Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I’m honestly more interested in bot detection than anything else at this point.

It seems like it should be perfectly legal to detect and then hold the connection open for a long period of time without giving a useful response. Or even send highly compressed gzip responses designed to fill their drives.

Legal or not, I can’t see any good reason that we can’t make it painful.



Make it painful if they abuse the site.

We all benefit from open data. Polite scrapers are just fine and a natural part of the web ecosystem.

Google has been scraping the web all day every day for decades now.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: