Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> And if you 429 Google’s bots they will reduce your pagerank. That’s straight up extortion from a company that also sells cloud services.

Googlebot uses different IP space from gcp



They use the same bank accounts and stock ticker. This is basically a non sequitur.

The point is they’re getting paid to run cloud servers to keep their bots happy and not dropping your website to page six.


I thought the argument was that if you run on gcp you can masquerade as googlebot and not get a 429 which is obviously false. Instead it looks like the argument is more of a tinfoil hat variety.

btw you don't get dropped if you issue temporary 429s only when it's consistent and/or the site is broken. that is well documented. and wtf else are they supposed to do if you don't allow to crawl it and it goes stale?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: