I think that's the point, you'd use robots.txt to direct Googlebot/Bingbot/etc a...

		jsheard 10 months ago \| parent \| context \| favorite \| on: I use zip bombs to protect my server I think that's the point, you'd use robots.txt to direct Googlebot/Bingbot/etc away from countermeasures that could potentially mess up your SEO. If other bots ignore the signpost clearly saying not to enter the tarpit, that's their own stupid fault.