It's not about not wanting people to access the content, it's about not wanting AI bots to do so. But yes, I agree, the only realistic defense we have is to remove it from the publicly-accessible web.