part of the secret that makes any search engine unique is the knowledge that a site at x.com exists and they know there is a forum at x.com/forums which is not visible by simply crawling from the root x.com. On the other hand, I would love an open web cache for my work.