It may be of interest that Donal Blaney is the only blogger I've ever heard of whose blog set a robots.txt file to stop the Internet Archive Foundation from indexing it.
Google's Web-in-RAM systems are rather less fastidious than Brewster Kahle's outfit (which invented quite a bit of the technology behind Google and other big clouds, like the containerised data centre). The site: command, for example, remains informative.
1 comment:
He doesn't seem to be disallowing it specifically (as of just now he has a disallow "/" - as of the archive's copy he has a disallow of "/search" but not everything so I don't see why the archive doesn't have anything of that date.
Lots of matching searches though, a bit of an obsession, methinks.
Post a Comment