At 11:38 AM 1/16/96 -0800, Timothy C. May wrote:
I've been thinking a lot about the problems and opportunities that are coming up as more and more "spiders" (Web searchers, crawlers) are indexing directories and files on systems they can find.
For the sake of this post, the files and whatnot these spiders and super-spiders can hit constitute a universe I'll call "spiderspace," as it semi-euphoniously matches cyberspace and cypherspace.
[stuff deleted]
Implications for Cypherpunks?
First, an alert for you to be very careful about what you make accessible to the outside world. It's no longer just a matter of people taking the time to rummage through your subdirectories, it's now trivial to find things with the new Web search engines.
Second, what is out there in spiderspace is incredibly useful for building dossiers, for compiling correlations, and for doing competitive analyses.
Third, more and more kinds of files are going into spiderspace. This may include files compiled by others, such as files containing Web accesses! (All it takes is for someone to keep a record of site accesses, subscriptions, etc., and then put record in a searchable place: it then becomes trivial to search on a name and find out interesting things.) --Tim May
Consider this: In about 10-20 years, the people who have been using Internet about now will come into the age from which (statistically) American presidents are usually chosen. Look at what just one letter Clinton sent (draft-dodge, etc) did. Now imagine literally YEARS of messages online, archived on terabytes of optical tape, searchable...