whitehouse.gov/robots.txt

[Anatoly Vorobey]

On Wed, Dec 10, 2003 at 12:56:24PM +0100, Eugen Leitl wrote:
> Can somebody with a webspider crawl these documents, and put it up
> on the web?
> 
> 	http://www.whitehouse.gov/robots.txt

All or nearly all of them are duplicates of same documents
elsewhere in the directory tree; "X/text/" and "X/iraq/" are
supposed to be copies of "X/", with images removed in the first 
case. I suspect that downloading them all would just confirm that.

--
avva