Search engines and https
While trying to submit the cypherpunks.to website to a few search engines, I noticed that none of them seems to support indexing of https URL's. Is anybody here aware of a search engine that indexes secure web pages? And if there is no such search engine, what are the thoughts on using https to deliberately keep pages out of indices? Thanks, -- Lucky Green <shamrock@cypherpunks.to> PGP v5 encrypted email preferred. "Tonga? Where the hell is Tonga? They have Cypherpunks there?"
Lucky Green <shamrock@cypherpunks.to> writes:
While trying to submit the cypherpunks.to website to a few search engines, I noticed that none of them seems to support indexing of https URL's. Is anybody here aware of a search engine that indexes secure web pages? And if there is no such search engine, what are the thoughts on using https to deliberately keep pages out of indices?
One thing you could do is to have your server use http (no s) iff the connection comes from a known search engine. Reasonably easy to do -- set up an http server, and block all sites, and put in allow directives for the search engines. It's not as if you're insisting on client certs for the HTTPS server. Adam
One thing you could do is to have your server use http (no s) iff the connection comes from a known search engine.
Why not set up http pages with just "description" and "keywords" headers for the search engines, and a link to the https page for users that come from the search engine?
On Wed, 19 Nov 1997, Adam Back wrote:
One thing you could do is to have your server use http (no s) iff the connection comes from a known search engine. Reasonably easy to do -- set up an http server, and block all sites, and put in allow directives for the search engines.
Then the search engine would list the wrong URL. The idea is to get people to use crypto. In fact, as soon as I find the time, cypherpunks.to will reject weak crypto browsers and provide those unfortunate enough to use such a browser with pointers to upgrade options.
It's not as if you're insisting on client certs for the HTTPS server.
Actually, there are pages on the server that do require client certs. Of course these pages need not be listed in any search engine. But now that you mention it, it might be fun to set up an automated enrollment page and require client certs for everything. -- Lucky Green <shamrock@cypherpunks.to> PGP v5 encrypted email preferred. "Tonga? Where the hell is Tonga? They have Cypherpunks there?"
Lucky Green <shamrock@cypherpunks.to> writes:
On Wed, 19 Nov 1997, Adam Back wrote:
One thing you could do is to have your server use http (no s) iff the connection comes from a known search engine. Reasonably easy to do -- set up an http server, and block all sites, and put in allow directives for the search engines.
Then the search engine would list the wrong URL.
Yes; so make the failure page from your http server say: connect to https://www.cypherpunks.to. More elegant, but a bit more difficult, would be to do the right thing. That is: 1. connection on https port & coming from *.altavista.com allow. 2. connection on https port & coming from somewhere else insist on SSL (Would this work? Wander if their spider will talk to https port with http?)
The idea is to get people to use crypto. In fact, as soon as I find the time, cypherpunks.to will reject weak crypto browsers and provide those unfortunate enough to use such a browser with pointers to upgrade options.
Kewl. Anyone know where those patches to convert for netscape 40 bit crippleware to 128 bit crypto are? Save download time -- a version 4 browser is quite large to download. Adam
-----BEGIN PGP SIGNED MESSAGE----- In <199711191459.OAA02975@server.test.net>, on 11/19/97 at 02:59 PM, Adam Back <aba@dcs.ex.ac.uk> said:
Kewl. Anyone know where those patches to convert for netscape 40 bit crippleware to 128 bit crypto are?
Save download time -- a version 4 browser is quite large to download.
ftp://ftp.replay.com has the patches. - -- - --------------------------------------------------------------- William H. Geiger III http://users.invweb.net/~whgiii Geiger Consulting Cooking With Warp 4.0 Author of E-Secure - PGP Front End for MR/2 Ice PGP & MR/2 the only way for secure e-mail. OS/2 PGP 2.6.3a at: http://users.invweb.net/~whgiii/pgpmr2.html - --------------------------------------------------------------- -----BEGIN PGP SIGNATURE----- Version: 2.6.3a Charset: cp850 Comment: Registered_User_E-Secure_v1.1b1_ES000000 iQCVAwUBNHMeVo9Co1n+aLhhAQH4qQP/ZNTBU6l4MGDblZ19/AN2lqv7MaKcoTKj GJzhLkGoDBve49sZ0JSIKfMA2q+UL9fIgTjBYdewhvPNDrb1stmUCWHGybfg57XR u2vGg4AfykWCmYWDAjTACMtJr4jZ8K5hRYbe8Q7/BaimU/zMcO+SMFX7fIGYkavG eiHKdE0Ebig= =rvXb -----END PGP SIGNATURE-----
Lucky, Infoseek will actually store id/passwords to get into sites that it really wants to index. I've forwarded your question to the infoseek list. I'll let you know what I find out about https. - Joi
It's not as if you're insisting on client certs for the HTTPS server.
Actually, there are pages on the server that do require client certs. Of course these pages need not be listed in any search engine. But now that you mention it, it might be fun to set up an automated enrollment page and require client certs for everything.
-- Lucky Green <shamrock@cypherpunks.to> PGP v5 encrypted email preferred. "Tonga? Where the hell is Tonga? They have Cypherpunks there?"
-- PGP Key: http://pgp.ai.mit.edu:11371/pks/lookup?op=get&search=0x2D9461F1 PGP Fingerprint: 58F3 CA9A EFB8 EB9D DF18 6B16 E48D AF2A 2D94 61F1 Home Page: http://domino.garage.co.jp/jito/joihome.nsf To subscribe to my personal mailing list send mailto:friends-subscribe@ji.to
participants (5)
-
Adam Back -
Joichi Ito -
Lucky Green -
ulf＠fitug.de -
William H. Geiger III