Some ideas I had regarding searching:
   - The web used to be some kind of its own index, back in the old days,
   where you could get from one webpage to another using links. If you get
   to think about it, links are not much different than a DHT: Every
   website has links to some sites that are similar to that site, and
   maybe to some sites that are a bit different. In order to find
   something specific you work your way through the links just like you
   would do in a DHT. The introduction of great search engines eliminated
   the need to put these kind of links in websites. So that's one
   solutions, links. Kind of primitive, but I think it used to work.
   Maybe I could put it in a less primitive way: Assume that you search
   for X. An example for a new search method could work like this: You are
   given 5 different things, and each of those you pick the one closest to
   X. Then again you are given 5 things, and you pick again the one
   closest to X. And so on. Maybe after a few iterations you get what you
   want. Just a strange idea, though maybe it could made practical
   somehow. I like it because it contains no analysis of words or phrases.
   - Crowdsourcing the creation of the index. I think it was mentioned in
   some of the messages on this thread. I believe that even the best
   algorithms and analysis methods are not good enough to index websites
   the right way. On top of that, All that SEO (Search engine
   optimization) that is so popular these days makes a lot of websites
   full of fluff show up in the top search results, which I think is
   really a shame. I suggest to those of you who didn't, to check out
   freenet, just to see what it's like to have some real content out
   there. It doesn't have much content, though the little it has is worth
   seeing.
   Regarding crowdsourcing the creation of the index, I suggest doing it
   using some kind of incentives for people to do it, and in the same time
   some trust mechanisms to make sure that the index crowdsourcing is not
   abused. I don't write here about the technological tools to do this,
   though I do believe we are getting close to make this kind of thing
   possible. Though, I still don't fully understand how to classify the
   websites: Maybe keywords, or some kind of similarity network/metric
   between websites.
   Anyways, These were my 50 cents.
   On Wed, Jan 1, 2014 at 5:00 AM, coderman <[1]coderman@gmail.com> wrote:

   On Tue, Dec 31, 2013 at 6:37 AM, Lodewijk andr de la porte
   <[2]l@odewijk.nl> wrote:
   > I'd like to ask people to wonder what Search Engines really do for
   us. Where
   > is the catalog? Where is the cultivated list of good resources?
   >
   > Do search engines provide the same level of guidance to its users
   that a
   > written overview can?

     what you want more than traditional search is resource discovery,
     which includes recommendation and per-peer-perspective reputation.
     this is an area where centralized search is incapable or
     untrustworthy
     enough compared to fully decentralized options.
     done centrally, that central trusted party would be privy to all
     your
     inter-peer interactions. in decentralized fashion this exposes only
     limited information to each peer. (central services usually paying
     the cost of the infrastructure to analyze all to all interactions by
     selling your private information to third parties, or delegating to
     those who do...)

   > Why don't we create a distributed website catalog? It's harder, as
   anti-spam
   > is the core feature. But competing with Google seems rather foolhardy
   at the
   > moment.

     public web is a small slice of all that is of interest. just put a
     internet [3]archive.org copy on a hidden Tahoe-LAFS and everyone
     gets a
     copy of the public web for local querying. (better yet, make a PIR
     LAFS ;)
     ... this would need a little coding *grin*

References

   1. mailto:coderman@gmail.com
   2. mailto:l@odewijk.nl
   3. http://archive.org/