Some ideas I had regarding searching:
- The web used to be some kind of its own index, back in the old days, where you could get from one webpage to another using links. If you get to think about it, links are not much different than a DHT: Every website has links to some sites that are similar to that site, and maybe to some sites that are a bit different. In order to find something specific you work your way through the links just like you would do in a DHT. The introduction of great search engines eliminated the need to put these kind of links in websites. So that's one solutions, links. Kind of primitive, but I think it used to work.
Maybe I could put it in a less primitive way: Assume that you search for X. An example for a new search method could work like this: You are given 5 different things, and each of those you pick the one closest to X. Then again you are given 5 things, and you pick again the one closest to X. And so on. Maybe after a few iterations you get what you want. Just a strange idea, though maybe it could made practical somehow. I like it because it contains no analysis of words or phrases.
- Crowdsourcing the creation of the index. I think it was mentioned in some of the messages on this thread. I believe that even the best algorithms and analysis methods are not good enough to index websites the right way. On top of that, All that SEO (Search engine optimization) that is so popular these days makes a lot of websites full of fluff show up in the top search results, which I think is really a shame. I suggest to those of you who didn't, to check out freenet, just to see what it's like to have some real content out there. It doesn't have much content, though the little it has is worth seeing.
Regarding crowdsourcing the creation of the index, I suggest doing it using some kind of incentives for people to do it, and in the same time some trust mechanisms to make sure that the index crowdsourcing is not abused. I don't write here about the technological tools to do this, though I do believe we are getting close to make this kind of thing possible. Though, I still don't fully understand how to classify the websites: Maybe keywords, or some kind of similarity network/metric between websites.
Anyways, These were my 50 cents.