-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 12/30/2013 11:21 PM, coderman wrote:

i have more to say later, but one effort from back in early 2000 is alpine:

What about YaCy? http://yacy.de/en/index.html

but other approaches which are not a feasible replacement include: - the old skewl (mostly)flooding broadcasts like gnutella - fragile, hard to defend constructs like DHTs as keyword indexes - aggressive caching with local search (110% useful, but not sufficient alone) - distributed (but better somehow) search engines on darknets, etc.

What aspects would constitute feasible replacements? - -- The Doctor [412/724/301/703] [ZS] PGP: 0x807B17C1 / 7960 1CDC 85C9 0B63 8D9F DD89 3BD8 FF2B 807B 17C1 WWW: https://drwho.virtadpt.net/ "You knew the job was dangerous when you took it!" --Super Chicken -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlLHPmsACgkQO9j/K4B7F8HxgQCfamx4+RWlapLfH6iIhfvKYaAv E7IAn3Hv0zJ7PEK7yltRP0wgJboq2/YI =1ghA -----END PGP SIGNATURE-----

I'd like to ask people to wonder what Search Engines really do for us. Where is the catalog? Where is the cultivated list of good resources? Do search engines provide the same level of guidance to its users that a written overview can? Why don't we create a distributed website catalog? It's harder, as anti-spam is the core feature. But competing with Google seems rather foolhardy at the moment. Maybe the word catalog isn't right, catalogs are too static and not discovery targeted at all. Maybe a Yahoo! answers type of tagging/cataloging would work rather well. Anyway: think about it guys! I'm sure there's a better way than "this keyword is also in this page which links to other good pages"!

Some ideas I had regarding searching: - The web used to be some kind of its own index, back in the old days, where you could get from one webpage to another using links. If you get to think about it, links are not much different than a DHT: Every website has links to some sites that are similar to that site, and maybe to some sites that are a bit different. In order to find something specific you work your way through the links just like you would do in a DHT. The introduction of great search engines eliminated the need to put these kind of links in websites. So that's one solutions, links. Kind of primitive, but I think it used to work. Maybe I could put it in a less primitive way: Assume that you search for X. An example for a new search method could work like this: You are given 5 different things, and each of those you pick the one closest to X. Then again you are given 5 things, and you pick again the one closest to X. And so on. Maybe after a few iterations you get what you want. Just a strange idea, though maybe it could made practical somehow. I like it because it contains no analysis of words or phrases. - Crowdsourcing the creation of the index. I think it was mentioned in some of the messages on this thread. I believe that even the best algorithms and analysis methods are not good enough to index websites the right way. On top of that, All that SEO (Search engine optimization) that is so popular these days makes a lot of websites full of fluff show up in the top search results, which I think is really a shame. I suggest to those of you who didn't, to check out freenet, just to see what it's like to have some real content out there. It doesn't have much content, though the little it has is worth seeing. Regarding crowdsourcing the creation of the index, I suggest doing it using some kind of incentives for people to do it, and in the same time some trust mechanisms to make sure that the index crowdsourcing is not abused. I don't write here about the technological tools to do this, though I do believe we are getting close to make this kind of thing possible. Though, I still don't fully understand how to classify the websites: Maybe keywords, or some kind of similarity network/metric between websites. Anyways, These were my 50 cents. On Wed, Jan 1, 2014 at 5:00 AM, coderman wrote:

On Tue, Dec 31, 2013 at 6:37 AM, Lodewijk andré de la porte wrote:

...

I'd like to ask people to wonder what Search Engines really do for us. Where is the catalog? Where is the cultivated list of good resources?

Do search engines provide the same level of guidance to its users that a written overview can?

what you want more than traditional search is resource discovery, which includes recommendation and per-peer-perspective reputation. this is an area where centralized search is incapable or untrustworthy enough compared to fully decentralized options.

done centrally, that central trusted party would be privy to all your inter-peer interactions. in decentralized fashion this exposes only limited information to each peer. (central services usually paying the cost of the infrastructure to analyze all to all interactions by selling your private information to third parties, or delegating to those who do...)

...

Why don't we create a distributed website catalog? It's harder, as anti-spam is the core feature. But competing with Google seems rather foolhardy at the moment.

public web is a small slice of all that is of interest. just put a internet archive.org copy on a hidden Tahoe-LAFS and everyone gets a copy of the public web for local querying. (better yet, make a PIR LAFS ;) ... this would need a little coding *grin*

On 2014-01-02 02:22, realcr wrote:

Some ideas I had regarding searching:

- The web used to be some kind of its own index, back in the old days, where you could get from one webpage to another using links. If you get to think about it, links are not much different than a DHT: Every website has links to some sites that are similar to that site, and maybe to some sites that are a bit different. In order to find something specific you work your way through the links just like you would do in a DHT. The introduction of great search engines eliminated the need to put these kind of links in websites. So that's one solutions, links. Kind of primitive, but I think it used to work.

Search humans, instead of search engines. Human authority instead of AI. As a matter of fact, it still does work.

Dnia czwartek, 2 stycznia 2014 13:04:17 Sean Lynch pisze:

On Wed, Jan 1, 2014 at 3:46 PM, James A. Donald wrote:

...

As a matter of fact, it still does work.

It works far less, though, since most people expect others to rely on search engines, so they don't bother to link anymore.

Here's a thought: browser extension that stores your "personal" web index, and gives you a typeahead menu when you write about concepts in your index, prompting you to convert phrases to links. Like the way Facebook always wants to convert the names of people and pages to tags. Even if it were just primed with Wikipedia, that would drastically reduce the amount of Google searching people need to do when reading stuff you write.

In Firefox it's called "The Awesome Bar", and it sifts through your history and bookmarks (I bookmark a lot, and tag these pretty exactly, which helps immensely). The downside, of course, is that it works only for links that I have already visited. So here's the idea: sharing bookmark tags and links with each other, via some extention for example, and making "The Awesome Bar" (damn, I hate that name) sift through bookmarks/tags of people in your "network" (what that means would have to be defined, but as Mozilla Sync can already store bookmarks, the data can already be on a server, just use it). -- Pozdr rysiek

Dnia niedziela, 5 stycznia 2014 19:52:12 stef pisze:

On Sun, Jan 05, 2014 at 07:39:40PM +0100, rysiek wrote:

...

So here's the idea: sharing bookmark tags and links with each other, via some extention for example, and making "The Awesome Bar" (damn, I hate that name) sift through bookmarks/tags of people in your "network" (what that means would have to be defined, but as Mozilla Sync can already store bookmarks, the data can already be on a server, just use it).

omnom[1] should be able to serve as the server-side, you still need to develop some kind of client-side extension though.

[1] omnom https://gitorious.org/tagr/omnom/source/419b512734021b71c01500514b5ae87d0b7f 3ab7:features.txt

Humm, I think I have already came across omnom some time ago. Well, I'll have to look into it. A short question: what would you say are the most important advantages of omnom over Mozilla Sync? -- Pozdr rysiek

Dnia niedziela, 5 stycznia 2014 23:12:28 stef pisze:

On Sun, Jan 05, 2014 at 08:22:51PM +0100, rysiek wrote:

...

Humm, I think I have already came across omnom some time ago. Well, I'll have to look into it. A short question: what would you say are the most important advantages of omnom over Mozilla Sync?

i don't know mozilla sync. one huge advantage of omnom is, that it snapshots the pages you bookmark as they are rendered in your firefox. i guess, that could also be useful for searching bookmarked pages... also there are hooks in the code that connect multiple omnom instances and other services for the federation of tags, but that is quite dead code.

I'm bought! Mozilla Sync has neither federation nor snapshotting. Me gusta mucho. -- Pozdr rysiek

Dnia piątek, 17 stycznia 2014 10:55:33 Sean Lynch pisze:

...

In Firefox it's called "The Awesome Bar", and it sifts through your history and bookmarks (I bookmark a lot, and tag these pretty exactly, which helps immensely).

I'm talking about anytime you type into text boxes.

Which "text boxes"? Any form on Teh Intertubes? THe AwesomeBar or SearchBar?

The goal of this proposal was to return to the hypertextual nature of the web in order to reduce our dependence on centralized indexes. However, I find your proposal to improve the utility of the AwesomeBar interesting.

It's easy (it just requires a habit of decent tagging), and effective -- when I remember an information I found important from a website I visited, it's usually in my bookmarks, tagged properly. This means 95% of the time as far as information I have already seen is concerned, the AwesomeBar reaching down to my bookmarks is enough to get what I need, no need to go to Google here.

...

The downside, of course, is that it works only for links that I have already visited.

So here's the idea: sharing bookmark tags and links with each other, via some extention for example, and making "The Awesome Bar" (damn, I hate that name) sift through bookmarks/tags of people in your "network" (what that means would have to be defined, but as Mozilla Sync can already store bookmarks, the data can already be on a server, just use it).

An even simpler proposal: assuming the AwesomeBar doesn't already include live bookmarks in its autocomplete functionality, add it. Then anyone can simply publish their bookmarks via RSS and anyone else can import them. Then someone can just add functionality to create live bookmarks that pull signed and possibly encrypted (with Ed25519/Curve25519 of course) RSS feeds from a DHT.

Not that easy, as everybody would need to publish their bookmark RSS/Atom channels not entirely in accordance with how it is being done usually on the 'Net. Usually, only the first 10-30 headlines/items are in the RSS/Atom channel, the older ones (it is assumed) are already cached in users' RSS/Atom readers. Firefox does not cache live bookmarks, so each time you only get the current 10-30 items, all older are "lost". This makes sense with regard to the intended use of this functionality in Firefox (and other browsers), but unfortunately makes it harder to implement interesting bookmarks sharing the way you described. -- Pozdr rysiek

On Tue, Dec 31, 2013 at 6:37 AM, Lodewijk andré de la porte wrote:

I'd like to ask people to wonder what Search Engines really do for us. Where is the catalog? Where is the cultivated list of good resources?

Well, in Google's case, the list is curated by those doing the linking, but Google is trading richness of metadata for coverage.

Do search engines provide the same level of guidance to its users that a written overview can?

No, but they cover far more of the Web than a manually curated index ever could. They can answer questions like "what was that article I read last week on this topic?" and "what other pictures exist of this person?" Nobody's going to be writing written summaries of every single news article and blog post.

Why don't we create a distributed website catalog? It's harder, as anti-spam is the core feature. But competing with Google seems rather foolhardy at the moment.

I think this is a good idea. Spam can be handled by just signing all the pages and having signed white and blacklists to create a web of trust/distrust. Proof-of-work could be used when creating new signing identities in order to make the blacklists useful.

Maybe the word catalog isn't right, catalogs are too static and not discovery targeted at all.

I imagine something as simple as StumbleUpon, just "I like/dislike this", perhaps with tags. One could add a signed inverted index as well to facilitate searching by phrase.

Maybe a Yahoo! answers type of tagging/cataloging would work rather well.

Anyway: think about it guys! I'm sure there's a better way than "this keyword is also in this page which links to other good pages"!

Been thinking about it for a while ;-)

Dnia wtorek, 31 grudnia 2013 13:42:59 stef pisze:

On Sun, Dec 29, 2013 at 12:17:07AM -0800, Jesse R. Taylor wrote:

...

but I've seen very little focus on the need for development of alternatives to corporate search engines.

[disregarding the corporate focus] i can warmly recommend https://searx.0x2a.tk

Hummm, what is this, who runs this, is it distributed or centralised (as far as control is concerned)? -- Pozdr rysiek