On Monday, December 12, 2005 Nazareno Andrade wrote:

A nice paper which you may find useful in this thread:

High Availability, Scalable Storage, Dynamic Peer Networks: Pick Two

Yes, it is an interesting approach - thank you! However, I'm not sure if their results directly apply to P2P nets. They are talking about six nines and replication factor of 20 to 80. They would likely commit suicide if they would try to actually use Gnutella for rare content. Any improvement would be nice - and forget about six nines. Also, despite introducing an interesting approach, this article results are very hard to verify and to reproduce, which is absolutely necessary if one would want to repeat their calculations with some different assumptions about the system requirements. For example, much of their conclusions are based on the Gnutella trace from April of 2003. Back then Gnutella was more than an order of magnitude smaller, and it would be interesting to repeat the calculations for today's situation. But the properties of this trace are not explicitly listed anywhere, being hidden in multiple charts and obscure statements like "only 5,000 of the 33,000 Gnutella hosts were usually available" (This, by the way, is a total mystery to me, since in April of 2003 Slyck's stats archive lists Gnutella at about 90,000 simultaneous nodes, so I have no idea where these 5,000 or 33,000 came from and what their meaning might have been.) To put it shortly, they have an interesting methodology, but I do not trust any one of their conclusions, as far as the caching in P2P file-sharing network is concerned. All their reasonings should be repeated for the reliable network statistical data, and with the set of requirements that reflects the needs of P2P users, not the need for a six nines-reliable data storage. I suspect that then the conclusions might prove to be a bit different. Best wishes - S.Osokine. 12 Dec 2005. -----Original Message----- From: p2p-hackers-bounces@zgp.org [mailto:p2p-hackers-bounces@zgp.org]On Behalf Of Nazareno Andrade Sent: Monday, December 12, 2005 10:22 AM To: Peer-to-peer development. Subject: Re: [p2p-hackers] p2p in some place or other Hi there. A nice paper which you may find useful in this thread: High Availability, Scalable Storage, Dynamic Peer Networks: Pick Two (HotOS XI) Peer-to-peer storage aims to build large-scale, reliable and available storage from many small-scale unreliable, low-availability distributed hosts. Data redundancy is the key to any data guarantees. However, preserving redundancy in the face of highly dynamic membership is costly. We use a simple resource usage model to measured behavior from the Gnutella file-sharing network to argue that large-scale cooperative storage is limited by likely dynamics and cross-system bandwidth - not by local disk space. We examine some bandwidth optimization strategies like delayed response to failures, admission control, and load-shifting and find that they do not alter the basic problem. We conclude that when redundancy, data scale, and dynamics are all high, the needed cross-system bandwidth is unreasonable. http://pmg.csail.mit.edu/~rodrigo/p2p-scl.pdf regards, Nazareno Matthew Kaufman wrote:

Alen Peacock:

...

I'd add: what is the self-interested motivation for a node to agree to cache the content in the first place?

This could be some external motivation like "I want anonymously-posted files about certain political views to be available for all to see" or "my corporate IT department says that we have to use this distributed collaboration tool"

...

If proactive caching were turned on by default in my p2p filesharing client, don't I have a very real incentive to turn this off in my own node to preserve bandwidth, disk space, and perhaps limit any legal liability?

In the general "filesharing" case? Absolutely. But that's not the only use for P2P technology or even P2P file transfer.

...

...which is similar to many of the arguments made against pre-fetching in traditional caching literature: how do you ensure that you prefetch the right content, especially when the cost of prefetching the wrong content is very high?

Actually, if you're replicating content to other nodes in order to ensure availability or create more downloadable nodes in order to speed future downloaders, it is more like the RAID arguments than the cache arguments.

The real question is, IF you had a high-availability file sharing system, what files would you want to make available on it? (The answer is probably *not* the long tail of all files ever seen on generic file sharing services)

Matthew Kaufman matthew@matthew.at www.amicima.com

_______________________________________________ p2p-hackers mailing list p2p-hackers@zgp.org http://zgp.org/mailman/listinfo/p2p-hackers _______________________________________________ Here is a web page listing P2P Conferences: http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences

-- Nazareno. ======================================== Nazareno Andrade LSD - DSC/UFCG Campina Grande - Brazil http://lsd.dsc.ufcg.edu.br/~nazareno/ OurGrid project http://www.ourgrid.org ======================================== _______________________________________________ p2p-hackers mailing list p2p-hackers@zgp.org http://zgp.org/mailman/listinfo/p2p-hackers _______________________________________________ Here is a web page listing P2P Conferences: http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences _______________________________________________ p2p-hackers mailing list p2p-hackers@zgp.org http://zgp.org/mailman/listinfo/p2p-hackers _______________________________________________ Here is a web page listing P2P Conferences: http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences ----- End forwarded message ----- -- Eugen* Leitl <a href="http://leitl.org">leitl</a> http://leitl.org ______________________________________________________________ ICBM: 48.07100, 11.36820 http://www.ativel.com 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE [demime 1.01d removed an attachment of type application/pgp-signature which had a name of signature.asc]