load balancing web proxy and server networks

[Adam Back]

We might also think about adding hot spot avoidance to distributed web
hosting mechanisms.  A distributed web space which distributed itself
to meet demand would be useful.  (An example of hot spots being
perhaps Dan Farmer's web site after he released Satan ... he switched
the web server off to stop his T1 being saturated, until the deluge
subsided).

An automatic load balancing distributed web server would be a kind of
next generation web proxy cacheing and mirroring scheme, dynamically
replicating and cacheing data to meet demand.  Robustness could be
improved also by the redundancy introduced.

Those browsing documents would pay for their web accesses at the
service rate they desired.  A market in web data would then give web
proxies / mirroring agents an economic incentive to automatically
replicate and migrate data.

Web caches already have some protection from various laws.


One other idea for an eternity service host we could leach off was
suggested to me by Ian Brown <I.Brown at cs.ucl.ac.uk>.  His suggestion
was that it would be nice if we could con proxy caches into cacheing
eternity web space for us.

One way to do this for example would be to actually host our own web
pages, but to configure our web server to only serve to a list of web
proxies -- anybody else would get access denied.  The .htaccess
mechanism would be sufficient for this purpose.

Adam