State of the Freenet, and tentative roadmap

[Eugen Leitl]

http://archives.freenetproject.org/message/20081113.185252.f691bed8.en.html

Author: Matthew Toseland
Date:  
To: Discussion of development issues, support
Subject: [freenet-support] State of the Freenet, and tentative roadmap
	
Delete this message
Reply to this message
gpg: Signature made Thu Nov 13 18:52:57 2008 UTC using RSA key ID BEC4F4C3
gpg: Can't check signature: public key not found
CHANGES SINCE 0.7

Lots of work has been done on Freenet since 0.7 was released. The main
highlights in code already shipped and available are sdiz's new datastore
(which should greatly improve CPU usage and I/O), lots of debugging,
profiling and optimisation, and two major improvements to routing:
friend-of-a-friend routing and turning off location swapping on opennet.
These two should improve data retention and performance significantly. We
have also implemented empirical measurements of network performance, which
have proved useful although they give a rather noisy signal. And there are
two major projects which have not yet been deployed, which are discussed
below: the db4o branch, and the new Freetalk plugin. Other changes include
better bandwidth limiting. The conference in July came up with many good
ideas; some of these are already in 0.8, some will be postponed to later
versions, but it was great to meet up with everyone and make sure we're all
on the same page, including the theoreticians. All this, plus some small
further changes and a lot of debugging, should make for a 0.8.0 some time in
the next few months which will be a major improvement on 0.7.0, and will have
taken much less than the nearly 3 years it took to reach 0.7.0!

FINANCES

We currently have $6863.24., including $4820 remaining from Google. This is
shown live (updated hourly) on the web site. Emu costs around #80/month, I
cost around #1440/month; the weak pound obviously helps matters, but when I
started work on Freenet I only cost $1250/mo; now I cost nearly twice that
(in a few months time I might be prepared to take a small pay cut, but it
won't cover the difference). This makes keeping going from individual
donations difficult, and over the last 18 months we've largely continued from
large donations from individuals or companies - Google's and John Gilmore's
being the biggest. Hopefully shipping 0.8 (or 0.8 alpha 1) combined with an
urgent appeal for funds will generate us some more publicity and more cash,
and more volunteers, but it's an open question whether we will make enough to
keep going for more than a few months.

DB4O STATUS

The db4o branch is an attempt to put most of the client layer into a
lightweight object database, so that we have no more hours of reloading the
pending requests from the datastore after a restart, faster startups, and
vastly less memory usage for large download queues. It has achieved most of
these goals already, but it has taken far longer than anticipated, and this
is why I haven't been working much on the mainstream stable Freenet over the
last 5 months. However it should be ready soon: I will post a new jar for
wider testing soon, and with a few more weeks of work it should be ready to
merge.

FREETALK (FORUMS IN FREENET) AND WEB INTERFACE USABILITY

Chat is essential for any community, especially an anonymous one; without a
good anonymous chat system, the chances of new users continuing to use
Freenet are low IMHO. We had Frost, but an unknown person has effectively
destroyed Frost with denial of service attacks (made possible by Frost's weak
design). FMS is the chat system currently used most widely on Freenet, and
its architecture should be immune to the attacks on Frost, but FMS has some
serious problems: It is written in C++, so is difficult to bundle with
Freenet, and cannot be an obvious part of the web interface or inlined in
freesites; it uses both a newsreader and a web browser, so is hard to use;
and it has had some exploitable bugs, and may still have them. I have stopped
using FMS because I don't have time to do a proper code review (and I'm not
sure I'm competent to do so), and nobody I trust has done one. So we clearly
need a new chat system.

Thanks to the massive efforts of batosai, saces and especially xor/p0s, the
Freetalk plugin should be ready by the time we ship 0.8. This will be a menu
item on the web interface, just under Browse Freenet, and will be an
officially code reviewed and supported plugin. Hopefully its web interface
will also be embeddable in freesites, as a site forums system. Freemail may
also be added to the top level GUI, if and when it becomes sufficiently
stable and has a good web interface; we had a Summer of Code student working
on one but that didn't work out.

There has been lots of discussion on making the user interface more friendly.
The main principle here is to make what you can do with Freenet more obvious,
and hide technical details in submenus. This of course means all the major
functions of Freenet must be bundled and code reviewed as plugins, and
accessible from Fproxy (experience suggests most users won't use them if
they're not obvious and two clicks away!). Real time chat among darknet peers
has also been discussed, with a more "social networking" feel for darknet,
since that's exactly what it is. Dieppe has built some mockups (especially
the first one):
http://doc-fr.freenetproject.org/Fproxy_mockup
http://amphibian.dyndns.org/freenet/browse-mockup/html/browse.html
http://amphibian.dyndns.org/freenet/mockup2/mockup2/html/browse.html

TENTATIVE ROADMAP

0.8:
- ALREADY IMPLEMENTED: New compression algorithms, FOAF, no swap on opennet,
salted hash datastore, lots of minor refactoring, ...
- Merge the db4o branch.
- Some work on plugin dependancies, plugin updating.
- More work on searching: adjacent word matching, fix the spider, make it look
better, possibly make it embeddable in freesites and support adding indexes
from freesites.
- LOTS of debugging. Debug current network problems, debug bootstrapping
taking ages ...
- Hopefully bundle a working and reasonably easy to use Freetalk.
0.9:
- Rest of the UI refactoring.
- Splitfile crypto randomization (greatly improves security for large
inserts).
- Tunnels. Greatly improve security, significant performance cost.
- Bloom filters. Greatly improve performance, significant security cost.
- BUT PUT THEM TOGETHER, and you get greatly improved security for the
paranoid at a slighty performance cost, improved security and current
performance or better for the moderately paranoid, and greatly improved
performance for the not so paranoid.
- Possibly some low level changes (e.g. streams, a proposal to greatly reduce
the bandwidth cost of padding).
0.10:
- Passive requests: These are needed for efficient, large scale Web of Trust
(a vital component of Freetalk, but also probably useful for filesharing and
other things in future). They are also very helpful for other applications
such as streaming, and can save significant amounts of bandwidth for
downloads in many cases.
- Long-term requests: Closely related to passive requests. Needed to deal with
uptime issues, and some of the more serious steganographic transport plugins.
Uptime issues are especially important: other p2p networks also have to deal
with them, often not very well, and the problem will only get worse as more
and more people only have laptops.
- Possibly transport plugins: This would enable Freenet to pretend to be HTTP
traffic, or Skype(tm), or whatever.
1.0
- DEBUG DEBUG DEBUG DEBUG

Post 1.0:
- Better support for sneakernet and other high latency transports. The main
remaining issue is how to assign locations on a low uptime / high latency
network.
- Content filters for *everything* - PDFs for example are quite feasible, and
strategically important, but a lot of work.
- Better darknet support - targeted swapping, better fragmented network
support, etc.

SECURITY SUMMARY

There are 3 main classes of attack on Freenet itself that are of particular
concern: harvesting nodes, statistical attacks to find out what your peers
are doing, and key-based searches where the attacker moves across the network
and slowly closes in on the source of some large content. All of them are
much harder if you use darknet mode and pick your Friends carefully. Right
now Freenet's security is nowhere near what it should be; if you have to
stake your freedom or worse on it, think carefully about the risks and the
alternatives (which are usually much easier to block). The features mentioned
for 0.9 should greatly improve security against these attacks. However, for
many purposes, especially if you use a darknet, Freenet is still better than
many of the alternatives.

NEAR FUTURE/NEAR PAST

On Saturday I released builds 1170, 1171, 1172 and 1173. 1170 included at
least a month's worth of work on trunk, all since 1166, which wasn't released
earlier as it was too unstable. 1173 was self-mandatory, meaning that every
node had to upgrade immediately, or lose connectivity. This has caused some
short term problems for the network; it was unavoidable, sorry folks. The
1170 changes are quite significant: If you have a freesite, please reinsert
it (it will load significantly faster in most cases). Much of the recent
chaos (addressed to some degree in 1174 and 1175) may have been caused by my
being absent from trunk development, building db4o. That's nearly finished
and I'm spending some time on trunk at the moment.