Tor 0.1.0.10, the first stable release of the 0.1.0.x branch, is
finally ready.
This release features cleanup on Windows, including making NT
services work; many performance improvements, including libevent to
use poll/epoll/kqueue when available, and pthreads and better buffer
management to avoid so much memory bloat; better performance and
reliability for hidden services; automated self-reachability testing by
servers; http and https proxy support for clients; and much more support
for the Tor controller protocol.
http://tor.eff.org/download.html
o Fixes on Win32:
- Make NT services work and start on startup on Win32 (based on
patch by Matt Edman). See the FAQ entry for details.
- Make 'platform' string in descriptor more accurate for Win32
servers, so it's not just "unknown platform".
- REUSEADDR on normal platforms means you can rebind to the port
right after somebody else has let it go. But REUSEADDR on Win32
means you can bind to the port _even when somebody else already
has it bound_! So, don't do that on Win32.
- Clean up the log messages when starting on Win32 with no config
file.
- Allow seeding the RNG on Win32 even when you're not running as
Administrator. If seeding the RNG on Win32 fails, quit.
o Assert / crash bugs:
- Refuse relay cells that claim to have a length larger than the
maximum allowed. This prevents a potential attack that could read
arbitrary memory (e.g. keys) from an exit server's process.
- If unofficial Tor clients connect and send weird TLS certs, our
Tor server triggers an assert. Stop asserting, and start handling
TLS errors better in other situations too.
- Fix a race condition that can trigger an assert when we have a
pending create cell and an OR connection attempt fails.
o Resource leaks:
- Use pthreads for worker processes rather than forking. This was
forced because when we forked, we ended up wasting a lot of
duplicate ram over time.
- Also switch to foo_r versions of some library calls to allow
reentry and threadsafeness.
- Implement --disable-threads configure option. Disable threads on
netbsd and openbsd by default, because they have no reentrant
resolver functions (!), and on solaris since it has other
threading issues.
- Fix possible bug on threading platforms (e.g. win32) which was
leaking a file descriptor whenever a cpuworker or dnsworker died.
- Fix a minor memory leak when somebody establishes an introduction
point at your Tor server.
- Fix possible memory leak in tor_lookup_hostname(). (Thanks to
Adam Langley.)
- Add ./configure --with-dmalloc option, to track memory leaks.
- And try to free all memory on closing, so we can detect what
we're leaking.
o Protocol correctness:
- When we've connected to an OR and handshaked but didn't like
the result, we were closing the conn without sending destroy
cells back for pending circuits. Now send those destroys.
- Start sending 'truncated' cells back rather than destroy cells
if the circuit closes in front of you. This means we won't have
to abandon partially built circuits.
- Handle changed router status correctly when dirserver reloads
fingerprint file. We used to be dropping all unverified descriptors
right then. The bug was hidden because we would immediately
fetch a directory from another dirserver, which would include the
descriptors we just dropped.
- Revise tor-spec to add more/better stream end reasons.
- Revise all calls to connection_edge_end to avoid sending 'misc',
and to take errno into account where possible.
- Client now retries when streams end early for 'hibernating' or
'resource limit' reasons, rather than failing them.
- Try to be more zealous about calling connection_edge_end when
things go bad with edge conns in connection.c.
o Robustness improvements:
- Better handling for heterogeneous / unreliable nodes:
- Annotate circuits with whether they aim to contain high uptime
nodes and/or high capacity nodes. When building circuits, choose
appropriate nodes.
- This means that every single node in an intro rend circuit,
not just the last one, will have a minimum uptime.
- New config option LongLivedPorts to indicate application streams
that will want high uptime circuits.
- Servers reset uptime when a dir fetch entirely fails. This
hopefully reflects stability of the server's network connectivity.
- If somebody starts his tor server in Jan 2004 and then fixes his
clock, don't make his published uptime be a year.
- Reset published uptime when we wake up from hibernation.
- Introduce a notion of 'internal' circs, which are chosen without
regard to the exit policy of the last hop. Intro and rendezvous
circs must be internal circs, to avoid leaking information. Resolve
and connect streams can use internal circs if they want.
- New circuit pooling algorithm: keep track of what destination ports
we've used recently (start out assuming we'll want to use 80), and
make sure to have enough circs around to satisfy these ports. Also
make sure to have 2 internal circs around if we've required internal
circs lately (and with high uptime if we've seen that lately too).
- Turn addr_policy_compare from a tristate to a quadstate; this should
help address our "Ah, you allow 1.2.3.4:80. You are a good choice
for google.com" problem.
- When a client asks us for a dir mirror and we don't have one,
launch an attempt to get a fresh one.
- First cut at support for "create-fast" cells. Clients can use
these when extending to their first hop, since the TLS already
provides forward secrecy and authentication. Not enabled on
clients yet.
o Reachability testing.
- Your Tor server will automatically try to see if its ORPort and
DirPort are reachable from the outside, and it won't upload its
descriptor until it decides at least ORPort is reachable (when
DirPort is not yet found reachable, publish it as zero).
- When building testing circs for ORPort testing, use only
high-bandwidth nodes, so fewer circuits fail.
- Notice when our IP changes, and reset stats/uptime/reachability.
- Authdirservers don't do ORPort reachability detection, since
they're in clique mode, so it will be rare to find a server not
already connected to them.
- Authdirservers now automatically approve nodes running 0.1.0.2-rc
or later.
o Dirserver fixes:
- Now we allow two unverified servers with the same nickname
but different keys. But if a nickname is verified, only that
nickname+key are allowed.
- If you're an authdirserver connecting to an address:port,
and it's not the OR you were expecting, forget about that
descriptor. If he *was* the one you were expecting, then forget
about all other descriptors for that address:port.
- Allow servers to publish descriptors from 12 hours in the future.
Corollary: only whine about clock skew from the dirserver if
he's a trusted dirserver (since now even verified servers could
have quite wrong clocks).
- Require servers that use the default dirservers to have public IP
addresses. We have too many servers that are configured with private
IPs and their admins never notice the log entries complaining that
their descriptors are being rejected.
o Efficiency improvements:
- Use libevent. Now we can use faster async cores (like epoll, kpoll,
and /dev/poll), and hopefully work better on Windows too.
- Apple's OS X 10.4.0 ships with a broken kqueue API, and using
kqueue on 10.3.9 causes kernel panics. Don't use kqueue on OS X.
- Find libevent even if it's hiding in /usr/local/ and your
CFLAGS and LDFLAGS don't tell you to look there.
- Be able to link with libevent as a shared library (the default
after 1.0d), even if it's hiding in /usr/local/lib and even
if you haven't added /usr/local/lib to your /etc/ld.so.conf,
assuming you're running gcc. Otherwise fail and give a useful
error message.
- Switch to a new buffer management algorithm, which tries to avoid
reallocing and copying quite as much. In first tests it looks like
it uses *more* memory on average, but less cpu.
- Switch our internal buffers implementation to use a ring buffer,
to hopefully improve performance for fast servers a lot.
- Reenable the part of the code that tries to flush as soon as an
OR outbuf has a full TLS record available. Perhaps this will make
OR outbufs not grow as huge except in rare cases, thus saving lots
of CPU time plus memory.
- Improve performance for dirservers: stop re-parsing the whole
directory every time you regenerate it.
- Keep a big splay tree of (circid,orconn)->circuit mappings to make
it much faster to look up a circuit for each relay cell.
- Remove most calls to assert_all_pending_dns_resolves_ok(),
since they're eating our cpu on exit nodes.
- Stop wasting time doing a case insensitive comparison for every
dns name every time we do any lookup. Canonicalize the names to
lowercase when you first see them.
o Hidden services:
- Handle unavailable hidden services better. Handle slow or busy
hidden services better.
- Cannibalize GENERAL circs to be C_REND, C_INTRO, S_INTRO, and S_REND
circ as necessary, if there are any completed ones lying around
when we try to launch one.
- Make hidden services try to establish a rendezvous for 30 seconds
after fetching the descriptor, rather than for n (where n=3)
attempts to build a circuit.
- Adjust maximum skew and age for rendezvous descriptors: let skew
be 48 hours rather than 90 minutes.
- Reject malformed .onion addresses rather then passing them on as
normal web requests.
o Controller:
- More Tor controller support. See
http://tor.eff.org/doc/control-spec.txt for all the new features,
including signals to emulate unix signals from any platform;
redirectstream; extendcircuit; mapaddress; getinfo; postdescriptor;
closestream; closecircuit; etc.
- Encode hashed controller passwords in hex instead of base64,
to make it easier to write controllers.
- Revise control spec and implementation to allow all log messages to
be sent to controller with their severities intact (suggested by
Matt Edman). Disable debug-level logs while delivering a debug-level
log to the controller, to prevent loop. Update TorControl to handle
new log event types.
o New config options/defaults:
- Begin scrubbing sensitive strings from logs by default. Turn off
the config option SafeLogging if you need to do debugging.
- New exit policy: accept most low-numbered ports, rather than
rejecting most low-numbered ports.
- Put a note in the torrc about abuse potential with the default
exit policy.
- Add support for CONNECTing through https proxies, with "HttpsProxy"
config option.
- Add HttpProxyAuthenticator and HttpsProxyAuthenticator support
based on patch from Adam Langley (basic auth only).
- Bump the default BandwidthRate from 1 MB to 2 MB, to accommodate
the fast servers that have been joining lately. (Clients are now
willing to load balance over up to 2 MB of advertised bandwidth
capacity too.)
- New config option MaxAdvertisedBandwidth which lets you advertise
a low bandwidthrate (to not attract as many circuits) while still
allowing a higher bandwidthrate in reality.
- Require BandwidthRate to be at least 20kB/s for servers.
- Add a NoPublish config option, so you can be a server (e.g. for
testing running Tor servers in other Tor networks) without
publishing your descriptor to the primary dirservers.
- Add a new AddressMap config directive to rewrite incoming socks
addresses. This lets you, for example, declare an implicit
required exit node for certain sites.
- Add a new TrackHostExits config directive to trigger addressmaps
for certain incoming socks addresses -- for sites that break when
your exit keeps changing (based on patch from Mike Perry).
- Split NewCircuitPeriod option into NewCircuitPeriod (30 secs),
which describes how often we retry making new circuits if current
ones are dirty, and MaxCircuitDirtiness (10 mins), which describes
how long we're willing to make use of an already-dirty circuit.
- Change compiled-in SHUTDOWN_WAIT_LENGTH from a fixed 30 secs to
a config option "ShutdownWaitLength" (when using kill -INT on
servers).
- Fix an edge case in parsing config options: if they say "--"
on the commandline, it's not a config option (thanks weasel).
- New config option DirAllowPrivateAddresses for authdirservers.
Now by default they refuse router descriptors that have non-IP or
private-IP addresses.
- Change DirFetchPeriod/StatusFetchPeriod to have a special "Be
smart" default value: low for servers and high for clients.
- Some people were putting "Address " in their torrc, and they had
a buggy resolver that resolved " " to 0.0.0.0. Oops.
- If DataDir is ~/.tor, and that expands to /.tor, then default to
LOCALSTATEDIR/tor instead.
- Implement --verify-config command-line option to check if your torrc
is valid without actually launching Tor.
o Logging improvements:
- When dirservers refuse a server descriptor, we now log its
contactinfo, platform, and the poster's IP address.
- Only warn once per nickname from add_nickname_list_to_smartlist()
per failure, so an entrynode or exitnode choice that's down won't
yell so much.
- When we're connecting to an OR and he's got a different nickname/key
than we were expecting, only complain loudly if we're an OP or a
dirserver. Complaining loudly to the OR admins just confuses them.
- Whine at you if you're a server and you don't set your contactinfo.
- Warn when exit policy implicitly allows local addresses.
- Give a better warning when some other server advertises an
ORPort that is actually an apache running ssl.
- If we get an incredibly skewed timestamp from a dirserver mirror
that isn't a verified OR, don't warn -- it's probably him that's
wrong.
- When a dirserver causes you to give a warn, mention which dirserver
it was.
- Initialize libevent later in the startup process, so the logs are
already established by the time we start logging libevent warns.
- Use correct errno on win32 if libevent fails.
- Check and warn about known-bad/slow libevent versions.
- Stop warning about sigpipes in the logs. We're going to
pretend that getting these occassionally is normal and fine.
o New contrib scripts:
- New experimental script tor/contrib/exitlist: a simple python
script to parse directories and find Tor nodes that exit to listed
addresses/ports.
- New experimental script tor/contrib/ExerciseServer.py (needs more
work) that uses the controller interface to build circuits and
fetch pages over them. This will help us bootstrap servers that
have lots of capacity but haven't noticed it yet.
- New experimental script tor/contrib/PathDemo.py (needs more work)
that uses the controller interface to let you choose whole paths
via addresses like
"<hostname>.