The next gen P2P secure email solution
Tue Dec 24 01:20:51 PST 2013
This thread pertains specifically to the use of P2P/DHT models
to replace traditional email as we know it today. There was
a former similarly named thread on this that diverged... from the
concept and challenge of P2P/DHT handling the transport and
lookups... back to more traditional models. This thread does not
care about those antique models, please do not take it there.
In short, we're attempting to examine and develop some form
of new transport that looks somewhat like a mix between secure
anonymous networks, string@pubkey node addressing, massive
decentralized dht-like scaling and finally a user facing daemon that
moves messages into and out of local spools for use by normal
Pasting in a very rough and unflowing thread summary to date
for interested people to pick up and discuss, draft, etc.
> [pgp/smime email encryption, etc]
> What is the gap we have to close to turn this on by default?
How many times has this been rehashed the last six months?
You can't fix email as we know it today using todays bolt-ons,
protocols and corporate stakeholders/services trying to profit from it.
The only way to have any real global seamless success is to go
ground up with a completely new model. IMO, that will be some
form of p2p message system where every address is a crypto key,
masked for grandma by her contact list, decrypted out your p2p
daemon and piped into your local mail processing (MUA/filter/lists)
and filesystem (encryption). At least that way your local mail tools
will still work (no one will give those up anyway).
The problem is the antique centralized backend, it needs bypassed.
You've got neat stuff like Tor, bittorrent, bitcoin, etc already... so
boost email into the 2020's the same way. Then let the old world
email services try to keep up, and slowly die like everything else.
On Mon, Nov 25, 2013 at 1:01 AM, ianG <email@example.com> wrote:
> On 23/11/13 15:30 PM, Ralf Senderek wrote:
>> On Sat, 23 Nov 2013, David Mercer wrote:
>>> But of course you're right about actual current usage, encrypted email
>>> is an
>>> epic fail on that measure regardless of format/protocol.
>> Yes, but it's about time we do something about that. Do we *exactly know
>> why* it is such a failure?
> It's an interesting question, and one worth studying for pedagogical
> motives. From my experiences from both sides, it is clear that both sides
> failed. But for different reasons.
> Hence, I've concluded that email is unsecurable.
Obviously. It will never be able to escape the non-body
header content and third party routing, storage and analysis with
any form of patching over today's mail. And it's completely
ridiculous that people continue to invest [aka: waste] effort in
'securing' it. The best you'll ever get clients down to is exposing
a single 'To:' header within an antique transport model that
forces you to authenticate to it in order to despam, bill, censor
and control you.
That system is cooked, done and properly fucked. Abandon it.
What the world needs now is a real peer to peer messaging
system that scales. Take Tor for a partial example... so long
as all the sender/recipient nodes [onions] are up, any message
you send will get through, encrypted, in real time. If a recipient
is not up, you queue it locally till they are... no third party ever
needed, and you get lossless delivery and confirmation for free.
Unmemorable node address?, quit crying and make use of your
local address book. Doesn't have plugins for current clients?,
so what, write some and use it if you're dumb enough to mix
the old and new mail.
The only real problem that still needs solved is scalability...
what p2p node lookup systems are out there that will handle
a messaging world's population worth of nodes [billions] and
their keys and tertiary data? If you can do that, you should
be able to get some anon transport over the p2p for free.
Anyway, p2p messaging and anonymous transports have
all been dreamed up by others before. But now is the
time to actually abandon traditional email and just do it.
If you build it, they will come.
I'm strongly against most the ideas to abbandon current email systems,
because the results will be to create wallet garden.
We need something interoperable with existing systems or the system will
just be used by a bunch of paranoid people or fostered by the marketing
of few cryptography company acquiring customers, not user.
It would be interoperable with current end user interfaces/tools, but not
directly with firstname.lastname@example.org.
As with everything else, today's seeds grow into tomorrows garden, sometimes
you just have to thorougly plow under last year's chaff first, that's by design.
E-mail is basically business correspondence.
- E-mail is stored.
- E-mail is sent to many people outside your personal social network.
- Business recipients of email are often subject to corporate and/or
regulatory policy constraints that are in conflict with end-to-end
The above list of features can be greatly expanded, and the
consequences elaborated, but I doubt may on this list truly need
to be re-educated about email.
So I will confidently predict that end-to-end secure email will
remain a niche service used by a tiny minority.
Even businesses that one might expect to implement at least encryption
to the "gateway", are in many cases choosing to out-source their
gateway to 3rd-party providers (anti-spam and anti-virus offerings
only work with un-encrypted email, and in many cases the provider
also operates the entire mail store).
[and others also said: tls, dane, skype, smime, ca's, smtp, etc]
Nothing here changes, only the backend transport between nodes.
Once your node decrypts the message to your local system pipes,
you can do all this and more with it. Including running a 'business' node
and doing whatever you want with the plaintext after your node daemon
passes it to you. This is not about those ancient protocols. It's also
about 'messaging' not strictly 'email'... those lines are already well
blurred, no need to distinguish between them anymore. A 'light'
messaging client could simply ignore the non transport email headers,
or use your standard msg@nodekey address.
Please do not continue to talk about antique tradional centralized
systems in this thread, except of course to bash and route around them.
The speed of a real P2P/DHT replacement at scale is a research challenge.
this would make an interesting bet! i too believe this to be
impossible given the constraints.
I'm not so sure about this, look at all the global resources being poured
into traditional email, and attempts to 'fix' it. Now redirect fractional 1%
of those resources and put them into a P2P replacement. That's ftw.
Say hello to Bote mail on I2P.
I2P provides encrypted anonymizing networking, Bote mail provides DHT
based serverless encrypted mailing with public crypto keys as
addresses (ECDSA or NTRU).
http://i2p2.de and i2pbote.i2p (if you don't have I2P installed, add
.us to visit it via an inproxy).
There is also I2P Messenger that is encrypted P2P IM within I2P also
using public keys as addresses.
You may have a look of "I2P Bote" it is severless, encrypted mail
system, address is the public key, P2P based... nice tool.
> You may have a look of "I2P Bote" it is severless, encrypted mail
> system, address is the public key, P2P based... nice tool.
As in another post of mine, I'll be looking at that again.
My first take was that it stores the messages in the DHT,
which didn't seem scalable or reliable at all. I may be
wrong as I read more later.
> Afterwards you can add the "I2P Bote plugin", the serverless mail
> system. SMTP- and POP3 support was on the ToDo list some times ago, I
I think that's working now. And is the general idea, create a strong
overlay network with a frontend MUA's can speak to.
As an aside: If you can make that overlay net present an IPv6 tunnel
interface on the local host, that lets you use any IPv6 enabled
app over it. I'm doubting the world needs a dozen application
specific overlay networks. More like just a few classes of network.
- message based store and forward
- low latency IPv6 transport
- data storage and retrieval
Bote mail doesn't have to be used for it's anonymous properties, for
me that is just a bonus. For many people it is more than enough to be
able to know that it is impossible for anybody else than the intended
recipient to read the message thanks to public key addressing.
Guaranteed end-to-end security if you can verify the address.
I don't think anything that doesn't fundamentally rely on public key
addressing is the (long term) future. There will inevitably issues
otherwise, including more issues of the type we have with CA:s today.
For those who want to make it more user friendly, nothing stops you
from adding a transparent "address translation layer" where servers
are involved. When you want to send a message to a person with a human
readable address at a server, then the server could then reply with
the public key based address to the mail client, and the user would
have the option to see what that address is. It could even be logged
by the client, with TOFU/POP style trust system that reduces the
amount of trust you have to place in the server. No need to trust
anybody with plaintext.
I suggest such interfacing with the current legacy system will only prolong
it's long past due extinction. Build a better system and let them come to you,
not the other way around. And bolting in exits will only make it harder to
do correctly and optimally what you need to do as a P2P system. Please,
just forget about interfacing with the legacy transport system. If you really
need that you can run your own p2p daemon node that acts as your
automated gateway between the two. This is mostly about design of
the p2p side, not that.
It is my understanding of the proposed replacement for email. Magic
email addresses that in fact correspond to an identifier of a public
key, for example the hash of a rule that identifies the public key,
and which result in your message not in fact being passed along by
If you're not going to send directly to the very long account@nodepubkey, then
yes, you'll need to create another layer on top to hold your h(key). However,
the key must still be obtainable behind that since that is what the messages
will ultimately be encrypted and routed to. It's not much of a stretch beyond
that to ensure userland mail tools can handle say 8k long addresses. I'm not
against such a [vanity/shorthash] layer.
Bote mail and I2P messenger are two pieces of serverless software that
ALREADY is public key addressed within I2P. Have you tried them? You
need to add the public keys of the recipients to be able to send a
message to start with, although the DHT based search system Seedless
allow you to publish Bote mail addresses to the network.
(IMAP support for Bote mail is planned but not yet implemented, right
now it has a local web interface.)
Maybe Namecoin could work together with them to enable you to register
your key addresses to your nickname in a secure manner, but then you
still have to have a globally unique nickname and tell people the
> If you are so sure, can you tell us how the next generation secure email
> solution will solve the "trust problem", please.
Though unclear, that sounds like the old trust of a CA/PKI system problem.
> How does the p2p daemon
> find the correct crypto key, so that every user can rely on its invisible
In general I suggest that people wish to use messaging with each other
once they already know them (or have some other trusted web to them).
As in, Hey John, nice to meet ya today, what's your key (address), I'll
message you later. Or Hey Jane, what's John's address. Same for
employers, businesses, etc. Such peer groups bootstrap and grow
very fast. Thus the perceived need for a cold lookup of Ralf, isn't much of
a real one.
Once you know the address (node crypto key), you put it 'To: <key>',
mua hands to spool, p2p daemon reads spool, looks up key in DHT and
sends msg off across the transport to the far key (node) when it is
reachable. Hopefully the transport looks like I2P/Tor in being a secure
random hop layer. In fact, those could probably be used today, they
have the keys as nodes and user facing ports for inbound/outbound
daemons. They just need scaling work to n-billion nodes (users,
aka: the hard part). People are already plugging postfix, bittorrent,
etc into these networks.
Tor is not currently addressible at the user level by the full key,
it 'shortens' the key into a 16char onion address. As you may be
hinting at... yes, that is bad... collisions, and needing secondary lookup
layers into the full key. Tor may be moving to full key addressibility
soon, see tor-dev for that.
I2P (and Phantom, and probably GnuNet) are addressible with full keys.
So you can send to 'account@key' with them if you want, and keep the
John/Jane/Ralf human style lookups in your MUA addressbook (once
you know them) without needing a secondary lookup layer into the full key.
No, I am not sure. But when looking at some of the p2p transport
layers that have come along so far, it seems like a fairly strong
possibility for a new backend transport model while retaining user
level mail tools... mutt, maildrop, mailman, Thunderbird, etc. Most
of what you'd need there is support for very long addresses and
split horizon handoff to local daemon/spool based on recognizing
what the destination net is... .onion, .i2p, etc.
I'd like to read what Pond and I2P-Bote are doing with some parts of
this as well.
I don't believe you need a trusted CA/PKI service to successfully
bootstrap users and their addresses/keys into a new global messaging
system. If I want to know what some unknown like Bruce's key is, I'll
look it up on his website, social net, list posts, etc. If that's what you
I feel like I walking in halfway into a conversation, I'm guessing this
started on the cryptography list that I'm not on.
Your DHT comment caught my attention though. What in particular about
DHTs don't seem scalable or reliable?
Seems like DHTs are regularly in the 5-10M range and I don't see any
reason that DHTs couldn't be 10 times that.
Any reasonable churn rate and reliability could be handled with
replication. The bit-torrent DHT for instance claims that 45% of users
that bootstrap from a central node are reachable 15 minutes later. So
typical setups involve 8 nodes per bin, and 20 bins. So every 15
minutes you ping 160 hosts, only reach 45%, and do some work to
repopulate the missing slots.
Given the simplicity of the bit-torrent DHT I think there's plenty of
room for improvement. Larger routing tables are obvious (at the cost of
more network bandwidth to track peers).
The most promising idea for DHT improvements I've seen is to divide
peers into 3 latency groups. High, medium, and low. Much like L1
cache, L2 cache, and main memory. That way common queries are very
fast, yet all queries still to find keys globally.
Bittorrent is already in the 100m node range. That's not enough. This
needs to replace every possible messaging user on the planet over
the duration of their actiive lifetime. That's at least a couple billion nodes.
Don't forget, you can always use disk to cache things.
> Need a system for handing one's keys around that protects end users from
> the horrifying sight of actual keys or actual strong hashes of keys.
But at the same time the system has to not say, "I can't deliver your
message to that person because an invisible gnotzaframmit that I won't
describe to you seems to be unavailable to me in the flabogrommit."
Address books as usual. Name layer if need be. We are humans, we
learn new lexicons, we write manuals, that part is nothing to be afraid of.
Being afraid only holds you back.
> I don't believe you need a trusted CA/PKI service to successfully
> bootstrap users and their addresses/keys into a new global messaging
> system. If I want to know what some unknown like Bruce's key is, I'll
> look it up on his website, social net, list posts, etc. If that's what you
> You can use an untrusted CA to bootstrap. I show how it can be done at:
This is an interesting idea, because it provides certificates on
demand for ordinary users, if they decide to sign up to a certain
certs are then being used for other purposes, so the site does act as a
bootstap for using crypto. The one thing that this proposal relies on is
the availability of a common piece of software (user agent) that stores
the private key for the user. It's this part of the picture where things
There can be no non-distributed/redundant elements in this p2p system,
aka: no 'sites', certainly none with a realworld IP on them, and none where
very high percentages of them vanishing will disrupt the system for others.
This must replace email, therefore system disruption is intolerable.
More information about the cypherpunks