2004: The Year That Promised Email Authentication

[R.A. Hettinga]

<http://www.circleid.com/print/855_0_1_0/>
 

CircleID

2004: The Year That Promised Email Authentication

By: Yakov Shafranovich
>From CircleID
Addressing Spam
December 27, 2004

 As the year comes to a close, it is important to reflect on what has been
one of the major actions in the anti-spam arena this year: the quest for
email authentication. With email often called the "killer app" of the
Internet, it is important to reflect on any major changes proposed, or
implemented that can affect that basic tool that many of us have become to
rely on in our daily lives. And, while many of the debates involved myriads
of specialized mailing lists, standards organizations, conferences and even
some government agencies, it is important for the free and open source
software (FOSS) community as well as the Internet community at large, to
analyze and learn lessons from the events surrounding email authentication
in 2004.

 "THE GHOST OF CHRISTMAS PAST"

 The quest for email authentication did not start from scratch.
Authentication systems are a well known field in computer security, and
have been researched for quite some time. Nevertheless, it is only during
this past year that email authentication has gained a prominent push mainly
due to the ever increasing spam problem. As well known, the original email
architecture and protocols was not designed for an open network such as the
Internet. Therefore, the original designers failed to predict the virtual
tidal wave of junk email that took advantage of lack of authentication in
the Internet email. As the result, a junk email filter is considered one of
the essential tools any Internet citizen must have in his toolkit today.

 The push towards email authentication started in earnest with the
publication of a proposal called RMX by a German engineer called Hadmut
Danisch in early 2003. While other previous proposals have been published,
none have gained any kind of traction. Hadmut's proposal on the other hand
coincided with the opening of the Anti-Spam Research Group (ASRG) of the
Internet Research Task Force (IRTF), which as an affiliate body of the
IETF. The IETF created and currently maintains the Internet email
standards, and an IETF affiliate was a logical body to work on addressing
the spam problem on the Internet at large. Being that the ASRG brought
together a sizable chunk of the anti-spam world, RMX gained more exposure
that none of the previous work in the field ever had. What followed was a
succession of proposals forked off the original RMX proposal until the
spring of 2004 when most of them were basically confined to the dustbin of
history together with RMX. In the end, only two proposals with any sizable
following were left: Sender Policy Framework (SPF) and Microsoft's
Caller-ID.

 The author of SPF, Meng Wong, managed to attract a large community to his
proposal, giving it a much larger deployed base than any competitor. In
many ways this effort can be compared to some of the open source projects,
except this time this was an open standard rather than a piece of software.
On the other side of the ring, so to speak, was Microsoft which surprised
the email world with their own proposal called Caller-ID at the RSA
conference in early 2004. Eventually, the IETF agreed to consider
standardization of email authentication by opening a working group called
MARID in March of 2004. With the merger of SPF and Microsoft's new
Sender-ID proposal, hopes were running high about the coming success of
email authentication and the coming demise of spam. Yet, ironically this
working group earned itself a record by being one of the shortest in the
existence of the IETF - it has lasted a little over six months until being
formally shutdown in September of 2004.

 "ALL THAT IS GOLD DOES NOT GLITTER"

 During the work of IETF's MARID group the quest for the email
authentication begun to permeate circles outside the usual cadre of
anti-spam geeks. Technology publications, and even the mass media have
begun to take note of the efforts occurring on an obscure mailing list
tucked away among 200 other even more obscure groups, prodded in many cases
by the public relations spokesmen of various companies in the anti-spam
space, including Microsoft. Yet in many ways that was one of the fatal
blows to the group and any hope of a common standard for email
authentication.

 Several major issues arose during the operation of the working group. The
first major issue that has been bubbling beneath the surface was technical
in nature. SPF has come from a group of proposals that worked with the
parts of the email infrastructure that was unseen by most users. This
included email servers that exchanged email among ISPs and were unseen. In
the technical lingo this type of authentication was known as "path
authentication". It focused on authenticating the path the message took
place between servers, and dealt with machines instead of end users.
Sender-ID approached the problem from a different viewpoint. Prodded by
financial companies and the fact that Microsoft itself makes more email
client software than server software, Sender-ID dealt with the end user. It
focused on "message authentication", based on the path the message took.
While the goals make have been admirable, many technical questions arose as
to whether Sender-ID would work. Most of them were rooted in the basic
differences between path authentication vs. message authentication, and
remained unresolved.

 The second major issue that arose was one of intellectual property rights.
Microsoft filed for patents on parts of Sender-ID and was not forthcoming
with information during the operation of the MARID WG. While the actual
patent application were eventually published towards the end of life of the
WG that came too late. The damage to the trust among the group members, and
different parts of the community has already been done. The main point of
contention was not necessarily the patents applications themselves - rather
it was the mandatory patent license that Microsoft had drawn up. The
language in the Sender-ID patent license was construed in a way that
prevents use by any software licensed under the General Public License
(GPL). Whether that was intentional or not we may never know, but the trust
between Microsoft and the FOSS community which was strenuous at best was
broken.

 The third major issue which played itself outside the mailing lists and
hallways of the anti-spam world was the media. Given that the spam problem
was only increasing, the media pounced on what was seen as the golden grail
for stopping spam. Unfortunately, as most reporters are not knowledgeable
in either Internet architecture or email protocols, they frequently
reported email authentication as the final cure for spam. These created
great expectations for email authentication which were blown away once the
hard truth settled in: email authentication did not stop spam. Unlike what
many had believed, email authentication did not address the spam problem
directly. Rather, it was only the first step towards a larger solution with
reputation and accreditation systems planned for the future. However, as
this truth sunk in, many of the companies and community members were not as
positive towards email authentication as before.

 The various disagreements, technical and non-technical, led some of the
group participants to create their own alternatives proposals or look to
crypto-solutions such as Yahoo's DomainKeys. As a result, any useful work
of the MARID group slowed to a crawl with the IETF eventually shutting down
the group. A major factor in that decision was letters from two large
members in the FOSS community against Sender-ID: the Apache Foundation and
the Debian Project.

 "LET'S VISIT UNCLE SAM"

 With the shutdown of MARID WG in September of 2004, both Sender-ID and SPF
were left to fend for their own. While some have assumed that Sender-ID was
left of the dead after being rejected by the IETF shortly before the
closure of MARID, Microsoft was quietly gathering support for Sender-ID
among the industry. Microsoft's goals become clear at the FTC's Email
Authentication Summit in November of 2004: Sender-ID was pushed as an
accepted email authentication standard to be mandated by the FTC. Among the
sizable PR gains that Microsoft gained was the endorsement of Sender-ID by
AOL, and a letter signed by representatives of 25 major email companies and
ISPs, a list which curiously included Meng Wong, the author of SPF. The PR
advantage was so great, that SPF was not even listed on the FTC's website
for the conference. At the same time, other alternative proposals such as
CSV and BATV have begun promulgating among the industry, all of which born
during the death throes of MARID.

 The SPF community being faced with the choice of joining or rejecting
Sender-ID, was split. Majority of the community as judging by the mailing
list traffic opposed Sender-ID/SPF combination. Nevertheless, some members
including Meng Wong, the original author, endorsed Sender-ID. This has led
to a lot of infighting with an election of an "SPF Council". At this time,
the SPF community is the midst of a political discussion about its future.

 At the same time, a separate low-key effort in the IETF is taking place to
address some of the cryptography solutions for Internet email. Proposals
such as Yahoo's DomainKeys, Cisco's IdentifiedMail, etc. seek to achieve
"message authentication" promised by Sender-ID but on a much more solid
technical ground and with less IPR and PR issues. This effort is purposely
left low key with even the mailing list itself hard to find, and certainly
no media stories promising the end of spam. The IETF-MAILSIG effort as this
is now called seeks to avoid the same problems that doomed MARID with hopes
of developing useful technologies to reduce spam. Nevertheless, this effort
was high-key enough for some of the companies involved to show case it at
the FTC's summit. Needless to say, the FTC is staying silent on its plans.

 WHAT THE FUTURE HOLDS

 While we still don't have workable email authentication, the Sender-ID/SPF
saga did accomplish a lot in many other ways. These events have shown to
the technology community at large that the FOSS world plays an ever
increasing role in the Internet as whole. The Apache Foundation and the
Debian Project carried enough weight to the IETF to consider their opinion,
marking probably the first time that FOSS opinions carried significant
weight in the standards process.

 This debacle has also lead to an increased awareness of the growing
problems in the patent system with Sender-ID being cited as a prime example
of a patent system gone wrong. While smaller sagas such as PanIP's rampage
on small e-businesses, Acacia's assault of video streaming and other
similar incidents have been happening for a while, the Sender-ID/IETF story
has brought this issue to the forefront of the Internet community for at
least a short time. What has followed has been positive developments with
governments, corporations and individuals recognizing the increasing
problems in today's patent system and some beginning to seek reform.
 As for spam, Microsoft, Cisco, the SPF community and many others are still
working on it. Some of the positive developments coming out of the
Sender-ID episode have been an increased awareness of how the email
architecture actual works and the increased realization that better
coordination among the Internet community is necessary.

 As for email authentication - there is still 2005...

-- 
-----------------
R. A. Hettinga <mailto: rah at ibuc.com>
The Internet Bearer Underwriting Corporation <http://www.ibuc.com/>
44 Farquhar Street, Boston, MA 02131 USA
"... however it may deserve respect for its usefulness and antiquity,
[predicting the end of the world] has not been found agreeable to
experience." -- Edward Gibbon, 'Decline and Fall of the Roman Empire'