2004: The Year That Promised Email Authentication

[Tyler Durden]

I see RAHWEH is back from visiting the relatives...

-TD



>From: "R.A. Hettinga" <rah at shipwright.com>
>To: cryptography at metzdowd.com, cypherpunks at al-qaeda.net
>Subject: 2004: The Year That Promised Email Authentication
>Date: Mon, 27 Dec 2004 16:49:01 -0500
>
><http://www.circleid.com/print/855_0_1_0/>
>
>
>CircleID
>
>2004: The Year That Promised Email Authentication
>
>By: Yakov Shafranovich
> >From CircleID
>Addressing Spam
>December 27, 2004
>
>  As the year comes to a close, it is important to reflect on what has been
>one of the major actions in the anti-spam arena this year: the quest for
>email authentication. With email often called the "killer app" of the
>Internet, it is important to reflect on any major changes proposed, or
>implemented that can affect that basic tool that many of us have become to
>rely on in our daily lives. And, while many of the debates involved myriads
>of specialized mailing lists, standards organizations, conferences and even
>some government agencies, it is important for the free and open source
>software (FOSS) community as well as the Internet community at large, to
>analyze and learn lessons from the events surrounding email authentication
>in 2004.
>
>  "THE GHOST OF CHRISTMAS PAST"
>
>  The quest for email authentication did not start from scratch.
>Authentication systems are a well known field in computer security, and
>have been researched for quite some time. Nevertheless, it is only during
>this past year that email authentication has gained a prominent push mainly
>due to the ever increasing spam problem. As well known, the original email
>architecture and protocols was not designed for an open network such as the
>Internet. Therefore, the original designers failed to predict the virtual
>tidal wave of junk email that took advantage of lack of authentication in
>the Internet email. As the result, a junk email filter is considered one of
>the essential tools any Internet citizen must have in his toolkit today.
>
>  The push towards email authentication started in earnest with the
>publication of a proposal called RMX by a German engineer called Hadmut
>Danisch in early 2003. While other previous proposals have been published,
>none have gained any kind of traction. Hadmut's proposal on the other hand
>coincided with the opening of the Anti-Spam Research Group (ASRG) of the
>Internet Research Task Force (IRTF), which as an affiliate body of the
>IETF. The IETF created and currently maintains the Internet email
>standards, and an IETF affiliate was a logical body to work on addressing
>the spam problem on the Internet at large. Being that the ASRG brought
>together a sizable chunk of the anti-spam world, RMX gained more exposure
>that none of the previous work in the field ever had. What followed was a
>succession of proposals forked off the original RMX proposal until the
>spring of 2004 when most of them were basically confined to the dustbin of
>history together with RMX. In the end, only two proposals with any sizable
>following were left: Sender Policy Framework (SPF) and Microsoft's
>Caller-ID.
>
>  The author of SPF, Meng Wong, managed to attract a large community to his
>proposal, giving it a much larger deployed base than any competitor. In
>many ways this effort can be compared to some of the open source projects,
>except this time this was an open standard rather than a piece of software.
>On the other side of the ring, so to speak, was Microsoft which surprised
>the email world with their own proposal called Caller-ID at the RSA
>conference in early 2004. Eventually, the IETF agreed to consider
>standardization of email authentication by opening a working group called
>MARID in March of 2004. With the merger of SPF and Microsoft's new
>Sender-ID proposal, hopes were running high about the coming success of
>email authentication and the coming demise of spam. Yet, ironically this
>working group earned itself a record by being one of the shortest in the
>existence of the IETF - it has lasted a little over six months until being
>formally shutdown in September of 2004.
>
>  "ALL THAT IS GOLD DOES NOT GLITTER"
>
>  During the work of IETF's MARID group the quest for the email
>authentication begun to permeate circles outside the usual cadre of
>anti-spam geeks. Technology publications, and even the mass media have
>begun to take note of the efforts occurring on an obscure mailing list
>tucked away among 200 other even more obscure groups, prodded in many cases
>by the public relations spokesmen of various companies in the anti-spam
>space, including Microsoft. Yet in many ways that was one of the fatal
>blows to the group and any hope of a common standard for email
>authentication.
>
>  Several major issues arose during the operation of the working group. The
>first major issue that has been bubbling beneath the surface was technical
>in nature. SPF has come from a group of proposals that worked with the
>parts of the email infrastructure that was unseen by most users. This
>included email servers that exchanged email among ISPs and were unseen. In
>the technical lingo this type of authentication was known as "path
>authentication". It focused on authenticating the path the message took
>place between servers, and dealt with machines instead of end users.
>Sender-ID approached the problem from a different viewpoint. Prodded by
>financial companies and the fact that Microsoft itself makes more email
>client software than server software, Sender-ID dealt with the end user. It
>focused on "message authentication", based on the path the message took.
>While the goals make have been admirable, many technical questions arose as
>to whether Sender-ID would work. Most of them were rooted in the basic
>differences between path authentication vs. message authentication, and
>remained unresolved.
>
>  The second major issue that arose was one of intellectual property 
>rights.
>Microsoft filed for patents on parts of Sender-ID and was not forthcoming
>with information during the operation of the MARID WG. While the actual
>patent application were eventually published towards the end of life of the
>WG that came too late. The damage to the trust among the group members, and
>different parts of the community has already been done. The main point of
>contention was not necessarily the patents applications themselves - rather
>it was the mandatory patent license that Microsoft had drawn up. The
>language in the Sender-ID patent license was construed in a way that
>prevents use by any software licensed under the General Public License
>(GPL). Whether that was intentional or not we may never know, but the trust
>between Microsoft and the FOSS community which was strenuous at best was
>broken.
>
>  The third major issue which played itself outside the mailing lists and
>hallways of the anti-spam world was the media. Given that the spam problem
>was only increasing, the media pounced on what was seen as the golden grail
>for stopping spam. Unfortunately, as most reporters are not knowledgeable
>in either Internet architecture or email protocols, they frequently
>reported email authentication as the final cure for spam. These created
>great expectations for email authentication which were blown away once the
>hard truth settled in: email authentication did not stop spam. Unlike what
>many had believed, email authentication did not address the spam problem
>directly. Rather, it was only the first step towards a larger solution with
>reputation and accreditation systems planned for the future. However, as
>this truth sunk in, many of the companies and community members were not as
>positive towards email authentication as before.
>
>  The various disagreements, technical and non-technical, led some of the
>group participants to create their own alternatives proposals or look to
>crypto-solutions such as Yahoo's DomainKeys. As a result, any useful work
>of the MARID group slowed to a crawl with the IETF eventually shutting down
>the group. A major factor in that decision was letters from two large
>members in the FOSS community against Sender-ID: the Apache Foundation and
>the Debian Project.
>
>  "LET'S VISIT UNCLE SAM"
>
>  With the shutdown of MARID WG in September of 2004, both Sender-ID and 
>SPF
>were left to fend for their own. While some have assumed that Sender-ID was
>left of the dead after being rejected by the IETF shortly before the
>closure of MARID, Microsoft was quietly gathering support for Sender-ID
>among the industry. Microsoft's goals become clear at the FTC's Email
>Authentication Summit in November of 2004: Sender-ID was pushed as an
>accepted email authentication standard to be mandated by the FTC. Among the
>sizable PR gains that Microsoft gained was the endorsement of Sender-ID by
>AOL, and a letter signed by representatives of 25 major email companies and
>ISPs, a list which curiously included Meng Wong, the author of SPF. The PR
>advantage was so great, that SPF was not even listed on the FTC's website
>for the conference. At the same time, other alternative proposals such as
>CSV and BATV have begun promulgating among the industry, all of which born
>during the death throes of MARID.
>
>  The SPF community being faced with the choice of joining or rejecting
>Sender-ID, was split. Majority of the community as judging by the mailing
>list traffic opposed Sender-ID/SPF combination. Nevertheless, some members
>including Meng Wong, the original author, endorsed Sender-ID. This has led
>to a lot of infighting with an election of an "SPF Council". At this time,
>the SPF community is the midst of a political discussion about its future.
>
>  At the same time, a separate low-key effort in the IETF is taking place 
>to
>address some of the cryptography solutions for Internet email. Proposals
>such as Yahoo's DomainKeys, Cisco's IdentifiedMail, etc. seek to achieve
>"message authentication" promised by Sender-ID but on a much more solid
>technical ground and with less IPR and PR issues. This effort is purposely
>left low key with even the mailing list itself hard to find, and certainly
>no media stories promising the end of spam. The IETF-MAILSIG effort as this
>is now called seeks to avoid the same problems that doomed MARID with hopes
>of developing useful technologies to reduce spam. Nevertheless, this effort
>was high-key enough for some of the companies involved to show case it at
>the FTC's summit. Needless to say, the FTC is staying silent on its plans.
>
>  WHAT THE FUTURE HOLDS
>
>  While we still don't have workable email authentication, the 
>Sender-ID/SPF
>saga did accomplish a lot in many other ways. These events have shown to
>the technology community at large that the FOSS world plays an ever
>increasing role in the Internet as whole. The Apache Foundation and the
>Debian Project carried enough weight to the IETF to consider their opinion,
>marking probably the first time that FOSS opinions carried significant
>weight in the standards process.
>
>  This debacle has also lead to an increased awareness of the growing
>problems in the patent system with Sender-ID being cited as a prime example
>of a patent system gone wrong. While smaller sagas such as PanIP's rampage
>on small e-businesses, Acacia's assault of video streaming and other
>similar incidents have been happening for a while, the Sender-ID/IETF story
>has brought this issue to the forefront of the Internet community for at
>least a short time. What has followed has been positive developments with
>governments, corporations and individuals recognizing the increasing
>problems in today's patent system and some beginning to seek reform.
>  As for spam, Microsoft, Cisco, the SPF community and many others are 
>still
>working on it. Some of the positive developments coming out of the
>Sender-ID episode have been an increased awareness of how the email
>architecture actual works and the increased realization that better
>coordination among the Internet community is necessary.
>
>  As for email authentication - there is still 2005...
>
>--
>-----------------
>R. A. Hettinga <mailto: rah at ibuc.com>
>The Internet Bearer Underwriting Corporation <http://www.ibuc.com/>
>44 Farquhar Street, Boston, MA 02131 USA
>"... however it may deserve respect for its usefulness and antiquity,
>[predicting the end of the world] has not been found agreeable to
>experience." -- Edward Gibbon, 'Decline and Fall of the Roman Empire'