End of the road for SMTP?
By Paul Festa
CNET News.com

August 1, 2003, 4:00 AM PT

URL: http://zdnet.com.com/2100-1105-5058610.html

The protocol that has defined e-mail for more than two decades may have a fatal flaw: It trusts you.

Developed when the Internet was used almost exclusively by academics, the Simple Mail Transfer Protocol, or SMTP, assumes that you are who you say you are.

SMTP makes that assumption because it doesn't suspect that you're sending a Trojan horse virus, that you're making fraudulent pleas for money from the relations of deposed African dictators, or that you're hijacking somebody else's computer to send tens of millions of ads for herbal Viagra.

In other words, SMTP trusts too much--and that has spam foes, security mavens and even an original architect of today's e-mail system agitating for an overhaul, if not an outright replacement, of the omnipresent protocol.

I would suggest they just write a new protocol from the beginning, Suzanne Sluizer, a co-author of SMTP's immediate predecessor and a visiting lecturer at the University of New Mexico, said in an interview.

In my experience in computers--which at this point, is quite extensive--trying to fix problems in the existing thing is almost always more difficult than just sitting down and thinking about what you want and coming up with something new, she added.

Sluizer co-authored the 1981 Mail Transport Protocol, SMTP's direct predecessor, while she was a technical staffer at the of the University of Southern California's Information Sciences Institute in Marina del Rey, Calif.--the birthplace of such fundamental Internet protocols as the Transmission Control Protocol/Internet Protocol (TCP/IP).

Having battled the scourge of unsolicited junk e-mail on a catch-as-catch-can basis for years, e-mail experts are at their wits' end as spam load increases day by day with no sign of slowing.

America Online, AOL Time Warner's online unit, said in April that it had blocked 2.4 billion pieces of spam in a single day. Despite that feat, many spam messages probably made it through to AOL's 34 million members. Some estimates put the worldwide proportion of spam to legitimate e-mail at around 50 percent.

The root of spam:

Companies, from AOL to small start-ups, are tackling the spam problem by testing out technological fixes. These include collaborative spam-blocking filters and so-called challenge response methods, which require a typed-in response to foil automated registrations of free e-mail accounts. In addition, spam is keeping lawyers, legislators and lobbyists busy, as states and nations criminalize spam and recipients of it go after the senders.

Still, spam keeps coming. That has led technologists to re-examine SMTP as the root of e-mail's evils, just as it has always been lauded as the source of the medium's remarkable power and popularity.

At issue is the protocol's lack of a comprehensive way of verifying an e-mail sender's identity. This makes it easy for people to mask their identities by forging return addresses and taking over victim machines to conduct their activities.

The flaws are so severe, some now believe, that the protocol that gave rise to the most significant explosion in written communication since Gutenberg may no longer be capable of serving its purpose in a world of con artists, pornographers, virus authors and unscrupulous spammers.

You have to remember the era in which this protocol was designed, said Sluizer, the self-described grandmother of SMTP. Back in the time we were doing this work, we were talking about hundreds or maybe thousands of sites on what was then called the Arpanet. We were looking at connecting with a few in Europe and some smaller networks in the U.S. It was a trusted situation, and the protocols were developed on the basis of that trust. So it's very surprising to me that we are using the same protocols coming up on 25 years later, because you need different things in a commercial environment than you need in a research environment.

While critics generally agree on what SMTP lacks, debate abounds on how to fix it.

Some who worked on the protocol in its early days argue that it is flexible enough to have successfully evolved over the years--having absorbed numerous revisions and extensions--and that the authentication problem can be partially solved with existing technologies.

Authentication in SMTP is not that hard, Paul Hoffman, director of the Internet Mail Consortium and author of numerous computer-related books, wrote in an e-mail interview. There is already a protocol for doing it, namely running SMTP over SSL/TLS. And, yes, I wrote it. (The SMTP over SSL/TLS protocol is available at the Internet Engineering Task Force's Web site.)

The hard part, according to Hoffman and others, is establishing the trust relationships required to back up any computer-based authentication scheme--in other words, verifying that a person is who he or she claims to be.

The problem worsens, Hoffman said, when trying to design a system that authenticates mail servers, rather than individuals. In part, this is because a third party would have to determine whether an e-mail server is responsible for sending spam. That kind of responsibility--voluntarily assumed by operators of various spam blacklists--could be onerous and expensive if applied to the Internet as a whole.

Who is paying this third party for both the time and the legal risk in doing this? Hoffman asked.

Number crunch:

Some say rewriting SMTP from the ground up would be prohibitively difficult because of the protocol's global user base, which is estimated to be in the hundreds of millions.

The difficulty of changing the transfer technology as a way of managing unsolicited bulk e-mail is the installed base, said Rodney Tillotson, the chair of the Anti-Spam Working Group for the Reseaux IP Europeens (RIPE), a consortium of European Internet service providers.

There are thousands or millions of SMTP servers transferring and delivering mail, and getting them all changed will take years, during which time the (unsolicited bulk e-mail) problem probably remains unsolved, Tillotson said. Proposals requiring a change to desktop mail software are even harder to deploy.

Sluizer counters this by suggesting two protocols--SMTP and a new one, with tighter authentication--could easily coexist, with e-mail applications supporting both side by side. In that way, people using one protocol would not be prevented from exchanging mail with those using another.

The RIPE antispam group isn't alone in conducting an online debate about changing fundamental protocols to stem the tide of spam. The Internet Engineering Task Force (IETF) this spring established a research group to come up with ideas on how to attack the problem from the protocol level.

But critics call the IETF's efforts belated and say that efforts to solve the spam crisis can't wait while a standards body deliberates.

Given that it's taken six-plus years for the IETF to get around to deciding spam is a big enough issue that they should charter a 'research group' to look at it, I just can't bring myself to be hopeful that we'll see the IETF ratifying any major overhauls to SMTP before the decade is out, Ray Everett-Church, chief privacy officer of the ePrivacy Group, said in an e-mail interview.

Paul Judge, chair of the IETF's Anti-Spam Research Group (ASRG) declined to answer questions for this story, citing both the group's desire to maintain focus and the quantity of proposals under consideration.

The view that an entirely new e-mail specification should be written isn't making headway within the IETF. Many of the organization's members argue that a practical solution could ride atop the present protocol or at least be backward-compatible with it.

Add, not replace?

Among others arguing for a less-radical fix is the ePrivacy Group, which markets the SpamSquelcher spam-control software for Internet service providers. In April, the group published its Trusted E-mail Open Standard (TEOS). That proposal builds on top of SMTP, rather than replacing it outright.

TEOS, according to is authors, lets people and organizations identify themselves more reliably and include machine-readable descriptions or assertions about their e-mail's content. It also establishes an encrypted, spoof-proof trust stamp that appears in the body of the message. The ePrivacy Group recommends the formation of an international, cross-industry body to maintain the standard.

Some of those tackling the problem are looking at amending protocols other than SMTP. Microsoft, for example, advocates a change to the domain name system (DNS) that would make it harder for spammers to disguise their identity.

The DNS is a distributed database, maintained by a number of different companies that provide domain names for Web site and e-mail addresses. The problem with the system, spam-fighters say, is that like SMTP, it provides no system for authentication.

One of the things we want to do is attack this issue of spoofing, said Harry Katz, program manager of Microsoft's Exchange server group. That's job one, in terms of putting a curb on spam, and we think we can do that (by) making a minor enhancement to the DNS.

The minor enhancement Microsoft is preparing to release would let individuals, companies and other organizations publish the identification numbers of their mail servers in the DNS database.

That would let an e-mail recipient compare the message's actual originating address with the address indicated in its header. A difference there could help a spam filter determine that a header is spoofed, increasing the likelihood that the message is spam. Such messages could easily be filtered or rejected.

The IETF's antispam research group has been entertaining a DNS alteration of its own, ever since the group was started this spring.

Conveniently, the DNS is flexible enough to allow for a change without requiring a major revision to the system protocol, but it would require a concerted implementation by various Internet mail companies. Microsoft--with its Hotmail Web mail service, its MSN mail service, and others under its control--could single-handedly give such a system a sizeable implementation boost.

There are people muttering darkly that it's a lost cause, there's no way that it's fixable and we have to start from scratch, said Katz. I would disagree with that. Analogies are dangerous, but while we may have legitimate concerns about traffic on the roads, do we have to tear down the interstates in response? The answer is 'no,' and there are things we can do that, over time, will make a significant dent in the problem.

Open to risk:

Katz warned that, in the rush to fix e-mail, the industry risks harming the openness that gave rise to the Internet's success in the first place.

For example, a spam solution should not block all unsolicited mail, he said. That could prevent the reunion of long-separated friends and relatives, as described in so many e-mail success stories. Nor should a solution put an end to bulk mailings per se.

Even Microsoft's soon-to-be-proposed DNS modifications will have to tread a fine line to make sure they don't bar third parties from sending out legitimate ads.

There's a balance that has to be struck, Katz said. We want to ensure that people can communicate easily and effectively over the Internet. That said, over time you're going to see that the system will be tightened down more than it is today. It just has to be, because of problems like spam and viruses. But you still need to leave the door open for the classic scenario, when your long-lost high school buddy contacts you by e-mail. That's one of the great strengths of the Internet.