Want CNET to notify you of price drops and the latest stories?

Spam blockers may wreak e-mail havoc

CNET News.com's Declan McCullagh writes that the growing use of technology to stop spam may also inadvertently prevent legitimate e-mail messages from reaching their destinations.

Declan McCullagh Former Senior Writer
Declan McCullagh is the chief political correspondent for CNET. You can e-mail him or follow him on Twitter as declanm. Declan previously was a reporter for Time and the Washington bureau chief for Wired and wrote the Taking Liberties section and Other People's Money column for CBS News' Web site.
Declan McCullagh
5 min read
Here's an unhappy prediction: The explosion of spam-blocking technology could herald the death of much legitimate e-mail.

I wrote about patents relating to this technology, known as challenge-response technology, last week. Basically, when your mailbox is protected by a challenge-response system, people who try to contact you will be greeted with a response saying something like "click on this link to deliver this message" or "type in the word you see in the box above." The idea is to block increasingly obnoxious spam bots but still let actual humans get in touch with you.

In theory, well-designed challenge-response utilities won't challenge mail from known correspondents or mail that you've actually asked to receive. Unfortunately, many current challenge-response systems are poorly designed, which could wreak havoc on mailing lists and other legitimate communications. This could make e-mail far less useful than it is today.

It's already starting to happen. SpamArrest.com began challenging mailing list messages last year. Recently Mail-block.com and iPermitMail.com followed suit.

When that happens, the operator of the mailing list receives a message--from each subscriber using the poorly designed challenge-response utility--that asks the list operator to respond to the challenge. Replying to a handful of challenges is no big deal, but if many subscribers start using poor challenge-response software, it will pose a serious problem for mailing list operators. Big corporations may be able to afford to hire someone to sit in front of a computer and spend all day proving they're not a spam bot, but nonprofit groups, individuals and smaller companies probably can't.

Challenge-response systems, ironically, share some characteristics with spam: In small quantities, both are only mildly annoying to the recipient. But as quantities increase, they make it more difficult to use e-mail at all. MailFrontier.net is a good example: It prevents its users from signing up to mailing lists unless the list operator manually intervenes to answer the challenge, a process that is exactly backward.

The enormous growth in spam means that challenge-response technology will become more popular. EarthLink recently announced it would make a challenge-response system available to its customers by the end of May, and the field is wide open, with no market leader so far.

EarthLink's announcement has alarmed veteran list operators, who view it as a model that other Internet service providers may follow. Dave Farber, the University of Pennsylvania computer scientist who runs the "interesting people" list, warned his subscribers: "If I start getting a flood of challenges from EarthLink IPers that require my response I will most likely declare them spam and you will stop receiving IP mail. I fully expect this to be the case for almost all the legitimate mailing lists you are on and count on."

This could make e-mail far less useful than it is today.
Editors at TidBits, the popular Macintosh newsletter that boasts about 50,000 subscribers, wrote a message on May 13 to readers: "Be warned that we will not answer any challenges generated in response to our mailing list postings. Thus, if you're using a challenge-response system and not receiving TidBits, you'll need to figure that out on your own."

It's worth remembering that, while they may not be as glamorous as the Web, peer-to-peer applications, or instant messaging software, mailing lists are the Internet's oldest form of mass communication. They date back to the original "MsgGroup" list in 1975, which the same Dave Farber--then at the University of California at Irvine--helped to create. Then the famous "sf-lovers" list came along, and the rest is, well, history.

Nowadays just about every organization uses mailing lists of some type, from Hotwire.com's cheap airfare announcements to the left-leaning activists at MoveOn.org who organized a massive e-mail campaign against the Iraq war. Professional organizations use them to contact members; companies offer deals to existing customers; and advocacy groups rely on lists to rally support for political causes. And that's not counting services like Yahoo Groups and Topica.

Another downside to challenge-response systems is that they can be exploited by spammers, yielding false negatives in addition to false positives. Some challenge-response systems require only that the sender reply to the challenge; others require only that a hyperlink in the challenge be followed.

A more pernicious problem is that challenge-response systems trust the "From:" line of a message. If challenge-response systems become sufficiently widespread, spam bots may start trying to guess at who your correspondents are--and then forge the "From:" header appropriately--by subscribing to discussion lists or following links from your personal or company home page. Digital signatures are probably the only way to prevent that kind of attack.

John Levine, an author, moderator of the comp.compilers Usenet newsgroup and veteran Internet hand, offers a gloomy worst-case prediction. "So what will the effect of this be?" Levine asks. "You won't be able to trust that mail from your friends is actually from your friends, since an increasing fraction will be spam leaking through your challenge system. What will people do? Given the basic principle of challenge systems, which is that it's someone else's job to solve your spam problem, people will dump their white lists and start challenging every message."

At least right now, because challenge-response systems are so easy for programmers to create, there are plenty of them, and the potential for market dominance has attracted some companies of dubious virtue. SpamArrest spammed advertisements to people who e-mailed its customers (imagine if AOL or MSN claimed the right to spam anyone who's ever sent you mail). Mail-block.com has been blocked by Outblaze.com, a large mail provider, for spamming. And MailWiper.com has been caught spamming.

For a challenge-response system to work properly, it will need to be tightly integrated with the mail client--so it knows who you contacted--and it should understand popular mailing list software such as Majordomo, Mailman and Listserv. It's easier for challenge-response companies that sell Web-based e-mail. For people using software like Eudora and Outlook, that probably means plug-ins or an e-mail proxy server that let the challenge-response system keep track of your outgoing messages.

Brad Templeton, chairman of the Electronic Frontier Foundation and author of one of the first challenge-response systems, compiled a useful list of design principles for challenge-response systems earlier this month. Templeton's list has some recommendations: Never challenge any mail that's a reply to a private message you sent; use multiple e-mail addresses; and never challenge mailing-list messages.

All these should be obvious, but many challenge-response systems just don't follow them. Fortunately, the Internet Engineering Task Force's Anti-Spam Research Group is spending some time trying to devise a reasonable standard.

Challenge-response systems may turn out to be the only way to inoculate ourselves against the spam epidemic. Or they may not. But their designers and users should think twice before trusting the future of Internet e-mail to buggy and problematic technology.