Study finds filters catch only a fraction of spam

A test of 15 email services over a 10-day period in March showed that AT&T WorldNet was the most effective at filtering spam among those rated, blocking 73 percent of unsolicited junk messages sent. The closest competitors were America Online, which blocked 40 percent of test spam messages, and Yahoo mail, which blocked 36 percent. Brightmail, which provides spam filtering services for AT&T, sponsored the independent eTesting Labs study.

In addition, the report found that the controversial free spam filter run by the Mail Abuse Prevention System (MAPS), known as the Realtime Blackhole List (RBL), blocked just 2 percent of 922 spam messages sent. The RBL also blocked one of 220 legitimate test emails sent to check the services for accuracy--the only service to do so.

Spam, or unsolicited bulk email, may be the most reviled by-product of the Internet, but controlling it has proven slippery. Lawmakers have so far struggled to create effective anti-spam legislation, and technology has offered only limited relief.

While Internet service providers have taken several steps to reduce the amount of spam reaching their customers--in some cases filing lawsuits against spammers--they have to be cautious, according to anti-spam advocates.

"ISPs have to be very careful not to accidentally trash some real email that is not spam," said Jason Catlett, president of Junkbusters, an anti-spam organization. "A customer could get screaming mad if they're losing email."

Several Internet email service providers have recently taken steps to keep their customers' in-boxes free of unwanted junk mail. In December, Yahoo installed Spamguard for its Yahoo mail service, and Microsoft in March announced an "in-box protector" feature for its Hotmail service. Those filters direct messages identified as spam to a separate junk mail in-box folder.

Since the beginning of January, AT&T has been using Brightmail, which monitors email activity and updates its filtering rules on the fly 24 hours a day, according to the company. The way it filtered email was most effective among the services tested, but it still let through 249 out of 908 spam messages.

According to Brightmail CEO Gary Hermansen, his company sponsored the study to see how well such measures are working.

"The test was conducted to determine the effectiveness of spam filtering," he said. "You've got this nuisance of a fairly significant amount of spam that's coming to your email box."

The mail services tested were those run by Brightmail client AT&T WorldNet, AOL, Concentric Network, Erol's Internet, Excite Mail, GTE Internet, ICQmail, Juno Online Services, Lycos, Mail.com, Microsoft's Hotmail, MindSpring, MAPS, Prodigy and Yahoo Mail.

Testers used default spam filter settings and disabled special spam filtering options available on services such as AOL and Hotmail that would have blocked virtually all of the test emails.

Despite the test results, Yahoo said it is confident that it has a reliable anti-spam filter.

"We feel very confident that the way the Yahoo Mail Spamguard performs under real-world spam attacks is very, very accurate in catching spam," said Lisa Pollock, senior producer for Yahoo. "We know that from qualitative and quantitative evidence, including feedback from our users that tell us that."

Many of the email services tested did not offer special spam filters. Excite, for one, said it is looking at options to improve its performance in anti-spam protections.

"We don't utilize a filtering service per se, but our mail system allows us to filter mail on a manual basis," said Brian Dally, senior manager of messaging products at Excite.

Dally said Excite has mail system administrators that write rules to block spam. When Excite notices a large attack in the mail system in which they see a lot of illicit mail or spam going to a lot of people all from one place, they can examine it and then write a rule to block it.

Although Excite has been using this manual process, Dally said the company has been looking at options to improve its performance in anti-spam protections.

According to a customer survey conducted last year by research firm Gartner, 74 percent of customers believe that their ISPs should be responsible for fixing spam problems.

The Gartner survey also said that 7 percent of 13,000 people had changed ISPs to avoid unsolicited commercial email messages, and another 36 percent said they would change to another ISP to reduce the amount of spam they receive.

"Living in an email account with lots of spam is like living in a community with trash in the streets," Gartner analyst Joyce Graff said in the study.