A question of numbers

 

By Paul Festa and Stefanie Olsen
Staff Writers, CNET News.com
April 30, 2001, 4:00 a.m. PT

Jerry Ziegler thought he'd hit the jackpot last August when his company, eFront Media, was listed as one of the 20 most-trafficked sites on the Web.

"There was euphoria at the office," the former eFront president recalled of the July ranking by research firm Media Metrix. "We were having trouble raising the institutional money, but once those numbers came out, we got a commitment for a very large amount of money. It was a serious deal, with cash and connections, and it fell right into our laps because of Media Metrix."

There was just one problem: The ranking was a mistake.

The Web site aggregator's leap to the big leagues was based on incorrect data. When Media Metrix learned of the error several weeks later, it revised the monthly rankings--the only time the Net research company has ever had to restate its top-50 list. After the revision, eFront didn't even make the top 500 for July, although in later months it managed to crack the top 50.

The eFront case raises concerns about the soundness of the influential system that has become the de facto standard for counting Web traffic--important figures followed closely by companies, advertisers, investors and industry analysts. Although its methodology has long been considered inexact, Media Metrix rose quickly to prominence in the heady days of the Internet economy when traffic was the equivalent of digital gold paving the way toward initial public offerings.

Among the questions to emerge from theeFront controversy: How was the incorrect information submitted? Why wasn't it discovered before the rankings were issued? Have other Media Metrix rankings been tainted in similar cases that have not come to light?

"The whole system is set up based on some trust that the people they are measuring are not just making things up out of whole cloth," said John Corcoran, an analyst at CIBC World Markets. Although traffic numbers provide "valuable information," he added, "this incident with eFront...is not helpful to Media Metrix."

Media Metrix is hardly the first media measurement company whose numbers have been challenged. The Nielsen ratings, which extrapolate TV viewership figures from a handful of families surveyed, have been criticized for years. (Nielsen, in conjunction with online research company NetRatings, also measures Web traffic in competition with Media Metrix, which merged last year with market researcher Jupiter Communications to become Jupiter Media Metrix.)

Despite their detractors, Nielsen and Media Metrix have become widely accepted in the Internet and TV industries, respectively, as no alternative systems have proven more accurate. Unlike the TV ratings, however, online rankings can have far more urgent consequences: Traffic can be the fine line between a company's life and death in the fragile Internet economy, with investors not far behind.

Revealing message logs
Still, Media Metrix's revision of eFront's numbers drew little public attention until last month, when hundreds of pages of ICQ instant messaging logs between eFront Chief Executive Sam Jain and his correspondents, including some fellow executives, were apparently pilfered from his PC and posted on the Web. The logs, which read like transcripts of telephone conversations, include explosive and candid discussions regarding business partners, employees and affiliated Web sites.

But among the most damaging allegations to arise after the publication of the logs is that eFront intentionally misled Media Metrix to get the company listed among the Net elite. Although the rankings episode is not detailed in the logs, it has been the subject of many message-board postings and has been acknowledged by Media Metrix, eFront executives and affiliates of the company.

"It is what it is; it's obviously not a positive thing," said Stacie Leone, a Media Metrix spokeswoman, who described the incident as an anomaly. She said the company uses a 20-person team to keep track of and verify "roll-ups," or lists of Web addresses assigned to particular properties, on a monthly basis.

eFront controversy raises concerns about Web traffic accounting NEW YORK, New York, October 2, 2000 - Media Metrix today announced that it has revised its July 2000 Top 50 Web and Digital Media properties based on a revision to the definition of eFront Media. The revision was made because eFront Media unintentionally submitted inaccurate information to Media Metrix. The primary change affected by this revision is that eFront Media now is reported as a Web domain and not a property. Rank Property Unique visitors (000) 1 AOL network (proprietary/www) 62,545 2 Microsoft sites 50,298 3 Yahoo sites 49,045 4 Lycos 32,119 5 Excite network 27,115

Media Metrix revised its Top 50 list of digital media and Web properties in the U.S. for July 2000. eFront, which initially came in at No. 18 because of some faulty information, did not even belong among the Top 500 Web sites.
Rank	Property	Unique visitors (000)
1	AOL network (proprietary/www)	62,545
2	Microsoft sites	50,298
3	Yahoo sites	49,045
4	Lycos	32,119
5	Excite network	27,115
6	Go network	21,848
7	About.com sites	18,282
8	AltaVista network	17,391
9	Time Warner Online	16,365
10	NBC Internet	15,434
11	Amazon.com	15,383
12	Real.com network	14,723
13	eUniverse network	13,958
14	Viacom Online	13,634
15	LookSmart	13,374
16	eBay	12,528
17	Go2Net network	12,527
18	Ask Jeeves	11,765
19	CNET Networks	10,728
20	ZDNet sites	9,936
Source: Media Metrix

"Because there are hundreds of those URLs coming at us...they need to provide us with documentation that they outright own or have a partnership with the company that has agreed to let them roll up that property into their traffic," Leone added.

As the eFront fiasco illustrates, however, this trust-but-verify approach is not foolproof.

In an October press release, Media Metrix said it revised the rankings after learning that eFront "unintentionally submitted inaccurate information." But Ziegler and two other eFront executives told CNET News.com that Jain was responsible for submitting data he knew to be inaccurate. Jain denied the charge and blamed Ziegler for filing the paperwork with Media Metrix.

Potential for abuse
Regardless of who was responsible for the submission, the episode underscores the potential for abuse of the counting system used by Media Metrix. Although the company maintains that it would have caught the discrepancy eventually on its own, it acknowledges that it discovered the problem after being alerted by a rival of eFront.

"In an ideal world, yes, these URLs are checked before the numbers are issued. But many of those sites are small mom-and-pop sites, and it's a matter of checking hundreds and hundreds of URLs, which would hold up the entire reporting process," Leone said. "In the case where they have so many partners, we might also do a basic spot check (of the URLs) and get most of the letters (from both parties verifying who should get credit for the traffic). I can't say for certain whether we did or not in the eFront case."

The research company defends its system and has not changed any procedures following the eFront controversy. "Media Metrix felt we had sufficient evidence to aggregate this site and report these numbers. Some of the information turned out not to be right," Leone said.

She added that Media Metrix executives consider the incident "a blip on their radar."

When it changed eFront's ranking, Media Metrix reclassified the Web site's designation from a "property"--a branded entity that encompasses other sites, such as the way the Microsoft Network includes Expedia and CarPoint--to a "domain." As a domain, eFront was no longer able to count traffic of affiliated sites, relying instead only on traffic generated at its primary site.

Yet the revision largely escaped major news media. Even after the eFront ICQ scandal, a Los Angeles Times story described the company as being "among the 20 most-frequently visited Web sites, garnering about 12.4 million viewers in July, according to Media Metrix." There is no mention of the restated rankings in the March 17 story.

The intoxicating power of such high traffic is precisely what drew Jennifer Moss, Webmaster for BabyNames.com, to eFront.

The company's concept was to sell advertising across a network of hundreds of small Web sites, many it had bought in exchange for eFront equity and the promise of three years' monthly payments. But Moss and other Webmasters say they received at most a handful of the payments and are now seeking to regain control of their sites. Ziegler concurred that eFront had trouble making payments to its affiliates.

Throughout 2000, eFront was rapidly acquiring an eclectic mix of Web sites with the goal of taking the network public.

The efforts appeared to pay off with the July top-50 list, which caught the attention of Salon.com. Last August, the online magazine wrote that the buy-and-hold strategy drove eFront "to No. 18 on the Media Metrix list, with 12.5 million visitors. The 14-month-old company is a huge roll-up of some 150 small content sites, ranging from gaming to tech news and loopy entertainment."

eFront trumpeted the BabyNames alliance in a press release dated Oct. 3--one day after Media Metrix issued its revised top-50 list and bumped eFront off it. Rather than having 12 million unique visitors, eFront had fewer than 400,000 "uniques," according to the revision.

That was particularly bitter news to Moss, who said she had agreed to sell her Web site to eFront and become an affiliate partly because of its apparent success in cracking the top 20.

"The month after we signed with them, they were suddenly (removed from) the charts," lamented Moss, who also is owner of Los Angeles Web consultancy Moss Mallory Interactive. "That bothered us because that particular ranking was one of the major reasons that made our decision to go into this deal."

Former eFront Chief Technology Officer Matt Levine, who resigned after the ICQ logs became public, called the Media Metrix episode the beginning of the end for several of eFront's executives. "Media Metrix was the straw that started putting pressure on the camel's back," Levine said. "That really started to put strains on relationships."

That has become abundantly clear in the dispute between Jain and former company President Ziegler, who has filed a lawsuit in Superior Court in Orange County, Calif., against eFront and Jain seeking back pay.

"Sam (Jain) would call companies all over the world, trying to get them to be our affiliates," Ziegler said. "Some of them wouldn't respond. He said, 'If they won't respond to me, they won't respond to anyone.' Sam took these and submitted them to Media Metrix and said they were affiliates."

Jain called the allegations "absolutely not true," saying in an interview that "Jerry (Ziegler) himself was in charge of our affiliates program. I had nothing to do with it."

Moss didn't particularly care how Media Metrix got the bad information or why it wasn't discovered sooner. For her, the damage had already been done.

Like other Webmasters who did business with eFront, Moss says she did not receive the promised monthly payments, and for months she struggled to regain control of the BabyNames Web site. She finally prevailed this month after arguing that the revised Media Metrix numbers provided a legitimate reason to break the affiliation because traffic was a major motive for joining eFront.

"We can dispute whether the original contract was brought about in good faith," Moss said in a March interview with CNET News.com. "We believe it wasn't."

CIBC analyst Corcoran said eFront was hardly the only Internet company apparently obsessed with traffic numbers. It was just a "more visible example that caused a little more embarrassment," he said.

To understand companies such as eFront, Corcoran said, it is important to view them in the broader context of the Internet gold rush--and the lengths to which some people may go to strike it rich.

"There was a big temptation to pump up the numbers two years ago," Corcoran said. "These companies had business models where they knew they weren't going to show any profit for some time, so companies were judged by an unusual set of criteria: page views and eyeballs."