AltaVista serving up out-of-date listings

AltaVista hasn't fully updated its database of millions of Web pages since July, according to company spokeswoman Kristy Kaspar. Most search engines refresh their databases every month. The Palo Alto, Calif.-based company "has fallen behind schedule," she said.

The company's housekeeping troubles come as its popularity has plummeted. In September 2000, Jupiter Media Metrix reported that AltaVista had about 13.9 million unique visitors for the month. A year later, the company drew only 6.8 million visitors for the same period. In contrast, Google shot from 5.7 million visitors in September 2000 to 18 million visitors last month.

"AltaVista has lost a huge amount of its audience," said Danny Sullivan, editor of industry newsletter Search Engine Watch. "How Google has shot up the charts is in direct proportion to how AltaVista has declined.

"AltaVista has gone from being one of the top search engines...to looking like they're struggling to be a third-tier player. This is just another blow that the service didn't need."

AltaVista, once known as a premier search provider among Internet cognoscenti, has seen its fortunes ebb along with those of its once red-hot owner, CMGI. The search company has undergone a number of strategy makeovers, including a failed attempt to challenge Yahoo and AOL Time Warner in the portal business.

It followed those forays with a retrenchment billed as a return to its roots. In February, the company touted enhancements to its site as "further demonstration of the company's commitment to search leadership," according to a press release. At the time, the company redesigned its site for faster load times and "reformulated its relevancy algorithms and increased the size of its Web page index for more relevant, accurate results."

While it might seem that technology is the cause for the delay, Kaspar said that's not the case. "It's not something that's broken. It's just a big challenge to get as many pages as you can," she said.

"We are unfortunately just behind schedule. We know that all of this is imperative. Within the next weeks we will be back to the most updated index." She added that the company has "crawled" the Web pages across the Internet but has not updated the index yet.

Paid vs. free listings
AltaVista catalogs between 500 million and 600 million Web pages and typically refreshes them within 45 to 60 days. Other search services, including Fast Search, update their engines every 30 days. Google typically updates every 28 days.

AltaVista has more than 40 customers that license its search index and more than 1,000 portals powered through those partners, including MyWay, InfoSpace/Go2Net and CNET Networks, publisher of News.com.

Some Web pages found on AltaVista search are getting refreshed. But those are mainly from AltaVista's paid-inclusion programs, which were launched this summer as a way for the company to diversify revenues during an online-advertising drought.

AltaVista sells expedited placement in its search engine to companies including eBay and Amazon that list more than 500 Web addresses at once. It also sells "express inclusion" to small and midsized companies submitting fewer domain names. Both programs, which are working, update Web addresses on a weekly basis. Its free URL submission service is updated every four to six weeks.

Defending the company's commercial inclusions, Kaspar said they are much easier to add to the index than a broader update. The company must go through many steps to refresh its database of Web pages, including checking pages for duplications and running the data through relevancy algorithms, she said. Meanwhile, commercial listings can be expedited because companies submit their addresses through a direct XML feed into the index. In addition, express inclusions are handled by an AltaVista partner, InfoSpider.

Sullivan called the company's inability to update search results "inexcusable" and said it feeds suspicions that paid listings take priority over generic search results.

"This is something people have been paranoid about," he said. "If you're going to start charging people to submit, does that mean Web sites that can't afford to pay will get overlooked?"