Inktomi braces for search competition

With the release of its fourth product line, Inktomi is both breaking new ground and keeping a nervous eye on upstart competitors challenging its traditional applications.

Paul Festa Staff Writer, CNET News.com
Paul Festa
covers browser development and Web standards.
Paul Festa
3 min read
With the release of its fourth product line, Inktomi is both breaking new ground and keeping a nervous eye on upstart competitors challenging its traditional applications.

Inktomi is well aware of its position as the 800-pound gorilla and No. 1 target to beat in most of its product areas, which include cache, search, a shopping engine, and now semiautomated directory-generation technology, announced today.

In discussing the company's last earnings report, executives warned analysts that a new wave of competitors in these comparatively new market spaces will present a fresh challenge.

For its caching business, Inktomi recently moved to partner with a partial competitor, Sandpiper Networks.

Under the partnership, Inktomi's Traffic Server caching product and Sandpiper's Footprint distributed network for handling Web traffic will support each other. Inktomi also announced a "substantial" investment in Sandpiper.

On the portal services side of its business, Inktomi has shored up its defense against the competition by hunkering down on basic search technology.

As Web surfers are well aware, search engines can leave much to be desired in honing or prioritizing results of tens of thousands of links.

The demand for improved search technology has been demonstrated by a new wave of start-ups moving in on the territory of refined searches. These challengers include Direct Hit and Google.com, which recently enjoyed a spectacular first round of venture financing.

Another more recent entry into the area of refined search results is Yep.com, produced by WebsideStory and based on information gleaned from their HitBox tracker tool.

Search technology start-ups are employing a variety of methods to refine search results. Some analyze the link structure of the Web to prioritize sites that are linked to more frequently. Others are moving up sites that previous searchers have chosen and stayed at longer.

And at least one of them, Direct Hit, has succeeded in taking some business away from Inktomi.

Search engine and longtime Inktomi customer HotBot, acquired by Lycos, recently made Direct Hit its default search engine, partially displacing Inktomi. Inktomi still powers HotBot searches for more arcane or complicated queries.

With the Directory Engine announced today, Inktomi also revealed that it has been implementing search-honing technologies, developed in conjunction with the Directory Engine, and announced the deployment of what it calls "concept induction."

"This is the cornerstone of how we do the population of the Directory Engine categories," Inktomi chief technology officer Paul Gauthier said.

"We crawl the entire Web, hundreds of millions of documents, and analyze all the words on all those pages: how they co-occur with each other, how words used in one context mean something different than they do in another context," he added. "We also factor in all the links on the Web. That gives us lot of information if two documents are linked together."

While Inktomi has not been first in implementing and promoting these refining technologies, Gauthier said the company will benefit by having a more comprehensive set of them.

"There is a number of these start-up companies, and typically what they have is one specialized technology that is their Holy Grail," Gauthier said.

"But they're only pieces of a solution. What we've been building is a framework that encompasses and rationalizes a number of them," he said. "We're harnessing all these different techniques, and the impact on the search engine has been dramatic. There's been a tremendous improvement in relevance."

Techniques Inktomi already has implemented include contextual analysis, link analysis, analysis of the words linked in the context of the connected pages, and other unspecified methods.