Infoseek has built a new spider technology for the new engine called Infoseek Ultra, which detects new and changed pages and update the index in real time based on how often pages change. This new technology will offer users the most current sites and information, according to Infoseek.
In building the index, Infoseek found more than 80 million URLs, but after filtering out duplicate pages and dead links only 15 million were found to be unique.
"The practice of using huge numbers of URLs to describe comprehensiveness has become commonplace," said Steve Kirsch, founder and chairman of Infoseek. "As we built the index for Infoseek Ultra, we realized that what really counts is unique content. We discovered that more than 50 percent of the URLs on the Internet are dead or duplicate links and add no value to the user. Our approach is to filter out all the garbage so the consumer doesn't have to."
Infoseek Ultra performs more than 1,000 queries per second and allows users to use multiword queries without a decrease in performance. The search engine runs on a single Sun Microsystem Ultra Enterprise 4000.
A beta version of the search engine is currently available at Infoseek's Web site. The service will go online later this year.