HolidayBuyer's Guide

Spying an intelligent search engine

Innovation in Web search using artificial intelligence may lead to the day when you could expect the Web to do the tedious tasks for you.

While most would agree that Google has set the current standard for Web search, some technologists say even better tools are on the horizon thanks to advances in artificial intelligence.

Search is like oxygen for many people now, and considering Google's breakthroughs in Web document analysis, supercomputing and Internet advertising, it can be easy to think this is as good as it gets. But some entrepreneurs in artificial intelligence (AI) say that Google is not the end of history. Rather, its techniques are a baseline of where we're headed next.

Proponents of AI techniques say that one day people will be able to search for the plot of a novel, or list all the politicians who said something negative about the environment in the last five years, or find out where to buy an umbrella just spotted on the street. Techniques in AI such as natural language, object recognition and statistical machine learning will begin to stoke the imagination of Web searchers once again.

"This is the beginning for the Web being at work for you in a smart way, and taking on the tedious tasks for you," said Alain Rappaport, CEO and founder of Medstory, a search engine for medical information that went into public beta in July.

"The Web and the amount of information is growing at such a pace that it's an imperative to build an intelligent system that leverages knowledge and exploits it efficiently for people," he added.

Medstory is not alone. Other young companies such as stealth start-up Powerset and image specialist Riya are also looking to turn arcane computing techniques into business success stories.

Unlearning "keywordese"
In the eyes of a search engine, the Web is essentially a body of words on billions of pages, along with the hyperlinks that connect the words. One of Google's big breakthroughs was to link those words efficiently, measuring relevance by the appearance of words on a page, and the number of hyperlinks pointing to that page, or its popularity.

As a rule, search engines don't understand the words--they're merely programmed to match keywords that are more significant on a page, closer together or linked more often from other pages. So in essence, when someone types in "woolly mammoth," he or she sends the search engine on a wild goose chase for those words, not the animal.

As a result, search engines miss the nuance of the human language. For example, Google might render a simple search for "books by children" by scouting for the pages that include the words "books" and "children," but it would eliminate the so-called stop word--in this case, "by"--because stop words are those that occur on almost every page. Yet those stop words occur so often because they are important to the meaning of a phrase. "Books by children" is different from "books about children," and different still from "children's books."

Barney Pell Barney Pell

Barney Pell, founder of a yet-to-be-launched AI search engine, calls the restrictive language of search engines "keywordese."

"Search engines try to train us to become good keyword searchers. We dumb down our intelligence so it will be natural for the computer," said Pell, whose company, Powerset, is based in Palo Alto, Calif.

"The big shift that will happen in society is that instead of moving human expressions and interactions into what's easy for the computer, we'll move computers' abilities to handle expressions that are natural for the human," he said.

Powerset, which hasn't divulged its launch date yet, is using AI to train computers not just to read words on the page, but make connections between those words and make inferences in the language. That way a search engine could think through and redefine relevance beyond the most popular page or the site with the most occurrences of keywords entered in a search box.

"The next step will be to be able to recognize and find everything in the world that isn't words on a Web page, like recognizing concepts or descriptions of events, like a plot of a story where boy meets girl. If you put in words for that now you end up with a lot of other stuff," said Esther Dyson, a blogger of Release 0.9, which is published by ZDNet and owned by CNET Networks. CNET Networks is publisher of, and Dyson has personally invested in Powerset.

Why is the time right to experiment with AI and search? One of the biggest hurdles to building AI into a search engine is that it can be impractical on a large scale. The computational power needed to calculate results efficiently can be enormously expensive, critics say. But the effects of Moore's law are pushing down prices for computers, CPUs and bandwidth, and the opportunities are ripe. Search is also a lucrative business. Google, after all, is worth $6 billion annually, thanks to targeted advertisements linked to its results.

Most of the search community believes advances in Web search from Google and others will now take place incrementally, by squeezing a bit more from Google's Pagerank, or by tuning relevance, or indexing hard to find files. But for the next leap to happen, executives like Pell say, a new architecture must be built.

Taking the pulse of search
Medstory, for example, is applying artificial intelligence techniques to one area of knowledge, health care, which is a fast-changing industry rife with inefficiencies in how people get up-to-date information. Tackling a specific body of knowledge is more economical and efficient, Rappaport said. But the tricks that Medstory is using to extract more knowledge from medicine for consumers and professionals could be applied to other industries, like finance or entertainment.

Alain Rappaport Alain Rappaport

"Our job is to get rid of the noise, so the amount of information is smaller than the Web," he said. "On the other hand, we have to compute things, not just ranking and linking, and that's computationally intense."

Rappaport said one of the more recent progressions in AI has been in moving from relying on humans to catalog connections between various data to programming computers to do the work, or what he calls the automation of knowledge structure. Tom Mitchell, chair of machine learning at Carnegie Mellon University, calls it machine learning for statistical language processing, or learning algorithms that allow computers to read text.

Rappaport won't disclose the secret sauce of the company's technology; however, he said, it's a 24/7 process in computing that connects valuable pieces of information together, such as linking one document that explains symptoms of a disease to another document with analysis of a therapeutic drug for that disease.

"It's continuously extracting information it can then build itself with," Rappaport said.

The techniques surface in Medstory as serendipitous bits of knowledge that would otherwise take a Web searcher hours or days to acquire.

For example, a search on the term "lower cholesterol" returns a detailed set of results that dives into topics on a subject, such as drugs, symptoms and nutrition information. Within each subtopic, there are lists the most relevant drugs, symptoms or nutritional supplements to lower cholesterol. Under "nutrition," for example, the most relevant link is olive oil. Point the cursor over olive oil and a page pops up with the health benefits of olive oil for high-cholesterol sufferers. Unless someone already knew that, the information could take a long time to find in a search engine like Google.

Medstory was founded in 2000 as software licensed to biotechnology firms and other health care companies, but the service recently launched to the public, and Rappaport said he plans to roll it out more widely soon.

Mitchell of Carnegie Mellon has predicted that computers will be able to read the Web by 2015--he's so sure he's bet a lobster dinner on it, and a few people have taken him on the gamble.

How these services present AI search results is also very important. Google revolutionized search for the masses partly because of its simple interface, with no ads, no clutter, just a search box. For Powerset and Medstory to capture new searchers, they must help raise expectations with what turns up in the results. Technologies like speech recognition will fuel advances.

An upstart called Riya is part of the trend, too, because its technology is about recognizing more than just nouns, or keywords. Its photo software attempts to string together a sentence with a verb, such as "find a girl who looks like this girl for me on"

A match for the ratty couch
Imagine uploading a picture to the Web of your favorite ratty couch, and then asking a search engine to find another one like it. The tool wouldn't just produce a similar couch but it might even point to a store where you could buy it.

Right now, most image search engines rely on keywords, or descriptive text that is linked to a photo in order to retrieve a list of results that match a Web surfer's keyword query. That method can be unreliable, however, if photos or images lack sufficient descriptions.

Munjal Shah Munjal Shah

Riya, in contrast, looks inside the photo to extract information about its qualities using AI. The San Mateo, Calif.-based company, which launched a beta site in March, uses algorithms to calculate the photo's shape densities, patterns and textures--among many characteristics--and extract them into a mathematical representation of the photo, or what Riya's CEO Munjah Shah calls a visual signature. Each photo is represented by 6,000 numbers, Shah said, and the company uses AI to match one visual signature to another.

"All the AI is the technology to extract that information and compare two visual digital signatures," he said, without divulging the secret sauce to the technology. "It's never exact, but it's more like how do you create a fuzzy effect. That's where we've built up a lot of technology to be able to do better than most people."

The field of AI called computer vision, which encompasses facial detection and recognition, is coming of age for several reasons. One is that computing and CPUs are much cheaper and powerful now, Shah said. The dual-core computers that have come out with 8 processors have one-eighth the running cost yet are eight times more powerful, for example.

On economics and accuracy
For software to merely detect a face in a photo--not to recognize who it is--took enormous expense and time just a few years ago. Riya takes between one second to detect a face within a small photo, about a quarter of a megabyte, and 20 seconds for a photo file as large as 7 megabytes.

"To do this even a few years ago, with a billion photos on the Web just looking for a face, you would have to spend a fortune on computing, an obscene amount of money. Detection takes a lot of computing power and it's just gotten cheap enough," he said.

"The economics of it work but you still have to worry about accuracy," he added.

Recognizing an object or person in a photo is a different problem, but it's faster than detection once the computers calculate a visual signature, Shah said. Riya, for example, can compare one face to 250,000 photos in one second to find relevant matches.

Riya will be able to do many tricks, apart from couch-shopping. A Web surfer, Shah said, will be able to point to a photo of a girl on and ask the search engine to find other girls that look like her on a dating site like Similarly, someone could upload a favorite dress pattern and ask the engine to find similar patterns. Riya already offers a service called Myphotos, which lets people upload photos, train the program (e.g. "this is a picture of me and my mom") and ask it to find other photos like it in a set.

Riya, founded in August 2004, has 50 employees, including 14 specialists in computer vision. The company started out strictly in facial recognition, but realized it had a larger opportunity in object recognition. It recently raised $15 million in venture capital, bringing its total funding to $19 million. It plans to make money by selling advertising and collecting fees from merchants that benefit from Riya's visual search engine. For example, it will collect a bounty if someone ends up buying that new couch on eBay.