Face recognition comes to photo albums

The company, founded by a group of facial recognition Ph.D.s from Stanford University, has created a Web site that will search through digital photo albums through "contextual recognition" to find matches, co-founder Munjal Shah said during a presentation at PC Forum, a three-day conference taking place here. Give the Web site a couple of pictures of your mother-in-law as a sample, and it will find the other pictures of her on your hard drive.

Contextual recognition is an amplified version of facial recognition, according to Shah. The software looks at a person's face, but will also look at the shirt they are wearing and other cues to find a match. Additionally, it will search for text in the images, so if people want to find a photo where they posed by a "Welcome to Florida" sign, they can search on the word "Florida" and the Riya search engine will find it.

Other companies, such as 3VR Security, are trying to bring facial recognition and image search to security cameras.

Riya's application can use cues other than facial structure because consumer photo albums are far more predictable than the data gleaned from photo recognition systems that law enforcement agencies use, the company said. Typically, individuals have photos of only a limited number of people, but they tend to have several pictures of each of these people. By contrast, FBI files contain thousands of photos, and only a few pictures of each suspect.

Thus, the task of matching a sample in a consumer's hard drive is easier. People also tend to wear clothes on one occasion and again on another, making it possible to search according to clothes.

"Traditional facial recognition can't do this because Osama bin Laden didn't walk by the security camera 10 times," Shah said.

The company has tweaked the software to search for matches under a variety of conditions. Two-dimensional photographs are mapped onto virtual 3D models so that pictures where someone's face is turned slightly away from the camera at an angle will pop up on a search where the samples fed into the computer are straight-on head shots. It will also compensate for people who tilt their head in a given photograph.

"You see that a lot, where people put their arm around a buddy and go like this (tilts head)," Shah said. "If we can see both eyes, we can get it."

Riya is roughly 70 percent accurate. If a hard drive contains 100 photos of a person, it will pull up 70 of them, miss 30 and pull up seven or so mistakes, Shah said. It works better on indoor shots than outdoor ones and can compensate for differing hairdos, lighting conditions and other environmental factors. You can also search for two faces at once, so that only the photos with both you and your wife show up, for instance. (It does not, however, work on pets, Shah admitted.)

Tests conducted by Shah (with a wide variety of his own family photos) on the floor at PC Forum generally met that standard. Searching on his own picture pulled up a number of photos, as well as a few shots of his brother.

The faces being searched can constitute a relatively small portion of the picture. The software can identify and pick out features that measure 100 pixels tall or larger.

The software has been tested by a few people in beta, and a larger public beta begins next week. To use the product, consumers have to download what the company calls an uploader.

The uploader tags photos for searches and then sends copies to Riya's Web site. The uploader also reduces the resolution of the photos sent to the Web site to 800 by 600 pixels, far lower than the typical digital photos taken on 5-megapixel cameras. The lower resolution, however, means that a consumer can upload several photos rapidly.

When the search is conducted, the users see the low-resolution photos. However, the company plans to improve the technology so that the search results will sync up to the higher-resolution photos on the hard drive. That way the user will be able to view the high-resolution photos after a search.

Riya also plans to license the technology to Web sites. Dating sites are one possibility, Shah said. Slightly off-kilter folks who go to sites equipped with Riya could download pictures of their old flames and search for someone who looks similar.

Investors in the company include Nokia Ventures and Leapfrog Ventures.