X

Search technology comes to the camera phone

Image recognition technology turns the camera phone into a text-free tool for promotions and Internet search.

Stefanie Olsen Staff writer, CNET News
Stefanie Olsen covers technology and science.
Stefanie Olsen
5 min read
What do MP3s have in common with dairy?

Milk marketing.

By early next year, cartons of milk sold by a European dairy manufacturer will have images of CD art printed on them. Accompanying each image will be a message urging people to take a picture of the art with a cell phone camera. Then, if the cell phone photographer sends the snapshot to a database operated by a marketing outfit, a free song will be sent to the phone from the band's sponsoring record label.

In the United States, people will also be able to use their cell phones to take a picture of a movie billboard, and then send the image to a special database that returns a film trailer, locates a theater showing the movie advertised on the billboard or allows the user to buy tickets to the movie.

It's all part of what several upstart companies have dubbed "mobile visual search"--technology promoted as an easier, more efficient way to get information on the go, without having to type on a tiny keypad.

It's possible, thanks to sophisticated object and facial recognition software that can match images with those scanned into an Internet-connected database. A match can trigger a range of possible results, including promotions, ring tones, pricing, maps and search results.

"One of the biggest problems of the wireless Web--it's not that people don't want it, it's that the Web is based on a keyboard and a mouse. We keep trying to simulate that with 12 keys on the phone," said Lauren Bigelow, chief marketing officer at Mobot, a mobile visual search company based in Lexington, Mass.

Facial and object recognition technology has been around for years, and has become a cornerstone of security applications used in airports and by the military. But now, companies see the opportunity to adapt the technology to improve the mobile marketing and search business.

Eventually, experts say, object recognition could be used to connect travelers with information on local restaurants, stores, museums or parks.

"If there's an economic reason to do it, it will happen," said Aaron Bobick, chairman of the interactive and intelligence computing division in the College of Computing at the Georgia Institute of Technology.

Of course, there are hurdles. Researchers say it will be another three years before 90 percent of mobile phones shipped in the United States will include a camera feature. Cell phone carriers need to adopt the technology. And consumers, of course, would have to learn how to use it.

Money phones
Neven Vision, a Santa Monica, Calif.-based company, has developed software that can be embedded in handsets or in a download from a Web site.

Neven's marketing application, called iScout, turns traditional forms of advertising such as magazines, billboards, newspapers, as well as any white space on a product, into a Web marketing tool. It has signed a deal with a dairy manufacturer in Europe, which executives won't yet name, to place iScout's "hyperlinks" on milk bottles for a year.

"All these surfaces in the physical world can now be hyperlinked, and they all turn into added information. A milk bottle gets a whole different value," said Hartmut Neven, chief executive officer of Neven Vision.

Neven's first campaign for iScout was for Coca-Cola in Europe. Advertisements began showing up in teen magazines in mid-October, asking readers to take a picture of a Coke can and send it to an SMS (Short Message Service) address to receive something back. Kids were sent a Java-based soccer game that they could play on the cell phone.

"It's a way to take something ubiquitous--the cell phone--for the teen girl and make the magazine interactive."
--Deb Burns, publisher, ElleGirl

Mobot, founded in 2003, also has a mobile visual search application. It licensed and developed software that can map the visual features of an image using its edges, shapes, textures and colors. In the United States, it has deals with companies such as the teen magazine ElleGirl and Warner Electric Atlantic, the music arm of the movie studio.

ElleGirl recently ran print advertisements for cosmetics company Loreal that offered readers samples, beauty tips and a sweepstakes entry for a trip to Paris if they took a photo and sent it in to its SMS short code.

"It's a way to take something ubiquitous--the cell phone--for the teen girl and make the magazine interactive," said Deb Burns, publisher of ElleGirl, who added that thousands of girls--between 5 percent and 10 percent of its readership--have participated in "Mobotized" ads in the magazine.

"Advertisers are always looking for something different and this is very accountable," she added.

Perfecting pictures
Neven Vision's technology is also being used in digital cameras and for security applications.

Docomo, Japan's largest mobile operator, and Vodafone, whose Japan unit also is a big player in that market, licensed Neven's software for their winter phone models, which will include facial recognition software designed to allow secure payment transactions. Right now, some Docomo phones with integrated cash-card functionality allow users to pay for items by passing their phone over a radio-frequency payment point. The Neven-embedded phones will store pictures of their owners that serve as network passwords.

Neven Vision is developing a similar application for mobile security. The company has built a "ruggedized" handheld device for police forces and military use. The device can store as many as 2 gigabytes of data, or 200,000 images, with biographical information such as name, license number and crime history.

Once a photo is taken with the device, its embedded software will launch a search for a match in its database. Because such information is regularly updated, the device can be synchronized to and brought current with a network database whenever it's being charged or connected to the computer.

The Los Angeles Police Department has been testing the device for the last year to aid in the capture of wanted gang members. It has made arrests with the device every day, according to Neven.

He expects the police department to eventually get cameras with resolution high enough to analyze the iris of a suspect's eye. "As image resolution increases, we can squeeze more and more information out of a facial picture, such as skin analysis and iris analysis," he said.