X

Forget rainbow vomit. Google Lens is AR you can actually use

Google's new technology lets you search with your phone's camera. It's augmented reality beyond photo filters.

Richard Nieva Former senior reporter
Richard Nieva was a senior reporter for CNET News, focusing on Google and Yahoo. He previously worked for PandoDaily and Fortune Magazine, and his writing has appeared in The New York Times, on CNNMoney.com and on CJR.org.
Richard Nieva
4 min read
google-io-2017-1995.jpg
Enlarge Image
google-io-2017-1995.jpg

Google Lens can tell you what type of flower you're looking at. Just point your camera at it.

James Martin/CNET

Snapchat made a big mark on augmented reality by putting digital flower crowns on real people. Now Google may make its own mark by showing you digital information about real flowers.

On Wednesday, the search giant announced a big push into augmented reality, which overlays digital images on what you'd normally see through a camera.

The new technology, announced at the company's I/O developer conference, is called Google Lens. It's a way to use your phone's camera to search for information. For example, point your camera at that flower and Google will tell you what kind it is. Point it at a book, and you get information on the author and see reviews. Ditto with restaurants: You'll be able to see reviews and pricing information on a little digital card that appears above the building on your phone's screen.

With Lens, Google can "understand what you're looking at and help you take action," CEO Sundar Pichai said on stage Wednesday. "We can give you the right information in a meaningful way."

Google is first launching Lens in the Assistant, its digital helper software, and in Google Photos. It will be available at an unspecified date "later this year."

Google Lens marks a big, ambitious attempt by a mainstream company to get into augmented reality in a way we haven't much seen yet. Snapchat, Facebook and Instagram (owned by Facebook) use AR for now to make you laugh and smile with filters like rainbow vomit or Iron Man masks. That stuff is important, but Google is taking a different approach when it comes to AR: utility.

"There are these other aspects of AR that involve creative expression," Aravind Krishnaswamy, head of engineering at Google Photos, said in an interview. "That's not what we're trying to do."

Others are trying to get the AR engine revving. Pinterest has a similar feature also called Lens, but it's mostly focused on buying. Microsoft has long been pushing AR through its HoloLens headset, but there's no consumer product in sight. Because Google is employing its technology in widely used Android phones, it has the scale to reach a massive audience once it launches.

For Google, it's important to expand search beyond its iconic homepage. The world is a lot different than when Larry Page and Sergey Brin started the company as graduate students at Stanford. Search was a thing for desktop computers and typing on a keyboard. Now -- who knows? It's for tapping on a smartphone screen, barking orders at a device in your kitchen or, if Mark Zuckerberg gets his way, eventually sending your thoughts straight to a computer.

Google knows the way of the world is changing, and it's trying to get out in front.

'Get stuff done'

Right now, AR is having a moment. Last September, when Snapchat renamed itself Snap and started calling itself a "camera company," people were baffled. But the young upstart realized the time was right for phone cameras to take advantage of augmented reality. Facebook, sensing that Snapchat was onto something, decided to build the future of its app around smartphone cameras, too. Last month, Zuckerberg announced a new augmented reality platform, which invites outside developers to create AR experiences for Facebook.

The promise of augmented reality is profound. With a good pair of AR glasses, you might be able to tell if your kid has a fever just by looking at him or her. You might never have to buy a TV again if you could just project a big screen over your eyes. And you'll never experience the awkwardness of forgetting someone's name, because it will appear right over their head.

Of course, we're not there yet. But Google thinks it can lead the way because of everything it knows about you and the world. That's where being a search company comes in handy. Google is taking its big trove of data and trying to make it three-dimensional.

"It's our core mission, accessing the world's information," Gummi Hafsteinsson, product lead for the Assistant, said in an interview, citing Google's mission statement. "Images are just another part of that."

What Google has going for it is scale, said Chirag Dekate, a research director at Gartner. Google has a good shot at being accurate with the information it presents in AR because of Google's knowledge about your surroundings. Companies like that have a "distinct data advantage," he said.

To be sure, Facebook and Snapchat have AR plans beyond photo filters. Facebook is working on AR glasses, but that will take years. And Snapchat has its Spectacles glasses, which will likely be a vehicle for more advanced AR. But for now, they are mostly just for taking snaps.

Google's goal is to be helpful. David Lieb, product lead for Google Photos, in an interview summed up the approach: "We're doing this to help you get stuff done."

CNET Magazine: Check out a sampling of the stories you'll find in CNET's newsstand edition, right here.

Life, disrupted: In Europe, millions of refugees are still searching for a safe place to settle. Tech should be part of the solution. But is it? CNET investigates.