Google Lens is turning into what Google Glass never was
Commentary: Trust me, this is a good thing.
- Nearly 20 years writing about tech, and over a decade reviewing wearable tech, VR, and AR products and apps
Google Glass
Years ago, Google Glass was pitched as a magic eye on the real world. I wore one. It never did what people really thought it could do.
The crazy thing is, Google is getting to that magic world fast... through phones . The latest updates to Google Lens -- demoed at this week's Google I/O developers conference, and coming soon to end users -- are closer than ever to what we thought magical real-time augmented reality would be. And, maybe, this is the stepping stone to where future AR headsets are destined to be.
Recognizing text and translating or copying it. Recognizing objects and searching for related matches. Seeing posters and popping out videos and related news. Getting directions and information that floats in front of my vision.
At the Google I/O developer conference where bigger-scale VR news has thus far been minimal, Google's continued evolution of Google Lens stole the show. Google Lens is Google's camera-based AR technology that expands the powers in Android phone's existing camera app. It's not a separate app, it's baked into Android. Lens debuted last year, but is evolving a lot faster in 2018 and is spreading to more Android phones. When I saw the on-stage demo of how Google Maps will use Android cameras to recognize the world and overlay head-up directions (and animated foxes, too), it made me think of what I imagined Google Glass might someday do. Or, what all AR headsets might do.
There aren't many good AR headsets right now, and that's mainly because they still don't do much. Google Glass-alike smartglasses still exist, but they mainly deliver heads-up information from a phone. They don't use cameras to intelligently analyze the world on the fly... yet. Devices like the Magic Leap One promise a world mixing real and virtual things, but you can't buy one yet. The smartest AR still requires smarter cameras to recognize the world and interact with it.
And maybe, that's where smart AR needs to remain for now. At least, according to Google. "On the experience side, we're trying to meet users where they are," says Aparna Chennapragada, Google's VP of Product for AR and VR. "It's how we introduce people to these changes in technology. In Lens, we absolutely we see this as one of the first steps that AR can make a difference. If you think about looking at the world, and how do you enhance what you see. And in this case we happen to do it with information." Chennapragada emphasizes AR on the practical side, describing Lens updates as focusing on what Google knows people do with their phone cameras, "how we can accelerate that user behavior for something they're doing already."
Google Lens with Smart Select
I tried Google Lens and its newest features, in a series of demos at Google's AR/VR pavilion at Google I/O. Translating text on a menu from Spanish to English. Copying text from a book jacket on a Zadie Smith novel. Looking at a bag, and searching for similar ones. Holding the phone to a photo of a dog and knowing what type of dog it is. Some of these ideas are ones I've tried before using tools like Google Translate. Others feel new by being integrated.
Lens is not quite as effortless as I'd ideally like, but it's even closer to being a magic eye on the world, a quick way to scan and search. And it doesn't just remind me of Google Glass: it reminds me of Google Tango.
Tango was Google's original magic camera: the AR project where Google had started to explore world-scanning camera tech. Tango's indoor mapping and way of scanning the world has made it bit by bit into ARCore, which is Google's augmented reality toolkit. The way its little sprinkle of fairy-dust precedes any sort of AR moment, as if its augmented fingers are already exploring.
It's also practical. I've realized, thanks to my family and my kid, that lots of people know what VR is, but few know AR. But they've tried it: Pokemon Go, or Snapchat Spectacles , or that Ikea app that makes furniture appear. Or Google Lens. Maybe it's apps, not "AR," that are all people care about.
Google Lens is pushing that territory further. This year, Lens feels like even more of a push toward the future of computer vision, where apps "know" what the camera is "seeing." It's what's needed before anyone makes a killer pair of smartglasses. And, on glasses, battery life can't last long enough to do the always-scanning stuff a phone can. In that sense, maybe this type of real-world-scanning AR is doing what glasses can't. Yet. "If you think about voice, that's the evolution of how these changes happen," says Chennapragada. "How you change the face of computing itself. We actually had voice actions in 2013, 2014 or so, and that was on the phone. You had a search bar, a microphone there, and you could do a handful of things. For us, I think in some sense there's an intelligence layer that needs to evolve first. There's a device, and there's intelligence. And you want to apply the intelligence layer on the device that everyone uses, which is a phone, and then really sharpen that."
Maybe a headset comes next, or maybe not yet. "And at some point, if it makes sense, a different form factor," Chennapragada says about an AR return to wearables. "But it's more about solving those use cases, and building that intelligence: in this case, a visual intelligence. That's one way to think about it."