Google's Dropping Location-Based AR Experiences Into Maps

Google is working on a mixed reality headset with Samsung and Qualcomm, but no further details of that collaboration emerged at Google's 2024 I/O developers conference. Instead, Google focused on AI. A few little AR-related bits of news did pop up, though: in addition to announcing Project Astra, an camera-enabled AI companion that could live in future glasses,  Google's also serving up more AR location-based experiences into Google Maps.

Maps has been Google's place where 3D and AR have been explored for a while now: head-up turn-by-turn directions, immersive 3D city views. Last year Google announced a framework to put 3D Maps data in location-based AR experiences from other app developers.

Apparently, finding those AR apps hasn't been easy. Google's new efforts will make certain AR experiences discoverable in Google Maps when looking around particular landmarks or cities. If you're in that place, holding up the Maps app will help make those experiences appear on your phone screen as you hold your phone up over a location – in the future, maybe that will be on glasses, too. If you're not nearby, the experiences will appear in Street View mode to virtually see the AR from a distance, in a sense.

The pilot program is starting in Singapore and Paris, where Google is incorporating some local arts AR experiences into Maps via the Singapore Tourism Board and a Google Arts and Culture project that will show pavilions from the 1900 Universal Exhibition overlaid onto actual Paris. 

We're still nowhere near a world where anyone is wearing everyday AR glasses yet, but Google's efforts suggest that the Maps app may be the best future doorway to exploring the world in AR when those glasses do arrive. Game developer Niantic, which has made AR games like Pokemon Go and Peridot for years, has a similar maps-centric vision. So does Snap.

Google's focus this year has been almost entirely on AI, but maybe the two will dovetail. Google's vision for glasses-based camera-enabled agents, Project Astra, is designed to be aware of environments, reacting accordingly. Would AI agents be what helps serve up hidden AR experiences in the future, too? Knowing how to manage overlaid information in the real world in ways that don't feel either spammy or invisible is a big challenge that no one's solved yet. Maybe our pocket map apps, which I already use for searching for pretty much everything I need when I'm wandering around, could be a central step in figuring that out.