Pokemon Go creator will sell its AR tech to spawn games like Harry Potter

Niantic, which is working on a new game called Harry Potter: Wizards Unite, showed off tech that maps what we see, and can be used way beyond games too.

Ian Sherr Contributor and Former Editor at Large / News
Ian Sherr (he/him/his) grew up in the San Francisco Bay Area, so he's always had a connection to the tech world. As an editor at large at CNET, he wrote about Apple, Microsoft, VR, video games and internet troubles. Aside from writing, he tinkers with tech at home, is a longtime fencer -- the kind with swords -- and began woodworking during the pandemic.
Ian Sherr
5 min read
Niantic/The Pokemon Company

I'm holding a phone about FaceTime length away from my face as I'm running around a room. If a certain company has its way, everyone will be doing this a lot more soon.

I'm playing a game created by Niantic, the startup that was the force behind the 2016 smash hit mobile game Pokemon Go . You know, the game that was the fastest in App Store history to hit $1 billion in revenue and still ranks among the most played in the world.

Pokemon Go's popularity was rooted in how simple it is: Grab a phone, point it at the real world and on the screen you see the sidewalk in front of you. Then, suddenly, a Pokemon -- or "pocket monster" -- pops out and bounces around. Your job is to capture it by effectively hitting it with a Pokeball. The Pokemon tagline: "Gotta catch 'em all."

The game's international sensation made Niantic a leader in a nascent field called augmented reality, in which computer images (in this case, Pokemon) are overlaid on the real world. But the technology is very much in its infancy. The Pokemon weren't convincingly in the real world since they just kinda hopped around the screen.

That's why Niantic, which is named after a whaling and gold rush-era ship that's buried beneath San Francisco's downtown, invited reporters to its spacious offices on the second floor of the city's historic Ferry Building to talk about the next-generation AR technology it's developing.


Pikachu, a Pokemon.


Each of the advancements Niantic showed us is designed to help our phones better understand the world around them. For example, the company has created technology to identify the ground, people, cars and other everyday objects your phone sees when you hold it up. If your phone can effectively identify enough items, Niantic can do things like make the Pokemon character Pikachu appear to run behind someone's legs as they're walking, hide behind a plant, or duck out of the way of a car.

It can also create new types of games, like Neon, a demo of which I played. There were three other people also holding phones at FaceTime position as they moved around the mostly empty room. The screen shows what the camera sees, as well as extras like little white orbs on the ground and a score above each of the player's heads.

My objective with Neon: Gather up the white orbs by walking over them, then tap on the screen at my opponents to fire a colorful ball of energy at them.

Neon is an example of what Niantic ultimately wants us all to do. It had me moving around the room to pick up the orbs on the ground and avoid being hit by my opponents. It also had our phones communicating, something Niantic described as a technological hurdle it had to work with cell carriers to clear. And Niantic identified my opponents with little badges that followed them around on the screen, which means the game tracked where they were in the room as they moved.

"It's the real world — plus," said Michael Jones, who works on real-world mapping efforts at Niantic. "We want to build games where everywhere you go, something magical is happening all around you and it fits into the world you live in."

Though Pokemon Go changed the way we think of this technology, few other game makers have followed Niantic's lead. Instead, larger companies like Apple , Google , Facebook , Microsoft and a promising startup called Magic Leap have pushed the technology by offering software tools to developers. They've also created their own apps and demos like Animoji (Apple), playful animals that direct you down a map (Google), filters that put characters and masks on your face (Facebook), Minecraft on your living room table (Microsoft) and the promise of futuristic work spaces with email, video games and more floating overlaid on the real world (Magic Leap).

To understand Niantic's ambitions, it's important to know the company's roots. Niantic is a startup that has raised more than $225 million, according to Crunchbase. That's in part because its CEO John Hanke and other people in its management team came from a satellite-imaging startup called Keyhole, which Google bought in 2004 and turned into Google Earth.

So it's no surprise Niantic has plans far larger than Pokemon Go. Like many of its bigger competitors, Niantic wants to become a leader of AR technology, and it plans to make its tools available to be used throughout the industry. 

That's not just for games either. Niantic believes its technology could help companies create robots and drones that know where they are in space, self-driving car makers do a better job of seeing what's around them, and much more.

Meanwhile, it's pushing further into the Pokemon Go formula, which itself was something the company honed with its first game, Ingress, which launched in 2013.

Though Niantic hasn't offered many details about its next big game, Harry Potter: Wizards Unite, Hanke said the company's experiences with Pokemon Go are "feeding the development" of Harry Potter .

Solving large problems

Like many of the sleek apps and gadgets these days, Pokemon Go is deceptively simple. It's when you start to think of its scale that you realize how tough this game is to make.

Niantic effectively has a map of the planet, over which it has scattered Pokemon; "PokeStops," where players can gather more items to help with their hunt; and "Gyms," where players battle their Pokemon for bragging rights.

With these new technologies, and the acquisition of British computer imaging startup Matrix Mill, Niantic says it'll be able to better identify where players are and what's happening in the world around them. Ross Finman, Niantic's AR research lead, said the company's "curating map data across the entire planet."

Niantic is aware that technology for tracking the world around you, identifying objects you see and adding that all into a worldwide map can do so much more than help drive a popular game. Companies making self-driving cars  are grappling with and inventing tech to handle these same issues, as are companies making drones and robots.

"Any type of computation that interacts with the physical world, whether it's AR or robotics, will need what we're doing," Finman said.

A look at how Niantic's technology teaches computers to understand what's in a room.


To facilitate that, Niantic said it plans to make the systems it's built for identifying objects, building worldwide maps, and helping people interact within that world available to other app and device developers in the future. It's called the Niantic Real World Platform, and the company says it'll begin working with outside developers later this year.

In the meantime, Niantic's hoping we all start seeing it as more than just the Pokemon Go company. For example, now I think of it as the Pokemon and Neon company.

Pokemon Go: Everything you need to know about the hit mobile game.

Pokemon Go a year later: What we learned from the hit game, and where AR is headed.