Apple's first crack at AR looks surprisingly good

Your Pokemon may look snazzier this fall.

Scott Stein Editor at Large
I started with CNET reviewing laptops in 2009. Now I explore wearable tech, VR/AR, tablets, gaming and future/emerging trends in our changing world. Other obsessions include magic, immersive theater, puzzles, board games, cooking, improv and the New York Jets. My background includes an MFA in theater which I apply to thinking about immersive experiences of the future.
Expertise VR and AR | Gaming | Metaverse technologies | Wearable tech | Tablets Credentials
  • Nearly 20 years writing about tech, and over a decade reviewing wearable tech, VR, and AR products and apps
Scott Stein
3 min read

Augmented reality: what the heck do you do with it? No one really knows for sure, but everyone's trying. Apple is making a concerted push towards AR with iOS 11, adding a developer toolset called ARKit that helps create augmented experiences on iPhones and iPads.

Google has AR. Facebook has AR. Microsoft has AR, too. Apple's approach is more like Facebook and Microsoft, aiming to bring convincing 3D objects into the real world. Of course, apps already do this: see Snapchat, or Pokemon Go. But these Apple AR demos looked nearly as good as Google's advanced Tango augmented reality phone hardware -- without the need for specialized hardware.

I got try a few experiences with Apple's AR toolkit on a new iPad Pro , and -- while it was just a quick demo -- I came away impressed.

What ARKit does

ARKit is a bit mysterious at the moment, but Apple details a few of its features for iOS 11. In particular, ARKit can combine camera information and motion sensing on the phone to track virtual objects realistically, not unlike what Google's next-gen Daydream camera tracking promises.

ARKit can also apparently recognize surfaces and edges, place objects properly and recognize environmental lighting to shade virtual objects properly. Furthermore, it adds extra graphics punch through Unreal Engine 4, Unity and Apple's Metal to offer some iOS-game-level graphics in AR apps.

The net result is a fairly convincing computer-generated object that appears to live in the real world, maintaining its position and lighting relative to the rest of the details on your screen.

It works with iPhone 6S or SE and later, iPad Pros, or Apple's new 9.7-inch iPad that debuted this spring. It uses the cameras already on these devices.


Augmented lamps cast convincing light.

Scott Stein/CNET

Augmented lamps, and Star Wars Holochess

One demo showed lamps and other objects placed on a wooden table in front of me. The table was real, the lamp and coffee and plant weren't.

I moved the map and coffee cup around, and shadows cast on the objects from the virtual lamp looked pretty convincing -- more than the typical AR demo. Objects tracked pretty steadily too, with no jittery leaping like I saw in my brief time with Microsoft's View Mixed Reality.


Fancy some Holochess?

Scott Stein/CNET

A demo of the Star Wars 3D Holochess, complete with Jabba hovering in the background, had geek appeal, and looked great graphically, but it wasn't nearly as impressive as the augmented reality lamp demo. Still, it shows how many games could get ported into quick AR-type experiences.

What impressed me the most were the small details: the shading, the graphics and the solid-looking feel of the objects on the table in crowded, live demos. Apple seems confident that its AR tools will work, and so far they seem a lot better than I was expecting -- especially considering that they run on existing hardware, and don't require the dual lens camera of the iPhone 7 Plus.

Life, Disrupted: In Europe, millions of refugees are still searching for a safe place to settle. Tech should be part of the solution. But is it? CNET investigates.

Does the Mac still matter? Apple execs tell why the MacBook Pro was over four years in the making, and why we should care.