Want CNET to notify you of price drops and the latest stories?

The problem with augmented reality: Tablets and targets

Augmented reality is showing up in all kinds of applications. But there are a few arenas where it still doesn't quite work.

Rafe Needleman Former Editor at Large
Rafe Needleman reviews mobile apps and products for fun, and picks startups apart when he gets bored. He has evaluated thousands of new companies, most of which have since gone out of business.
Rafe Needleman
3 min read

Is augmented reality for real? At the annual Augmented Reality Event conference in Santa Clara, Calif., this week, marketers, entrepreneurs, lawyers, and science-fiction authors (Daniel Suarez and Bruce Sterling) were all looking for ways to leverage a technology that could change the way we use computers and access data and media. Or not.

The challenge for many of the AR projects being shown and discussed at conferences like this is that to use them, you have to contort yourself around a tablet or smartphone, which becomes the window through which you see the augmented world. You might also have to print an AR "target" that your app uses as a beacon, over which it can display its imagery.

Sesame Street goes virtual
Watch this: Sesame Street goes virtual

In some applications, using a tablet-and-target system works just fine. In an art gallery, for example, holding up a smartphone to view supporting media about a painting is not asking too much. And for commerce, the system can work as well: I used the Panasonic Viera AR Setup Simulator app to see how a flat-screen TV would look in my living room before I bought it (I got an LG instead, but that's a different story).

But in other applications, in particular for books and games, and for some gimmicky marketing apps, you have to drag out your phone, load a specific app, point the camera it at a particular target (a sign, a gameboard, a book), wait for the software to recognize what you're pointing at, and then keep the phone pointed at the target while you interact with the app. It's cool that the Sesame Workshop is building apps that turn Bert and Ernie dolls into moving animated characters, but the app might be asking too much of its kid users. (See video.)

Daqri demonstrated a medical app that put a representation of the inside of the body over a printed map of its outside. "Medical schools don't have enough cadavers," CEO Brian Mullins told me. Rafe Needleman/CNET

These apps make for great demos. They feel almost magical, in fact. But the magic wears off once your arm gets tired. Or you have to put the phone down to interact with the real world (like turning a page).

Are goggles the answer? The founders of the AR software company Daqri told me they're very excited about augmented-reality eyewear coming out, from Google or anyone. It will make their education and training apps more usable, and solve the arm-fatigue problem. I still don't think that casual users will be very interested in having to put on special glasses just to see a 3D effect in a little app, and I seriously doubt that AR goggles are going to get parental approval for kids under the video game uptake age (about 12).

In other words, AR in media is more 3D TV than pop-up book: A feature everyone will eventually have, even if they don't use it much.

As I said, the other challenge with AR media apps is the need to have a "target," for the AR software to lock on to. This, however, is a temporary issue. While there are still apps that require uses to print out big, bold targets, newer apps are getting better at locking on to more subtle cues. And robot vision technologies under the heading of SLAM (Simultaneous Localization And Mapping) are migrating down to smartphones and tablets as processors get more powerful. This makes it possible for AR apps to overlay their virtual worlds over the real one without requiring an artificial target. Soon enough you'll be able to virtually place furniture you're thinking of buying in your house by just pointing your smartphone's camera into your room.

Total Immersion's Marie Geffroy tries on sunglasses with her demo. Rafe Needleman/CNET

One company at the Augmented Reality Event, Metaio, was showing its toolkit for creating targets from real-world objects. In a demo, I saw the system generate target data from the digital camera I handed to developer Jacob Ervin. This could make products' user manuals far more useful.

Augmented reality is becoming part of our landscape: Cars will be getting AR navigation overlays; shoppers will be using AR to try on clothes (see Total Immersion and other companies), and for product manuals, training, and vocational education there are wide open spaces for the technology. There are also some really great AR art projects. But augmented reality doesn't belong everywhere. As one person told me at the conference, "it used to be magic to float a shoe over an advertisement, but you have to do better now." Much better.