Augmented reality comes to mobile phones
It's been used for years in video gaming and for military applications, but now, thanks to advances in chips and wireless, AR appears destined for the mobile market.
If you thought location-aware search and other mobile mapping technologies were cool, hold onto your hat. A new wave of innovation in the mobile market will bring augmented reality to smartphones, allowing users to interact virtually with their surroundings.
Augmented reality, or AR, is a term that refers to technology that superimposes computer-generated content over live images viewed through cameras. The technology, which has been used in gaming and in military applications on computers, has been around for years. But thanks to more sophisticated devices, faster wireless broadband networks, and new developments at the chip level by companies like Qualcomm, it has become inexpensive enough to put into smartphones and tablets.
Even though these are still the early days for the technology--chip vendors like Qualcomm are just now giving demonstrations--augmented reality could have a major impact on smartphones in the coming years.
"The idea that a mobile device knows where I am and can access, manipulate, and overlay that information on real images that are right in front of me really gets my science fiction juices flowing," said Mark Donovan, senior analyst at ComScore. "It's just beginning now, and it will likely be one of the most interesting trends in mobile in the next few years."
Just as location-based services have begun to change how wireless subscribers use their cell phones and marketers reach an increasingly mobile audience, augmented reality will go a step further, bringing a wealth of collected data to users' fingertips.
Today, GPS and other location-based technologies allow people to track and find friends on the go. It allows them to "check in" at particular locations. In other words, wireless subscribers provide information about their surroundings, such as where they are, and that information is stored and shared with others via the Internet cloud. That information can be used so friends can locate you, or it can be used by marketers to send you coupons and other promotions.
But as these location services are married to augmented-reality technology, mobile subscribers have the opportunity to allow the crowd-sourced data about a particular location to tell them about their surroundings. Think of Flickr and Wikipedia information layered on top of real-time images you can see via your phone's camera and displayed on your smartphone's screen.
For example, imagine you're on a walking tour of New York City. You could stand in front of 97 Orchard St., the location for the Lower East Side Tenement Museum, and get a virtual tour guide to tell you about the building and its history. Images of how the neighborhood looked 100 years could be overlayed on top of the existing image of the neighborhood on your mobile phone screen.
Facts and other information about the neighborhood and the people who lived there could be embedded onto the screen. And by clicking on an icon, you could hear audio, see video, or read text about what happened there. Other visitors could leave virtual comments about the tour. Maybe someone would leave a virtual note letting you know of a good pizza place a block away. That pizza joint might also insert an icon offering you a coupon.
AR could also be very useful in education. Biology students, for example, could use an augmented-reality application and a smartphone to get additional information about what they are seeing as they dissect a frog.
Toying with AR
These are just a few examples of how AR on a handheld device could be used to enhance user's experience. Matt Grob, Qualcomm's senior vice president of engineering and head of corporate research development, recently showed off the company's augmented-reality technology at the EmTech conference at MIT in Cambridge, Mass. Grob and other Qualcomm execs have been demonstrating the new technology for the past few months at various conferences to drum up interest and excitement.
The company's Snapdragon processors and a new software developer kit for Android smartphones will help provide the necessary foundation for building and using augmented-reality technology on cell phones.
In the Qualcomm demo, the company teamed with toy maker Mattel to create a virtual update to a classic game called Rock 'Em Sock 'Em Robots. Using Qualcomm's technology and the embedded smartphone camera, players could see superimposed virtual robots on their smartphone displays. The robots appeared in the ring, which was a piece of paper printed with the static image of the ring and its ropes. Players used the buttons on their handsets to throw punches, and their robots actually moved around the ring as the players physically circled the table where the image of the ring was placed.
Grob said gaming offers a huge opportunity for Qualcomm and its augmented-reality technology. But he said others will also see the usefulness of the technology over time.
"Augmented reality certainly enhances the gaming experience," he said. "But it's also a nice way for advertisers to improve their reach to consumers by bringing in contextual awareness between the device and the physical world."
For example, marketers can insert animated coupons on top of real-life images of their products in stores. So when consumers walk by a box of cereal in the grocery store and look into their phone's screen, they could get an instant rebate.
While augmented-reality technology has been around for a while and is already being used in some PC games, Qualcomm has advanced the technology by making it more affordable and usable for portable devices.
Grob said Qualcomm's software developer kit for Android devices, which was announced earlier this summer, will be available in coming weeks. This will allow game developers and others to start developing applications using the augmented-reality technology. And these games and apps should be able to operate on current-generation Android smartphones. Grob said the technology is already mature enough for commercial use. The demonstration of the Rock'em Sock'em game used Google's Nexus One Android handsets.
Qualcomm's software developer kit will initially be available for Android, but other mobile platforms are expected to be added later.
ComScore's Donovan said he expects to see a lot more experimentation with augmented-reality applications on mobile phones over the next 12 to 24 months, with more large-scale consumer services hitting the market in the next few years.
"It's very early days still for the technology," he said. "You'll see some applications, but it could take a couple of years for someone to develop a meaningful use for augmented reality in mobile. But once they do, you'll see it really take off. And then we'll go from simply telling apps where we are, as we do today, to apps telling us about the world around us."
Update 7:30 a.m. PDT: Indeed, there are some augmented-reality applications already available. The iPhone 3GS and iPhone 4, which includes a built-in compass in the phone, support many augmented-reality apps. A company called PresseLite has developed a version of the Metro Paris Subway application that uses augmented reality. The overlays display information about Paris businesses when you look at the city via the iPhone camera.
Acrossair has developed a similar AR application for the New York Subway system. It allows you to hold up the phone and look through the camera and see on the screen where the closest subway is. It also points toward other New York City tourist destinations and landmarks and provides the approximate distance.
Another iPhone app called Panoramascope gives you information about your surroundings, such as the names of mountains you see on the horizon.
The Layar Augmented Reality Browser for Android phones allows developers to build AR into their apps. And some have already created some applications using it. For example, HPSC has created an augmented-reality version of the popular 1980s arcade game, Pac-Man. In this game, players run after a blinking dot and users get points for every dot that Pac-Man eats.
DNL Pro has created Message Central, which allows users to leave messages on customized billboards around them. All users are then able to read the message on a 3D augmented-reality object.
Surprise Me by Elipse Ad has created the Surprise Me app that allows friends to leave augmented-reality gifts for friends to find using their phones' cameras. The way it works is you open the application in the camera view and see what your friends have left you, or what other people may have shared. They could leave photos, audio files, and text gifts all around you.
Even though there are already many applications using augmented reality on the market, the technology has still not hit the mainstream. For the most part, the technology has been more of a novelty. But as devices, processors, and software add more horsepower and sophistication the level of information that can be added will increase, Donovan said.
"It's really only the beginning of what is possible with this technology," he said.