How Leap Motion tracks what it can't see
We're not at "Minority Report" quite yet, but the latest Leap Motion upgrade and its new Freeform software have brought us one step closer to the dream of next-gen computing.
Leap Motion, makers of the matchbox-sized motion tracker that launched back in July, has gone even further in its efforts to make interacting with computers in open space as accurate and powerful as possible. An upcoming software upgrade announced Wednesday will allow any Leap Motion device to perform far more precise tracking by viewing one's hand as a whole object without needing to see its every move, allowing for new and more accurate forms of interaction.
The upgrade, which has been in the works for about a year and marks the first substantial overhaul to the device's software, is going out to all 85,000 developers in Leap's open-source community shortly and making its way to all Leap devices in the coming months, said CEO Michael Buckwald. "It takes the incredible tracking in our current generation of Leap and takes it the next level by being able to track things even when the device can't see them," he said.
"The way our tracking works now is what you see is what you get," said CTO David Holz. The upgrade, Holz explained, will let Leap track the hand as a general object now, allowing for actions like pinching, crossing fingers, moving one hand over another, and hand-to-hand interactions like brushing and tapping fingers on one another.
While those more precise movements sound arbitrary on the surface, the added accuracy to the Leap Motion tracking opens up an entirely new level of depth to hand movement that developers can utilize for new inputs or increased functionality.
"This makes it easier for developers to build even better physical experiences. It's about bringing everything closer to that original vision: how to bring the incredible power of hands and fingers to computers and make that interface disappear into the background," Buckwald added. "It's about making it as close to reality as possible."
"A lot of these improvements are important, but you don't necessarily notice them until you're used to the first," Holz said, noting how consumers are now conditioned to smartphone upgrades. "Otherwise they would never release a new phone. They would just wait until they had one that was infinitely fast."
And much in the same vein, Buckwald said, of Elon Musk's demo of spacecraft part building with Leap Motion in August, the company is also releasing for free a creation software tool called Freeform that allows users to experiment with geometric creation and manipulation. Like playing with clay on a pottery wheel, the app lets users distort any number of premade objects -- as well as import their own -- with a slew of tools and variables like object material to create digital sculptures and even 3D-printable objects.
"There are already apps for the very high end, like the SpaceX video using Siemens software with a Leap," Buckwald said. Freeform is intended to be somewhere between those professional-grade applications and one's that everyday consumers will find accessible.
"The holy grail for this sort of this thing is that an 8-year-old could walk up, put their hands out, and then play with this. But someone who is older and has been using this for a while can really build something sophisticated with this," Buckwald added. "I think we've struck that balance."
Freeform is just one a growing number of applications Leap Motion has urged developers to create around motion tracking, from art and educational tools to musical instruments and video games. Since launching its Airspace app store, the company has seen its application count double and hopes to have 150 by the end of the year. It has also made in-roads in adoption, getting its device in thousands of retail stores in 12 countries, as well as embedding the Leap technology directly within HP's Envy 17 laptop through a unique partnership, integration Buckwald and his team hope to expand to many other devices and form factors.
"What were about is not just making things simpler for professionals, but also making it so that people who don't have has much ability with technology now can do more," said Holz.
"It doesn't matter if it's possible or even if it's reasonable. It has to be easy," Holz added. "Basically, we're going towards a point where you can have a indistinguishably real, but still in some way synthetic, digital reality that you can interact with, but is more powerful than the reality we normally have because we define all the rules. That's really interesting, but it's a long journey."
Update, 9:31 a.m. PT: Adds background on Leap Motion's Airspace app store and HP partnership.