Hacking Kinect: Setting sci-fi in motion

In the two-plus months since the creation of open-source drivers for Microsoft's Kinect made it possible to hack the popular motion controller, the device has been used as a sophisticated piano, to add voice control to popular video games, to create 3D renderings of a car's path, and even for a demo of soft-core porn.

Indeed, the open-source tools have snatched control of the device from Microsoft's proprietary hands, and each new day sees the emergence of some new kind of Kinect hack. But as the community of techies investing time in these types of imaginative--and unauthorized--works grows and each new hack pushes the envelope a little more, it's natural to wonder just how far hackers can go. What are, in other words, the outer limits of Kinect hacking?

"I truly believe that those who [say] that the sky is the limit [have] limited imagination," said Kristoffer Risanger, the founder of a site called Kinect Hacks that aggregates the latest efforts by the fledging community to do things with the device that Microsoft never intended. "There are virtually no limits to what can be done."

In other words, science fiction may be the best starting point.

If you're not familiar with the Kinect, it's Microsoft's new $150 motion-control camera system that lets people control all kinds of software with nothing more than their bodies. Where Nintendo's Wiimote translated users' movements of a physical controller into in-game changes, Kinect's camera captures the 3D movement of users' arms, legs, torsos, facial expressions--indeed, their whole bodies--and translates it on-screen. But when it was released, last November, it was tethered strictly to Microsoft's Xbox 360 video game console. But while there is already a wide variety of games from Microsoft and third-party developers that support Kinect, many people chafed at being limited to authorized uses of the device.

The story of Kinect hacking is one of a community refusing to be bound by Microsoft's directive on how the controller could be used. Within days of the release of the Kinect, the open-source hardware developers at Adafruit Industries had offered a four-figure bounty for the first person to come up with an open-source driver for the motion controller--a bounty that was raised after Microsoft said it was willing to call in law enforcement to protect its intellectual property rights. The money was quickly claimed, springboarding the entire Kinect hacking movement.

Before long, Microsoft softened its tune, with two of its people going on NPR and saying they were impressed with what the community was doing on its own. And not long after that, the nascent genre went where just about all new tech genres go. Kinect sex was born.

The question of what's next--and how far it may go--is obviously abstract. Risanger, who, professing to be overwhelmed by traffic, has put the Kinect Hacks site up for sale, said such a boundary has everything to do with the imaginations of those creating the hacks, though the Kinect technology itself has hardware limits--that will erode with succeeding iterations of the device. As well, Microsoft may try to sweep aside the whole idea of Kinect hacking by, as is rumored, releasing a Kinect software development kit for Windows. That would mean, experts say, that developers would have a vast scope for new, authorized, projects on Windows.

"Once Microsoft's SDK comes out," said Kyle Machulis, an expert on sex and digital toys and video games, and someone working on his own open-source Kinect projects, "anything on the Kinect is fair game."

Microsoft did not reply to a request for comment for this story.

To Philip Torrone, a principal at Adafruit, and therefore one of the people with a legitimate claim to kicking off the entire field of Kinect hacking, this is a very exciting time to be trying to find, or stretch, the controller's limits. "The Kinect as it is wasn't meant to do anything it's doing now," Torrone said. "We're only at version 1 [and] the best part about all these hacks will be all the things we cannot possibly imagine.

That said, with his prognostication hat on, Torrone sees a rich future of new projects made with the device.

Robots and mo-cap
"A few things come to mind, since the Kinect--once hacked--is a great input device specifically [for] whole-body movements," Torrone said. "I think we'll see the Kinect hackers start out using it to control machinery in some manner. There are some small robotics and telepresence examples already [but] that's just the start."

Torrone explained that he imagines Kinect telepresence projects along the lines of people employing the device as "puppet strings for controlling robots with vision systems over small and large distances." At the same time, he envisions the Kinect being used to run anything from industrial machinery to giant Burning Man projects. In each case, "the hacked Kinect allows the user to control extremely complicated machines just [by] using their bodies, distance, and shapes."

But limiting the Kinect to controlling Earth-bound projects is small potatoes, Torrone suggested. He finds it easy to imagine hacked Kinects "being used to control planetary or moon probes as we land new rovers in our solar system."

NASA, of course, might prefer to work within Microsoft-authorized uses.

At the same time, Torrone thinks that the Kinect could be a very attractive new tool for young robot enthusiasts and predicted that First, a large high school robotics organization, could start implementing the device in its competitions. And then there's the obvious military, industrial, and research uses. For one, Kinects could be employed to operate underwater robots, and for another, scientists might be able to use the controller to conduct microscopic studies. "I suppose we might see weaponized robots," Torrone said. "'Robot wars' with human puppeteers [using Kinects] and giant robots fighting is certainly going to happen."

Risanger believes that the Kinect will be a popular choice among amateur and hobbyist filmmakers and video game designers, giving Kinect users a low-cost motion-capture system. Mo-cap "has traditionally been insanely expensive, with just the basic equipment costing several thousand dollars," Risanger said. "The Kinect makes it possible to do motion-capture for just the small cost of one or two Kinects."

At the same time, he thinks that filmmakers and game designers might find the device useful for object scanning and green screen technology.

More Kinect sex
As vice president of business development for ThriXXX, the first acknowledged developer of a Kinect-based sex game--or at least a demo for such a game--Brad Abram has his own ideas about where Kinect hacking might go.

In the world of adult games, realism is one of the holy grails, and Abram foresees offering players the ability to get quite a bit more hands-on than is possible in any of today's sex-based titles.

On the one hand, because Kinect accepts voice commands, he sees games and 3D worlds where players can issue spoken directions to an on-screen avatar--such as "undress," or to take off this or that layer of clothing. And at the same time, the Kinect could make it possible to reach into the world to, as he says, "stroke and touch" an avatar.

"We're trying to figure out how to [do] 3D mesh morphing," Abram said, "so that when you reached in, you could actually be touching the [avatar's] breast. We've [already] got basic breast physics [down, such as] bounce. But now we want to make it deform."

Abram explained that some Kinect hacks have already demonstrated that users can experience a somewhat realistic sensation of touching and manipulating cloth. "The same thing could go for a breast or any other body part."

And expanding on what Risanger proposed about using Kinect for mo-cap, Abram thinks that mo-cap could easily be a valuable tool in the creation of sex games or adult 3D worlds, allowing users or designers to capture dancing, either vertical or horizontal, and capture some of that data as sex poses.

Of course, he said, full-on mo-cap sex would be difficult with capture technology like that of Kinect because of the problem of various limbs and body parts disappearing behind bodies. So multiple cameras and potentially different colored mo-cap suits might be necessary to overcome such limits.

Abram and Machulis both spoke optimistically about the work being done by PrimeSense, the Israeli company that designed the original reference hardware used in Kinect, on an open-source camera system. Abram said that PrimeSense is trying to keep its distance from anything sexual, but said the Israeli firm will have trouble stopping users from going in that direction. "It's a piece of hardware. It's a camera," Abram said. "It's like Panasonic saying you can't use [their] 3D cameras to shoot porn."

PrimeSense did not respond to a request for comment.

In the short term, Abram said, the word is that PrimeSense's camera might not be as powerful as Kinect but is likely to improve quickly. The prediction, he said, is that by March or April of this year, the camera could be on par with Microsoft's offering.

Either way, what's clear is that the Kinect hacking community is determined to push the limits of the new device far, far beyond what executives back in Redmond, Wash.--Microsoft's headquarters--ever imagined. Yet perhaps asking those on the front lines of designing and developing the hacks how far they can take them isn't the best approach to determining what's really possible.

After all, said Machulis, himself an accomplished engineer, maybe engineers aren't the ones best suited to imagining the device's true boundaries. Science-fiction writers, he said, may be the right people to ask. "It sounds," Machulis said, "like you want the people that create engineers' dreams."