Artist makes beautiful light with Microsoft's Kinect
After using a camera with an infrared filter to shoot some friends playing games with the hit motion-sensitive controller, Audrey Penven discovered a hidden cyberpunk treasure trove.
For months, we've known that Microsoft's Kinect could help make video games fun. But who knew that it projects such beautiful light?
Until San Francisco Bay Area artist Audrey Penven and some friends started taking pictures of themselves playing Kinect games, no one. But when Penven looked at the images, she realized she was on to something special.
In normal light, you can't even see the light put out by the Kinect, Microsoft's new motion control system for the Xbox 360. But with the help of a roommate's camera, which is modified to shoot infrared, Penven discovered scenes at once ghostly and straight from the cover of a Neal Stephenson novel.
Penven said she learned that the Kinect projects a known pattern of infrared structured light, and that when it picks this up with its built-in camera, the device figures out the shape of the 3D space based on the distortion of the pattern. "It uses infrared light so ambient visible light won't interfere with the process," Penven said. "I imagine this is also so it can remain invisible."
The images that resulted from Penven's photographic experiment show a cacophony of bright dots that encompass and enfold the people in them. They evince movement and wonder and hint at art. Yet the first time around, the light was little more than Kinect trying to gauge the movements of Penven and her roommates while they played a little Dance Central.
"I thought it was really amazing to see people defined by these infrared dots," Penven said of discovering the surprise in her photos. "I knew that infrared was used in some way by the Kinect to map out 3D space, but I didn't know what to expect when shooting with an infrared camera...I thought it was interesting that the human form could still be so recognizable, even when only shown in tiny dots. I loved the quality of light and the different way of looking at depth and form. [And] I was inspired by the way the Kinect was using a pattern invisible to human eyes to see us."
Related links
• 10 million a magic number for Microsoft's Kinect
• Culture hacker talks Kinect bounty hunt (Q&A)
• Bounty offered for open-source Kinect driver
• Hacker wins contest for open-source Kinect driver
• Microsoft announces plans for Kinect SDK
That inspiration led Penven to take what she had just inadvertently learned and run with it. Created with the help of artist and animator Aaron Muszalski, the result is her first-ever solo art exhibition, titled "Dancing with Invisible Light," which opens Friday at the Pictopia gallery in Emeryville, Calif., and which will run through April 29.
"With these images I was exploring the unique photographic possibilities presented by using a Microsoft Kinect as a light source," Penven writes in the invitation to the opening of the exhibit. "As a photographer, I am most interested in the nature and quality of light: how light behaves in the physical world, and how it interacts with and affects the subjects that it illuminates. For this shoot my models and I were essentially working blind, with the results visible only after each image was captured. Together, we explored the unique physicality of structured light, finding our way in the darkness by touch and intuition. Dancing with invisible light."
Unanticipated use of the Kinect
Though the Kinect has been an unqualified success as a video game accessory, selling more than 10 million units since its November debut, it's also been a huge hit in the hacker community.
Literally from day one, that community has been out to take the Kinect places where Microsoft never intended. A $3,000 bounty offered by the open-source hardware outfit Adafruit Industries for the first open-source driver for the device bore fruit almost immediately, and since then there's been a near free-for-all among people wanting to use the Kinect for things far outside of gaming.
And to some of those who have been following this movement since the beginning, Penven's work fits in beautifully.
"[It's] stunning. This is another great example of the tool being used in a way that [Microsoft] could not imagine," said Phil Torrone, a principle at Adafruit Industries. "Are they diamonds, are they points of light? It doesn't matter--it's just one of the many expressions the hacked Kinect has enabled for artists, designers, and even photography--something that's been around for almost two centuries."
To Kyle Machulis, a hacker and artist who has experimented with Kinect-created visualizations, Penven's work is deeply impressive, particularly given that the device has been on the market for such a short time.
"It's really amazing, the way she brings such beauty to [something] happening in millions of homes around the world right now," Machulis said. "We're only five months into the release of the Kinect and the technology is already becoming a bit of an afterthought to many consumers. But what's going on behind it still seems like magic even to those of us close to the technology, and [Penven's] pictures really bring that out."
Of course, Penven is hardly the only one using the Kinect to make art. Do a quick Google search on the term, and a seemingly endless supply of links pops up. They range in style from the art that can be captured on screen with a series of gestures to 3D printed representations of Kinect users' motions to a storytelling initiative that helps children gain confidence in their self-expression to an effort to use the Kinect to help blind people restore some sense of sight.
Just a little flash
While Penven's final images blossom with light, the reality is that when she and her models were shooting the pictures, they were in a dark room with nothing more than a camera flash to illuminate them. But throw in the infrared filter and the pictures burst into life.
"Most of what you see was done in camera," she explained. "For some shots, we experimented with longer exposures and movement. Color and contrast are the only adjustments I made after the fact. Infrared photography, by its nature, is a false color process. The infrared spectrum is represented by colors that we can actually see. I hadn't intended to change the color much from what came out of the camera, but I thought that the difference between the light from the flash and the Kinect was really cool. I decided to emphasize that by pushing the colors in different directions."
Although Penven and her friends discovered the artistic possibilities of the Kinect while playing Dance Central on an Xbox, she said that for the photos in the exhibition, they never hooked up the game console. So what you see in the images doesn't has no relationship to any game--it's nothing more than the million points of light from the Kinect rolling over Penven's models.
In the end, she came up with a whole set of images from the shoot. But one that she is using to promote the exhibit may do the best job of illustrating what the show is all about. In the image, a woman is seen half normal, and half flooded with purple dots. It screams cyberpunk.
"I love the contrast between the sides of her face," Penven said. "Right after this one was taken, and there was a group of us standing around the camera to see it, someone said that she looked like she fell out of a sci-fi story. That really stuck with me. It's like a traditional portrait of a half-digital human."