Facebook's new VR cameras will get you bobbing and weaving
The social networking giant's professional camera setup, Surround 360, has new tricks it says will make virtual reality feel much more... well, real.
Imagine you're watching a butterfly as it flits through the air and into the foliage.
Normally, as it disappeared into the leaves, you'd bob and turn your head trying to see if you could catch one last look from a different angle. But in a movie, you can't do that.
Think again.
Facebook has created new designs for its professional camera, the Surround 360, which works in tandem with virtual reality headsets. Among its new features is the ability to peer around the virtual world it's filming. For the viewer, that means you can lean forward or side to side from your chair and you'll see a different angle, just as if you were there.
This technology is called "six degrees of freedom," or "6DoF," referring to all the ways you can move (front, back, up, down, and side to side). But to you and me, it means being able to peer around a lamppost or underneath a table.
"It gives you the ability to feel much more closely like you're there," Facebook CTO Mike Schroepfer said Wednesday.
The cameras were shown publicly for the first time Wednesday at Facebook's F8 developer conference, which is taking place in San Jose, California.
The new cameras and software, the designs and code for which Facebook is posting on the web for free and offering through a group of partner manufacturers, are part of the company's stated promise to spend as much as $3 billion over the next decade to convince you and me that VR technology is worth buying.
So far, it's been a hard sell. While there are some compelling movies, games and other experiences, fewer than 100 million devices have landed in people's homes around the world. That's less than half the number of phones Apple alone sold just last year.
Facebook's bet is that by attracting better content, be it games, movies or whatever else, people will naturally come. Which is what leads us to the Surround 360 camera and the company's attempts to help filmmakers create the best-looking videos they can.
If you try a 360-degree video on Facebook today, such as last year's advertisement for the hit Marvel film "Deadpool," you'll see it's good enough for what it is. Filmmakers use a mix of cameras and software to create a sphere of video that envelops you so you can look in any direction and see what's going on. In the case of "Deadpool," that meant seeing the namesake character doing all sorts of wacky things in the video.
Because Facebook's new cameras can understand depth, you can insert things like these luminous butterflies in a way that they blend into the scene.
But with Facebook's new 6DoF feature, you'll effectively be able to move about a meter in any direction in the virtual world, making it all that much more real.
"Once you put on that headset, you want to make sure you have that amazing experience," Schroepfer said.
If Facebook can pull it off, VR just might become a standard technology in Hollywood studios, documentary productions and, yes, eventually even our home movies. That's a big "if" for now.
Facebook declined to disclose the estimated sticker price of its new camera setups, dubbed x24 and x6 for the number of cameras oriented in a sphere to capture video. Its original Surround 360 design unveiled last year cost $30,000. (Which, while eyebrow-raising for the average consumer, isn't that much when compared with the competition.)
Facebook chose to home in on expensive cameras because it wants to "establish a creative norm," said Brian Cabral, an engineering director on the camera effort. One of the things Facebook focused on in its tests, for example, was filming people, which the company said presents a different set of challenges than filming the northern lights or the top of Mount Everest.
To make the magic that lets you actually lean into a film and see different angles, Facebook uses a mix of multiple camera angles, which is part of why it went from a UFO-pie-tin design last year to a sphere about the size of a beachball in the case of the x24. The other thing that makes it possible is software that uses complex math to identify how close and how far various objects are in the frame and then attempt to estimate their depth.
"It allows us to take our expertise and computer vision and go to town," Schroepfer said Wednesday. "We're able to recreate views that didn't exist before."
The one thing to know is that while this is a gee-whiz idea, it's quite the letdown in practice so far. The images were blurry and crude compared with the crisp images I typically see in a VR movie. Worse, other demonstrations, from the likes of Intel earlier this year, were much more crisp and lifelike.
In a video Facebook shot at the California Academy of Sciences in San Francisco, the leaves of the trees in a rainforest exhibit had a shimmering look to them, something Cabral said wasn't so much a result of the cameras as it was a sign more work needs to be done on the software.
These new designs and software should be available by the end of the year.