Picture a 3D virtual representation of your living room, one you can fly over in a top-down view and even move through with the fluidity of a first-person video game. Now imagine having the ability to tinker with that space: Change the paint on your walls, drop in a new couch to see how it fits with the existing furniture, or perform accurate measurements of the room, all on a computer screen.
That's Matterport's vision for the future of 3D modeling, and it extends beyond home renovation. From architecture and construction to real estate and crime scene visualization, the scope of 3D models is expanding and the hardware and software that allow us to map our physical world are getting cheaper, faster, and better. Matterport is hoping it can be not only the Nikon and Canon, but also the Adobe of the burgeoning industry, offering a professional-grade camera and a cloud-processing platform for making 3D modeling exponentially easier and accessible.
- Chief of Google's Project Ara talks modular smartphones
- Watch a virus-like particle try to enter a cell -- in 3D
- Maker 3D-prints himself a prosthetic flexing fingertip
- BigRep 3D printer can print whole pieces of furniture
- eBay acquires PhiSix, which lets users try on 3D clothes virtually
If you've heard the name Matterport recently, it was likely in association with Project Tango, the mobile 3D mapping venture out of Google's Advanced Technology and Projects division, the research arm of Motorola Mobility that Google absorbed into Android before selling Motorola to Lenovo. Matterport got its hand on a prototype and last month released one of the first 3D models using Tango. Matterport, however, is aiming for a more pro-grade, less experimental market when it comes to its own breed of modeling.
On Thursday, the Mountain View, Calif.-based company -- created in 2011 and running on Y Combinator and VC funding amounting to $10 million -- launched its full platform suite, which includes the $4,500 Matterport Pro 3D camera alongside its subscription-based cloud service and Web player. Before that, an early-adopter program saw only a few dozen cameras get out into the wild to create more than 1,000 3D models. Matterport will sell the device and service directly, aiming primarily at contractors, real estate companies, and architects with the intention of drastically changing how those industries work with digital visualization.
Matterport modeling is all about automation
Because while 3D modeling is of course not new technology -- existing for years in architectural modeling, video game design, film CGI, and computer animation -- it's still an incredibly intensive and multi-step process. You need people to take measurements and draw out schematics to build an outline of a space, photographers and videographers to capture images for texture and depth, and designers and engineers working in 3D CAD software to construct it. Even then, there's still no easy way to cohesively stitch together all those parts into an easily transferable and reasonably-sized file.
"If you spend enough effort and time, you can make a model that looks like something we can make," said Matterport CEO Bill Brown in an interview with CNET. "You'll have spent $20,000 or $30,000, but you end up with something you can't distribute. You can give it to someone if they have a CAD package," he added.
It's no coincidence that Brown was tapped by Matterport's three co-founders -- Reactrix founder Matt Belle, PayPal alum Dave Gausebeck, and former SRI 3D guru Mike Beebe -- to be CEO. Brown came from Motorola Mobility, where he was general manager of a division called "converged consumer products," during its time under Google's wing, where Project Tango would later be hatched. The combined expertise of Brown and Matterport's co-founders results in a highly focused effort that's more practical than flashy, and less intent on doing something radical -- like Tango -- than it is in tacking what the company thinks is a dormant technology ready for acceleration in select fields.
That's how Matterport was able to take the 3D modeling process and automate away the most tedious aspects of it. "In a matter of an hour, you can do something that takes two days for people to do now," Brown said.
What exactly makes Matterport's camera that much of a leap? For one, it has the ability to capture geometric and texture data simultaneously, while offloading much of the intensive computing to combine that data to the cloud to be done after the fact. "It's shooting at 30 frames per second, so it's in essence taking a video as it spins," Brown explained.
With three sensors relaying information between 2D and 3D, the camera is able to take in a near-360 degree scope of the room in one motorized sweep that takes less than half a minute to complete. Combine anywhere from as little as four to six sweeps and as many as 15 to 20 of an area and you have a detailed model that can be compressed down to an average file size of 50 to 75MB. Even then, one can dig into the raw files of the sweeps and swap in stills, so that fuzzier portions of the model containing books on a shelf or a clock on the wall can suddenly have the fidelity of a hi-res photograph.
Even mirrors, which would have to reflect appropriately as one moved through the 3D model, can be dealt with. "We identity areas that are mirrors and replace those with digital mirrors," Brown said, adding that "a lot of the techniques and technologies we're using on the application side come from the computer gaming world." Like in video games, mirrors and differences in lighting can be easily replicated with video effects tools.
The key, however, to Matterport's efforts lies in not just small file sizes, feature sets, and affordability, but also in ease of use. "We've automated that entire process and got it to the point where anybody can operate the camera and the cloud processing figures out how to do everything," Brown said. "It puts this thing together like a jigsaw puzzle."
And it is true that anyone can operate the camera. It's as simple as pressing a button; I did it myself, on an iPad, while a Matterport camera 3D-mapped a studio room at CNET's San Francisco office. You have to manually move the camera to desired areas for new sweeps, and also have to walk around it as it's moving -- it pauses after each motorized rotation -- unless there's portion of the room that you can use to conceal yourself.
The end result, after sending the data to the cloud service and letting it build the model for roughly 30 to 45 minutes, yields a mix of awe and a strange sensation akin to animation's uncanny valley, thanks in part to the video game-styled movement you're employing within an unprecedented level of photo realism. Moving around can be done with directional arrows, including a jumping capability with the space bar. Or if you're on an iPad, which can run Matterport's 3D models via its Web player, you can use specialized touch commands like two fingers to strafe side to side or pinch-to-zoom to go from the top-down view into first-person mode.
The best description of the experience would be Google Street View meets the interactive panorama, the multimedia trend of the last few years that uses a series of shots from a singular point to create an immersive drag-able photograph.
In that vein, Matterport is aware its tackling more than just the current 3D modeling use cases. "We're trying to establish a new category of media," Brown said.
Next up: Movement modeling and mapping with mobile
Matterport's hardware and software has clear limitations. For one, the camera can't process glass and light-induced reflections, rendering any windows entirely translucent. And certain levels of light and drastic changes in those levels are also tricky, meaning Matterport can't be used outside except in only select conditions.
Most importantly though, the camera tech is hampered by movement, which Matterport has to tackle down the line. "It's going to be a specialized set of processing that we're going to need to figure out," Brown said of moving images. "I do think that at a high level view, we can tell the difference between something that's moving and something that's not moving. We can isolate the things that are moving and handle them appropriately."
That means that right now, not only must the camera remain still, but no one can sit anywhere within its sight unless he or she sits still, or else they'll be rendered into a kind of "2D plus," as Brown put it, resembling a paper cut-out.
That's an integral challenge for the 3D modeling field at large, and especially the large players in the 3D sensor industry, nearly all of which are communicating with Matterport, Brown said. Because the goal is not just to move 3D modeling capability to mobile devices, as Google is attempting with Tango, but to implement that idea as ubiquitously as the integration of cameras was to early feature phones.
"We kind of say down the road as you have 3D sensors in these mobile devices, you're not going to take 2D pictures anymore. You're always going to capture the 3D behind because theres a bunch of useful things that you can do with that 3D, even if it's just a still image," Brown said. "If you were going to walk through the space and shoot a video, you could build a 3D environment that in addition to that linear video could give a user the ability to not just pause, but back up on a path and say, 'I'm going to see what was to the sides as you were walking.'"
To achieve that, Matterport won't be focusing on drastic cost-cutting, multiple product tiers, or pushing to make its own mobile device, despite having originally started out attempting to develop a 3D mapping-capable handheld. "One of the big pushes that we have is to work with the folks on the mobile side to make sure that these devices are going to capture the right data," Brown said. "We'll probably just make our software work on those devices. I'd never say never because there might be a point where a partnership with one of those company that makes one of those devices makes sense. But it's not Matterport's focus," he added.
The company's longterm view then is to build out its software platform to the point where Matterport applications become the primary way to process 3D models, ones captured from mobile all the way up to professional-grade, DSLR-level cameras like the one Matterport is now offering.
"Theres a point, whether its 5 years down the line or whenever it is, where every time someone captures a 2D image, they're capturing 3D data," Brown said. "We're gong to be the company that makes that combination very useful."