When you hear about virtual reality, whether or not you've ever used it, you probably think of video games that transport you to a far-off space ship, or to an underwater cavern, or some mythical dungeon. Ready Player One, or Black Mirror. Sci-fi things. You know the drill.
VR's been a work in progress for five years at Facebook, with the technology improving but the masses not diving in. Despite promises and reports of Facebook making augmented reality headsets that blend the real and the virtual, like Microsoft HoloLens and Magic Leap, Mark Zuckerberg, Facebook's co-founder and CEO, says VR is still the way forward, and is more than a doorway to games. He still sees it as Facebook's bet on the future, not unlike the bet Steve Jobs made at Apple a decade ago on the iPhone, which helped kickstart the mobile revolution.
More importantly, he believes Facebook is the best company to lead the industry when it eventually takes off.
"For me, it's never been about just technology, it's been about how can you make technology that's more natural to people and a lot of that is about interacting with other people," Zuckerberg said in an exclusive interview two days before he was set to take the stage at the company's Oculus Connect developer conference to tout its VR dreams.
"The thing that we care about is delivering human connection and helping people come together," Zuckerberg said, leaning towards the theme of presence, and away from a model "that's more just around, here's your app, here's your content, I'm gonna pull it from a store."
Zuckerberg's not-so-veiled knock toward competitors with competing headsets and app stores include may sound like typical Silicon Valley infighting. The competitors are many: Google with its Daydream headset project, Valve's Index headset, Microsoft's Mixed Reality and even Apple's nascent VR and AR efforts. But it's also a sign of how serious Facebook is about backing VR and, soon, AR too.
When Zuckerberg announced that Facebook was buying then-startup Oculus VR in 2014 for more than $2 billion, the tech industry was confused why the social media giant would care about VR. But to Zuckerberg, it was -- and is -- a bridge to the AR future, which he believes is the next tectonic shift in tech. He describes it as a move from smartphones in our pockets to VR headsets that embed us in a computer-generated world. Then, eventually, to AR glasses that overlay computer images (dinosaurs, spaceships, fairy forests, avatars of our friends, mapping directions down the street) onto the real world.
"There's a lot of questions that people have about where's this all going, why is Facebook doing it," he said. "We have our social mission -- giving people a voice and bringing people together -- but you can think about what we try to do from a technology perspective as putting people at the center of your experience with technology."
To get there, Facebook's released a trio of headsets, all of which put a screen so close to your eyes it can trick you into thinking you're really in the computer-generated world. There's the low-end, low-power Oculus Go for $199; the more-capable completely wireless and self-powered Oculus Quest for $399; and the sharpest, most capable headset, the Oculus Rift S, which is also $399 but requires you to own a high-powered computer to power it.
Next, Zuckerberg says, is coming up with new tech that makes it so when you put on an Oculus headset, it really does convince you you're in that other, digital place.
"What AR and VR do is deliver a sense of 'presence,' where you actually feel like you're there with a person, it's a really deep connection," he said. "There's a bunch of software and experiences that we need to do directly around helping to facilitate people connecting."
Facebook's latest innovation is hand tracking, which allows you to interact with images in the Oculus Quest with your hands, instead of the controllers it has now. For example, you could end up poking in the air with your finger, and see it push a button in the virtual world.
Facebook believes this will be most helpful for businesses and schools, and in demos at Facebook's headquarters it worked reasonably well, if a little slowly.
Facebook isn't the only company building this technology, of course. Microsoft tracks hand movements with its HoloLens 2, as does Magic Leap with its namesake headset. Apple has programmed software powering the cameras on the iPhone and iPad to identify people's body parts as they move. Samsung and other phone-makers are offering similar features too.
But for Facebook, using your hands means accessibility, and an easier way to dive into VR for first-time users and workers. Leaving controllers behind could also be the first steps towards Facebook working out how to build an interface in augmented reality.
Zuckerberg spoke about Facebook's plans with VR, about how the privacy scandals that have rocked his company could affect his virtual worlds, and about what he thinks it'll take to make VR succeed.
Here are edited excerpts from our conversation.
While you're building all this new technology, you're still struggling with fake news, election interference and privacy issues. How do you think those issues might show up in VR or AR?
Zuckerberg: Certainly, there are lots of issues that we're working through, whether that's finding the right balance between free expression and safety on content moderation; a lot of civic work specifically on election integrity; privacy; making sure that the connections that we're helping to facilitate are positive for people's wellbeing; and that we mitigate things that could be negative or dangerous.
There are a lot of social questions that people have. That's been a lot of the journey of the company for the last few years -- making sure that we work through these well. Right now, there's still a lot of questions. I don't think that these are things ever get fully answered -- the threats evolve and you need to work on them. But I would hope that by the time that these ecosystems are mature, our approaches to those issues will also be quite mature and fully vetted, and people are broadly more accepting of what our approaches are.
There's a lot of talk about AR and its promise. Microsoft has HoloLens, Magic Leap has its device. You've talked about making AR glasses. Apple's reportedly researching glasses. Where does VR fit in?
VR is going to be a central part of this. I said before that I think VR is going to be bigger than people realize.
One basic analogy is if you think about how we use screens and the rest of our lives, phones are the ones we bring with us, but half of our time with screens is TVs -- and because there's a lot of time where you sit and you want like an immersive experience. And if you're going with the future, I think VR is TV is and AR is phones.
You're not going to bring VR -- like walk through the streets with it. But I would imagine that over time, one of the key use cases of AR glasses will be VR. And being able to go into a mode where you where you can do that. It'll never be as perfect as actual VR goggles.
So, I think that there will be a market for both just like we watch TV shows on our phones, but a lot of the time we would prefer to watch it on TV. I think both are important.
Certainly we're doing more long-term research now focused on AR as well. But I think the commitment to VR is certainly still strong, and that's the technology path that we're going to use to get there.
How is VR adoption doing right now?
It's doing well, I don't think we're sharing a number. But I think what we can say safely is that we are selling them as fast as we're making them.
A lot of why I'm so focused on the ecosystem, is because we are still at a scale where it is not economical for the largest AAA [top-tier] developers to say, "We're going to put all of our resources on making a game for this," because there aren't enough people who have it to make back all their money, or at least make that a better investment than building a game for Xbox.
We think that there's a critical mass and a lot of what we're focused on doing is getting to that critical mass. Part of what we're doing is we're investing a lot and buying and licensing content from developers for who it might not be economical on their own today, because there's a chicken and egg.
In the console world with the Microsoft Xbox, Sony PlayStation and Nintendo Switch, developers port a game over so it works on various systems.
I don't know that right now, anyone else is making anything quite like Quest. So, we're quite far ahead on that. Just getting all the compute to work and getting that to work within the thermal envelope. Also, we want the device to be affordable for people. So, we're pushing really aggressively on cost. And so on all these things, I think we're quite far leading on that.
Do you see the Quest continuing to evolve in the immediate future?
There's a lot more that needs to happen. The form factor today is not what we would want it to be in the future. It's still very bulky, right. There's a lot of research that's going into making that better in the future.
And you know, $400 is a good price, we think it's a very good price for what it can deliver. But I think over time working on that would also be right. I think having stuff that's higher end and lower [price], could be a direction to go. But there's many, many years of work ahead of us on this.
What about eye tracking? That's something that's considered key for VR and AR. And there are questions about what happens with eye-tracking data. Oculus hasn't done eye tracking yet.
There are a bunch of sensors that you want to add, but part of what needs to get worked out is, if you added every sensor imaginable, then you get to a device that's bulkier, that eats more energy. So that's going to be the basic constraints on this, is there's social acceptability -- can you get something that is reasonable? And can you get something that doesn't consume so much energy that either burns through the battery quickly or just gets too hot, right, to wear on your face?
So, assume that that's part of the question that we're trying to figure out: What do we need to deliver to deliver real presence? That's the big question for us. Other folks might be thinking about, OK, well, what can I make in glasses that would be useful? For us, it's the magical thing about AR and VR, compared to every other computing platform before, is that it actually could let you feel like you're there with a person that makes you feel more present, rather than pulling you away from people like we do with our phones and computers today. And to me, that's the holy grail of the mission of this company, making it so that you could be anywhere in the world, and you can feel present with people both for social, and for work.
What's still missing in VR for you to achieve presence? Now that you're adding hand tracking, what about haptics to feel things?
Haptics would be great, and you can imagine a number of ways to get there, whether it's a controller, or a band or something like that. But a lot of times, people will make trade-offs between the fidelity of the experience that they want and the natural ease of it. So, hands are the easiest, but haptics would be useful. And one thing that we're very focused on is expressive emotional avatars and delivering that well.
And extending out, what about Facebook's Libra cryptocurrency? We read a lot of William Gibson, Neal Stephenson books. Could having a currency like that be a helpful facilitator in connected virtual worlds?
I think payments need to work well. And as a company, we're taking a few approaches towards payments. I've talked publicly about how we're trying to build payments directly into WhatsApp and Messenger, and we're doing the country-by-country thing where we make it work with every country's banking system and payment system. And that's just, it's like a ground war, right, you like, you do specific work in each place. But I think that's going to be very easy for people to plug into.
We also just wanted to take an approach that was maybe more rethinking the whole payments approach from scratch. And that's a little bit more what we're trying to do with Libra. We serve a lot of people around the world, and a lot of them live in countries where there isn't a stable currency.
Having something that is more stable, that's based on a basket of different currencies from other places, that has rethought a lot of the payments stack, because it's based on ... some of the newer crypto technologies, could just make it so people can transfer money between countries more efficiently, and bring a lot of people around the world who haven't been able to be part of the modern financial system, into it. It's probably more where we're going with with Libra. But certainly, if that ends up working well, it'll open up a lot of opportunities down the road. I just think the most immediate opportunities are probably in developing countries, especially. But payments, I think, is obviously going to be critical.