Facebook's moonshots: Making brains type and skin hear

The social network finally unveils what its secretive Building 8 hardware lab has been working on for the past year. Communication might never be the same.

Richard Nieva Former senior reporter
Richard Nieva was a senior reporter for CNET News, focusing on Google and Yahoo. He previously worked for PandoDaily and Fortune Magazine, and his writing has appeared in The New York Times, on CNNMoney.com and on CJR.org.
Richard Nieva
7 min read

It looks like just another beige office park building next to a dental office in Menlo Park, California. Yet Building 8, across the street from Facebook 's main campus, houses the social network's biggest bets on out-there products.

The tech industry has a term for what people inside Building 8 work on: moonshots. Think potentially groundbreaking projects that could reshape Facebook's long-term future and even how all of us communicate.

CEO Mark Zuckerberg unveiled Building 8 (named for the number of letters in Facebook) at last year's F8 developer conference. He also revealed he'd recruited Regina Dugan from Google's Advanced Technology and Projects (ATAP) group to head Facebook's skunkworks efforts, as part of Zuckerberg's 10-year strategic plan.

Watch this: Facebook's Building 8 lab working on moonshots for mind and body

Since then, Facebook has given tantalizing hints about Building 8's mission, saying only that it's focused on "seemingly impossible" hardware in augmented and virtual reality, artificial intelligence, connectivity and "other important breakthrough areas," with "clear objectives for shipping products at scale." The one thing we knew for sure: The company had been amassing a dream team of hardware veterans from the likes of Apple, Motorola, Google and other industry heavyweights.

Some of that secrecy faded Wednesday, when the group unveiled its first two projects: a "brain-to-computer interface" that would allow us to send thoughts straight to a computer, and technology to "hear" or absorb language through vibrations on our skin.


Regina Dugan, who previously ran Google's Advanced Technology and Projects group, now heads Facebook's skunkworks efforts.

James Martin/CNET

"If I'm doing my job well, we should deliver things people didn't know to ask for," Dugan -- who previously headed Darpa, the Defense Department's famed tech arm -- tells me Monday from a working space on Facebook's campus. "There's the risk of failure. But that's precisely the price you pay for the honor of working on something new."

You might not expect off-the-wall hardware products from a social network known for the Like button, status updates and baby photos shared among nearly 2 billion people each month. Yet the company already has its hand in everything from virtual reality headsets to massive drones meant to blanket the earth with Wi-Fi signals. It can't afford to let up.

That's because Silicon Valley is on a constant quest to find the next big thing. Like Alphabet, Apple, Amazon and other major tech players, Facebook's future depends on ambitious moonshots that could open new business opportunities.

"Even if the technology is out there and never gets turned into a product, the R&D work that goes into it can often be used for things being created in the real world right now," says Jan Dawson, chief analyst at Jackdaw Research. Just as important, moonshots "attract and retain smart employees," says Dawson.

In the Valley, that's a competitive advantage in its own right.

"OK, computer"

Building 8's brain-to-computer project comes straight from the top. "One day, I believe we'll be able to send full, rich thoughts to each other directly using technology," Zuckerberg said in a Q&A on Facebook two years ago.

"You'll just be able to think of something and your friends will immediately be able to experience it too, if you'd like," he said.


Mark Zuckerberg kicks off the 2017 F8 developer conference in San Jose, California.

James Martin/CNET

Now it's up to Dugan and Mark Chevillet, the initiative's technical lead, to make the technology practical. And not in some nebulous future, either. Like Google's ATAP -- whose projects include fabric with built-in sensors and radar interfaces you could control with gestures -- Building 8's efforts all have two-year deadlines. That's possible in part because of a collaboration deal it signed in December with 17 universities, taking months out of the time it would normally take to ramp up a project, Dugan said in a Facebook post at the time.

So before the two-year period lapses, Building 8 hopes to develop a system that lets us "type" 100 words a minute on a computer, just by thinking what we want to say. That's about five times faster than people can type on smartphones, and quicker than most of us can type on a computer.

"Imagine what would be possible if you could type directly from your brain," Dugan says.

Building 8's technology works by using sensors that tap into the speech center of your brain -- the part that's active when you've thought of something to say, formed the words and are getting ready to speak them. The technology would then feed those signals to a computer, kind of like how speech-to-text software works. But instead of inputting an audio feed, you're inputting your neural activity.

When I asked if this project is particularly close to Zuckerberg's heart, Dugan tells me he's enthusiastic. (Facebook declined to make Zuckerberg available for this story.)


Mark Chevillet and Regina Dugan stand outside Facebook's secretive moonshots lab, F8, across the street from the company's main campus in Menlo Park, California.

James Martin/CNET

The challenge for Facebook is that many of the brain-to-computer projects rely on micro-electrodes implanted into the brain. Facebook is only working on "noninvasive" technology, with signals transferred through wearable sensors.

For sure, universities and researchers have been developing brain-computer interfaces for decades, typically aimed at helping stroke victims, people with ALS and paraplegics with spinal cord injuries regain basic communication or motor skills.

Three years ago, for example, a paraplegic man used a mind-controlled exoskeleton to kick off the World Cup in Brazil. A university collaboration called BrainGate has developed a system that lets people control a computer cursor by thinking about the movement of their own paralyzed hand and arm. And the BioSense lab at the University of California at Berkeley's School of Information is working on identifying people through their brainwaves, in what could become the ultimate personal ID protector.

Still, even researchers worry about the potential ethical complications of this sort of work. Some fear that a government could use it to monitor thoughts or to amp up an interrogation. There's also the fear that data could fall into the wrong hands.

"One of the challenges is we still don't know what the [brain] data means," says Nick Merrill, a UC Berkeley doctoral candidate working with the BioSense lab. "You have to consider the very consequential privacy problems that could happen if the data were leaked and mishandled."

Chevillet, a former program manager of applied neuroscience at Johns Hopkins University, counters that Building 8's tech isn't trying to read every random thought in your head. It's just tapping into the part of the brain for already formed speech. "These are things you want to say," he says. It's not concerned with other thoughts. "That's your stuff."

Dugan compares it to -- what else for a Facebook executive -- sharing photos. "You take many photos but choose to only share some of them," she says.

Now what?

Why speak directly from your brain? Facebook says your brain can process a terabyte of data every second, which is about the same as streaming 40 high-definition movies. The speed of thought is much faster than speaking, which functions more like a "1980s dial-up modem," Dugan says.

In other words, the speed matters.

The technology could be game-changing for something like augmented reality glasses, suggests Dugan. Simple "yes" and "no" buttons in front of your eyes could be helpful in a number of situations. For instance, answering "yes" to the question "Do you want to see in the dark?" might activate a night-vision mode. All you'd have to do is think of moving a cursor to the "yes" button, and visualize clicking it.

Facebook has already made a big bet on augmented reality. On Tuesday, the social network unveiled a platform that lets software developers create digital graphics that are overlaid on real-world images.

For the brain-to-computer project, Facebook is partnering with a team of more than 60 engineers and scientists from universities including University of California at San Francisco and Johns Hopkins to develop the technology.

Even though some universities are using the technology to develop mind-controlled limbs, Building 8 isn't taking that route. Chevillet tells me Facebook isn't working on prosthetics because the company's mission has more to do with communication.

"We're just focused on getting people to communicate better," adds Dugan.

And while Zuckerberg's vision of transmitting "full, rich thoughts" is further out in the future, the people in Building 8 say it's possible with this kind of research.

I hear you

Dugan's Building 8 team is also working on a project that could let you "hear" and decipher words through vibrations on your skin.

The concept is similar to braille, in which tiny bumps represent letters and other elements of language. But instead of running your hand over those bumps, you'd feel frequencies in different patterns on your forearm from a sleeve worn on your wrist. Each pattern represents a different word. The hope is that, in practice, the deaf could communicate quickly.


For Facebook's test, a subject was taught nine "vocabulary" words through a series of vibrations on the skin.


To try out the method, the company developed tests in which someone is taught nine "vocabulary" words, like sphere, cone, black and blue -- and assigned those words different vibrations. That person could then decipher words and phrases, like "blue sphere," based on the vibrations.

The project isn't as far along as the brain-computer interface initiative. This one doesn't yet have an end goal for the two-year deadline, Dugan says.

Dugan says the aim of both projects -- and everything else at Building 8 -- is to take us beyond the phone as the primary communication tool.

"This thing has allowed us to connect with people far away, but at the expense of people sitting next to us," she says, holding up her own phone.

"It's the first time in a long time we've been able to crawl out of this little black box and be in the room again."

CNET's Laura Hautala and Roger Cheng contributed to this report.

Tech Enabled: CNET chronicles tech's role in providing new kinds of accessibility.

Logging Out: Welcome to the crossroads of online line and the afterlife.