Facebook's new Portal smart displays: Who's listening and what's happening to your data?
The tech giant compromised your privacy and broke your trust. Now, it has a new set of "Smart Cameras" and always-listening microphones for your living room.
Ry CristSenior Editor / Reviews - Labs
Originally hailing from Troy, Ohio, Ry Crist is a writer, a text-based adventure connoisseur, a lover of terrible movies and an enthusiastic yet mediocre cook. A CNET editor since 2013, Ry's beats include smart home tech, lighting, appliances, broadband and home networking.
ExpertiseSmart home technology and wireless connectivityCredentials
10 years product testing experience with the CNET Home team
Each comes with an AI-powered "Smart Camera" that can track you as you move about in the frame during a call, and each comes with the same microphones for voice-activated controls as the originals. You can say, "Hey, Portal," to wake it up and make a video call or any other number of functions, and you can say, "Alexa," to access the full capabilities of Amazon's digital assistant, too.
"For more than two years, Facebook's public disclosures presented the risk of misuse of user data as merely hypothetical when Facebook knew that a third-party developer had actually misused Facebook user data," the SEC said.
In other words, anyone considering bringing into their home a Facebook-branded device with always-listening microphones and an AI camera really ought to consider Facebook's privacy practices, first. So, let's do that.
Is Facebook eavesdropping? (Is anybody not eavesdropping?)
Facebook's Portal devices use the same sort of microphones as other smart displays, which means they're always listening for the wake words (in this case, "Hey, Portal"). When you say the wake words, the device activates and begins recording the audio of whatever you say next. From there, the device sends that audio snippet to Facebook's cloud in order to figure out how to respond.
That's the same way other voice-activated smart assistant gadgets work, including Amazon's Alexa devices, Google Home smart speakers and Google Nest smart displays, and assistants like Apple's Siri and Microsoft's Cortana, too. It's also how those voice assistants work when you use them on your phone.
The logical follow-up: What do these companies do with those audio snippets, some of which may contain background conversations not intended to be shared? Is anybody listening to them?
As it turns out, the answer is yes. Both Amazon and Google admitted earlier this year that they had hired contractors to listen to recordings like that in order to help refine their respective assistants' capabilities. Apple and Microsoft soon followed suit. Human review of user audio was a standard practice across the board for all of them.
And Facebook? The company admitted that it, too, paid contractors to transcribe Messenger calls for the purpose of improving the service. After a brief pause, that practice is back on as these new Portal devices launch, though now you have the option of saying, "no thanks."
"If you have storage enabled, which is the default, then those [audio snippets] may be reviewed by humans, by a team of trained reviewers to improve voice services overall," a Facebook spokesperson said. "Those, of course, can be deleted individually, or you can also just disable storage entirely, and then they can't be reviewed or stored."
That's a good option -- but many might prefer to have human review off by default, and available only for those that opt in. So far, Apple is the only company that claims to take that approach.
In most cases, companies like Facebook that have acknowledged human review of user audio say that they hire outside contractors for the job.
"There are vendors who are really specialized at this and who are really good at this," a Facebook spokesperson explained. "Getting the right people to help review voice transcripts makes the service a lot better. It makes it more inclusive."
But Facebook adds that company employees may listen to user audio, too.
"There are specialists out there who are vendors," the spokesperson added. "We're happy to use them. They're under very strict protocols in terms of how we engage with them. But we also use a mix of employees when it's appropriate."
And what constitutes "appropriate?" I asked Facebook, but haven't received an answer yet. I'll update this space if that changes.
Can I cover up that camera?
Yes. You can cover each Portal's camera with a physical shutter when it isn't in use.
Good for Facebook -- that's better than the Google Nest Hub Max and the Amazon Echo Show, each of which skipped the shutter in favor of a digital kill switch that disables the camera electronically without covering it. Amazon at least seemed to realize that consumers were more comfortable with a shutter when it added one to its most recent smart display, the mini Echo Show 5.
As for the Portal, Facebook adds that the camera's movement-tracking features and the microphone's audio enhancement capabilities are all processed locally on each Portal device, and never sent to Facebook's cloud. Meanwhile, Messenger calls are encrypted in transit, and WhatsApp calls are encrypted end-to-end. Facebook stresses that it does not listen to, view or keep the contents of your Portal video calls.
What about when I'm not using it?
All of that is good -- but unless you've switched them off or closed the shutter, the camera and microphone are always on. Are they gathering any other data on you?
"When Portal's camera and microphone are on, which you can control, we collect camera and audio information," reads Facebook's supplemental data policy for its Portal devices. It goes on to describe the data that's collected whenever you make a call, which Facebook says is the same data collected from other Messenger-enabled devices during calls.
"When using Portal's camera or microphone to make a call, we collect technical information about your call, such as volume level, number of bytes received, or frame resolution."
Smart displays let Amazon, Facebook, Google show you answers to your questions
That sounds innocuous enough, and in the latter version Facebook specifies a bit more to characterize the data as purely technical information, but I still default to skepticism when a company uses terms like "can include" or "such as" to detail the various types of data it collects. After all, both phrases connote a nonexhaustive list, which raises an obvious follow-up question: Can that list include anything else?
And what about when you're not making a call? If the device is just sitting there plugged in on your kitchen counter, is it quietly taking notes on your day-to-day routine?
I asked Facebook these questions and several others about the data policy, but have not received answers. I'll update this space if that changes.
In the interest of transparency, I've pasted the text from Facebook's data policy for Portal devices into a Google Doc and annotated it with the exact questions I asked the company with respect to each section. You can see that document, along with what Facebook did and did not answer, by clicking here. As of writing this, the company hasn't answered any of my questions, but I'll add those answers to each annotation as I receive them.
But the cameras in Facebook's Portal devices don't use facial recognition technology at all, the company says.
"Right now we understand faces, but we don't understand your face," a Facebook spokesperson explained. "So, we can look at the features like eyes and mouth and things like that. It helps us place things like when you put on an AR mask, knowing how to overlay it on top of your face."
That distinguishes the Portal devices from the Google Nest Hub Max, which uses facial recognition in order to show individual users personalized bits of information from their Google accounts. Google calls the feature Face Match, and initially told us that all of the processing for it happens locally, on each user's device. That's true in real time, when you're using the feature, but the product's fine print makes it clear that your face data may occasionally get sent to Google's cloud so it can ensure the feature works in multiuser households, and so it can test new features before pushing them to users' devices.
"We occasionally use the images you provide during setup to generate a face model in the cloud for a couple of reasons, all related to improving your product experience specifically on Nest Hub Max, and motivated by the fact that we have more computing power available in the cloud," Google said.
"This only uses the images originally taken on the phone during the setup process," a Google spokesperson emphasized. "Any processing on our servers is temporary, and all face models processed in the cloud are permanently discarded. The only feature that has access to the enrollment images and subsequent face data is Face Match -- this data is not used for anything apart from this feature on the device.
Is this thing going to affect what ads I see?
Yes. Whenever you use Facebook or a Facebook device like the Portal, part of the fine print is that you're agreeing to share data with the company, and that the company may use that data to target ads at you that may be relevant to your interests.
"We collect the same information that other Facebook properties collect," a Facebook spokesperson said. "So, when you're using Facebook Watch, for example, we'll know what you're watching and we'll use that to prioritize what things to show you. And there are things that will be used for our ad system there."
The Portal's primary use case is video calls -- do the calls you make impact the ads that make it into your feed?
"The best way to think of this honestly is it's like a mobile phone," a Facebook spokesperson said. "And it's running Messenger, it's running Facebook Watch, and it's running WhatsApp. And so it's using the same services in the same way that your phone uses those services. And so when you're using Messenger, we have an awareness of, you know, who the call was initiated by and who received the call and how long the call was. And you can imagine it informing advertising on Facebook. Like, for example, 'this is a person who makes video calls.'
"That's relatively unlikely," the spokesperson added. "The amount of data we're generating here is very, very small. It's certainly not material. It's certainly not the point of the product. But it's also something that could happen."
In the case of the gang members' voice calls, Facebook argued that it couldn't carry out the FBI's demands because Messenger has end-to-end encryption, meaning that it'd have to completely rewrite the app to do so. In the end, the Justice Department wasn't able to make Facebook break its own encryption, even after reportedly trying to hold the company in contempt of court.