X

Google's new Nest Hub tracks your sleep with Soli -- no camera required. Here's how it works

The technology previously known as "Project Soli" is a presence-sensing radar chip that avoids many of the privacy pitfalls of front-facing cameras.

Dale Smith Former Associate Writer
Dale Smith is a former Associate Writer on the How-To team at CNET.
Dale Smith
5 min read
screen-shot-2021-03-16-at-1-02-44-pm.png
Google

Like the two generations of smart screens that came before it, Google's  new $100 Nest Hub smart display combines smart home control à la Google Assistant with one of the most gorgeous screens on the market today, all without the benefit -- or the security and privacy concerns -- of a front-facing camera. Unlike its predecessors, however, the new Nest Hub can respond to physical gestures like hand waves, same as its bigger, camera-equipped sibling, the Nest Hub Max. But it gets even better.

The new Nest Hub can also track your sleep cycle, noting every fidget, barrel-roll and midnight snack run you make, then analyzing and aggregating that data into a cohesive sleep report. There are no battery-powered wearables required, and again, no cameras involved, whatsoever. Instead, the next-level tech behind these new features is a miniature radar chip Google has dubbed "Soli."

Soli, whose market debut in 2019's Pixel 4 met a rather lukewarm reception, allows the Nest Hub to sense its surroundings even better than a camera. That's because it's not encumbered by pesky visual problems that otherwise beleaguer camera lenses -- wonky ambient light, obstacles (like blankets) in the line of sight, etc.

Here we'll take a deeper dive into everything we know about Google's Soli -- what it does, how it works and where it comes from, plus what you can expect if you plan to buy an updated Nest Hub when it hits shelves March 30.

What is sleep tracking and why is this a big deal?

Despite the popularity of wearables like Apple Watch and Fitbit fitness trackers, sleep tracking has yet to really catch on. That could be because even the most energy efficient activity trackers need to be recharged at some point, and what better time to do that than when you sleep? That said, few behaviors have as profound and widespread an effect on everything from our energy level to our mood as the quality of our sleep.

nest-hub-lifestyle-your-evening-with-sleep-sensing

The updated Google Nest Hub can track your sleep activity using radar. 

Google

To track your activity during the night, the new Nest Hub uses the Soli chip to detect nearby motion and movement, then analyzes the input to determine what you're up to -- whether you're climbing into bed, sleeping soundly or waking up. It also uses microphones, temperature sensors and light sensors to record any coughing, snoring or other disturbances, like changes to the room's temperature or an errant light affecting your sleep. Google says it's tested Soli's analysis against established, clinical-grade sleep trackers and reports that Soli was at least as good if not better than other methods of tracking sleep activity. 

How does the new Nest Hub track your sleep activity?

In a word: radar. The Soli chip is basically just a very small radar that detects the room's electromagnetic field. As different objects move through that field, they change the signals the radar receives. The chip then processes those anomalies to figure out what's happening -- whether someone just stood up, or rolled over, or wiggled their pinkie toe.  

What information can you track with the new Nest Hub?

We won't have a complete picture until we can test the updated Nest Hub ourselves, but Google has stressed that the new Sleep Sensing feature tracks what it considers to be the three most important sleep metrics: duration of sleep, consistency of sleep schedule and how restful the sleep you're getting actually is. 

nest-hub-lifestyle-nest-cam-feed

The Nest Hub avoids a number of privacy pitfalls by omitting a camera.

Google

Google says sleep tracking on the Nest Hub also detects coughing and snoring episodes, as well as your breathing rate while you sleep. All of this info is accessible either directly on the Nest Hub or using the Google Fit app on Android and iOS , which Google says will also provide actionable, personalized recommendations based on individuals' sleep metrics. 

Google doubles down on privacy with on-device processing

The Nest Hub is our favorite smart display overall not just because it offers the most spectacular screen among the competition (thanks to the way it adjusts color temperature based on the quality of ambient light), but also for what it doesn't offer -- a camera.

Although several smart displays by Amazon have physical camera covers, as does Google's own Nest Hub Max, there's something especially reassuring about a device that has no camera to begin with. It's easier to put a Nest Hub in your bedroom or bathroom, knowing there's no way anyone you know or, heaven forbid, someone you don't, can wind up seeing things you'd rather they not see -- accidentally or otherwise.

nest-hub-lifestyle-kitchen

The updated Nest Hub goes on sale later this spring, but you can preorder it starting today.

Google

On top of that, although the updated Nest Hub uses both audio and presence sensing to help determine your nighttime activity, Google says both the audio feed and the raw Soli data are processed on-device, with only the overall "sleep event data" making its way to Google's servers. Additionally, users can reportedly pause sleep tracking or delete their sleep data whenever they'd like.

Soli was originally used to distinguish apples from oranges (really!)

Originally announced at Google I/O in 2015, Project Soli was touted as a radar sensor capable of detecting sub-millimeter motions at higher speeds and with greater accuracy than cameras could capture. One of the first implementations of the microchip was a machine called RadarCat, which researchers at St. Andrews University in Scotland developed to distinguish between three-dimensional objects -- apples and oranges, literally.

The failed Pixel 4 and 4 XL introduced Soli to the mainstream, briefly

The Soli chip didn't appear in a mass-market device until 2019, when it was featured in the Pixel 4. There, it worked in tandem with the phone's front-facing camera, which didn't need to be turned on, to recognize gestures -- even through clothes or in the dark. For example, users could swipe the air in front of the phone to skip Spotify tracks and the phone would automatically detect their presence and light up the screen.

This graph illustrates what Google's Soli chip "sees" while you're sleeping.

Google

Unfortunately, however, the Pixel 4's relatively underwhelming stats plus a price tag generally perceived as too high (given its hardware) led to the Pixel 4's discontinuation not even a full year after launch, replaced by the less expensive Pixel 4A, which did not include a Soli chip. Neither, for that matter, did the Pixel 5, which debuted in December 2020.

What gestures can you use with the new Nest Hub?

Google says the new Nest Hub supports what it calls Quick Gestures, which likely are similar to what you could do with the Pixel 4 -- e.g., wave your hand in front of the device to silence an alarm, wave your hand to the right to skip forward or left to skip backward during audio playback, tap the air above the display to pause or unpause, etc.

Will sleep tracking cost money? How much?

It appears so, but how much it will cost or when Google will start charging is still up in the air. Google says the feature will be available as a "free preview... until next year."