Hearing aids are getting smarter. Think AI, health tracking

This is part of CNET's "Tech Enabled" series about the role technology plays in helping the disability community.

When Shannon Conn puts her hearing aids in her ears in the morning, a few things happen.

The lights in her bathroom turn on.

The coffee maker starts brewing.  

When someone rings the doorbell, the chime streams straight into her ear.

Conn, a 43-year-old special education advocate from College Grove, Tennessee, wears the Oticon Opn, which features the ability to link up with other connected devices. She's put that capability to good use, connecting her Opn to her smart home and setting up commands that allow her hearing aids to trigger her house's morning routine.

"I can't live without them," Conn says. "I haven't said that in a really long time about anything."

The Oticon hearing aids represents a growing crop of hearing assistance devices that are benefiting from an injection of technology. Helping is the growing interest in augmented hearing, or so-called hearables -- think wearables for your ears -- that's leading to new features that can be applied to a wide range of devices. The result: smarter and more accessible devices for people dealing with hearing loss.

It's a large crowd. Roughly 48 million Americans, or a fifth of the country, deal with some degree of hearing loss, according to the Hearing Loss Association of America. It's not just age-related hearing loss -- it affects 15 percent of school-age children as well. 

Traditionally, hearing aids required a medical prescription and could be pricey. (Insurance typically doesn't cover their expense.) The high cost and stigma meant that not all people with hearing loss actually wear hearing aids.

"People miss out when they have uncorrected hearing, and they don't even know what they're missing," says Chuck Bergen, a 64-year-old hearing aid user from Apple Valley, Minnesota.

A new law passed last year, the Over the Counter Hearing Aid Act, allows adults with mild to moderate hearing loss to buy cheaper devices without having to see a hearing professional. It also directs the US Food and Drug Administration to designate a new category of hearing aid, which means traditional headphone makers and technology companies can start offering devices for people with hearing loss.

Watch this: At CES 2018, cute robots, smart hearing aids and KodakCoin (The 3:59, Ep. 337)

20:19

"We think we could reach a lot of people with the benefits of our technology," says John Roselli, head of Bose's wellness team.

That's why Bose, along with other notable tech players like Samsung's Harman, are joining traditional hearing aid makers like Oticon in this area. The aim isn't just about improving sound. New hearing devices could pack in sensors for things like health tracking and fall detection and smarts for language translation.  

Connected hearing aids

It was two years ago when Conn tried the Oticon Opn as part of a trial. She vividly remembers turning them on and hearing her daughter speak.

"I just burst out in tears," she says.

Conn also got Opn as rechargeable hearing aids for her now 17-year-old daughter, who developed the same condition --  neural hearing loss -- and started to see her grades drop. With Opn, she was able to rest her iPhone near the teacher and have the lecture stream straight into her ear through her hearing aids, a capability found in hearing aid devices under Apple's "Made for iPhone" program.

Oticon has taken advantage of the Opn's connected capabilities in fun ways, including streaming a live performance from rock band Styx across the nation to its users. At CES 2018, the company showed off a hearing fitness app that serves as an exercise tracker for your ears.

The app uses the hearing aid's audio sensors, which can work with other fitness trackers like a Fitbit or Apple Watch to offer advice to users on how to protect or improve their hearing. For instance, the app will give users goals on how long they should wear the Opn, or if it knows you're dealing with a crowded atmosphere (like a big trade show like CES), it will encourage you to keep using the device.

"People start to disengage if they don't have good hearing," Sheena Oliver, vice president of marketing for Oticon, said in January of the potential frustration with hearing aids.

Next up: AI, motion sensors and more

Starkey, another Made for iPhone partner, has long offered the ability for users to stream audio directly to their hearing aids.

But Starkey's plans are much bigger.

The hearing aid maker is working on inertial sensors like accelerometers and gyroscopes. The first such sensor will show up in a hearing aid in the middle of the year and will allow it to track physical activity and eventually detect falls, which are a major cause of death for seniors.

The sensors, along with the hearing aid's microphone and digital signal processor, also will allow users to tap the hearing aid to awaken a smart assistant and nod or shake their head to respond to inquiries.

Starkey will use artificial intelligence to classify normal activities of the hearing aid user and send an alert to a loved one if something seems out of place, says Achin Bhowmik, Starkey's chief technology officer.

"Using AI to turn these into real-time health monitoring devices is an enormous opportunity for the hearing aid to transform itself," he says.

By the end of this year, Starkey also will be bringing natural language translation capabilities to its hearing aids. And it plans to eventually integrate optical sensing to detect heart rate and oxygen saturation levels in the blood. It's also looking at ways to noninvasively monitor the glucose levels in a user's blood and include a thermometer to detect body temperature.

"We want to turn the hearing aid from a single-purpose device to a multipurpose device," Bhowmik says. "We want to go from not many people wanting to use it to many people wanting to use it."

Control what you hear

Bose, one of the world's biggest wireless headphone makers, has been working on Hearphones, which can "augment your sense of hearing by letting you control what you hear," says Kathy Krisch, director of Bose's Hear business. "Their purpose for being is to hear and converse better in noisy places."

Noise cancellation drowns out background sounds, while directional microphones give you the ability to focus in on what's directly in front of you or better hear people in a group. You can also control the volume and tonality, all through the headphones' companion smartphone app.

Watch this: Bose AR aims for augmented audio instead of visuals

01:14

Samsung's Harman business -- which owns the JBL, Harman Kardon and AKG audio lines -- offers something similar. Technology called "ambient aware" on its Everest JBL headphone line lets you better hear conversations in noisy environments. For instance, when a flight attendant stops to talk to you, you can tap a button that lets "the audio signals closely aligned to speech come through," says Kevin Hague, vice president of technology strategy at Harman.

Then there's the flipside of hearing better -- the desire to hear nothing, like while you're sleeping. Bose in November launched a campaign on Indiegogo to build Noise-Masking Sleepbuds, earbuds that play white noise while you sleep, masking a noisy environment or snoring spouse. A prototype of the earbuds have emerged on eBay for $800.

"We know a lot about the ear," says John Roselli, head of Bose's wellness team. "The key here was to get something small enough that it could fit in your ear very comfortably all night long."

Early supporters on the Indiegogo crowdfunding site started getting their earbuds last month. Bose plans to start selling them in North America later this year, followed by a full global rollout in the fall.

Smarter hearables

Going into this year, an Ericsson study said one of the hottest tech trends was augmented hearing, or the idea that wireless headphones will get smaller, last longer and do more things.

That could include the ability to overlay smarts and signals on top of what you're hearing in the real world. Bose's augmented reality sunglasses feature speakers built into the arms of the glasses that speak information out loud. They can tell you about what you're seeing, help you learn a foreign language and automatically bring up music based on whether you're in the office or the gym.

Harman has been working on its own version of audio augmented reality. "You can have cues that are actually injected into the audio stream," Hague said. If a car is speeding down the street and you're walking across it, your headphones could beep in the direction of the car. If there's a jackhammer nearby, you could program your headphones to always remove that sound while keeping all other noises around you, Hague said.

Eventually, Hague believes smart earbuds, possibly paired with smart glasses, could replace the traditional smartphone.

"Hearables go with you and are with you all the time," he said. "You could have ubiquitous computing as you're going along, the same thing a cell phone today gives you."

For Conn, her first step into augmented hearing tech meant visiting a world she wasn't comfortable with. After all, how many people are familiar with If This Then That instructions? But as she grew comfortable with the tech, she learned to appreciate the little things like quiet murmurings from her children.  

"I can't say how much it drastically changed my life," she said.

First published April 5, 5 a.m. PT. 
Update, 11:33 a.m. PT: To note that Bose's prototype Sleepbuds showed up on eBay. 

Tech Enabled: CNET chronicles tech's role in providing new kinds of accessibility.

Hands-on with a robot duck:  CNET got to spend time with My Special Aflac Duck which was designed to help kids suffering from cancer.