What’s Next for AI Hearing Aids? Hearing in Your Own Friend Bubble
Semantic hearing could let users create acoustic bubbles, isolate chosen voices, and control individual sounds in their environment.)
:format(webp))
I want to live in a bubble. Well, not all of the time. But when I’m in a crowded restaurant with friends, it would be great to be able to create an acoustic bubble that surrounds our table. Inside the bubble, we could hear each other clearly because all the noise from the outside would be silenced.
I would also like to be able to train my aids to target and prioritize certain sounds such as my wife’s voice or the doorbell.
Bubbles and targeted hearing are no longer wishful thinking. They already exist, at least in the lab of Shyam Gollakota, PhD, and his remarkable team at the University of Washington in Seattle. These revolutionary technologies will one day be in your hearing aids, and that day may be arriving sooner than you might think.
:format(webp))
Dr. Stefan Launer, Sonova’s VP of audiology and health innovation, notes two important milestones in the evolution of hearing aids. “We made the big leap from analog to digital some 30 years ago, and that opened a new door to innovations that wouldn’t have been possible with analog. It’s the same now. With the introduction of AI and with deep neural networks, we have made another big leap and we can drive innovation forward.”
:format(webp))
The first wave of the AI revolution is here now. Today, most higher-end hearing aids use deep neural networks (DNN) to identify and extract speech in noisy environments. For anyone with hearing loss, it is remarkable technology that goes a long way toward solving the “cocktail party problem.”
“These DNN algorithms are really fantastic,” says Launer, “but they also have their limits because they extract the strongest speech signal. In other words, they tend to process the loudest voice, but that voice may not be the person you are talking with.”
In a room full of competing voices, the AI and your brain can struggle trying to ignore that loud man sitting 20 feet away to let you focus on the person you are talking with. But what if your hearing aids could solve that issue by creating a bubble around you?
Hearing Bubbles
Dr. Gollakota and his team have developed algorithms that can do just that. Using multiple microphones, the DNN can gauge the distance to the source of a sound. If you choose a 3-meter distance, for example, every noise beyond that will be silenced. Only the sounds within the bubble would be passed on to your ears.
“What that means is that in real time I can pick the people who are within my bubble and remove the people who are outside it,” says Gollakota. “And this is transformational, I think, for both people with hearing loss and for people who have normal hearing. And that is what we actually achieved and why I am super excited about this.”
“The bubble really makes people feel like they’re living in science fiction.”
You can see and hear for yourself in this demo video provided by Gollakota’s team.
So far, this amazing technology is confined to Gollakota’s lab—and it requires custom-built headphones, mics, and other hardware. But will it work in hearing aids? Gollakota replies with a smile, “I don’t want to scoop my own work.”
But rest assured, bubbles are coming to hearing aids.
As Arthur C. Clarke famously put it, “Any sufficiently advanced technology is indistinguishable from magic.” And bubbles aren’t the only magic being conjured up now.
Targeted Hearing
These emerging AI systems and DNNs are trainable. For example, they can also be taught to identify and extract certain sounds. In effect, you will be able to teach it to prioritize the sounds you have targeted. That could be your child’s voice, an oven timer, a car horn, or virtually any sound you consider important.
“The trick we have proposed is what we call ‘look once to hear’,” explains Gollakota, “Basically, if I want to focus on you, I’m going to look at and listen to you for 3 seconds or 5 seconds, then click a button on my hearing aids.”
“Now that it knows what you sound like, the neural network is just going to extract your voice. I don’t need to look at you anymore. It knows what you sound like, even though other people around you are talking; it can just keep listening to you while removing everyone else.”
Of course, this would be a boon for anyone stuck in the cacophony of a crowded room or who finds themself at the other end of the house when the doorbell rings.
Launer offers another powerful example of what targeted hearing can do. “For a child with a hearing impairment, it could be trained to recognize and emphasize the voice of their teacher.”
Be Your Own Audio Engineer
Anyone who has been to a concert has probably spotted an audio engineer at their mixing board working a line of slide controls. They use these to adjust the volume of the band’s microphones and various instruments.
What if you could do the same thing in your daily life? And, in effect, become your own audio engineer? That’s the concept detailed in a recent paper by Gollakota and his team. It’s a concept they call, appropriately enough, “Aurchestra’.
“What it does is give the user fine-grained volume control of the individual sounds of their acoustic environment. Say you’re sitting on a beach and you want to hear the sound of the waves. But there’s a barking dog, someone is playing a guitar a little too loudly, and people are chatting nearby. With a phone app, you will be able to lower the volume of each of those intruding sounds and lift the volume of the waves.”
“We have demonstrated that we can do this all in real time.”
Semantic Hearing
Professor Gollakota and his colleagues coined the term “semantic hearing” to describe this emerging technology in their landmark 2023 paper, Semantic Hearing: Programming Acoustic Scenes with Binaural Hearables. The word semantic is defined as “relating to meaning”. It was chosen because these AI systems are based on the meaning and classification of sounds.
To take this technology from the lab to the real world, Gollakota has co-founded Hearvana AI. “Our goal is to bring this technology to not just hearing aids, but also to devices like earbuds and smart glasses as well. We really want to transform how people hear.”
Hearing aid manufacturers are watching all of this closely, and they also have teams of researchers working on similar technologies. It’s only a matter of time before Gollakota’s magic algorithms or others like them appear in hearing aids. The big question for people with hearing loss like me (and others who just think this would be cool) is when?
“I think the only constraint right now is having the right hardware to run our algorithms on. So far only a few companies have the necessary AI accelerator chips. That limits how fast we can deploy our algorithms, but I would say that probably we will see them appear in the next 2 to 3 years.”
Sonova’s Launer offers a similar timeline, “Over the course of the next, say, 2 to 5 years, we will see lots of specific algorithms introduced, probably incrementally.”
When it comes to AI, we ain’t heard nothing yet.
Digby Cook
ContributorDigby Cook is a veteran journalist with a wide range of experience in television news, documentaries, and newspapers. As a person with severe to profound hearing loss, his interest in the science of hearing is both professional and personal.