A new Seattle startup is taking speech translation to a whole new dimension.
Hearvana, founded this year by University of Washington computer science researchers, translates speech from multiple speakers in real time — while preserving their spatial positions and vocal identities.
The idea is that you could be in a noisy setting surrounded by people speaking different languages, and still understand what’s being said.
“Our system transforms your entire auditory space into your own language, preserving the unique voices of speakers, their 3D spatial positions, and doing it all in real time, even as people move,” Shyam Gollakota, co-founder of Hearvana, wrote on LinkedIn.
Gollakota, a leading inventor who heads up the UW’s Mobile Intelligence Lab, told GeekWire that the company is “creating AI breakthroughs that are shaping the future of sound.”
“Our AI algorithms enable on-device superhuman hearing capabilities and will be part of billions of earbuds, hearing aids and smartphones,” he said. “It is an exciting time.”
Hearvana says its translation features go beyond what’s available on existing wearables such as Meta’s Ray-Ban glasses.
The startup’s underlying tech — dubbed “spatial speech translation” — was detailed in a research paper published last month. The study tested translation from French, German, and Spanish into English. It outperformed baseline systems in user studies and participants praised its translation accuracy, speaker fidelity, and spatial realism.
Hearvana is being incubated at the AI2 Incubator in Seattle.
“Hearvana is my favorite kind of startup as it addresses a familiar pain point — we all struggle to hear in noisy settings like a restaurant or a party — with deep AI technology,” said Oren Etzioni, technical director and partner at AI2 Incubator.
Etzioni, the former CEO of the Allen Institute for AI, called Gollakota a “world-class computer scientist.”
Gollakota has a track record of turning research into startups.
He previously co-founded Sound Life Sciences, a UW spinout that developed an app to monitor breathing that was acquired by Google in 2022.
He’s also the co-founder of Wavely Diagnostics, which uses a smartphone app to detect ear infections.
Gollakota last year won a $100,000 award as one of six researchers honored as part of this year’s Infosys Prize.
His research focuses on wireless tech, battery-free devices, WiFi sensing and imaging, medical diagnostics via smartphones, and more.
Malek Itani, a research assistant and PhD student at the UW’s computer science school, is a co-founder of Hearvana. Itani was an intern at Meta, where he worked on smart glasses.
Gollakota and Itani published research last year on a headphone prototype that uses AI to create a “sound bubble” and can learn the distance for each sound source in a room.
Read the full article here
