SBIR-STTR Award

Noisy rooms with multiple active sound sources create problems for hearing-impaired listeners. Unwanted masking sounds reduce the quality and intelligibility of speech that listeners want to hear, especially listeners with hearing deficits. We propose a novel assistive listening system called HWIW ("Hear What I Want") that"scrubs" (i.e., removes) noise and other unwanted audio components from complex real-world environmentscontaining multiple acoustic sources. HWIW has been designed for integration into NIH's Open SpeechPlatform initiative for hearing aids and other personal audio devices. HWIW will leverage STAR Corp's MultipleAlgorithm Source Separation (MASS) application framework of "pluggable" acoustic separation signalprocessing modules. MASS is compatible with the Open Speech Platform and available on GitHub. HWIW is a room-centric system that delivers listener-specific audio to users through their smartphones. HWIW employs multiple microphones distributed around a room and connected to a room-specific dedicatedserver. An initial HWIW setup procedure is used to name permanently positioned "noisemakers" in the roomsuch as speakers and appliances and characterize their acoustic radiation and reflection patterns. The HWIW Room Server processes audio signals from multiple HWIW mics to scrub the noisemaker-generated sounds from any microphone in the room a listener chooses to monitor. Multiple listeners are supported simultaneous-ly. Each listener uses a HWIW Listener App to specify which mic to monitor for sounds of interest and which of the known noisemakers to scrub. The HWIW Room Server computes an individualized scrubbed audio streamfor each listener and transmits it wirelessly to their Listener App. The Listener App outputs this audio stream tothe listener's hearing aid, personal audio device, or earbuds as a standard line level or Bluetooth audio signal. HWIW is room-centric, sensor image-based, latency-optimized, and listener-aware. Important system components are embedded in the acoustic space itself, rather than in the user's ear (the hearing aid). HWIW calculates the acoustic image of masking sounds in sensor response mixtures so that images of unwanted sounds can be removed. It computes the latency of its signal processing and balances the quality benefits oflonger-latency scrubbing against the perceptual advantages of faster response times. HWIW employs listener-specific acuity profiles, information about the sound-isolating properties of each listener's hearing aid or earpiece, and the listener-specified masking sounds to determine whether which maskers are audible given thelistener's acuity; and thus what the optimal noise scrubbing strategy is for that listener. In Phase I, we will implement three HWIW MASS scrubbing modules, and a prototype of the Listener App.We will objectively measure the ability of the scrubbing modules to scrub noise from microphone responses,calibrate those measurements against perceived residual noise, and evaluate Listener App useability. The HWIW system will help hearing impaired listeners hear what they want more clearly in noisy rooms.

Public Health Relevance Statement:
Project Narrative Listeners with hearing deficits find noisy rooms, such as many classrooms, burdensome because speech is hard to understand in rooms with multiple interfering sound sources. We propose to improve hearing healthcare by developing a hearing aid-compatible system that would identify the location and sounds of interfering acoustic sources and deliver an enhanced rendition of speech sounds from which interfering sources have been "scrubbed". This product would use multiple microphones permanently placed in a classroom or other complex space to isolate individual sound sources, identify and remove echoes, and upon request deliver personalized enhanced speech signals, from which masking sounds had been removed, to the ears of multiple listeners in the room via their cellphones and preferred listening devices.

Project Terms:
<21+ years old>