RELATED RESEARCH · PERCEPTION & ASSISTIVE SYSTEMS
A sound may matter in one setting and distract you in another. Assistance needs to fit your surroundings, your attention and the moment you need it.
Alongside our speech and vision work, we explore hearing, environmental context and adaptive feedback: how a system could connect what it senses to assistance a person actually wants.
See why the surroundings matter
SIGNAL, SETTING, PERSON
Imagine an assistive device detecting a short notification tone. Identifying a sound is only one part of deciding whether and how to draw someone’s attention to it.
THE SITUATION
Quiet.One notification tone stands out from the background.
If the person has asked for notification support, a visual or tactile cue might help. The system still needs a way to respect preference and avoid redundant alerts.
Detection should connect to a chosen form of assistance.
THE SITUATION
Crowded.Voices and several similar tones overlap.
Assistance should not turn an ambiguous signal into a confident claim. Research needs to consider uncertainty, repeat requests and what happens when the system cannot provide a useful interpretation.
A good result includes knowing the limits of the interpretation.
THE SITUATION
Engaged.The person is listening to someone speaking.
Recognizing a tone does not establish that it deserves attention now. The research question includes how a person can choose, correct or pause assistance as the situation changes.
More feedback is not always more help.
WHY IT IS DIFFICULT
Recognizing a signal is not enough. You also need to consider who receives the cue, where they are and what they are trying to do.
Background noise, competing inputs and uncertainty about the source can change the interpretation. Our interests include audio and multimodal context, while keeping the difference between a detected signal and an inferred meaning explicit.
Preferences and needs vary between people and over time. We ask how correction and interaction could guide assistance without requiring someone to continually configure or supervise it.
Signal accuracy alone cannot describe a helpful experience. Timing, missed or unnecessary cues, attention, effort and user control would need to be examined for a specific task and population.
RESEARCH SCOPE
Our direction includes hearing, assistive experiences and adaptive feedback. Sensory restoration remains a longer-term research interest within this area.
Potential investigations include context-aware cues, ways to incorporate user correction, and evaluation that considers real environments alongside controlled conditions. These questions could be relevant to assistive technology teams, wearable-product builders and researchers.
Research stage. No clinical product is available; no clinical efficacy or regulatory approval is claimed.
Discuss a perception research question ↗
Start with a specific assistive task, intended users and clearly described environments. Define the limits of the system and distinguish useful assistance from missed, unnecessary or misleading feedback.
A proposed study would examine interpretation and the experience of the person using the assistance. Any work involving participants would need appropriate consent, domain expertise and a study design suited to the task. Clinical questions would require a separate clinical and regulatory pathway.
Related reading: Amershi et al., Guidelines for Human-AI Interaction (2019) considers interaction over time, correction and user control.
PUT YOUR QUESTION TO WORK
What is the person trying to do? Which signals matter? Where does current assistance fall short? These are useful starting points for a device team, an accessibility practitioner or a research collaborator.
We welcome discussion of a bounded research question, the expertise it requires and how to evaluate it responsibly.
Discuss an assistive experience