RACHEL GROSSMAN-KAHN
INCLUSIVE UX DESIGNER
SoundWatch
An Android app that provides glanceable, always-available sound feedback on smartwatches for d/Deaf and hard-of-hearing users
Client: The Makeability Lab. Sponsored by the National Science Foundation
Role: UX/UI Designer
Awards: Microsoft Best Artifact
Press: Forbes
Author on Publication: SoundWatch: Deep Learning for Sound Accessibility on Smartwatches
​
TLDR
SoundWatch is a smartwatch app designed to give Deaf and hard-of-hearing users real-time awareness of important environmental sounds—like fire alarms, doorbells, and speech—through visual and haptic feedback. When I joined the team, the app was already functional, but had never been touched by a UX designer. I stepped in to simplify the interface for glanceability, refine the interaction patterns, and shape the experience to better serve real users.
What made this project so meaningful was that it was built by and for the Deaf community—led by a hard-of-hearing innovator, tested by D/deaf users, and grounded in real needs. My background as an ASL interpreter helped me bridge design decisions with cultural insight, ensuring the interface respected and reflected how Deaf people actually live and communicate.The app launched in 2020 and reached over 500 downloads within the first month—a major milestone for a niche audience on Android smartwatches.
​
DESIGN CHALLENGE​
Many common and useful sounds and alerts are inaccessible to the D/deaf and hard of hearing community. While many Deaf people live perfectly full lives without utilizing sound, there are still many people that want to be assisted in accessing the sounds in their environment. ​
​
SOLUTION
We created a smartwatch app on Android to provide glanceable, always-available sound feedback to people who are D/deaf or hard of hearing. Led by a hard-of-hearing innovator who wanted to build for his community, SoundWatch uses machine learning to alert its users to important sounds occurring in their environment. (It was important to me that this project was conceived and led by someone in the Deaf community, as I wanted to make sure this was a project that satisfied a real problem, and didn't want to make assumptions about what a Deaf person needs.) With everything from critical fire alarms to speech alerts, users know they can rely on SoundWatch to make their environment more accessible. We provide visual plus haptic sound feedback designs, a quantitative examination of smartwatch-based deep learning sound classification models and architectures. It has implications for future portable sound awareness systems.
PROCESS
I joined the small team well into the development of the app. Research, initial development, programming and initial usability testing were already completed, as well as two papers were published about the project. However, until I joined there had never been a UI/UX Designer on the team, so developers had made all design decisions. This meant that there hadn’t been a lot of thought put into the visual design. Even though I had to work a little backwards, this gave me a great opportunity to come in and make an impact on the project, to make sure it was user friendly for the Deaf community, and had consistent design patterns. One potential issue I identified right off the bat was that many D/deaf users would be signing, meaning that their hands would be moving quickly. This would impact viewing the interface on the watch face. Therefore, I made it a priority to simplify the UI and amount of content on the screen, to make it glanceable.
I continued jumping in by doing an initial evaluation of their UI with notes for improvement. I then tested the app by wearing the smartwatch for two weeks and created a Journey Map, recording my findings on its accuracy and ease of use, which I then shared with the team. (We later did user testing with eight D/deaf participants because it is crucial to test with actual users. As I am not Deaf I therefore have limitations with the product.) As I journeyed, I considered several key UX questions. How is a smartwatch-based sound awareness app adopted and used by DHH people? How does the app affect the lives of DHH people, their understanding of sounds and their environment? How do DHH people customize the sound feedback in different contexts? What are the culture-specific customization preferences (if any)?​​
As the project continued, I gained more responsibility, such as designing Sound Watch’s logo and ads. Because of my background as an ASL interpreter for 10 years, I also provided insight into the Deaf community and was able to get feedback on sound awareness systems from my network. I spent most of my time working closely with the main developer testing several iterations of the app as they were produced, and providing design feedback. I contributed to changes such as providing a “tutorial” and “about” page to improve the user experience.
CHALLENGES
This was my first project as the sole UX Designer on a team, a role that I embraced. It meant people came to me as the expert, so I had to be ready to provide answers and feedback to everything UX related. It was also my first Android based project, so using a new platform was difficult at first. Because I was working with developers and programmers, there was new language and processes I had to learn. One part that was hard but satisfying was learning a bit of code so that I could install each updated version of the app to my watch and phone.
FINAL PRODUCT
This project started in September, 2018 and was released in October, 2020. Within a month we reached over 500 downloads which is significant considering the small population that is HOH and also owns an Android smartwatch. The app was well received and inspired a companion app, called ProtoSound which I worked on as well. ProtoSound allows users to record custom sounds to be recognized. In the spirit of innovation, ​I am proud that Sound Watch is open-source and all its components are available to the public.
