Sandbox Hackathon 2022 - App for deaf people that converts conversations into a group chat in real time. Before the hackathon, I noticed a big pain among deaf/HH people is the inability to feel included in vocal conversations. Current TTS/STT services offer very basic assistance, but in the current day and age, deaf people deserve much more.
After 8 hours of intense coding fueled by energy drinks, pizza, and our desire to help, we finally finished building SpeakUp.
SpeakUp is an app that converts real-life conversations into a group chat in real-time. It uses a deep learning technology called Speaker Diarization that segments speech into its respectable speaker before converting it into text in real-time. This way, deaf people don't feel isolated in fast and unpredictable conversations with hearing people. It also converts speech to text with an auditory "typing" status to prevent interruptions.
Building such an app from scratch has been an inspiring learning experience and it couldn't be done without the brilliant engineers Cody Arvonen and Sonny George! Seriously talented and a pleasure to work with.