Live-Streamed Game Collects Sounds To Help Train Home-Based Artificial Intelligence
From yawning to closing the fridge door, a lot of sounds occur within the home. Such sounds could be useful for home-based artificial intelligence applications, but training that AI requires a robust and diverse set of samples. A video game developed by Carnegie Mellon University researchers leverages live streaming to collect sound donations from players that will populate an open-source database.
"This data could be used to create extremely useful technologies," said Jessica Hammer, an assistant professor with joint appointments in the HCII and the Entertainment Technology Center. "For example, if AI can detect a loud thud coming from my daughter's room, it could wake me up. It can notify me if my dryer sounds different and I need to change the lint trap, or it can create an alert if it hears someone who can't stop coughing."
Hammer and her team developed the game, "Rolling Rhapsody," specifically to be played on the live-streaming platform Twitch. The streamer controls a ball, which they must roll around to collect treasure scattered about a pirate base. Viewers contribute to the game by collecting sounds from their homes using a mobile app. "When they submit sounds, they are donating them to the database for researchers to use, but those sounds are also used as a part of the game on the live stream, incentivizing viewers with rewards and recognition for collecting many sounds or unique sounds," Hammer said.
Privacy is paramount, and all players and viewers must opt in and provide consent to upload sounds. Additional privacy measures have also been taken, including opportunities for viewers to redact sound files that may have accidentally captured something personal. They can delete submissions, choose to store sounds locally and withdraw their consent at any time. "We can collect data in a way that's fun and feels good for everybody involved," Hammer said.
See the full story here: https://www.cmu.edu/news/stories/archives/2020/august/game-collects-sounds.html