Designed by Cheng Zhang, assistant professor of information science in the Cornell Ann S. Bowers College of Computing and Information Science, and Cornell University doctoral student Ruidong Zhang, the SpeeChin is a compact infrared camera hanging on a necklace that’s worn at chest level. The camera points upwards, capturing high-contrast video of the wearer’s chin movements, which, after some training, can be used to figure out what someone is saying without them making any sound. The location of the camera is not only more covert than mounting a camera to someone’s face to record their mouth movements, it also sits at an angle where other people’s faces can’t be captured, ensuring no privacy concerns.
The researchers tested the SpeeChin with 20 participants; 10 of them spoke 54 simple phrases including digits and common voice assistant commands in English, and 10 spoke 44 simple words and phrases in Mandarin Chinese. After a training period, the chin-tracking camera was able to recognize commands in English with 90.5% accuracy, and commands in Mandarin Chinese with 91.6% accuracy. This was with the participants uttering the various phrases while remaining stationary. When asked to speak these phrases while walking, the accuracy dropped as a result of variations in each person’s movements including their gaits and the added movement of their heads.
It’s a problem that could potentially be resolved with a longer training session that included the participants both standing and walking while working through the library of phrases and commands, as well as improved camera equipment that was better able to track chin movements through more resolution or higher frame rates. Here’s to hoping the researchers continue to develop the technology, because with more advanced speech recognition capabilities, the world would be a more peaceful place where no one had to make a sound.