/ privacy

Everything is a Microphone: VibraPhone

A very intriguing research paper (PDF) just came out of the University of Illinois Department of Electrical and Computer Engineering. PhD candidate Nirupam Roy and associate professor Romit Roy Choudhury have devised an ingenious use for the vibration motors commonly found your phone, activity tracker, smartwatch, and any other electronic device that can vibrate: a ubiquitous microphone.

Dubbed VibraPhone, the researchers realized that the construction of a vibration motor is very similar to that of a microphone diaphragm. Microphones work at their simplest level by having a moving piece (a diaphragm) that vibrates in sync with the sound vibrations that travel through the air. These vibrations are converted to electrical signals and digitized or output through an analog source (as an aside, this is basically a speaker in reverse, in fact speakers make decent microphones, try this by plugging your headphones into a microphone jack - one of the ears should work well as a microphone).

A vibration motor (or "vibra-motor") is a small motor that vibrates at high speed when an electrical charge is applied - this is what makes your phone or activity tracker or even xbox controller vibrate. The same properties that allow this small vibrating element to vibrate also allow it to be used as an (admittedly poor quality due to its relatively large mass) diaphragm to record audio. A vibra-motor, in essence, becomes a makeshift microphone by running in reverse and taking vibrations in the air and measuring the small vibrations they cause on the motor. All that's needed is the code (or analog wiring) to record the signals created from small changes in air pressure (sound waves).

Now the idea of using a vibrating device isn't new (the authors cite Gyrophone, AccelWord (an accelerometer mic), and several other vibration recording mechanisms as related work) and though the use of a vibration motor is novel, the real breakthrough here is in the processing of the recorded audio. Take a look below:

Here you can see standard waveforms of the word "Often." If you take a look at the last panel, the one recorded with a standard microphone, you can see how the waveform should look. Compare that to the pre-processed vibration motor signal and you can see they look almost nothing alike - something the audio recording confirms with the original recording being nearly unintelligible. After processing, however, the vibration motor signal is very similar to the clean mic and while not crystal clear, is quite discernible and good enough for decoding speech.

The study measured the ability of manual listeners and automated speech recognition software to decode the recorded the speech. At 80dbSpl (approximately the loudness associated with talking directly into a phone) the processed vibra-motor did better than 80% accuracy with both manual listeners and automated (untrained) speech recording software (the regular microphone had near 100% accuracy). At 60dbSpl (approximate loudness of a normal conversation), the accuracy was about 60% for automated recognition and near 70% for manual decoding. All of this is achieved with an algorithm that still has lots of potential for optimization, especially through machine learning.


The paper's authors are quick to point the potential applications for this. Many activity trackers sport vibration motors (for alarms) but no microphones (which could be damaged by water) and currently don't support voice commands. This technology could bring voice control with low power requirements to these devices using little more than a firmware update. That's exciting and a very cool application for an already ubiquitous electronic component.

Of course we know every positive technological advancement comes with huge potential abuses. VibraPhone is certainly no exception. Imagine, if you will, malware that turns your phone, your fitbit, even your playstation controller into a microphone recording everything you say. Imagine installing an app that only asks for notification permissions (to access the vibration motor) that now can record your conversations. Imagine your activity tracker which you wear everywhere recording every word across your day. Now imagine how people could abuse that information.

Granted, it's not practical to record everything, but with keyword recognition and automated speech recording ("Hey, Siri!" "Okay Google!") it's trivial to turn recording on only when it counts. Ubiquitous eavesdropping just a firmware update away - if your phone mic isn't already doing that anyway.