Muscle-Based PC Interface Lets You Literally Point and Click, No Mouse Required

A research collaboration between Microsoft, the University of Washington and the University of Toronto is developing a muscle-controlled interface enabling Minority Report-esque, gesture-driven interaction with computers. It’s perhaps the most promising of the billion or so Minority-Report-aspiring prototype interfaces.

For now, the system consists of six electromyography (EMG) sensors and two ground electrodes ringing the upper right forearm that sense finger movement in the right hand. Two more sensors on the left upper-forearm sense squeezing movements in the left hand. The software must be carefully trained via machine-learning algorithms to associate specific gestures with specific electrical signals in the arm, but once set the interface works very well, recognizing gestures 85 percent of the time (up to 90 percent for some gestures). The software also continues to learn, improving accuracy over time. The training also works both ways; the researchers found that their human subjects quickly learned to make small adjustments to help the computer recognize gestures more accurately.

In the lab, the sensors are wired and placed individually, but because they don’t target specific muscles, they should be equally successful at interpreting gestures if placed in an arm band that users could slip on without assistance. Rather than try to target specific muscle movements, the algorithms focus on the magnitude of muscle activity, the rate of activity, and the wave-like patterns produced across several sensors at once when the arm moves. By triangulating these three distinct data streams, the computer learns to recognize individual gestures as well as differentiate between them.

The researchers see a number of applications for the technology, from changing songs on MP3 players while occupied with tasks like jogging to playing video games like Guitar Hero without a controller. The goal is more seamless integration of computer power into our lives. While it may seem subconscious to you, all that pointing and clicking requires a good deal of your attention. New and novel interfaces like this could render such systems obsolete, replacing them with more passive, better-integrated ways to interact with the machines in our lives. Here’s hoping that whatever arm band peripheral Microsoft develops for this interface is half as cool as Cruise’s three-finger, light-up gloves.

Technology Review