MIT Gesture-Based Computing System Uses Motley Clown Gloves for Finer, Faster Control

Technology just got a bit more psychedelic: Robert Wang, a graduate student at MIT, has invented a gesture-based computing system consisting of a standard webcam and a colorful pair of Lycra gloves.

Unlike other glove-based interfaces, which rely on pieces of tape attached to the fingertips, Wang’s system can read the wearer’s entire hand. As a result, the interface can translate complex gestures from the gloved hand to the onscreen 3-D model with remarkable precision and almost no lag time.

With the help of Jovan Popovic, an associate professor of electrical engineering and computer science, Wang carefully selected and arranged colors that a computer can accurately distinguish from one another and from background objects. For example, colors on fingertips can recur on the back of the hand, but not the front, lest the interface confuse which patches it should respond to. While early designs used dots and a variety of shapes and colors, the current model uses 20 irregularly-shaped patches in 10 different colors.

While configuring the software, Wang also had to figure out a new algorithm to help the computer match images of the actual glove to digital models of the glove-wearing hand in various positions. After the webcam captures an image of the actual hand, it crops the image size to 40 pixels by 40 pixels. The software taps into a huge database of hand positions, finds the corresponding position, and projects an image onto the screen. Since the system relies on matching images rather than on calculating relative distances, it can display the virtual gestures almost instantly.

Wang imagines that video game players, designers, and engineers will someday use his system to easily maneuver 3-D models of products and large structures. Wang’s gloves are already nabbing attention, not just for their bright appearance, but because Lycra is cheap to manufacture and can stretch to fit all hand sizes. The system itself can quickly adjust to the shape of a wearer’s hand. The glove user presses his hand against an 8.5-by-11-inch sheet of paper on a flat surface in front of a webcam, and voila, the system has learned. Wang hopes that in time, people will apply his technology to bigger pieces of clothing, like shirts, in order to aid digital animation or assess athletic performance.

In the meantime, check out this fun “commercial” for the presentation Wang did on his multicolored gloves:

MIT