How Kinect Changed Gaming, Art and Even Medicine

Kinect Hacks Click here to get a bigger view of this amazing image.

In November 2010, Microsoft released Kinect, a motion-sensing accessory for its Xbox 360 gaming console. Kinect could measure depth by sending out thousands of small infrared dots to create a 3-D map of a room, and its microphones could pinpoint sound in space. Such hardware would not be confined to gesture-based videogames. Within a few days, engineers strapped the $150 device onto a robot vacuum and wired the machines together to allow the robot to see and hear. For developers and hackers, Kinect’s promise proved irresistible (and affordable): It could give machines sense. And in the seven months following its release, the device inspired a flurry of development as it became a tool for art, leisure (see “lounge chair” and “channel surfing”), work (for checking e-mail or assisting in surgery) and even play (Tetris in augmented reality).

KINECT HIGHLIGHTS

December 5, 2010: Sighted quadrocopter
Researchers at Hybrid Systems Laboratory, at the University of California at Berkeley, mounted a Kinect on a quad-rotor UAV. Kinect continually mapped the copter’s surroundings, allowing it to avoid obstacles automatically.

January 18, 2011: Surgical equipment
Researchers at the University of Washington wrote code to connect a surgical robot to a Kinect. Kinect made a detailed, live-updating 3-D scan of patients, which the robot could use to better perform operations.

January 24, 2011: Holography
Using Kinect’s infrared sensor to build a 3-D model, Massachusetts Institute of Technology students projected holograms at 15 frames per second to reenact Princess Leia’s plea for help.

February 3, 2011: Draw-a-button
Robotics engineer Garratt Gallagher created an interface in which Kinect recognizes buttons drawn on a piece of white paper in black pen, and it detects when someone presses them.

February 21, 2011: Video camera
Music-video director Dan Melamid used Kinect processing software to shoot Clipse’s “Come N Get It.” The video is filled with “point clouds” and trippy color blocks thanks to the machine’s motion-capture system.