MIT's Kinect Hack MIT

The Kinect's more official uses, namely its games, have pretty much avoided tracking of individual fingers in favor of full-body tracking. That's fine for traditional motion games of the sort pioneered by the Nintendo Wii, but the Kinect has potential far beyond ping-pong and dancing games--and a lot of that future depends on finger tracking for more delicate controls. Luckily, MIT's Robot Locomotion Group and Learning Intelligent Systems teams took it upon themselves to show that the Kinect can absolutely recognize ten fingers and some relatively minute gestures.

The video, which, yes, is highly reminiscent of a near-decade-old Steven Spielberg movie featuring a diminutive action star and (if I remember correctly) a lot of shiny stuff blowing up, manages to show some basic photo manipulation using finger and hand gestures. Grabbing photos with two fingers, stretching with both hands, and shunting out of frame with a fast sweep of the hand--this is one of the most polished hacks we've seen yet. This kind of tracking is going to be essential if Microsoft wants to take the Kinect beyond just gaming, be it content creation or simply more detailed interfaces.


Don't forget to check out our rapidly growing coverage of Kinect hacks, and our essential guide to setting up the Kinect for intended or illicit use.

[YouTube via Kotaku]

12 Comments

The recognition seemed to flicker on and off. And did anyone else notice just how close he was standing to the Kinect?

Its camera's resolution is just not good enough for single-digit tracking. Maybe one of the Hackinectors (yes, I coined that word just now) can try modding the hardware with a higher-res camera (both for the depth cam and the color cam)? That would help a lot!

-IMP ;) :)

Air guitar! Air keyboard! Air drums!

To IceMetalPunk,

The Kinect is already out of date! Imagine what companies are working on behind closed doors, not to mention the speeds at which computer processors are reaching!

In this particular experiment with the Kinect, students are utilizing hardware that is mass produced and readily available to the public, but what have they developed with their own hardware?

The answer, is G-Speak: John Underkoffler on Ted Talks released a video that's really inspiring and gives us a glimpse at what is to come.

Go to Youtube and search "John Underkoffler points to the future of UI" and you'll see some really amazing data manipulation!

It like the defying gravity guys off of AGT. They should have won that show.

Isn't the kinect cheap off the shelf hardware? As far as I understood it it's the software, image recognition and knowing where a hand is behind a body and such, that makes it a true advancement?

With this we could create a game that allows people who are learning sign language on their own to practice interactively

To sykosmurph,

You are correct in that software is key in the utilization of hardware, and MIT has the best programmers out there!

What I was talking about in the hardware aspect was the speed at which the camera and processors can send/receive signals. Interpreting that data is 100% software, which MIT seems to be making great strides in.

To truthseeker,

That's a great idea! What I've been imagining is applying hand-tracking to my favorite games, such as Starcraft, Warcraft, EVE Online, and other action-intense games. I can't wait to play a strategy game without a mouse and keyboard.

Ever see the movie "Gamer" where the character Simon Silverton plays 'Slayers' using only his body movements?

@Dubby: I watched the video you suggested. While it's kind of cool, it's not "amazing". In fact, the entire thing is just hand tracking with the original Minority Report UI (actually the original, as he says :P ). But that's all software with gesture tracking. The thing is that we shouldn't need to learn special gestures to work with these kinds of UIs. We should be able to use natural gestures and have the computers recognize their meanings, even when we're redundant.

And since we've gone all out hacking the Kinect, we might as well use that as our base board.

-IMP ;) :)

So far Ive been unimpressed with kinect , this hack really was cool but it will never be available on the xbox .

Well done MIT, well done.

hacking an information gathering tool marketed as a gaming device is hardly amazing. Kinect is being used as a marketing and information gathering device. There is even a microsoft exec who said that they "hoped" to use the "data" being gathered from kinect users for enhanced targeted marketing. Your shirt logo, your brand of shoes. Proabably gathering all data possible from reflexes , pupil dilation, heartrate, and anything else they could jam in there.

Allegedly, there are tangible objects, and persons created out of black ops technology, this is the atari of gaming....
Except it's reporting to microsoft, size 10 shoe, IQ 88, likely impaired! hah!

Well done indeed, but can anyone remember Jonny Mnemonic? Just one step closer to being able to get into Web 3D!

Popular Tags

Regular Features



June 2013: American Energy Independence

Five amazing, clean technologies that will set us free, in this month's energy-focused issue. Also: how to build a better bomb detector, the robotic toys that are raising your children, a human catapult, the world's smallest arcade, and much more.


Online Content Director: Suzanne LaBarre | Email
Senior Editor: Paul Adams | Email
Associate Editor: Dan Nosowitz | Email
Assistant Editor: Colin Lecher | Email
Assistant Editor: Rose Pastore | Email

Contributing Writers:
Rebecca Boyle | Email
Kelsey D. Atherton | Email
Francie Diep | Email
Shaunacy Ferro | Email

circ-top-header.gif
circ-cover.gif
bmxmag-ps