Remember that awesome scene in Minority Report when Tom Cruise just wiggles his hands in the air to sift through information? Today's featured Invention Award winner brings it to life.
When he's wearing the SixthSense, a combination miniature projector, webcam and notebook computer, Pranav Mistry can snap photos just by making the shape of a frame with his fingers.
He can conjure a phone keypad in the palm of his hand and tap the virtual numbers to place a call. The system can even recognize a book in front of the camera, retrieve its Amazon listing from the Web, and project its rating on the cover. Watching Mistry, a graduate student in the Massachusetts Institute of Technology's Media Arts and Sciences program, demonstrate the device is like witnessing a magic show. But he and his adviser, Pattie Maes, a digital-interface specialist at MIT's Media Lab, expect the SixthSense to do a lot more than evoke wonder. Within a few years, they hope, it will let people operate smartphones without touching a button, do instant research on objects around them, and generally offer the kind of enhanced-reality experience that's now confined to science fiction.
Maes hit on the idea last October while discussing g-speak, a real-world version of the gesture-controlled interface in the movie Minority Report. She liked the notion of using hand signals to manipulate digital content but wanted something cheaper that you could walk around with, projecting content and interacting with it anywhere you liked. Mistry, nicknamed "Zombie" because of his aversion to sleep, turned out a prototype in just three weeks.
Although the system has evolved considerably since then, the basic concept has stuck. A pocket projector and a webcam hang on Mistry's chest, both wired to a laptop in his backpack, and he wears four different-colored marker caps or pieces of tape on his thumbs and index fingers. When he switches on the system, the webcam starts capturing video and streaming it back to the computer. Then the computer's vision algorithms take over. The real brains of this system, this software filters out background imagery, determines x and y coordinates for each cap or tape color in the video frame, and tracks them over time. The computer discerns which colors are moving which way, so it can follow freehand gestures. These, in turn, trigger various functions.
Say, for instance, Mistry wants to know the time. He traces a small circle on his wrist with his index finger, and the computer tracks the red marker cap or piece of tape, recognizes the gesture, and instructs the projector to flash the image of a watch onto his wrist. For book-recognition, Mistry activates the program with a gesture, and the system snaps a photo of the book, compares it with book-cover images it finds online, computes a match, and retrieves and projects the ratings. Future functions will similarly rely on computer vision algorithms. "It recognizes what's in front of the user and augments those things with relevant information," Maes explains.
This summer, Mistry will begin working with Samsung engineers to compress the entire system into one of the company's new smartphones, which has a built-in projector. With further improvements to the algorithms, eventually even the markers and tape could go away and the device could track fingers alone, making it even easier to enhance your surroundings anywhere you go.
Here's a video from the TED conference of Pattie Maes presenting SixthSense:
Inventor: Pattie Maes and Pranav Mistry
Time: 8 months
Is It Ready Yet? 1 2 3 4 5
wouldnt it be obvious to take away the projector and replace it with microdisplay sunglasses???
and put the webcam on the glasses too. that way, the webcam can see exactly what you're seeing
Wow, that a great invention, hope it will become popular in near future, I will wait to have that magic fingers..
Some great ideas there and I bet alot of them aren't too far away from happening. The world would be a much better place for it http://www.mypetstop.co.uk/Services/Dog-Training-Manchester-Newcastle-Leeds/
Modern science and technology more and more developed, many items have joined the science and technology elements, they hope, will allow people to operate fixed-button smart phones, this moment study of the surrounding objects, generally to provide an enhanced experience the real world that is now limited to science fiction . This is a good idea.In fact, high-tech elements are also able to join the glasses, you can see here
thanks . your hard work speaks a lot in your article. Tons of information you have shared with us. I am really thankful to this information. Keep posting.
i am staring at the webcam on my macbook wondering why it wudn't fit on glasses which should really be your display. add mic and earbud to one side of glasses for phone functions and what not, bluetooth to the otherside to connect it to the smartphone in ur pocket. if you could rig the frames to actually be the battery rather than the traditional rectangular and other shaped batteries that wud need a separate resting place, that would be one sweet rig.
Thank you for sharing these important messages I thank you for this informative article.www.wangshanghai.com
The Sixth Sense - New GROUNDBREAKING Book in 2012! The Sixth Sense leads to Enlightenment
DNA Healing Code - Hardwired in ALL Humans
As an example of what I mean by “Groundbreaking” info that can be found in the 2012 release of my new book on The Sixth Sense (not yet titled )…..
There is a sequence required to communicate with Infinite Intelligence – to gain insight and/or much more complicated yet achievable, HolisticDNA Energy Healing. The sequence required is on multiple simultaneous levels, not just “step by step” like an instruction manual for assembling a piece of furniture.
The “Key” sequence has to do with applying known facts, beliefs, emotions and faith (not religious) – if not applied exactly as detailed, the Sixth Sense will remain dormant, and not be “activated”. As an example:
in theory --
you won’t find this in any existing text, which is why The Sixth Sense is so rarely utilized and hard to confirm. This will allow the Scientific Community to experiment and confirm my claims — remember, just a few decades ago, it was impossible to have a Man walk on the moon. Time for the next impossible to be challenged and confirmed — real
Steve Meyer HolisticDNA