A Hand-Tracking Sensor For Virtual Reality Headsets

One of ten brilliant innovations from our 2015 Invention Awards
Nimble Sense could let VR users grab and throw a digital object or even pluck the strings of a virtual guitar. Courtesy Nimble VR

We may earn revenue from the products available on this page and participate in affiliate programs. Learn more ›

2015 Invention Award Winner Category: Gaming
Inventors: Robert Wang, Chris Twigg, Kenrick Kin, Shangchen Han
Company: Nimble VR
Invention: Nimble Sense
Development cost to date: Undisclosed
Maturity: 2/5

When peering through a virtual reality headset, players can’t see their hands–or use them to interact with their digital surroundings. To solve VR’s interaction problem, software developers Robert Wang, Chris Twigg, Kenrick Kin, and Shangchen Han formed a start-up, courted investors, and got to work.

The four had spent years working with motion-tracking camera systems and developing algorithms that monitor and analyze skeletal structure. With their experience, it didn’t take long to craft software that tracks joint angles and knuckle placement to create realistic virtual hands. But they couldn’t find a camera sensitive enough to pick up such data. “We played with them all,” Wang says. “There was no depth-sensing camera good enough for VR hand tracking.” Existing videogame motion trackers, such as Microsoft’s Kinect, are designed to detect large objects, such as a bouncing 8-year-old several feet away, not subtle hand movements.

The team set out to design its own specialized camera, Nimble Sense, small enough to fit over an Oculus headset and with a field of view optimized for VR. It monitors users’ hands, and software then digitally mimics every gesture. Facebook-owned Oculus VR was so convinced it bought the start-up, Nimble VR, last December. Now, the developers are working to make Nimble Sense even better. “There are still a lot of interesting problems left to be solved,” Wang says.

This article was originally published in the May 2015 issue of Popular Science. For more extraordinary innovations and tips on how to be an inventor, click through the rest of our Invention Awards feature.