Surgical robots might allow precise operation in tiny places our unwieldy human hands can't go, but using those robots removes the surgeon's valuable sense of touch. At the University of Washington, a group of engineering students decided to use a hacked Microsoft Kinect to give that sense back.
Engineers and scientists have been working for years to fix the inherent lack of touch sensitivity that goes along with using surgical robots. The sense of touch is tremendously important to a surgeon; without it, you could easily nick an artery or drag along bone without realizing it. The solution is to implement some kind of force feedback, kind of like a more detailed version of the rumbling or vibrating you feel while playing videogames. That led this group of University of Washington graduate engineering students to one of our favorite pieces of gaming hardware, the Microsoft Kinect.
To add force feedback, surgeons need a highly detailed, live-updating, 3-D scan of whatever is being sliced and/or diced. That equipment is normally exceedingly expensive, done with CT scanners or what have you, but the UW team lit on a much cheaper and easily hacked substitute: the Kinect. The Kinect senses 3-D environments by spitting out thousands of tiny infrared dots and then measuring the rebounding light--exactly what they need for this purpose, and at the price of $150. The Kinect has other advantages as well: They can assign "off-limits areas," around, say, vital organs, so the tools can't venture in and cause undue damage.
Before the Kinectified robots are actually used in surgery, some adjustments will have to be made. The Kinect's relatively low-resolution 640 x 480 cameras will have to be upgraded, and the sensors will have to be scaled down to focus on the comparatively small area of surgery (compared to, you know, the giant living room the Kinect normally requires). But the team definitely plans to move forward--after all, a few simple upgrades will still make the contraption both cost-effective and really cool.
You can check out our ongoing coverage of Kinect hacks here, and don't forget to read our guide to setting up the Kinect.
Five amazing, clean technologies that will set us free, in this month's energy-focused issue. Also: how to build a better bomb detector, the robotic toys that are raising your children, a human catapult, the world's smallest arcade, and much more.


Online Content Director: Suzanne LaBarre | Email
Senior Editor: Paul Adams | Email
Associate Editor: Dan Nosowitz | Email
Assistant Editor: Colin Lecher | Email
Assistant Editor: Rose Pastore | Email
Contributing Writers:
Kelsey D. Atherton | Email
Francie Diep | Email
Shaunacy Ferro | Email
That's cool. Maybe it will drop the cost of surgeries since you don't need a $500,000 machine to do the same surgery now.
It'll be great if it also comes with achievements!
This won't likely change the cost of the surgical equipment, because:
#1 The testing and clinical trials involved to satisfy FDA regulation are extremely long and expensive.
#2 Microsoft isn't going to allow it's hardware to be used for an application like this, because if the person undergoing surgery dies, they could be liable.
#3 The article already mentions that higher resolution cameras would be needed, does anyone honestly think they can just buy better camera off of EBay, plug them in and it will work....(I love how the author says a few simple upgrades, yeah right, good luck with that)
The project is an interesting proof of concept, and maybe it will lead to some improved equipment, but possible product it is not.
@Timias-
Wow, apparently you aren't as analytical as you're making yourself out to be.
First fault of logic- this is not a testing or clinical trial for the FDA. This is adapting an existing peice of technology for use with another existing peice of technology that has already undergone the appropriate testing for surgical use. The testing process for the software interface would be much less arduous. Its like using universal joystick on a gaming system, you're not altering the hardware of the system, just the way that you access it.
Secondly- the camera has little to do with the infared matrix of the kinect. The camera, as per the article, would need to provide a clearer IMAGE, and the dot matrix for the infared tech would be scaled down to provide a more accurate depiction of a smaller area.
And finally, Microsoft WOULD WHOLEHEARTEDLY endorse an application of their technology that would be used to save lives. Saying they could be blamed for someone dying is like saying Windows 7 caused someones printer to explode. The proper licensing and disclaimers would of course need to be litigated, but why would Microsoft turn down such a profitable venture.
Formulate a logic argument, lets just not use conjecture.
@timias, @pheonixashes, etc.
The biggest flaw in the idea that this will drop surgeries is that this is in no way replacing existing da vinci surgical equpiment with a microsoft kinect. It is adding the kinect technology to the existing da vinci surgical system, an already established surgical method.
Although the system gives a wide range of motion of the instruments beyond what traditional laproscopic tools gives you, and it also gives the surgeon 3D vision of the surgical field, a major issue was that lack of tactile feedback. I had spoken with a Intuitive Surgical (makers of Da Vinci) representative, he was aware of the issue, and it seemed that he did not know of any plans to add this feature. I'm sure they would welcome this additional feature, as this is a major reason that many laproscopic surgeons still prefer the manual approach.
The truth probably lies somewhere in the middle.
1. There will be testing involved. Probably this could come under the classification of "substantially equivalent", but that only reduces the required work, it doesn't eliminate it. EVERY new medical device must be submitted to the FDA.
2. It likely won't be the actual MS hardware, but a licensed version upgraded to meet requirements. You can't have essential equipment malfunctioning when somebody fires up some electrically noisy surgucal equipment ( some of which is amazingly noisy in the electrical sense). Things like that. Also, they usually want it to be immune to liquid spills and the like.
3.Upgrading the cameras etc may well require significant effort.
The tech could make for a cheaper and maybe even better system, but it has to be robust.
@pheonixashes
I am not trying to appear analytical, I am an engineer for a company that makes health care equipment in the US, and I have gone through the process to change/add electronic components on Class 2 medical devices, the Kinect would be used on a Class 3 device and therefore would need to have much more rigorous testing including clinical trials.
Second a higher resolution camera can't be simply dropped in, the higher resolution feed requires faster hardware to handle the extra bandwidth, and it is unlikely the consumer grade equipment was built to this spec. (Since cost is the primary design issue, there would be little room to add extra capabilities that aren't being used.)
Third as far as Microsoft goes, covering their legal butts is much higher on their priorities than this invention. If you doubt me, you need to get an actual job with a big company to learn how the real world works.
@ Nokia
You might not be replacing the Da Vinci system, but you would be significantly modifying the way it was used, if your force feedback system got it even a little wrong you could kill someone, what if it applied force to the surgeon at the wrong time, or that some EMI from the surgical system caused the Kinect system to lock up? Or the color was off just a little and the surgeon mis-identified a tumor for something else?
I admire your optimism and ingenuity, but all I can say is good luck.
@Ford2go
There is no way this would be substantial equivalent, when it is a completely new idea, that doesn't exist in any robotic surgical equipment to date (per the article).
If they can't use the kinect, then it will end up costing them thousands of dollars (minimum), since the only way they can get it that cheap is to build it in units of millions.
All this arguing is irrelevant!
This is just a starting point and will never be the final product.
But, you have to give them credit for coming up with a good idea and trying it out.
Man didnt start out with sharpened rocks tied to spears...
I cant help but be sucked into this virtual argument. I have much less knowledge on this subject than the people arguing, but it seems to me that timias really seems to know he stuff and he is probably correct on most of his conjectures (at least to a point). However, Microsoft could create the proper legal paperwork so that they are not responsible if their technology causes the death of someone, with a cost that could be dwarfed by the better reputation the company will get for helping to save lives. In business philanthropy can go a long way.
LOL. Regardless of who does what, when and how, everyone has their respective areas of expertise. My point, in stating what I stated, was that one cannot minimize someone elses area of expertise. Timia you may very well have that area of expertise, but you CREATE the device, engineer it, someone else CREATES the control mechanism. This is not a virtual argument and rest assured I have a job in the REAL world. Step off of your high horse and sit down with the rest of us, your engineering degree doesn't place you above my psychology degree. This is a FORUM, look up the definition of the word, I stated to pose a thoughtful logical argument and not use conjecture, which you did after my post. Very nice job in that. But don't downgrade someone elses area of expertise, are you an electronic engineer as well as a software programmer and designer. I highly doubt it but if you are good job you. There are TEAMS of people that are working on this, not just one person. Im sure their collective IQ's will figure this out where you and I may not be able to. Funny though how you took personal offense to something that was posted, we could really analyze that, but you're not paying me by the hour. :) LOL
Medical innovation??
Why does it take a video game console release to improve a system like this? You would think that it would be the video game industry trying to use the advanced tech of the surgical robot, not the other way around.
The innovation is produced by the gaming industry, but the industry of saving people's lives haven't put much effort into improving their own devices.
Lets just hope we don't hear news of microsoft suing surgeons for hacking their xbox.
Off topic but this was the first comment I've posted since implementing the new CAPTCHA system and I just want to know that for a technology based company this site is just ridiculously bad. You would think someone at POPSCI would recognize that this is a terrible way to run a blog.
The CAPTCHA doesn't even work, i need to confirm that I'm a human 3 times before posting when I've posted many legitimate comments before from the same name/ip. Let the humans you know comment! Prevent spammers from dating sites and such from becoming members! Its not that hard, there are plenty of more popular blogs that do a way better job without annoying their legitimate community of commenters.
i am going to be a heart surgeon so this will make it funner.