Augmented Reality

At the Hospital




The first clinical medical experiment with augmented reality is being conducted at the University of North Carolina at Chapel Hill. Patients admitted for routine breast biopsies and possible lumpectomies are randomly assigned to the AR test. Instead of the radiologist's usual practice of looking up at a sonogram screen and then back again at the patient, ultrasound images are seen through the physician's headgear as projected directly onto the patient's body. This provides a sort of virtual X-ray vision throughout the procedure. Breast lumps and other possibly cancerous anomalies show up as ghostly white outlines against an uneven gray background. And the position- and orientation-sensing technology in the head-mounted display lets the radiologist "see" where to guide a biopsy needle with unprecedented precision. The hoped-for outcome of this AR application includes fewer complications and shorter recovery times for existing procedures, as well as the development of new surgical techniques.




For brief procedures such as biopsies and laparoscopic (minimally invasive) surgery, a head-mounted AR display offers an ideal solution for combining actual and computer worlds. But for longer operations or spot checks of lengthy procedures, a head-mounted device may prove less desirable than handheld display equipment or images that are projected directly onto the real world. "Head-mounted displays in their present state have serious limitations," says George Stetten, an assistant professor of bioengineering at the University of Pittsburgh. "Your field of view is limited, the resolution is not as good as natural vision, and it can be cumbersome in the operating room."




Stetten has developed a handheld ultrasound transducer that casts an image directly through the part of the patient's body being examined, like a black-and-white beam from an X-ray flashlight. In this system, the overlaying of computer-generated sonograms on flesh and bones is all done with mirrors.