With the wave of a hand, a life-size human skeleton appears in front of him. In the next few seconds, the bones are overlaid with the body’s organs, then its musculature, then the skin and outer appearance of a man. Pinching the fingers in the space in front of him separates the body’s different systems into four silhouettes. A pinch and drag to zoom in on the heart and spin it around as it beats.
This is an anatomy app created by researchers at Case Western Reserve University designed to teach anatomy using Microsoft’s HoloLens. The first models of the the augmented reality platform were slated to ship at the end of March, and the apps just started coming on line accordingly; the Case Western app, released in mid-April, is the first third-party app for HoloLens. But to the researchers who created it, this is just the beginning—future iterations of this app, and others like it, could make the future of medical education more interdisciplinary, training the next generation of medical school graduates to be better doctors.
Gross anatomy is one of the first classes that medical students take, and the word “gross” fits in more ways than one. Students learn about the inner-workings of the human body on cadavers, a trial-by-fire of gore and guts through which new medical students must learn to steel their stomachs. This practice of training doctors on dead bodies has been around since the foundation of medicine. “If you ask physicians what they remember about gross anatomy, it’s the smell,” says Mark A. Griswold, professor of radiology at Case Western Reserve University and one of the main developers behind the anatomy app. “It’s just not an easy environment.”
Some students can’t work with cadavers at all because they have averse reactions to the formaldehyde or other chemicals used to preserve them. But more importantly, Griswold says, students only get one perspective on the body, from the outside in. “As soon as you cut your way in, you’ve lost the context as to where that was when you put it together,” he says. That’s particularly relevant for doctors who go on to become surgeons and radiologists, who rely on a sophisticated knowledge of human anatomy. Thanks to different types of medical imaging such as MRI or laparoscopic cameras snaked through the veins, these doctors are given limited points of view on the body. “If you show a radiologist a 2D image of the body and ask them what is 10 centimeters below this, they don’t know,” Griswold says. “They’ve been trained to look through stack of images and they have no idea of absolute scale.”
The big advantage of the Case Western app is that it shows students where each of these parts fit in the context of the rest of the body. “We can quickly show you all the relevant anatomy you need to know, quickly switching between parts of anatomy, and can display it in a way where we always maintain context,” Griswold says. Skin can be made transparent on the anatomical model in the app, so students can see where the intestines sit relative to the diaphragm and pelvic bones and major arteries. And, importantly, it’s all at the scale of a real human body.
Griswold admits that the current version of the app is relatively simple. “It’s just meant to show what’s possible,” he says. Users go on a self-guided tour of the body, working their way through its various systems, complete with labels.
Unlike with other virtual reality systems, it doesn’t take any time to get your bearings in augmented reality, Griswold says. He has gotten violently ill from wearing other headsets for just a few minutes, but he’s worn the HoloLens for more than nine hours with no ill effects. Some people complain about the limited field of vision offered in the HoloLens, Griswold adds, but that’s exactly why he doesn’t get sick—unlike other virtual reality headsets, this device doesn’t limit a user’s peripheral vision.
The HoloLens app doesn’t have the tactile component of working with a real cadaver, but Griswold considers that a compromise in favor of the physiological context and ease of use that it provides to students.
Griswold believes that augmented reality platforms have the potential to change medical education. In fact, he’s banking on it. In 2019, the Case Western medical school is moving to a new campus that it shares with the Cleveland Clinic. The goal is to educate doctors alongside nurses and other healthcare professionals to foster more collaboration. This new building has no facilities to host cadavers for gross anatomy class. That means that, in the next three years, Case Western needs to be approved by the proper medical boards to use HoloLens to teach that class. “In four years, we’re replacing hundreds of years of medical education,” Griswold says. “I’m optimistic and terrified. But it’s the first piece of technology that it’s almost a given that it will work.” He thinks similar apps could make their way into college and high school anatomy classes, too.
But when it comes to augmented or virtual reality in medicine, the implications go far beyond education. The HoloLens, or something like it, could make it easier for radiologists to tell surgeons where to operate on the body, giving them a “common language,” as Griswold says. A virtual assessment of a person’s stress on his knee could determine if an ACL will re-tear after it’s been repaired. It could help doctors better communicate diagnoses and treatments directly to patients, showing them directly on a simulated body what needs to be done. Medical professionals can also role-play difficult conversations—about difficult diagnoses, about end-of-life care—with virtual patients before addressing the real ones.
The uses for virtual reality platforms extend beyond medicine, too. “We’re working with people in polymer science who want to understand how plastics and molecules work together,” Griswold says. That information is also of interest to geneticists and chemists who want to understand the complex movements of molecules in 3D. Civil engineers can use virtual reality to test the structural integrity of new buildings or bridges. Even choreographers are interested—VR platforms can give them the capacity to stage a performance in two places at once
The technology is improving so rapidly that Griswold believes many of these applications will become possible in the near future. “2016 will be this year of change in virtual and augmented reality,” he says.
If you don’t yet have a HoloLens, you can try out this and other apps on your PC on the HoloLens Emulator.
