Virtual reality headsets are not yet commonplace, but they already have the potential to revolutionize many aspects of our lives—how we view entertainment like movies and video games, how we see the world, and even how we do science.
Now researchers at Weill Cornell Medicine in New York City have started using the technology to better understand the genetic mutations that drive cancer. They’ve developed a new program for the Oculus Rift VR headset that lets users see and interact with 3D models of microscopic proteins. Called “IPM VR,” short for Institute of Precision Medicine VR, the program aims to make it easier for researchers to pinpoint where and how a person’s DNA has mutated to cause cancer. Their goal is to help doctors all over the country better understand the mutations in order to quickly find the best treatment to stop the disease.
A couple of us here at Popular Science recently got the opportunity to test IPM VR ourselves, as well as to learn more about the program from the researchers who developed it. Here’s what we discovered.
Why do we need to pay attention to proteins?
In order to understand the value of IPM VR in fighting cancer, you first need to know about how cancer is formed in the body. It all goes back to DNA, the blueprint by which your body, and the body of every living creature on Earth, manufactures proteins.
Everyone has some mutations in their DNA. Some are inherited, others are acquired from environmental factors such as smoking or sun, and others simply happen spontaneously new cells form. Most of these are benign, but the wrong combination of mutations can sometimes cause cells in a particular organ or area to grow out of control, which can eventually inhibit the normal functioning of the body. That runaway growth process is cancer, in all of its various forms.
Tumors usually contain about a thousand genetic mutations, but only a handful drive the cancer to grow and move throughout the body. Figuring out which mutations are the drivers of a particular cancer can be the genetic equivalent of a needle in a haystack. But if doctors can do it, they can target only those cells with that particular mutation, allowing healthy cells to continue to function — a better outcome for the patient overall. That’s the field of precision medicine.
It’s only by looking at proteins — the body’s molecular building blocks created from the genetic instructions in our DNA — that a genetic mutation becomes evident, says Olivier Elemento, a specialist in precision medicine at the Meyer Cancer Center at Weill Cornell. “A mutation at the DNA level does nothing—it’s only when the gene is transcribed and made into a protein that the mutation expresses itself, and the function of a mutation becomes important,” he says.
Proteins are three-dimensional structures, but they’re often depicted flat on paper. If clinicians don’t see the proteins in 3D, they can’t fully understand the role of the mutations, Elemento says. When clinicians are trying to determine if a new mutation could be driving cancer, they check to see if it’s close to a hotspot, a concentration of genetic mutations that indicates that the error is common in other patients with the same type of cancer.
“In a one-dimensional representation of a protein, if a mutation falls in a hotspot, it’s a no brainer—it’s important,” Elemento says. But because proteins fold, a mutation that might seem far away in a 2D representation could actually be positioned very close to a hotspot in the three-dimensional protein.
That might sound like a small difference, but the significance for the patient could be huge: It shows the clinicians that the mutation might be driving the cancer, and that knowledge might help them select a drug that could target it. It’s not an exaggeration to say that this information could make the difference between life and death for a patient.
Using virtual reality to look for cancer-causing mutations
Scientists at Cornell used an early developer version of the Oculus Rift VR headset and gaming software to develop IPM VR, giving researchers and clinicians an immersive way to explore a patient’s unique genetic mutations.
The Oculus Rift VR headset is equipped with a motion sensor that detects what direction you’re looking. IPM VR also relies on the presence of a few external cameras not included with the headset, which further track the motion of your body and hands.
After researchers sequence all the genes in a patient’s tumor, the software projects all of the mutations onto a particular protein, the shape of which the researchers downloaded from the protein data bank, a digital archive of 3D protein shapes.
The clinician wearing the headset can change the view to see only the mutation hotspot or only the mutation in question, navigating through the simulated protein with hand and arm gestures. For more information, the clinician can pull up two-dimensional “documents” pertaining to the patient—the report about the type of cancer, or the patient’s medical history from the electronic medical record.
In my brief time using it, it was a truly impressive piece of software to operate. It was my first time using an Oculus VR headset and I flailed clumsily to move through the protein, while my more VR experienced colleague Dave Gershgorn got his sea legs much quicker. He navigated to the mutation hotspots with ease. You can watch our experiences in the video at the bottom of this article.
If my ineptitude is any indication, it will take clinicians a little while to get used to IPM VR and the Oculus Rift headset more broadly. The researchers behind the software are still tweaking some of these features to see what best fits clinicians’ needs, says Alexandros Sigaras, a research associate in computational biomedicine at Weill Cornell Medicine. And though many researchers around the country are working on different aspects of precision medicine, Sigaras and Elemento believe that theirs is the only one integrating patient information into virtual reality.
How VR could become a game-changer for medical treatment
Right now this tool is primarily for research, Sigaras says, but the Cornell team is hoping that it will soon make its way to clinicians all over the country. “The idea of using Oculus is not to replace every computer—it’s to enhance information that would otherwise be very hard to get,” Sigaras says.
He’s aiming to make the complex information about genetic mutations and protein shapes instantly intuitive to clinicians, who wouldn’t have to learn to use another piece of software on their computers. Plus, clinicians would be able to collaborate more seamlessly—using their own headsets, specialists in different disciplines could see the same information to discuss a particular case.
Ideally, Sigaras would like the software to contain all of a patient’s information so that clinicians can analyze it without the distractions and technical requirements offered by a computer. That’s especially important for precision medicine, where the immense amount of genetic and health data can easily overwhelm doctors looking for the best way to treat a patient.
Of course, most doctors don’t yet have virtual reality headsets. But Sigaras, Elemento, and their team strongly believe that the platforms will soon be commonplace. “These Oculus devices are going to democratize themselves very quickly,” Elemento says.
Even at $599 for the recently announced Oculus Rift consumer edition, plus the separate high-powered graphics PC needed to run them, the setup isn’t very expensive compared to other medical equipment, he says. He and others believe that price is sure to drop, too, as more competitors enter the field of VR (though HTC’s upcoming Vive Pre VR headset is actually slightly more expensive at $699).
“I think that in maybe five years, maybe 10, every clinician will have one of these,” Elemento adds. He hopes that his team’s software will be among the first pioneers to make virtual reality attractive to doctors.
To get there, though, the precision medicine software needs to provide a clear advantage to clinicians, Sigaras says. They’re still figuring out just what kinds of information doctors would want to see in virtual reality, and what they prefer to see on a computer or even in hard copy.
The Oculus Rift VR headset is merely a means to an end to this team, which seems to be somewhat platform-agnostic. Sigaras has tested similar software on other virtual and augmented reality platforms including Google Glass, and even smartphone-powered VR headsets like Google Cardboard. He’s also trying to get a Microsoft HoloLens to try his software in augmented reality, too.
In the end, Sigaras and Elemento care most about getting more information to clinicians more quickly and intuitively. “The real question is: How do you make your clinician more productive?” Sigaras says. “We want to enable clinicians to go through an immense amount information at a faster pace.” More often than not, fighting cancer is a race against time, he adds, and if a virtual reality tool can give doctors an advantage in finding the right treatment more quickly, patients will reap the benefits.
