The Quest to Read the Human Mind

If a few very smart neuroscientists are right, with enough number crunching and a powerful brain scanner, science can pluck pictures—and maybe one day even thoughts— directly from your brain

By Lisa Katayama Posted 02.09.2010 at 10:09 am 139 Comments

The Mind Reader Coherent Images

It’s after dark on a warm Monday night in April, and I’m lying face-up in a 13-ton tube at the Henry H. Wheeler, Jr. Brain Imaging Center at the University of California at Berkeley. The room is dimly lit, and I am alone. A white plastic cage covers my face, and a blue computer screen shines brightly into my eyes. I’m here because a neuroscientist named Jack Gallant is about to read my mind. He has given me strict instructions not to move; even the slightest twitch could affect the accuracy of what he’s about to do. As I stare straight up, I notice an itch on my thigh. Don’t scratch it, I tell myself. I try to keep my thoughts blank as the beeping gets faster and the fMRI machine—the scanner that will detect changes in blood flow in my brain—powers up.

Gallant assures me that the random thoughts in my head will not affect his results. Today he’s just concerned with what I see and how that registers in the visual cortex, a region at the back of the brain that processes what my eyes take in. It doesn’t matter that I’m thinking about what to eat for dinner, or that I’m worried about getting a parking ticket on Oxford Street. The only important thing, he says, is for me to keep as still as possible, and soon he’ll have enough information to re-create the pictures I’ve been staring at without ever having seen the images himself.

For the past 10 years, Gallant has been running a neuroscience and psychology lab at Berkeley dedicated to brain imaging and vision research. He’s one of a few neuroscientists in the world on the verge of unlocking the key to mind reading through brain-pattern analysis using magnetic resonance scans and algorithms. By showing me a series of random photographs and evaluating fMRI readings from my primary visual cortex, Gallant says his technique can reconstruct imagery stored in my brain. His current method takes hours of analysis, but his objective is to hone the technology to the point where it can deduce what people are seeing in real time.

If successful, it could influence the way we do just about everything. Mind-reading machines could help doctors understand the inner worlds of people with hallucinations, cognitive disabilities, post-traumatic stress disorder and other impairments. Judges could use them to sneak a look into suspects’ brains by having them reenact the experience and reading their visions. Such machines could also determine whether someone using the insanity defense is faking it, or whether someone claiming self-defense truly feared for his life. On the flip side, the technology raises serious ethical concerns, with critics worrying that it could one day make our private thoughts vulnerable to snoops and hackers.

I ponder all this as I lie motionless in the brain scanner, staring straight ahead while Gallant and two of his lab researchers flash several dozen photographs in front of my eyes, a few seconds at a time. I see sheep grazing in a meadow, a rock formation, a pond and a profile of a guy who looks like Einstein. I’m not actually supposed to be looking at these pictures—my job is to stare at the white dot in the middle of the screen. “Seeing” doesn’t happen entirely in the conscious realm, Gallant explains. The visual cortex works like a camera, automatically absorbing information through the retina and registering the imagery in the brain.

Ten minutes feels like an eternity, but finally the fMRI announces the conclusion of its program with another loud beep. The researchers remove me from my bind and escort me to the control room, where a giant monitor is displaying 30 scanned images of my brain from different angles. I see bunches of white squiggly lines and light gray V shapes inside rows of gray circles. “That’s it? That’s my brain?” I ask, my head foggy from having tried so hard to stay still. It surprises me that all the goings-on in my mind can be reduced to a bunch of geometric shapes. Gallant tells me that brain activity is basically just a bunch of neurons firing—an estimated 300 million in the primary visual cortex alone, according to the latest research.

To help make sense of the shapes, the brain scanner divides them up into a grid of three-dimensional cube-like structures called volume pixels, or voxels. To me, each voxel looks like a random mix of whites, grays and blacks. But to Gallant’s computer model, which can see more-precise data in those shades, the voxels are a meaningful matrix of zeroes and ones. By crunching this matrix, it can transform the shapes back into a remarkably accurate rendering of the Einstein Guy or the grazing sheep. Gallant and his team didn’t have time to generate enough scans of my brain to make their algorithm work, but they showed me some convincing results from other volunteers. “It’s not perfect,” says Shinji Nishimoto, one of Gallant’s postdocs, “but we’re getting pretty close.”

As I leave the lab, my thoughts secure in my head, I feel a bit uneasy knowing that they may not stay that way for long. Gallant’s “neural decoding”—a term he prefers to “mind reading”—is getting faster and more sophisticated all the time. In fact, last October, his lab managed to re-create entire video clips just by analyzing the brain patterns of people watching them. In one example, a reconstructed video of an elephant walking through the desert shows a blotchy Dumbo-shaped mass plodding across the screen. The fine details are lost, but the rendering is nonetheless impressive for having been pulled from someone’s brain. And it’s not just Gallant who’s making progress. Using similar technology, other researchers are unlocking memories and dreams.

Beyond the fuzzy realm of the paranormal, mind reading could simply be a question of having the right tools. “As long as we have good measurements of brain activity and good computational models of the brain,” Gallant wrote in a supplement to a paper he published in Nature in 2008, “it should be possible in principle to decode the visual content of mental processes like dreams, memory, and imagery.”

fMRI, How It Works: Your brain generates a unique pattern of activity for every image it sees. By analyzing thousands of fMRI scans [right], software can match patterns in your brain to specific objects. Kevin Hand

What's on your Mind?

Remarkably, scientists can predict with near-perfect accuracy the last thing you saw just by analyzing your brain activity. The technique is called neural decoding. To do it, scientists must first scan your brain while you look at thousands of pictures. A computer then analyzes how your brain responds to each image, matching brain activity to various details like shape and color. Over time, the computer establishes a sort of master decoding key that it can later use to identify and reconstruct almost any object you see without the need to analyze the image beforehand.

Previous Article: Proof of Life in Three Martian Rocks May Come This Year

Next Article: Bulletin of Atomic Scientists Turns Back Doomsday Clock, Gives Us An Extra Minute To Live

139 Comments

Subscribe Now

What's on your Mind?

139 Comments

Popular Tags

Science

Photo Galleries

Science

Innovation Challenges

Popular on Popsci

Most Viewed

Science

Most Emailed

Science

Most Commented

Science

Today on PopSci.com

Footer Menu