Pixel-Perfect Digital Clones Are Ready For Their Close-Ups

Sit back while The Matrix Reloaded boots up the next generation of virtual filmmaking.

by Illustration: Jason Lee; Matrix: Courtesy Warner Bros.

THE "U-CAP" RINGER
To create three-dimensional photo-realistic faces, each of Reloaded´s lead actors performed inside the "universal capture" setup at Fox Studios in Sydney, Australia. The studio arrayed five Sony CineAlta high-definition cameras around the actors´ faces (1) and zoomed in on hundreds of subtle features such as enlarged pores, wrinkles and whiskers (2). Proprietary vision algorithms then processed the data to reconstruct precise replicas of the face (3).
Illustration: Jason Lee; Matrix: Courtesy Warner Bros.

Perhaps the only thing more outlandish than the rumored $300 million budget fueling the wildly anticipated Matrix sequels, The Matrix Reloaded and The Matrix Revolutions, is the films' new jaw-dropping visual effects wizardry. When Reloaded opens in theaters on May 15, viewers will spy the most realistic computer-generated faces ever made, claims visual effects supervisor George Borshukov. His team
at Esc Entertainment in Alameda, California, has spent the past three years designing digital mugs that precisely mimic the faces of Keanu Reeves, Carrie-Anne Moss and Hugo Weaving, the sequel's stars.

"Creating a believable synthetic face is the ultimate challenge in computer graphics," says Borshukov. That's because humans are trained from birth to make and recognize more than 10,000 complex facial expressions, most of them too subtle to be accurately simulated in standard computer-generated renderings.

To create photo-realistic digital copies of the actors' faces, Esc had to first invent an ultraprecise facial mapping technique, dubbed "universal capture." Unlike standard motion capture techniques, in which a camera records facial movements by tracking painted-on dots, universal capture uses five Sony CineAlta high-definition digital cameras arrayed around a live, line-reading actor. The cameras zoom in and track minute facial imperfections, like pores or whiskers. The 3-D information then streams from the cameras
(at about one gigabyte per second) into a proprietary suite of computer programs that extract the actors' facial expressions, stretch virtual skin and grow synthetic hair. The results are impressive: In an epic fight scene, 100 clones of the trilogy's main villain, Agent Smith (Hugo Weaving) battle Matrix hero Neo (Keanu Reeves). Whose face is real is anyone's guess. "I'll be curious to see how many people realize that some faces in the movie are 100 percent computer generated," says Borshukov. "It's going to be a very interesting
psychological experiment for the audience."