Mask-Bot Displays 3-D Human Faces With a Plastic Superstructure

A new robot face can display a realistic virtual visage from any angle, making telepresence somewhat less creepy by using actual human features. Mask-Bot, as it’s known, displays three-dimensional heads on a transparent plastic mask.

It uses a “talking head animation engine” to filter face emotions according to the emotional feel of spoken words, so the faces accurately reflect the speakers’ expressions. It could be used for video conferences or other situations where other telepresence ‘bots may not be exactly what you want to look at, according to its designers. “With Mask-bot, you can create a realistic replica of a person that actually sits and speaks with you at the conference table,” said Takaaki Kuratate of the Institute for Cognitive Systems at the Technical University of Munich.

Mask-Bot is designed in the spirit of other 3-D masks, most notably the singing busts at Disneyland’s Haunted Mansion. But it uses on-board rear projection instead of Disney’s front-end projection. The projector’s high-compression fish-eye lens is 12 centimeters from the mask, which is coated with luminous paint to better project the image. It’s bright enough to work in daylight, the researchers say. Using only a 2-D photograph, algorithms can extrapolate a properly proportioned 3-D image of a person’s face and project it onto the mask, according to a news release from TUM.

The weirdest part is the animation engine, which uses a database of facial emotion data that was previously collected using a motion-capture system. The system selects a facial expression that best matches the sound of a person’s voice, and extracts facial coordinates from each expression, which it can then assign to any projected face. “Emotion synthesis software delivers the visible emotional nuances that indicate, for example, when someone is happy, sad or angry,” TUM explains. It can convert text in English and Japanese (German is coming soon) into audio, using a male or female voice.

Mask-Bot was developed by an intercontinental partnership at TUM and Japan’s National Institute of Advanced Industrial Science and Technology, which has been involved in some of our favorite humanoid robot projects. A second-generation version will incorporate the mask into a fully bodied mobile robot, according to TUM. “These systems could soon be used as companions for older people who spend a lot of time on their own,” Kuratate says. Maybe … or perhaps just to personify Skynet once it becomes aware.