Using breath-analysis software and mouth-movement observations, engineers in Japan have taught a robot how to sing. The divabot, an HRP-4 with a creepily realistic tilting head, blinks and opens her mouth as she croons, even mimicking the facial expressions of the human singer.
Researchers used a real singer as a model, recording her every move as she sang.
Masataka Goto, who leads the Institute of Advanced Industrial Science and Technology’s media interaction group, explains that researchers developed a software program called VocaListener to synthesize notes, and a program called Vocawatcher to analyze a singer’s facial tics as she belts out a tune. The robot’s head therefore follows the roll, pitch and yaw movements of the real singer.
The team even helped the fembot breathe realistically, by modeling the breath sounds of various people.
Like robotic shape-note singing, the robot learns where sounds will emerge. It can reproduce them using Vocaloid software developed by Yamaha.
Masataka says he believes the entertainment industry must embrace robots if they are ever to become widely accepted.
“We hope the entertainment industry will be able to make widespread use of robots,” he says.
Lady Gaga, I hope you’re listening.