Algorithm Can Tell If Your Smile is Happy or Born of Frustration

You may not notice it — at least not as much as you notice when you smile sarcastically — but you smile when you’re frustrated. It’s more like a surprised grimace than a happy grin, but the difference is subtle. So subtle that humans can hardly detect it, actually — but a computer can. New research with smile-detection could help people interpret others’ expressions, including people with autism, according to scientists at MIT.

Students in MIT’s Media Lab are hoping to improve face-to-face communication, and working with computers to break down the cues that differentiate emotions. Grad student Ehsan Hoque and his colleagues recorded volunteers’ expressions in a range of situations.

First, people were asked to act out expressions of happiness or frustration; second, they watched a video of a cute baby and then filled out an arduous form, only to have its contents deleted when they hit “submit” — an exercise in frustration. Then Hoque and colleagues compared the facial expressions. Ninety percent of the frustration fakers did not smile, but when they were genuinely frustrated, they did, he found. There’s little difference in the actual smile, but its progression is what counts. A delighted smile grows slowly, building gradually to a full-on grin. A frustrated smile erupts instantly but fades fast.

When it comes to smiling, timing can be everything, Hoque tells MIT News. Remember Herman Cain? MIT also notes Gordon Brown, the former British prime minister, whose grin was supposedly phony because of its odd timing.

A more nuanced view of smiling could be used to train computers to monitor human emotions more effectively, the researchers say. That could help computers determine what we want from them or when we’re frustrated or overworked, perhaps. It could also help people who have difficulty ascertaining emotions, such as autistic patients, they say.

The work appears in IEEE Transactions on Affective Computing.

MIT News