How Facebook Used Science To Design More Emotional Emoticons

In 1872, Charles Darwin published The Expression of the Emotions in Man and Animals, a book that cataloged emotional expressions in humans and their link to the animal world. In the book, Darwin described more than 50 universal emotions. Now Facebook, with the help of a psychologist who studies emotions and a Pixar illustrator, has turned some of the emotions Darwin described in the 19th century into a set of emoticons. The hope: to create emoticons that better capture the vast range of human emotion.

“This all began we were looking at the kind of issues people were reporting to Facebook,” Facebook engineer Arturo Bejar tells Popular Science. “The reports had to do with things Facebook didn’t need to act on, but things people should address–what should happen when you say something that’s upsetting to me or put up a photo I didn’t like?”

Around that time, he met Dacher Keltner, a psychology professor at UC Berkeley who studies emotions and social interaction, and invited him to become a scientific partner with Facebook in early 2012, “getting people to be kinder and more polite to make for more compassionate communication,” as Keltner describes.

They started looking at how compassion research could help Facebook address the kind of interpersonal conflicts the company saw emerge in issue reporting. When people inserted a little more emotion into their messages asking friends to take down photos, Facebook found, the friend was more likely to respond or comply rather than just ignore the message.

So Facebook started thinking about how to add more emotional information to Facebook messaging. “There’s all this communication that happens when you’re talking to someone face-to-face–you can see that they’re nodding and you can see their smile–that is not present when you’re communicating electronically,” Bejar explains. “One of the questions that we asked was, ‘Wouldn’t it be great if we had a better emoticon that was informed by science?'”

Over the last few months, they’ve been working on just that with the oversized cartoon emoticons Facebook Stickers. Within Facebook Stickers is the Darwin-inspired, compassion-research-based set of emoticons called Finch. It’s “an appealing character who appears to think, emote, and communicate,” according to a statement from Matt Jones, a Pixar designer who worked with Keltner (independent of his involvement with Pixar) to illustrate the more emotionally accurate faces.

Sketches

While traditional emoticons can add a little more emotional context to a text-only conversation, they can’t provide much nuance. It can be hard to tell what they mean. Is : | disappointed? Indifferent? Is this : $ your foot in your mouth or are you just spewing money?

We may never be able to replicate the emotional context of a face-to-face conversation with a graphic. A 2007 study of emoticon use in online chatting warned that even though emoticons are meant to inject nonverbal expressions into text-only conversations, they differ because we have to use them consciously and deliberately:

Although it is conceivable that emoticons could become habitual and less conscious over time, it is still not clear how emoticons are interpreted in [computer-mediated communication]: as iconic and unconscious like nonverbal facial expressions or, like wording, as deliberately encoded elements of intentional communication. It is plausible that emoticons have a limited range compared to emotions in real life.

Yet Keltner thought that by incorporating some of the principles from Darwin’s seminal work on emotion, he could add a touch of the richness he felt existing emoticons lacked. “I’m naïve about emoticons because I’ve never sent one in my life, but I’ve looked at them–it’s just missing a lot of important things in our emotional lives,” Keltner says.

Sympathy, for example, can be hard to really get across in traditional emoticon form. “It’s an under-appreciated emotion in Western culture,” Keltner explains. “We now know what it looks like and sounds like because of science. They created this dynamic emoticon that when you see it, it’s really powerful.”

Embarrassed / Sympathetic

He also wanted to be able to express embarrassment. Research has shown that blushing serves an evolutionary purpose, showing that people regret whatever social boundary they’ve just overstepped. “When you show you’re embarrassed to somebody else, it’s like saying ‘I’m sorry.’ It triggers forgiveness and trust,” Keltner explains.

To put more emotional data in one graphic, it had to be dynamic and move, just like a human facial expression. In studies on emotion perception, people can identify emotions better in dynamic faces than in still faces, according to Lisa Feldman Barrettt, a psychology professor at Northeastern University who is beginning research into emoticons. Movement before and after the expression provides a context that helps identify the emotion.

However, she warns that without the context of a broader conversation, it’s difficult to just look at an emoticon and comprehend the emotion it’s trying to convey.

Grateful

“The assumption is they’re going to add information, because somebody can just look at one of them and know what emotion is being signaled, and I think that’s highly debatable,” she says. “On its own, I think some of those are unclear as to what emotion they would be signaling. Facial movements by themselves are not sufficient to signal an emotion.”

There’s potential to roll out more emotions for Finch eventually: Keltner and Jones created a total of 50 emotions together, though the Facebook set so far has only used 16.

While Finch is the most scientific set, the other Sticker sets Facebook currently offers have varying degrees of emotion, according to Bejar. He’s eager to see how people respond to them. “We want to see how people use them to relate to each other,” he says. There’s even the possibility that in the future, Finch could find a voice.