Robots photo
SHARE

In a small, sparsely furnished room, a young boy in a black T-shirt backs himself into a corner. He’s cautious. Cameras capture his movements, and microphones record every sound. But this doesn’t intimidate him; he doesn’t even seem aware that he’s being observed. His mom, sitting nearby, is not the object of his focus either. Brian (his name has been changed here to protect his privacy) is autistic, and he’s staring across the room at a two-wheeled, gray, humanoid robot with big, cartoonish eyes. The machine, Bandit, is roughly Brian’s size, and it has been trying to engage him by slowly rolling toward him.

Bandit uses infrared sensing and cameras to calculate Brian’s position. Seeing that the boy is backing away, the robot tries a different approach. It stops moving and makes a “come-here” gesture, waving him closer. It works. Brian approaches and then stands alongside Bandit, shoulder-to-plastic-shoulder. Bandit stops moving, and Brian backs off. The boy is like a boxer sizing up an opponent. Finally, emboldened, Brian steps up to the robot and leans his face toward it, curious and confident. For the researchers observing the interaction through a two-way mirror in an adjoining room, this small gesture is an encouraging sign. The boy is warming up to the machine, and that’s the point.

This unusual pair is part of a research initiative at the University of Southern California to build robots sympathetic and sensitive enough to serve as both therapists and playmates to kids with autism. Bandit is programmed to perform simple facial expressions and movements, and researchers are working to give the robot the ability to make complex decisions in response to the child’s behavior. This way, Bandit and robots like it could draw socially detached kids into simple games, like Simon Says or hide-and-seek and, ultimately, social activities with people. As USC computer scientist and project leader Maja Matari´c explains, “The robot is a catalyst for social interaction.”

In its current form, Bandit has only rudimentary social skills. For instance, it cannot yet understand speech; a researcher in the other room must command the robot to respond if the child speaks to it. But early results are encouraging. Matari´c’s team has conducted experiments similar to the interaction between Bandit and Brian with 14 other autistic children, most between five and nine years old. Some of the kids were incapable of speech, while others could talk in full sentences but were prone to physical tics like hand-flapping or obsessions with moving objects like trains. The interactions lasted on average about five minutes—not long enough to produce permanent behavioral changes—but many of the children became more sociable, and more vocal, with a robot in the room.

That may seem surprising, since robots are hardly known for warmth and sociability. Yet there is increasing evidence that kids with autism respond more naturally to machines than they do to people. Psychologist Simon Baron-Cohen, the director of the Autism Research Center at the University of Cambridge in England, along with other autism experts, believes that robots, computers and electronic gadgets may be appealing because they are predictable, unlike people. You can pretty much guess what a computer is going to do next about 90 percent of the time, but human interactions obey very few entirely predictable laws. And this, Baron-Cohen explains, is difficult for children with autism. “They find unlawful situations toxic,” he says. “They can’t cope. So they turn away from people and turn to the world of objects.”

Perfecting a robot that bridges the gap will call for advances in almost every facet of modern robotics: artificial intelligence, machine vision, mechanical and electrical design, signal processing. Among other requirements, robot therapists must be subtle enough to respond to their patients without scaring them away, and smart enough to identify and react to autistic behaviors. For instance, if a child starts rocking back and forth, which can be a sign of increased anxiety, the robot should be able to recognize the motion and modify its behavior until the child appears more comfortable. “Yes, there’s work to be done,” Matari´c acknowledges. “But it’s doable. What I want is a robot that can sell for the price of a laptop, a decade from now.”

Computer scientist Maja Mataric's humanoids are capable of expressive gestures and facial expressions.

Mother of Invention

Computer scientist Maja Mataric’s humanoids are capable of expressive gestures and facial expressions.

Help Wanted

Autism is commonly defined as a developmental disorder characterized by social and communication difficulties and a tendency to fixate on objects and details. A series of standard assessments typically determines the diagnosis. In one common test, the Autism Diagnostic Observation Schedule, an examiner interacts with a child for 30 to 45 minutes, conducting playful activities, such as games with bubble blowers, and basic communication exercises. The examiner looks for blank expressions, lack of empathy, an apparent inability to respond to one’s name, and other telltale behaviors.

Examiners record, tally, and analyze as much as possible, but a diagnosis is rarely clear-cut. Autism is difficult to precisely define, in part because the condition is a spectrum disorder. It encompasses a wide range of people—up to 1.5 million in the U.S., according to the Autism Society—from adults who require constant care to independent and professionally successful intellectuals. Among children, the prevalence statistics have jumped from one in every 150 a few years ago to one in 110 kids today. Experts debate the cause (the increase may simply stem from a greater awareness of the disorder) but agree that there is now a shortage of qualified therapists. And therapy is work. To achieve results, a single child can spend 40 hours per week moving from one specialist to the next. Therapists might use music therapy to teach the child how to take turns; encourage artistic expression; use role-playing and drama therapy to teach social skills; and practice looking and pointing at a single object, and then discussing it, to work on communication and shared attention.

Each of these methods requires specialized training, and there aren’t enough experts. The size of the gap isn’t yet clear, but “without question, there are currently more people with autism than there are therapists who know how to assess and help them,” says Matthew Goodwin, the director of clinical research at the Massachusetts Institute of Technology’s Media Lab, where he’s building wearable devices to gauge stress levels in autistic people.

Yet autism is treatable, and with therapy, children can acquire basic social skills, a degree of independence and, in some cases, self-sufficiency as adults. Regardless of where the child starts on the spectrum, researchers agree that the training is critical early in life, when the brain is more flexible and open to change. “Every day you’re not taking advantage of brain plasticity, you’re reinforcing things that are potentially difficult to undo later,” Goodwin says. Hoping to do what’s best for their children, families in outlying, rural areas can drive for hours or even across the country to urban centers, where it’s easier to find care. Some simply pack up and move.

Automated therapists would not only increase the amount of available therapy but would also make it available wherever a family happened to live. Matari´c envisions a day when parents will be able to bring Bandit-like ‘bots home right after diagnosis for round-the-clock therapy. They could customize them, adjusting the facial expressions and body language depending on the needs or comfort level of the child. Or the robot could adjust itself, gradually becoming less predictable to slowly increase a child’s tolerance for the uncertainty of real social situations. They won’t replace human therapists, but robots could offer an added advantage by giving parents and caregivers a systematic way to track improvements or setbacks in their child’s therapy.

To that end, MIT electrical engineer Rosalind Picard has co-founded a company called Affectiva to develop sensor wristbands that record movement, temperature and perspiration. Incorporated into child-robot therapy sessions, the wristbands could pick up hidden physiological cues and inform robots if the child might be getting anxious.

Likewise, a ‘bot could begin recording audio and video anytime the child’s vitals began to change, and flag that clip for analysis. Later, a therapist or parent could study the videos to see if there was anything in particular that might have caused the child to grow uncomfortable.

Mataric plays a mirroring game with humanoid Bandit.

Robot See, Robot Do

Mataric plays a mirroring game with humanoid Bandit.

The Man-Machine Divide

Matari´c used to research military applications, but as she raised a family, she decided that she “didn’t want to be the mommy who builds killer robots.” And Bandit does not resemble a killer in the slightest. When I first encountered the robot at Matari´c’s USC lab last July, it was powered down and standing still. Compact arms with bulging triceps hung at its side. Its rounded, hairless gray head resembled a mannequin’s, and its torso sat on a base stacked with a Linux computer, speakers, a router, batteries, and an electric motor to drive the wheeled “legs.” Its small stature, playful face, and moveable, rubber-covered wire lips all gave it a harmless, come-play-with-me charm. For additional kid appeal, the group attached one of the bubble blowers used in diagnostic tests to the base and added a clicking noise that many autistic kids find soothing.

I was transfixed by Bandit’s humanlike face and silvery eyes, but researchers have been careful to avoid making it look too human. A realistic face could be off-putting, since autistic children often find other children intimidating. Matari´c wanted the difference to be clear. “The robot is not a substitute for other people,” she says. “It’s a catalyst. It’s not like the movie AI. I didn’t want to do the perfect boy who’s a robot.”

Later that day, USC computer scientist Pierre Johnson invited me to observe Bandit perform a mirroring game similar to the ones used by human therapists. The researchers were still refining its abilities, and Johnson wanted to see how well its cameras and vision software tracked the movements of his arms. The idea is for Bandit to eventually be able to perfectly imitate a child, spreading its arms wide in response to the child doing the same, lifting its arms over its head, raising one hand but not the other, and so on. The game could serve as a more entertaining way of teaching eye contact or even just engaging with another person.

Most of the formal child-robot experiments take place at a hospital in Los Angeles (the name is withheld here to protect the children’s privacy). A therapist begins with a brief introduction showing what the robot can do—here’s how the head and arms move, how the bubble blower works. Then each child and parent are left alone with Bandit for roughly five minutes. The therapist and other experts observe the interaction through the two-way mirror while cameras and microphones record everything. Afterward, researchers analyze the recordings, noting, for instance, how long and how frequently the child looks at the robot, how much time he spends within an arm’s distance of it, and how often he speaks to the robot and to his parent.

The results of such tests have verified some of the USC group’s assumptions and challenged others. As hoped, the higher-functioning autistic children were more vocal when the robot was in the room and, importantly, they engaged their parent in the interaction. They weren’t closing themselves off with the robot; they were actually more sociable than usual. At the same time, some of the lower-functioning children rejected the robot. “We had some kids that would go right up to the robot. They were incredibly intrigued,” Matari´c says. “Then you get kids who kind of come in and look at the robot and approach it slowly. Then there are children who are just not even coming near.”

To demonstrate this range of responses, computer scientist David Feil-Seifer, the other lead on the project, showed me a series of videos. In one clip, a lanky, dark-haired boy approaches Bandit instantly. He stands in front of the robot, kneels down, and presses the toy buttons on its base, triggering Bandit to respond with “Woo hoo!” Bandit is constantly turning to face the boy. Pulling an imaginary train whistle, the boy asks, “Robot, can you make a train sound for me? Can you make a train sound?” He tells the robot, “Say ‘choo-choo!’ “

For this boy, Bandit is clearly a draw, and he also doesn’t forget his mother. He turns toward her repeatedly, a clear sign of an important social skill called joint attention—which, in this case, is an awareness that he and his mother are focused on the same thing. Lack of this skill is often one of the first signs that a child may be autistic, and although the boy didn’t learn the skill from Bandit, the robot did bring it out. It inspired the boy to be sociable.

In another video, however, Bandit is a repellant. A blond boy is desperately trying to avoid the robot. The boy’s hands are pulled up inside the cuffs of his shirt. His father sits in the corner, watching, but the boy is no more comfortable. In a high-pitched voice, he says, barely above a whisper, “Go. . .go. . .go. . .” The robot, trying to encourage play, moves closer, but the boy presses himself against the wall.

Whereas human therapists experiencing this kind of resistance could easily recalibrate their approach, perhaps speaking more softly or moving more slowly, robots still require the skilled hand of engineers to fully refine their behavior. Most recently, when researchers noticed that the motor that drives Bandit’s wheels emitted a high-pitched whir that upset some of the kids, they were forced back to the lab to figure out how to reduce the noise. (A software upgrade did the trick.) In another instance, the robot frightened a boy by wheeling too close and nodding its head, prompting researchers to look for ways to make Bandit capable of knowing when it’s out of sync with the patient, just like a human therapist would. “Ideally the robot would either stop moving, stop making noise, or try to do something soothing,” Feil-Seifer says.

The group is also working with signal-processing experts at USC to develop technology that would interpret facial expressions, gestures or changes in tone, allowing a robot to determine on its own whether the child is happy, upset or frustrated. “One of our challenges is not just recognizing what the kid is doing,” says USC electrical engineer Emily Mower, “but how they’re doing.”

Unlocking the Potential

Ten years might sound like a long time to perfect a robot that’s already interacting with kids. Indeed, Matari´c says they could probably make a consumer version of Bandit in far less time, but it would cost thousands of dollars, and that would subvert one of their central goals. Bandit needs to be affordable, costing no more than $1,000, Matari´c says, because families with autistic children already have enough expenses. Yet this price goal adds certain constraints to the design process. Take that loud motor, for example. Matari´c’s team could easily switch to quieter motors, but these would be far more expensive.

For now, the USC team is busy trying to broaden Bandit’s social skills and enhance its ability to read and react to a child’s mood. They are also expanding the scope of their trials. This summer, the group will begin a larger study, after which the researchers will have a much clearer picture of exactly how effective the robot can be in helping children with autism.

Matari´c is cautiously optimistic—as a mother herself, she doesn’t want to give false hope to parents of autistic children—but she does occasionally drop her reserve. She recalls one child, a high-functioning nine-year-old autistic boy who struggled to communicate and interact with others. Bandit seemed to change that. Playing with the robot, the boy was more chatty and interactive with his mother. But as he tried to involve Bandit in a game of tag, he became frustrated. The robot didn’t understand him—the scientists hadn’t programmed in the ability to play this game. When the boy realized that Bandit wasn’t going to comply, he stunned the observers by saying, “Now I know how my teachers feel.”

Matari´c was astonished. This was totally unexpected behavior. Even the boy’s mother was surprised. Empathy is one of those skills autistic children typically lack; this boy wasn’t supposed to be aware of his teachers’ frustration. “That’s a profound level of self-understanding and introspection, and if these kids have it, it’s not coming out in their interactions with other people and other kids,” Matari´c says later. “To have it come out with the robot is fantastic. It’s unlocking all this great potential that the kids have.”

Contributing editor Gregory Mone’s new novel, FISH, came out last month.