Before trying robot judges, let’s learn from robot referees

Automated rulings in sports can help inform the development of criminal justice tech.

In 2012, the International Football Association Board—the 131 year old rulemaking organization for soccer around the world—said it would allow use of goal-line technology. The decision came after nearly a decade of arguments, bad calls, and angry fans, and is still a point of discussion today.

That type of extended and bitter debate repeats any time the governing body of any sport considers adding technology to the mix. Those constant conversations caught the attention of Meg Jones and Karen Levy, who study law and technology at Georgetown and Cornell, respectively.

Jones says it’s quite tricky to figure out how people feel about the law enforcement technologies she studies, like automatic cameras at traffic lights, police body cameras, and courtroom computer programs. But she found that when you ask people what they think about the use of automated technology to enforce rules in sports, they express much stronger opinions. “We had the realization that people are really, really good at talking about it,” Jones says. “They have very developed and articulate reasons for why they feel the way that they do.”

With that in mind, the pair set out to understand the way different sports made decisions about the use of technology to enforce rules, with the goal of providing a framework to understand the way people feel about automated enforcement in the legal system more generally. They presented the results of that research at the annual meeting of the Society for Social Studies of Science last week. Though the rules of a sport and the laws of society aren’t exactly in the same ballpark, Jones says, the comparison is useful. “We are having the same conversations, so there might be something to learn from sport.”

A sporting chance

The study looked at four sports—football, baseball, tennis and golf—and teased out the ways that each did, or did not, embrace automated systems to enforce the rules.

One of the most important factors in the decision to add or eschew technology, Jones says, was the culture and integrity of the sport.

Football has a very different identity and fan base from golf, for example, and that shows in the way each uses technology. Football embraced the use of instant replay to assist human referees. But in golf, where players call their own fouls, a video replay in the moment would fundamentally change the culture of the sport. “You really take away the honesty, the gentlemanliness of the golfer,” Jones says, “in a way that completely changes what a golfer is.”

All sports aimed to maintain what Jones and Levy call an element of essential predictability, or “sporting chance,” in the enforcement of the rules. Even though technology could make perfect enforcement possible, that’s not quite what fans are after, Jones explains. The drama and adversity generated by bad calls are important aspects of the ritual of watching a game.

“People enjoy watching humans be humans,” she says.

Law and order

At the outset, the goal of this research was to use sports as a way to answer questions about law enforcement. The criminal justice system is much more complicated than an athletic institution, the problems are weightier, and the consequences more dire. But even though it’s not a not a perfect parallel, Ryan Calo, who studies law and technology at the University of Washington, says that the research helps explain how people respond when machines lay down the law.

“They’re actually getting at this really hard question of what it is that bothers us about the ability of machines to engage in perfect enforcement,” he says. “That’s what I love about this contribution.” Imperfect enforcement may in fact be an important element of the legal system, Calo says. He points to jaywalking as an example: it’s illegal in both Seattle and New York City, but you’re way less likely to get busted for it in the Big Apple.

The results of the study, Calo notes, suggest that people value the idea of human judgement. “It gives you a sense that, regardless of whether it may be objectively more fair, there is a meaning in the decision being made by a person,” he says.

For Jones, the research shows how important it is to consider culture and values when integrating technology into enforcement. “You need people sitting down saying, what is the essence of this social activity, and how does integrating any technology into the enforcement of these rules alter, eliminate, or reinforce different values,” she says. “If you ignore culture, you’re going to be sorry.”

The governing bodies of sports are generally good at having nuanced conversations about the values of their institutional structures, Jones says. The legal system might need more practice. “They’re constantly having to ask themselves what their sport is,” she says. “We often don’t do that in different legal contexts.”

Questions about the role of technology in the legal and criminal justice system are critical, says Calo, and sports might help answer them. “It’s about what role systems should play in decisions where people’s lives are at stake,” he says. “And sports can give us some sense of where people’s intuitions law.”