MIT Harnesses Online Crowds to Beat Darpa Balloon Challenge in Just 9 Hours

The Pentagon's DARPA agency wanted to know how to filter trustworthy information from social networks; MIT had the answer

Groups of friends and strangers spent more than a month preparing for perhaps the greatest social networking competition in history. All wanted to be the first to find 10 red weather balloons scattered across the continental U.S. on December 5, and claim a $40,000 prize from the Pentagon’s DARPA agency.

Riley Crane, a postdoctoral researcher in the Human Dynamics group at MIT’s Media Lab, only heard of the DARPA Network Challenge a few days before balloon D-Day, when a friend from Switzerland emailed the info and jokingly asked which MIT team would solve the case. Crane forwarded the message to some colleagues and asked if anyone had a $40,000 idea to claim victory.

MIT’s team went on to win the DARPA challenge within nine hours of the first balloon launch this past Saturday. Success depended upon figuring out how to filter trustworthy information from thousands of misleading tips that represented spam or noise.

“It became clear early on that there were a lot of people submitting false information,” Crane told PopSci. “That was definitely the biggest challenge.”

Finding ways to get around that problem was DARPA’s main interest in setting up the competition. Online social networks teem with unverified information from millions of interconnected friends, acquaintances and strangers. DARPA wanted to see how perfect strangers loosely connected by social networks could work together toward finding truthful nuggets of info, despite a swarm of disinformation.

“The general question was the establishment of trust in adversarial situations in social networks,” said Peter Lee, director of DARPA’s Transformational Convergence Technology Office.

Lee spent Saturday standing next to an 8-foot red weather balloon tethered inside Union Square in San Francisco, California. DARPA representatives also accompanied nine other balloons in public locations across the U.S., all visible for team spotters or random strangers to see.

Competitors took many different approaches to sift through a cascade of online tips about the balloons. Community website Fark used a small group of coordinators to collect info and deploy spotters who could check up on suspected balloon locations. The Open Red Balloon Project tried to provide intelligent guesses about balloon locations to netizens, so that a swarm of submissions to DARPA’s website might hit upon the correct locations for all 10 balloons.

Much of the action took place invisibly in the form of private info submissions at different team websites. But Twitter provided a sense of frothy excitement on December 5, as people scrambled to pin down exact locations for tempting tips or pictures, tweeted encouragement, and occasionally admitted to the overwhelming nature of the contest.

“The amount of disinformation on the Red Balloon challenge is frustrating,” said a Twitter user named “floppyhead” late Saturday morning.

Teams were split as to how best to motivate their participants. Participants such as Harvard’s Project Red Balloon promised to donate all the winnings to charities for causes such as AIDS research. Others, such as, dangled shares of DARPA’s prize money as the incentive for people to provide correct info.

The MIT researchers created incentives for people to sign up trusted friends or acquaintances who might have a chance of spotting balloons. They split all the DARPA prize money per correct submission: $2000 per balloon to the first person who sent in correct coordinates, $1000 to the person who invited them, $500 to whoever invited the inviter, and $250 to whoever invited them. The last $250 went to charity.

Crane’s MIT team hit upon their solution for obtaining trustworthy information in a series of impromptu group huddles. The group eventually consisted of Crane and another postdoc, Manuel Cebrian, as well as grad students Galen Pickard, Wei Pan and Anmol Madan.

That “economics meets epidemics” approach differs from a pyramid or Ponzi scheme that rewards only the first people to sign up, according to Crane. It encouraged people to feel part of a trusted team, and to help themselves as well as science and charity.

“Your reward was directly based on whether you were responsible for getting us key information,” Crane explained. “It didn’t matter if you were the first or millionth to sign up. What mattered was whether you found a balloon, or if you signed up someone who found a balloon.”

Crane spent two sleepless nights coding the MIT team’s website, and also made continuous changes on the day of the contest based on feedback from submitters. His team’s efforts paid off handsomely when DARPA announced MIT’s victory a mere nine hours after the balloons went up.

The MIT team has not revealed exactly how they verified the correct balloon locations among all their submissions, but DARPA has already made plans to debrief them and the other 49 top teams that took part in the challenge. The agency received about 800 submissions.

“There were a number of very competitive teams,” DARPA’s Lee told PopSci. “It was a really exciting down-to-the-wire finish.”

For his part, Crane expects to use his motherlode of data for more than just social-networking research. The MIT system might someday apply to real-world problems such as helping police track down missing children or criminals.

“We’re scratching the surface of the potential to mobilize human networks on a scale that would not have been possible five years ago,” Crane said.