Manuel Cebrian and his team just won the Tag Challenge, a State Department-sponsored competition to find five fake jewel thieves in five countries within 12 hours. Team Crowdscanner featured some of the same people who won DARPA’s Network Challenge, a strange hunt for red balloons placed around the country. As with the balloon challenge, some team members thought Tag Challenge was an impossible task — and in some ways it might have been, because no one found all five “suspects.” But three out of five isn’t bad, especially given the complexity of the task.
Cebrian, a research scientist at the University of California - San Diego, was on the team that won the balloon challenge while a postdoc at MIT back in 2009. Part of the challenge lay in filtering useful information from noisy social networks, as well as building a trustworthy network of tipsters and searchers. This time, the team leader was Iyad Rahwan, associate professor and head of Computing & Information Science at the Masdar Institute in Abu Dhabi. Cebrian was the only team member based in the U.S., he said.
The suspects were hired actors in Washington, D.C., New York, London, Stockholm, and Bratislava. Mug shots of each suspect were released at 8 a.m. local time, and participants had 12 hours to track them down, using only the photograph and each suspect’s back story posted on the Tag Challenge website. For instance, the D.C. suspect, "Teresa Bay," was described as a coffee addict who was arrested for counterfeiting Starbucks gift cards. She was found in a Starbucks.
Team Crowdscanner couldn’t find the suspects in London or Stockholm, despite broad coverage in Europe, Cebrian said — but things fared better when the competition moved to the U.S.
The Tag Challenge team leadership announced the team’s win via Twitter over the weekend, but have not released any other details yet. Cebrian said he was still waiting for data so he can start piecing together how it all went down, and which methods worked better than others. Meanwhile, he shared some insight with PopSci about what happened.
PopSci: How did you get involved in this?
Manuel Cebrian: I study network science, which we also call computational information science — how information spreads in large patterns, especially internationally. We almost never have any chances for gathering this type of information. People talk a lot about social networks, but from a scientific standpoint, we are really just starting to study them. We have no data about how they organize, how they get recruited.
The balloon challenge and this competition are not only excellent places to compete and win — we share the money, we don’t make any money on this — but we collect a lot of data, so it’s a very unique opportunity. It helps us scientists really understand things like the Arab Spring and the Occupy protests. The only way we have to collect this data is by participating in these challenges.
PopSci: So what did you do to build up Team Crowdscanner?
MC: We kept the same incentive structure we did in ’09 with the balloon challenge, but we simplified it a little bit so people could make money by either finding suspects, or recruiting the person who found the suspects. We still have to determine how we divide the prize now. The idea is you would get $500 for finding them, and $100 for recruiting the person who found them. We also pay some money for spreading the word. If you just spread the word, and the people you recruited didn’t find them, you would still make $1 for every person you recruited, for the first 2,000 people.
The goal is to kick-start it, to get to what we scientists call the tipping point.
We found people on two levels. We tried to target mainstream media sites — we talked to journalists, tried to get some articles started in the hopes that things will somehow propagate. Somebody might read your tweet, and send it to 10 or 20 friends. Then we ourselves spread it; we have a few hundred friends on Facebook and so on. So we can gather data at both levels. We can see how successful are journalists, mainstream media sites, versus normal people like us.
Mainstream media sites are always more effective; the question is how much. We believe mainstream media sties were at least 100 times more effective in this competition than individuals — between 100 and 1,000 times more effective.PopSci: How did people sign up?
Then if you see one of the suspects, you come back to our website and use your login and upload a picture. We even deployed an Android app where you could just take a picture, and immediately upload it to our site.
PopSci: When did this all start, and how did you cover all these different time zones?
MC: The team leader and the developers were based in Abu Dhabi in the United Arab Emirates. For those guys, it started at 10 a.m. We had some people at Southampton University in the UK, and I am in California, so we were covering all the time zones nicely. It seems trivial, but you need to have someone in different time zones. If the other side is really tired and it’s 3 a.m. for them, I am still fresh and I can verify what people were submitting.
PopSci: How long did it take to find your three suspects?
MC: Well, we had given up on Bratislava. They don’t have huge Internet penetration, they don’t even speak English, most of the population, so we had given up. But three hours into it, we had computer scientists from a university in Bratislava that heard about our team through Slashdot, and he saw one of the suspects and sent us a picture.
We were amazed that we got to find the first person in a relatively short amount of time in Bratislava. He wasn’t hiding, but he wasn’t in the open doing ridiculous things either. He was acting like a normal person.
Then around 1 p.m. New York time, we found the suspect in Grand Central Station. That was from a grad student from Harvard who found out about us through some mailing list. So again, the social network was working very well in that case.
PopSci: How will you determine how to pay all the participants?
MC: The incentives are designed to maximize the payment to the people who helped us. We didn’t find the whole set of suspects, though. The money should go mostly to the people who took the picture, then to the people who recruited them, and also to the people who spread the word.
In the meantime, we have been gathering all sorts of information — every single mention of our team, we are tracking it. It requires some fine analysis to figure out how much we need to pay everyone. It is harder for the spreaders.
PopSci: What surprised you during this competition? What did you learn?
But you know, three out of five is an interesting number. It means that this is not trivial. If it had been five out of five, maybe [that means] it would have been easier. For me, it means we need to be very careful about how much power do we attribute to social networks. We couldn’t find the five people; we struggled to find three. So can social networks do something incredible? Yes. Can they do whatever you want them to do? Absolutely not. Social networks are not a magical bullet; we need to understand the limits.
We will study the data to find out why we didn’t find the people in London and Stockholm — we want an itinerary of the suspects to compare it with our data, and so on. We want to create a model for how this unfolded, and find out were we lucky or unlucky. Until we have that kind of understanding, we don’t know if we got really lucky, and three is a lot, or if we got really unlucky and missed the guys in London and Stockholm by one hour, or by one recruit. We are really excited to get the whole picture.