Manuel Cebrian and his team just won the Tag Challenge, a State Department-sponsored competition to find five fake jewel thieves in five countries within 12 hours. Team Crowdscanner featured some of the same people who won DARPA’s Network Challenge, a strange hunt for red balloons placed around the country. As with the balloon challenge, some team members thought Tag Challenge was an impossible task — and in some ways it might have been, because no one found all five “suspects.” But three out of five isn’t bad, especially given the complexity of the task.
Cebrian, a research scientist at the University of California – San Diego, was on the team that won the balloon challenge while a postdoc at MIT back in 2009. Part of the challenge lay in filtering useful information from noisy social networks, as well as building a trustworthy network of tipsters and searchers. This time, the team leader was Iyad Rahwan, associate professor and head of Computing & Information Science at the Masdar Institute in Abu Dhabi. Cebrian was the only team member based in the U.S., he said.
The suspects were hired actors in Washington, D.C., New York, London, Stockholm, and Bratislava. Mug shots of each suspect were released at 8 a.m. local time, and participants had 12 hours to track them down, using only the photograph and each suspect’s back story posted on the Tag Challenge website. For instance, the D.C. suspect, “Teresa Bay,” was described as a coffee addict who was arrested for counterfeiting Starbucks gift cards. She was found in a Starbucks.
Team Crowdscanner couldn’t find the suspects in London or Stockholm, despite broad coverage in Europe, Cebrian said — but things fared better when the competition moved to the U.S.
The Tag Challenge team leadership announced the team’s win via Twitter over the weekend, but have not released any other details yet. Cebrian said he was still waiting for data so he can start piecing together how it all went down, and which methods worked better than others. Meanwhile, he shared some insight with PopSci about what happened.
PopSci: How did you get involved in this?
Manuel Cebrian: I study network science, which we also call computational information science — how information spreads in large patterns, especially internationally. We almost never have any chances for gathering this type of information. People talk a lot about social networks, but from a scientific standpoint, we are really just starting to study them. We have no data about how they organize, how they get recruited.
The balloon challenge and this competition are not only excellent places to compete and win — we share the money, we don’t make any money on this — but we collect a lot of data, so it’s a very unique opportunity. It helps us scientists really understand things like the Arab Spring and the Occupy protests. The only way we have to collect this data is by participating in these challenges.
PopSci: Is it a problem that it’s not necessarily organically driven, that it’s driven by a competition?
MC: It’s true that the spreading of a brand, or a certain behavior, is not competitive. Either you like the brand or you don’t. But even in brands, there are competitive scenarios — like Nike and Reebok can be competing to get some sneaker out. When you buy one, you don’t have the money to get the other one, so whoever gets to you first is going to win the prize. That’s exactly what happened with this competition.
PopSci: So what did you do to build up Team Crowdscanner?
MC: We kept the same incentive structure we did in ’09 with the balloon challenge, but we simplified it a little bit so people could make money by either finding suspects, or recruiting the person who found the suspects. We still have to determine how we divide the prize now. The idea is you would get $500 for finding them, and $100 for recruiting the person who found them. We also pay some money for spreading the word. If you just spread the word, and the people you recruited didn’t find them, you would still make $1 for every person you recruited, for the first 2,000 people.
The goal is to kick-start it, to get to what we scientists call the tipping point.
We found people on two levels. We tried to target mainstream media sites — we talked to journalists, tried to get some articles started in the hopes that things will somehow propagate. Somebody might read your tweet, and send it to 10 or 20 friends. Then we ourselves spread it; we have a few hundred friends on Facebook and so on. So we can gather data at both levels. We can see how successful are journalists, mainstream media sites, versus normal people like us.
Mainstream media sites are always more effective; the question is how much. We believe mainstream media sties were at least 100 times more effective in this competition than individuals — between 100 and 1,000 times more effective.
PopSci: How did people sign up?
MC: Whenever you would click [on our site], you have a unique identifier for that recruitment ID. If you read it in the New York Times, we know you found out about it in the New York Times. If you read about it from me, you know it comes from Manuel. You could join via Twitter, Facebook, email … we just keep track of the unique way in which the information was spreading.
Then if you see one of the suspects, you come back to our website and use your login and upload a picture. We even deployed an Android app where you could just take a picture, and immediately upload it to our site.
PopSci: When did this all start, and how did you cover all these different time zones?
MC: The team leader and the developers were based in Abu Dhabi in the United Arab Emirates. For those guys, it started at 10 a.m. We had some people at Southampton University in the UK, and I am in California, so we were covering all the time zones nicely. It seems trivial, but you need to have someone in different time zones. If the other side is really tired and it’s 3 a.m. for them, I am still fresh and I can verify what people were submitting.
PopSci: How long did it take to find your three suspects?
MC: Well, we had given up on Bratislava. They don’t have huge Internet penetration, they don’t even speak English, most of the population, so we had given up. But three hours into it, we had computer scientists from a university in Bratislava that heard about our team through Slashdot, and he saw one of the suspects and sent us a picture.
We were amazed that we got to find the first person in a relatively short amount of time in Bratislava. He wasn’t hiding, but he wasn’t in the open doing ridiculous things either. He was acting like a normal person.
Then around 1 p.m. New York time, we found the suspect in Grand Central Station. That was from a grad student from Harvard who found out about us through some mailing list. So again, the social network was working very well in that case.
In D.C., there was nothing going on until 7:20 p.m. One of the suspects was hiding in a Starbucks Coffee, around George Washington University. And this is incredible: The person who submitted the location was a professor, David Alan Grier, who is the president-elect of the IEEE Computer Society. He is a really high-profile computer scientist — it was like if Barack Obama had found the person. He wrote to us saying he is an expert in crowdsourcing, and learned about us through a mailing list. He was walking on his way home from work and thought, ‘What if I just had a look at a couple of coffee places?’ And the suspect was there. It seems like a movie to us! It’s really unbelievable.
PopSci: How will you determine how to pay all the participants?
MC: The incentives are designed to maximize the payment to the people who helped us. We didn’t find the whole set of suspects, though. The money should go mostly to the people who took the picture, then to the people who recruited them, and also to the people who spread the word.
In the meantime, we have been gathering all sorts of information — every single mention of our team, we are tracking it. It requires some fine analysis to figure out how much we need to pay everyone. It is harder for the spreaders.
PopSci: What surprised you during this competition? What did you learn?**
MC: We thought this was impossible. We really thought this was beyond what social networks can achieve. So this sets new limits on social media’s abilities. If the balloon challenge was the former record, this is definitely the new one.
But you know, three out of five is an interesting number. It means that this is not trivial. If it had been five out of five, maybe [that means] it would have been easier. For me, it means we need to be very careful about how much power do we attribute to social networks. We couldn’t find the five people; we struggled to find three. So can social networks do something incredible? Yes. Can they do whatever you want them to do? Absolutely not. Social networks are not a magical bullet; we need to understand the limits.
We will study the data to find out why we didn’t find the people in London and Stockholm — we want an itinerary of the suspects to compare it with our data, and so on. We want to create a model for how this unfolded, and find out were we lucky or unlucky. Until we have that kind of understanding, we don’t know if we got really lucky, and three is a lot, or if we got really unlucky and missed the guys in London and Stockholm by one hour, or by one recruit. We are really excited to get the whole picture.
