This outbreak simulator predicted the strange twists in our pandemic behavior

In 2018, a deadly outbreak was underway at Sarasota Military Academy Preparatory School in Florida. Groups of middle schoolers congregated with hazmat suits and masks, distributing medicine, checking one another’s health statuses, and occasionally carrying the infected into isolation on a stretcher. 

Looking back, the situation was prescient, even if not quite real. The students running around were part of an app-based educational simulation, and the pathogen they were dealing with was a virtual one spread through proximity-sensing Bluetooth communications. Every other element was no more than a stage prop.  

This experiential education platform, called Operation Outbreak, started as a simple lesson plan by Todd Brown, a former teacher from southwest Florida. It began in 2015, he recalls, with a single question by a seventh grader. From there, it “has blown up to what we’re doing now,” Brown says. 

The question was: What if an Ebola-like virus came to the US? A discussion and a subsequent two-week lesson plan spawned from that inquiry, delving into details on who would be in charge of what, where funding comes from, what things would be like on the ground, and how different groups reported to one another. From there, they launched an early analog version of the simulation, where the virus was spread through a close-contact sticker system. They broke students into groups of doctors, triage personnel, epidemiologists, the media, the government, and the general population.

Then, that same year, on a whim, Brown reached out to Pardis Sabeti, a computational geneticist and a professor at Harvard University. 

Sabeti had previously researched infectious diseases such as the Ebola virus. And from 2014 to 2016, she and Andrés Colubri, a computational researcher in her lab, had been working on apps that could support diagnosis, prognosis prediction, and digital contact tracing. “It’s something that anyone who’s been in an outbreak recognized as needed,” Sabeti says. “The whole world came to it during COVID, [but] we came to it before and during Ebola and during the mumps outbreak because it became clear to us that it was spreading through contacts.” 

At the time Sabeti was learning about Brown’s work, she and Colubri were discussing ways to get good data and model the different facets of an outbreak. They were drawn to the schoolyard project that they heard about from Brown because they saw it as an opportunity to make the students’ experience as engaging and hyper-realistic as possible, and also gain high-quality insights into how actual humans would respond to mitigation methods like vaccination or quarantine. “At that point, those pieces kind of came together, and it was like wait, the best thing to do is simulate an outbreak,” she says. “And it’s best simulated on the phone [via an app] where you can make all of those pieces that would make the simulation more realistic and gather the data.”

In late 2017, Colubri, Sabeti, and Brown introduced the first version of the Operation Outbreak smartphone app and accompanying web platform.

Here’s how it works 

Operation Outbreak is a project geared towards schools and educational facilities. All participants must download the Operation Outbreak app (available in iOS and Android app stores) to set up the simulation. 

While Operation Outbreak mainly partners with the schools (which can reach out to the team to receive a code to activate the app), they are working toward a more decentralized approach where any school or organization can create a simulation and use the app. “We’re working through that right now in terms of what’s the best level of supervision that may or may not be needed,” says Kian Sani, program manager for Operation Outbreak, now an outreach project run by The Broad Institute of Harvard and MIT. 

The virtual pathogen flows across phones through Bluetooth. Each user’s app will show an avatar that displays their health status (depicted as a happy face emoji or sick face emoji). The app uses software like p2pkit, Herald, and Estimote to detect proximity and contact with nearby devices. It records all interactions between users that are less than 10 feet apart at a resolution of one second. 

The virtual pathogen can be transmitted if a phone is close to an infectious participant, and the probability of transmission over a certain time period is preset in the parameters of the simulation ahead of time. The app can also model the effects of interventions such as diagnostic tests, vaccines, masks, or personal protective equipment. Students get these by scanning QR codes that decrease their probability of catching the virus. For example, in the versions where asymptomatic spread is activated, users can have avatars that appear healthy when they’re carrying the virus and they can only discover that they’re infected by scanning the diagnostic test QR code, says Sani.

Every interaction and contact that happens during the simulation is recorded into a web server that can be used to observe what’s happening in real-time or analyze patterns post-simulation (like the ‘God View’ feature in video games). The web portal also allows organizers to get data on the number of cases, and the health status of each participant, as well as tweak different parameters for the outbreak including number of users, duration, symptoms, recovery time, viral transmissibility, money and resource allotments, and more. It creates a microcosm where public policies, economics, and personal decisions play out within the bounds of the technology. 

Visualizing the web of social contacts.

“Every outbreak [simulation], there were more insights that were being generated in real-world response,” Sabeti says. “Essentially these outbreaks could emulate everything you could experience from police tensions to government mistrust to vaccine passports and people cheating the system.” 

Though the team is still running simulations during the COVID pandemic, some of the shockingly relevant insights they’ve noticed stemmed from trials that were run in 2018 and early 2019. For example, in one of these pre-COVID setups, the participants that were assigned government roles decided to lie to the general population in order to get them to do certain things. Then a whistleblower leaked it to the media who reported it to the general population. 

“Everything went haywire,” says Brown. “These were young kids in which we were seeing these kinds of behaviors.” 

In another run, kids were restricted from going places based on their health status. “There are always people that don’t want to abide by rules so they can get what they want as an individual. So they would screenshot the healthy emoji to fake the passport to then get through to certain areas.”

There is also a financial system built into the simulation that can randomize the amount of money that families received to help them, and the wealth was not evenly distributed. In these scenarios, although some with “grotesque” amounts of money decided to help those without, there were also people who used their extra capital to buy up supplies like masks and try to resell them at a higher price even though “you didn’t get anything for being wealthy at the end,” Brown notes. 

“To watch some of these things, watch people protest the government’s decisions, watch people get angry with one another… It was surreal,” Brown says. He sees some “cosmic strangeness” in how things have since aligned between the simulations and real life. One year to the date before the US declared a state of emergency, the team penned an 2019 op-ed in Wired about why they shouldn’t wait for a pandemic to happen to prepare people for how things might go down. That same year, they started running a version of the SARS-inspired virus that spread asymptomatically, to make it challenging for participants. Former students even emailed him during the COVID pandemic saying how it was bizarre that the adults were acting almost exactly like they had in middle school. “It was very Twilight Zone,” says Brown.

Initially, the goal for the students in the simulation was to survive. But they’ve been testing other metrics for winning as a group, such as specific percentage of survival. “I think it changes with some of the new additions to the app with vaccination strategies and masking,” says Brown. In a recent 2021 run with Brigham Young University, they even used the simulation to explore ideas for incentivizing more vaccinations.   

Models for real pandemics

In addition to using this technology to create a more immersive setting for students, the Operation Outbreak team hopes that their simulation data could help create better predictive models on how real-life pandemics play out. The team has already published some preliminary findings on this subject in a Cell commentary and a preprint article on medRxiv. 

One benefit is that these mock pandemics can allow the team’s researchers to completely map out everything that is happening—who infected who—which is hard to keep track of in a real outbreak. “We always have a partial view of things during a real outbreak. We need to test everybody—that’s something that’s not possible to do,” says Colubri. “With the data from this simulation, we have the advantage of getting the ground source data, kind of the answer of what happened.” 

They can then use this data to infer how groups of people might interact with each other and spread the pathogen through the community and construct exposure risk models or new algorithms that might predict what happens in a real outbreak. 

Another feature is that Operation Outbreak’s simulations benefit from human input. “There’s a whole method of running outbreak simulations with agent-based models, which is essentially like running a Sim City simulation where the computer simulates each agent,” Colubri says. “But here, the individuals are real people who might do things you did not anticipate. That element of unpredictable behavior that we have in this simulation is something that sets it apart from these other models where everything is already programmed into the system.” 

From here, there are many attractive possibilities for this type of research, although much of it remains speculative at the moment. For one, Operation Outbreak could be used as a testing ground for trying out public policies or intervention techniques. Alternatively, they could use the simulation-derived models as a foundation for constructing a real-time risk calculator that can take data from real-world Bluetooth contacts (it could look something like the system Apple and Google partnered up to create in April 2020). 

They’re not the only group thinking about incorporating social relationships into models of pandemics. Several research groups have looked into how the dynamics of social contacts, cultural influences on behavior, and factors like restrictions and speed of vaccine rollout can impact the way populations respond at-large to such emerging public health threats.  

Ran Xu, an applied statistician and an assistant professor at the University of Connecticut who is not involved in Operation Outbreak, says that he sees value in such simulation exercises. 

“Disease simulations have been used widely in policy planning and for education purposes,” he says. “This one seems particularly interesting as the users can play a part during a disease outbreak and see how their behaviors affect the disease transmission. Many articles have pointed out the important role of human behavior in epidemiology.” He and his colleagues published a paper last month in PLOS Computational Biology that showed that detailing how disease affects behavior and vice versa is important for building better pandemic forecasting models. But any behavioral insights should be tested more rigorously afterwards with real-world data as well, he says. 

There are some limitations to consider, too. “This may not fully represent what will happen in the next disease outbreak depending on how seriously people play this,” Xu notes. “But this applies to any simulation.”

An outlook on outbreaks

Outside of Operation Outbreak, Sabeti is working on a number of other pandemic-related projects, most converging at the intersection of genetics, epidemiology, and technology. Separately, Sabeti and her colleagues have been working on a symptom-tracking app for COVID infections called Scout, which they have tested at Colorado Mesa University. 

Sabeti believes digital platforms that enable contact tracing, health status monitoring and virus tracking are going to be critical for containing and managing the spread of novel infections. But privacy concerns remain a hurdle. “What is fascinating to me is that everyone is so freaked out about their privacy in these outbreaks,” she says. “We’re on TikTok sharing every one of our personal thoughts. But somehow, the data that might save you from a deadly virus is one thing you want to protect.” 

“The lack of trust is so high and it’s gotten so much worse over the course of this,” Sabeti adds. “It’s insane how much we have to overcome to get to a point where this will work and these technologies [such as contact tracing and health-monitoring apps] won’t be scary to people.” 

Sabeti estimates that she spends more time on Operation Outbreak than any other project in the lab. This time is broken down into periods of tweaking the app and coming up with simulation plans, but also editing a textbook on outbreak science (expected to come out sometime next year). “It’s probably the most important thing I could be doing—educate a generation of people who understand these technologies and get the point,” she says.

Education, in fact, remains the core mission of Operation Outbreak. “You might argue that people have been living through the pandemic for over two years, why would we want to play it with an app. The answer is, the information that we have received through the news and social media is often conflicting, [with] lots of misinformation, which really doesn’t help,” Colubri says. “By creating these immersive simulation scenarios, we could help make sense of what happened.” 

The goal for the project is that in addition to raising compelling questions about human behavior in the event of a disaster, it can also prepare the next generation of scientists, public health officials, and health responders to think about their positions in these situations, understand how their roles work in conjunction to other parts of society, and grasp how what they do as an individual or a collective might impact other people.

“I’ve asked a group of teachers: How many people are happy with how the United States as a whole has responded to this? And virtually no one raises their hand,” Brown adds. “How would this pandemic have been different if the general person walking on the street would’ve had a little better understanding of outbreak science, awareness, and response? What would the results have been at this point compared to what they have been in reality?” 

The ‘empathy’ factor

A key component of this simulation is the post-outbreak exercise, which Brown included at the start of the program to provide students with an opportunity to reflect on why they or others acted one way or another, and discuss whether their decisions worked best for themselves or for everyone in the group.

“We were having situations in which communication, government trust, belief or understanding of basic science principles, those things we see are always on the bottom of the decision chain. Because of that, it changed the way people behaved,” says Brown. “But kids learn. They realize that that’s probably not the best way to do something.” 

Often through repeated runs, the students become more considerate, they learn to cooperate, they learn to convey their needs, and eventually, they get better at beating the pathogen.  

“What’s interesting about it is in a way it gives the students empathy for what the circumstances are for somebody else, in a situation that’s not very high stakes and volatile,” Sabeti adds.  

The researchers behind the Operation Outbreak project stress the present urgency of properly informing the younger generations about how to navigate and respond to these stressful and chaotic global health emergencies, an item that’s lacking in most school curriculums, especially as statistics show that COVID-like outbreaks are only expected to become more common and more severe. 

COVID, for which there are effective vaccines, has already killed over one million people in the US. But surprisingly, it is still not the virus that Sabeti considers “the big one.” 

“It’s a very scary time,” Sabeti says. “What are we doing if we actually get something that is the ‘big one’ that could kill any one of us instantly in all ages? How would this all go down?”