Can Technology Save the Military From a Data Deluge?

Our Drones are Feeding a Video Glut U.S. Dept. of Homeland Security

The U.S. military has a data problem. If knowing is half the battle then it’s the half the Pentagon should never lose, at least in theory. But in practice, the military’s data problem is significant, vexing, and given the current pace of acceleration, technologically intimidating. Just two years ago, there were roughly a dozen NATO aircraft flying surveillance missions over Afghanistan at any given time. Now, there are more than 50 Predator and Reaper drones in the air at once, and all of them are dumping fat streams of data to the ground all the time.

Meanwhile, more drones are joining the fights in Afghanistan and Pakistan, in the Horn of Africa and over Iraq and Yemen and elsewhere. Intelligence, surveillance, and reconnaissance (ISR) sensors are proliferating on conventional manned aircraft. The data streams are outpacing the DoD’s capacity to organize and store them and generating so much noise, so much unusable intel, that analysts can’t sort the relevant information from the useless. And all this is happening in an environment where anything slower than real time can take a toll in human lives.

“We’re swimming in sensors, and we need to be careful we don’t drown in the data,” says Dave Deptula, CEO and managing director for defense technology problem-solver MAV6. This isn’t the first time Deptula has said this, and it won’t be the last. In his previous post as the first deputy chief of staff for intelligence, surveillance and reconnaissance, Lt. Gen. Deptula was in charge of planning and implementing the entire U.S. Air Force’s ISR strategy, and he saw the data flood topping its levees firsthand.

“We’re swimming in sensors, and we need to be careful we don’t drown in the data.”“The unavoidable truths are that data velocities are accelerating and the current way we handle data is really overwhelmed by this tsunami” Deptula says. “So we’re going to have to begin exploring different ways to meet the growing challenges of hyper-scale workloads.”

Simply growing the military’s rackspace--and that’s been much of the strategy for dealing with the problem to date--isn’t going to tame the flood. The DoD doesn’t just need new storage methods, but completely new concepts of operation that blend novel storage architectures, all kinds of digital semantics, and--critically--a healthy dose of artificial intelligence.

Someday soon, computer programs will view, tag, organize, and store hundreds and thousands of video streams simultaneously, deciding what sensor data is relevant to the fight at hand, what needs immediate attention, and what needs to be filed away. Language interfaces will let analysts instantly search their databases with natural language queries like something straight out of Star Trek. Drones themselves will even become computerized intelligence analysts, combing through their own data streams in realtime to highlight only the choicest bits of intel. These technologies are already in the works, and this is how technology will save the military from its technology.