Humans still rule drone racing, but NASA’s AI pilot might change that

In a California warehouse in October, quadcopter drones zoomed and buzzed, racing through an obstacle course of black-and-white checkered arches. On one team: drones guided by software and AI, the work of a team from NASA’s Jet Propulsion Laboratory. On the other: a drone steered by a human professional—Ken Loo, a Google engineer and Drone Racing League pilot.

The official results? Score one for flesh and blood. The human-piloted drone completed the course faster, on average flying the laps more than two seconds quicker than the software-powered craft.

The competition highlights the different ways that humans and machines actually learn in situations like these, as well as how an AI-piloted drone works in the first place. Here’s how the system works—and why future races could have a very different outcome.

How the drone knows where it is

For the NASA drones to successfully fly around a course, the devices need to know where they are in space. For that, they use two onboard cameras—one that looks forward, and the other, down, a common setup for mid-to-high-level consumer drones. Other onboard sensors measure the drone’s acceleration and rotation. Drones that fly around outside can make use of GPS, but that’s not an option when flying indoors, in a complex environment, at speeds of 30 to 40 mph.

The drone also needs an onboard three-dimensional map of the course at hand, so it can match what it sees with the cameras to that internal map and know where it actually is. That process is known as relocalization. It can relocalize as many as a few times every second, says Robert Reid, the project lead at the Jet Propulsion Laboratory.

“As long as you stay close enough to the existing map,” he says, “we’re very unlikely to crash.”

For this research (which Google funded) the NASA team used technology from Google Tango, an augmented reality platform that runs off two smartphones, the Lenovo Phab 2 Pro and the Asus ZenFone AR. Crucially, that same tech can also create the type of three-dimensional map a drone needs to fly in situations like this.

How it navigates

Like a racecar driver learning a course, the drone needs to know the best lines to take to get where it’s going quickly. “We either hand-carry the drone around the course, or we manually fly it,” Reid says, “so we can teach the drone where the race track is.”

But that’s just the beginning. From there, the team figures out the best route for the drone to take by modeling it on computers. That process allows the humans to participate and make sure that the path is actually a safe one that keeps their pricey drone in one piece. In other words, for this competition, the drone wasn’t figuring out the best way to fly all on its own—people were involved. In that sense, it wasn’t a true, independent artificial intelligence system like those that automatically power, for example, language translation on Facebook.

From there, after the drone is programmed with the route, it’s off to the races. Reid stresses that while the route planning actually took place offboard the drone, in the future it could happen using just the drone’s onboard computer.

Learning to fly

For the actual race in October, both the NASA team and Loo had to learn a new course and get ready in a matter of hours. But the way the human and the AI-powered drone actually did that was different.

Loo learned quickly by flying the course multiple times, Reid says. But the NASA team did things differently. “We only need to fly once, and then we can sit there for a few hours crunching numbers to get better,” he says. Interestingly, that optimization process—using algorithms to figure out the best route—took a lot of time.

“The human pilot has to learn by flying—whereas we can record it, and learn without even flying the drone,” Reid says.

Had the NASA team had more time that day to run the software and figure out the best route to take around the course, the resulting race times could have been different—the AI drone might have beaten the human.

Reid says that they’re working on boosting the efficiency of their algorithms, so that in the future, it might take less time to calculate the fastest route. And after that, it’s off to the races. And remember: unlike a person, software doesn’t get tired.