When a drone from a company called Zipline is zipping through the air at some 70 mph, the ideal scenario is that no other low-flying, but faster, aircraft smashes into it. Zipline drones deliver health-focused supplies (like blood) via parachute in countries such as Rwanda and Ghana. They also make commercial deliveries from a Walmart in Arkansas. In Rwanda, they’ve even been delivering semen from bulls (and pigs), for the purposes of insemination with a focus on, in the case of the bull semen, genetic diversity and milk production.
Like any other aircraft operator, Zipline doesn’t want any midair collisions. Keenan Wyrobek, the company’s CTO, says that in the US especially, navigating the airspace can be tricky. Low-flying planes from regular folks out cruising for fun in a Cessna, or someone operating a crop duster, or a helicopter, may not have a transponder announcing their location. “A lot of these planes, they’re just not required to carry transponders,” Wyrobek says. “There’s kind of a wild west spirit of aviation in this country.”
While a human flying a small plane has the responsibility to look ahead and avoid hitting another aircraft such as a hovering helicopter, in the case of the drone, the UAV needs to take action to avoid being hit itself. “It’s actually the job of the drone to see that Cessna coming up on them, and get out of the way,” Wyrobek says. The pokey cowboy, in this instance, has to make way for the cowhand galloping much faster.
That raises some questions, which are concerns that don’t just apply to this one company: How can a drone, with no human on board to use their eyes to look out for traffic that might smash into it, get out of the way of a fast-flying plane? How can it identify the threatening traffic in the first place?
[Related: An electric aircraft just completed a journey of 1,403 miles]
The solution to this larger problem, in Zipline’s case at least, does not involve radar, cameras, lidar, or other sensors, which tend to be approaches commonly used in the autonomous car space. Instead, the drone company has decided to employ microphones that can listen for other aircraft, and then have the drone get out of their way.
The setup goes like this: A total of eight microphones, each one placed on a probe protruding from the leading edge of the 11-foot wing, comprise the sensor array tasked with detecting other aircraft. The system needs to be able to ignore the ambient noise of the drone itself—the air around the drone, and its own propeller sounds—and just listen for other flying machines. “The array is important to both help with getting enough signal-to-noise to hear the planes far away, but also to figure out where the planes are,” Wyrobek says. That way it can “triangulate where those planes actually are coming from.”
To do this trick, the drone relies on a small amount of onboard computing power. “It’s a combination of signal processing techniques—like beam forming—as well as machine learning, AI-based techniques in order to actually localize where that aircraft is,” he says. A small onboard GPU helps with this job, as those types of chips are good at handling AI-related tasks. Wyrobek says that fortunately, the microphones don’t produce that much data. “The actual information pipe is so small, [so] it’s not a big compute load,” he adds.
To build the system, they collected training data that included some 15,000 planned interactions between a drone and a human-crewed aircraft like an airplane or helicopter. Of course, microphones aren’t going to be too helpful in the case of a hot air balloon or a glider, but Wyrobek says that one fortuitous benefit of this approach is that fast-moving aircraft also tend to be louder, meaning that the signal from a quicker-moving threat is stronger.
For now, the company is waiting for regulatory approval to let the software onboard the drone make decisions to take evasive action to avoid an aircraft that might hit it, which would involve a maneuver such as proactively turning out of the way and entering a holding pattern until the coast is clear. Currently, the microphones are installed in some of the drones, even if the whole system isn’t switched on, yet. “The microphones think they’re in control, but they’re not,” he says; the team examines the data from them post-flight to “make sure what it wanted to do is what it should have done.”
Wyrobek anticipates using the sound-based detection and avoidance software in many regions where they operate. “In most places, I think we’ll use it,” he says. “As we scale, we want to keep increasing our safety, and this is a way to do that.” From an airspace management perspective, that sounds good.