Whale-monitoring robots are oceanic eavesdroppers with a mission

About 20 miles offshore from Martha’s Vineyard, a yellow robotic buoy is bobbing in place on the water. Like most data-collection buoys, this moored robot is eavesdropping on the world under the waves. But unlike most buoys, which function like floating weather stations, this one from the Woods Hole Oceanographic Institution (WHOI) is listening for whales in the area in real-time. Just this week, it detected the presence of a sei whale and a fin whale. 

That buoy is one out of the many robots deployed by WHOI off of the East Coast and West Coast of the US. These buoys are autonomous platforms tuned into the melodies of a range of different whales: sei whales, fin whales, blue whales, humpback whales, but most notably, right whales, which are critically endangered. 

Right whales play a vital role in the ocean food web. Like other filter feeders, they eat zooplankton and tiny crustaceans, then recycle and redistribute nutrients like iron back into the ocean as they poop. Whales also act as valuable carbon stores, and when they die and sink, their corpses transform into pop-up habitats for critters on the ocean floor. But right whale numbers have been dwindling again since around 2010, despite a slight uptick in population in the early 2000s after commercial whaling laws were revised in the late 1900s. Currently, it’s estimated that there are only 360 of the animals left. The most common causes of right whale deaths are entanglement in fishing gear and ship strikes, though it’s suspected that climate change may also become a threat. Other human activities in the ocean that can affect their behaviors include loud, unusual noises such as construction or sonar. 

However, some scientists think that using sea-faring robots to detect the presence of whales could help humans navigate more carefully around them. In March, WHOI announced that it was collaborating with a French shipping company to figure out how to incorporate information from the robots into their business operations.

Mark Baumgartner, a marine ecologist at WHOI whose lab operates these buoys, along with robotic gliders that can move across the sea to scope out where whales are, says that they’re also working with wind energy companies, NOAA, the US Navy, state agencies, and Canadian researchers on ways to use these tools to reduce the risk of harms to the animals. 

On the West Coast, these buoys help monitor the activity of these large marine mammals as part of the Whale Safe system that maps whale and ship movement off the coast of California.

[Related: Whale ‘roadkill’ is on the rise off California. A new detection system could help.]

How the robots work

Researchers from WHOI are currently employing 7 buoys and 4 gliders to help with this issue. Both the buoys and the gliders share the same basic instruments and software. More specifically, the software is one written by Baumgartner for identifying whale sounds and creating “pitch tracks” that get sent to researchers in his lab. 

The software is conceptually simple and is best explained with an analogy. “Imagine you’re sitting down and you’re playing something on the piano and there’s a magic box on the piano that listens to what you’re playing and spits out the back sheet music of everything you’ve played,” Baumgartner says. “You can take that sheet music to a musician and the musician can read the notes on the sheet music and say you were playing ‘Mary had a Little Lamb.’ The musician didn’t have to hear what you were playing. They just have to read the notes.” 

This system works similarly. In essence, it identifies sounds and creates compact representations of sound from a spectrogram, or “pitch tracks,” which are analogous to notes on a piece of sheet music. Then, it compares those to pitch tracks within an existing library of whale calls. The robots will forward the audio clips that contain whale sounds, according to their analysis, to a dedicated server ashore. The human analysts back at the lab listen to these clips and make the final call on if whales were detected or not as well as put in notes about the features of sound and the species that made it. 

“A lot of whale sounds are distinctive by species. North Atlantic right whales will make different sounds than a fin whale,” Baumgartner says. “The pitch tracks for each of the species are distinctive.”

[Related: Birders behold: Cornell’s Merlin app is now a one-stop shop for bird identification]

The hardware components of the listening system include a computer inside each glider and buoy and an underwater microphone. These robots also have particle motion sensors that can help them get a bearing on which direction the sound is coming from. The computers send the data back to the lab through an iridium satellite system. After coming off the satellite, it goes through processing, and gets put on a central website that displays it publicly. The data is also shared with the National Oceanographic and Atmospheric Administration. 

The Slocom glider can run for three to four months on a lithium-ion battery, and the buoy runs on a stack of alkaline batteries that last for a year. The moored buoy was designed at the WHOI to be very quiet so the onboard instruments can listen to ocean sounds effectively. 

Both the buoys and the gliders are capable of two-way communication. The gliders make a “phone call home” every two hours to a computer in Baumgartner’s lab in Woods Hole, Massachusetts. It sends not only pitch track data but all kinds of information about how the glider is doing, where it is, and where it thinks it’s going. The researchers can tell it to go somewhere different, or troubleshoot issues that come up onboard.

“The reason we chose different platforms to put this technology in is because sometimes you want to monitor over a relatively small area for a long period of time. Buoys are great for that,” says Baumgartner. “Autonomous vehicles are good if you want to do a much larger area.” 

As it exists today, both robots act like the flashing lights in front of an elementary school that caution cars to slow down. But Baumgartner and his team hopes that these robots could one day tell any ships or possibly fishermen to be careful when whales are around either through email, text, an alert app, government software, or perhaps some other form of communication. 

Near real-time detection

Baumgartner has been studying whales and ocean acoustics for over a decade. Back in 2005, he and his collaborators were deploying gliders with passive acoustic monitoring abilities. But soon, they realized that while collecting sounds and analyzing them months later was good for science, it wasn’t useful for conservation or management. That’s when they started shifting the system to do more real-time detections. Since 2012, they’ve cycled through two versions of the ocean-faring robots, but the software has stayed more or less the same. 

[Related: Why ocean researchers want to create a global library of undersea sounds]

In the US, NOAA has set up a program called slow zones that creates a box around areas where whales have been detected visually or acoustically. It asks mariners going through to slow down to ten knots or less or avoid the area altogether. “This program has been up for about two years. And we’ve had a lot of slow zones over the East Coast over the past winter, triggered because of our buoys and gliders,” Baumgartner says. “We know that when ships go slower, they’re less likely to hit and kill whales.”

Canada, on the other hand, has been closing off areas to fishing and imposing mandatory speed limits when whales are present. 

Without this system, the way to spot traveling whales is to survey the ocean from a plane, drone, or a ship. These methods are useful for getting a visual reading on the whales, and you can tell from photos if they’re entangled, injured, sick, or dead. But flying instruments are often hampered by weather and wind conditions. With imaging, if there are too many white caps on the ocean, it would be impossible to make out a whale. The robots, especially the moored buoy, meanwhile, can listen all the time. But only when and if whales are calling in the area.

Although this technology is helpful, especially for mitigating risks in areas where right whales aren’t very common, it can’t solve this problem alone. “These systems are just a constant reminder. But it’s not enough to let people know that they’re there. Continuing to advocate for stronger protections for right whales based on this information is really important,” says Baumgartner. “There are lots of other solutions that we need to be paying attention to.” 

Robots alone will not save the whales

Canada’s mandatory speed restrictions imposed in areas where whales are present is what Baumgartner says is the more effective approach. “That voluntary-versus-mandatory [distinction] is really important because compliance with mandatory ship speed restrictions is actually quite good. The compliance with voluntary slow downs is quite bad,” he says. 

Additionally, Baumgartner and his team are thinking about marine industries can apply the information the robots supply. One problem they’re still trying to solve is who to get this information to. Because when a ship wants to come on shore, it needs to be on time to unload, have longshoremen scheduled to be in place, and have truckers ready to receive the containers. “That’s a business problem that the captain will not solve alone. Who do you provide this information to?” says Baumgartner. “Do you provide it to the captain, do you provide it to the pilots, the company, schedulers and business planners, or to the people who arrange logistics on shore at the port, the Coast Guard so they can notify ships?” Their project with the French shipping company, CMA CGM Group, is aimed at finding an answer to that question. 

Meanwhile, offshore wind companies, which fund the operations of some of Baumgartner’s buoys, are also interested in seeing if the robots can inform construction schedules, and advise best practices for ships going back and forth to maintain wind farms.

[Related: These free-floating robots can monitor the health of our oceans]

The robots are not a holistic solution for all industries operating near whales at sea. For example, they might not be practical for preventing right whales from getting entangled in fishing gear in the US. Fisheries operate differently in Canada compared to in the US. “They fish [in Canada] with a lot less gear. They have seasons where guys fish and when the season’s over they have to gather up all the gear and go home,” he adds. “The American fishery is not like that at all, it’s a year-round fishery, so guys just don’t have the equipment to gather up their gear and bring it home.” 

In addition to robots for whales, Baumgartner and others are looking into ways that both scientists and manufacturers can innovate ropeless fishing to make it cheaper and easier for fishermen to incorporate into their businesses. That way they can still fish even when whales are there, and regulatory bodies won’t have to close off entire areas because a whale was detected nearby. 

“We’re not going to stop shipping, we’re not going to stop fishing. We have to find a way for these industries to be sustainable, to not have the impact that they’re having on the ocean today and on the animals that live in it,” Baumgartner says. “One of the potential ways to mitigate these risks that we pose to the animals is to change industrial practices when we know whales are around.”