Inside the simple counting software that makes biologists’ jobs a little easier

On most autumn days, David Sopjes can be found in the Eel River in Northwest California counting fish. As a retired high school science teacher and citizen scientist, Sopjes has spent the last 10 years monitoring the Chinook salmon population in the Eel River, which he says has the third-largest watershed in the state. Every fall, Sopjes counts the salmon as they wait for the winter rains to reproduce. 

“They don’t eat anymore. They have one thing on their mind, and that’s just sex,” Sopjes says. 

Before getting a drone three years ago, Sopjes and his colleagues counted the salmon by snorkeling in the river and standing on paddleboards, both of which greatly disturbed the fish and weren’t very accurate. 

The drone produced clear photographs of the salmon, but counting the fish in the images using pen and paper was tedious. While scouring the internet for a better method to count and organize his data, he found a software called DotDotGoose and has been using it ever since. 

Designed at the American Museum of Natural History’s Center for Biodiversity and Conservation, DotDotGoose is a free, open-source tool that assists researchers with manually counting objects in images. Peter Ersts, the senior software developer at the center, created DotDotGoose in May 2019. He got the idea through discussions with colleagues. 

[Related: Alexa will tag along on an uncrewed mission to the moon]

At the time, the most popular ways for conservation researchers to tally up different categories of animals in photos were very hands-on. “A lot of people were still just literally projecting images onto a dry erase board, circling the animals and turning the projector off, and then counting them as they wiped [off the markings],” Ersts says. “I saw a need for a really simple tool that allows you to quickly and easily put dots on an image.” 

Although the tool’s only been online for about two and a half years, it’s already helping many researchers across the world. Since finding DotDotGoose, Sopjes says he’s counted thousands of fish, and the accuracy of his data improved “dramatically,” so much so that the California Department of Fish and Wildlife became interested in using his datasets. The accurate record of the total fish paired with images from the drone provided a useful way for Sopjes to track every fish.

How it works

DotDotGoose has a very simple interface that allows users to import images they want to analyze. Then, they can divide different objects in the images into “classes” or categories. For example, Sopjes sets the classes as different life stages of salmon. Each category corresponds to a dot color. 

To count each class, researchers can click on each object in the image to place the dot. DotDotGoose tallies the number of dots by class as they’re placed. Users can add custom notes, latitude and longitude coordinates, or other data points to describe the image.

DotDotGoose was originally intended to count animals for conservation research, but Ersts has seen users repurpose it to count inventory in warehouses, components on circuit boards, and even flowers on tomato plant candidates for the Guinness World Record.

Why it’s useful

Rochelle Thomas, a graduate student in Columbia University’s Department of Ecology, Evolution, and Environmental Biology, has used DotDotGoose with real-life geese.

From 1995 to 2019, Thomas’ advisor, Robert Rockwell, had taken aerial photographs of lesser snow goose flocks in the Hudson Bay region of Canada. In the early years of the project, Thomas says Rockwell would print out the photographs to count the geese by hand. 

When Thomas joined the project in 2018, she tried counting the geese using Photoshop, but it was hard to count geese simultaneously by species and age. She was introduced to Ersts while he was building DotDotGoose and became the program’s beta-tester. The program’s name is a nod to her work with lesser snow geese.

“I spent many days putting dots on geese, and it just kind of came [to me] to call it DotDotGoose,” Ersts says. 

[Related: AI turned a Rembrandt masterpiece into 5.6 terabytes of data]

Compared to similar software, such as Photoshop and ImageJ, Thomas likes that DotDotGoose was built with a conservation biologist in mind and allows her to note the quality of the image or insert information indicating the presence of water in the photo. 

“Conservation biologists and ecologists are sitting on just tons and tons of photographic data,” Thomas says. 

And while the current manual version of the program already makes the data more manageable for analysis, she thinks that making counting in DotDotGoose more automated could further help support research projects like her’s.

The future of DotDotGoose

Ersts has had plans to semi-automate the process since its inception.

“If you’re able to save those coordinates of your locations on an image, you then basically have a training set that you could use [to power] a machine learning model to help automate things in the future,” Ersts says. “[But] it’s quite a challenging task to automate this when you really start thinking about all the different types of data that exists.”

Ersts imagines that researchers might train it with a couple of similarly oriented photos unique to their project that contain the same types of objects. 

But even a tailored, automated DotDotGoose would have its limits. Images with many objects bunched together would be very difficult to parse through. And while an automated version of the program could free up researchers’ time, Ersts says a human would still need to be a part of the process, at least to check the computer’s work.