Boston Tested Crowd-Watching Software That Catalogues People's Skin Color

Just because technology can do something doesn't mean it should.

A Navy Performance During Boston Harborfest, 2007

This is very much not the Boston Calling concert, but it's at City Hall Plaza and surrounded by the same brutalist architecture.Dave Kaylor, U.S. Navy, via Wikimedia Commons

The Boston Calling Music Festival in May 2013 had a great lineup: Fun., Dirty Projectors, and Of Monsters and Men, to name a few. The event also included ten cameras that recorded over 50 hours of video surveillance footage on the thousands of concert goers at Boston's City Hall Plaza. An investigative series by the Boston area free weekly DigBoston recently unearthed this surveillance, which was done on behalf of the city.

Of course, video surveillance is nothing new. What's particularly interesting about this footage is that it was used in conjunction with a new type of "people-parsing" software. For these concerts, IBM provided the city with a software dashboard for surveillance, and it went beyond just recognizing faces. A "people search" tool available in the dashboard shows that the software recognized and categorized a number of searchable facets about concert attendees. These include "'baldness,' 'head color,' 'skin tone,' and clothing texture," making it possible to search the footage for images of everyone by outfit as well as by race.

According to Boston officials, the tests at both the May and September 2013 Boston Calling concerts were to evaluate the new software for managing large public events, and they used both existing cameras and existing data storage technology. Saying the tests were just for evaluating software is a tad misleading. With the right software, an off-center picture of a face can be reconstructed into a 3-D image accurate enough for police use, and facial recognition software is increasingly sophisticated. New software isn't just a part of how surveillance is done -- it's what transforms video from an formidable 50 hour block of time into a database of searchable images.

Courtesy of DigBoston, watch a small slice of the surveillance footage below, taken from the concert's beer garden, below:

Read the first part of the revelations at DigBoston, and look for Part Two there tomorrow.