In the heat of battle or an active shooter situation, it’s hard for untrained observers to correctly identify the weapon currently trying to kill them. Reporters, present company included, will often be vague when first describing the weapons used in an attack, with the all-encompassing “small arms,” or the still-broad “rifle,” “pistol,” and “automatic weapon” doing a lot of the heavy lifting. What if, though, a computer algorithm could look at publicly available images of a person holding a weapon, say from social media, and automatically detect what that weapon was? A new project from Justin Seitz’s automating open source intelligence project aims to do just that.
First, the scale of the problem. There are a staggering amount of guns in existence. Not just guns, which the Small Arms Survey estimated as at least 875 million in 2007, but also a staggering number of types of guns. And it’s not just recent guns; NATO troops in Afghanistan recovered multiples weapons dating back to the World War I and even one from the 1870s. “Jane’s Guns Recognition Guide,” a gun identification reference manual intended for law enforcement and other legal authorities, contains extensive detail on almost 400 types, and includes a list of over 1,100 other brand names for guns. Even that depth of knowledge comes prefaced with the disclaimer that “the contents of this book cannot be encyclopaedic.”
Fortunately for Seitz’s project, a computer program should be able to do more than a reference manual. First, to teach the program to accurately identify weapons, Seitz fed the Imagga tagging tool images of just the guns themselves, no people attached. Then he tried with images of people holding guns. The tool tried to tag other things, like “fashion” instead, but with some manual cropping, it recognized the weapons again. Manually cropping images defeats the point of an automated weapon-identifier, so he coded a python function that, in essence, looks at the image in chunks, tags them, and then only keeps the tagged sections with desired keywords (“gun”, “weapon,” “glock”, etc).
Seitz details the entire process in a granular level at the blog post, with notes about what specific lines of code do and why a person would want them to do that. He concludes part one of this series (there’s another to follow), with the module taking a previously tricky image and correctly labelling it “weapon.”
That’s a great start for the project, and it’s easy to see how it could be expanded to not just identify the existence of a weapon in a photograph, but label the kind of weapon too. That’s still a ways off though, so don’t throw away the reference manuals just yet.
