Software Now Better Than Humans At Recognizing What You Were Trying To Draw

Here come the Pictionary robots

Researchers at Queen Mary University of London used these kinds of sketches to inform their artificial neural network, Sketch-a-Net.Mathias Eitz, James Hays, and Marc Alexa

Artifical neural nets have been all over the news in the last few weeks—they dream, nightmare-ify music videos, and now they can beat you in Pictionary.

Researchers at the Queen Mary University of London designed their neural network to analyze drawings on varying levels of abstraction, and figure out what is depicted. This technology could have wide use in shopping, criminal identification, and even finding the exact picture of a cat you're looking for on the internet.

The program, called Sketch-a-Net, beat a human sketch recognition benchmark by a very close margin of 74.9 percent recognition to 73.1 percent for humans. It uses eight layers of processing to break a sketch apart.

Like other neural networks, the Queen Mary team had to train Sketch-a-Net by showing it a variety of human sketches matched with keywords. To ensure the computer knew a variety of sketch styles and figures, researchers used the Technical University of Berlin sketch dataset, the largest set of its kind at 20,000 sketches. The TU-Berlin team originally sourced the sketches from Amazon's Mechanical Turk, an anonymous horde of internet task-doers, who sketched 80 objects in 250 categories. Researchers at Queen Mary altered each sketch slightly, reflecting and stretching them. These variations gave the software 31,000,000 sketches to work from.

The birds identified by Sketch-a-Net.Mathias Eitz, James Hays and Marc Alexa

With that training, the program was able to beat the human benchmark, and in certain categories far exceeded humans. In the recognition of different kinds of birds, Sketch-a-Net reached 41.5 percent accuracy, almost doubling the humans’ 24.8 percent score.

The team sees real-world use in the retail world, according to project supervisor and co-author Timothy Hospedales. If you’re shopping for a specific kind of couch, you could to draw the style you want and be matched with a few close options. Hospedales says the technology is also ripe for use by police, matching portraits made by sketch artists to photos of possible suspects.