Robots photo
SHARE

When you get a new coffee machine, you can probably figure out how to use it. But how does your robot know? (You know, that one you have lying around.) The Robot Learning Lab at Cornell University has a solution: a robot that can learn how to operate new machines.

The Robobarista, as the project is called, uses a “deep learning neural network” to figure out how different controls on a gadget work. Overall, it’s actually pretty close to how humans learn. For one thing, the robot consults instruction manuals for the device in question, which are often available online. Secondly, it has a database of how to interact with and use certain types of controls, assembled via a crowdsourced method, using a combination of Amazon’s Mechanical Turk and volunteers who show the robot how to perform a particular task using an interactive 3D model in a web browser.

Thus, when the robot is confronted by a control it finds familiar–say, a knob–it can first try to recognize it by comparing it to all the known objects in its database. Then, once it’s determined that said control is a knob, it can extrapolate how to physically operate it, based on all the knobs in its database and how people have demonstrated using them. Similar projects in the past have utilized crowdsourced knowledge to help robots figure out how to grab certain items or even build shapes with blocks, the latter using the same PR2 robot as the Robobarista project.

Because this learning method is so broad, it means that the Robobarista is able to interact with far more than just coffee makers; it can learn how to use everything from juicers to bathroom sinks. So rather than dedicated robots who only know how to do a single thing, you could have one robot that handles many different household tasks.

But no matter how good your robot gets at running the dishwasher or squeezing juice, it’ll probably be a little while before it learns how to make all that great latte art.