Robobarista autonomously makes latte, taught by the Internet

Cornell University's PR2 robot is able to follow a set of instructions to use an espresso machine to autonomously make latte.

Michelle Starr Science editor
Michelle Starr is CNET's science editor, and she hopes to get you as enthralled with the wonders of the universe as she is. When she's not daydreaming about flying through space, she's daydreaming about bats.
Michelle Starr
3 min read

PR2 in action. Cornell University

The education of PR2 is teaching the robot many things, and it is starting to progress forward -- moving on from an affogatto coffee, which simply involves pouring hot coffee over ice cream, to latte -- which is a little more complicated with the addition of steamed milk. Meet the Robobarista.

And it's partially thanks to the robot-loving people of the Internet.

The team at Cornell University that was teaching the Willow Garage-designed research robot to follow spoken verbal commands has collected enough crowd-sourced information from Internet volunteers to teach the robot to use objects it had never seen before -- to wit, an espresso machine. Moreover, it was able to do so autonomously, based on object recognition skills that allowed it to extrapolate the actions it needed to perform.

Think of it like this: you know what a toaster is. You have seen possibly dozens of toasters in your life. When confronted with a new model that you have never seen before, you still know that pushing the lever down activates the bread-toasting, while turning a knob will allow you to determine how light or dark your toast is cooked.

This is because you recognise a general set of objects as "toaster" and can extrapolate based on its function how to use it. Robots, on the other hand, are not intuitive, driven by pure logic, and need to be taught a little more comprehensively.

The problem with this is that teaching a robot about range of objects vast enough to allow this intuitive extrapolation is very labour intensive -- hence the crowdsourcing, which allowed humans from around the globe to log into a website to help catalogue objects for the PR2's database.

It was this database that allowed the PR2 robot to operate an espresso machine it had never seen before -- by transferring the information it knew about similar object parts in its database. And -- here's another exciting part -- it was able to do so using an English language instruction manual.

Cornell University

So the instruction "Push down on the handle to add hot water" told the robot that it needed to find and identify a handle on the unfamiliar espresso machine -- which it was able to do based on its knowledge of a urinal handle, which is pushed down on a similar trajectory. By drawing on its database in this fashion, PR2 was able to extrapolate how to use the machine, and successfully made a latte all on its own.

That's some pretty amazing stuff right there.

Of course, we're about to see a whole robotic kitchen hit the market, but the two are running vastly different systems; the robotic kitchen executes recipes by following a pre-programmed set of movements as one whole command; Cornell's PR2, on the other hand, can intuitively account for variables and work around them.

The team is still working with crowdsourcing to educate their robot, and you can sign up to assist in their efforts on the Robobarista website.

Eventually, the research will form part of Cornell's overarching Robo Brain project, which is also currently crowdsourcing the education of a robotic memory bank.