A human step for artificial intelligence
Robots are getting smarter.
They still need instructions, but they no longer need to be so detailed. Now you can just say “make me a bowl of noodles”, rather than telling them that they need to heat the water, too.
This is because of work being done by a scientist in California, who is designing them to be able to take casual instructions from humans. In his Robot Learning Lab at Cornell University, Ashutosh Saxena is teaching robots to understand instructions in natural language from various speakers, account for missing information and adapt to their environments.
Equipped with 3-D cameras, the robots use Saxena's computer vision software to scan and identify objects. They are able to associate objects with their capabilities: a pan can be poured into or poured from, for example, while a stove can heat things by putting objects on it.
They can also pick a number of objects to perform the same task. If you tell them to “heat water”, they will be able to do this using a microwave if a stove is not available.
The way that Saxena and his team are doing this is by using a "machine learning" technique to train the robot's computer brain to associate entire commands with flexibly defined actions. The computer is fed animated video simulations of the action which are accompanied by recorded voice commands from several different speakers.
Then, when the robot is given an instruction, it decides what to do based on probability. If it is told to "take the pot to the stove”, it calculates the probability of a match with an instruction it has heard before, and if the probability is high enough, it declares a match and carries out the same action.
Because of this, the more instructions that a robot hears, the more competent and accurate it will become at carrying out tasks. is a website which has been set up to crowd source instructions for the Cornell robots for just that reason. It allows you to teach a simulated robot to perform any kitchen task.
Aditya Jami, visiting researcher at Cornell, is helping Tell Me Dave to scale the library to millions of examples. "With crowdsourcing at such a scale, robots will learn at a much faster rate," Saxena says.