Robotic chefs have been featured in science fiction for decades, but in reality, cooking is a challenging problem for a robot. Several commercial companies have developed prototype robot chefs, although they are not yet available for purchase and still have a long way to go to match the skill of human chefs.
Now, engineers from the University of Cambridge have trained a robotic ‘chef’ to watch and learn from cooking videos and recreate the dish itself.
The researchers programmed their robotic chef with a ‘cookbook’ of eight simple salad recipes. After watching a video of a human demonstrating one of the recipes, the robot was able to build on its knowledge as it progressed and come up with a ninth recipe on its own.
The results demonstrate how video content can be a valuable and rich source of data for automated food production and could enable easier and cheaper deployment of robot chefs.
“We wanted to see whether we could train a robot chef to learn in the same incremental way that humans can – by identifying the ingredients and how they go together in the dish,” said Grzegorz Sochacki from Cambridge’s Department of Engineering, the paper’s first author.
Researchers used a publicly available neural network to train their robot chefs. The neural network had already been programmed to identify a range of different objects, including the fruits and vegetables used in the eight salad recipes (broccoli, carrot, apple, banana, and orange).
The robotic chef was able to watch 16 videos, each one of a human demonstrating a recipe, before being able to identify which recipe was being prepared and make it.
The robot chef recognized the correct recipe 93% of the time, even though it only detected 83% of the human chef’s actions. The robot could also detect slight variations in a recipe, such as making a double portion or normal human error.
By correctly identifying the ingredients and the actions of the human chef, the robot could determine which of the recipes was being prepared. The robot could infer that if the human demonstrator held a knife in one hand and a carrot in the other, the carrot would then be chopped up.
“Our robot isn’t interested in the sorts of food videos that go viral on social media – they’re simply too hard to follow,” said Sochacki. “But as these robot chefs get better and faster at identifying ingredients in food videos, they might be able to use sites like YouTube to learn a whole range of recipes.”
Journal reference:
- Grzegorz Sochacki, Arsen Abdulali, Narges Khadem Hosseini, Fumiya Iida. Recognition of Human Chef’s Intentions for Incremental Learning of Cookbook by Robotic Salad Chef. IEEE Access, 2023; DOI: 10.1109/ACCESS.2023.3276234
