Toyota uses VR to train robots as home care assistants

A Japanese tech giant Toyota Research Institute (TRI) is using virtual reality to train robots to help with domestic activities. The proposal is to offer a better quality of life to the elderly, who may have difficulties in performing some tasks at home. It targets a knowledge of fleet learning, which means that after a robot learns a command, it teaches the rest of the fleet.

A virtual reality training system allows humans instructors to see the same things the robot is seeing in real-time and in 3D from their sensors and cameras. A human instructor asks the robots to perform arbitrary tasks with a variety of objects, such as holding a handle, for example. A human can then teach them everyday skills by annotating the scene with simple notes and instructions.

“Operating and navigating in home environments is very challenging for robots. Each home is unique, with a different combination of objects in different configurations that change over time,” TRI wrote in a press release. To address this diversity, we teach the robot to perform arbitrary tasks with a variety of objects instead of programming it to perform predefined tasks. So when the robot sees a specific object or scenario again, even if the scene has changed a bit, it knows what actions can be taken.”

“At present, our system can successfully perform a relatively complex human-level task, about 85% of the time. This includes letting the robot automatically try again if it recognizes that it has failed at a specific behavior. Each task is comprised of about 45 independent behaviors, meaning all individual behavior results in recoverable success or failure 99.6% of the time,” wrote the TRI.

However, the system is not perfect. In a video above, TRI reminds viewers that it creates research prototypes, not product concepts. Still, the project can change the way robots learn and support those who need help with basic household tasks.