NVIDIA Research team has developed a new AI agent that can teach robots complex skills, such as spinning a pen with high speed and precision. The video below demonstrates how the robotic hand can match the human performance in this challenging task.
The AI agent, called Eureka, can automatically write reward algorithms that guide the robots to learn new skills. It has enabled robots to master almost 30 different tasks, such as opening drawers and cabinets, throwing and catching balls, and using scissors, among others.
“Reinforcement learning has enabled impressive wins over the last decade, yet many challenges still exist, such as reward design, which remains a trial-and-error process,” said Anima Anandkumar, senior director of AI research at NVIDIA and an author of the Eureka paper. “Eureka is a first step toward developing new algorithms that integrate generative and reinforcement learning methods to solve hard tasks.”
The paper reports that Eureka-generated reward programs outperform expert human-written ones on more than 80% of tasks. This leads to an average performance improvement of more than 50% for the bots.
Eureka works with the GPT-4 LLM and generative AI to write software code that rewards robots for reinforcement learning. It doesn’t require task-specific prompting or predefined reward templates but rather adapts to the goals and preferences of the developer. The AI agent also uses human feedback to modify its rewards for results more accurately aligned with a developer’s vision.
Using GPU-accelerated simulation in Isaac Gym, Eureka can quickly test and optimize reward functions for various robotic tasks and domains. Eureka then constructs a summary of the key stats from the training results and instructs the LLM to improve its generation of reward functions. In this way, the AI is self-improving. It’s taught all kinds of robots – quadruped, bipedal, quadrotor, dexterous hands, cobot arms, and others – to accomplish all kinds of tasks.
The research paper provides in-depth evaluations of 20 Eureka-trained tasks based on open-source dexterity benchmarks that require robotic hands to demonstrate a wide range of complex manipulation skills.
“Eureka is a unique combination of large language models and NVIDIA GPU-accelerated simulation technologies,” said Linxi “Jim” Fan, one of the project’s contributors, in an official statement. “We believe that Eureka will enable dexterous robot control and provide a new way to produce physically realistic animations for artists.”
