Advancements in Robot Training Using Virtual Environments

Freddo the robot walks across the office and takes a plastic bottle offered by a staff member. The speed at which Freddo has been trained to walk, recognize the bottle, and grasp it is notable, taking just a few minutes to develop those skills and upload them to Freddo. His developers state that rival systems could take days to achieve similar capabilities. Vsim, a British start-up based in Cambridge, was founded by Michelle Lu and Kier Storey, who aim to create software that will enable robots to navigate and perform tasks in homes and workplaces. Storey notes the peculiarities of robotics, stating that while humans find tasks like gymnastics difficult, robots can perform them reasonably well, whereas fine dexterity remains challenging for robots. Freddo's skills were developed in a virtual environment, allowing for the performance of tasks in computer simulations millions of times. Once the optimal solution is identified, it can be uploaded to the hardware, in this case, Freddo. This method of training robots is common, with tech giant Nvidia offering a system called Isaac Sim, which Lu and Storey previously worked on. In 2022, they established Vsim to build their own training system and tools. Starting from scratch allowed them to optimize the software for the powerful computer chips used in AI, known as graphics processing units (GPUs). Storey explains that the algorithms used in robotic simulations date back to the 1970s and 1980s but are not well-suited for GPUs. Within months, they realized their system could operate much faster than previous models. Lu states that after 18 months, they have developed a fully functional, high-performance simulator. The software's efficiency allows it to run on the hardware carried by Freddo, enabling the robot to conduct tens of thousands of simulations while moving. Storey explains that the robot can anticipate various scenarios about a second into the future, which is crucial for navigating unstructured environments like homes. Lu adds that unexpected events, such as actions by humans or animals, require the robot to adapt quickly to maintain safety and mission focus. Vsim currently employs 10 engineers, while Nvidia has a significant presence in the AI chip market and a leading robotics software division. Nvidia does not manufacture robots but provides software for organizations to train and control them, including virtual simulation training systems and a world model called Cosmos, which helps robots understand real-world physics. However, even with Nvidia's resources, the software offers only a basic understanding of the real world. Spencer Huang, director of product for robotics at Nvidia, emphasizes that while basic manipulation tasks are manageable, complex tasks require more advanced capabilities. Huang notes that Nvidia has begun using AI agents to assist in building virtual environments for robot training and validating training outcomes. Additionally, robots can be trained by observing human demonstrations. Rika Antonova, an associate professor at the University of Cambridge, has been involved in robotics since 2015 and focuses on developing software and hardware for complex robot behaviors. She works with MuJoCo, an open-source training system owned by Google's DeepMind, which is user-friendly for researchers and small start-ups. Antonova acknowledges Vsim's fast simulation approach as promising, stating that it allows for the simulation of millions of samples in seconds, enabling real-time adjustments to robot motion. However, she notes that simulated environments are still rough approximations of reality, limiting training effectiveness. Antonova highlights challenges in modeling certain objects and tasks in simulation, such as deformable objects and cutting. Both Nvidia and Vsim are addressing these challenges. Lu mentions that their system has reduced approximation errors by using accurate simulations for training models that perform effectively in real-world scenarios. A second robot, named Nacho, is expected to assist in developing this technology and provide Freddo with companionship.