Robotics: Firms race to improve robot training systems

Freddo the robot walks across the office and takes a plastic bottled offered by a staff member.

Given that a robot recently beat Usain Bolt's 100m sprint record, it's not the most startling achievement.

But the speed by which Freddo has been trained to walk, recognise the bottle and grasp it is impressive. It took just a few minutes to develop those skills and upload them to Freddo. His developers say rival systems could take days to attain such skills.

I'm at Vsim, a British start-up based in Cambridge. Founders Michelle Lu and Kier Storey hope one day their software will control robots that can navigate and do useful tasks in the home and workplace.

"It's a weird situation with robotics because actually the stuff that we find as humans to be incredibly difficult, like gymnastics, you can get robots to do reasonably well. The stuff that humans are really good at, like fine dexterity, is really hard in robots," Storey says.

Freddo's skills were honed in a virtual environment, where a task can be performed in a computer simulation millions of times. Once the optimum solution (known as a policy) is found, it can be uploaded and used by the hardware - in this case Freddo.

Such virtual simulations are a common way to train robots. Tech giant Nvidia has a system called Isaac Sim which works that way - Lu and Storey both worked on an early version of it.

In 2022 they decided to set up Vsim, to build the their own training system environment and other tools.

As they were starting from scratch Lu and Storey could optimise the software to exploit the powerful computer chips used in AI, known as graphics processing units or GPUs.

"The underlying algorithms that we were using for most of these robotic simulations they hark back to the 1970s and 1980s, but those algorithms are not really brilliant fits for GPUs," Storey says.

Within months they realised their system could work much faster than anything they had seen before.

"Eighteen months in and we actually have a completely functional, super high-performance simulator," says Lu.

The software is so efficient that it can run on the hardware carried by Freddo. That means the robot can run tens of thousand of simulations while it is moving around.

"It can look about a second, or so, ahead into the future for 20,000 different kind of combinations of things that might happen," Storey explains.

And that would be vital for a robot moving around an unstructured environment like the average home.

"Things outside of the robot's control, like humans, animals or even other robots, could do things that require a change of strategy. These unexpected events could happen very quickly and the robot needs to be able to quickly adapt to ensure its actions remain safe and on-mission," Lu says.

Vsim is a start-up with 10 engineers working on its tech. Nvidia is at the other end of the industry. It dominates the market for computer chips used for AI and has a leading robotics software division, with hundreds of engineers.

It does not build robots, instead it has a suite of software designed to let organisations train and control robots.

That includes virtual simulation training systems and a so-called world model , called Cosmos, which gives a robot an understanding of the physics of the real world and how its environment might change as it moves around.

But even with the powerful computer resources available to Nvidia, the software only gives a rudimentary understanding of the real world.

"Manipulation, - where I just grab a bottle, that's not too hard. The problem is when you start doing long-horizon tasks, where I say: 'I want you to take the bottle and I want you to fill it up and I want you to go pour'," says Spencer Huang, director of product for robotics at Nvidia.

But he's confident that good progress is being made. This year Nvidia has started using AI agents to help build virtual environments to train robots and validate whether the solutions from training work or not.

"When we talk about creating the [virtual] world and actually scanning it in - a lot of that is actually manual labour.

"We're just throwing agents at it... it's basically given us a huge workforce," Huang says.

Simulation is not the only method for training robots. They can also be trained by watching human or video demonstrations.

Rika Antonova has been working in the field of robotics since 2015 and is currently an associate professor at the Department of Computer Science and Technology at the University of Cambridge.

Her research is focused on developing software and hardware that can aid robots to learn complex behaviour.

Antonova works with a training system called MuJoCo, owned by Google's DeepMind since 2021. It's open-source software, which means researchers can use it for free, and are allowed to tinker with the code.

"It is very, very user-friendly. So for research groups or for small start-ups, that's useful," she says.

She says that Vsim's approach - very fast simulation - is promising.

"If you have a very, very fast simulator, then you can simulate hundreds of millions of samples in that few seconds that your robot is thinking about how to adjust its motion, and then you can change the motion almost in real time," she says.

But those simulated environments are still rough approximations of the real world, which limits what can be trained.

"There are certain things that are hard to model in simulation, like highly deformable objects and cutting," she says.

It's a challenge that Nvidia and Lu and Storey at Vsim are working on.

Lu says their system has "reduced approximation, using accurate simulations to train models that genuinely work in reality as well as they do in simulations."

Soon a second robot, to be called Nacho, will be helping develop that tech.

Lu says that should speed up their development process and ensure their software can run on different machines.

And, of course, provide Freddo with some company.

The critical tech staying safe by going underground

Why airlines are warning over lithium-ion batteries