Programming a robot used to mean scripting every single step. These five models are why that's changing fast.
Not long ago, getting a robot to perform a task meant programming every single step. Today, robots are beginning to understand language, recognize objects, and learn new skills. The AI models for robotics behind that shift are mostly a new class called vision-language-action (VLA) models, which take in what a robot sees plus an instruction, and output the actions to carry it out. Here are five driving the transformation.
1. RT-2: the model that connected language to robot actions
First up is RT-2 from Google DeepMind. Released in 2023, it's the model that named the vision-language-action category.
Its core trick was representing robot actions as text tokens, which let a single model train on both web-scale image and text data and real robot data. The result was robots that could connect what they see with instructions written in natural language. Instead of following fixed commands, they could start to work out what a human actually meant, including for objects and phrasings they'd never seen in robot training data.
2. OpenVLA: bringing vision-language-action models to open source
Another major step came with OpenVLA, a 7-billion-parameter model from Stanford and Toyota Research Institute.
Its contribution was access. It brought powerful vision-language-action models into the open-source world, so any lab or developer could download the weights, fine-tune on their own robot, and experiment with modern robotics AI. That turned VLAs from something a handful of large teams could explore into something the whole field could build on.
3. NVIDIA GR00T: a foundation model for humanoid robots
NVIDIA is taking a different approach with GR00T. Rather than targeting one task, GR00T N1 is designed as an open foundation model to help humanoid robots perform a wide range of everyday activities.
It uses a dual-system design: a vision-language model handles the reasoning about what it sees and hears, while a separate action module generates the actual motor commands. From one set of weights it can drive single-arm, bimanual, and humanoid bodies, which is exactly the kind of generality that humanoid robots need to be practical.
4. LeRobot: an open platform for AI-powered robots
Then there's LeRobot from Hugging Face. Calling it a model undersells it. It's an open robotics platform with datasets, pre-trained models, simulation environments, and tools that help developers build AI-powered robots faster.
The idea mirrors what Hugging Face did for language models: gather the data, the models, and the tooling in one place so people can start from proven work instead of scratch. We took a closer look at how the LeRobot codebase is structured if you want to explore it yourself.
5. Pi-Zero: one model across many robots and tasks
Finally, Pi-Zero, written as π0, from Physical Intelligence. Its goal is ambitious: a single AI model that adapts across different robots and different tasks.
It pairs a pre-trained vision-language model with a flow-based action expert, and trains across data from many robot types rather than one. That cross-embodiment approach, one brain for many bodies, is a real step toward general-purpose robotics, and it's shown up in demos doing everyday work like folding laundry and clearing tables.
What these AI models mean for the future of robotics
These five take different approaches, but they're aimed at the same future. Robots are moving past following instructions and toward learning, adapting, and interacting with the world around them, which is the core promise of embodied AI.
One thread runs through all of them: scale. Each depends on huge amounts of training data, and much of it comes from simulation, where a model can practice millions of times before it touches real hardware.
FAQ
What are vision-language-action (VLA) models?
VLA models take in what a robot sees along with a language instruction, and output the robot's actions directly. They let a robot follow natural-language commands and generalize to objects and situations it wasn't explicitly trained on, rather than executing hard-coded steps.
What is RT-2?
RT-2 is a vision-language-action model from Google DeepMind, released in 2023. It represents robot actions as text tokens so a single model can learn from both web data and robot data, which lets it apply general knowledge from the internet to physical control.
What is the best open-source robotics AI model?
OpenVLA is a common starting point, since its weights are openly available and it can be fine-tuned on a new robot with limited data. Hugging Face's LeRobot is the go-to open platform, providing datasets, pretrained models, and tooling around models like these.
What is NVIDIA GR00T?
GR00T is NVIDIA's open foundation model for generalist humanoid robots. It combines a vision-language model for reasoning with an action module for motor control, and a single set of weights can operate single-arm, bimanual, and humanoid robots.
What is Pi-Zero (π0)?
Pi-Zero is a vision-language-action model from Physical Intelligence built for general robot control. It's trained across data from many different robots, with the goal of one model that adapts to different bodies and different tasks.
How are these robotics AI models trained and tested?
Largely in simulation, where a policy can be trained and evaluated at massive scale before it runs on expensive hardware. Tools like Drift generate the simulated robots and environments used for that testing, while the models themselves are trained on top of those simulations.
Top comments (0)