TL;DR — Vision-language-action (VLA) models like OpenVLA give robots a slow, high-level 'brain' that plans grasps and subgoals, while a classical controller still closes the real motion loop at hundreds of Hz. Open-weight VLAs let labs and factories fine-tune manipulation policies on their own hardware without shipping video to the cloud, but the models are still compute-bound multi-billion-parameter systems that need real GPUs, not microcontrollers.
A robot arm doesn't think the way a chatbot does. It can't afford to. Ask a language model a question and a half-second of latency is invisible. Ask a robot to close its fingers around a coffee mug and a half-second of hesitation is a dropped mug. That mismatch — between how slow it's safe for a "brain" to think and how fast a body has to move — is the whole engineering story of open-weight AI in robotics, and it's why the field ended up splitting the job into two systems instead of one.
The split brain: intent at 1-10 Hz, motion at 1,000 Hz
No published on-robot vision-language-action (VLA) system runs the actual balance or motion control loop. Instead, a VLA or LLM component runs at roughly 1-10 Hz on onboard compute — Jetson-class boards or Hailo-10H NPUs — and produces task-level intent: a target pose, a grasp point, a subgoal, according to a detailed breakdown of on-robot inference constraints from promptquorum.com. A separate classical controller — PID or MPC, usually on a microcontroller or dedicated real-time core — closes the actual motion loop at 100 to 1,000 Hz. The VLA says "pick up the mug." The controller figures out, a thousand times a second, exactly how many degrees to rotate each joint to get there without knocking anything over.
This isn't a workaround, it's the architecture. A multi-billion-parameter model doing vision and language reasoning is compute-bound work — the kind that wants a real GPU, not a microcontroller — as semianalysis.com put it plainly in a piece on where robots actually do their thinking. Nobody is routing an emergency stop through a language model. Safety interlocks stay deterministic; the LLM only gets to propose, never to directly actuate.
OpenVLA: what open-weight looks like in the wild
The reference point for open VLA work is OpenVLA, a 7-billion-parameter open-source model pretrained on 970,000 robot episodes from the Open X-Embodiment dataset. It's built from a fused SigLIP + DinoV2 visual encoder feeding a Llama 2 7B backbone that predicts tokenized actions, decoded straight into joint commands. On out-of-the-box evaluation across the WidowX (Bridge V2) and Google Robot (RT-series) platforms, OpenVLA's authors report it outperforming the prior generalist policies RT-1-X and Octo — and notably outperforming RT-2-X, a 55-billion-parameter closed VLA nearly eight times its size.
The more interesting number for practitioners is adaptation speed. The team fine-tuned OpenVLA onto two new physical setups: a stationary Franka Emika Panda arm controlled at 5 Hz, and the Franka-DROID rig running at 15 Hz — using parameter-efficient fine-tuning rather than a from-scratch training run. That's the open-weight pitch in miniature: you're not building a manipulation policy from zero, you're adapting a pretrained one to your particular arm, your particular gripper, your particular lab bench.
Where this actually gets used
Three constraints keep showing up in real deployments, and they explain why open weights matter here specifically rather than just being ideologically nice to have:
Academic and startup manipulation research. Labs running Franka arms or WidowX setups don't have RT-2-X's training budget or Google's fleet of robots. OpenVLA gives them a checkpoint that already generalizes to unseen objects, positions, and instructions, and lets them spend their compute on fine-tuning to a specific task rather than on pretraining a vision-language-action stack from scratch — a cost gap that's the difference between a PhD project shipping and not shipping.
Multi-embodiment fleets without a single vendor lock-in. OpenVLA supports multiple robot platforms out of the box and is designed to be adapted to new robot setups via parameter-efficient fine-tuning. That matters for anyone running a mixed fleet of arms, grippers, and mobile bases who doesn't want a separate closed model contract per robot body.
Sites that can't or won't send camera feeds anywhere. The same logic that keeps factory-floor LLMs air-gapped — process recipes and order histories being a legal and competitive-intelligence risk, not just a technology one, as one industrial deployment writeup put it via dev.to — applies directly to a robot arm's camera stream. A manipulation policy watching a proprietary assembly line is watching something a competitor would pay to see. Running the VLA on local Jetson-class compute instead of a cloud API means that footage never leaves the cell.
Field robots without reliable connectivity. Warehouse pickers, agricultural robots, and disaster-response platforms operate exactly where the "just call an API" model breaks: intermittent or nonexistent connectivity, and a task-level decision that has to happen in under a second regardless. The split-brain architecture — slow VLA for intent, fast local controller for motion — is the only version of "AI-driven robot" that survives a dead network connection.
The ceiling, and where closed models still lead
Be honest about the gap. Google DeepMind's Gemini Robotics 2 announcement demonstrated a humanoid — Apptronik's Apollo — walking to a table, picking up a watering can, and placing it on a shelf from a single natural-language instruction, with a companion "Gemini Robotics On-Device 2" model that DeepMind says can adapt to a new bi-arm embodiment in a few hours using fewer than 200 examples. That's a vendor claim about a closed model, not an open one, and whole-body humanoid control is a meaningfully harder problem than tabletop manipulation — but it's a real signal of where the frontier is moving, and it's the bar open VLA projects are implicitly racing toward.
The honest trade-off: open-weight VLAs give you inspectable, fine-tunable, deployable-anywhere manipulation policies today, at the 7B-parameter, single-arm, tabletop scale that OpenVLA operates at. They don't yet give you out-of-the-box whole-body humanoid coordination or multi-robot teaming. If your job is teaching a Franka arm a new grasp without sending video to anyone, open weights already do it. If your job is a humanoid crossing a room, the frontier is still mostly closed.
Credits & sources
Technical detail on the VLA control-loop split and edge compute constraints comes from promptquorum.com. The compute-bound argument for why VLAs need real GPUs rather than microcontrollers is from semianalysis.com. Model architecture, benchmark, and fine-tuning figures for OpenVLA are drawn from the project's own page at openvla.github.io, maintained by its research authors. Details on Gemini Robotics 2 and its on-device variant come from <a href="https://deepmind.google/blog/gemini-robotics-2-br
Appendix — the field in one chart
Top comments (0)