Robotics Has a 95% Data Gap. World Models Multiply Data — They Don't Create It.

Robotics is having its "GPT-2 moment" — everyone can feel that the architecture works, and everyone is quietly panicking about the data.

Here is the number that reframed the problem for me. As of early 2026, the global stock of high-quality embodied interaction data — real robots doing real tasks, recorded well enough to train on — sits at roughly 500,000 hours. Estimates for what a general-purpose embodied foundation model needs run to 10 million hours or more. That is a shortfall north of 95%, and unlike text, you cannot scrape your way out of it. Nobody crawled the internet for 10 million hours of a gripper failing to open a drawer.

The field's answer, and the theme of this year's Embodied AI Workshop, is world models as data engines: train a generative model of environment dynamics, then use it to synthesize the trajectories you cannot afford to collect. GigaWorld-0 and similar systems render texture-varied scenes, novel viewpoints, and ego-centric translations to inflate a small real dataset into a large training corpus for vision-language-action (VLA) models. It works. Reported scale gains are 10–100x.

It also quietly recreates every failure mode we already learned the hard way in LLM land — and if you have been anywhere near a data pipeline in the last two years, the shape of the trap should look extremely familiar.

The synthetic-data loop has a known failure curve

We have run this experiment before. When a model's outputs become a meaningful share of the next model's training inputs, distributional tails collapse first. Rare events, unusual lighting, weird object geometries, the 3am edge cases — those thin out generation over generation while the mean of the distribution looks fine and your aggregate metrics stay flat or even improve.

In text, this shows up as blandness. In embodied AI, it shows up as a robot that handles the demo flawlessly and destroys a coffee cup that happens to be slightly translucent.

The mechanism is worth being precise about, because "synthetic data is bad" is the wrong lesson. Synthetic data is excellent at interpolation — filling in the space between things your real data already covers. It is structurally incapable of extrapolation into physics it never observed. A world model trained on rigid-body manipulation will happily generate confident, plausible, entirely wrong video of a cloth fold. The rendering is beautiful. The dynamics are fiction. Your VLA model trains on that fiction and learns a policy that fails on contact.

So the practical question is not "real or synthetic." It is: which slices of the distribution must be real, and how do you know when a synthetic slice has drifted?

Three rules that seem to hold

Working with teams building perception and action datasets, a few heuristics have earned their keep.

1. Real data anchors the contact boundary. Anywhere the robot touches the world — grasp initiation, slip, deformation, force feedback — needs real recordings. This is where simulators and world models are weakest and where policy failures are most expensive. Synthetic augmentation is fine for backgrounds, lighting, camera pose, distractor objects. It is not fine for the physics of the moment of contact.

2. Failure data is worth more than success data, and nobody collects it. Most robot datasets are demonstration datasets: an expert teleoperates the task correctly, N times. The resulting policy has never seen recovery. It does not know what "the grasp is slipping" looks like because that frame was never labeled, or the take was discarded. Deliberately collecting near-misses and recoveries — and annotating the transition point where things went wrong — changes downstream robustness more than another 10,000 clean demos. This mirrors what we found with coding and computer-use agents: models fail mid-trajectory, and mid-trajectory correction data is what fixes them.

3. You need a contamination check between your world model and your eval set. If the same generative model produced both training trajectories and evaluation scenarios, your benchmark is measuring self-consistency, not competence. This is the embodied version of the benchmark contamination problem that hit SWE-bench this year, where audits found frontier models could reproduce verbatim details of supposedly held-out tasks. Hold out real-world evaluation episodes, recorded on hardware, that no generative model in your stack has ever seen. Yes, it is slow and expensive. It is also the only number you can trust.

What this means for annotation, concretely

The unglamorous truth is that embodied data quality is mostly an annotation problem, and embodied annotation is much harder than the image-classification work most vendors are tooled for.

A single manipulation episode may need:

• Temporal segmentation — where does "reach" end and "grasp" begin? Boundaries are fuzzy and annotator disagreement is high without a tight rubric.
• 3D spatial labels — LiDAR or depth-derived bounding boxes and pose, consistent across frames, not just per-frame boxes that jitter.
• Cross-modal alignment — RGB, depth, joint states, force/torque, and language instruction all on one clock. Sub-100ms drift between streams silently teaches the model the wrong causality.
• Outcome and failure-mode labels — did this succeed? If not, at which frame did it become unrecoverable, and why?

That last one is the expensive one, because it requires judgment rather than tracing. An annotator has to understand the task well enough to say "this grasp was doomed at frame 412, when the approach angle drifted past the object's stable face." That is domain expertise, not clickwork — the same category of judgment that reasoning and preference data demands, which is why teams increasingly staff these projects with people who understand the domain rather than generalist labelers. It's the model of work we've built multimodal annotation and reasoning and human-feedback data programs around at SyncSoft.AI, especially on LiDAR and video for automotive and ADAS clients — and the pattern generalizes cleanly from driving scenes to manipulation.

Two practical notes from doing this at volume:

Inter-annotator agreement is your leading indicator. If two qualified annotators disagree on where a phase boundary sits more than ~10% of the time, your rubric is broken, not your annotators. Fix the rubric before you scale the batch. Teams that skip this ship a dataset with a noise floor higher than the effect they are trying to measure.

Multi-pass review is not optional on temporal data. A single-pass QA process catches spatial errors reasonably well and temporal errors badly, because reviewing a boundary means scrubbing the clip, not glancing at a frame. Budget for it.

A pipeline shape that works

If I were standing up an embodied data program today:

1. Collect a real seed set covering your task family, deliberately including failures and recoveries. Small — hundreds of hours, not thousands. Prioritize diversity of objects and conditions over volume of repetitions.
2. Annotate it to a rubric you have already stress-tested on 50 episodes with three annotators, measuring agreement. Iterate the rubric until agreement stabilizes.
3. Fit the world model on that seed, then generate augmentation — but tag every synthetic episode with provenance. You will need to ablate it later.
4. Train, then ablate. Train one variant on real-only, one on real+synthetic. If synthetic helps on your real held-out eval, keep it. If it only helps on synthetic evals, you have measured nothing.
5. Evaluate on real hardware episodes with clean provenance. Track per-slice performance, not just aggregate success rate. Aggregate numbers hide tail collapse for a long time.
6. Re-seed quarterly. As the policy improves, its failure distribution shifts. Yesterday's failure data is about yesterday's policy. This is the part everyone skips and everyone regrets.

The uncomfortable summary

World models are a genuine breakthrough and they will absolutely be part of how the embodied data gap closes. But they multiply the data you have; they do not create data you never had. A 100x multiplier on a biased seed set gives you a very large, very confident, very biased dataset — and the metrics will look great right up until deployment.

The teams that win the next two years of robotics will not be the ones with the biggest synthetic pipeline. They will be the ones who were disciplined about which 5% of their data had to be real, and rigorous about proving their evaluations weren't quietly grading their own homework.

That is a boring conclusion. Most durable ones are.

---

I work at SyncSoft.AI, where we build annotation, reasoning-data, and model evaluation pipelines for AI teams — including LiDAR and multi-sensor work for automotive and robotics. If you're wrestling with a sim-to-real data gap of your own, happy to compare notes.