Building a Practical Taxonomy for AI World Models

The term world model is used almost everywhere in AI now.

But the more often it appears, the less clear it sometimes become.

A reinforcement learning researcher may use the term for a latent dynamics model. A robotics team may use it for an action-conditioned simulator. A video generation company may describe a large generative model as a world model. An autonomous driving company may use the same expression for a system who creates traffic scenarios.

All of these systems have something in common.

But they are not solving exactly the same problem.

That was the starting point for our report:

State of World Models 2026: Taxonomy, Benchmarks and Open Challenges

We did not wanted to build another leaderboard with one global score. That would have been easy to understand, but probably misleading.

Instead, we tried to answer a more basic question:

How can we describe world models in a consistent way before trying to compare it?

A working definition

For the report, we used this definition:

A world model is an AI model that learns a representation of an environment and uses it to predict, simulate, evaluate or support action inside that environment.

The definition is intentionally broad.

It can include:

• model-based reinforcement learning
• video prediction
• robotics simulators
• embodied AI
• autonomous driving models
• spatial and 3D models
• procedural or agentic environments

But not every generative model should automatically be called a world model.

A model may generate a beautiful video and still fail to preserve object identity, physical consistency, spatial structure or the consequence of actions.

That is one of the main problems in the current discussion.

A model can look like it understands a world without being really useful for acting inside it.

Why a universal ranking does not really work

Imagine three models:

1 One creates extremely realistic videos
2 One helps a robot to plan movements
3 One generates rare traffic situations for autonomous driving

Which one is the best world model?

There is no serious answer without more context.

The first may be better visually.

The second may be better for planning.

The third may be much more useful for safety testing.

Putting all three into one score would hide most of the important differences between them.

That is why we moved toward a taxonomy rather than a global ranking.

The classification framework

The framework uses a set of practical fields.

Field	Description
Name	Public model or benchmark name
Organization	Company, lab or institution
Publication year	First public release, paper or announcement
Domain	Robotics, video, driving, RL, embodied AI, agents
Input modalities	Text, image, video, action, state or sensor data
Output modalities	Video, state, action, trajectory or simulation
Action-conditioned	Yes, no, partial or unknown
Representation	Latent, pixel, token, 3D, symbolic or hybrid
Temporal horizon	Short, medium, long or procedural
Evaluation type	Perceptual, physical, functional or planning
Availability	Paper, code, weights, dataset, demo or closed
Limitations	Known weaknesses or missing informations

This is not meant to be a perfect academic ontology.

It is a practical tool for navigating a field that is changing very fast, maybe too fast sometimes.

Domain still matters

The first thing to identify is what kind of environment the model is supposed to represent.

Current categories include:

• reinforcement learning
• robotics
• embodied AI
• generative video
• autonomous driving
• games
• industrial simulation
• spatial intelligence
• software agents

This may sounds obvious, but it changes almost everything.

A robotics model and a video model may both predict future states. But one is expected to help a machine act, while the other may mainly be expected to generate plausible scenes.

The evaluation should reflect this difference.

Function matters too

Two models can work in the same domain and still have very different purpose.

A world model may be designed for:

• future-state prediction
• simulation
• planning
• policy evaluation
• synthetic data generation
• representation learning
• counterfactual reasoning

Some systems combine several of these functions. Others are very specialised.

A visually rich model may be excellent for simulation but difficult to use for explicit planning.

A small latent model may be almost impossible to interpret visually, but very useful for an agent.

Internal representation

World models can represent environments in very different ways.

Some work directly with pixels or video frames.

Others use:

• latent vectors
• tokens
• object-centred states
• spatial grids
• 3D geometry
• symbolic variables
• hybrid representations

Each choice come with trade-offs.

Pixel-based systems preserve more visual detail, but they can be expensive and difficult to reason over.

Latent systems are more compact, but their internal states are usually harder to interpret.

Object-based and symbolic systems may support planning more naturally, but they can require stronger assumptions about the environment.

There is no obvious winner at this moment.

Time horizon is often overlooked

A model that predicts the next state is not necessarily able to simulate a useful future.

We therefore distinguish between:

• next-state prediction
• short-term continuation
• medium-length rollouts
• long-horizon prediction
• procedural planning

This matters because many models look strong over a few frames and become unstable over longer period.

Small errors accumulate.

Objects drift. Geometry changes. The state slowly stop making sense.

For planning, this problem becomes even more serious because every mistake can affect all the next decisions.

Action conditioning may be the key distinction

One of the most important questions is whether the model understands actions.

A passive model estimates:

P(sₜ₊₁ | sₜ)

An action-conditioned model estimates:

P(sₜ₊₁ | sₜ, aₜ)

Where:

• sₜ is the current state;
• aₜ is the selected action;
• sₜ₊₁ is the predicted next state.

The difference looks small on paper.

In practice, it changes the whole use case.

A passive model answers:

What may happen next?

An action-conditioned model answers:

What may happen if I do this?

For robotics, autonomous driving and software agents, that second question is usually the one which is useful.

Evaluation is still fragmented

Current benchmarks evaluate different parts of world modelling.

Some focus on visual quality.

Others look at physical consistency, long-horizon reasoning, action conditioning or downstream performances.

We separated evaluation into four broad groups.

Perceptual evaluation

This asks whether the output looks convincing.

Examples include:

• video quality
• image quality
• temporal smoothness
• human preference

These measures are useful, but limited.

Physical evaluation

This asks whether the environment behaves in a plausible way.

Possible criteria include:

• gravity
• collision behaviour
• motion continuity
• object permanence
• geometry preservation
• spatial relations

This is much harder to evaluate automatically and results are not always easy to compare.

Functional evaluation

This asks whether the model actually helps with a task.

For example:

• does it improve robot planning?
• does it reduce real-world interactions?
• does it help evaluate a policy?
• does it generate useful training data?
• does it improve task success?

This kind of evaluation is often more important than visual quality, but also more expensive to perform.

Planning evaluation

This asks whether the model helps to make multi-step decisions.

A model may need to compare possible trajectories, estimate risk, select actions or maintain state over a long sequence.

Current systems still struggle a lot on this.

The perception-functionality gap

One of the most useful ideas in current world-model research is the gap between perception and functionality.

A model can generate a sequence that looks excellent to a human viewer and still be unusable for a robot.

A small error in object position may be almost invisible in a video.

For a robot trying to grasp that object, the same error can make the full prediction useless.

This is why visual realism cannot be the only target.

A useful world model should help an agent making better decisions.

That sounds simple, but it changes the way models should be evaluated.

A simple data structure

A world-model catalog can be represented with a fairly simple JSON structure.

{
  "name": "Example World Model",
  "organization": "Example Lab",
  "year": 2026,
  "domains": [
    "robotics",
    "embodied-ai"
  ],
  "inputs": [
    "video",
    "actions"
  ],
  "outputs": [
    "future-video",
    "trajectory"
  ],
  "action_conditioned": "yes",
  "representation": [
    "latent",
    "video"
  ],
  "temporal_horizon": "medium",
  "evaluation": [
    "perceptual",
    "functional"
  ],
  "availability": {
    "paper": true,
    "code": false,
    "weights": false,
    "dataset": true
  },
  "limitations": [
    "Limited public implementation details",
    "Evaluation restricted to simulated environments"
  ]
}

This makes it easier to:

• filter by domain;
• compare action-conditioned systems;
• identify open-source projects;
• separate robotics models from video models;
• track benchmarks;
• find missing information.

It also avoid rewriting every profile by hand whenever the taxonomy changes.

Unknown is better than guessing

A lot of frontier models are not documented well enough.

In those cases, the correct value is often:

unknown

Not:

probably yes

A company may claim that a model understands physics without publishing the evaluation protocol.

A demo may suggest action conditioning without clearly documenting the model interface.

A paper may describe a dataset without releasing it.

The catalog should reflect what is known, not what seems probably true.

This is especially important for proprietary systems, where public documentation can be incomplete.

Evidence and interpretation should be separated

A good model profile should clearly distinguish between:

1. documented information
2. external evaluation
3. editorial interpretation

For example:

Documented:
The model accepts video and action inputs.

Externally evaluated:
The model was tested on benchmark X.

Editorial interpretation:
The available results suggests that long-horizon consistency remains limited.

This makes the information easier to trust and easier to correct later.

Benchmarks need their own structure

Benchmarks should not be mixed directly into the same table as models.

They need separate fields.

A benchmark record may look like this:

{
  "name": "Example Benchmark",
  "year": 2026,
  "domains": [
    "video",
    "embodied-ai"
  ],
  "evaluates": [
    "physical-consistency",
    "action-conditioning",
    "functional-utility"
  ],
  "evaluation_methods": [
    "automated-metrics",
    "human-evaluation",
    "downstream-task"
  ],
  "public_dataset": true,
  "public_code": true,
  "limitations": [
    "Limited environment diversity"
  ]
}

Models can then be connected to the benchmarks they were tested on.

This helps preserve:

• benchmark version;
• result date;
• reported score;
• source reference;
• evaluation conditions.

That last point is important because benchmark results can become outdated quite fast.

Versioning is essential

The world-model landscape changes too fast for a static database.

Models are updated.

Benchmarks evolves.

New papers correct earlier claims.

A basic versioning approach could be:

v1.0.0 — Initial public dataset
v1.1.0 — New models and benchmarks
v1.1.1 — Corrections and metadata fixes
v2.0.0 — Major taxonomy revision

Each release should include:

• a publication date;
• a changelog;
• a dataset snapshot;
• a methodology version;
• citation information.

This prevents a report or dataset to change silently after publication.

What we exclude

A world-model directory can easily become a general AI catalog.

Some exclusion rules are needed.

For now, the framework excludes:

• static image generators without temporal dynamics;
• general language models without a world-state use case;
• traditional simulators without a learned component;
• marketing claims without enough technical evidence;
• projects with too little public information;
• systems where “world model” is used mainly as branding.

This does not mean those systems are not useful.

It simply helps to keep the scope understandable.

Open source is not the same as reproducible

A project may provide only a paper.

Another may provide code but no weights.

Another may provide weights but no training pipeline.

So availability needs more detail than just “open” or “closed”.

For example:

{
  "paper": true,
  "code": true,
  "training_code": false,
  "weights": false,
  "dataset": true,
  "evaluation_scripts": true,
  "demo": true
}

This gives a much more realistic picture about reproducibility.

The framework still has limits

The taxonomy is not finished.

The term world model is still unstable.

Many systems belongs to several categories.

Public information is incomplete.

Different communities use different evaluation standards.

And no structured dataset can capture all the nuance of a paper or architecture.

The framework should therefore be used as a navigation tool.

It does not replace reading the original research.

Why publish it openly

The main reason to make the taxonomy public is simple : it needs to be corrected.

A closed database would become outdated very quickly.

An open structure allows researchers, engineers and readers to:

• report missing models
• correct factual errors
• suggest better sources
• challenge a classification
• propose new fields
• point out benchmark weaknesses

That does not mean every suggestion must be accepted.

But the logic behind classification decisions should stay visible.

What comes next

The next steps are:

• publish a structured public dataset
• add benchmark-level metadata
• document sources more precisely
• create domain-specific comparison pages
• maintain a public changelog
• open a contribution process
• publish future versions of the report

The full report is available here :

State of World Models 2026

Archived publication and DOI:

Zenodo record

Project website:

world-models.io

Feedback, corrections and missing references are welcome.

The framework is still early, and it will probably need to evolve together with the field.

Regards,