Agents Need Feedback Loops. World Models Add a Way to Look Ahead.

I find “LLM agents versus world models” an unhelpful framing. They address different parts of the same problem.

An agent organizes work toward a goal: reason, plan, choose tools, act, and evaluate the result. A world model predicts how an environment might change when an action is taken.

The useful question is how to connect them:

Goal → Plan → Simulate → Choose → Act → Observe → Update
         ↑                                         │
         └─────────────────────────────────────────┘

That architecture matters more to me than whether the next reasoning model tops a benchmark. Better models help, but autonomous work depends on what happens between model calls—and whether the system can recover when reality disagrees with its plan.

Start With the Workflow, Not the Chat Interface

GPT, Claude, Gemini, Qwen, DeepSeek, and Grok have expanded what developers can do with language models: reasoning, coding, analysis, content generation, and interaction with software.

A conventional application still often has this shape:

User → Prompt → LLM → Answer

That is enough for plenty of tasks. It is not enough for a request like:

Analyze the AI API market in the United States and create a go-to-market strategy.

This is a workflow, not a single question:

Understand objective
→ Search the web
→ Collect competitors
→ Analyze market data
→ Organize findings
→ Develop hypotheses
→ Draft strategy
→ Evaluate
→ Revise

The transition to agents is the transition from producing an answer to managing that workflow.

The LLM supplies reasoning and language capabilities. The surrounding system lets it decompose objectives, select tools, call APIs, browse, execute code, inspect results, and revise plans.

I would not call tool calling alone the important architectural change. The important part is closing the loop.

A Coding Agent Makes the Difference Concrete

Consider “fix the payment issue in this project.”

A plausible workflow is:

Analyze codebase
→ Locate bug
→ Read logs
→ Modify code
→ Run tests
→ Discover another issue
→ Modify code again
→ Run tests
→ Commit changes

The second modification is the interesting part. The system has to incorporate evidence produced by its own actions rather than keep executing its original plan.

More generally:

Goal
 ↓
Reasoning
 ↓
Planning
 ↓
Tool selection
 ↓
Action
 ↓
Observation
 ↓
Evaluation
 ↓
Re-planning
 ↓
Next action

This gives AI work a time dimension. Instead of an interaction lasting seconds or minutes, an agent workflow might run for 30 minutes, several hours, or several days. Weeks or longer remain a possible future horizon, not something to assume from a good demo.

That suggests an evaluation lens:

Task Completion Rate × Task Horizon

I treat this as a useful heuristic, not a standardized metric. How often does the system finish successfully, and how long can it keep working effectively?

Reasoning and coding benchmarks still matter. They just do not describe the whole system.

A Plausible Plan Is Not an Environment Model

Now move the same architecture into robotics.

Ask a robot to put a cup from a table into a cabinet. A language model can produce a sensible outline:

1. Locate the cup
2. Move toward the cup
3. Grasp the cup
4. Locate the cabinet
5. Move toward the cabinet
6. Place the cup inside

The outline leaves out most of what determines success.

The robot needs information about the cup’s 3D position, orientation, and weight. It must account for its trajectory, friction, collision constraints, table geometry, cabinet location, and grasping point.

“Grasp the cup” is a task description. It is not a prediction of what will happen during the grasp.

That is the gap a world model is meant to address:

Current state + Action
          ↓
      World model
          ↓
    Predicted future state

For one proposed grasp, the prediction might be that the cup falls. Another approach—say, approaching from the right—might have a higher predicted probability of success. Those are illustrative outcomes, not guarantees about a particular robot or model.

The distinction is between acting to discover the result and comparing predicted results before acting.

Prediction Adds a Loop Before Execution

Without a world model:

Think → Act → Observe → Correct

With one:

Think
→ Imagine candidate actions
→ Simulate outcomes
→ Compare possible futures
→ Choose
→ Act
→ Observe
→ Update

The observation step does not disappear. A prediction is still something the system must check against the environment.

This capability is particularly relevant to robotics, autonomous vehicles, games, video generation, physical AI, simulation, and agents operating in real-world environments.

Divide Responsibilities Before Combining Models

My working division is simple:

• Agent: What should I try to accomplish, and which action should I choose?
• World model: What is likely to happen if I take that action?
• Memory: What observations and experience should carry forward?
• Tools, APIs, and robots: How does the system act?

A rough comparison makes the intended roles clearer. These are architectural tendencies, not measured rankings that apply to every implementation.

Capability	LLM agent’s role	World model’s role
Goal understanding	Primary	Supporting
Language	Central	Not central
Reasoning	Primary	Partial or supporting
Task planning	Organizes the plan	Supplies predictions
Tool use	Selects and invokes tools	Not central
Environment understanding	Often limited	Central
World dynamics	Often limited	Central
Future prediction	Limited	Core capability
Physical simulation	Limited	Core capability
Decision making	Chooses actions	Informs choices
Long-term learning	Still needs improvement	Potential foundation

A combined architecture could look like this:

Goal
 ↓
Agent ←─────────────────────────────┐
 ↓                                  │
Planning ← World-model predictions  │
 ↓                  ↑               │
Candidate actions ──┘               │
 ↓                                  │
Selected action                     │
 ↓                                  │
Environment                         │
 ↓                                  │
Observation → Memory ───────────────┘

The LLM contributes language, knowledge, abstraction, and reasoning. The agent runtime manages goal-directed execution. The world model provides predictions about environmental changes.

None of those responsibilities requires treating the entire system as one monolithic model.

Experience Can Feed Prediction

The relationship can also run in the other direction.

An acting agent produces trajectories:

State₁ → Action₁ → State₂ → Action₂ → State₃

The useful unit is:

State → Action → Next state

Collections of these transitions can help a world model learn how actions change an environment. Its predictions can then help the agent choose actions more likely to achieve the goal.

Agent
→ Action
→ Environment
→ Experience
→ World model
→ Prediction
→ Better planning
→ Agent

This is a possible feedback loop, not an automatic improvement mechanism. The source of value is the connection between observed transitions and future planning.

Over time, agents and world models may become tightly integrated components rather than separate products.

Why Video Belongs in This Discussion

Video generation is interesting here because a sequence contains more than appearance. It contains change.

A stronger predictive model needs to represent relationships among:

Objects → Movement → Interaction → Physics → Future state

Take a ball rolling off a table. A visually convincing clip is one requirement. Predicting the event involves additional questions:

• Why is the ball moving?
• How does the table’s slope affect it?
• How does velocity change?
• What happens after a collision?
• Where will the ball be next?

That motivates a possible progression:

Video generation → Video prediction → World model

I would keep “possible” attached to that progression. Realistic generation alone is not evidence that a model reliably understands physical dynamics.

Still, it explains why this discussion extends beyond text models into video, 3D, robotics, simulation, physics, and spatial intelligence.

The Infrastructure Problem Is Choosing Capabilities

Once an agent uses several kinds of intelligence, model access stops being a single-provider integration problem.

A research workflow might involve:

User task
→ Planner
→ Reasoning model
→ Search
→ Vision model
→ Coding model
→ Summarization model
→ Final answer

Across providers, developers may have to handle different APIs, SDKs, authentication, pricing, context limits, model IDs, rate limits, response formats, and billing systems.

A unified multi-model API such as CometAPI is relevant at that integration boundary. The broader architectural opportunity is a gateway that helps select capabilities, rather than merely exposing a longer model list.

That distinction matters: access to models is not the same thing as deciding which model should handle the next step.

Routing Is a Policy Decision

A router could accept a request like this:

{
  "task": "research",
  "budget": 1.5,
  "latency": "fast",
  "quality": "high"
}

This is a conceptual routing request, not a documented API contract. The example does not define the budget’s units.

The router would need to consider:

• Task type and required capabilities
• Cost
• Latency
• Quality and success rate
• Context requirements
• Tool-use capability
• Reliability
• Current availability

The resulting architecture is straightforward:

Agent
 ↓
Model router
 ├── GPT
 ├── Claude
 ├── Gemini
 ├── Qwen
 ├── DeepSeek
 └── Grok

The difficult part is the policy behind those branches.

I separate that into three responsibilities.

Model Intelligence: What Fits This Task?

A capability map might begin with candidate families:

Coding    → Claude / GPT / Qwen
Reasoning → GPT / Gemini / DeepSeek
Image     → Model A
Video     → Model B
Voice     → Model C
Simulation→ Model D

These are illustrative assignments, not comparative benchmark results. A useful capability layer needs evidence about the actual task, not just a broad label such as “coding.”

Routing: What Should Run Now?

The best fit in principle may not be the best call under the current cost, latency, context, or availability constraints.

That is why routing should be distinct from a static model ranking.

Evaluation: Did the Choice Work?

An evaluation layer can connect model-level results to workflow outcomes:

Model
→ Benchmark
→ Real-world task
→ Agent evaluation
→ Latency
→ Cost
→ Reliability
→ Recommendation

For an agent, the important result is not just whether an individual response looks good. It is whether that response helped the workflow succeed.

That moves a gateway toward a model decision system rather than a directory.

Model Catalogs May Need a Different Taxonomy

Today, a catalog often groups endpoints by output modality:

Models
├── Chat
├── Image
├── Video
└── Audio

If predictive environment models become available through APIs, a future catalog could distinguish capabilities more explicitly:

Models
├── Language
│   ├── Reasoning
│   ├── Coding
│   └── Agent
├── Perception
│   ├── Vision
│   ├── Audio
│   └── Multimodal
├── World Model
│   ├── Video World Model
│   ├── 3D World Model
│   ├── Physics Model
│   └── Robotics Model
└── Generation
    ├── Image
    ├── Video
    ├── Audio
    └── 3D

This is a prospective taxonomy, not a claim that these endpoints are already available.

The corresponding runtime might connect reasoning, perception, and prediction through a shared access layer:

Application
 ↓
Agent runtime
 ↓
Model gateway
 ├── Reasoning: GPT / Claude / Gemini / Qwen
 ├── Perception: Vision / Audio / Multimodal
 └── World model: Video / 3D / Robotics
 ↓
Action
 ↓
Environment
 ↓
Observation
 ↓
Memory
 ↓
Agent runtime

The infrastructure provider would not need to train every underlying model. Its job would be reliable access and coordination.

A possible progression is:

Phase 1: Unified API
         Multiple models through one interface
 ↓
Phase 2: Model gateway
         Models, billing, authentication, monitoring, routing
 ↓
Phase 3: Agent infrastructure
         LLMs, vision, coding, search, tools, memory, evaluation
 ↓
Phase 4: Broader intelligence infrastructure
         Agent models, world models, video, robotics, simulation

I read those as possible scopes of responsibility, not a guaranteed product roadmap.

Documentation Becomes Part of Model Selection

There is also a discovery problem. Developers search for provider APIs, coding models, reasoning models, agent models, open-source models, video models, and world models.

Comparisons such as GPT versus Claude, Claude versus Gemini, and DeepSeek versus Qwen are part of the same selection process.

A useful model page should therefore go beyond pricing and an endpoint:

Overview
→ Capabilities
→ Benchmarks
→ Agent performance
→ World-model capability
→ Latency
→ Pricing
→ Use cases
→ Alternatives
→ Comparisons
→ API documentation

Not every category will apply to every model. The point is to make suitability discoverable.

Model releases also create a recurring information lifecycle:

Pre-launch → Launch → Benchmark → Comparison → Adoption

That can support search-engine optimization and generative-engine optimization, but I care more about the developer outcome: a connected body of evidence that makes model selection less speculative.

Treat the Timeline as a Hypothesis

One proposed progression looks like this:

Period	Possible emphasis
2024–2025	LLMs and reasoning
2025–2026	Agents, tool use, and computer use
2026–2027	Agents with memory, multimodality, and world models
2027–2028+	World models, planning, long-term memory, and real-world action

These are overlapping directions, not clean industry transitions or established forecasts.

The more useful prediction is a change in evaluation emphasis: from “which model has the highest score?” toward “which system can reliably complete real tasks over longer horizons?”

That is also why I am cautious about defining AGI as one sufficiently large LLM. A candidate architecture may instead be a complete loop:

Goal
→ Intelligence
→ World model
→ Planning
→ Action
→ World
→ Observation
→ Memory
→ Learning
→ Intelligence

Connecting these components does not establish general intelligence. It does provide a concrete architecture to evaluate.

For developers, the distinction is practical: an agent manages action, a world model supplies predictions, and memory carries experience forward. The system becomes useful when those pieces cooperate well enough to turn a goal into a verified result—not merely a convincing plan.