DEV Community

Cover image for Genie 4 and world models: why a playable world is harder than a pretty video
AIHubMix
AIHubMix

Posted on

Genie 4 and world models: why a playable world is harder than a pretty video

#ai #llm #worldmodels #gamedev

On October 27, 2026, Google DeepMind announced Genie 4, its newest world model. A video generator hands you a finished clip. Genie 4 generates a world you step into: it responds to your keyboard every frame, you can rewrite the scene mid-simulation with plain English, and the world now carries its own live sound — wind, footsteps, music, spoken dialogue — instead of a soundtrack laid over video afterwards.

As of launch it is a research preview for trusted testers, "coming soon" to Google AI Ultra subscribers through Project Genie — the consumer world-exploration product, currently running Genie 3 — and to early access partners of the Gemini API and Google AI Studio. The Project Genie waitlist is open; no dates and no pricing have been published. Nobody can book time inside Genie 4 yet, so the interesting part right now is the field it belongs to, and the engineering problems it exists to solve.

What a world model is, actually

Strip the marketing off and a world model is a neural network that has learned how some world behaves well enough to keep simulating it: given the current state of the world and an action, it predicts what the world looks — and now sounds — like next. Feed a system like that enough video, and what emerges is something like a game engine you never wrote: no physics code, no hand-built assets, no level design. The rules, the scenery, the clouds over the hills all come out of the weights.

The idea has a founding artifact. In 2018, David Ha and Jürgen Schmidhuber's World Models paper showed that a small agent could be trained inside a generative model of its own environment — trained, in the authors' phrase, "entirely inside of its own hallucinated dream generated by its world model" — and then transferred back into the real game. That remains the cleanest one-line definition: a world model is a place to practice.

Companies have since drawn two different conclusions from the same idea:

  • Predict in pixels. OpenAI argued in February 2024, in "Video generation models as world simulators", that scaling video generation is "a promising path towards building general purpose simulators of the physical world". Its evidence: Sora rendering Minecraft gameplay while simultaneously controlling the player with a basic policy. Make the video good enough, the argument goes, and world understanding comes along with it.
  • Predict in abstractions. Yann LeCun's 2022 position paper A Path Towards Autonomous Machine Intelligence takes the opposite view: a world model should predict future states in an abstract representation space, not in pixels — partly because pixel-level prediction burns most of a model's capacity on detail nobody can verify, and partly because an agent plans by comparing imagined outcomes, not imagined frames.

Both camps agree on the goal — a simulator deep enough to train agents in, without paying real-world prices for every mistake. They disagree about which representation gets there first. DeepMind's Genie work sits on the pixels side, with a twist: pixels first, but pixels you can act through, and now with a language model standing behind them.

Why a playable world is harder than a pretty video

The distinction that trips people up: a world model is a different order of difficulty from a video generator, not a bigger video generator. The differences are structural:

Contrast Video generator World model
What you get a finished clip an ongoing environment
Who acts in it nobody — it plays without you you, every frame
Each frame comes from a prompt, denoised once your last action plus the world's current state
State nothing to keep the world remembers what it has generated
Sound edited onto a finished clip emerges live with the world
When it ends the clip ends when you stop — or when drift sets in

A note on that table: it describes the ideal contrast between the two model types. Individual products straddle it — Runway's GWM Worlds 2 and Decart's Oasis are video-style models built to be playable.

But a playable world model commits its maker to problems a clip generator gets to dodge:

  1. Speed. Film-quality video models can spend minutes on a shot. A world model gets a fraction of a second per frame, because interactive experiences run at 24 to 60 frames per second, and any slower response breaks the illusion. It is an architecture problem, not just an output-quality one.
  2. Memory. Everything the model has shown you, it has to keep. Walk away from a landmark and come back a minute later: if the landmark renders differently, the world dies instantly. This is why consistency — how much coherent, remembered experience a world can sustain — is the number people in the field actually monitor.
  3. Causal physics. Lifelike footage is easy to fake; cause and effect are not. A 2025 benchmark called Physics-IQ paused real recordings of everyday events and asked video models — Sora, Runway, Pika, Lumiere, Stable Video Diffusion, VideoPoet — to continue them, then scored the continuations against what actually happened next, across fluids, optics, solid mechanics, magnetism, and thermodynamics. Its verdict: physical understanding in those models was "severely limited, and unrelated to visual realism". A world model has to thread that needle, or nothing trained inside it can be trusted to transfer out.

The Genie line: from a toy to two minutes (reported)

Generation Released You give it What it proved
Genie February 2024 a single image playable 2D world, actions inferred from video
Genie 2 December 2024 a single image playable 3D world with object permanence, up to about a minute
Genie 3 August 2025 a text prompt first real-time generation: 720p at 24 fps, several minutes
Genie 4 October 27, 2026 text and events, live native audio; a Gemini "director" in the loop; ~2 minutes (reported)

Genie 4's resolution and frame rate are unpublished. Genie 2's one-minute figure and Genie 3's numbers come from DeepMind's announcements; the two-minute figure above is TechCrunch's readout of DeepMind's launch briefing (reported here).

Two Genie-line details carry the concept, not just trivia:

The first Genie invented its own controls. The 2024 Genie paper described an 11-billion-parameter foundation world model trained on raw internet video with no action labels — and found that when a person presses "left" or "jumps" in the training footage, the network can extract a latent action space from the changes between frames. Playability appeared without anyone labeling an action, which is why "controller" in a world model does not mean buttons a designer chose. It means whatever actions the model can infer — and what DeepMind's generations have expanded ever since, from platformer moves to text-steered world events.

Consistency is the increment that tells the story. Genie 2 could hold a coherent world for roughly a minute, per DeepMind's December 2024 announcement. Genie 3 stretched usable interaction to several minutes with memory about a minute back. Genie 4 doubles the working horizon to roughly two minutes, per the TechCrunch report above. Read the sequence as the shape of the problem: the hard end of interactive worlds is not rendering. It is keeping hundreds of consecutive remembered states aligned at interactive speed.

What's new in Genie 4

DeepMind's announcement centers on four capabilities. Read them together, not as "Genie 3 plus extras":

1. You steer with language, live. Natural-language edits — change the weather, move the camera, introduce a new person or object — apply to the simulation in real time, and Gemini adapts your wording to the scene that actually surrounds it. Between your inputs, Gemini keeps watching and steering so the world progresses cohesively. The craft is not rendering a scene; it is responding to changes inside one.

2. Physics and people. Genie 4 simulates real-world physics scenarios — aerodynamics and collisions are DeepMind's examples — and its "most dramatic leap", per DeepMind, is people: characters that move naturally, respond organically to their environments, and can be interactively steered in real time.

3. Native audio. Genie 4 is, by DeepMind's own description, "our first real-time world model to generate audio natively". Every setting carries its own ambient sound, from wind in trees to surf on a beach — and the frontier piece is speech: characters converse and react to your inputs and to events in the world, in real time, rather than receiving a dubbed track afterwards.

4. A reasoning layer inside the loop. The fourth is where Genie 4 breaks with the Genie 1–3 recipe. Under the hood, DeepMind says Gemini Omni — the Gemini family's multimodal model that turns images, audio, and text into video — anchors the simulation in real-world appearance and behavior, while other Gemini models supply the reasoning. Concretely, Gemini 3.8 Flash runs continuously as a "guiding agent in the loop": as Genie 4 renders, it observes the scene, breaks the goal down into steps, reviews incoming frames, verifies each step, and replans when the world shifts or the user intervenes.

That reasoning layer also has a named partner: an experiment with Gemini 4 Argon and Gemini Robotics, where Argon decomposes a user-instructed task into steps that are interleaved as text captions into the simulation as it renders — so the simulated task keeps getting steered toward completion while it runs. That is the door to "practice the task in a dreamed world before spending the real one", which is robotics' whole pitch for world models: infinitely varied, safely dangerous, re-runnable training environments, at effectively no marginal cost per retry.

What still doesn't work

DeepMind names the remaining gaps itself, and they read as the field's to-do list:

  • Coherence holds over short sessions only. Visual details and object permanence still shift and drift — especially as a simulation runs longer, after multiple complex edits, or as you navigate through the scene.
  • Complex physical interactions, real-time audio accuracy, rendering legible text, spatial accuracy, and long-term consistency are all active research areas.

One longer way to say it: nobody has cracked forever-consistent generated worlds yet. World Labs' Marble — reconstruct-and-persist 3D worlds, exportable as Gaussian splats or meshes — is a rival architecture approaching the same ambition from the other side. And DeepMind's stated next step is hybrid world models: let a language-model stack carry the world's structured, logical layer while Genie renders what it looks, sounds, and responds like, instead of asking one giant network to do both at once. As Tejas Kulkarni, a research director at Google, put it during the launch briefing per TechCrunch: "What we'll increasingly see in the future are these hybrid world models, where you have a combination of Gemini writing code and text, and Genie ingesting that in real time and simulating reality at different levels of abstraction."

Two safety-relevant facts before this reaches general consumers: all Genie 4 video and audio output carries SynthID watermarking, and DeepMind says it ran safety evaluations before opening the preview, deliberately to trusted testers first.

Who else is building this — and the money

Genie 4 landed into a field already in motion, keyed to the two schools above:

  • Decart's Oasis (October 2024, built with chip startup Etched) was the field's proof that small can play: a 500-million-parameter model open-sourced for local runs, with a hosted demo running a larger checkpoint, improvising a Minecraft-shaped world at 360p and 20 fps on an NVIDIA H100 — no game engine anywhere in the loop.
  • World Labs' Marble is spatial intelligence — the term Fei-Fei Li's company uses for its own bet on world models: persistent, navigable 3D worlds from text, images, or video, viewable without time limits and exportable as Gaussian splats, meshes, or video.
  • NVIDIA Cosmos is the industrial branch: open world-foundation-model checkpoints for generating synthetic training data and simulating futures for robots and autonomous vehicles — the same "dream before reality" logic, applied to machines with actuators as well as cameras. NVIDIA's "physical AI" stack (Cosmos, plus the Isaac GR00T robot foundation models) is pitched exactly along the lines of DeepMind's robotics experiment.
  • Runway's GWM Worlds 2 shipped a couple of months before Genie 4: playable generated worlds with audio, from a company built on video generation — the pixel school's proof that the video-model/world-model boundary is blurring from the other direction.

And around that map, the money: Amazon–Decart (The Wall Street Journal reports Amazon is in talks to acquire Decart for roughly $7 billion), AMD–World Labs (AMD announced on September 28, 2026 that it is acquiring World Labs for $8.2 billion), and Rivian's buyout of the world-model startup Third Dimension — announced October 22, 2026 — to bring simulation in-house for self-driving testing. Game studios, robot builders, and carmakers are all shopping for the same thing — worlds as training environments. That is why Google keeps chasing this, and why the reported two-minute horizon is a research goal rather than a marketing line: a simulator is only useful as a classroom if it holds together long enough to learn in.

The bet

DeepMind frames the whole line of work as "a crucial aspect of building AGI". The claim behind it: an intelligence that will act in the world has to learn from causes, not labels, and a controllable, infinitely variable, safely dangerous world is the only classroom that scales. That is the strong version of the bet — and it is a bet. Whether pixel-level simulation becomes the substrate for embodied intelligence, or one substrate among several with the LeCun camp's abstract latents carrying part of the load, nobody can prove it yet.

What can be said about Genie 4 specifically: the officially defensible first is native audio — by DeepMind's description, the first real-time world model that generates audio natively. The bigger claims are ours, and they should carry their weight as arguments rather than sound like settled facts: this reads to us as the first interactive world at this fidelity that you can speak to and that speaks back, and as the clearest public case so far of a reasoning model sharing the renderer's loop — the closest the field has come to making "world model" mean both halves of the phrase: the pixels, and the mind moving through them.

Sources

Top comments (0)