Picture a talented friend who can do something most people cannot: hold a genuine conversation about a painting while simultaneously sketching it from memory, then explain their artistic choices in writing, then generate a variation in a different style — all in one unbroken flow of thought. No pausing to switch hats. No handing the problem to a different colleague. Just one mind, moving fluidly between seeing, understanding, reasoning, and creating.
For years, artificial intelligence couldn't do this. The systems that were brilliant at understanding images were separate creatures from the systems that could generate them. And the systems that could generate text were fundamentally strangers to the ones that could generate pictures. We built specialists and called them state-of-the-art. A new paper from Inclusion AI suggests we may finally be moving past that era.
The Specialist Trap
To understand why building a unified AI brain has been so hard, consider how the field arrived at where it is today.
When researchers wanted to build systems that could understand images — answering questions like "Is the dog happy?" or "What's in the background?" — they built what are called vision-language models. These work a bit like a translator with two desks: one desk for images, one for text, with a bridge between them. The model looks at an image, converts it into a kind of abstract summary, then reasons about that summary using language skills. It became excellent at this. Ask it what's on a table in a photo and it will describe every item with unnerving precision.
But when researchers wanted to build systems that could generate images — creating a picture from a text description — they took an entirely different road. They built diffusion models, which work through a process analogous to developing a photograph in a darkroom. Imagine a blank sheet of photographic paper coated in a fog of random chemical noise. The developer's job is to gradually coax a clear image out of that noise by applying the right chemistry in the right sequence. Generation-focused AI works the same way: it starts with pure randomness and, step by step, refines it into something coherent. These models became extraordinarily good at producing images, but they weren't built for conversation.
The result was a landscape of powerful specialists who couldn't collaborate. Your image-understanding model couldn't create anything new. Your image-generating model couldn't reason about what it made. Asking one system to understand a photograph and then produce a variation of it was like asking a translator and a painter to work together when they speak different languages and have never met.
A Common Alphabet
The fundamental problem was that text and images were written in incompatible scripts. Text arrives as words — discrete, enumerable, easy to shuffle around and reason about. Images arrive as a continuous wash of pixel values: 16 million possible colors per pixel, no obvious boundaries, no clean units. Trying to process both in the same system was like trying to play chess and checkers on the same board with the same pieces.
The solution the LLaDA2.0-Uni researchers found starts with a step that sounds simple but is actually the keystone of everything else: they translated images into the same kind of discrete alphabet that text already uses.
Think of it this way. If you wanted to describe a piece of music to someone who only reads sheet music, you wouldn't play them the recording — you'd transcribe it into notes and rests on a staff. The transcription loses some nuance (the exact timbre of the violin, the subtle swell of dynamics), but it captures the essential structure in a form the reader can work with. The researchers built something similar for images, using a component called SigLIP-VQ, which stands for a particular kind of image encoder paired with vector quantization.
Vector quantization is the sheet music step. Imagine you have a vast library of small visual "stamps" — maybe 16,384 different ones, each representing a distinct visual pattern: a soft edge, a bright diagonal, a particular texture. When you feed an image into the tokenizer, it breaks the image into small patches (like cutting a photograph into a grid of tiny tiles) and asks, for each patch: which stamp in our library is closest to this? The answer — "stamp number 7,341" — is a discrete token. Do this for every patch and you've converted a continuous photograph into a sequence of numbers, just like text.
Now text and images speak the same language. A sentence like "a red barn at sunset" and a photograph of a red barn at sunset can both be represented as sequences of tokens. The same reasoning machinery can process either.
The Crossword Puzzle Model
Here is where the paper's central gamble becomes interesting, because the reasoning machinery they chose is not the dominant approach in the field.
Most large language models today generate text the way a novelist types: one word at a time, left to right, never going back. The model commits to each word before seeing what comes next, which works remarkably well but has limitations — especially for tasks where you might want to revise your global plan as you go, or fill in a document non-sequentially.
The LLaDA2.0-Uni system instead uses what researchers call a discrete diffusion model. The analogy here is a crossword puzzle.
Imagine you're handed a crossword grid where every square has been filled in with random letters — pure noise. Your job is to fix it, guided by the clues. You don't start at 1-Across and work linearly. Instead, you scan the whole grid for places where you're most confident ("7-Down, three letters, 'feline'? That's CAT, obvious"), fill those in, then let those anchors guide the harder squares. You're refining the whole grid simultaneously, converging toward correctness from many directions at once. When you're mostly done, you revisit the remaining uncertain squares with fresh eyes, because now the surrounding letters constrain them.
Discrete diffusion works the same way. The model starts with a sequence of masked tokens — imagine every word in a sentence replaced by a [?] — and iteratively fills them in, guided by the content it's already committed to. It can fill any position at any time, not just left to right. This means it can develop a global sense of what a response should look like before committing to individual words. For images, this is especially powerful: it can simultaneously work on the sky of an image and the ground, letting each inform the other.
The Council of Specialists
Running a model that processes both high-resolution images and complex language simultaneously is computationally expensive — the kind of expensive that makes the electricity bill of a small city seem modest. The researchers addressed this with an architectural choice called Mixture of Experts, or MoE.
Think of a large hospital emergency department. When a patient arrives, a triage nurse assesses the situation and routes them: chest pain goes to cardiology, a broken bone to orthopedics, a rash to dermatology. Not every doctor sees every patient. Most doctors sit idle for any given case while the relevant specialist handles it.
The MoE backbone works the same way. Inside the model, there are many specialized sub-networks — the "experts." When the model processes a given input, a routing mechanism decides which subset of experts should activate. An image-heavy input might activate different experts than a text-heavy one. The result is a model with the capacity of a very large system but the computational cost of a much smaller one, because only a fraction of the total machinery runs at any moment.
This is not a new idea in AI research, but combining it with a diffusion-style architecture for both text and image tokens simultaneously is precisely the kind of integration that makes this work notable.
Reconstructing the Canvas
Even after all this machinery processes an image as tokens, you eventually need to convert those tokens back into actual pixels that a human can see. The gap between "a sequence of stamp numbers" and "a beautiful, coherent image" is where many unified systems stumble, producing outputs that look smeared or incoherent.
The researchers added a dedicated diffusion decoder for this final step — essentially a specialized refinement engine that takes the abstract token sequence and reconstructs it into a high-fidelity image. Think of it as the difference between reading sheet music notation and actually hearing an orchestra perform it. The notation captures the structure; the performance fills in all the richness that makes it real.
To make this fast enough to be useful, they used a technique called few-step distillation. Normally, the diffusion process requires dozens or hundreds of refinement steps — like developing a photograph through a long sequence of chemical baths. Distillation compresses this wisdom: a "teacher" model that takes a hundred careful steps trains a "student" model to achieve comparable results in just a few. The student learns not the teacher's process but the teacher's outcomes, skipping the intermediate labor.
LLaDA2.0-Uni LLaDA-O Lumina-DiMOO Figure 1: Benchmark Perfo Authors are listed in alphabetical order based on last nam 1
The Integrated Mind
What all of this amounts to is a system that, for the first time in this configuration, can genuinely interleave text and images in its reasoning without handing off between different specialized models.
Consider what this means concretely. Imagine asking the system: "Here's a painting. What mood does it evoke, and can you generate a photograph that captures the same feeling in a real-world setting?" A siloed system would have to pass the image to an understanding model, extract a description, pass that description to a generation model, and hope the handoff preserved what mattered. LLaDA2.0-Uni processes the original image and generates the new photograph within the same computational stream. The understanding and the creation are happening in the same mind, informed by each other.
The paper calls this "interleaved generation and reasoning," and it's the feature that most distinguishes this architecture from its predecessors. The model can generate a paragraph of text, then generate an image that continues the narrative, then reason about both together — without the artificial seams that separate-model pipelines inevitably produce.
What Changes in the Real World
The most interesting applications of a system like this are not in the lab but in the workflows where the gap between understanding and generation currently costs time, fidelity, and money.
Consider medical imaging. A radiologist today looks at a scan, forms a judgment, and dictates a report — two separate steps, often using separate tools. A system that can simultaneously examine a CT scan and draft a structured report, then modify the report and highlight the corresponding region of the scan when a colleague asks a follow-up question, collapses multiple handoffs into a single workflow. The bottleneck shrinks.
Consider education. A teacher designing a history lesson might want to explain the significance of a photograph from 1945, generate a map showing the troop positions it references, create a timeline that incorporates both, and then produce a quiz that uses all three. Today, each of those steps requires a different tool and a manual bridge between them. A unified reasoning-and-generation system makes the bridges automatic.
Or consider design iteration, where a product designer needs to produce a concept, explain its rationale to a client, modify it based on feedback, and document the changes — all in a single collaborative session. The ability to reason about what's on the canvas and alter it within the same cognitive loop changes the pace of that process entirely.
What Remains Unanswered
It would be wrong to leave this without noting what the paper doesn't address, because the gap between benchmark performance and real-world deployment is always wider than a research paper can acknowledge.
The authors report that their model "matches specialized VLMs in multimodal understanding while delivering strong performance in image generation." That hedge — "matches" rather than "surpasses," "strong" rather than "best" — is doing meaningful work. The specialized models that focus only on understanding images remain better at it. The specialized models that focus only on generating images remain better at that. What LLaDA2.0-Uni offers is not supremacy in any single domain but competence across all of them simultaneously. Whether that trade-off is worth making depends entirely on the use case.
I'm also skeptical, from the abstract alone, about how the discrete tokenization of images holds up at the extremes. The sheet music analogy works well for capturing structure, but it loses expressiveness. A violin's exact timbre doesn't survive transcription. Similarly, the process of converting an image into a vocabulary of 16,384 stamps and then reconstructing it will introduce artifacts and losses, particularly for images with complex textures or fine detail. The paper claims "high-fidelity" reconstruction, but what "high fidelity" means at scale, across diverse real-world imagery, is a question that requires more than a benchmark table to answer.
Finally, the computational reality is sobering. A Mixture of Experts architecture is cheaper to run than a naive model of the same theoretical capacity, but "cheaper" is relative. Running a system like this in a consumer product, at scale, remains a significant engineering challenge. The gap between "this works in a research paper" and "this works on your phone" is still large.
None of this diminishes the intellectual accomplishment. Building a system that can read an image as a sequence of meaningful symbols, reason about those symbols and text symbols simultaneously using a diffusion process, and then reconstruct coherent images from the output — all within one integrated architecture — represents a genuine step toward the flexible, general-purpose AI systems the field has been working toward for years. The question is never whether a new approach is perfect. The question is whether it moves the frontier in a direction worth moving. This one does.
📄 https://arxiv.org/abs/2604.20796
tags: multimodal, diffusion, imagegeneration, unifiedai
🇰🇷 Korean version on Velog: https://velog.io/@tkdnel1002/f4k7gl8o
Top comments (0)