DEV Community: Davide Mibelli

RAG in Production: What the Tutorials Don't Tell You

Davide Mibelli — Thu, 14 May 2026 13:00:00 +0000

I built a RAG system that scored 91% on our internal eval suite. It retrieved the right chunks four out of five times in every benchmark we ran. We shipped it. Users thought it was broken.

The gap between "works in evaluation" and "works in production" is the thing every RAG tutorial skips. This article is what I learned closing that gap across three different production deployments — a customer support bot, an internal knowledge base, and a document Q&A tool for a legal team.

Why your evals lie to you

The typical RAG eval flow: take 50 question-answer pairs, run retrieval, score chunk relevance, measure answer quality. The benchmark looks good. Production does not.

The problem is evaluation datasets are clean. Real user questions are not. Users ask ambiguous things, reference context from earlier in the conversation, use company-specific jargon that is not in your embeddings vocabulary, and ask questions that span multiple documents. Your 50-pair eval dataset does not cover any of this.

The more subtle problem: retrieval correctness is not the same as answer usefulness. A chunk can be semantically relevant to the query but contain outdated information, contradict another retrieved chunk, or be missing the specific number the user actually needs. Cosine similarity does not catch any of this.

Before you optimize retrieval metrics, instrument what users actually do. In my customer support deployment, the clearest signal was not retrieval recall — it was how often users rephrased their question immediately after getting an answer. That rephrasing rate was the real quality metric.

Chunking is where it actually breaks

The default chunking strategy in most tutorials: split every 512 tokens with 50-token overlap. This is almost always wrong.

The problem is that 512 tokens is an arbitrary number based on older embedding model limits, not on the structure of your documents. A 512-token chunk cut out of the middle of a legal clause or a technical procedure is often meaningless without the surrounding context.

What actually works depends on your document type:

For structured documents (FAQs, product docs, knowledge base articles): chunk by logical unit — one question-answer pair, one procedure step, one concept section. Use your document's own structure as the chunking boundary. If your docs use consistent heading patterns, split on those.

For long-form prose (contracts, reports, research papers): hierarchical chunking. Keep a parent chunk of 1000–2000 tokens for context retrieval, and child chunks of 150–300 tokens for precise matching. At retrieval time, return the child chunk for relevance scoring but pass the parent chunk to the LLM as context.

For code documentation or READMEs: file-level or function-level chunks, never mid-function splits.

The overlap parameter also matters more than people expect. Overlap exists to avoid losing information at chunk boundaries, but it inflates your vector store size and retrieves duplicate context. I found that semantic chunking — splitting at sentence boundaries that mark topic shifts — eliminated the need for overlap almost entirely.

Retrieval is rarely the problem you think it is

When RAG produces wrong answers, the instinct is to improve retrieval. Better embeddings, more chunks, higher top-k. This is usually the wrong lever.

In my experience, retrieval is failing only about 30% of the time when users report bad answers. The other 70% is one of:

The right chunk was retrieved but the LLM ignored it — this is a prompt engineering problem, not a retrieval problem
The answer requires synthesizing across multiple chunks — retrieval returned individually correct chunks but the LLM could not connect them
The question is genuinely unanswerable from the knowledge base — the document does not exist or is outdated

To separate these, add logging at both retrieval and generation time. Log the top-k chunks and the final answer separately. A human spot-check of 20 failure cases per week will show you very quickly which category you are actually in.

When retrieval genuinely is the problem, the highest-leverage fix is hybrid search: dense retrieval (embeddings + cosine similarity) combined with sparse retrieval (BM25 keyword matching). Dense retrieval handles semantic similarity. BM25 handles exact matches — product names, error codes, version numbers, any term where "sounds like" is the wrong answer.

from langchain.retrievers import EnsembleRetriever
from langchain_community.retrievers import BM25Retriever
from langchain_chroma import Chroma

dense_retriever = Chroma(...).as_retriever(search_kwargs={"k": 10})
sparse_retriever = BM25Retriever.from_documents(docs, k=10)

ensemble = EnsembleRetriever(
    retrievers=[dense_retriever, sparse_retriever],
    weights=[0.6, 0.4]  # tune based on your query distribution
)

The weight split between dense and sparse depends on your documents. Technical documentation with lots of product names and version numbers benefits from higher BM25 weight. Conversational knowledge bases lean toward dense.

Reranking changes the answer quality more than anything else

If there is one thing to add to a RAG pipeline that makes the biggest difference in production answer quality, it is a cross-encoder reranker between retrieval and generation.

The retrieval step uses bi-encoder embeddings — query and document are embedded independently, similarity is a dot product. This is fast but imprecise. A cross-encoder takes the query and a candidate chunk together as a single input and scores their relevance jointly. Much more accurate, but too slow to run over your entire corpus.

The standard pattern: retrieve top-20 with the fast bi-encoder, rerank with the cross-encoder, pass top-3 or top-5 to the LLM.

from sentence_transformers import CrossEncoder

reranker = CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2")

def rerank(query: str, chunks: list[str], top_n: int = 5) -> list[str]:
    pairs = [(query, chunk) for chunk in chunks]
    scores = reranker.predict(pairs)
    ranked = sorted(zip(scores, chunks), reverse=True)
    return [chunk for _, chunk in ranked[:top_n]]

In the legal document system, adding reranking reduced hallucinations in answers by roughly 40% — not because retrieval improved, but because the LLM stopped having to sort through marginally relevant context and could focus on the actually relevant chunks.

The infrastructure issues no tutorial shows

Context window budget. Top-5 chunks at 500 tokens each is 2500 tokens before you add the system prompt and the user message. With GPT-4 this is fine. With smaller models or high-volume APIs where you want to minimize token cost, you need explicit context budgeting. Know your LLM's context window, subtract your fixed prompt overhead, and set your chunk count and size to fit.

Stale chunks. Documents in production change. Your chunk embeddings do not update automatically. You need a pipeline that detects document changes (by checksum or last-modified timestamp) and re-embeds only the changed documents. I've seen teams manually re-index their entire corpus monthly as a workaround. That is not a solution for any corpus over a few thousand documents.

Conflicting information. If the same concept is documented in multiple places with different details, retrieval will return both. The LLM will either pick one arbitrarily or produce a contradictory answer. The fix is upstream — deduplicate your knowledge base and establish a single source of truth. Retrieval cannot save you from bad source data.

Context window management

One problem that grows slowly and then all at once: as your knowledge base expands and you increase top-k to improve recall, your context window fills up. You pass 10 chunks at 500 tokens each, plus system prompt, plus conversation history, and suddenly you are at 7000 tokens per request on a model with an 8000-token limit.

The naive fix is to increase top-k and hope the model attends to the right parts. The correct fix is explicit context budgeting:

MAX_CONTEXT_TOKENS = 4000  # reserve headroom for prompt + answer
TOKENS_PER_CHUNK = 400     # approximate after chunking

max_chunks = MAX_CONTEXT_TOKENS // TOKENS_PER_CHUNK  # = 10

chunks = rerank(query, retrieve(query, k=20), top_n=max_chunks)

Calculate the budget before retrieval, not after. If you are hitting the limit regularly, reduce chunk size rather than reducing top-k — smaller chunks at the same top-k gives the model more coverage with the same token budget.

What to actually monitor

Stop measuring retrieval precision in isolation. In production, instrument:

Rephrasing rate: how often a user asks a follow-up that is essentially the same question reworded. High rate means the answer was not useful, regardless of retrieval metrics.
Answer rejection rate: if your UI has thumbs-down feedback, track it. Correlate failures with retrieved chunk IDs to identify which documents produce bad answers consistently.
Latency by pipeline stage: retrieval, reranking, and generation each have different latency profiles and different optimization paths. Aggregate P95 latency tells you nothing useful about where to look.
Unanswerable rate: how often the LLM says "I don't have information on this." Below 5% and your system is probably hallucinating answers it should refuse. Above 30% and your knowledge base has coverage gaps.
Chunk age: track when each chunk was last re-indexed. Any chunk older than your document update frequency is potentially stale. This one takes ten minutes to add to your ingestion pipeline and saves hours of debugging mysterious wrong answers three months from now.

The RAG tutorial teaches you to build a pipeline. Production teaches you to instrument one. The teams I've seen succeed were instrumenting before they had users, not after.

What is the failure mode you hit first in your own RAG deployment?

Originally published on Medium.

Spring Boot JWT Authentication: The Complete Setup Most Tutorials Get Wrong

Davide Mibelli — Tue, 12 May 2026 05:18:42 +0000

I've read probably forty Spring Boot JWT tutorials over the years. They all show you the same thing: how to generate a token on login, how to validate it on each request, and how to wire up SecurityFilterChain. And they all stop right there.

What they skip is everything that matters in production: refresh token rotation, token revocation, and not sending tokens in JavaScript-readable headers when you don't have to. I've inherited two codebases where a "working JWT implementation" turned out to be a security hole you could drive a truck through. This article is the setup I now use by default.

What the typical tutorial gives you

The standard walkthrough produces something like this:

@Bean
public SecurityFilterChain filterChain(HttpSecurity http) throws Exception {
    http.csrf(AbstractHttpConfigurer::disable)
        .sessionManagement(s -> s.sessionCreationPolicy(SessionCreationPolicy.STATELESS))
        .authorizeHttpRequests(auth -> auth
            .requestMatchers("/auth/**").permitAll()
            .anyRequest().authenticated())
        .addFilterBefore(jwtAuthFilter, UsernamePasswordAuthenticationFilter.class);
    return http.build();
}

This is fine. The filter reads the Authorization: Bearer <token> header, validates the signature, sets the SecurityContext. You can deploy this. The problem is what comes next.

The access token has an expiry — typically 15 to 60 minutes. What happens when it expires? In most tutorials: the user logs in again. In production: users complain, sessions die mid-work, and someone "fixes" it by setting expiry to 7 days. Now you have a long-lived token you can't revoke.

The right structure: short-lived access + rotating refresh

The pattern I use in every Spring Boot project now:

Access token: 15 minutes, stored in memory (JS variable or React state), never in localStorage
Refresh token: 7 days, stored in an HttpOnly; Secure; SameSite=Strict cookie, not accessible to JavaScript
Rotation: every refresh request issues a new refresh token and invalidates the old one
Revocation store: a small table (or Redis set) of invalidated refresh token IDs

Here's the refresh token entity:

@Entity
@Table(name = "refresh_tokens")
public class RefreshToken {
    @Id
    private String id; // UUID, stored in the cookie value

    @ManyToOne(fetch = FetchType.LAZY)
    private User user;

    private Instant expiresAt;
    private boolean revoked;

    @CreationTimestamp
    private Instant createdAt;
}

And the service that handles rotation:

@Service
@Transactional
public class RefreshTokenService {

    private final RefreshTokenRepository repo;
    private final Duration refreshExpiry = Duration.ofDays(7);

    public RefreshToken create(User user) {
        RefreshToken token = new RefreshToken();
        token.setId(UUID.randomUUID().toString());
        token.setUser(user);
        token.setExpiresAt(Instant.now().plus(refreshExpiry));
        token.setRevoked(false);
        return repo.save(token);
    }

    public RefreshToken rotate(String oldTokenId) {
        RefreshToken old = repo.findById(oldTokenId)
            .orElseThrow(() -> new InvalidTokenException("Refresh token not found"));

        if (old.isRevoked() || old.getExpiresAt().isBefore(Instant.now())) {
            // Possible token reuse attack — revoke entire family
            revokeAllForUser(old.getUser());
            throw new InvalidTokenException("Refresh token expired or revoked");
        }

        old.setRevoked(true);
        repo.save(old);

        return create(old.getUser());
    }

    public void revokeAllForUser(User user) {
        repo.revokeAllByUser(user);
    }
}

The reuse detection is the part most tutorials omit entirely. If an attacker steals a refresh token and uses it, you now have two parties trying to use it. When the legitimate client tries to rotate and finds its token already consumed, you revoke everything and force a new login. This is the refresh token family pattern from the OAuth 2.0 Security Best Current Practice spec.

The cookie setup

The refresh token goes out in the response as a cookie, not in the JSON body:

@PostMapping("/auth/login")
public ResponseEntity<AccessTokenResponse> login(
        @RequestBody LoginRequest req,
        HttpServletResponse response) {

    User user = authService.authenticate(req.email(), req.password());

    String accessToken = jwtService.generateAccessToken(user);
    RefreshToken refreshToken = refreshTokenService.create(user);

    ResponseCookie cookie = ResponseCookie.from("refresh_token", refreshToken.getId())
        .httpOnly(true)
        .secure(true)
        .sameSite("Strict")
        .path("/auth/refresh")
        .maxAge(Duration.ofDays(7))
        .build();

    response.addHeader(HttpHeaders.SET_COOKIE, cookie.toString());

    return ResponseEntity.ok(new AccessTokenResponse(accessToken));
}

Note .path("/auth/refresh") — the cookie is scoped to the refresh endpoint only. The browser won't send it on any other request. This reduces the attack surface on CSRF (though SameSite=Strict already handles most of that).

The refresh endpoint reads the cookie:

@PostMapping("/auth/refresh")
public ResponseEntity<AccessTokenResponse> refresh(
        @CookieValue(name = "refresh_token", required = false) String refreshTokenId,
        HttpServletResponse response) {

    if (refreshTokenId == null) {
        return ResponseEntity.status(HttpStatus.UNAUTHORIZED).build();
    }

    RefreshToken newRefreshToken = refreshTokenService.rotate(refreshTokenId);
    String newAccessToken = jwtService.generateAccessToken(newRefreshToken.getUser());

    ResponseCookie cookie = ResponseCookie.from("refresh_token", newRefreshToken.getId())
        .httpOnly(true)
        .secure(true)
        .sameSite("Strict")
        .path("/auth/refresh")
        .maxAge(Duration.ofDays(7))
        .build();

    response.addHeader(HttpHeaders.SET_COOKIE, cookie.toString());

    return ResponseEntity.ok(new AccessTokenResponse(newAccessToken));
}

The JWT service itself

Nothing exotic here, but I'll include it for completeness. I use io.jsonwebtoken:jjwt-api (JJWT 0.12.x) with Spring Boot 3:

@Service
public class JwtService {

    @Value("${app.jwt.secret}")
    private String secret;

    private static final Duration ACCESS_EXPIRY = Duration.ofMinutes(15);

    private SecretKey key() {
        return Keys.hmacShaKeyFor(Decoders.BASE64.decode(secret));
    }

    public String generateAccessToken(User user) {
        return Jwts.builder()
            .subject(user.getId().toString())
            .claim("email", user.getEmail())
            .claim("roles", user.getRoles())
            .issuedAt(new Date())
            .expiration(Date.from(Instant.now().plus(ACCESS_EXPIRY)))
            .signWith(key())
            .compact();
    }

    public Claims validateAndParse(String token) {
        return Jwts.parser()
            .verifyWith(key())
            .build()
            .parseSignedClaims(token)
            .getPayload();
    }
}

The secret must be at least 256 bits (32 bytes) for HS256. Generate it once and store it in your secrets manager, not in application.yml committed to git:

openssl rand -base64 32

The filter

@Component
@RequiredArgsConstructor
public class JwtAuthFilter extends OncePerRequestFilter {

    private final JwtService jwtService;
    private final UserDetailsService userDetailsService;

    @Override
    protected void doFilterInternal(
            HttpServletRequest request,
            HttpServletResponse response,
            FilterChain chain) throws ServletException, IOException {

        String header = request.getHeader(HttpHeaders.AUTHORIZATION);
        if (header == null || !header.startsWith("Bearer ")) {
            chain.doFilter(request, response);
            return;
        }

        String token = header.substring(7);
        try {
            Claims claims = jwtService.validateAndParse(token);
            String userId = claims.getSubject();

            if (userId != null && SecurityContextHolder.getContext().getAuthentication() == null) {
                UserDetails userDetails = userDetailsService.loadUserByUsername(userId);
                UsernamePasswordAuthenticationToken auth = new UsernamePasswordAuthenticationToken(
                    userDetails, null, userDetails.getAuthorities());
                auth.setDetails(new WebAuthenticationDetailsSource().buildDetails(request));
                SecurityContextHolder.getContext().setAuthentication(auth);
            }
        } catch (JwtException e) {
            // Token invalid or expired — let the request proceed unauthenticated
            // The security config will reject it at the endpoint level
        }

        chain.doFilter(request, response);
    }
}

I catch JwtException broadly here rather than differentiating expired vs. tampered. From the filter's perspective the right behavior is the same: don't authenticate. The client should catch the 401 and attempt a refresh.

Logout

Logout needs to revoke the refresh token and clear the cookie:

@PostMapping("/auth/logout")
public ResponseEntity<Void> logout(
        @CookieValue(name = "refresh_token", required = false) String refreshTokenId,
        HttpServletResponse response) {

    if (refreshTokenId != null) {
        refreshTokenService.revoke(refreshTokenId);
    }

    ResponseCookie cleared = ResponseCookie.from("refresh_token", "")
        .httpOnly(true)
        .secure(true)
        .sameSite("Strict")
        .path("/auth/refresh")
        .maxAge(0)
        .build();

    response.addHeader(HttpHeaders.SET_COOKIE, cleared.toString());

    return ResponseEntity.noContent().build();
}

Setting maxAge(0) tells the browser to delete the cookie immediately.

What I skip in this setup

A few things I deliberately leave out of a standard deployment:

JWT blocklist for access tokens. Once you have 15-minute expiry and a working refresh flow, blocking access tokens before expiry is usually not worth the latency of a blocklist check on every request. If you need immediate revocation (e.g. "terminate all sessions now"), revoke all refresh tokens — the access tokens will expire naturally within 15 minutes.

RS256 asymmetric signing. Useful if multiple services need to verify tokens without sharing the secret. In a monolith or a small microservices setup where the auth service also issues tokens to itself, HMAC-SHA256 is simpler and faster.

Remember me / device management. Out of scope here, but the refresh token table already gives you the foundation: add a deviceName column, show the user their active sessions, let them revoke specific ones.

The only thing preventing most teams from shipping this instead of the tutorial version is the extra thirty minutes it takes to add the refresh token table and rotation logic. It's worth it.

What does your current JWT setup look like — are you doing refresh rotation, or relying on long-lived tokens?

Originally published on Medium.

I Tested Claude Code, OpenCode, and Codex for 30 Days — Here's My Verdict

Davide Mibelli — Wed, 06 May 2026 10:02:19 +0000

In March 2026, I made a rule: every piece of non-trivial code I write has to go through at least one AI coding agent before I commit it. Not for autocomplete — for the whole thing. Architecture decisions, test generation, refactoring, debugging. The full workflow.

I used three tools in rotation: Claude Code, OpenCode, and OpenAI Codex CLI. All on the same projects — a Spring Boot microservice, a Flutter mobile app, and a personal side project in Python. Thirty days, 100+ hours of active coding across all three.

A note on scope: I deliberately chose terminal-first agents, not IDE-integrated tools like Cursor or GitHub Copilot. My workflow already lives in the terminal, and I wanted agents that reason about entire codebases rather than just the file I have open. If you're evaluating Cursor, this comparison won't map directly.

Here's what actually happened.

This is a preview. Read the full article on Medium: I Tested Claude Code, OpenCode, and Codex for 30 Days — Here's My Verdict

Spring Boot 4.0 Migration: What Nobody Tells You About the Breaking Changes

Davide Mibelli — Wed, 29 Apr 2026 13:14:24 +0000

I upgraded two production applications to Spring Boot 4.0 the week it went GA. I read the migration guide, skimmed the release notes, and thought the hardest part would be the Jackson 3 change everyone was talking about.

The Jackson 3 change was not the hardest part.

Two applications, a few days of debugging, and one very confusing test suite later, I have a clear picture of what actually breaks — and what the official guide glosses over.

This is not a walkthrough of the migration guide. This is the stuff that costs you real time.

This is a preview. Read the full article on Medium: Spring Boot 4.0 Migration: What Nobody Tells You About the Breaking Changes

Stop Fighting CSS: The Google Stitch + Antigravity Stack for Solo Developers

Davide Mibelli — Fri, 24 Apr 2026 09:07:34 +0000

Most developers I know have the same problem: the logic is solid, but the UI looks like it was built in a hurry — because it was.
I spent two days on a login flow for a side project before I gave up and tried a different approach: Google Stitch for the UI, Antigravity as the IDE, and the MCP bridge between them. Here's what the workflow actually looks like.

The core idea

Google Stitch gives you pre-configured UI patterns — "Stitches" — that are already accessible and mathematically sound. You pick a functional category (Hero, Card, List), set one primary color and one font, and it handles the rest. The MCP connection to Antigravity means you never manually export manifests: change a button style in the Stitch web UI, and it reflects in your running simulator instantly.

The implementation

Once you've connected Stitch to Antigravity via MCP: Configure Server in the command palette, your components become requestable by name:

import { useStitch } from '@antigravity/react-hooks';

function DashboardCard({ userId }) {
  const { Component, loading } = useStitch('UserCard');
  if (loading) return <Placeholder />;
  return (
    <Component
      username={userId.name}
      onAction={(ev) => console.log('User tapped:', ev)}
    />
  );
}

Your React component has no idea what UserCard looks like. It just requests it. If you decide a List works better than a Card, you change it in Stitch — the code stays the same.

What I didn't expect

The MCP connection is bidirectional. You can send performance feedback from Antigravity back to Stitch, so the design tool knows which components are causing frame drops on real devices. The documentation barely mentions this — it took me a while to find it.

The full guide

I wrote a more detailed walkthrough on Medium covering the full setup, the Gravity Fields fetch pattern, and the telemetry configuration: https://medium.com/p/02aa7c97e131

What's your approach for UI as a solo dev — do you start from the design or the logic?

From DALL-E to gpt-image-2: The Architectural Bet That Finally Fixed AI Text

Davide Mibelli — Thu, 23 Apr 2026 21:04:44 +0000

This article was originally published on Medium.

Two years ago, if you asked an AI to design a menu for a Mexican restaurant, you’d get a beautiful layout of “enchuita” and “churiros.” It looked like food, and the font looked like letters, but it was essentially a visual fever dream. The “burrto” became a classic meme in dev circles — a reminder that while AI could paint like Caravaggio, it had the literacy of a toddler.

Yesterday, OpenAI launched ChatGPT Images 2.0 (gpt-image-2). I ran the same test. The menu was perfect. Not just the spelling, but the hierarchy, the prices, and the specialized diacritics. It is no longer just “generating pixels.” It is communicating.

This isn’t a minor version bump or a better training set. It’s a total architectural pivot that signals the end of an era. If you’ve spent the last three years building workflows around diffusion models, it’s time to rethink your pipeline.

1. Why text was broken (and how they fixed it)

To understand why gpt-image-2 works, you have to understand why DALL-E 3 failed at spelling. Diffusion models — the tech behind almost every major generator until now — work by denoising. They start with static and try to “find” an image. Because text pixels make up a tiny fraction of a training image, the model learned the texture of text rather than the logic of characters. To a diffusion model, an “A” is just a specific arrangement of lines, not a semantic unit.

OpenAI has quietly abandoned diffusion. While they won’t officially confirm the guts of the system, the PNG metadata and the model’s behavior tell the story: this is an autoregressive model.

It generates images the same way GPT-4 generates code — by predicting the next token. By integrating image generation directly into the language model pipeline, the model isn’t “drawing” a word; it’s “writing” an image. When the architecture treats a pixel and a letter as parts of the same conceptual stream, the “enchuita” problem simply vanishes.

2. The end of the CSS overlay hack

For those of us in agency work or product dev, AI images have always been a “background only” tool. If a client wanted a marketing banner with a specific CTA, we’d generate the art, then use a graphics library or CSS to overlay the text. It was the only way to ensure the brand name wasn’t spelled “Gooogle.”

Gpt-image-2 changes that calculus. With near-perfect rendering of Latin, Kanji, and Hindi scripts, the “post-processing” stage of the workflow is suddenly on the chopping block. You can now generate multi-paneled assets or social media posts where the text is baked into the composition with proper lighting and perspective.

But there’s a catch for your budget. At approximately $0.21 per high-quality 1024x1024 render, this is roughly 60% more expensive than the previous generation. If you’re at a high-volume startup, that’s a significant line item.

3. Thinking before rendering

The most impressive part of the new model isn’t the resolution — it’s the “thinking mode.” Borrowed from reasoning models like o3, the generator now spends compute time planning the layout before it touches a single pixel.

I watched it handle a prompt for “a grid of six distinct objects, each with a label in a different language.” Previous models would lose count by object four and turn the labels into Sanskrit-flavored gibberish. Gpt-image-2 paused, “thought” (generating reasoning tokens), and then executed. It can count. It can follow layout constraints.

This moves AI generation from “creative toy” to “reliable infrastructure.” Reliability is what we actually need in production. I’d much rather pay more for a single correct image than spend credits on ten “cheap” re-rolls.

4. The DALL-E eulogy

OpenAI is shutting down DALL-E 2 and 3 on May 12, 2026. Not moving them to a legacy tier — shutting them down.

This is a massive signal. It’s an admission that the diffusion approach hit a ceiling that no amount of fine-tuning could break. By retiring the DALL-E brand in favor of a unified ChatGPT Image model, OpenAI is betting that the future of Multimodality is a single, unified architecture.

The wall between “thinking” and “seeing” is being torn down. We used to have a brain (LLM) that sent instructions to a hand (Diffusion model). Now, the brain is doing the drawing itself.

5. What I’m still worried about

Despite the polish, there are gaps. The knowledge cutoff is December 2025. If you need a render involving a trend or news event from early 2026, you’re reliant on the web search tool, which adds latency and even more cost.

Furthermore, the pricing model is now “tokenized” for images. Thinking mode adds a variable cost based on how many reasoning tokens the model uses to plan the composition. This makes it incredibly hard to predict API costs for complex apps. You aren’t just paying for an image; you’re paying for the “brain power” required to frame it.

6. The 2026 reality check

If you are building a simple placeholder tool, stick to cheaper, older models. But for any workflow where the image is the content — marketing, UI prototyping, or localized assets — the shift to autoregressive generation is a one-way door.

We’re entering a phase where the term “image model” feels dated. We just have models. They happen to output pixels sometimes and Python code others. The fact that it can finally spell “Burrito” is just the first sign that the gap between human intent and machine execution has finally closed.