DEV Community: Umesh Malik

How to Increase SEO Traffic in the AI Era: 10 Techniques (2026)

Umesh Malik — Wed, 29 Jul 2026 17:49:55 +0000

How to increase SEO traffic in the AI era comes down to one uncomfortable truth: you no longer earn traffic by ranking — you earn it by being the source an AI reaches for, and then by giving the reader a reason to click through anyway. Put plainly, AI-era SEO is the practice of structuring your content and technical surfaces so AI engines retrieve it, quote it, and attribute it to you — while the occasional click still lands. This is the hands-on companion to SEO in the AI Era: The 2026 GEO Playbook. That post explained what changed and why. This one is the field guide: ten genuine, white-hat techniques you can start running this week, no budget and no ten-person content team required.

TL;DR

Growth now has two dials, not one. Citation share (do AI answers quote you?) and residual click-through (does the reader still visit?). You have to move both — optimizing only for rank moves neither.
The highest-leverage technique costs nothing: rewrite the first 60 words of your top pages into self-contained, quotable answers. It's the single change that most reliably turns a ranking page into a cited one.
First-party data is the only durable moat. Models synthesize summaries for free; they cannot synthesize your benchmark, your invoice, your incident. One original number beats a thousand rephrased paragraphs.
Genuine techniques only. Everything here is white-hat: no link schemes, no astroturfing, no AI-filler at volume. Those tactics violate search-engine spam policies and AI engines punish them too. Growth that survives is growth you'd be happy to explain.
Distribution is half the job. Being on Reddit, YouTube and the two forums that own your niche puts you inside the sources the models already trust — often faster than ranking your own domain.
Measure citations, not average CTR. In a world where ~68% of searches end without a click, average CTR falls even as your real traffic and revenue grow. Track the funnel that actually pays.

How to Increase SEO Traffic in the AI Era, Concretely

In the AI era, traffic grows when your content gets retrieved and quoted inside AI answers, when those answers earn the residual click, and when classic organic still sends the qualified visitors AI engines can't intercept. Three overlapping funnels, not one. The mistake almost everyone makes is pouring effort into the middle of the old funnel — more keywords, more backlinks, more word count — while the new one quietly decides who wins.

Here's the mental model I use. Ranking gets you into the candidate set an AI engine draws from. Structure and specificity decide whether you get extracted from that set. Entity authority decides whether you get attributed. And genuine value decides whether the human bothers to click. Every technique below moves exactly one of those four gates — I'll tell you which.

💡 Key insight: You can't "increase SEO traffic" as a single number anymore. You increase citation share and residual clicks separately, and they respond to different levers. Confusing the two is why so much effort produces so little movement.

The Ten Techniques, Ranked by Effort-to-Payoff

I ordered these by payoff per hour, not by importance. The first three you can do this week with content you already have. The last three are where you leave everyone else behind — and where almost nobody is competing yet.

The technical half — techniques 09 and 10 — is a weekend of work if you're already on Cloudflare. I documented the exact implementation in Agentic Browsing in PageSpeed Insights, the performance side in How to Fix Core Web Vitals (a stable, fast page is one screenshotting agents can actually read), and the callable endpoint end to end in Deploy an MCP Server on Cloudflare Workers and How to Build an MCP Server.

The Rewrite That Turns a Ranking Page Into a Cited One

Technique 01 sounds abstract until you see it side by side. Same facts, same page — only one version can be lifted into an AI answer without a human editing it first.

Genuine vs. Spam: The Line You Don't Cross

Every "grow your traffic fast" thread eventually recommends something that works for a month and then torches your domain. In the AI era the blast radius is bigger, because both search engines and the models learn to distrust the pattern. Here's the honest split.

Why the spam column is worse than useless now
Classic spam tactics degraded slowly — a penalty here, a de-index there. AI engines add a second, harsher feedback loop: once a model learns your content is low-trust or synthetic, it stops reaching for you across every query, and there's no "reconsideration request" for a model's retrieval prior. The genuine techniques are slower, but they're the only ones that compound instead of detonate.

Does AI Search Traffic Actually Convert?

If you only looked at CTR you'd conclude the effort isn't worth it. Look at composition instead.

The mechanism is intent compression. Someone who spent four turns in ChatGPT refining "I need a vector DB for a 50M-embedding hybrid-search workload" has already done the comparison shopping. When they land on you, they're at the end of the funnel. That's why techniques 04 and 05 — original data and comparison tables — pay off twice: they're what gets you cited, and they're what closes the visitor who arrives pre-qualified.

The One-Page Weekly Routine

Techniques don't move traffic; repeated techniques do. This is the entire routine, small enough to actually keep.

Once a quarter, layer in the bigger swings: publish one piece of genuinely first-party data (technique 04), refresh your top 20 pages (08), and — if you haven't yet — ship llms.txt, schema, a Markdown mirror and an MCP endpoint (09–10). The weekly routine moves extraction and attribution; the quarterly swings move the candidate set.

Common Mistakes That Cap Your Growth

Optimizing for a rank you already have. Position #3 and not being quoted? More backlinks won't fix it — rewrite the passage so it's extractable. Ranking is the qualifier; structure is the prize.
Chasing volume over evidence. Ten AI-written pages restating the docs will get you cited zero times. One page with a real benchmark gets cited repeatedly. Publish less, prove more.
Treating llms.txt as a keyword dump. It's a map — an H1, a real description, links to your best content. Stuffing it is the 2007 meta-keywords mistake in a new file. Ship it because it's cheap and now a Lighthouse audit, not because it's a proven ranking lever.
Blocking every AI crawler in a panic. Blocking retrieval bots (OAI-SearchBot, PerplexityBot, ChatGPT-User) cuts you out of the only path to being cited. Decide per bot — it's reasonable to allow retrieval crawlers while restricting pure-training ones.
Reporting average CTR as your headline metric. With ~68% of searches ending click-free, average CTR falls while total clicks and revenue climb. Report total clicks, AI-channel conversions and citation share instead.
Abandoning classic SEO. AI Overviews are largely assembled from pages that already rank, and cited brands earn meaningfully more organic clicks than uncited ones. GEO is a layer on top of technical SEO — an unindexed page is uncitable.

The Bottom Line

Increasing SEO traffic in the AI era isn't a new trick bolted onto the old playbook — it's a different objective that happens to share a name. You're no longer buying a rank and collecting the clicks it pays out. You're earning a place in the answer, and hoping the reader still wants the source.

The good news is that the techniques that work are the ones you'd want to do anyway: write clearer answers, prove your claims with real numbers, show up genuinely where your audience already is, and make your site trivially easy for a machine to read. None of it requires a budget or a team. All of it compounds. And unlike the shortcuts, none of it blows up in your face a month later.

Start with technique 01 on your single best page this week. Then read part one — SEO in the AI Era: The 2026 GEO Playbook — for the full strategic picture behind why these ten moves are the ones that matter.

Sources

Written for umesh-malik.com — no-fluff technical writing on AI, Web Dev, and Engineering. This is part two of the AI-era SEO series — start with SEO in the AI Era: The 2026 GEO Playbook for the strategy behind these techniques.

Originally published at umesh-malik.com

Keep reading on umesh-malik.com:

SEO in the AI Era: The 2026 GEO Playbook for Winning AI Search Traffic

Umesh Malik — Mon, 27 Jul 2026 15:44:19 +0000

SEO in the AI era is no longer a competition for ten blue links. It's a competition to be the source an AI model reaches for when it writes the answer — and then, occasionally, to earn the click that follows. The ranking is still real. The click is now optional.

TL;DR

The click collapsed, not the search. Zero-click searches hit roughly 68% in early 2026, up from about 45% a decade ago. Pew Research measured click-through at 8% when an AI Overview is present vs 15% when it isn't, and Seer Interactive clocked organic CTR falling 61% (1.76% → 0.61%) across 3,119 informational queries. Google's AI Mode is worse still: a ~93% zero-click rate.
The remaining traffic is dramatically better. Similarweb clickstream data (Apr–May 2026) puts ChatGPT referral conversion at 7.1% — second only to paid search. Ahrefs found AI search was 0.5% of traffic but 12.1% of signups. Fewer visitors, far higher intent.
GEO is not "SEO but with AI in the title." Generative Engine Optimization optimizes for extraction and citation, not position. The peer-reviewed GEO research found quotations lift AI visibility 41%, statistics 32%, cited sources 30%, and fluency 28% — levers that have no equivalent in classic SEO.
There is no single algorithm to game. An analysis of 680M citations found only 11% of domains are cited by both ChatGPT and Perplexity. You are optimizing for a fragmented committee, not one crawler.
The technical bar moved from "crawlable" to "callable." Clean semantic HTML, llms.txt, structured data, a Markdown mirror of every page, and increasingly an MCP endpoint. Machines are now a first-class audience with their own read path.
Track it or you'll conclude the wrong thing. Only ~14% of marketers separate AI search as a channel; most of it is silently misfiled as "direct" in GA4. You cannot manage a channel you can't see.

Your Traffic Didn't Drop Because You Got Worse at SEO

Here's the conversation I keep having. Someone shows me a Search Console chart where impressions are flat or up, average position is stable or improving — and clicks have fallen off a cliff since late 2025. Then they ask what they broke.

They broke nothing. The search engine stopped being a referral machine and started being an answer machine. Your content still ranked. It just got read, summarized, and delivered to the user inside a chat box with a citation chip they didn't click.

This is the single most important mental shift of the last two years, and most SEO advice hasn't caught up. The industry is still optimizing for a rank position that increasingly determines whether you get quoted, not whether you get visited. Those are different objectives with different tactics, and confusing them is why so many teams are working hard and losing ground.

💡 Key insight: In the AI era, ranking is the qualifier and citation is the prize. Position #1 that never gets extracted is worth less than position #6 that gets quoted in every answer.

What Is GEO (Generative Engine Optimization)?

Generative Engine Optimization (GEO) is the practice of structuring content, entities and technical surfaces so that AI systems — ChatGPT, Claude, Perplexity, Google AI Overviews and AI Mode, Copilot — retrieve your page, extract a specific claim from it, and attribute that claim to you in a generated answer.

The distinction that matters: classic SEO optimizes a document for a position. GEO optimizes a passage for retrieval and quotation. An AI engine doesn't rank your page — it chunks it, embeds the chunks, retrieves the two or three most relevant, and synthesizes. Your unit of competition shrank from "the article" to "the paragraph."

That single fact drives almost every practical tactic below. If your best insight is buried in paragraph nine of a section that requires the previous eight paragraphs for context, it cannot be extracted cleanly, so it will not be cited. A self-contained, factual, quotable paragraph beats a beautifully-argued essay that only works as a whole.

GEO, AEO, AI SEO — the naming is a mess
You'll see Generative Engine Optimization, Answer Engine Optimization, LLM SEO and AI Search Optimization used interchangeably. They describe the same job: earning visibility inside generated answers rather than inside a list of links. Pick one term and move on — arguing about the acronym is the least valuable thing you can do this quarter.

What Actually Changed — Nine Shifts, One Table

Most "AI changed SEO" posts wave at the vibe. Here is the concrete delta, item by item.

Three of those deserve unpacking, because they're the ones people get wrong.

1. Query fan-out means you can't see the query anymore

Google's AI Mode doesn't run your keyword. It decomposes the user's task into a set of synthetic sub-queries, runs them in parallel, and assembles the result. You will never see those sub-queries in Search Console. This is why keyword-level reporting is quietly becoming fiction, and why topical coverage beats keyword targeting: you want to be a plausible answer to a whole neighborhood of questions you cannot enumerate.

2. The citation market is fragmented, and that's good news

An analysis of 680 million citations across ChatGPT, Google AI Overviews and Perplexity found that only 11% of domains are cited by both ChatGPT and Perplexity. Each engine has different retrieval logic, different index freshness, different trust priors. And critically, even the most-cited domain on any platform rarely exceeds ~5% of total citations — versus classic SEO, where the top 10 results eat roughly two-thirds of clicks.

Translation: the AI citation market is less winner-take-all than the blue-link market ever was. A small, sharp, well-structured site can get cited alongside a Fortune 500 in a way it could never outrank one.

3. Crawling exploded; referrals didn't follow

This is the uncomfortable part. Cloudflare Radar's crawl-to-refer ratios (May 2026) show what the exchange actually looks like:

Cloudflare attributed 51.8% of AI crawler requests to training, 35.7% to mixed training-plus-retrieval, and only 9.3% to search-only purposes. So most of the machine attention on your site is not shopping for a link to send you. Deciding what to do about that — block, allow, or monetize — is now a real strategic call, not a robots.txt afterthought.

Why Bother? Because the Traffic That Survives Is Better

If you stopped at the CTR numbers you'd conclude the web is over. It isn't. The composition changed.

Three independent datasets point the same direction. Similarweb's clickstream panel puts ChatGPT referral conversion at 7.1%, behind only paid search (7.8%) and ahead of direct, organic, social and email. A 12-month GA4 study by Visibility Labs found ChatGPT traffic converting at 1.81% vs 1.39% for non-branded organic — a more modest but real 31% edge. Semrush reports AI-driven visitors converting at roughly 4.4x standard organic.

The mechanism is intent compression. Someone who spent four turns in ChatGPT narrowing "I need a vector database for a 50M-embedding workload with hybrid search" has already done the comparison shopping. When they land on you, they're at the end of the funnel, not the start. Traditional organic sends you people still browsing; AI sends you people already decided.

The honest caveat
Volume is small. AI referrals are typically low single-digit percentages of total sessions for most sites right now, and every one of these conversion studies uses a different panel, attribution window and definition of "AI traffic." Treat the direction as solid and the exact multiplier as marketing. The right posture is "this is a high-value emerging channel worth engineering for," not "abandon organic search."

The GEO Playbook — Nine Moves That Actually Move Citations

The peer-reviewed GEO research gives us the closest thing to a measured baseline: quotations increased AI visibility by 41%, statistics by 32%, cited sources by 30%, and fluency optimization by 28%. Notice what's absent from that list — keyword density, word count, exact-match headings. The levers changed.

Here's the playbook I actually run, in priority order.

For the technical half of moves 07 and 09, I documented the exact implementation — Worker, discovery files, MCP endpoint and all — in Agentic Browsing in PageSpeed Insights, and the endpoint itself in How to Build a Production MCP Server.

The Rewrite That Doubles Your Citation Odds

Abstract advice is easy to nod at and hard to apply. Here is the same content, written both ways.

How to Measure AI Traffic When Analytics Lies to You

Most AI traffic is invisible by default because AI clients strip or mangle the referrer, so GA4 files the session under "direct." Only about 14% of marketers currently track AI search as a distinct channel — which means most teams are either underestimating a growing channel or crediting it to the wrong one.

Fix it in three layers:

Layer 1 — a referral channel group. Create a custom channel in GA4 matching AI hostnames on the session source. A regex that covers today's field:

chatgpt\.com|chat\.openai\.com|perplexity\.ai|claude\.ai|copilot\.microsoft\.com|gemini\.google\.com|you\.com|phind\.com

Note that google.com referrals from AI Overviews and AI Mode look identical to classic organic — you cannot cleanly separate them in GA4 today. Don't pretend otherwise in your reporting.

Layer 2 — server logs for the crawl side. Referrals only show you the ~1% that clicked. Your access logs show the other side of the trade: who's reading you. Filter by user agent for GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, PerplexityBot, Google-Extended, Applebot-Extended, Bytespider, CCBot. Compute your own crawl-to-refer ratio per bot. That ratio is the honest scoreboard for whether the exchange is working for you.

Layer 3 — citation tracking. Run your 20 highest-value questions through ChatGPT, Perplexity, AI Mode and Claude on a fixed monthly cadence and record whether you're cited. It's manual and it's noisy — answers vary run to run — but it's the only direct read on the metric that matters. Tools exist for this; a spreadsheet and a recurring calendar block works fine to start.

The metric to actually report
Stop leading with average CTR. In a world where 68% of searches end without a click, a falling average CTR can coexist with growing total clicks and growing revenue. Report total clicks, AI-channel conversions, and citation share. Average CTR is now a ratio whose denominator is being inflated by impressions you were never going to convert.

Common Mistakes That Are Costing You Citations

Optimizing for a rank you already have. If you're position #3 and not being quoted, more backlinks won't fix it. Rewrite the passage so it's extractable. Ranking gets you into the retrieval candidate set; structure gets you into the answer.
Treating llms.txt as a keyword dump. It's a map: an H1, a real description, and links to your genuinely best content. Stuffing it is the 2007 meta-keywords mistake in a new file. Adoption is still early and even Google has been lukewarm — ship it because it's cheap and now an explicit Lighthouse audit, not because it's a proven ranking lever.
Publishing AI-written filler at volume. The engines are retrieving from a corpus increasingly full of generated text. Undifferentiated content has no reason to be picked. First-party data, original benchmarks and named opinions are the only durable moat, and they're the one thing a model can't synthesize from the existing web.
Blocking every AI crawler in a panic. Blocking ClaudeBot and GPTBot cuts training use — and also cuts you out of the retrieval paths that share user agents or infrastructure. Decide deliberately per bot: it's reasonable to allow retrieval bots (OAI-SearchBot, PerplexityBot, ChatGPT-User) while restricting pure training crawlers. Just know which you're doing and why.
Chasing an "AI visibility score" from a tool. Every vendor has an index and none of them can see inside the models. Use them for directional trend, never as a KPI. Your own citation spot-checks and server logs are more honest.
Abandoning classic SEO. AI Overviews are largely assembled from pages that already rank. Brands cited inside AI Overviews earn roughly 35% more organic clicks than uncited competitors. GEO is a layer on top of technical SEO, not a replacement for it — an unindexed page is uncitable.

Being PRO at SEO in the AI Era

Everything above gets you to competent. Four things separate the professionals.

1. You publish things that cannot be synthesized. The web is filling with plausible restatements of existing knowledge, and models are excellent at producing those for free. The only content with a structural advantage is content the model cannot generate: your benchmark run, your production incident, your pricing comparison with real invoices, your opinion with your name on it. Every hour spent producing first-party evidence is worth ten spent rephrasing documentation.

2. You own an entity, not a keyword list. Pick a narrow territory and be visibly, consistently the person or brand associated with it across every surface a model reads — your site, GitHub, YouTube, conference decks, forum answers, other people's posts. Entity strength is what breaks ties, and ties are most of the game now.

3. You treat machines as a first-class audience with their own read path. Humans get the designed page. Agents get semantic HTML, a Markdown mirror, llms.txt, structured data, and a callable endpoint. Same content, three formats, one source of truth. Most sites are still serving agents a JavaScript-heavy page and hoping.

4. You measure the funnel, not the vanity metric. Impressions → citations → clicks → conversions, with the AI channel broken out. You'll often find the channel that looks worst on CTR is the best on revenue per session. Teams that report on the wrong end of that funnel keep optimizing away their most valuable traffic.

Two of those overlap with work you may already have done: the CLS and performance items are covered in my Core Web Vitals optimization guide, and the callable layer is a weekend's work if you're already on Cloudflare — see Deploy an MCP Server on Cloudflare Workers.

The Bottom Line

SEO isn't dying. It's being demoted from "the traffic channel" to "the qualification round." Ranking still decides whether you're in the retrieval candidate set. Everything after that — whether you get extracted, quoted, attributed, and occasionally clicked — is a different discipline with different levers, and almost nobody is running it deliberately yet.

That's the opportunity. The citation market is fragmented enough that a small site with sharp, specific, well-structured, genuinely original content can sit in the same generated answer as a company with a hundred-person content team. That was never true of the blue links.

So: write answers, not articles. Publish evidence, not summaries. Serve machines a format they can read without guessing. Measure citations, not average CTR. And stop optimizing for a click-through rate that the interface itself decided to take away from you.

Sources

Written for umesh-malik.com — no-fluff technical writing on AI, Web Dev, and Engineering. Want the technical half in depth? Read Agentic Browsing in PageSpeed Insights: How to Make Your Website AI-Ready next.

Originally published at umesh-malik.com

Keep reading on umesh-malik.com:

Claude Opus 5: Benchmarks, Pricing & Two Breaking Changes

Umesh Malik — Fri, 24 Jul 2026 18:55:22 +0000

Claude Opus 5 landed on July 24, 2026, and it quietly did the most disruptive thing a model launch can do: it kept the old price. Same $5 / $25 per million tokens as Opus 4.8, and it more than doubles Opus 4.8's score on Anthropic's hardest agentic coding benchmark. In the API it's claude-opus-5.

That combination breaks the mental model most teams have been running on. For the last two months the ladder was simple — default to Sonnet, escalate to Opus, and pay double for Fable 5 when the task was genuinely brutal. Opus 5 collapses the top two rungs: it lands within half a percentage point of Fable 5 on real-world coding evals at roughly half the cost per task. The interesting question is no longer "when do I escalate to Fable?" It's "is there still a reason to?"

There's a catch, and it's the part the launch post doesn't lead with. Two API behaviors changed, and if you migrate by swapping the model string, one of them will silently truncate your responses and the other will 400 your requests.

TL;DR

Claude Opus 5 is a drop-in upgrade at Opus 4.8's exact price — $5 per million input tokens, $25 output, 1M context, 128K max output. No price increase, no new tier.
It gets near-Fable 5 capability at half Fable's cost. Within 0.5% of Fable 5 on CursorBench 3.2, past Fable 5's peak OSWorld 2.0 score at about a third of the budget, and 43.3% on Frontier-Bench v0.1 versus Fable 5's 33.7%.
Two breaking changes will bite a string-swap migration. Thinking is now on by default (so a tight max_tokens can truncate mid-answer), and thinking: {"type": "disabled"} returns a 400 at xhigh or max effort.
Re-sweep your effort levels. low and medium are meaningfully stronger on Opus 5 than on any earlier Opus — they're now the primary cost lever, not a compromise.
The play: make Opus 5 your default ceiling, keep Sonnet 5 for high-volume work, and reserve Fable 5 for the narrow band where you've measured that Opus 5 falls short.

What Is Claude Opus 5?

Claude Opus 5 is Anthropic's model for complex agentic coding and enterprise work — a frontier-class model priced at the mid-flagship tier. It has a 1-million-token context window, up to 128K output tokens per request, adaptive thinking that's on by default, high-resolution vision, and a May 2026 knowledge cutoff. You call it as claude-opus-5 on the Claude API, anthropic.claude-opus-5 on Amazon Bedrock, and claude-opus-5 on Google Cloud and Microsoft Foundry.

The one-sentence version: it's the model that made "escalate to the premium tier" a much rarer decision. Anthropic positions Fable 5 as the answer when you need the highest available capability, and Opus 5 as the answer for everything else demanding — which, after you look at the benchmarks, turns out to be nearly everything.

Three things define it in practice:

It verifies its own work. This is the headline behavioral change over Opus 4.8. It opens pages in a browser at desktop and phone widths, checks its output against reality, and iterates before handing back.
It finishes in fewer turns. Across effort levels, Opus 5 averaged roughly 9 percentage points higher accuracy while using about a third fewer turns and tool calls, and 60% less wall-clock time. Fewer turns is not a vanity metric — it's the token bill.
It's the most aligned Opus Anthropic has shipped. It scored 2.3 on their automated overall-misalignment audit, the lowest of any recent model, with the lowest rates of deceptive behavior and the least susceptibility to being tricked into misuse.

💡 Key insight: The story of Opus 5 is not a capability jump — those happen every few months now. It's a price-tier jump. Frontier-class results arrived on the shelf below the frontier-class price.

Claude Opus 5 Benchmarks

Anthropic leaned on a newer, harder set of evals for this launch — the older saturated ones stopped separating models. Here's what the numbers actually say.

Two results deserve a second look.

Frontier-Bench v0.1 is the one that matters for engineering teams. Opus 5 more than doubles Opus 4.8 (43.3% vs 18.7%) and clears Fable 5 (33.7%) — at half Fable's price. When a mid-tier model beats the premium tier on the premium tier's home turf, the ladder has changed shape.

ARC-AGI 3 is the one that matters for everyone else. 30.2% roughly triples the previous best any model had posted, and sits about four times GPT-5.6 Sol's 7.8%. ARC-AGI is deliberately built to resist memorization — it tests reasoning on problems the model hasn't seen a template for. A jump that size on that benchmark is not benchmark-tuning.

The customer quotes line up with the numbers rather than contradicting them, which is rarer than it should be. Devin called it "approaches Fable-level performance at half the cost." Cursor: "near Fable 5 intelligence at Opus speed and cost." Lovable called it the "biggest leap in the Opus family since 4.5," with "far less variance run to run" — and run-to-run variance is the thing that actually decides whether you can put a model in an unattended pipeline.

Where Opus 5 is still behind
Opus 5 deliberately does not advance the frontier on dual-use capability. It stays behind Mythos 5 on biology research and offensive cybersecurity: on OSS-Fuzz it identifies vulnerabilities at a similar rate but is considerably less successful at developing exploits. Its cyber classifiers are also less restrictive than Fable 5's — expected to intervene around 85% less often — with flagged requests falling back to Opus 4.8 by default.

The Real Cost Math

This is the section that changes what you do on Monday. Opus 5 did not get a price increase. It's $5 per million input tokens and $25 per million output — the same numbers Opus 4.8, 4.7, 4.6, and 4.5 all carried.

Run the arithmetic on a real workload. Take an agentic run that consumes 500K input tokens (large context, re-sent across turns) and 100K output tokens:

Opus 5: $2.50 + $2.50 = $5.00
Fable 5: $5.00 + $5.00 = $10.00
Sonnet 5: $1.50 + $1.50 = $3.00

But raw per-token price understates the gap, because Opus 5 needs fewer tokens to finish. It hit comparable quality while generating about 26% fewer tokens than Opus 4.8 at max reasoning, with a third fewer tool calls. Applied to the run above, that's closer to $3.70 of real spend — against $10.00 on Fable 5 for a result within half a percentage point on CursorBench. That's not a 2x saving. It's closer to 2.7x.

Two pricing details worth knowing before you budget:

Fast mode is a separate line item. speed: "fast" gives you up to 2.5x output throughput, priced at $10 / $50 — double the base rate, and Claude API only (not Bedrock, Google Cloud, or Foundry). It also draws on its own rate-limit pool.

Opus 5 has its own rate-limit bucket. Opus 4.8, 4.7, 4.6, and 4.5 share one combined Opus limit. Opus 5 does not draw from it. Shifting traffic over neither frees headroom on the old bucket nor inherits it — check your tier's Opus 5 limits before you move volume, or you'll rate-limit yourself on launch day.

The Two Breaking Changes

Here's the part that costs people an afternoon. Everything from the Opus 4.7 API surface still holds — budget_tokens is gone, sampling parameters (temperature, top_p, top_k) are rejected, and last-assistant-turn prefills 400. If you're already on 4.8, those are clean. Two things are genuinely new.

1. Thinking is on by default — and it eats your `max_tokens`

On Opus 4.8 and 4.7, omitting the thinking parameter meant no thinking. On Opus 5, omitting it runs adaptive thinking. The wire value didn't change; the default did.

This is a silent cost and truncation change, not just a behavior one. max_tokens is a hard cap on thinking plus response text. A route that ran thinking-off on Opus 4.8 and sized max_tokens tightly around its expected answer can now truncate mid-response — with no error, just a shorter reply and stop_reason: "max_tokens".

from anthropic import Anthropic

client = Anthropic()

# On Opus 4.8 this ran with NO thinking. On Opus 5 it thinks —
# and thinking tokens come out of the same 2000-token budget.
response = client.messages.create(
    model="claude-opus-5",
    max_tokens=2000,          # ← now shared between thinking and the answer
    messages=[{"role": "user", "content": "..."}],
)

The fix is one of two lines: raise max_tokens to leave room, or opt out explicitly with thinking: {"type": "disabled"} — subject to the next change.

2. Disabling thinking 400s at `xhigh` and `max` effort

You can only disable thinking at effort high or lower. Pair thinking: {"type": "disabled"} with xhigh or max and the request returns a 400.

# 400 on Claude Opus 5 — disabled thinking above "high" effort
client.messages.create(
    model="claude-opus-5",
    max_tokens=4096,
    thinking={"type": "disabled"},
    output_config={"effort": "xhigh"},   # ← invalid combination
    messages=[{"role": "user", "content": "..."}],
)

The trap is that this is validated per request, not per conversation. A later call that bumps effort to xhigh while thinking is still disabled gets rejected even though every earlier call in the same session succeeded. Audit every call site, not just the first one.

The honest recommendation on disabled thinking
Don't disable it. On Opus 5 the thinking-off path has two documented failure modes: the model occasionally writes a tool call into its visible response text instead of emitting a structured tool_use block — the turn succeeds, the call never runs, no error is raised — and it can leak internal <thinking> tags into the response. Running thinking-on at low or medium effort is cheaper and avoids both. If you truly must stay thinking-off, add "You may say a brief sentence before using a tool," delete any don't-reason instruction (it makes tag leakage worse), and ask generically for no internal XML tags.

Effort: Re-Sweep It, Don't Reuse It

Opus 5 supports all five effort levels — low, medium, high, xhigh, max — with high as the API default. The guidance genuinely changed from 4.8, and copying your old settings over is the most common way to leave money on the table.

Effort	When to use it on Opus 5
`max`	Genuinely frontier problems where correctness outranks cost. Can overthink simpler tasks.
`xhigh`	The starting point for coding and agentic work. Set `max_tokens` to at least 64K here.
`high`	The default. Most intelligence-sensitive workloads that aren't long-horizon agentic.
`medium`	Now a real option, not a compromise. Sweep it — quality often holds.
`low`	Subagents, scoped tasks, latency-sensitive paths. Stronger here than on any earlier Opus.

The line from Anthropic's own effort docs is unusually direct: low and medium are stronger on Opus 5 than on earlier Opus models, and you should "use them liberally as your primary control for token cost and response time wherever your evals show quality holds." That is the opposite of the reflex most teams built up over the 4.x series, where dropping below high meant visibly worse output.

One trap: effort controls thinking volume, not visible response length. If Opus 5's answers are too long, lowering effort won't reliably fix it — you have to prompt for brevity. A short conciseness instruction cut user-facing response length by about 20% in testing; changing effort didn't.

What Else Is New

The refusal path deserves a code snippet, because it's the one that crashes production:

response = client.beta.messages.create(
    model="claude-opus-5",
    max_tokens=16000,
    betas=["server-side-fallback-2026-07-01"],
    fallbacks="default",     # routes by refusal category, no model list to maintain
    messages=[{"role": "user", "content": "..."}],
)

if response.stop_reason == "refusal":
    handle_refusal()
else:
    print(response.content[0].text)

A refused request is a successful HTTP 200, not an exception. Code that reads response.content[0] unconditionally breaks on it. Check stop_reason first.

Opus 5 vs Fable 5 vs Sonnet 5

{#snippet newContent()}

Opus 5 is now the right answer for almost all of it. It's within half a percentage point of Fable 5 on real-world coding evals at half the cost per task, beats Fable 5 outright on Frontier-Bench and OSWorld 2.0, finishes in fewer turns, and carries a four-months-fresher knowledge cutoff. It's also the most aligned Opus shipped, with less restrictive cyber classifiers than Fable 5 — so fewer false-positive refusals on benign security work.

{/snippet}
{#snippet oldContent()}

Fable 5 still wins a narrow band. The very longest unattended runs, where Anthropic still recommends it for extended autonomous operation, and cyber/bio-adjacent research where its higher dual-use ceiling is the point. If you're paying double, you should be able to name the specific task where you measured Opus 5 falling short — not "it's the flagship."

{/snippet}

And Sonnet 5 hasn't been displaced — it's just been re-scoped. It remains the right default for high-volume backends, chat, RAG, and anything latency-sensitive, where $3 / $15 and fast turnaround beat a capability ceiling you never touch. Just remember its tokenizer inflates counts by roughly 30%, which eats into the apparent saving versus Opus 5.

The heuristic I'd use in July 2026: Sonnet 5 for volume, Opus 5 as the ceiling for everything hard, Fable 5 only where a measured gap justifies double. That's a simpler ladder than we had last month, and it's cheaper at the top.

The Honest Ledger

The Migration Checklist

That seventh item is the counterintuitive one and it's worth repeating: delete your verification scaffolding. Instructions like "include a final verification step" or "double-check your answer" were good practice on earlier models. On Opus 5 they cause over-verification, and removing them reduces it with no capability regression. It's a delete, not a rewrite — which inverts a prompting rule most teams have internalized. If you maintain a shared prompt library or a house CLAUDE.md, that's the line to carve out.

FAQ

The Bottom Line

Claude Opus 5 is the rare launch where the interesting number is the one that didn't change. Same $5 / $25 as the model it replaces, near-Fable capability, fewer turns to get there, and a knowledge cutoff four months fresher than the flagship. If you're running Opus 4.8 today, this is a migration where the ceiling goes up and the bill goes down — those don't come along often.

Just don't swap the string and walk away. Thinking is on by default now, which means the max_tokens you tuned last quarter is quietly a truncation risk, and disabled thinking plus xhigh effort is a 400 waiting for whichever call site raises effort first. Fifteen minutes of auditing buys you the whole upgrade.

Then do the thing most teams skip: re-sweep your effort levels. The single biggest cost lever in this release isn't the price tag — it's that low and medium finally hold quality on hard work. Measure it on your own evals rather than trusting your Opus 4.8 instincts, and the same workload can get better and cheaper at the same time.

Next: if you're deciding where Opus 5 fits alongside your tooling, read the Claude Code vs Cursor production breakdown, and the Sonnet 5 tokenizer gotcha before you assume Sonnet is always the cheaper option.

Sources

Introducing Claude Opus 5 — Anthropic's launch announcement (benchmarks, alignment audit, customer results)
Models overview — Anthropic docs: pricing, context window, knowledge cutoff, platform IDs
Effort — Anthropic docs: effort levels and the Opus 5 recommendations
Model migration guide — Anthropic docs: the Opus 4.8 → Opus 5 breaking changes
Anthropic launches Claude Opus 5 — VentureBeat's launch coverage

Written for umesh-malik.com — no-fluff technical writing on AI, Web Dev, and Engineering.

Originally published at umesh-malik.com

Keep reading on umesh-malik.com:

Vercel AI SDK in Production: Streaming, Tool-Calling & the Gotchas Nobody Tells You (2026)

Umesh Malik — Mon, 20 Jul 2026 21:43:47 +0000

TL;DR

The Vercel AI SDK in production is easy to start and easy to get wrong — it collapses a week of streaming plumbing into an afternoon, then quietly leaves the hard parts to you.
The demo is streamText + useChat. Production is everything around it: aborting generations, tool-calling loops that don't run forever, error and retry UX, rate limiting, and cost you can attribute.
Wire the abort signal all the way through. Stopping only on the client keeps the model generating — and billing — in the background.
Rate limiting and cost control are your job, not the SDK's. Add a 429 gate before streamText, and log usage from onFinish per user.
Default your routes to the Node runtime; reach for Edge only when global cold-start latency is a real, measured problem.

Most Vercel AI SDK tutorials lie by omission

You've seen the tutorial. Install ai and @ai-sdk/react, drop a streamText call in a route handler, wire up useChat, and — magic — tokens stream into a chat bubble. Ship it Friday.

Then real users show up. Someone fires ten prompts and cancels each one; your model keeps generating all ten in the background because "stop" only stopped the UI. A tool call loops on itself and burns 40 model calls for one question. A provider hiccups and your chat just... freezes, with no error, no retry, nothing. At the end of the month you get a bill and no idea which feature caused it.

The Vercel AI SDK is genuinely excellent — it's the fastest way to build AI features in a TypeScript app. But the gap between a working demo and a production feature is real, and it's exactly the part nobody writes about. This post is that part.

Everything below targets AI SDK 5 (the transport-based useChat) on Next.js 15 with the App Router. The patterns port to Vue and Svelte too — the core is framework-agnostic.

What the Vercel AI SDK actually is

The Vercel AI SDK is a TypeScript toolkit that gives you one API to talk to any LLM provider, plus framework hooks that stream model output into your UI. It has two halves you should keep straight in your head:

AI SDK Core — server-side functions like streamText, generateText, and tool. Provider-agnostic: swap @ai-sdk/openai for @ai-sdk/anthropic and the rest of your code doesn't change.
AI SDK UI — framework hooks like useChat and useCompletion that consume a stream and manage message state for you.

The whole value proposition is that you write a Route Handler with streamText and drop useChat into a client component, and the SSE-style streaming, message parsing, and state updates are handled. You don't hand-roll an event stream parser. That's real leverage — and it's why the SDK is worth using over raw fetch.

💡 Key insight: The SDK owns the transport — getting tokens from a model to a React state update. It owns nothing about policy — who's allowed to call, when to stop, what a failure looks like. Production is all policy.

Why the Vercel AI SDK in production isn't the demo

Here's the demo, honestly the smallest useful version. A route handler:

// app/api/chat/route.ts
import { openai } from '@ai-sdk/openai';
import { streamText, convertToModelMessages, type UIMessage } from 'ai';

export async function POST(req: Request) {
  const { messages }: { messages: UIMessage[] } = await req.json();

  const result = streamText({
    model: openai('gpt-5.6'),
    messages: convertToModelMessages(messages),
  });

  return result.toUIMessageStreamResponse();
}

And the client:

'use client';
import { useChat } from '@ai-sdk/react';
import { useState } from 'react';

export function Chat() {
  const { messages, sendMessage, status } = useChat();
  const [input, setInput] = useState('');

  return (
    <div>
      {messages.map((m) => (
        <div key={m.id}>
          <strong>{m.role}:</strong>
          {m.parts.map((part, i) =>
            part.type === 'text' ? <span key={i}>{part.text}</span> : null,
          )}
        </div>
      ))}
      <form
        onSubmit={(e) => {
          e.preventDefault();
          sendMessage({ text: input });
          setInput('');
        }}
      >
        <input value={input} onChange={(e) => setInput(e.target.value)} />
      </form>
    </div>
  );
}

That's it. It streams. Note two things AI SDK 5 changed that trip people up: useChat no longer manages your input state — you own it with useState — and messages are made of parts, not a single content string, because a message can interleave text, tool calls, and tool results.

This works. It's also naive in five specific ways. Let's fix each one.

1. Tool-calling loops that actually terminate

Tool calling is where the SDK earns its keep — and where a naive setup quietly runs up your bill. A tool is a function the model can decide to call:

import { tool, stepCountIs } from 'ai';
import { z } from 'zod';

const result = streamText({
  model: openai('gpt-5.6'),
  messages: convertToModelMessages(messages),
  tools: {
    getWeather: tool({
      description: 'Get the current weather for a city.',
      inputSchema: z.object({
        city: z.string().describe('City name, e.g. "Berlin"'),
      }),
      execute: async ({ city }) => {
        const data = await fetchWeather(city);
        return { tempC: data.tempC, condition: data.condition };
      },
    }),
  },
  stopWhen: stepCountIs(5),
});

Two production details. First, inputSchema is a Zod schema — it's fed to the model and validates the model's arguments before your execute runs, so malformed tool calls never reach your code. Use .describe() liberally; the descriptions are prompt engineering.

Second — and this is the one that bites — stopWhen: stepCountIs(5). Without a stop condition, a multi-step tool loop (model calls a tool, gets a result, decides to call another) can iterate far more than you expect when the model gets confused. stepCountIs(5) caps the loop at five steps. Set it deliberately based on how many tool hops your feature legitimately needs. An uncapped loop is an uncapped bill.

💡 Key insight: A tool-calling agent without a step cap is the AI equivalent of an infinite loop with a network call inside. Always bound the loop.

2. Aborting a generation without leaking cost

Users cancel. They rephrase mid-stream, hit stop, or navigate away. useChat gives you this for free — almost:

const { messages, sendMessage, status, stop } = useChat();

// status is 'submitted' | 'streaming' | 'ready' | 'error'
{(status === 'submitted' || status === 'streaming') && (
  <button type="button" onClick={stop}>Stop</button>
)}

Calling stop() aborts the in-flight fetch. Here's the trap: aborting the fetch does not, by itself, stop the model. The provider request keeps running server-side unless you propagate the abort. Wire the request's signal into streamText:

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-5.6'),
    messages: convertToModelMessages(messages),
    abortSignal: req.signal, // <-- the line everyone forgets
  });

  return result.toUIMessageStreamResponse();
}

req.signal aborts when the client disconnects. Pass it to streamText and the provider call is genuinely cancelled — you stop paying for tokens the user will never see. Skip this line and every "stop" is cosmetic: the UI freezes the output while the model finishes generating, invisibly, on your dime.

3. Error handling and retry UX

Providers fail. Rate limits, timeouts, transient 500s — at any real volume you'll hit all of them. The demo has no error path; the freeze is the error handling. Do better.

On the client, useChat surfaces an error object and a regenerate function:

const { messages, error, regenerate, status } = useChat();

{error && (
  <div role="alert">
    Something went wrong.
    <button type="button" onClick={() => regenerate()}>Retry</button>
  </div>
)}

On the server, control what leaks to the client. By default the SDK masks error details in the stream (good — don't leak provider internals or keys). When you need a real message, pass an onError mapper to your response:

return result.toUIMessageStreamResponse({
  onError: (error) => {
    // log the real error server-side; return a safe message to the client
    console.error('[chat] stream error', error);
    return 'The model is temporarily unavailable. Please retry.';
  },
});

The rule: log the truth on the server, show something safe and actionable on the client. A user staring at a frozen cursor churns. A user who sees "temporarily unavailable — retry" clicks retry.

4. Rate limiting — the SDK won't do it for you

Nothing in the AI SDK stops a single user from hammering your route. LLM calls cost real money per request, so an unthrottled chat endpoint is a standing invitation to run up your bill — accidentally or not. Add a gate before you call streamText:

import { Ratelimit } from '@upstash/ratelimit';
import { Redis } from '@upstash/redis';

const ratelimit = new Ratelimit({
  redis: Redis.fromEnv(),
  limiter: Ratelimit.slidingWindow(20, '1 m'), // 20 requests/min/user
});

export async function POST(req: Request) {
  const userId = await getUserId(req); // your auth
  const { success } = await ratelimit.limit(userId);
  if (!success) {
    return new Response('Rate limit exceeded. Slow down.', { status: 429 });
  }

  const { messages } = await req.json();
  const result = streamText({
    model: openai('gpt-5.6'),
    messages: convertToModelMessages(messages),
    abortSignal: req.signal,
  });
  return result.toUIMessageStreamResponse();
}

Key it on a user ID where you can, IP as a fallback. Per-user limits survive shared networks and NAT; pure IP limits punish everyone behind one office router. Return 429 early so you never pay for the model call you were about to reject.

5. Cost you can actually attribute

"Our AI bill went up" is a useless sentence if you can't say which feature or which user. The provider dashboard gives you a monthly total. onFinish gives you per-request truth:

const result = streamText({
  model: openai('gpt-5.6'),
  messages: convertToModelMessages(messages),
  abortSignal: req.signal,
  onFinish: ({ usage }) => {
    // usage carries inputTokens, outputTokens, totalTokens
    logUsage({ userId, feature: 'chat', usage });
  },
});

Log token usage per user and per feature, then multiply by your model's price. Now "the bill went up" becomes "the summarize feature's output tokens tripled after we widened the context window" — a sentence you can act on. This is also how you decide when to route cheap requests to a smaller model: you can't optimize a cost you don't measure. If you're weighing models, the Claude Sonnet 5 guide walks through the real cost math on exactly this tradeoff.

Edge vs Node: pick Node by default

The SDK streams fine on both the Edge and Node runtimes, and a lot of tutorials reach for export const runtime = 'edge' reflexively. Don't, unless you've measured a reason to.

Concern	Edge runtime	Node runtime
Cold start	Very fast	Slower
Streaming	Works	Works
Node APIs / many SDKs	Partial / unavailable	Full
Long generations	Tighter platform limits	Generous timeouts
Vector DB / ORM clients	Often unsupported	Supported

Edge wins on global cold-start latency. But the moment you need a database client, a Node-only SDK, or a long generation, Edge fights you. Default to Node, and move a route to Edge only when startup latency is a measured problem for that specific route. Premature Edge adoption is a top source of "works locally, breaks in prod."

Common mistakes

Stopping on the client only. No abortSignal: req.signal means "stop" is cosmetic and you keep paying. The single most common production bug.
No stopWhen on tool calls. An uncapped multi-step loop is an uncapped bill.
Treating useChat like v4. In AI SDK 5 you own the input state and read message parts, not a content string. Copy-pasting old tutorials breaks in confusing ways.
No rate limit. One script, or one frustrated user mashing send, and your endpoint is a money leak.
Leaking raw provider errors to the client instead of mapping them with onError.
Reflexive Edge runtime that breaks the moment you add a DB client.

Best practices, in order

Always pass abortSignal: req.signal into streamText. Non-negotiable.
Always set stopWhen when tools are involved. Pick the number deliberately.
Gate the route with a 429 before calling the model, keyed on user ID.
Log usage in onFinish per user and feature from day one — retrofitting cost attribution is painful.
Map errors with onError; log the real one, return a safe, actionable message.
Start on Node. Promote to Edge per-route only when you've measured a latency win.
Read message parts and render text, tool calls, and tool results distinctly — it's how you build transparent, debuggable agent UIs.

The Vercel AI SDK vs raw fetch vs LangChain.js

Dimension	Vercel AI SDK	Raw `fetch` + SSE	LangChain.js
Streaming to React	Built-in (`useChat`)	Hand-rolled parser	Wrapper, less UI-native
Provider switching	One line	Rewrite per provider	Abstracted
Tool calling + loop control	First-class (`tool`, `stopWhen`)	DIY	First-class, heavier
Bundle / footprint	Small	Smallest	Largest
Best for	Web apps with streaming UI	Total control, minimal deps	Complex chains/orchestration

My take: for a web app that streams AI output to a UI, the Vercel AI SDK is the right default — you get the transport for free and keep control of policy. Reach for raw fetch only when you need absolute control and minimal dependencies, and for LangChain.js when your orchestration is genuinely complex (multi-agent graphs, elaborate retrieval chains). For most product teams, that's not day one.

The runnable example

The complete, runnable version of everything above — route handler with abort, rate limit, and usage logging, a tool-calling loop with a step cap, and a client with stop and retry — lives in a self-contained project you can clone and run: vercel-ai-sdk-production on GitHub. Copy the patterns, not just the happy path.

FAQ

Conclusion

The Vercel AI SDK isn't the hard part of shipping AI features — it's the easy part, and it's very good at being easy. The hard part is the production layer the SDK deliberately leaves to you: when to stop, who's allowed to call, what failure looks like, and what it all costs. Wire the abort signal through, cap your tool loops, gate the route, and measure your tokens, and you've closed the gap between a Friday demo and something you can actually put in front of users.

Next, put this to work on something real: building a retrieval-augmented chatbot with the same SDK. Until that follow-up lands, build the RAG pipeline it sits on and get clear on RAG vs fine-tuning so you're grounding on the right foundation. If you're deploying the model-facing infra too, here's how to deploy an MCP server on Cloudflare Workers.

Sources

Written for umesh-malik.com — no-fluff technical writing on AI, Web Dev, and Engineering.

Originally published at umesh-malik.com

Keep reading on umesh-malik.com:

React Server Components in 2026: The Mental Model, the use client Boundary & When Not to Use Them

Umesh Malik — Mon, 20 Jul 2026 21:43:15 +0000

TL;DR

React Server Components are components that render only on the server and ship zero JavaScript to the browser — the default in Next.js 15's App Router.
The mental model: your app is two graphs — a server graph (default) and a client graph (opt-in with use client). The whole skill is knowing where to draw the line.
use client marks a boundary, not a file — everything imported into a client module joins the client bundle.
The payoff is Core Web Vitals: teams report 60–70% less client JavaScript, which directly improves LCP and INP.
Server Components are not always the answer. Anything interactive — state, effects, event handlers, browser APIs — must be a Client Component. Don't fight that.

Server Components aren't experimental anymore — they're the default

For two years React Server Components were the thing everyone had opinions about and nobody fully understood. That era is over. In 2026, Next.js 15 ships RSC as the default, Remix adopted them, and the rest of the ecosystem is falling in line. If you're writing React and still treating every component as a client component, you're shipping JavaScript you don't need to.

Here's the part that trips people up: RSC isn't a feature you turn on. It's a change in the default. In the App Router, every component is a Server Component unless you say otherwise. The question flipped from "should this be a server component?" to "does this really need to be a client component?"

This post gives you the mental model that makes RSC click, the exact rules for the use client boundary, and — the part tutorials skip — when Server Components are the wrong tool.

What React Server Components actually are

React Server Components are components that render exclusively on the server and send only their resulting UI to the browser, shipping zero JavaScript for themselves. That last clause is the whole point. A Server Component's code — its imports, its data-fetching, its formatting libraries — never reaches the client bundle.

This is different from SSR, and the confusion between the two is the root of most RSC misunderstanding:

Server-Side Rendering (SSR) renders your components to HTML on the server, then ships the JavaScript and re-runs (hydrates) it on the client. The JS still goes over the wire.
React Server Components render on the server and ship no JavaScript for those components at all. There's nothing to hydrate.

So a Server Component can await your database directly, import a heavy markdown parser, or read the filesystem — and none of that weight lands on the user's phone.

// app/page.tsx — a Server Component (no 'use client')
import { db } from '@/lib/db';

export default async function Page() {
  const posts = await db.post.findMany(); // runs on the server, ships no JS
  return (
    <ul>
      {posts.map((p) => <li key={p.id}>{p.title}</li>)}
    </ul>
  );
}

No useEffect to fetch, no loading spinner, no client-side data library. The data is there before the component renders.

💡 Key insight: SSR is about when you render (on the server, first). RSC is about where the code lives (on the server, permanently). One ships JS and hydrates; the other ships none.

Think in two graphs

The mental model that makes everything else obvious: your component tree is split into two module graphs.

The server graph is the default. Components here render on the server, can be async, can touch server-only resources, and ship no JS.
The client graph is opt-in. Components here are your familiar React — useState, useEffect, event handlers, browser APIs — and they ship JavaScript.

use client is the doorway between them. Everything on the far side of that door is client code. The entire practice of RSC is deciding where to put the door — as far down the tree as possible, so the interactive leaves are client components and everything above them stays on the server.

The `use client` boundary

use client at the top of a file marks a boundary, not just that one file. Once a module is a client module, every module it imports is pulled into the client bundle too.

'use client'; // this file and everything it imports is now client code

import { useState } from 'react';

export function Counter() {
  const [n, setN] = useState(0);
  return <button type="button" onClick={() => setN(n + 1)}>{n}</button>;
}

The mistake this causes: put use client too high in the tree and you drag half your app into the client bundle. A single use client at the top of a layout can turn every child into client code, silently erasing the RSC benefit.

The rule: push use client to the leaves. Keep the interactive `` a client component; keep the page that renders it a server component.

Server Components as children — the pattern that unlocks it

"But my interactive component needs to wrap server content" is the objection everyone hits. The answer is the composition pattern that makes RSC actually usable: a Client Component can render Server Components passed to it as children (or any prop).

`tsx
// ClientShell.tsx
'use client';
import { useState } from 'react';

export function ClientShell({ children }: { children: React.ReactNode }) {
const [open, setOpen] = useState(true);
return (

setOpen(!open)}>Toggle
{open && children}

);
}
`

`tsx
// page.tsx — a Server Component
import { ClientShell } from './ClientShell';
import { ServerData } from './ServerData'; // stays on the server!

export default function Page() {
return (

);
}
`

`` renders on the server and stays there, even though it's displayed inside a client component. The client component receives it as already-rendered UI — it never imports it, so it never pulls it into the client bundle. Interactivity on the outside, server rendering on the inside. This is the pattern that lets you keep the boundary low.

Streaming with Suspense

Server Components pair with `` to stream UI progressively. Wrap a slow server component and the rest of the page ships immediately while the slow part streams in when ready:

`tsx
import { Suspense } from 'react';

export default function Page() {
return (
<>

  }>
     {/* streams in when its data resolves */}

</>

);
}
`

The user sees the shell instantly instead of waiting for the slowest query. If you want the deeper story on out-of-order streaming, I wrote about streaming HTML out of order without JavaScript — RSC streaming is the React-flavored version of the same idea.

The Core Web Vitals payoff

This is why I care about RSC, and why you should. Less client JavaScript is the most direct lever on Core Web Vitals there is. Teams moving to RSC report 60–70% reductions in client bundle size, and that shows up where it counts:

LCP improves because the browser parses and executes less JS before it can paint.
INP improves because the main thread isn't clogged hydrating components that never needed to be interactive.

If you've been fighting Web Vitals by code-splitting and lazy-loading your way around a giant bundle, RSC attacks the problem at the source: it never sends the bundle. Pair it with the fundamentals in my Core Web Vitals guide and the React performance techniques post, and you're optimizing the cause, not the symptom.

When NOT to use Server Components

Bold take, honestly held: Server Components are the right default, and the wrong tool for anything interactive. Reach for a Client Component — without guilt — when you need:

State or effects — useState, useReducer, useEffect.
Event handlers — onClick, onChange, anything user-driven.
Browser APIs — window, localStorage, IntersectionObserver, geolocation.
Client-only libraries — most animation, charting, and map libraries assume the DOM.
Context that changes on interaction — live theme toggles, open/close state.

The failure mode isn't "used a client component." It's using client components by default out of habit, or contorting a genuinely interactive feature to stay on the server. Draw the boundary deliberately: server by default, client at the interactive leaves.

Common mistakes

use client at the top of the tree — drags everything below it into the client bundle and erases the benefit.
Confusing RSC with SSR — expecting hydration where there is none, or thinking SSR already gave you the JS savings.
useState/useEffect in a Server Component — a hard error; those need the client graph.
Importing a Server Component into a Client Component — pulls it client-side. Pass it as children instead.
Fetching in useEffect out of habit — in a Server Component you just await the data.
Accessing window in a Server Component — it doesn't exist there; guard or move to the client.

Best practices, in order

Server by default. Only add use client when a component genuinely needs the client graph.
Push the boundary to the leaves — small, interactive client components; server everything above.
Compose with children to keep server content inside client shells without importing it.
await data in Server Components instead of client-side fetching where you can.
Wrap slow server components in `` to stream and protect perceived performance.
Treat client JS as a budget — every use client spends it. Measure your bundle.

Common questions, answered

Question	Short answer
Is RSC the same as SSR?	No — SSR ships and hydrates JS; RSC ships none for server components.
Do Server Components replace Client Components?	No — they're the default; client components handle interactivity.
Can a Client Component render a Server Component?	Yes, if passed as `children`/props — not if imported.
Where should `use client` go?	At the interactive leaves, as low in the tree as possible.

FAQ

Conclusion

React Server Components aren't a new API to memorize — they're a new default to internalize. Think in two graphs, keep the use client boundary at the interactive leaves, compose server content into client shells with children, and you get dramatically less JavaScript for free. That's not a micro-optimization; it's the most direct Core Web Vitals win available to a React app in 2026.

Put it to work: attack your bundle at the source with RSC, then close out the fundamentals with the Core Web Vitals guide and React performance techniques. And if you're weighing where React and the framework layer are headed, Cloudflare viNext and Next.js on Vite is the wider context.

Sources

Written for umesh-malik.com — no-fluff technical writing on AI, Web Dev, and Engineering.

Originally published at umesh-malik.com

Keep reading on umesh-malik.com:

Build a RAG Chatbot in Next.js: Retrieval, Streaming & Citations (2026)

Umesh Malik — Mon, 20 Jul 2026 21:43:14 +0000

TL;DR

A RAG chatbot in Next.js is a chat UI that retrieves your own data before it answers — so the model responds from your documents, not just its training set.
The loop is three steps: embed the question → search a vector store → inject the hits into the prompt → stream a grounded answer.
Use Supabase + pgvector for storage and the AI SDK for embeddings and streaming. No separate vector database required.
Render citations in the UI. A RAG answer with no visible sources is just a chatbot with extra steps — citations are what make it trustworthy.
Most RAG chatbots fail at retrieval, not generation. If the right chunks don't surface, no prompt saves the answer.

A chatbot that makes things up is a liability

A plain LLM chatbot is confidently wrong about your business. Ask it about your refund policy, your API, or last quarter's numbers and it will invent something plausible, because it has never seen your data. That's not a bug you can prompt your way out of — the information simply isn't in the model.

Retrieval-augmented generation fixes this by fetching the relevant facts at query time and handing them to the model as context. The model stops guessing and starts summarizing what you gave it. Done right, it also cites where each claim came from, so a human can verify it.

This post builds the whole thing in Next.js 15 with AI SDK 5: the retrieval step, the streaming chat UI, and — the part most tutorials skip — citations rendered in the interface plus the guardrails that keep the model honest.

One boundary up front: this is the product layer. If you want how the underlying pipeline works — chunking strategy, embedding choice, hybrid search, reranking — build the RAG pipeline from scratch first. This post assumes you have documents in a vector store and focuses on wiring them into a chatbot users actually trust.

What a RAG chatbot in Next.js actually is

A RAG chatbot in Next.js is a chat interface where every user question triggers a retrieval step — the app embeds the question, finds the most relevant chunks of your own content, and passes them to the LLM as grounding context before generating a streamed answer. Strip away the framing and it's a three-stage request:

Retrieve — turn the question into a vector, find the nearest chunks in your store.
Augment — build a prompt that says "answer using only this context," with the chunks inlined.
Generate — stream the model's answer back to the UI, with citations.

The model never sees your whole knowledge base. It sees the handful of chunks retrieval decided were relevant. That's the entire game — and it's why retrieval quality, not prompt cleverness, decides whether your chatbot is useful.

💡 Key insight: RAG doesn't make the model smarter. It makes the model informed. Everything good about the answer traces back to what retrieval put in front of it.

The vector store: Supabase + pgvector

You don't need a dedicated vector database. pgvector runs inside the Postgres you probably already have, which means one less service to operate. Enable the extension and create a table:

create extension if not exists vector;

create table documents (
  id bigserial primary key,
  content text not null,
  source text,               -- url or title, for citations
  embedding vector(1536)     -- text-embedding-3-small = 1536 dims
);

-- cosine-distance similarity search
create or replace function match_documents (
  query_embedding vector(1536),
  match_count int default 5
) returns table (id bigint, content text, source text, similarity float)
language sql stable as $$
  select id, content, source,
         1 - (documents.embedding <=> query_embedding) as similarity
  from documents
  order by documents.embedding <=> query_embedding
  limit match_count;
$$;

The <=> operator is cosine distance; 1 - distance gives you a similarity score you can threshold on. Add an index (ivfflat or hnsw) once you have real volume.

Embedding and retrieval with the AI SDK

The AI SDK gives you embed for a single value and embedMany for batches. Embed the user's question, then hand the vector to your match_documents function:

// lib/retrieve.ts
import { embed } from 'ai';
import { openai } from '@ai-sdk/openai';
import { createClient } from '@supabase/supabase-js';

const supabase = createClient(process.env.SUPABASE_URL!, process.env.SUPABASE_KEY!);

export async function retrieve(query: string) {
  const { embedding } = await embed({
    model: openai.textEmbeddingModel('text-embedding-3-small'),
    value: query,
  });

  const { data, error } = await supabase.rpc('match_documents', {
    query_embedding: embedding,
    match_count: 5,
  });
  if (error) throw error;

  // Drop weak matches — a bad hit is worse than no hit.
  return (data ?? []).filter((d) => d.similarity > 0.75);
}

That similarity filter matters more than it looks. A low-relevance chunk doesn't just waste tokens — it actively misleads the model, which will dutifully summarize garbage if you feed it garbage. Threshold, don't just take the top 5.

Wiring retrieval into the streaming route

Now connect retrieval to generation. The route embeds the latest question, retrieves context, builds a grounding system prompt, and streams the answer. It reuses every production pattern from the Vercel AI SDK production guide — abort signal, error mapping:

// app/api/chat/route.ts
import { openai } from '@ai-sdk/openai';
import { streamText, convertToModelMessages, type UIMessage } from 'ai';
import { retrieve } from '@/lib/retrieve';

export const runtime = 'nodejs';

export async function POST(req: Request) {
  const { messages }: { messages: UIMessage[] } = await req.json();

  // The question is the last user message's text.
  const last = messages.at(-1);
  const question = last?.parts.find((p) => p.type === 'text')?.text ?? '';

  const chunks = await retrieve(question);
  const context = chunks
    .map((c, i) => `[${i + 1}] (${c.source})\n${c.content}`)
    .join('\n\n');

  const result = streamText({
    model: openai('gpt-5.6'),
    abortSignal: req.signal,
    system: [
      'You answer using ONLY the context below.',
      'If the context does not contain the answer, say you do not know.',
      'Cite sources inline with their bracket numbers, e.g. [1].',
      '',
      context || '(no relevant context found)',
    ].join('\n'),
    messages: convertToModelMessages(messages),
  });

  return result.toUIMessageStreamResponse({
    // Attach the retrieved sources so the client can render citations.
    messageMetadata: () => ({ sources: chunks.map((c) => c.source) }),
  });
}

Two decisions that make or break trust. First, the system prompt says "answer using ONLY the context" and "say you do not know" — this is your primary hallucination guardrail. Second, messageMetadata ships the retrieved sources alongside the stream so the UI can show them.

Rendering citations in the chat UI

A grounded answer nobody can verify isn't grounded in practice. Surface the sources with the message:

'use client';
import { useChat } from '@ai-sdk/react';
import { useState } from 'react';

export default function Chat() {
  const { messages, sendMessage, status } = useChat();
  const [input, setInput] = useState('');

  return (
    <div>
      {messages.map((m) => (
        <div key={m.id}>
          <strong>{m.role}: </strong>
          {m.parts.map((p, i) => (p.type === 'text' ? <span key={i}>{p.text}</span> : null))}

          {/* Citations from messageMetadata */}
          {m.role === 'assistant' && m.metadata?.sources?.length > 0 && (
            <ul>
              {m.metadata.sources.map((src, i) => (
                <li key={i}>
                  [{i + 1}] <a href={src}>{src}</a>
                </li>
              ))}
            </ul>
          )}
        </div>
      ))}

      <form
        onSubmit={(e) => {
          e.preventDefault();
          sendMessage({ text: input });
          setInput('');
        }}
      >
        <input value={input} onChange={(e) => setInput(e.target.value)} />
      </form>
    </div>
  );
}

Now every answer arrives with a "Sources" list. Users can click through and check. That single UI affordance is the difference between a demo and something a team will actually rely on.

Grounding guardrails that stop hallucinations

The system prompt is necessary but not sufficient. Layer these:

Threshold retrieval (shown above). No chunk clears the bar? Return "I don't have information on that" instead of an empty-context guess.
Instruct the "I don't know" path explicitly. Models default to helpfulness; you have to make refusal an allowed, expected outcome.
Keep context tight. Five focused chunks beat twenty loose ones — more context dilutes attention and raises cost.
Cite by construction. Number the chunks in the prompt and require bracket citations. If the model can't point to a chunk, it shouldn't make the claim.
Log the retrieved chunks per answer. When the bot is wrong, you'll almost always find retrieval, not generation, was the culprit.

💡 Key insight: You cannot debug a RAG chatbot by reading its answers. You debug it by reading what retrieval fed it. Log the chunks.

Common mistakes

Skipping the similarity threshold — feeding low-relevance chunks that mislead the model.
No "I don't know" path — the model invents an answer when context is empty.
No citations in the UI — users can't verify, so they either over-trust or don't trust at all.
Dumping the whole knowledge base into context — expensive, slower, and worse answers from diluted attention.
Blaming the model for retrieval failures — most wrong answers are wrong chunks. Fix retrieval first.
Embedding the raw multi-turn history instead of the actual question — retrieval quality craters.

Best practices, in order

Threshold every retrieval and return a graceful "I don't know" when nothing clears the bar.
Ground hard in the system prompt — "only this context," and refusal is allowed.
Render citations with every assistant message; make them clickable.
Keep context small and focused — quality of chunks over quantity.
Log retrieved chunks per query for debugging and evals.
Reuse production patterns from the AI SDK — abort signal, rate limit, cost logging — a RAG route is still a model route.
Measure retrieval with an eval set before you tune prompts. If recall is bad, prompting is rearranging deck chairs.

RAG vs the alternatives

Approach	Best for	Cost / effort	Freshness
RAG chatbot	Answering from your own, changing data	Moderate	Live — update the store
Fine-tuning	Style, format, narrow domain behavior	High, retrain to update	Frozen at training time
Long-context stuffing	Small, static doc sets	Low setup, high per-call cost	Manual

For a chatbot over documents that change, RAG is the default — it's cheaper to keep fresh and it can cite. Fine-tuning teaches behavior, not facts; the two are complementary, not competing. If you're deciding, RAG vs fine-tuning breaks down exactly when each wins.

The runnable example

A complete, runnable RAG chatbot — Supabase schema and match_documents function, an ingestion script that chunks and embeds with embedMany, the streaming route with grounding, and the citations UI — is in a self-contained project: rag-chatbot-nextjs on GitHub. Clone it, point it at your own docs, and you have a grounded chatbot in an afternoon.

FAQ

Conclusion

A RAG chatbot is only as good as what retrieval feeds it — and only as trustworthy as the citations it shows. Get retrieval right, ground the model hard, and render sources users can click, and you've built a chatbot a team will actually rely on instead of quietly distrust. The Next.js and AI SDK plumbing is the easy 20%; the retrieval quality and the trust affordances are the 80% that decides whether it ships.

From here: harden it with the production patterns from the AI SDK guide, deepen retrieval with the from-scratch pipeline, and settle the RAG vs fine-tuning question for your use case.

Sources

Written for umesh-malik.com — no-fluff technical writing on AI, Web Dev, and Engineering.

Originally published at umesh-malik.com

Keep reading on umesh-malik.com:

Kimi K3 vs Claude Fable 5: The Full Head-to-Head Benchmarks, Pricing, and Where the Open Model Wins (2026)

Umesh Malik — Sun, 19 Jul 2026 10:25:14 +0000

Kimi K3 vs Claude Fable 5 is the matchup nobody in the West saw coming this fast. On July 16, 2026, Moonshot AI shipped Kimi K3 — a 2.8-trillion-parameter open-weight model — and it did something no Chinese lab had done before: it beat Anthropic's flagship Claude Fable 5 outright on multiple frontier benchmarks, including topping the Frontend Code Arena.

Let me be honest up front, because the headlines are not: K3 does not dethrone Fable 5 overall. Fable 5 still wins the broader set of evaluations and leads the composite intelligence index. Moonshot's own launch post admits K3 trails Fable 5 and GPT-5.6 Sol on aggregate.

But that framing misses the real story. Where Kimi K3 wins, it wins the lanes that matter most for agent builders — long-horizon coding, terminal automation, agentic browsing, and frontend generation — and it does it at roughly one-third the price. For a huge class of production workloads, "slightly behind the absolute frontier, at 70% lower cost, with open weights" is not second place. It is the new default.

TL;DR

Kimi K3 launched July 16, 2026 from Moonshot AI: a 2.8T-parameter mixture-of-experts model (16 of 896 experts active per token), 1M context, native multimodal. Open weights followed within weeks — the largest open model ever released.
It genuinely beats Claude Fable 5 on Terminal-Bench 2.1 (88.3 vs 84.6), SWE-Marathon (42.0 vs 35.0), BrowseComp (91.2 vs 88.0), and the Frontend Code Arena (#1, 1,679 Elo).
Fable 5 still wins overall: it leads the Artificial Analysis Intelligence Index (59.9 vs 57.1) and takes FrontierSWE (86.6 vs 81.2), DeepSWE (70.0 vs 67.5), GDPval, JobBench, and vision.
K3 costs about 70% less: $3 / $15 per 1M tokens vs Fable 5's $10 / $50 — roughly 3.3x cheaper, flat across the full 1M window.
The right move is workload routing, not wholesale switching: K3 for agentic/terminal/browsing/high-volume, Fable 5 for frontier-difficulty engineering, vision, and knowledge work.
Big caveat: the two ran through different harnesses (KimiCode vs Claude Code/Codex), which limits clean model-to-model conclusions.

Kimi K3 vs Claude Fable 5: The Benchmark Scorecard

Here is the full head-to-head on vendor-reported, max-effort numbers. I've marked the winner of each row honestly — including the ones where the open model loses.

Count it up and the pattern is clear: Fable 5 wins the broader board, but every single one of K3's wins is in the agentic / long-horizon / automation lane. That is not a coincidence — it is what Moonshot optimized for.

💡 Key insight: K3 and Fable 5 are not fighting for the same crown. Fable 5 is the frontier-intelligence ceiling; K3 is the agentic-throughput value leader. Which one is "better" depends entirely on whether your workload looks like a hard exam or a long, cheap, repetitive grind.

Watch the Agentic Gap Animate

The wins that matter for agent builders are the long-horizon ones — the tasks where the model has to keep going for dozens or hundreds of steps without losing the thread. Here is how K3 stacks up against Fable 5 on exactly those, as an animated reveal.

Where Kimi K3 Actually Wins

Strip away the leaderboard noise and K3's edge is specific and real.

The most quotable single result is the Frontend Code Arena: in blind developer testing, K3 ranked first at 1,679 Elo, ahead of Fable 5. That is a human-preference benchmark, not a synthetic one — real developers picked K3's UI code more often. For anyone shipping frontend work, that is the result to actually test against your own prompts.

If you want context on how fast Chinese labs have been closing this gap, I traced the earlier jump in DeepSeek V4's challenge to US AI rivals — K3 is that trajectory reaching the frontier.

The Cost Story Is the Real Weapon

Benchmarks get the headlines. Price gets the migration.

Do the arithmetic on a real agentic workload. A long-horizon coding agent that consumes, say, 20M input and 4M output tokens across a run costs about $260 on Fable 5 and about $120 on K3 — and K3 actually scores higher on SWE-Marathon. That is the entire pitch in one line: more agentic stamina, less than half the bill.

Interactive: The Same Prompt, Two Models

Toggle between the two to see how the tradeoff plays out on a representative agentic-coding task.

{#snippet oldContent()}

Frontier ceiling, premium bill. Fable 5 is the stronger single-shot reasoner and edges ahead on frontier-difficulty subtasks. Adaptive reasoning effort lets it dial compute up on the hard steps. But at $10 / $50, a 40-step run that burns millions of tokens gets expensive fast, and on the long-horizon version of this task it actually scores lower than K3 (SWE-Marathon 35.0).

Pick it when: the task has genuinely frontier-hard steps, needs top-tier vision, or a single wrong answer is costly.

{/snippet}
{#snippet newContent()}

Agentic stamina, one-third the cost. K3 leads on SWE-Marathon (42.0), Terminal-Bench (88.3), and BrowseComp (91.2) — exactly the skills this loop stresses. At $3 / $15 the same 40-step run costs less than half as much, and open weights mean you can self-host to cut it further.

Pick it when: the workload is long, repetitive, high-volume, or agentic — and cost per run matters more than peak IQ.

{/snippet}

Under the Hood: The Largest Open Model Ever

K3's specs are a statement of intent as much as an engineering result.

The design choices tell you what Moonshot cares about:

Sparse mixture-of-experts — 2.8T total parameters, but only 16 of 896 experts fire per token, so inference cost tracks a far smaller active model. This is how they hit frontier quality without frontier-class compute per request.
Kimi Delta Attention — a linear-attention variant that keeps the 1M-token context affordable and flat-priced instead of ballooning cost at long context.
Native multimodal — images and video in, not bolted on.
Open weights — the whole thing is downloadable, which is why "largest open model ever" is not just a spec-sheet flex. It changes who can build on the frontier.

The honest caveat you must not skip
These benchmark numbers come from different software harnesses — K3 ran through KimiCode, Fable 5 through Claude Code and Codex. Harness quality materially affects agentic scores, so treat every cross-model gap here as directional, not surgical. Before you migrate anything, re-run your prompts through your harness. Vendor benchmarks start the conversation; they don't end it.

How to Use Kimi K3

You have four realistic paths, from zero-effort to full control.

Because the API is OpenAI-compatible, switching an existing integration is often a base-URL and model-string change:

import OpenAI from 'openai';

// Point the standard SDK at Moonshot's OpenAI-compatible endpoint.
const client = new OpenAI({
  apiKey: process.env.MOONSHOT_API_KEY,
  baseURL: 'https://api.moonshot.ai/v1'
});

const res = await client.chat.completions.create({
  model: 'kimi-k3',
  messages: [
    { role: 'user', content: 'Refactor this module and run the tests until they pass.' }
  ]
});

console.log(res.choices[0].message.content);

Two launch-day quirks to plan for
At launch K3 fixes temperature at 1.0 and offers only max thinking effort — you can't dial reasoning down for cheap, fast extraction yet. If your workload needs low-latency, low-effort calls, benchmark that specifically before assuming K3 is a universal Fable 5 replacement.

A Migration Plan That Won't Burn You

Timeline

The Verdict

The takeaway is not "Kimi K3 killed Claude Fable 5." It didn't. Fable 5 is still the sharper model on the hardest single tasks, and the intelligence index says so.

The takeaway is that the frontier is no longer a single-vendor, closed-weights club, and the price of "good enough to ship" just fell by 70%. For agent builders running long, repetitive, high-volume workloads, K3 is the most disruptive release of the summer — not because it's the smartest model, but because it makes the second-smartest model cheap and open.

Route your workloads. Send the agentic grinds to K3, keep the frontier-hard exams on Fable 5, and measure cost per successful task. That's how you turn this rivalry into a lower bill without giving up quality where it counts.

For the other side of this matchup, read the Claude Fable 5 deep-dive and the GPT-5.6 Sol / Terra / Luna guide — the third model in the frontier race K3 just crashed.

FAQ

Sources

Explore more: LLM Engineering — RAG, Fine-Tuning & Production LLMs

Originally published at umesh-malik.com

Keep reading on umesh-malik.com:

Streaming HTML Out of Order Without JavaScript (2026)

Umesh Malik — Tue, 14 Jul 2026 07:19:26 +0000

For thirty years, HTML has streamed one way: top to bottom, in the exact order the server wrote it. If the slow database query lived in your page header, everything below it waited. That constraint is finally breaking — and the fix needs zero JavaScript.

Streaming HTML out of order flips that: you send the fast parts of a page immediately, leave labeled holes where the slow parts go, and fill those holes later in the same response — in whatever order the data actually arrives. No client framework. No hydration. No fetch() on the client. The browser's own HTML parser does the reordering.

This started as a clever hack built on Declarative Shadow DOM. In 2026 it became a real standards-track feature — Declarative Partial Updates — shipping behind a flag in Chrome 148. Here's how it works, where it came from, and whether you should care yet.

TL;DR

Streaming HTML out of order means sending page sections in the order they become ready on the server, not the order they appear in the document — so a slow widget never blocks the fast content around it.
It's been possible without JavaScript since 2024 using Declarative Shadow DOM (<template shadowrootmode>) plus named <slot> elements. You stream a shell with placeholder slots, then stream the real content later and the browser slots it into place.
The 2026 evolution is Declarative Partial Updates (DPU) — a WICG proposal from Chrome's Barry Pollard and Noam Rosenthal — that removes the Shadow DOM requirement. You drop <?marker> / <?start>…<?end> processing instructions where content goes, then stream <template for="…"> elements to fill them.
The declarative half is tracked in a WHATWG HTML pull request (<template for>) authored by Philip Jägenstedt. There's also a JS half — setHTML, streamHTML, appendHTML, streamAppendHTML — for updates after initial load.
Status (mid-2026): DPU is behind chrome://flags/#enable-experimental-web-platform-features in Chrome 148. Firefox and WebKit are interested but haven't shipped. It is not production-ready — but Declarative Shadow DOM streaming is, today, cross-browser.

What "streaming HTML out of order" actually means

Streaming HTML out of order is the technique of sending an HTML document in chunks where later chunks patch content into positions defined earlier in the stream, so the browser can render each section as soon as its data is ready rather than waiting for document order.

Compare the two models directly.

In-order streaming (the last 30 years): The server writes the response top to bottom. If your page is a header, a personalized recommendations widget backed by a 900ms query, and a footer, the bytes leave the server in that order. The user stares at a header while the query runs, because the footer literally cannot be sent before the widget above it.

Out-of-order streaming: The server sends the header, then a placeholder for the widget ("Loading…"), then immediately the footer — the whole shell arrives in ~50ms. When the 900ms query finishes, the server sends the widget's real HTML as a trailing chunk, tagged to fill the placeholder. The browser swaps it in. The user saw a complete, laid-out page almost instantly, and the slow part filled in without blocking anything.

💡 Key insight: Out-of-order streaming decouples the order you author HTML from the order you can afford to compute it. Slow data stops being a layout-blocking tax on everything below it.

If that sounds like React Suspense or streaming SSR — it is the same idea. The difference is the machinery. React ships a runtime that receives the out-of-order chunks and repositions DOM nodes with JavaScript. The techniques in this post push that job down into the browser itself, so the reordering happens in the HTML parser with no script at all.

Why this matters right now

Perceived performance is the whole game. Largest Contentful Paint, Time to First Byte, and how "alive" a page feels are dominated by how fast something useful appears — not by when the slowest widget resolves. (If you're optimizing these numbers, my Core Web Vitals optimization guide covers the metrics this technique moves.)

The old workaround was to make everything fast enough to send in order, or to punt the slow parts to client-side fetch() after load — which trades a blocked render for a JavaScript bill, a loading spinner, and a layout shift. Out-of-order streaming gives you the best of both: one HTTP response, first paint at the speed of your fastest content, and no client runtime.

This lands at a moment when the whole industry is rediscovering server-driven HTML — HTMX, Astro, Qwik, React Server Components, and the Cloudflare viNext rebuild of Next.js all lean on streaming. A native, framework-free primitive for out-of-order delivery is a big deal: it's the plumbing every one of those tools has been faking in userland.

The no-JavaScript technique that started it — Declarative Shadow DOM and slots

Before any new proposal existed, developers found a way to stream out of order with zero JavaScript using two features that already shipped: Declarative Shadow DOM (DSD) and <slot>.

Here's the mechanism. A <slot> inside a shadow root is a placeholder. The browser fills it with any light-DOM child of the host element carrying a matching slot="…" attribute — and crucially, it does this whenever that child appears in the stream, even if that's much later than the slot itself.

So you stream a shell first:

<article>
  <template shadowrootmode="open">
    <h1>My page</h1>
    <slot name="reviews">
      <p>Loading reviews…</p>   <!-- fallback shown immediately -->
    </slot>
    <footer>Site footer</footer>
  </template>
</article>

The browser paints the heading, the "Loading reviews…" fallback, and the footer right away. Then — later in the same response, after your slow query resolves — you stream the real content as a light-DOM child targeting that slot:

<article>
  <!-- ...the template above already streamed... -->
  <div slot="reviews">
    <p>★★★★★ Actually great.</p>
    <p>★★★★☆ Pretty good.</p>
  </div>
</article>

The moment that <div slot="reviews"> is parsed, the browser removes the fallback and slots the reviews into place. No script ran. No event fired. The HTML parser did it. That's out-of-order streaming, and it's been Baseline across Chrome, Firefox, and Safari since early 2024.

Why DSD needed a rename to make this work
Declarative Shadow DOM first shipped in Chrome 90 (2021) using a shadowroot attribute that attached the shadow root at the template's closing tag — which defeats streaming, since you'd have to buffer the whole template. The fix, driven by Google's Mason Freed in a Chromium "Intent to Prototype", renamed it to shadowrootmode and attached the root at the opening tag. That one change is what unlocked streaming and got Firefox and WebKit on board.

The catch: this only works inside Shadow DOM. Every out-of-order region needs a custom-element host and a shadow root, you inherit shadow-DOM style scoping whether you want it or not, and it's awkward for the common case of "just patch this one <div> in my normal document." That limitation is exactly what the next proposal set out to remove.

Declarative Partial Updates — the native proposal

Declarative Partial Updates (DPU) is the web-platform proposal that turns out-of-order streaming into a first-class HTML feature with no Shadow DOM requirement. It was introduced in a Chrome for Developers article on May 19, 2026 by Barry Pollard and Noam Rosenthal, with a full WICG explainer and a WHATWG HTML pull request (#11818) for the <template for> element authored by Philip Jägenstedt.

The explainer states the problem plainly:

"Streaming of HTML existed from the early days of the web… However, it always had the following major constraints: 1. HTML content is streamed in DOM order. 2. After the initial parsing of the document, streaming is no longer enabled."

DPU attacks both constraints. Two mechanisms, two audiences.

Mechanism 1 — declarative out-of-order streaming (no JavaScript)

You mark a spot in the document with a processing instruction, then fill it later with a <template for> whose for attribute matches the marker's name.

Single-point insertion — drop content at a marker:

<div>
  <?marker name="placeholder">
</div>

<!-- ...anything else streams here... -->

<template for="placeholder">
  Here is some <em>HTML content</em>!
</template>

Temporary placeholder — show fallback content until the real thing arrives, using a <?start> / <?end> pair to define the range to replace:

<div>
  <?start name="reviews">
  Loading…
  <?end>
</div>

<!-- ...footer, other sections, all stream immediately... -->

<template for="reviews">
  Here is some <em>HTML content</em>!
</template>

When the <template for="reviews"> streams in, the browser replaces everything between <?start name="reviews"> and <?end> — the "Loading…" text — with the template's content. Notice there's no custom element and no shadow root. This patches a plain <div> in the normal document.

Multiple / chunked updates — a template can contain its own nested marker, so you can stream a list one item at a time:

<ul id="results">
  <?start name="results">
  Loading…
  <?end>
</ul>

<template for="results">
  <li>Result One</li>
  <?marker name="results">
</template>

<template for="results">
  <li>Result Two</li>
  <?marker name="results">
</template>

Each <template for="results"> appends its <li> and leaves a fresh <?marker name="results"> for the next chunk. This is how you stream search results or a chat transcript row-by-row, declaratively.

Why processing instructions instead of an attribute?
An earlier design used a contentmethod attribute, but the WICG explainer notes it "doesn't support replacing arbitrary ranges of nodes, only an element or all of its children." Processing instructions let the server define an arbitrary range — a start/end marker pair — to swap out, which matters for frameworks whose patch boundaries aren't known ahead of time.

Mechanism 2 — streaming updates after load (this half needs JavaScript)

The second half of DPU modernizes DOM insertion for content that arrives after the initial page — think live dashboards or infinite scroll. It replaces the messy innerHTML / insertAdjacentHTML landscape with a consistent, sanitizer-aware, Streams-integrated set of methods:

const el = document.querySelector('#content-to-update');

// Static: parse a string of HTML into the element
el.setHTML(html, options);
el.appendHTML(html, options);

// Streaming: pipe an HTTP response straight into the DOM
const response = await fetch('/api/content.html');
response.body
  .pipeThrough(new TextDecoderStream())
  .pipeTo(el.streamHTMLUnsafe());

There are Unsafe variants (streamHTMLUnsafe, streamAppendHTMLUnsafe) that skip sanitization, mirroring the setHTMLUnsafe naming already standardized with the HTML Sanitizer API. This half obviously uses JavaScript — but the first half, the out-of-order document streaming, does not.

Old slot technique vs Declarative Partial Updates

Both give you no-JavaScript out-of-order streaming. Here's how to choose.

…<?end>', tone: 'positive' }] },
{ label: 'Post-load streaming updates', cells: [{ text: 'Not really its job', tone: 'neutral' }, { text: 'Built in — streamHTML / streamAppendHTML', tone: 'positive' }] },
{ label: 'Production-ready in mid-2026', cells: [{ text: 'Yes', tone: 'positive' }, { text: 'No — explicitly for developer testing', tone: 'negative' }] }
]}
/>

The honest recommendation: if you need out-of-order streaming in production today, use Declarative Shadow DOM with slots. It's cross-browser, it's stable, and the Shadow DOM constraints are livable for coarse-grained regions. Reach for Declarative Partial Updates when you're prototyping the future or building a framework's rendering layer that you can gate behind feature detection and a polyfill.

How to try Declarative Partial Updates today

.", status: "done" },
{ date: "Mid-2026", title: "Chrome 148 developer testing", description: "DPU available behind chrome://flags/#enable-experimental-web-platform-features. Firefox and WebKit interested; not yet shipped.", status: "active" },
{ date: "Future", title: "Cross-browser standardization", description: "Pending WHATWG HTML + DOM spec merges and multi-implementer support before it's safe for production.", status: "upcoming" }
]}
/>

To experiment right now:

Enable the flag. Open chrome://flags/#enable-experimental-web-platform-features in Chrome 148+ and turn it on.
Stream from a real server. The effect only shows if bytes actually arrive over time — flush the shell, do your slow work, then flush the <template for>. A buffered response that sends everything at once will look correct but won't demonstrate the streaming win.
Use a polyfill for reach. The proposal ships with polyfills so you can adopt the API shape today and let it fall through to native behavior as browsers catch up.
Feature-detect, don't assume. Gate on the presence of the API (e.g. checking for Element.prototype.streamHTML) before relying on it.

Do not ship this to users yet
Both the Chrome team and the WICG explainer are explicit: Declarative Partial Updates is for developer testing and feedback. The syntax — the marker and start/end processing instructions, plus the template-for element — can still change before it's standardized, and only one browser exposes it. Prototype freely; don't put it on your production critical path.

Common mistakes and pitfalls

Most people who try out-of-order streaming trip on the same few things:

Buffering the response and expecting magic. If your framework or reverse proxy buffers the full HTML before sending, there is no "out of order" — everything arrives at once. You must flush the shell early. Check your server and any nginx/CDN buffering in front of it.
Assuming Suspense-style streaming is JavaScript-free. React's out-of-order streaming is not the same class of thing — it ships a client runtime. If your goal is "works with JS disabled," you need DSD/slots or DPU, not a framework's streaming SSR.
Fighting Shadow DOM styles. With the slot technique, your global CSS doesn't pierce the shadow root. Plan for ::part(), inherited custom properties, or accept scoped styles — don't discover this after your design looks unstyled.
Forgetting the fallback content. The whole point of <slot> children or the <?start>…<?end> range is what the user sees before the slow content arrives. Ship a real skeleton or "Loading…" state, not an empty hole that causes layout shift.
Reaching for DPU in production. It's one browser, behind a flag, with a syntax that may change. Treat it as R&D.

Best practices

Default to Declarative Shadow DOM + slots for production out-of-order streaming. It's the only cross-browser, no-JavaScript option that's actually stable in 2026.
Flush aggressively. Send the shell, headers, and any above-the-fold static content in the first chunk. The perceived-performance win is entirely about getting something painted fast.
Order your slow work by user value, not document order. Stream the widget the user actually looks at first, even if it's lower on the page.
Keep JavaScript optional, not required. Use the declarative techniques for the initial render; layer the JS streamHTML methods on top only for genuinely post-load updates.
Follow the standard, not one browser. Track the WHATWG PR and WICG explainer for syntax changes, and gate any DPU usage behind feature detection + polyfill.

A concrete example: a product page with a slow reviews panel

Picture a product page. The product info comes from cache in 20ms. The reviews come from a service that takes 700ms. Personalized recommendations take 1.2s.

In-order: first byte of anything useful waits behind the slowest thing above it. Best case, you reorder your markup so slow things sit at the bottom — but then your layout is dictated by latency, not design.

Out of order: you flush the full page shell — header, product info, empty reviews slot, empty recommendations slot, footer — in ~30ms. LCP fires on the product image almost immediately. At 700ms the reviews <template for="reviews"> streams in and fills its slot. At 1.2s the recommendations do the same. One request, no client fetch, no spinner-driven layout shift, and the page is fully interactive-looking the entire time.

That's the shape of nearly every modern page: fast core content plus a few slow, independent widgets. Out-of-order streaming is purpose-built for it — and it pairs naturally with server-first stacks like the ones I covered in the Cloudflare viNext teardown and FastAPI's native SPA support.

FAQ

/ <?start>/<?end> and .",
tag: 'Basics'
},
{
question: 'What is in HTML?',
answer: "It's the element at the center of the Declarative Partial Updates proposal (WHATWG HTML PR #11818, by Philip Jägenstedt). Its for attribute matches a processing-instruction marker earlier in the stream; when the template is parsed, the browser moves its content into the marked location — enabling out-of-order streaming without JavaScript.",
tag: 'Syntax'
},
{
question: 'Which browsers support Declarative Partial Updates?',
answer: "As of mid-2026, only Chrome 148 exposes it, behind chrome://flags/#enable-experimental-web-platform-features. Firefox and WebKit have signaled interest but have not shipped. Declarative Shadow DOM streaming, by contrast, is Baseline across all three since early 2024.",
tag: 'Support'
},
{
question: 'Is this a replacement for React Suspense or streaming SSR?',
answer: "For the reordering itself, arguably yes — it does natively what frameworks do with a client runtime. But React Suspense also coordinates hydration and interactivity, which these HTML primitives don't. Expect frameworks to build on top of DPU rather than be replaced by it.",
tag: 'Comparison'
},
{
question: 'Why does my out-of-order stream render all at once instead of progressively?',
answer: "Almost always response buffering. If your app server, framework, or a reverse proxy/CDN buffers the full response before sending, the browser receives everything at once. You must flush the shell early and ensure nothing downstream re-buffers the stream.",
tag: 'Troubleshooting'
},
{
question: 'Should I use this in production?',
answer: "Use Declarative Shadow DOM + slots in production — it's stable and cross-browser. Keep Declarative Partial Updates to prototypes and framework R&D until it standardizes and ships in more than one browser.",
tag: 'Adoption'
}
]}
/>

Conclusion

HTML's top-to-bottom, in-order delivery was a 30-year-old assumption, not a law of physics. Breaking it — sending pages in the order data is ready rather than the order it's written — is one of the most consequential quiet upgrades the web platform has shipped in a decade, and the best part is that its core doesn't cost a single kilobyte of JavaScript.

The pragmatic move today: use Declarative Shadow DOM and slots for real out-of-order streaming now, and start prototyping with Declarative Partial Updates so you're fluent when it lands. Flush your shells, kill your buffering, and stop letting your slowest widget dictate when the whole page appears.

If this was useful, read the Core Web Vitals optimization guide next — it covers the exact metrics that out-of-order streaming is designed to move.

Sources and further reading

Declarative partial updates — Chrome for Developers (Barry Pollard, Noam Rosenthal), May 19, 2026
WICG Declarative Partial Updates explainer — problem statement, goals, and full syntax
WHATWG HTML PR #11818 — <template for> for out-of-order streaming — Philip Jägenstedt
Chromium Intent to Prototype: streaming declarative shadow DOM — the shadowrootmode rename that unlocked streaming
MDN — <template> shadowrootmode — Declarative Shadow DOM reference and Baseline status

Written for umesh-malik.com — no-fluff technical writing on AI, Web Dev, and Engineering.

Originally published at umesh-malik.com

Keep reading on umesh-malik.com:

OpenAI GPT-5.6 Complete Guide: Sol, Terra, Luna Benchmarks, Pricing, and API (2026)

Umesh Malik — Sat, 11 Jul 2026 15:07:56 +0000

GPT-5.6 is OpenAI's July 2026 model family — three tiers (Sol, Terra, and Luna) that share a 1.05M-token context window and range from $1 to $30 per million tokens. Sol is the coding-and-reasoning flagship, Terra is the balanced everyday model, and Luna is the fastest and cheapest.

OpenAI released GPT-5.6 on July 9, 2026, and for the first time in the GPT-5 line the headline is not a single model. It is three: Sol, Terra, and Luna.

That naming change is the whole story. Instead of shipping one frontier model and a -pro step-up, OpenAI split the release into a capability-and-cost ladder where each rung is tuned for a different budget and workload. Sol is the flagship. Terra is the everyday workhorse. Luna is the speed-and-price play. And crucially, they all share the same 1.05M-token context window, the same February 16, 2026 knowledge cutoff, and the same platform features — so you can move up and down the ladder without rewriting your integration.

The short answer: GPT-5.6 is OpenAI's most efficiency-focused release yet. Sam Altman is on record calling the family "orders of magnitude more efficient and cost-effective than previous versions," and the coding numbers back the claim — Sol reportedly finishes agentic coding tasks with 54% better token efficiency than the previous generation. If you run models at scale, this release is less about a new capability ceiling and more about doing the same work for a fraction of the tokens.

TL;DR

GPT-5.6 launched July 9, 2026 as a three-model family: Sol (flagship), Terra (balanced), and Luna (fast and cheap).
All three share a 1.05M-token context window, 128K max output, and a February 16, 2026 knowledge cutoff.
Sol is the best coding model in the family, scoring 80 on the Artificial Analysis Coding Agent Index — about 2.8 points above the previous frontier competitor — and 91.9% on Terminal-Bench 2.1 with ultra thinking.
Sol is the first model to clear the halfway mark on Agent's Last Exam, at roughly 50.9% in code mode.
Terra lands just above the previous frontier tier on coding, and Luna outperforms the last generation's flagship while being the cheapest option.
Pricing per 1M tokens: Sol $5 / $30, Terra $2.50 / $15, Luna $1 / $6.
New ultra thinking and max reasoning modes push the ceiling on hard, long-horizon tasks.
OpenAI calls GPT-5.6 its strongest cybersecurity model yet, tuned for defensive work like threat modeling, code review, and blue teaming.
API names: gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-luna; the alias gpt-5.6 routes to Sol. Available in ChatGPT, Codex, the API, and GitHub Copilot.

What GPT-5.6 Actually Is

GPT-5.6 is not "GPT-5.5 but smarter." It is a repackaging of the frontier into three price points, and that is a more interesting decision than another benchmark bump.

Here is the mental model:

Sol — the flagship. Built for complex work across coding, knowledge work, research, cybersecurity, science, computer use, and design. This is the one you reach for when the task is genuinely hard.
Terra — the middle tier. A deliberate balance of capability, speed, and cost for everyday production work. It is the model most teams will actually run by default.
Luna — the floor. The fastest and lowest-cost member of the family, aimed at high-volume, latency-sensitive, or cost-capped workloads.

The important part is what they have in common. Every tier gets the same 1.05M context window, the same 128K max output, and the same knowledge cutoff. There is no "the cheap model also has a smaller brain for context" catch. You are trading raw reasoning depth for price and speed — not memory.

Naming note
The GPT-5.6 family uses celestial codenames — Sol (sun), Terra (earth), Luna (moon) — instead of the old -mini / -pro suffixes. It is a marketing rebrand, but it maps cleanly onto a real axis: Sol is the brightest and most expensive, Luna is the smallest and cheapest, Terra sits in between.

1. The Real Headline Is Efficiency, Not a New Ceiling

Most model launches lead with "we beat the benchmark." GPT-5.6 leads with "we beat it for less."

That is a genuine shift. The standout claim is that Sol is 54% more token-efficient on AI coding tasks than the previous generation. On head-to-head coding runs, OpenAI says Sol uses less than half the output tokens of a comparable frontier model, finishes in less than half the time, and costs about a third less to complete the same task.

For anyone paying a real API bill, that math matters more than a two-point benchmark win.

Why does this land now? Because the bottleneck for most production LLM systems in 2026 is not "the model cannot do it." It is "the model does it, but the token bill and latency make it uneconomical at scale." A model that produces the same answer with half the tokens changes which use cases are actually viable.

💡 Key insight: GPT-5.6's most important number is not a benchmark score — it is the token count it takes to reach that score. Efficiency is the feature.

2. Coding and Agents: Where Sol Actually Wins

Sol is positioned as the best coding model in the family, and the public numbers are strong.

The Terminal-Bench and Agent's Last Exam numbers are the ones agent builders should care about. They measure long-horizon, multi-step task completion — the model has to plan, run commands, read output, recover from errors, and keep going without a human babysitting each step. Clearing 50% on Agent's Last Exam is a milestone; most models still stall well before the halfway point.

For a concrete sense of how Codex-class coding behaves inside a real agent loop, I walked through building frontend UIs with Codex and Figma — the same plan-run-inspect-iterate pattern these benchmarks are trying to measure.

Where Terra and Luna land on coding
Terra performs just above the previous frontier competitor on the coding index, and Luna outperforms the previous generation's flagship while being the cheapest option in the lineup. In other words: even the budget tier of GPT-5.6 is roughly a frontier model from one generation ago.

3. Thinking Modes: Ultra and Max

GPT-5.6 introduces higher-effort reasoning modes that let you dial compute up for the hardest work.

Ultra thinking — the top reasoning setting, used to post Sol's record 91.9% on Terminal-Bench 2.1. Reserve it for genuinely hard, long-horizon problems where extra deliberation pays for itself.
Max mode — a strong high-effort tier that still hit 88.76% on the same benchmark without going all the way to ultra.

The practical rule is the same as it has always been with reasoning models: effort is a cost dial, not a free upgrade. Higher thinking modes consume more (billed) reasoning tokens and add latency. Use them where the task genuinely needs multi-step planning — agentic coding, deep research, complex analysis — and drop back to lower effort for extraction, formatting, and simple transforms.

Reasoning tokens still cost real money
As with every recent OpenAI reasoning model, the tokens spent "thinking" in ultra and max modes are billed as output and consume your context budget even though you never see them. Leave headroom, and measure incomplete responses before you cap max_output_tokens aggressively.

4. Cybersecurity: OpenAI's "Strongest Yet"

OpenAI describes GPT-5.6 as its strongest cybersecurity model to date, explicitly tuned to help with defensive security work:

threat modeling and architecture review
security-focused code review and vulnerability triage
patch generation and remediation guidance
blue-team workflows and detection engineering

This is a meaningful positioning choice. A model that is good at finding and fixing vulnerabilities is, by definition, also more capable in the offensive direction — so OpenAI pairs the capability with monitoring and access controls, and frames the sanctioned use cases around defense. If your security team has been waiting for a model strong enough to sit inside real review pipelines, Sol is the one to evaluate.

If you are thinking about agents with this much capability touching production systems, the governance questions in the agentic AI enterprise security model apply directly here.

5. The 1.05M Context Window Is Shared — but Read the Fine Print

Every GPT-5.6 tier ships with a 1,050,000-token context window and 128,000 max output tokens. That is a real capability, and the fact that even Luna gets the full window is genuinely useful.

But the same caveat from every large-context model still holds: a big window is not perfect recall. Long-context retrieval quality degrades at the far edge of the window, and giant prompts carry hidden cost and latency. Treat 1M context as a tool for broad synthesis and large working memory, not as a replacement for retrieval discipline.

Sol vs Terra vs Luna: The Decision

If you only remember one section, make it this one.

The simplest rule:

Default to Terra. It is the balanced everyday model and will be the right call for most production traffic.
Escalate to Sol when the task is genuinely hard — agentic coding, deep research, security review, or anything where a wrong answer is expensive.
Drop to Luna for high-volume, latency-sensitive, or cost-capped work where "good and fast and cheap" beats "best."

I go much deeper on this — including the break-even math and a routing strategy — in the companion post: GPT-5.6 Sol vs Terra vs Luna: which one to actually use.

Using GPT-5.6 in the API

The models are available as gpt-5.6-sol, gpt-5.6-terra, and gpt-5.6-luna, with the alias gpt-5.6 routing to Sol. Here is a minimal call:

import OpenAI from 'openai';

const client = new OpenAI();

const response = await client.responses.create({
  model: 'gpt-5.6-terra', // default workhorse; swap to sol/luna as needed
  input: 'Summarize this incident timeline and propose three remediation steps.',
  reasoning: { effort: 'medium' }
});

console.log(response.output_text);

Escalating a single hard request to Sol with a higher thinking mode is a one-line change:

const hard = await client.responses.create({
  model: 'gpt-5.6-sol',
  input: 'Audit this auth module for vulnerabilities and produce a patch.',
  reasoning: { effort: 'high' } // dial up for long-horizon, high-stakes work
});

Migration and Rollout

What GPT-5.6 Still Does Not Solve

The release is strong, but read the tradeoffs before you over-index on the launch numbers.

1. The knowledge cutoff is February 16, 2026

Impressive, but still a cutoff. For genuinely current facts you still need web search or your own retrieval layer. Do not assume the model "knows" anything after mid-February 2026.

2. A 1M window is not 1M of perfect recall

Shared context across all three tiers is great, but far-edge retrieval still degrades. Keep your retrieval discipline.

3. Higher thinking modes cost real time and money

Ultra and max deliver the record scores, but they are slower and more expensive. They are not a default; they are an escalation.

4. Cheaper does not mean free to misroute

The whole point of three tiers is routing. If you send everything to Sol out of caution, you lose the entire cost advantage of the family. If you send everything to Luna to save money, you will pay for it in quality failures. The value is in matching the tier to the task.

5. Powerful cyber capability is a double-edged surface

A model strong enough to be OpenAI's best defensive security tool is also more capable in the wrong hands. Treat security-related agent workflows as privileged, with logging, scope limits, and human confirmation.

FAQ

Final Take

The most important thing to understand about GPT-5.6 is that OpenAI stopped competing purely on the capability ceiling and started competing on capability per dollar.

Splitting the release into Sol, Terra, and Luna — all sharing the same 1.05M context and platform features — turns model selection into a routing problem instead of an all-or-nothing bet. The flagship is genuinely strong on coding and agents. But the release's real weapon is that even the cheap tier is roughly a last-generation frontier model, and the flagship finishes the same work with half the tokens.

If you run LLMs at any real scale, evaluate GPT-5.6 with a cost-per-successful-task lens, not just a leaderboard lens. Then set Terra as your default, escalate to Sol where it earns its price, and push volume to Luna. That is the whole game this release is built around.

Next, read the companion deep-dive on choosing between Sol, Terra, and Luna, and the breakdown of the ChatGPT super-app reform and Apps SDK that shipped alongside it.

Sources

Explore more: LLM Engineering — RAG, Fine-Tuning & Production LLMs

Originally published at umesh-malik.com

Keep reading on umesh-malik.com:

GPT-5.6 Sol vs Terra vs Luna: Which One Should You Actually Use? (2026)

Umesh Malik — Sat, 11 Jul 2026 15:07:56 +0000

GPT-5.6 Sol vs Terra vs Luna is the wrong way to frame the choice — the real question is not which model is best, but which one to use for this request. OpenAI shipping GPT-5.6 as three models is not a marketing gimmick. It is a routing problem handed to you.

The old question was "should I pay for the pro model?" The new question is sharper: for this specific request, what is the cheapest tier that still gets it right? Get that wrong in the expensive direction and you burn money sending trivial requests to the flagship. Get it wrong in the cheap direction and you ship quality failures to users. This post is about getting it right.

The short answer: make Terra your default, escalate to Sol only when the task is genuinely hard, and push high-volume work to Luna. That single rule will beat both "always use the best model" and "always use the cheapest" on cost-per-successful-task — which is the only metric that actually matters.

TL;DR

Default to Terra. It is half the price of Sol, keeps the full 1.05M context, and is strong enough for most production work.
Escalate to Sol for hard, agentic, or high-stakes tasks — coding agents, deep research, security review. Sol scores 80 on the Artificial Analysis Coding Agent Index and uses far fewer tokens to finish, which partly offsets its higher price.
Drop to Luna for high-volume, latency-sensitive, or cost-capped paths. At $1 / $6 it is 5× cheaper than Sol and still beats the last generation's flagship.
Sol's per-token price is 2× Terra and 5× Luna, but Sol's token efficiency means the effective gap on a completed task is smaller than the sticker price suggests.
The winning pattern is difficulty-based routing: classify the request, send it to the cheapest capable tier, and escalate on failure or low confidence.
Because all three share one API surface and context window, mixing tiers in a single app is trivial — it is a model-string change, not a rewrite.

The Pricing Math (Do This First)

Everything downstream depends on the raw numbers, so start here.

Two things jump out.

First, the gaps are clean multiples: Terra is exactly 2× cheaper than Sol, and Luna is 5× cheaper. That makes routing decisions easy to reason about — moving a request from Sol to Terra literally halves its token cost.

Second, and this is the part people miss: Sol uses fewer tokens to finish the same task. OpenAI cites roughly 54% better token efficiency on coding work, with Sol using less than half the output tokens of a comparable frontier model. So the effective cost gap between Sol and Terra on a completed task is smaller than the 2× per-token headline — sometimes much smaller on long agentic runs where Sol simply finishes in fewer steps.

💡 Key insight: Compare models on cost per successful task, not cost per token. A pricier model that finishes in half the tokens and half the retries can be cheaper in practice.

Cost Per Task, Not Cost Per Token

Here is the trap. A naive cost model says "Sol is 2× Terra, so always prefer Terra." But real workloads have retries, failures, and multi-step loops, and those wreck the naive math.

Consider an agentic coding task:

On Luna, it might take 4 attempts and still fail once, burning tokens on dead ends.
On Terra, it succeeds in 2 attempts.
On Sol, it succeeds first try, in half the tokens, with ultra thinking.

Suddenly "the expensive model" can be the cheapest path to a shipped result, because you paid once instead of three times. This is why routing beats a fixed choice: the right tier depends on how hard the request is, and you often do not know until you try.

The cheap-model tax
Sending a hard task to Luna to save money frequently costs more, because you pay for repeated failed attempts, longer loops, and eventually a human cleanup. Cheap-per-token is not cheap-per-outcome. Measure the outcome.

Coding: When Sol Actually Earns Its Price

Coding is where the tiers separate most clearly.

The practical read:

Use Sol when the coding task is open-ended and agentic — "understand this repo, plan the change, edit across files, run the tests, fix what breaks." That is where its long-horizon strength and token efficiency compound.
Use Terra for scoped, everyday engineering — a well-defined feature, a code review, a bug with a clear repro. It is the right default for most day-to-day work.
Use Luna for short, mechanical edits and high-volume operations — bulk refactors with a clear pattern, autocomplete-style suggestions, or anything where speed and price dominate.

If you want to see the agentic-coding loop that Sol is optimized for, my walkthrough of building UIs with Codex and Figma and the field notes on Claude Code auto mode in production both show what "the model keeps going without a babysitter" actually looks like in practice.

The Routing Strategy

This is the part worth stealing. Do not pick one model — pick a router.

Because all three models share the same API surface, implementing this is genuinely a model-string swap:

import OpenAI from 'openai';

const client = new OpenAI();

// A crude but effective starting router.
function pickModel(req: { difficulty: number; highStakes: boolean }) {
  if (req.highStakes || req.difficulty >= 0.8) return 'gpt-5.6-sol';
  if (req.difficulty >= 0.4) return 'gpt-5.6-terra';
  return 'gpt-5.6-luna';
}

async function run(input: string, req: { difficulty: number; highStakes: boolean }) {
  let model = pickModel(req);
  let res = await client.responses.create({ model, input });

  // Escalate one tier if the cheap answer fails your validator.
  if (!isValid(res.output_text) && model !== 'gpt-5.6-sol') {
    model = model === 'gpt-5.6-luna' ? 'gpt-5.6-terra' : 'gpt-5.6-sol';
    res = await client.responses.create({ model, input });
  }
  return res.output_text;
}

Start simple, then measure
You do not need a fancy ML router on day one. Length thresholds, task-type rules, and "escalate on validation failure" will already beat a fixed single-model choice. Add sophistication only where the logs show it pays.

GPT-5.6 Sol vs Terra vs Luna: The Decisions, Made Explicit

Migration Checklist

FAQ

Final Take

GPT-5.6 is the first OpenAI release where the smartest move is not choosing a model — it is choosing a router.

Make Terra your default, escalate to Sol when the task is genuinely hard, and push volume to Luna. Measure cost per successful task, not cost per token, and let the data move your thresholds. Do that, and you get most of the flagship's quality on the requests that need it while paying Luna-and-Terra prices for everything else.

That is the entire point of splitting the frontier into three tiers. Use it.

For the full release picture — benchmarks, cybersecurity positioning, and the API — start with the GPT-5.6 complete guide. And if you are building on the platform, the ChatGPT Apps SDK and super-app reform shipped in the same window.

Sources

Explore more: LLM Engineering — RAG, Fine-Tuning & Production LLMs

Originally published at umesh-malik.com

Keep reading on umesh-malik.com:

ChatGPT Apps SDK and the Super App Reform: How Apps in ChatGPT Work (2026)

Umesh Malik — Sat, 11 Jul 2026 15:07:24 +0000

OpenAI just turned ChatGPT into a platform — and the ChatGPT Apps SDK is how developers ship to it. The company stopped treating ChatGPT as a chatbot and started treating it as an operating system.

The 2026 reform — a "super app" redesign reportedly codenamed Aria — folds AI agents, the Codex coding tool, image generation, and a growing catalog of third-party apps directly into the main ChatGPT interface. For its ~900 million weekly users, the chat box is becoming a launcher: you ask for a playlist and Spotify renders inside the conversation, you ask for a place to stay and Booking.com shows up as an interactive card, you sketch an idea and Canva opens inline.

The thing that makes this real for developers is the Apps SDK — and the most important detail is that it is built on the Model Context Protocol (MCP). OpenAI did not invent a proprietary plugin format this time. It standardized on the same open protocol the rest of the industry is converging on, and added a UI layer on top.

The short answer: ChatGPT is now a distribution platform, and the Apps SDK is how you ship to it. If you build software, "being inside ChatGPT" is about to be a channel you have to think about — the way "being in the App Store" became one in 2008.

TL;DR

OpenAI reworked ChatGPT into a super app: agents, Codex, image generation, and third-party apps live inside the main interface, reaching ~900M weekly users.
The Apps SDK is how developers build those apps, and it is built on MCP (Model Context Protocol) — an open standard, not a closed plugin format.
An app is an MCP server that lists tools, executes tool calls, and returns widgets — web UI components ChatGPT renders inline in the conversation via a sandboxed iframe.
The UI talks to ChatGPT over a JSON-RPC bridge (postMessage) and can request inline, picture-in-picture, or fullscreen display via window.openai.requestDisplayMode.
Launch partners included Booking.com, Canva, Coursera, Figma, Expedia, Spotify, and Zillow.
Apps are available to logged-in users outside the EEA, Switzerland, and the UK on Free, Go, Plus, and Pro.
Developers submit apps through a submission portal; approved ones land in the App Directory to be discovered and shared.
Because it is MCP-based, the same app can run in other MCP Apps–compatible hosts — you build the UI once.

What the "Super App" Reform Actually Is

For years, ChatGPT was a text box with a model behind it. The reform changes the shape of the product: the conversation becomes a surface where different capabilities and apps render directly.

Concretely, the redesigned ChatGPT pulls several things into one place:

Agents that can take multi-step actions on your behalf
Codex, the coding tool, available in-line for engineering work
Image generation as a first-class capability, not a mode you switch to
Third-party apps — Spotify, Canva, Figma, Booking.com and more — that render interactive UI inside the chat

Codex and Figma sitting side by side in that same launch lineup is not a coincidence — see how to turn Figma designs into production React with Codex for that workflow in practice.

The strategic move is obvious once you name it: OpenAI wants ChatGPT to be the place you start a task, not the place you go to get text you then paste elsewhere. That is the definition of a super app — one interface that handles many jobs — and it is the same playbook WeChat ran in China a decade ago.

Why this is not 'plugins, again'
The 2023 plugin era failed partly because it was a closed, bespoke format with weak UI and weak discovery. The 2026 reform fixes all three: it is built on the open MCP standard, apps render rich inline UI, and there is a real App Directory for discovery. Same ambition, much better foundation.

Why Building on MCP Is the Real Story

The single most important technical decision in this release is that the Apps SDK builds on the Model Context Protocol.

MCP is the open standard for connecting models to external tools and data. If you have already built an MCP server — for Claude, for an internal agent, for anything — you are most of the way to a ChatGPT app. And because ChatGPT implements the same iframe-and-bridge model that the broader MCP Apps effort defines, a UI you build for ChatGPT can run in other MCP Apps–compatible hosts too.

That is a genuinely different bet than a proprietary plugin store. It means:

You build once. The tool contract and UI bridge are standardized.
You are not fully locked in. MCP is an open protocol with multiple hosts.
The ecosystem compounds. Every MCP investment across the industry now points at ChatGPT distribution too.

If you are new to MCP, start with my guide to building an MCP server and the walkthrough on deploying an MCP server on Cloudflare Workers — the same server you build there is the foundation for a ChatGPT app.

How the ChatGPT Apps SDK Works

An app in ChatGPT has two halves: an MCP server (the logic) and an optional UI component (the interface). This is the core of the ChatGPT Apps SDK — here is the anatomy.

At minimum, your MCP server implements three capabilities:

The UI half runs inside a sandboxed iframe and communicates with ChatGPT over a standard bridge — ui/* JSON-RPC messages over postMessage. Your component can read state, call your tools, and negotiate how it is displayed:

// Inside your app's UI bundle, running in the ChatGPT iframe.

// Ask ChatGPT to present the widget larger for an interactive task.
await window.openai.requestDisplayMode({ mode: 'fullscreen' }); // or 'inline' | 'pip'

// Call one of your MCP server's tools from the UI and render the result.
const result = await window.openai.callTool('search_listings', {
  location: 'Lisbon',
  checkIn: '2026-08-01'
});
renderListings(result.structuredContent);

The bridge is the whole trick
Your UI never talks to the network directly for app logic — it talks to ChatGPT over the bridge, and ChatGPT relays to your MCP server. That is what keeps apps sandboxed, consistent, and portable across MCP Apps hosts. Build the UI once; the host handles presentation.

The display modes

window.openai.requestDisplayMode lets a widget negotiate how it appears:

Building an App: The Shape of the Work

You do not need to learn a new framework. If you can write an MCP server and a small web UI, you can build a ChatGPT app.

Availability caveat
At launch, apps in ChatGPT were available to logged-in users outside the EEA, Switzerland, and the UK, on Free, Go, Plus, and Pro. If your audience is primarily European, factor that rollout gap into any launch plan — the distribution surface is not yet global.

The Opportunity — and the Risk

This is a real distribution channel, but building on someone else's platform always cuts both ways.

For brands, the mental shift is the same one search and app stores forced: users increasingly start inside an aggregator. If people begin their travel, design, or media tasks in ChatGPT, being absent from the App Directory is a discovery problem — the same way being absent from Google once was.

Readiness Checklist

The Bigger Picture: This Pairs With GPT-5.6

The super app reform did not ship in a vacuum. It arrived in the same window as the GPT-5.6 model family, and the two reinforce each other. A more capable, more efficient model makes agentic, multi-app workflows economically viable; a super app surface gives that model somewhere to do things. Cheaper tokens plus a distribution platform is how "AI agent that gets real work done" stops being a demo.

If you are building agents that call many tools, the same discipline I wrote about in the agentic AI enterprise security model applies: capability without governance is a liability, and apps inside ChatGPT are no exception.

FAQ

Final Take

The ChatGPT super app reform is OpenAI's clearest bet yet that the interface, not just the model, is the product.

By building the Apps SDK on MCP, OpenAI turned "being inside ChatGPT" into a channel you can reach with mostly standard, reusable work — and turned ChatGPT into a distribution platform sitting in front of ~900 million weekly users. That is a genuine opportunity and a genuine dependency at the same time.

The right posture is pragmatic: reuse your MCP investment, ship an app as a channel, design for portability, and keep your product's center of gravity your own. Do that, and the super app is upside without the classic platform trap.

Next, read the GPT-5.6 complete guide that shipped alongside it, and — if you are building the backend — my MCP server guide.

Sources

Explore more: AI Coding Agents & Developer Experience

Originally published at umesh-malik.com

Keep reading on umesh-malik.com:

How to Build Enterprise-Grade AI Agents for Free (MaxKB, 2026)

Umesh Malik — Tue, 07 Jul 2026 19:57:26 +0000

TL;DR

How to build enterprise-grade AI agents for free: self-host MaxKB — an open-source (GPLv3, ~22k GitHub stars) agent platform — and point it at a local model like DeepSeek or Llama via Ollama, so you pay zero API tokens and no data ever leaves your server.
"Enterprise-grade" isn't a checkbox — it's five things: answer precision, cost control, data-sovereign security, access control, and observability. Free tools can nail four of them out of the box.
The one honest catch: MaxKB's Community edition is free forever but capped (2 users, 5 apps, 50 knowledge bases). SSO, LDAP, and RBAC live in the paid Pro tier ($1,920/yr) — you can replace them yourself for $0 with more effort.
Precision comes from your RAG pipeline, not the model. Chunking, hybrid search, a reranker, and a "cite or refuse" prompt matter more than which LLM you pick.
MaxKB vs Dify vs n8n: pick MaxKB for a knowledge-grounded Q&A agent, Dify for a broad LLM app builder, n8n when the agent is one step in a bigger automation.

Most "Free AI Agent" Guides Are Lying to You

Here's how to build enterprise-grade AI agents for free in 2026: self-host the open-source platform MaxKB, point it at a local model, and you get a document-grounded, tool-using agent with $0 API cost and zero data leaving your network.

Most other "free AI agent" tutorials are either toys or bait. The toy version wires ChatGPT to a prompt and calls it an "agent." The bait version is free until step 7, when you hit a paywall, a per-token API meter, or a "contact sales" wall right as it gets useful. Neither gives you something you'd actually put in front of customers or run your internal knowledge base on.

This guide is the version I wish existed. We're going to stand up a real, enterprise-grade AI agent — one that answers from your documents with citations, takes actions through tools, runs entirely on infrastructure you control, and costs $0 in API fees — and I'll be honest about exactly where "free" stops and money starts. No hand-waving.

The vehicle is MaxKB, and by the end you'll have a working agent plus a clear-eyed view of the five things that separate a demo from something you can trust in production.

What Is an Enterprise-Grade AI Agent?

An enterprise-grade AI agent is an AI system that answers or acts on your organization's own data with measurable accuracy, keeps that data under your control, enforces who can do what, stays observable, and does all of it at a cost you can predict. "Enterprise-grade" is about trust and control — not about how big the model is.

The five-question test
Point any agent at five questions: is it accurate on our data, is the cost predictable, does our data stay ours, can we control who does what, and can we see why it answered the way it did? Five yeses is enterprise-grade. Anything less is a demo.

People throw the phrase around like it means "expensive." It doesn't. A $30/month SaaS chatbot can be less enterprise-grade than a well-configured open-source stack running on a $40 VPS. What makes an agent enterprise-grade is whether it holds up on five pillars:

Keep these five in mind. Everything below maps back to them. The good news: a free, self-hosted stack wins pillars 1, 2, 3, and 5 outright. Pillar 4 is the one place "free" gets an asterisk, and we'll deal with it head-on.

The $0 Stack: MaxKB + a Local Model

The cheapest enterprise-grade agent in 2026 is MaxKB running against a local LLM, both self-hosted on one machine. That's the whole trick. MaxKB gives you the agent platform; a local model kills the API bill; self-hosting solves data sovereignty.

MaxKB (short for Max Knowledge Brain, built by the 1Panel team) is an open-source platform for building enterprise-grade agents. Its stack is boringly solid — Vue frontend, Django backend, LangChain under the hood, and PostgreSQL + pgvector as the vector store — which means no exotic dependencies and one-command deployment.

Out of the box it gives you a full RAG pipeline (upload docs or crawl a site → automatic chunking → vectorization), a visual workflow engine, and MCP tool-use so the agent can call external tools. Crucially, it's model-agnostic — it'll talk to OpenAI, Claude, and Gemini or to local models like DeepSeek, Qwen, and Llama.

That last part is the money-saver. Point MaxKB at a local model served by Ollama and your per-token cost drops to exactly zero.

How to Build Enterprise-Grade AI Agents for Free, Step by Step

Here's the honest, end-to-end walkthrough. You need a machine with Docker installed — a laptop works for testing; for a local model you'll want at least 16GB of RAM (more if you run larger models). Every step below is free.

Step 1 — Run MaxKB

One command. This maps a data volume so your knowledge bases survive restarts:

docker run -d --name=maxkb --restart=always \
  -p 8080:8080 \
  -v ~/.maxkb:/var/lib/postgresql/data \
  -v ~/.python-packages:/opt/maxkb/app/sandbox/python-packages \
  1panel/maxkb

Open http://localhost:8080 and log in with the default credentials shown in the MaxKB docs. Change the password immediately — that's the first line of your security checklist.

Step 2 — Serve a local model for $0

Install Ollama, then pull a model. DeepSeek and Qwen punch far above their weight for RAG in 2026:

# install ollama, then:
ollama pull deepseek-r1:7b        # reasoning model, runs on modest hardware
ollama pull nomic-embed-text      # embeddings for the RAG pipeline

Ollama now serves an OpenAI-compatible endpoint at http://localhost:11434. This is the whole reason your token bill is zero — inference happens on your hardware, not someone's metered API.

Step 3 — Connect the model in MaxKB

In the MaxKB UI go to Model Settings → Add Model, choose the Ollama provider, and point it at your Ollama host. If MaxKB runs in Docker and Ollama runs on the host machine, use http://host.docker.internal:11434 as the base URL. Add both the chat model (deepseek-r1:7b) and the embedding model (nomic-embed-text). No API key required.

Precision tip
Use a dedicated embedding model (like nomic-embed-text) for the knowledge base, not your chat model. Retrieval quality — and therefore answer precision — depends far more on the embedding model than on the chat model. This is the single most common mistake people make and never diagnose.

Step 4 — Build the knowledge base

Create a knowledge base, then either upload documents or paste a URL to crawl. MaxKB handles splitting, vectorizing, and indexing into pgvector automatically. Two settings decide your precision:

Chunk size — too big and retrieval pulls in noise; too small and it loses context. Start around 500–800 tokens with overlap, then tune against real questions.
Segment cleanup — strip navigation boilerplate and repeated headers before indexing. Garbage in the index is the number-one cause of confidently wrong answers.

Step 5 — Create the agent and give it a tool

Create an application, attach your knowledge base, and set a system prompt that enforces grounding — the "cite or refuse" rule:

Answer only from the provided knowledge base.
Cite the source document for every claim.
If the answer isn't in the knowledge base, say
"I don't have that information" — never guess.

Then wire an MCP tool so the agent can act, not just answer — look up an order, create a ticket, query a database. MaxKB's function library and MCP support let you register tools the workflow can call. If you're new to MCP, start with how to build an MCP server and deploying one on Cloudflare Workers.

Step 6 — Embed it

MaxKB generates an embeddable chat widget and a REST API. Paste the widget script into any page, or call the API from your backend. Zero front-end code required — this is the "zero-coding integration" MaxKB is built around.

The Five Pillars, Judged Honestly

A running agent isn't the same as an enterprise-grade one. Let's grade the free stack against the five pillars — including where it falls short.

Pillar 1 — Effectiveness & precision

Precision comes from the retrieval layer, not the model. This is the most important sentence in this article. Teams burn weeks swapping models when their real problem is a bad chunking strategy or no reranking. RAG works by retrieving relevant chunks and forcing the model to answer from those chunks, which is what crushes hallucinations.

The precision levers, in order of impact:

Chunking — right size + overlap, boilerplate stripped.
Hybrid search — combine keyword and vector search so exact terms (part numbers, names) aren't lost to fuzzy semantics.
Reranking — reorder the top-k so the best chunk lands in the model's context, not just a relevant one.
A grounding prompt — "cite or refuse," as above.

Nail those four and a 7B local model will out-answer a frontier model with a sloppy pipeline. If you want the full theory, read building a RAG pipeline from scratch.

Pillar 2 — Cost control

This is where self-hosting quietly wins. A metered API bills you more as you succeed; a per-seat SaaS bills you more as your team grows. A self-hosted local model turns both into one flat server bill that barely moves.

The tradeoff is real and worth stating: self-hosting trades a variable money cost for a fixed operational cost — you run the server, you patch it, you own uptime. For a small internal agent that's a rounding error. At scale it's a massive saving.

Pillar 3 — Security & data sovereignty

The strongest argument for this stack. When the model runs locally and MaxKB runs on your server, no document ever touches a third-party cloud. For anyone handling PII, health, financial, or regulated data, that alone can be the difference between "allowed" and "not allowed." You're not sending your knowledge base to an API you don't control.

That's the architecture being secure. You still have to harden the deployment:

For the broader threat model of agents that take actions, see the agentic AI enterprise security model.

Pillar 4 — Access control (the honest asterisk)

Here's where free ends. MaxKB's Community edition caps you at 2 users, and SSO, LDAP, and RBAC are Pro-tier features. If you need "marketing can only see the marketing knowledge base, support leads can edit agents, everyone logs in with Okta" — that's the paid tier, or DIY work you take on yourself (an auth proxy in front, separate instances per team). I'd rather tell you that now than let you discover it at rollout.

Pillar 5 — Observability

MaxKB logs conversations and lets you inspect what was retrieved for a given answer, which is enough to debug precision and iterate on chunking. It's not a full LLM-observability suite — if you need deep tracing and eval dashboards you'll add tooling — but for "why did the agent say that?" you have what you need for free.

Where Does "Free" Actually End?

No dodging it. Here's the exact line between $0 and paid, so you can plan.

My take: start on Community. It is genuinely free and genuinely capable. Only pay when access control across multiple teams becomes a real, present need — not a hypothetical one. Most people building their first agent are nowhere near the 2-user wall.

MaxKB vs Dify vs n8n: Which Free AI Agent Platform Should You Use?

"Free AI agent platform" returns a dozen tools that do overlapping-but-different things. Here's how the honest contenders compare — and none of these are the same product.

The key insight: Ollama isn't a competitor to MaxKB — it's a component of the stack. MaxKB and Dify compete; n8n plays a different game (orchestration). Here's how I'd choose:

Common Mistakes That Kill Free Agents

FAQ

Bottom Line

Free and enterprise-grade are not opposites in 2026 — that's the myth this guide exists to kill. Self-host MaxKB, point it at a local model, invest in your retrieval pipeline, harden the box, and you have an agent that answers from your data with citations, keeps that data on your own hardware, and costs nothing per token. The only honest asterisk is multi-team access control, and you now know exactly when that bill arrives.

Start on the Community edition today. Build the narrow, useful agent — the one that answers your support questions or your internal docs — get the retrieval right, and let the results decide whether you ever need to pay.

If this was useful, read building a RAG pipeline from scratch next to push your agent's precision further, or why 77% of AI agents never reach production to make sure yours is in the 23% that do.

Sources

MaxKB — 1Panel-dev/MaxKB on GitHub (open-source, GPLv3)
MaxKB official documentation
MaxKB project site
Ollama — run local models

Written for umesh-malik.com — no-fluff technical writing on AI, Web Dev, and Engineering.

Originally published at umesh-malik.com

Keep reading on umesh-malik.com:

DEV Community: Umesh Malik

How to Increase SEO Traffic in the AI Era: 10 Techniques (2026)

TL;DR

How to Increase SEO Traffic in the AI Era, Concretely

The Ten Techniques, Ranked by Effort-to-Payoff

The Rewrite That Turns a Ranking Page Into a Cited One

Genuine vs. Spam: The Line You Don't Cross

Does AI Search Traffic Actually Convert?

The One-Page Weekly Routine

Common Mistakes That Cap Your Growth

The Bottom Line

Sources

SEO in the AI Era: The 2026 GEO Playbook for Winning AI Search Traffic

TL;DR

Your Traffic Didn't Drop Because You Got Worse at SEO

What Is GEO (Generative Engine Optimization)?

What Actually Changed — Nine Shifts, One Table

1. Query fan-out means you can't see the query anymore

2. The citation market is fragmented, and that's good news

3. Crawling exploded; referrals didn't follow

Why Bother? Because the Traffic That Survives Is Better

The GEO Playbook — Nine Moves That Actually Move Citations

The Rewrite That Doubles Your Citation Odds

How to Measure AI Traffic When Analytics Lies to You

Common Mistakes That Are Costing You Citations

Being PRO at SEO in the AI Era

The Bottom Line

Sources

Claude Opus 5: Benchmarks, Pricing & Two Breaking Changes

TL;DR

What Is Claude Opus 5?

Claude Opus 5 Benchmarks

The Real Cost Math

The Two Breaking Changes

1. Thinking is on by default — and it eats your max_tokens

2. Disabling thinking 400s at xhigh and max effort

Effort: Re-Sweep It, Don't Reuse It

What Else Is New

Opus 5 vs Fable 5 vs Sonnet 5

The Honest Ledger

The Migration Checklist

FAQ

The Bottom Line

Sources

Vercel AI SDK in Production: Streaming, Tool-Calling & the Gotchas Nobody Tells You (2026)

TL;DR

Most Vercel AI SDK tutorials lie by omission

What the Vercel AI SDK actually is

Why the Vercel AI SDK in production isn't the demo

1. Tool-calling loops that actually terminate

2. Aborting a generation without leaking cost

3. Error handling and retry UX

4. Rate limiting — the SDK won't do it for you

5. Cost you can actually attribute

Edge vs Node: pick Node by default

Common mistakes

Best practices, in order

The Vercel AI SDK vs raw fetch vs LangChain.js

The runnable example

FAQ

Conclusion

Sources

React Server Components in 2026: The Mental Model, the use client Boundary & When Not to Use Them

TL;DR

Server Components aren't experimental anymore — they're the default

What React Server Components actually are

Think in two graphs

The use client boundary

Server Components as children — the pattern that unlocks it

Streaming with Suspense

The Core Web Vitals payoff

When NOT to use Server Components

Common mistakes

Best practices, in order

Common questions, answered

FAQ

Conclusion

Sources

Build a RAG Chatbot in Next.js: Retrieval, Streaming & Citations (2026)

TL;DR

1. Thinking is on by default — and it eats your `max_tokens`

2. Disabling thinking 400s at `xhigh` and `max` effort

The `use client` boundary