DEV Community: Maksim Danilchenko

I Benchmarked My AI Carb Counter on 400 Meal Photos: 8 Models, 1 Holdout Set

Maksim Danilchenko — Thu, 08 Oct 2026 09:09:15 +0000

I build Soba, an iPhone app that looks at a photo of a meal and estimates the carbs in it. People with type 1 diabetes use numbers like that to dose insulin, so "it seems to work" wasn't a good enough answer. I ran it on 400 meal photos from two public research datasets where the carbs were weighed or recorded, and published the method, the misses and the per-meal data on the Soba accuracy page.

This post is the engineering side of that test: how the benchmark was set up, which of eight models won, and the bugs a production-path test turns up that a separate eval harness can miss. The full write-up on my blog has the per-meal-size breakdown and the comparison with published ChatGPT studies.

The setup

The photos come from Nutrition5k (Google cafeteria plates, every ingredient weighed, fixed overhead camera) and SNAPMe (95 people in the US photographing their own meals and logging them in a food record). The first is clean, controlled data. The second looks like what users actually shoot.

A few choices did most of the work:

A script with a fixed seed drew 25 meals from each quarter of the carb range, per dataset, so no part of the range ended up under-sampled by chance.
Cafeteria plates photographed within the same ten minutes often look nearly identical, so the script capped how many it took from one ten-minute window. The confidence intervals come from a bootstrap that resamples whole clusters (all photos from one SNAPMe participant, or from one ten-minute cafeteria window), because near-duplicate photos aren't independent samples.
There are two disjoint sets of 200. All error analysis and prompt work happened on the first set; the second stayed untouched until a result needed confirming.
Every photo was downscaled to 1,024 px on the long side like the app does and sent through the same server code, with the same instructions and model settings. There's no eval-only prompt.
The main metric is the share of meals within 10 g of the reference carbs, roughly one bread unit. Failed scans count as misses (there were none).

The results in one table

Setup	Meals	Within 10 g	Median error
Photo only, all meals	400	68.5%	5.2 g
Photo + text hint	400	72.8%	4.5 g
Photo only, cafeteria (Nutrition5k)	200	77.5%	3.9 g
Photo + depth data, cafeteria	200	82.0%	3.1 g
Photo only, own photos (SNAPMe)	200	59.5%	6.8 g
Photo only, meals with 60 g+ carbs	42	19.0%	25.4 g

The headline 68.5% has a 95% interval of 64.0% to 72.9%. The last row shows the main failure. In 38 of the 42 largest meals the estimate was too low, with the median estimate at 64% of the real amount. In 30 of those 38, the model judged the food to weigh at least 15% less than it did, while its carbs per 100 g were usually close.

Every meal in the test. The shaded band is ±10 g; points below the dashed line are underestimates.

The per-meal CSV is public, so you can check the headline numbers yourself with the standard library:

import csv, io, statistics, urllib.request

URL = "https://soba-app.com/accuracy/soba-carb-benchmark-2026-10.csv"
with urllib.request.urlopen(URL) as resp:
    rows = list(csv.DictReader(io.TextIOWrapper(resp, "utf-8")))

errs = [abs(float(r["soba_total_carbs_g"]) - float(r["reference_total_carbs_g"])) for r in rows]
print(f"within 10 g: {sum(e <= 10 for e in errs) / len(errs):.1%}")
print(f"median error: {statistics.median(errs):.1f} g")

within 10 g: 68.5%
median error: 5.2 g

Eight models, and why the winner didn't ship

I ran eight models on the first 200 meals with Soba's prompt, one run each through OpenRouter, with the request settings Soba had before October 2026. These were separate runs, so production's numbers differ slightly from the table above:

Model	Within 10 g	Median time
Gemini 3.7 Flash	73.0%	3.7 s
Gemini 3.8 Flash	71.5%	4.8 s
Gemini 3.5 Flash-Lite	69.0%	2.1 s
Gemini 3.6 Flash (production)	67.5%	3.3 s
GPT-6 Luna	65.0%	6.9 s
GPT-6 Luna Pro	63.0%	11.7 s
GPT-6.1 Sol	58.0%	13.1 s
Claude Sonnet 5.5	53.0%	6.0 s

Gemini 3.7 Flash beat production by 5.5 points. That's enough of a lead to make a switch look obvious. But when you pick the best of eight on the same 200 samples, part of the winner's lead is luck, so I ran both models on the untouched second set. They tied at 70.5%, and 3.7 Flash costs more. Production stayed on Gemini 3.6 Flash.

Two more rows need a footnote. Claude Sonnet 5.5 returned an empty food item for 41 of the 200 photos through the provider OpenRouter routed it to; on the other 159 photos it scored 66.7%, right in the pack. If the harness had silently dropped empty answers, that model would have looked competitive, and the 41 empty answers would never have shown up. And GPT-6.1 Sol overestimated carbs by 10 g on average, a bias you'd only notice by looking at signed errors.

Letting Gemini 3.6 Flash reason longer raised the share within 10 g by 4.1 points over all 400 meals, but the share within 5 g and the number of errors above 30 g stayed the same, and each scan took 6.6 seconds instead of 3.5. More thinking pushed medium errors under the line and did nothing for the large ones. I kept the fast setting.

Bugs that only a full run finds

The test pushed 3,614 requests through production code. Two problems fell out that normal traffic never surfaced:

The fallback model couldn't run. Soba sent a temperature parameter that the OpenAI backup didn't accept, and the request allowed only providers that support every parameter in it (OpenRouter's require_parameters option). So the OpenAI backup would have failed on every scan the moment Gemini went down. Removing the parameter moved the main model from 68.9% to 68.5%, inside run-to-run noise. The backups are now Gemini 3.7 Flash and GPT-6 Luna.
Repetition loops. In 9 of 3,614 requests (0.25%), Gemini repeated one phrase over and over, and the scan took 35 to 87 seconds. A cap on answer length now cuts those off early.

A spot check of ten photos wouldn't have caught either one. The loops only appeared in the latency tail of 3,614 requests, and the dead fallback only when the backup itself was called.

What extra inputs added

Since the dominant error was portion weight, I tested what extra inputs add:

A text hint naming the foods (copied from the dataset records, no amounts) lifted the share within 10 g from 68.5% to 72.8%. It helped most on ambiguous photos, like a coffee with creamer that Soba had read as caramel sauce (162 g estimated without the hint, 5.5 g with it, 2.7 g in the food record). It didn't fix the main failure: large meals were still underestimated in 37 of 42 cases with a hint. The gain was also lopsided between the two sets (8.0 points vs 0.5), and real users won't type as precisely, so I treat it as an upper bound.
Depth data lets the app tell the model how wide the frame is in centimeters, on iPhones with LiDAR or multiple rear cameras. Replayed from Nutrition5k's depth recordings on its 200 cafeteria photos, it lifted the share within 10 g from 77.5% to 82.0%. Large meals were still underestimated in 12 of 13 cases, and angled photos haven't been measured yet.
The weight editor in the UI is the fallback for big plates. The app shows a weight range with each estimate and recalculates when the user enters a real weight. The real plate weight fell inside the shown range on 97 of 200 weighed plates, about half, so treat the range as a guide; typing in a weight from a kitchen scale is the reliable fix.

Takeaways for anyone benchmarking a vision LLM

Keep a holdout set and don't touch it while tuning. Mine stopped a model switch that the first-round numbers made look obvious.
Break results down by the size of what you're estimating. The 68.5% average hid a 19% result on the biggest meals.
Count failures as misses, and look at signed errors. Empty answers and systematic bias both disappear in a careless average.
Run the eval through the production code path. Mine found a dead fallback and a 0.25% latency tail that a separate eval harness could easily have missed.
Find the bottleneck before you shop for models. Here it was portion weight on large plates, and the inputs that carry scale (depth, a weight the user enters) go after it more directly than a model swap. Even depth left 12 of 13 large meals underestimated.

The full method, the limits (US-only meals, overhead camera on one dataset, self-reported references on the other) and the 400-row CSV are on the Soba accuracy page. The blog version has the meal-size table and how the numbers compare with published ChatGPT-4o and ChatGPT-5 studies.

Soba isn't a medical device. Its numbers are estimates: check them against labels and a scale, and follow your diabetes care plan.

LLM Food Recognition in Production: What Shipping soba Taught Me

Maksim Danilchenko — Mon, 05 Oct 2026 08:48:52 +0000

TL;DR

I spent the last few months building soba, an iOS app that photographs a meal and returns carbs, glycemic index, and portion weights for people who count carbohydrates. The recognition backend went through one model migration, one full prompt rewrite, and a stack of validation code. Three things carried almost all of the improvement: picking the model with a benchmark instead of vibes, forcing strict JSON Schema through OpenRouter, and rewriting the prompt around scene scale rather than food identity. Model choice turned out to be the smallest lever of the three. Every model I tested misjudges portion weight by 25–35%, so the UX for correcting weight beats another round of model shopping.

Fact	Value	Source	Verified
Best nutrition MAPE on DiningBench	24.45% — Gemini-3-Pro (reported)	arXiv:2604.10425	Aug 29, 2026
GPT-5 nutrition MAPE on same bench	32.17% (reported)	arXiv:2604.10425	Aug 29, 2026
gemini-3.1-pro-preview price	$2 / $12 per 1M tokens	OpenRouter	Aug 29, 2026
gemini-3.6-flash price	$1.50 / $7.50 per 1M tokens	OpenRouter	Aug 29, 2026
Cost per scan (my prod)	$0.005–0.01	soba server billing	Aug 29, 2026
Scan latency (my prod)	2.6–6 s	soba server logs	Aug 29, 2026

What soba does

The app is aimed at people with type 1 diabetes: carb counting several times a day, every day. You point the camera at a plate, and a few seconds later you get a list of items with weights, net carbs per 100 g, and a glycemic index estimate. There's a barcode path for packaged food too. Under the hood it's a Go server (internal/recognize/) that sends the photo to a multimodal LLM and turns the answer into something a diabetic can act on.

That framing shaped every technical decision. A person dosing insulin needs grams of digestible carbohydrate, an honest error bar, and no surprises when the model has a bad day.

Picking the model: benchmark first, then a live A/B

I picked the starting model off a benchmark instead of marketing pages. DiningBench evaluates 29 models on 3,021 dishes for exactly this task. Gemini-3-Pro led both fine-grained dish classification and nutrition estimation, with a 24.45% nutrition MAPE against 32.17% for GPT-5. That settled the starting lineup: soba launched on gemini-3.1-pro-preview.

Then I A/B-tested gemini-3.6-flash against the pro model on the live recognition path. Same prompt, same photos. Quality differences never rose above the noise floor of gram estimates. What did change:

Metric	gemini-3.1-pro-preview	gemini-3.6-flash
Cost per scan	~$0.012	~$0.005
Latency	7.2–8.3 s	2.6–6 s
Pricing (per 1M tokens)	$2 in / $12 out	$1.50 in / $7.50 out
Lifecycle status	preview, can vanish	stable

A scan that costs 58% less and returns in as little as a third of the time, at the same practical accuracy, was an easy call. The lifecycle column mattered more than it looks, too: preview models get deprecated whenever the vendor decides, and a health-adjacent app can't afford to wake up to a dead endpoint.

All of these models (pro, flash, GPT) miss portion weight by 25–35%. That error dwarfs any gap between frontier models on food identity. The accuracy budget went into scale anchoring in the prompt and a weight-editing UX in the app, and I stopped following model releases with any anxiety — the opposite of what I found comparing frontier models for coding, where the gaps are real. A multi-dataset study in Nutrients reports the same pattern, with prompt design moving nutrition accuracy as much as a model swap, and portion size has been the weak link in photo-based estimation since long before LLMs.

One more model decision that saves real money: not every subtask deserves the full recognition call. Estimating the GI of a barcode-scanned product is a lookup against well-known food knowledge, so it runs on the fast flash model at temperature 0 and costs a rounding error compared to a photo scan.

The API call, hardened

Everything goes through OpenRouter, and the request body carries most of the production hardening. Here's the shape of it, abbreviated:

{
  "models": ["google/gemini-3.6-flash", "google/gemini-3.5-flash", "openai/gpt-5.1"],
  "messages": [
    { "role": "system", "content": "<recognition prompt, below>" },
    { "role": "user", "content": [
      { "type": "text", "text": "Item names and notes language: German.\nThe photo is approximately 31 cm wide at the distance of the food.\nAnalyze this photo." },
      { "type": "image_url", "image_url": { "url": "data:image/jpeg;base64,..." } }
    ]}
  ],
  "temperature": 0.2,
  "reasoning": { "effort": "low" },
  "response_format": {
    "type": "json_schema",
    "json_schema": { "name": "meal", "strict": true, "schema": { "...": "..." } }
  },
  "provider": { "require_parameters": true, "data_collection": "deny" },
  "usage": { "include": true },
  "user": "u_9f3ab2c1"
}

Line by line, the parts that earn their place:

The models array is the whole failover story. Primary is gemini-3.6-flash; if Google has an incident, OpenRouter walks down to gemini-3.5-flash and then across vendors to openai/gpt-5.1. One line of config replaces a routing layer I never had to write, and a provider outage doesn't reach the user.

Structured outputs with strict: true mean the response is guaranteed to match my JSON Schema, so parsing is literally json.Unmarshal and nothing else. Strict mode has sharp edges: every field must be listed in required, and every object needs additionalProperties: false. Miss either and the request is rejected before any tokens flow. The abbreviated schema for a meal item looks like this:

{
  "type": "object",
  "additionalProperties": false,
  "required": ["name", "name_en", "grams", "grams_min", "grams_max",
               "weight_basis", "carbs", "fiber", "protein", "fat",
               "kcal", "gi", "gi_confidence"],
  "properties": {
    "grams":      { "type": "number" },
    "grams_min":  { "type": "number" },
    "grams_max":  { "type": "number" },
    "weight_basis": { "type": "string" },
    "gi":         { "type": ["number", "null"] },
    "gi_confidence": { "type": "string", "enum": ["high", "medium", "low"] }
  }
}

provider.require_parameters: true closes the gap the schema leaves open: it makes OpenRouter route only to endpoints that actually support json_schema. Without it, your request can land on a provider that silently ignores the schema and returns prose, and you'll waste an evening blaming the model for a routing problem.

reasoning.effort: "low" is the setting I'd check first on any cost surprise. Gemini 3.x and GPT-5.x burn hidden reasoning tokens that bill as output, and identifying rice doesn't need chain-of-thought (my post on efficient LLM reasoning covers why reasoning-token budgets got out of hand in the first place). Left at default, this setting alone can double the scan price.

Food photos are health data, so provider.data_collection: "deny" opts them out of provider-side training.

usage.include: true asks OpenRouter to attach the request's exact cost to the response (newer API revisions include the usage block by default). Every scan's cost is written to the database, and the server has a daily budget kill switch sitting on top of those numbers. When something goes wrong (a retry loop, a pricing change upstream) the stop-loss trips before the invoice does.

And user carries a stable pseudonymous ID so OpenRouter's abuse detection bans an abusive account rather than my whole API key.

On the client side, photos get downsampled to 1024 px via ImageIO before upload. They land at 100–250 KB of JPEG. I tested bigger inputs and they bought nothing: gram accuracy is limited by scene scale ambiguity, and more pixels don't resolve that.

The prompt: scale first, honesty second

The first version of my prompt read like everyone's first food prompt ("identify the foods in this photo and estimate nutrition") and produced confident nonsense. The rewrite starts from the observation the A/B forced on me. The dominant error is portion weight. Weight is unguessable without scene scale. So the system prompt now opens by assigning the model a job and a method, in that order. Verbatim:

You are a clinical nutrition analyst helping a person with type 1 diabetes count carbohydrates. Analyze the food photo and identify every distinct edible item (dishes, single products, packaged foods).

Then the scale hierarchy, the part that did the most work:

Work scale-first:
1) Anchor the physical scale. If the user message includes a depth-camera width measurement, it is the primary reference. Otherwise infer scale from objects of known size: dinner plate 24-26 cm, salad/dessert plate 20 cm, bowl 15-18 cm, fork/knife 19-21 cm, tablespoon 20 cm, glass 7-8 cm wide, credit card 8.6 cm, adult palm ~9 cm wide.
2) From that anchor estimate each item's covered area and layer height, then weight = volume x typical density of the cooked food.
3) For countable pieces (eggs, bread slices, syrniki, cookies, dumplings, sushi) prefer count x typical unit weight (egg ~55 g, bread slice 30-40 g, pelmeni ~12 g, syrnik ~60 g) over volume reasoning.

On devices with LiDAR or stereo depth, the app measures the physical width of the frame at the food's distance and passes it in. Everything else in the hierarchy is a fallback for when hardware can't help. And rule 3 exists because counting beats volume math. "2 eggs" lands within a few grams; "a pile of scrambled egg" doesn't.

Two fields in the schema exist purely to keep the model honest. weight_basis forces it to name the anchor it actually used: "plate 26 cm, rice layer 2 cm". That disciplines the estimate and makes failures debuggable, because you can see exactly where the reasoning slipped. And instead of one confident number, the model returns a grams_min/grams_max bracket with instructions to earn its width:

Keep it tight where the weight is nearly certain (label or package weight, countable pieces: 2 eggs, 3 slices) and widen it where it is not (a mound of rice seen from above, an opaque bowl, a sauce, a partly hidden portion, no scale anchor in frame). ... Do not default to a fixed percentage and do not collapse the bracket to hide uncertainty — an interval that turns out to contain the true weight is worth more to this user than a confident single number.

The rest of the prompt is domain policy, compressed. Effort goes where errors hurt: weight accuracy is prioritized for carb-dense items (bread, rice, pasta, sweets), while butter and greens get rough estimates by design. Nutrition comes back per 100 g in the as-served state (cooked pasta, not dry), with carbs meaning net digestible carbs and fiber in its own field. GI is nullable with an explicit ban on inventing numbers, plus a gi_confidence grade. If a nutrition label is visible in the frame, transcription beats vision, with fiber subtracted from carbs for US-style labels. And the model never gives insulin dosing advice; that line is not negotiable in a diabetes app.

The system prompt sets the method; a dynamically assembled user message carries each scan's variables. Four moving parts, composed per request:

Item names and notes language: German.
The photo is approximately 31 cm wide at the distance of the food.
User hint: this is buckwheat, not rice.
Analyze this photo.

Language first (without it, a German grocery shelf comes back with English item names), then the depth measurement when hardware provides one, then a free-text hint that the prompt instructs the model to trust over its own eyes. When there's no photo at all, the same pipeline accepts a text description of the meal ("two syrniki and a bowl of soup") and estimates typical portions from words alone.

Validation catches what the schema can't

JSON Schema only guarantees the shape of the response; a perfectly well-formed answer can still claim 150 g of carbs per 100 g of product. The Go server runs its own validator after unmarshal: carbs ≤ 100 g/100 g, macro sum ≤ 105 g/100 g, GI within [0, 110], and calories cross-checked against the Atwater formula:

atwater := 4*it.Carbs + 4*it.Protein + 9*it.Fat + 2*it.Fiber
if it.Kcal > 0 && math.Abs(it.Kcal-atwater)/atwater > 0.25 {
    it.Kcal = math.Round(atwater) // silently recompute — don't fail the scan
}
if it.GramsMin > it.GramsMax {
    it.GramsMin, it.GramsMax = it.GramsMax, it.GramsMin
}

The validator repairs instead of rejecting wherever it can. An inverted weight bracket gets swapped. A grams value outside its own bracket gets clamped. A meaningless bracket narrower than a gram gets zeroed out. The user paid for this scan with a real LLM call, so it never fails over a decorative field.

Violations are tiered by severity. Hard nonsense like carbs at 150 g/100 g triggers one retry with feedback: the model receives Your previous answer was invalid: <list of violations>. Return corrected JSON and a second chance. In practice one feedback round is enough: a paid scan almost never dies on a bad first answer. Cosmetic issues, like a word where the emoji field should be, just get cleaned up server-side without spending tokens on them.

There's plumbing around the edges that earns its keep, too. Client retries are idempotent via an X-Scan-ID UUID that lives for the scan's whole lifecycle, so a network error after the paid LLM call can't bill twice. Local development runs on MOCK_LLM=1, a deterministic mock that mirrors the real model's heuristics, so tests and UI iterations burn zero tokens. This kind of contract-first setup will be familiar if you've done structured document extraction; the failure modes rhyme even when the domain doesn't. And the broader discipline of packing exactly the right variables into each request — nothing more — is the same instinct I described in context engineering.

FAQ

Which LLM is best for food recognition?

On DiningBench (3,021 dishes, 29 models), Gemini-3-Pro leads nutrition estimation with 24.45% MAPE versus 32.17% for GPT-5. In my production A/B, though, gemini-3.6-flash matched the pro model within the noise of gram estimates at 58% lower cost — for this task, the flash tier is the sweet spot.

How accurate are LLMs at estimating food weight from a photo?

Portion weight is where the pipeline is weakest: every model I tested lands 25–35% off. A depth-camera scale reference, known-size objects in frame, and counting pieces instead of estimating volume all narrow the error, but an editable weight in the UI is still the most reliable fix.

How do you get strict JSON from Gemini through OpenRouter?

Use response_format with json_schema and strict: true, put every field in required, set additionalProperties: false on each object, and add provider.require_parameters: true so the request only routes to endpoints that honor schemas. The response then parses with a plain json.Unmarshal.

Can an LLM read nutrition labels from a photo?

Yes, and it should — my prompt makes a visible label override visual guessing entirely, the same way transcription beats inference in document OCR pipelines. The one trap is US-style labels counting total carbs: the prompt has to spell out subtracting fiber to get net carbs, or packaged foods come back inflated.

Sources

DiningBench — arXiv:2604.10425 — the 29-model food recognition benchmark that picked my starting model
soba on the App Store — the app all of this ships in
OpenRouter structured outputs docs — json_schema, strict mode, and require_parameters
Prompt engineering and model selection for LLM-based nutritional estimation — Nutrients — reports prompt design moving accuracy on par with model choice
Calorie estimation from photos, crowdsourcing study — JMIR — portion-size error predates LLMs by a decade

Bottom line

The recognition stack in soba got better three times: when a benchmark replaced my gut feeling, when strict schemas replaced "please return JSON", and when the prompt started arguing about centimeters instead of cuisine. It never got noticeably better from a model upgrade alone. If you're building anything that turns photos into numbers a person will act on, spend your week on scale anchors, honest intervals, and repair-first validation. The model is the least interesting part of the pipeline, which is exactly how you want it.

Faster Than Dijkstra: The New Shortest-Path Algorithm, Explained

Maksim Danilchenko — Thu, 13 Aug 2026 08:45:38 +0000

TL;DR

For 65 years, Dijkstra's algorithm was the fastest known way to compute single-source shortest paths, and most of us assumed it was optimal. In 2025, a team led by Ran Duan at Tsinghua University proved it isn't: their deterministic algorithm runs in O(m log^(2/3) n) time and beats Dijkstra on sparse directed graphs. A February 2026 follow-up pushed the bound even lower. The catch, and it's a big one: on real hardware, plain Dijkstra is still reported to be 3–4× faster, and it'll stay that way for any graph you'll ever load into memory.

The barrier that stood since 1959

Edsger Dijkstra published his shortest-path algorithm in 1959. You give it a weighted graph and a starting vertex; it hands back the shortest distance to every other vertex. It has been in every algorithms textbook, every routing stack, and every whiteboard interview since.

The version we actually reason about today pairs Dijkstra's greedy strategy with a Fibonacci heap, a result from Fredman and Tarjan in 1984 that gets you O(m + n log n) time for m edges and n vertices. On a dense graph, where m is close to n², the m term dominates and the algorithm is basically linear in the input. On a sparse graph, where each vertex has a handful of edges and m is roughly proportional to n, the n log n piece takes over. That log-n factor is the tax, and for decades it looked like a tax you had to pay.

Here is why the tax exists. Dijkstra settles vertices in strictly increasing order of distance: closest first, then next-closest, and so on. To always pull out the current minimum, it keeps its frontier in a priority queue and pays O(log n) per extraction. Do that n times and you have effectively sorted all n vertices by distance. Sorting n comparable things has a known lower bound of Ω(n log n), so as long as your algorithm produces a fully sorted output, you can't dip below it. Researchers called this the sorting barrier, and the open question was whether shortest paths actually require it, or whether the sorting was just an accident of how Dijkstra happens to work.

I wanted to feel the barrier before reading how it fell, so I dropped a textbook Dijkstra into a file and timed it. Here is the whole thing:

import heapq

def dijkstra(graph, source):
    dist = {v: float('inf') for v in graph}
    dist[source] = 0
    pq = [(0, source)]          # the heap IS the sorting barrier
    settled = set()
    while pq:
        d, u = heapq.heappop(pq)
        if u in settled:
            continue
        settled.add(u)
        for v, w in graph[u]:
            if d + w < dist[v]:
                dist[v] = d + w
                heapq.heappush(pq, (dist[v], v))
    return dist

g = {
    'A': [('B', 4), ('C', 1)],
    'B': [('D', 1)],
    'C': [('B', 2), ('D', 5)],
    'D': [('E', 3)],
    'E': [],
}
print("distances from A:", dijkstra(g, 'A'))

Running it prints the shortest distance from A to everything else:

distances from A: {'A': 0, 'B': 3, 'C': 1, 'D': 4, 'E': 7}

The direct edge A→B costs 4, but the detour A→C→B costs 1 + 2 = 3, so the algorithm picks 3. Standard stuff. Then I generated a random sparse graph (average degree 3, so m ≈ 3n) and let the same function loose on it:

n=  100,000  m=  299,999  dijkstra:  1.11s
n=1,000,000  m=2,999,999  dijkstra: 16.92s

Ten times the vertices, roughly fifteen times the wall-clock. That super-linear creep is the log n factor showing up in a real profiler (pure Python inflates the constants, so read these as shape rather than a benchmark). Every one of those seconds is the heap keeping the frontier in perfect sorted order. The Duan result asks a sharp question: what if it didn't have to?

What Duan and his co-authors actually did

The paper is Breaking the Sorting Barrier for Directed Single-Source Shortest Paths by Ran Duan, Jiayi Mao, Xiao Mao, Xinkai Shu, and Longhui Yin. It won a Best Paper Award at STOC 2025, the top theory conference, held in Prague that June. The headline result: a deterministic O(m log^(2/3) n)-time algorithm for single-source shortest paths on directed graphs with non-negative real weights, in the comparison-addition model. On sparse graphs that's asymptotically below O(m + n log n), so it's the first algorithm to prove Dijkstra isn't optimal for this problem.

The trick is to stop insisting on a fully sorted frontier. You don't need to know the exact order in which the next thousand vertices settle. You only need to keep finding correct shortest distances. So instead of one global priority queue, the algorithm runs a recursive routine the authors call BMSSP, for Bounded Multi-Source Shortest Path. Each call is handed a distance bound B and told: find every shortest path shorter than B, starting from a set of sources rather than a single one. It splits that job into smaller bounded sub-problems and recurses.

One blogger summed up the shape of it in a line I liked: "Dijkstra is to BMSSP as binary-tree sort is to merge sort." Dijkstra grows one ordered structure incrementally; BMSSP divides, conquers, and merges bounded pieces, and never pays for a total order it won't use.

The move that keeps the whole thing cheap is frontier reduction. Before recursing, a step called FindPivots runs a few rounds of Bellman-Ford-style relaxation out from the current frontier, just a handful of layers rather than the full graph. Most vertices it touches turn out to hang off a small number of "pivot" vertices that sit at the root of large shortest-path trees. The algorithm keeps only those pivots and drops the rest from the active set, shrinking the frontier by a factor of roughly k each time. A smaller frontier means fewer things to track, and fewer things to sort. Sorting was the whole tax. Stack about log^(2/3) n of these recursion levels on top of each other and the log-n cost collapses to log^(2/3) n.

That Bellman-Ford cameo is the satisfying part. Bellman-Ford is the other shortest-path algorithm everyone learns: slower, brute-force, relaxing every edge over and over. On its own it loses to Dijkstra badly. Bolted on as a frontier-pruning pre-pass, it's exactly the tool that lets you skip the sorting Dijkstra can't. The winning design fuses the two classics rather than replacing either.

The 2026 sequel went further

Barriers, once cracked, tend to keep cracking. In February 2026 an overlapping group (Ran Duan, Xiao Mao, Xinkai Shu, and Longhui Yin) posted A Faster Directed Single-Source Shortest Path Algorithm. Its full running time is O(m√(log n) + √(mn log n log log n)), which on sparse graphs simplifies to O(m √(log n · log log n)) and drops the exponent on the log from 2/3 toward 1/2. Same problem, same deterministic setting, tighter analysis and a refined recursion. Less than a year after the first paper, the frontier had already moved.

Algorithm	Result year	Time on sparse directed graphs	Beats Dijkstra?
Dijkstra + Fibonacci heap	1984 (Fredman–Tarjan)	O(m + n log n)	— (the baseline)
Duan, Mao, Mao, Shu, Yin (BMSSP)	2025	O(m log^(2/3) n)	Yes, asymptotically
Duan, Mao, Shu, Yin	2026	O(m √(log n · log log n))	Yes, further still

Neither result is a small tweak. Both are genuine progress on a problem that had not moved in decades, and the theory community treats them as landmark work. If you study algorithms, this is the most exciting thing to happen to shortest paths in a long time.

Where the speedup disappears in practice

Now the honest column of the ledger. "Faster than Dijkstra" is true in the asymptotic, worst-case, big-O sense, and that sense hides enormous constant factors.

A November 2025 paper by Lucas Castro, Thailsson Clementino, and Rosiane de Freitas, Implementation and Brief Experimental Analysis of the Duan et al. (2025) Algorithm, did the unglamorous work of actually coding it up and racing it against Dijkstra. They tested sparse random graphs, grids, and U.S. road networks up to 10 million vertices. Their finding is blunt: Dijkstra's algorithm stays 3 to 4 times faster in every scenario they tried. The recursion, the pivot bookkeeping, and the Bellman-Ford passes carry so much overhead that the asymptotic win never gets a chance to pay off. By their estimate, a graph would need to exceed 10^67 vertices before the worst-case bound overtakes Dijkstra in practice. There are about 10^80 atoms in the observable universe. You're never going to hit that input size.

None of this diminishes the theory. A worst-case upper bound is a statement about what is possible, and lowering it redraws what the next decade of research takes for granted. Practical speedups have a long history of arriving years after the asymptotic breakthrough that made them thinkable. Someone finds the idea that removes the constants, and a "useless" bound becomes a real library. The 2025 result opened that door. Whether anyone walks a fast version through it is a separate, open problem.

What this changes for my own code

Concretely, for the work I actually ship: nothing, and I think that is the right takeaway rather than a disappointing one. When I need shortest paths, I still reach for Dijkstra with a binary heap, or A* when I have a decent distance heuristic, or contraction hierarchies when I'm precomputing a road network and can afford the setup. Those are still the fastest tools for graphs that fit on a machine.

What did change is how I read a complexity result. Sitting next to my own timing numbers, a million-node graph chewing up 16.9 seconds with most of it inside the heap, the BMSSP paper is a clean demonstration that a bound you've quietly accepted as a law can just be a habit. The sorting barrier felt like a law of nature. It turned out to be a quirk of one particular algorithm, and the problem itself never demanded it. Worth keeping in mind the next time some O(n log n) in your own stack feels permanent. It might only be waiting for the right recursion.

If you want to reproduce the feel of it, the Dijkstra snippet above runs as-is on any Python 3, and the two papers below are both readable if you're comfortable with graph proofs. Start with the 2025 one; the introduction alone explains the sorting barrier better than most textbooks. For a broader tour of how algorithmic lower bounds have been falling lately, I wrote up the four color theorem dropping to O(n log n), another 2026 result where a barrier everyone treated as settled quietly gave way. And if your interest in "doing less work" is more about models than graphs, the same instinct drives efficient LLM reasoning and sparse attention, where the trick is likewise skipping computation you were told you needed.

FAQ

Is there an algorithm faster than Dijkstra?

Yes, as of 2025. The Duan et al. algorithm runs in O(m log^(2/3) n) time on directed graphs with non-negative weights, which is asymptotically faster than Dijkstra's O(m + n log n) on sparse graphs. A 2026 follow-up is faster still. In everyday practice, though, well-tuned Dijkstra remains 3–4× quicker on any real-world graph, because the new algorithm's constant factors are huge.

What is the new shortest path algorithm called?

The paper doesn't brand it with a catchy name, but its core recursive routine is BMSSP, short for Bounded Multi-Source Shortest Path. People usually refer to it as "the Duan et al. algorithm" or "the algorithm that broke the sorting barrier."

How does the new shortest-path algorithm work?

It avoids fully sorting the frontier. Dijkstra settles vertices in exact distance order using a priority queue, which forces an Ω(n log n) sorting cost. BMSSP instead solves many bounded shortest-path sub-problems recursively, and before each recursion a FindPivots step runs a few Bellman-Ford relaxation rounds to shrink the frontier down to a small set of "pivot" vertices. Fewer vertices to track means less sorting, and the log-n factor drops to log^(2/3) n.

Who discovered the new shortest path algorithm?

A team led by Ran Duan at Tsinghua University, with co-authors Jiayi Mao, Xiao Mao, Xinkai Shu, and Longhui Yin. They presented it at STOC 2025 in Prague, where it won a Best Paper Award.

Should I replace Dijkstra with it in production?

No. For graphs that fit in memory, which is essentially all of them, Dijkstra with a good heap, A*, or contraction hierarchies will beat it comfortably. The new algorithm is a statement about worst-case complexity, and it won't drop into your stack as a speedup. Treat it as proof the barrier is breakable rather than as your next routing library.

Sources

Breaking the Sorting Barrier for Directed Single-Source Shortest Paths — arXiv:2504.17033 — the original STOC 2025 Best Paper this post summarizes
Breaking the Sorting Barrier — ACM STOC 2025 proceedings — the peer-reviewed conference version
A Faster Directed Single-Source Shortest Path Algorithm — arXiv:2602.07868 — the February 2026 follow-up improving the bound
Implementation and Brief Experimental Analysis of the Duan et al. (2025) Algorithm — arXiv:2511.03007 — the benchmark study finding Dijkstra 3–4× faster in practice

Bottom line

Dijkstra's 65-year reign as the provably fastest way to find shortest paths is over, at least on paper. The Duan team showed the sorting barrier was never a law of the problem, only a feature of one very good algorithm, and they did it with a clever hybrid of Dijkstra and the humble Bellman-Ford. That's a beautiful result and it deserves the award it got. It also won't touch your production routing code for years, if ever, and the researchers who benchmarked it are the first to say so. Keep shipping Dijkstra. Just know, now, that it was never the finish line.

Go CLI Frameworks in 2026: Cobra vs urfave/cli vs Bubble Tea

Maksim Danilchenko — Mon, 27 Jul 2026 08:44:25 +0000

TL;DR

For subcommands, flags, and shell completions, the shape of kubectl or gh, reach for Cobra. For a single command with a handful of flags and the least ceremony possible, urfave/cli v3 gets you there with half the boilerplate. Bubble Tea plays a different game: it draws interactive terminal UIs (menus, spinners, forms), and you usually bolt it onto a Cobra command rather than pick between them. I built the same deploy tool three times to figure out where the lines actually fall, and the verdict depends more on which problem you have than which library is objectively best.

Why I rebuilt the same CLI three times

Two weeks ago I got tired of guessing. I maintain a small internal deploy tool at work, the kind of thing that reads a config, asks which environment you mean, and shells out to a few commands. (I walked through building a single-purpose CLI from scratch in Python in an earlier uv tutorial; this is the Go counterpart.) It started as a 40-line flag package script and had grown warts. So I rewrote it three times in one afternoon: once with Cobra, once with urfave/cli v3, once as a Bubble Tea TUI. Same behavior, three implementations, so I could compare boilerplate, binary size, and how each one felt under my fingers instead of trusting a table someone else made.

That exercise settled a few things I had been repeating without checking. urfave/cli v3 is a genuinely different library from the v2 most tutorials still show. Bubble Tea v2 quietly broke its own API in ways that will bite you if you copy an old snippet. And Cobra, the one everyone calls heavyweight, wrote itself faster than I expected because cobra-cli scaffolds the whole tree. This post is what I learned, with the code I actually ran.

If you want the wider Go tooling context, I keep a running Modern Go in 2026 guide that this piece slots into.

The split: parsers versus painters

The single most common mistake I see in "best Go CLI library" threads is treating these three as interchangeable, when two of them parse arguments and one of them paints screens.

Cobra and urfave/cli both answer the same question: given mytool deploy --env prod service-a, how do I route to the right code with the right values? They give you commands, flags, arguments, help text, and completions.

Bubble Tea answers a different question: how do I draw a live, keyboard-driven interface in the terminal (a selectable list, a progress bar, a text form) that redraws as the user types? It owns the screen rather than the argument vector. When people say "Cobra vs Bubble Tea," what they usually want is "Cobra for the command tree, Bubble Tea for the interactive bits," and the two compose cleanly.

Go's single static binary and fast startup are a big part of why so many CLIs get written in it, the same traits that keep it in the fight with Rust for systems work. For rough popularity: Cobra sits near 44k GitHub stars, urfave/cli near 24k, and Bubble Tea near 44k, reported at the time of writing and moving around, so treat them as signals rather than precision.

Feature comparison

Capability	Cobra	urfave/cli v3	Bubble Tea
Subcommand trees	Yes, deep nesting	Yes, flatter feel	N/A
Flag parsing	pflag (POSIX)	Built-in	N/A
Auto help / usage	Yes	Yes	Manual
Shell completions	bash/zsh/fish/pwsh	bash/zsh/fish	N/A
Config integration	Viper (same author)	Manual / altsrc	N/A
Interactive UI	No	No	Yes (core purpose)
Boilerplate for hello-world	Medium	Low	Higher
Scaffolding tool	`cobra-cli`	None	None
Latest line at writing	v1.10.x	v3.10.x	v2.x

That last row is easy to skim past. Both urfave/cli and Bubble Tea shipped major versions that changed their APIs, and a lot of the tutorials Google surfaces are still on the old ones.

Cobra: the one that scales

Cobra is what the big tools use. kubectl, gh, hugo, and a long list of infrastructure binaries route their commands through Cobra. The reason is nested subcommands: git remote add origin ... is three levels deep, and Cobra models that tree natively with per-command flags, help, and completions generated for free.

Here is the greeting command from my deploy tool, cut down to what runs:

package main

import (
    "fmt"
    "os"

    "github.com/spf13/cobra"
)

func main() {
    var verbose bool

    root := &cobra.Command{
        Use:   "greet [name]",
        Short: "Greet someone by name",
        Args:  cobra.ExactArgs(1),
        Run: func(cmd *cobra.Command, args []string) {
            if verbose {
                fmt.Fprintln(os.Stderr, "verbose mode on")
            }
            fmt.Printf("Hello, %s!\n", args[0])
        },
    }
    root.Flags().BoolVarP(&verbose, "verbose", "v", false, "enable verbose output")

    if err := root.Execute(); err != nil {
        os.Exit(1)
    }
}

Run it and the plumbing is already there:

$ go run . greet Ada --verbose
verbose mode on
Hello, Ada!

$ go run . --help
Usage:
  greet [name] [flags]

Flags:
  -h, --help      help for greet
  -v, --verbose   enable verbose output

The --help output, the flag validation, the arg count check (cobra.ExactArgs(1)): none of that is code I wrote. For a single command it feels like a lot of struct scaffolding. The payoff shows up at the third or fourth subcommand, when you add deploy, rollback, and status as children of the root and each one carries its own flags without you rewiring anything.

The other half of the Cobra story is Viper, from the same author. Viper layers config files, enviroment variables, and flags into one precedence chain, so --env on the command line overrides DEPLOY_ENV overrides config.yaml. If your tool has real configuration, that pairing is the reason to stay on Cobra. I lean on it whenever a tool needs to read a config file, which is most of them.

Where Cobra frustrates me: the empty-Run-function ritual for parent commands, and the fact that you scaffold a lot before the first useful line runs. The cobra-cli init command papers over that by writing the whole tree for you, and honestly it's the fastest way to start a serious multi-command tool in Go today.

urfave/cli v3: less ceremony, same job

urfave/cli is the library I reach for when the tool is one command with a few flags and I don't want to think about a command tree. v3 is the current line, and it's worth being explicit that it broke from v2: the action signature now takes a context.Context and a *cli.Command, and you run with cmd.Run(ctx, os.Args). Copy a v2 snippet into a v3 project and it won't compile, and the error messages aren't obvious. That single gotcha cost me ten minutes.

The same greeting, in v3:

package main

import (
    "context"
    "fmt"
    "os"

    "github.com/urfave/cli/v3"
)

func main() {
    cmd := &cli.Command{
        Name:  "greet",
        Usage: "greet someone by name",
        Flags: []cli.Flag{
            &cli.BoolFlag{Name: "verbose", Aliases: []string{"v"}},
        },
        Action: func(ctx context.Context, cmd *cli.Command) error {
            name := cmd.Args().First()
            if name == "" {
                return fmt.Errorf("name is required")
            }
            if cmd.Bool("verbose") {
                fmt.Fprintln(os.Stderr, "verbose mode on")
            }
            fmt.Printf("Hello, %s!\n", name)
            return nil
        },
    }

    if err := cmd.Run(context.Background(), os.Args); err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
}

Two things I like here over Cobra. The action returns an error, so failure handling is a normal Go return instead of an os.Exit buried in a closure. And there's no Args: cobra.ExactArgs(1) vocabulary to learn. You pull arguments off cmd.Args() and validate them yourself, which for a small tool is one line and reads plainly.

The tradeoff is that everything Cobra gives you automatically becomes your job at scale. Nested commands exist in urfave/cli via a Commands slice, but the ergonomics get thinner the deeper you nest, and there's no cobra-cli equivalent to scaffold them. My rule of thumb after this exercise: one-to-three commands, urfave/cli; a real tree with shared config, Cobra.

For a worked example of a single-purpose Go tool built end to end, I walked through one in the Go AI agent tutorial, and the argument-handling shape there is exactly the urfave/cli sweet spot.

Bubble Tea: when a flag isn't enough

Some tools shouldn't answer a question with a flag. "Which environment?" is nicer as an arrow-key menu than as --env prod, especially for a tool humans run interactively. That kind of prompt is what Bubble Tea is for. It implements the Elm architecture: a Model holding state, an Update that folds messages into new state, and a View that renders state to a string. It redraws on every keystroke.

Here is the environment picker from my deploy tool. This targets the classic Bubble Tea API (see the version note right after; v2 changes some of this):

package main

import (
    "fmt"
    "os"

    tea "github.com/charmbracelet/bubbletea"
)

type model struct {
    choices  []string
    cursor   int
    selected string
}

func (m model) Init() tea.Cmd { return nil }

func (m model) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
    if key, ok := msg.(tea.KeyMsg); ok {
        switch key.String() {
        case "up", "k":
            if m.cursor > 0 {
                m.cursor--
            }
        case "down", "j":
            if m.cursor < len(m.choices)-1 {
                m.cursor++
            }
        case "enter":
            m.selected = m.choices[m.cursor]
            return m, tea.Quit
        case "q", "ctrl+c":
            return m, tea.Quit
        }
    }
    return m, nil
}

func (m model) View() string {
    s := "Pick an environment:\n\n"
    for i, choice := range m.choices {
        cursor := " "
        if m.cursor == i {
            cursor = ">"
        }
        s += fmt.Sprintf("%s %s\n", cursor, choice)
    }
    return s + "\n(↑/↓ move · enter select · q quit)\n"
}

func main() {
    m := model{choices: []string{"dev", "staging", "production"}}
    final, err := tea.NewProgram(m).Run()
    if err != nil {
        fmt.Println("error:", err)
        os.Exit(1)
    }
    if fm, ok := final.(model); ok && fm.selected != "" {
        fmt.Printf("Deploying to %s\n", fm.selected)
    }
}

Running it gives you a live menu:

Pick an environment:

  dev
> staging
  production

(↑/↓ move · enter select · q quit)

Bubble Tea rarely travels alone. Lip Gloss styles the strings (colors, borders, padding), Bubbles ships pre-built components (text inputs, spinners, tables, paginators), and Huh wraps the whole thing into ready-made forms if you just want a questionnaire. That family is Charm, and it's why so many recent Go TUIs share a look.

What changed in Bubble Tea v2

If you're starting fresh in 2026, know that Bubble Tea v2 is a breaking release, and copying the snippet above verbatim into a v2 project won't fully work. The changes that bite:

Key messages now split into tea.KeyPressMsg and tea.KeyReleaseMsg. You can still match both with the tea.KeyMsg interface, but key.Type/key.Runes became key.Code/key.Text, and modifiers moved into key.Mod.
The big one: View() no longer returns a plain string. It returns a tea.View struct that declares content, cursor, alt-screen mode, colors, and more, which is the change most likely to break an old View() string method.
Imports moved to the vanity path charm.land/bubbletea/v2.

The mental model (model, update, view, messages) is identical. The signatures are what shifted. I kept the example on the v1 API because it's the version the vast majority of existing tutorials and libraries still target, and it runs today. When you upgrade, the migration is mechanical, just tedious.

Combining Cobra and Bubble Tea

This is the setup I shipped, and it's the answer to most "which one" arguments: use both. Cobra owns the command tree and flags; a Bubble Tea program launches inside a command's Run when the tool needs interaction. Falling back to a flag when one is supplied keeps it scriptable.

var deployCmd = &cobra.Command{
    Use:   "deploy",
    Short: "Deploy a service to an environment",
    RunE: func(cmd *cobra.Command, args []string) error {
        env, _ := cmd.Flags().GetString("env")
        if env == "" {
            // No flag given: ask interactively with Bubble Tea.
            m := model{choices: []string{"dev", "staging", "production"}}
            final, err := tea.NewProgram(m).Run()
            if err != nil {
                return err
            }
            env = final.(model).selected
        }
        fmt.Printf("Deploying to %s\n", env)
        return nil
    },
}

deploy --env prod runs non-interactively in CI; a bare deploy opens the menu for a human. You get Cobra's completions and help for free, and Bubble Tea only for the part that benefits from it. That split is why I stopped thinking of these as competitors.

Decision matrix

Your situation	Reach for
Multi-command tool, shared config, completions	Cobra (+ Viper)
One or two commands, minimal deps, quick	urfave/cli v3
Interactive menu, form, dashboard, or spinner	Bubble Tea (+ Lip Gloss / Bubbles)
Scriptable in CI and friendly for humans	Cobra + Bubble Tea together
A throwaway script, one flag	stdlib `flag` (skip all three)

The flag row isn't a joke. The standard library parses flags fine, and for a 50-line utility that nobody else runs, pulling in a framework is overhead you'll feel in build time and binary size for no benefit. I reach for a framework when the tool grows a second command or a real user other than me.

Getting started

# Cobra (+ scaffolding tool)
go get github.com/spf13/cobra@latest
go install github.com/spf13/cobra-cli@latest
cobra-cli init mytool

# urfave/cli v3
go get github.com/urfave/cli/v3@latest

# Bubble Tea + Charm libraries
go get github.com/charmbracelet/bubbletea@latest
go get github.com/charmbracelet/lipgloss@latest
go get github.com/charmbracelet/bubbles@latest

For anything Cobra-based, run cobra-cli init first and let it write the tree. It's the single biggest time-saver in this whole space. Then add commands with cobra-cli add deploy. If you're on Bubble Tea, start from an official example in their repo rather than a blog snippet, because the v2 API churn means half the snippets online are subtly stale.

FAQ

Is Cobra or urfave/cli better?

Neither wins in the abstract. Cobra is better for tools with deep subcommand trees and configuration (it powers kubectl and gh for a reason). urfave/cli v3 is better for small tools where you want less boilerplate and a normal error-returning action. Pick by the shape of your command tree rather than by popularity.

Is Bubble Tea a replacement for Cobra?

No. Bubble Tea draws interactive terminal interfaces; Cobra parses commands and flags. They solve different problems and compose well: Cobra for the command structure, Bubble Tea launched inside a command for the interactive parts.

What CLI library does the GitHub CLI use?

gh is built on Cobra, as are kubectl, hugo, and many other large Go tools. The draw is native nested subcommands, generated help, and shell completions across bash, zsh, fish, and PowerShell.

Do I need Cobra to use Bubble Tea?

No. Bubble Tea runs standalone as tea.NewProgram(model).Run(). You only pair it with Cobra when you also need a command tree and flags around the interactive UI. For a single interactive screen, Bubble Tea alone is enough.

What is the best Go CLI framework in 2026?

For most non-trivial tools, Cobra, because of subcommands, Viper config, and completions. For small single-purpose tools, urfave/cli v3. For interactive UIs, Bubble Tea. There's no one winner. The right answer depends on whether you're parsing commands, keeping it minimal, or drawing a screen.

Sources

Cobra GitHub repository and releases: the framework behind kubectl and gh, current v1.10.x line
urfave/cli GitHub repository: v3 releases and migration notes from v2
Bubble Tea GitHub repository and releases: Elm-architecture TUI framework, v2 breaking changes
Bubble Tea v2.0.0 release notes: the key-message split, tea.View return type, and import-path change
Charm: Lip Gloss, Bubbles, and Huh, the styling and component layer around Bubble Tea

Bottom line

The useful question is what your tool actually does, not which library wins a benchmark. If it parses commands, the choice is Cobra versus urfave/cli, and it comes down to how deep your subcommand tree goes and whether you want Viper config: Cobra for depth, urfave/cli for lightness. If it draws a screen, reach for Bubble Tea, which sits on top of whichever parser you chose rather than competing with it. My deploy tool ended up Cobra plus Bubble Tea, and rebuilding it three times was the fastest way I've found to stop guessing and just pick.

Kimi K3 Review: The 2.8T Open Model That Beats Claude on Paper

Maksim Danilchenko — Wed, 22 Jul 2026 08:45:26 +0000

TL;DR

Kimi K3 is the largest open-weight model anyone has shipped: 2.8 trillion parameters, a million-token context, and a price that undercuts Claude by roughly 3x. On the leaderboards it beats Claude Fable 5 at frontend coding and lands third overall on GDPval, which is genuinely impressive for weights you can download. After a week of feeding it real Go and Python work, the picture got messier. It's slow, it invents APIs that don't exist more often than any frontier model I've used this year, and most of its headline numbers are still Moonshot's own. Worth running for cost-sensitive batch work; I would not hand it an unsupervised agent loop yet.

Why I bothered testing this one

I review a lot of models and most open-weight releases don't survive contact with a real repo. DeepSeek V4 Pro and GLM-5.2 were the two that did. Both were cheap, competent, and good enough that I kept them in rotation for grunt work. When Moonshot dropped Kimi K3 on July 16 and the first benchmark screenshots showed it ahead of Claude Fable 5, I moved it to the top of my testing queue.

I ran K3 through the API (OpenRouter forwards to Moonshot's single hosted endpoint right now) for about a week: a Go service refactor, a pandas-to-Polars migration, a batch of PR reviews, and the usual pile of "explain this stack trace" prompts. I paid for the tokens myself, so the cost math below is real. What follows is what the leaderboards don't tell you: where a 2.8T open model actually helps, and where it quietly wastes your afternoon.

One caveat up front, and it's a big one: at the time of writing, the open weights aren't public yet. Moonshot says they land July 27, ten days after launch, under a modified MIT-style license. Everything in this review is based on the hosted API. If the weight release slips or the license has teeth, half the appeal evaporates.

What Kimi K3 actually is

Kimi K3 is a Mixture-of-Experts model with about 2.8 trillion total parameters. Moonshot reports 896 experts with 16 active per token, which is why a model this size can serve at Sonnet-ish prices: you only pay to run a sliver of it on each forward pass. The context window is 1,048,576 tokens, the same 1M ballpark the Gemini line reaches, and it's tuned hard for long-horizon coding and agent loops rather than chat.

At launch there are two variants: K3 Max for interactive chat and agent tasks, and K3 Swarm Max for large-scale parallel processing. Most people will only touch K3 Max. Swarm Max is aimed at the "fan out 200 requests and merge" workloads that batch pipelines and eval harnesses generate.

The "largest open-weight model ever" line is doing a lot of marketing work, but it's accurate. Nothing else with downloadable weights comes close to 2.8T. Whether you can run it locally is a separate question I'll get to.

Getting it running

The API is a plain OpenAI-compatible endpoint, so if you already have code hitting GPT or Claude, swapping in K3 is a two-line change. Here's the exact call I used to smoke-test it:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://openrouter.ai/api/v1",
    api_key=os.environ["OPENROUTER_API_KEY"],
)

resp = client.chat.completions.create(
    model="moonshotai/kimi-k3",
    messages=[
        {"role": "system", "content": "You are a senior Go engineer. Be terse."},
        {"role": "user", "content": "Write a worker pool in Go 1.26 that "
                                    "processes a channel of jobs with N workers "
                                    "and returns the first error via errgroup."},
    ],
    temperature=0.2,
)

print(resp.choices[0].message.content)
print("tokens:", resp.usage.total_tokens)

Output on my run (trimmed):

package main

import (
    "context"
    "golang.org/x/sync/errgroup"
)

func Process(ctx context.Context, jobs <-chan Job, n int) error {
    g, ctx := errgroup.WithContext(ctx)
    for i := 0; i < n; i++ {
        g.Go(func() error {
            for {
                select {
                case <-ctx.Done():
                    return ctx.Err()
                case job, ok := <-jobs:
                    if !ok {
                        return nil
                    }
                    if err := job.Do(ctx); err != nil {
                        return err
                    }
                }
            }
        })
    }
    return g.Wait()
}

tokens: 512

That's correct, idiomatic, and closes over the loop variable safely (Go 1.22+ semantics). No complaints on the code itself. The complaint is what the timer showed: that response took noticeably longer than the same prompt to Fable 5 or GPT-5.6. Which brings me to the thing the leaderboards bury.

The speed problem is real

Every hands-on account I've seen, including mine, lands on the same finding: K3 is slow. The New Stack ran a coding comparison and clocked it at roughly 4x slower than Claude Fable 5 on equivalent tasks. My informal timings weren't that clean, but "meaningfully slower" is not in dispute.

For an interactive agent loop, 4x slower is disqualifying. When I'm iterating with a coding agent, latency is the whole experience. A tool call that takes six seconds instead of two turns a ten-minute session into forty. I stopped using K3 for anything conversational within a day.

Where the slowness stops mattering is batch work. If you're running a nightly job that summarizes 5,000 support tickets or generates first-draft docstrings across a monorepo, nobody's watching the clock. There, K3's price does the talking. That split defines the whole model: great for offline, painful for interactive.

The benchmarks, with the asterisks attached

K3 earned its headlines on the benchmarks. As VentureBeat's launch coverage laid out, these are the numbers being quoted, and I'm labeling every one of them as reported, because most trace back to Moonshot's own eval runs or single-leaderboard placements rather than broad independent replication.

Benchmark	Kimi K3 (reported)	For reference	Placement
Frontend Code Arena	1,679	~48 pts ahead of Claude Fable 5	1st
GDPval-AA v2	~1,668	Fable 5 Max ~1,760 · GPT-5.6 Sol Max ~1,748 · Opus 4.8 1,600	3rd
AA-Briefcase	~1,547	Fable 5 Max ~1,583 · GPT-5.6 Sol Max 1,495	2nd
SWE-bench Verified	~65–67%	Same tier as top OpenAI/Anthropic models	—

Two things stand out. First, beating Claude Fable 5 on the Frontend Code Arena is a real result. Arena runs are human-preference votes on built UIs, and that one leaderboard is independent of Moonshot. Second, on GDPval it lands behind the top closed models but ahead of Opus 4.8, which is the fair "third place, and that's still very good for open weights" framing.

The SWE-bench figure needs the most salt. "65–67% in scaffold-assisted settings" means with a coding harness doing retries and test feedback around it. Bare-model SWE-bench numbers are always lower, and Moonshot hasn't published a clean methodology at the time of writing. Treat it as "strong tier," not a precise ranking against a specific competitor.

Kimi K3 vs Claude vs GPT: the head-to-head

Most people searching for K3 are really asking one question: should I use this instead of Claude or GPT? Here's the head-to-head that reflects a week of use plus the public data.

	Kimi K3	Claude Fable 5	GPT-5.6
Weights	Open (July 27)	Closed	Closed
Context	1M	1M	~400K (est.)
Input price / 1M	$3 ($0.30 cache-hit)	~$10	Premium
Output price / 1M	$15	~$50	Premium
Speed	Slow	Fast	Fast
Agentic reliability	Uneven	Best in class	Very good
Best for	Cheap batch, self-hosting later	Interactive coding, agents	General reasoning

K3 and Fable 5 prices are published/reported rates; GPT-5.6 figures are approximate.

On raw coding quality for a single well-scoped task, K3 is close enough to Fable 5 that you would not reliably tell them apart in a blind read. The New Stack's summary of "same results, one-third the cost, four times slower" matches my experience almost exactly: Fable 5 runs about $10 in and $50 out per million tokens, so K3's $3/$15 really is the cheaper column. If your bottleneck is API spend and not wall-clock time, that trade is a bargain.

The gap opens up in agent loops. Fable 5 and GPT-5.6 recover from their own mistakes; they read a failing test, form a theory, and fix it. K3 is more likely to double down on a wrong assumption, which is a known failure mode I dug into in the piece on multi-agent error cascades. If you want the current best-in-class agent, the Claude Fable 5 review still holds. K3 wins on price; Fable 5 wins on recovering from its own mistakes. Anyone doing head-to-heads through the Anthropic API will feel that difference within a session.

Where it falls apart: hallucination

This is the part that kept K3 out of my daily rotation. Independent testing has put its hallucination rate around 51%, up from a reported ~39% on its predecessor. I don't have a way to reproduce that exact figure, but the qualitative pattern showed up fast.

Twice in a week, K3 confidently used APIs that don't exist. In one pandas migration it wrote pd.read_parquet(path, engine="polars") as though pandas ships a Polars backend — it doesn't. In a Go task it referenced a stdlib helper that isn't in the standard library at all. Both times the surrouding code was clean and plausible, which is exactly what makes it dangerous: the wrong line reads as confidently as the right ones. A less careful reviewer merges it.

For a human-in-the-loop workflow where you read every diff, this is an annoyance you can catch. For an autonomous agent that writes, runs, and commits without a person checking each step, a coin-flip hallucination rate is a hard stop. The benchmark wins and this number live in tension, and you can't take one seriously without the other.

Pricing and how to run it

Through the API, K3 costs $3 per million input tokens on a cache miss, $0.30 per million on a cache hit, and $15 per million output tokens. That's Sonnet-class pricing for a model posting near-frontier coding scores, which is the entire pitch.

Local is the harder story. The weights drop July 27, but 2.8T parameters is not something you run on a workstation. Even at MoE's reduced active-parameter cost, you need serious multi-GPU hardware or a quantized community build to load it at all. "Kimi K3 on Ollama" is one of the top autocompletes for the model, and the real answer for now is: not on consumer hardware, and not on day one. Watch the Hugging Face repo and the quantization community; a usable GGUF will show up eventually, but it won't be quick. If you want a model you can actually run on a Mac today, Apfel is a far more realistic pick. Different weight class, but it fits in memory.

Who should use it, who should skip

Use Kimi K3 if: you run large offline batch jobs where cost dominates and latency doesn't, you want to hedge against closed-model lock-in, or you plan to self-host once the weights and a viable serving stack exist. The price-per-quality on non-interactive work is the best I've measured this quarter.

Skip it if: you want a snappy interactive coding assistant (Fable 5 or GPT-5.6, no contest), you're building an unsupervised agent where a hallucination becomes a committed bug, or you need a model you can run locally this week. For most solo developers writing code interactively, the slowness alone rules it out.

FAQ

Is Kimi K3 open source?

The weights are open; the training data and full pipeline are not. Moonshot says the weights ship July 27, 2026 under a modified MIT-style license (the same approach as Kimi K2.6), which makes K3 open-weight rather than fully open-source. You can download and run the model, but you can't fully reproduce it.

Is Kimi K3 better than Claude?

On some coding leaderboards, yes: it reportedly tops the Frontend Code Arena ahead of Claude Fable 5. In day-to-day use it's competitive on single-task code quality but slower and more prone to hallucination, so Claude remains the stronger pick for interactive and agentic work.

How much does Kimi K3 cost?

API pricing is $3 per million input tokens ($0.30 on cache hits) and $15 per million output tokens, roughly a third of what comparable closed models charge for similar coding quality.

Can you run Kimi K3 on Ollama or locally?

Not on consumer hardware. At 2.8T total parameters it needs substantial multi-GPU infrastructure even as a Mixture-of-Experts model. Quantized community builds may make smaller-scale local runs possible after the July 27 weight release, but there's nothing practical for a laptop at launch.

How many parameters does Kimi K3 have?

About 2.8 trillion total parameters in a Mixture-of-Experts design, with roughly 896 experts and 16 active per token, so only a small fraction runs on any given forward pass, which is what keeps inference costs down.

Sources

Kimi K3 on OpenRouter — official API pricing and hosting details
China's Moonshot AI releases Kimi K3, the largest open-source model ever — VentureBeat — launch coverage, specs, and benchmark placements
Moonshot releases 2.8-trillion-parameter Kimi K3 — Tom's Hardware — architecture and Frontend Code Arena result
Claude Fable 5 vs Kimi K3: same results, one-third the cost, 4x slower — The New Stack — independent coding comparison, latency, and pricing
Kimi K3 — DataCamp — model overview, variants, and open-weights timeline
Kimi K3 beat Claude and GPT at coding — why can't you fully trust it? — Memeburn — reported hallucination-rate testing

Bottom line

Kimi K3 is the most impressive open-weight release of the year and, for a specific job, a genuine bargain. If your workload is offline, cost-bound, and human-reviewed, download it the moment the weights land and point your batch jobs at it. But the leaderboard story and the daily-driver story are two different reviews. On the benchmarks it beats Claude; at my keyboard it was slow and it lied to me twice in a week. I'll keep it for batch grunt work and reach for Fable 5 the moment a task actually needs to be right.

Go's JSON v2 (encoding/json/v2): Faster, Stricter, and What Breaks

Maksim Danilchenko — Sun, 19 Jul 2026 08:41:43 +0000

TL;DR

Go's new encoding/json/v2 is the first real rewrite of the standard library's JSON handling in fifteen years. It's a reported 2–10x faster at unmarshaling, cleans up a pile of long-standing footguns, and moves configuration from magic struct tags into explicit function options. The catch: it changes several v1 defaults you may be quietly relying on — nil slices now encode as [] instead of null, duplicate object keys are rejected, and field matching is case-sensitive. As of Go 1.26 it lives behind GOEXPERIMENT=jsonv2, with the plan to make it the default in Go 1.27. This is a practical walkthrough of turning it on, the changes that will bite you, and how to migrate without breaking your API contracts.

Why a second JSON package at all

encoding/json shipped with Go 1.0 in 2012 and barely changed since. That stability paid off for a long time, then slowly turned into a tax. The v1 package carried a set of decisions that looked reasonable in 2011 and aged into daily annoyances: it silently accepts duplicate object keys, replaces invalid UTF-8 with the Unicode replacement character instead of erroring, and marshals a nil slice as null rather than an empty array. Each of those has bitten real production code, including mine.

The one that cost me an afternoon was the nil-slice-as-null behavior. I had a Go service returning a list of items to a TypeScript frontend. When the list was empty, the backend sometimes sent "items": null and sometimes "items": [], depending on whether a slice had been initialized or left nil somewhere upstream. The frontend called .map() on the response and blew up only on the null path. It was an intermittent, hard-to-reproduce crash that came down to a Go zero-value quirk. I fixed it back then with a make([]Item, 0) and a mental note. v2 fixes it at the source.

That is the spirit of the whole package. The official experimental announcement frames it as fixing "15 years of accumulated sharp edges" while keeping the API recognizable. If you've written Go, json.Marshal and json.Unmarshal still work the way you expect. They just have saner defaults and a much faster engine underneath.

If you're tracking where the language is heading more broadly, this fits the same evolutionary arc as Go 1.26's other standard-library changes: fewer surprises, more explicit control.

What's actually in the box

json/v2 is not one package but two, split along a clean line:

encoding/json/jsontext handles syntax: the raw grammar of JSON. It gives you a streaming Encoder and Decoder, plus Value and Token types, and it doesn't touch reflection at all. Think of it as the tokenizer layer.
encoding/json/v2 handles semantics: mapping Go values to and from JSON. This is where Marshal, Unmarshal, and the struct-tag machinery live.

That separation pays off in performance-sensitive code, because you can drop down to jsontext and stream tokens without paying for reflection when you don't need it. For everyday use you'll import v2 and never think about the split.

Turning it on

On Go 1.25 or 1.26, the packages are hidden unless you build with the experiment flag. There's no go get and no third-party module. It's already in your toolchain, just gated.

# One-off run
GOEXPERIMENT=jsonv2 go run main.go

# Persist it for a session
export GOEXPERIMENT=jsonv2
go build ./...
go test ./...

Once the flag is set, the import paths become visible:

// main.go — run with: GOEXPERIMENT=jsonv2 go run main.go
package main

import (
    "fmt"

    json "encoding/json/v2"
)

type User struct {
    ID    int      `json:"id"`
    Name  string   `json:"name"`
    Roles []string `json:"roles"`
}

func main() {
    u := User{ID: 7, Name: "Ada"} // Roles left nil on purpose
    b, err := json.Marshal(u)
    if err != nil {
        panic(err)
    }
    fmt.Println(string(b))
}

Run it and you get:

{"id":7,"name":"Ada","roles":[]}

Under v1, that same struct marshals to "roles":null. That single difference is the one most likely to change bytes on the wire in your service, so it's worth internalizing before anything else.

If you can't move off an older Go toolchain yet, the same code is available today as a standalone module, github.com/go-json-experiment/json, which is the upstream where v2 was developed. The import path differs but the API is the same, so it's a fair way to experiment on Go 1.23 or 1.24 before your production toolchain catches up.

The breaking changes, ranked by how likely they are to bite

Here's the full set of default changes, most disruptive first. Every one of them is controllable. v2 keeps the old behavior available as an option; it's just no longer the default.

Behavior	v1 default	v2 default	How to restore v1
Nil slice	`null`	`[]`	`json.FormatNilSliceAsNull(true)`
Nil map	`null`	`{}`	`json.FormatNilMapAsNull(true)`
Field matching	case-insensitive	case-sensitive	`json.MatchCaseInsensitiveNames(true)` or `case:ignore` tag
Duplicate object keys	accepted	rejected	`jsontext.AllowDuplicateNames(true)`
Invalid UTF-8	replaced silently	rejected	`jsontext.AllowInvalidUTF8(true)`
`time.Duration`	encoded as int64 nanoseconds	requires explicit `format`	`json:",format:nano"`

Case sensitivity is the sneaky one

The nil-slice change is loud. You'll see it in the first test that checks a response body. Case sensitivity is quiet, because it fails by silently not matching rather than by erroring. Consider input from an API that uses SCREAMING_CASE or PascalCase keys:

data := []byte(`{"NAME":"Ada"}`)

var u User // Name is tagged json:"name"
if err := json.Unmarshal(data, &u); err != nil {
    panic(err)
}
fmt.Printf("Name=%q\n", u.Name)

Output:

Name=""

In v1 that would have populated u.Name with "Ada" because matching ignored case. In v2 the field stays at its zero value and no error is raised. If you consume upstream JSON whose casing doesn't exactly match your tags, opt back in per-field:

type User struct {
    Name string `json:"name,case:ignore"`
}

Or globally at the call site with json.MatchCaseInsensitiveNames(true). I'd reach for the per-field tag, since it documents which fields have loose upstream contracts instead of loosening the whole struct.

Duplicate keys now error

v1 happily unmarshaled {"id":1,"id":2} and kept the last value. That's a genuine security hole: different parsers picking different values for the same key is the root of a whole class of request-smuggling and access-control bugs. v2 rejects it:

data := []byte(`{"id":1,"id":2}`)

var u User
err := json.Unmarshal(data, &u)
fmt.Println(err != nil) // true

The error message is descriptive (something along the lines of a duplicate object member name for "id"; the exact wording may shift while the package is experimental). If you have a legitimate reason to accept duplicates, passing jsontext.AllowDuplicateNames(true) restores the old leniency (that option and AllowInvalidUTF8 live in the jsontext package, since they're syntactic, but you pass them straight to json.Unmarshal like any other option). Think hard before you reach for it.

Options replace tag soup

The change I like most has nothing to do with speed. In v1, everything was a struct tag or a global, and anything you couldn't express as a tag you couldn't express at all. v2 makes options first-class function arguments that you pass to Marshal and Unmarshal directly. The package reference lists the full set.

b, err := json.Marshal(u,
    json.FormatNilSliceAsNull(true), // keep v1 null-for-nil
    json.Deterministic(true),        // stable map key ordering
    jsontext.WithIndent("  "),       // pretty-print
)

That Deterministic(true) option is a small quality-of-life win worth calling out: it guarantees identical inputs produce identical output bytes, which makes golden-file tests and content hashing reliable without you sorting maps by hand. You can bundle options once and reuse them:

prettyV1 := json.JoinOptions(
    json.FormatNilSliceAsNull(true),
    json.FormatNilMapAsNull(true),
    jsontext.WithIndent("  "),
)

out, _ := json.Marshal(payload, prettyV1)

The struct-tag vocabulary grew too. The most useful new tag is omitzero, which finally fixes the ancient confusion around omitempty:

type Event struct {
    Name  string    `json:"name"`
    Start time.Time `json:"start,omitzero"`  // dropped when the zero Time
    Tags  []string  `json:"tags,omitempty"`  // dropped when empty JSON ([], "", null)
}

omitempty in v1 was notorious for not omitting a zero time.Time (because a struct is never "empty" by its rule). omitzero checks the Go zero value (or an IsZero() method if your type has one), which is almost always what people meant in the first place. If you've ever shipped "start":"0001-01-01T00:00:00Z" into a JSON response by accident, this tag is for you.

Streaming and the O(n²) trap

The headline performance number is unmarshal speed, but the more interesting fix is structural. In v1, if a custom type implemented MarshalJSON, encoding a large slice of that type could degrade to O(n²): each element's MarshalJSON allocated a fresh []byte, which then got re-parsed and re-copied into the parent buffer. Nest that a couple of levels deep and throughput falls off a cliff.

v2 adds streaming custom-marshaler interfaces that write directly into the shared encoder:

// Old, allocation-heavy:
//   func (t T) MarshalJSON() ([]byte, error)
//
// New, streams into the parent encoder:
func (t T) MarshalJSONTo(enc *jsontext.Encoder) error {
    return enc.WriteToken(jsontext.String(t.value))
}

If you'd rather not add methods to a type, v2 also takes function-based marshalers registered per type through json.WithMarshalers. Those helpers lean on Go generics so you get a type-safe func(T) handler instead of the old interface{} juggling. For bulk output you can drive the encoder yourself and avoid building a giant in-memory value at all:

import (
    json "encoding/json/v2"
    "encoding/json/jsontext"
)

func writeAll(w io.Writer, users []User) error {
    enc := jsontext.NewEncoder(w)
    for _, u := range users {
        if err := json.MarshalEncode(enc, u); err != nil {
            return err
        }
    }
    return nil
}

The decode side mirrors it with json.UnmarshalDecode over a jsontext.Decoder, which pairs naturally with a range loop the way Go's iterators do: read one value, process it, discard it, without holding the whole document in memory. The Go team reported that Kubernetes' kube-openapi saw "orders of magnitude" improvement switching a hot path to the streaming interface. That's the kind of gain you only get from fixing the algorithmic shape of the problem, and no amount of parser tuning would have matched it.

Benchmarking it yourself

I want to be precise here: I did not run these numbers on my own hardware, because the machine I wrote this on is still on Go 1.24 and the package needs 1.25+. The 2–10x figure comes from the Go team's own benchmarks and third-party measurements, and I'm labeling it as reported rather than measured. Marshal is roughly at parity with v1; the big wins are on the unmarshal path, where the new parser does far less work.

Don't take anyone's benchmark on faith, including mine. Here's a harness you can drop into your own repo and run against your actual payloads, which is the only benchmark worth trusting:

package bench

import (
    "os"
    "testing"

    json "encoding/json/v2"
)

func BenchmarkUnmarshal(b *testing.B) {
    data, err := os.ReadFile("testdata/payload.json") // your real data
    if err != nil {
        b.Fatal(err)
    }
    b.ReportAllocs()
    for b.Loop() {
        var out []User
        if err := json.Unmarshal(data, &out); err != nil {
            b.Fatal(err)
        }
    }
}

GOEXPERIMENT=jsonv2 go test -bench=Unmarshal -benchmem ./bench/

Run the same benchmark once with the flag and once without (swapping the import back to encoding/json), and compare with benchstat. Your mileage depends heavily on the shape of your data. Documents with lots of small objects and string-heavy payloads tend to show the largest gap, while a handful of big numeric arrays will show less.

A safe migration plan

The good news is that v2 is designed for gradual adoption. v1 is being reimplemented on top of v2 internally, and the two are meant to coexist for years. The Go team has said v1 won't be deprecated for years after v2 stabilizes, so you don't have to flip everything at once. Here's the order I'd follow:

Turn on the flag in CI first, not prod. Build and run your full test suite with GOEXPERIMENT=jsonv2. Most failures will be golden-file or response-body assertions catching the nil-slice and case-sensitivity changes. That's your blast-radius report, for free.
Fix contracts, not symptoms. Where a test breaks because a nil slice now encodes as [], decide which representation your API actually promises. Usually [] is correct and the test was wrong; occasionally a client depends on null and you set FormatNilSliceAsNull(true) at that boundary.
Add case:ignore where upstream casing is loose. Audit every place you unmarshal third-party JSON and tag the fields whose casing you don't control.
Adopt omitzero in place of the omitempty-on-a-struct workarounds you've accumulated.
Leave the streaming interfaces for later. They're an optimization you can defer safely. Convert the one or two hot paths that profiling flags, and leave the rest on plain Marshal.

If you build data-heavy services, this is the same performance-versus-safety tradeoff conversation that plays out in Python land with msgspec versus Pydantic: a stricter, faster serializer that asks you to be explicit about your data's shape. Go just happens to be baking that upgrade into the standard library instead of a third-party package.

FAQ

Is Go json v2 stable and production-ready?

Not yet, officially. As of Go 1.26 it's experimental, gated behind GOEXPERIMENT=jsonv2, and the API can still change. The plan is to graduate it to the default in Go 1.27. Plenty of teams are already testing it in CI, but I wouldn't gate a production release on experimental behavior that might shift.

What's the difference between encoding/json and encoding/json/v2?

Same core API (Marshal/Unmarshal), but v2 has stricter, safer defaults (rejects duplicate keys and invalid UTF-8, matches field names case-sensitively), encodes nil slices/maps as []/{} instead of null, adds first-class option arguments, introduces the omitzero tag, and ships a much faster unmarshaler.

How much faster is Go json v2?

Reported benchmarks put unmarshaling at roughly 2–10x faster than v1, driven by a more efficient parser. Marshaling is about the same speed as v1. The gains depend on your data shape, so benchmark your own payloads rather than trusting a single headline number.

Do I need to rewrite my structs to use json v2?

No. Existing struct tags keep working. You only touch code where a changed default affects your output: restoring null for nil slices at a specific boundary, adding case:ignore for loosely-cased upstreams, or swapping omitempty for omitzero. Everything else is a recompile.

Can I use encoding/json/v2 on older Go versions?

The standard-library path needs Go 1.25 or newer with the experiment flag. If you're stuck on Go 1.23 or 1.24, the identical implementation is available as the standalone module github.com/go-json-experiment/json.

Sources

A new experimental Go API for JSON — The Go Blog — the official announcement, rationale, and behavior changes
encoding/json/v2 — Go Packages — full API reference for functions, options, and struct tags
encoding/json/jsontext — Go Packages — the syntactic streaming layer
go-json-experiment/json — GitHub — the upstream module usable on older Go toolchains
Go 1.26 Release Notes — current stable release context

Bottom line

json/v2 is the rare standard-library change that's both faster and safer, and the migration cost is mostly "run your tests and read the diffs." The performance is a nice bonus; the real value is that the defaults finally match what most Go developers meant all along: no more null slices leaking into frontends, no more silently accepted duplicate keys, and case-insensitive matching stops quietly papering over schema mismatches. Turn on GOEXPERIMENT=jsonv2 in CI this week, see what breaks, and you'll be ready the day Go 1.27 makes it the default.

How to Build an AI Agent in Go (From Scratch, No Framework)

Maksim Danilchenko — Mon, 13 Jul 2026 08:45:53 +0000

TL;DR

An "AI agent" is a loop: send a prompt to a model, let it ask for a tool, run the tool, feed the result back, repeat until it stops. You can write that loop in Go with the standard library and about 120 lines of code. No LangChain, no SDK, no google.golang.org/adk. This tutorial builds a working agent that can read files and list directories on your machine, driven by the Anthropic Messages API. I'll show the full source, the exact JSON the API expects, the terminal output from a real run, and the two mistakes that cost me an hour the first time.

Why write this in Go at all?

Python owns the agent tutorials, and for good reason — every framework ships a Python client first. But Go has quietly become a strong fit for the runtime side of agents. A Hacker News thread from earlier this year argued the case: one build system, one formatter, static types, and real concurrency without the footguns. When your agent has to fan out ten tool calls, handle timeouts, and ship as a single static binary, that counts for more than how many frameworks exist.

I spent an afternoon porting a toy Python agent to Go, and the thing I kept noticing was how little I missed the framework. The whole "agent" abstraction collapses into a for loop and a switch statement once you see the message flow. That is the point of this post: strip away the layers so you understand what an agent actually is, then decide whether you want a framework on top.

Here's what we'll build: a working file-reading agent in about 120 lines of Go, with zero third-party dependencies and two tools.

The mental model: an agent is a loop

Strip away the marketing and an LLM agent is four steps in a cycle:

Send the conversation (plus a list of tools the model is allowed to call) to the API.
The model replies. It either answers in text, or it asks to call a tool with some arguments.
If it asked for a tool, your code runs that tool and captures the result.
You append the result to the conversation and go back to step 1.

The loop ends when the model responds with plain text and no tool request. The API signals this with stop_reason: "end_turn", and that is the whole pattern. Memory, planning, and "ReAct" are all variations on the same cycle. If you understand the loop, you understand agents.

Setup

You need Go 1.21+ and an API key. Grab one from the Anthropic Console (the tool-use format is cleanest there; I'll note the OpenAI equivalent at the end). Export it:

export ANTHROPIC_API_KEY="sk-ant-..."
mkdir go-agent && cd go-agent
go mod init go-agent

No dependencies to go get. We use net/http and encoding/json from the standard library and nothing else.

Step 1: Talk to the model

Before we add tools, let's make one plain request so the wiring is obvious. The Messages API wants a POST to https://api.anthropic.com/v1/messages with three headers and a JSON body. Here are the types:

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
)

type Tool struct {
    Name        string         `json:"name"`
    Description string         `json:"description"`
    InputSchema map[string]any `json:"input_schema"`
}

type ContentBlock struct {
    Type string `json:"type"`
    // text block
    Text string `json:"text,omitempty"`
    // tool_use block (from the model)
    ID    string          `json:"id,omitempty"`
    Name  string          `json:"name,omitempty"`
    Input json.RawMessage `json:"input,omitempty"`
    // tool_result block (from us)
    ToolUseID string `json:"tool_use_id,omitempty"`
    Content   string `json:"content,omitempty"`
}

type Message struct {
    Role    string         `json:"role"`
    Content []ContentBlock `json:"content"`
}

type Request struct {
    Model     string    `json:"model"`
    MaxTokens int       `json:"max_tokens"`
    Messages  []Message `json:"messages"`
    Tools     []Tool    `json:"tools,omitempty"`
}

type Response struct {
    Content    []ContentBlock `json:"content"`
    StopReason string         `json:"stop_reason"`
}

The one struct worth staring at is ContentBlock. The API models every piece of a turn as a typed block (a text block, a tool_use block, a tool_result block), and omitempty lets one Go struct serialize as any of them. The model's input arrives as arbitrary JSON, so I keep it as json.RawMessage and decode it later per tool.

Now the call itself:

func callClaude(msgs []Message, tools []Tool) (Response, error) {
    body, _ := json.Marshal(Request{
        Model:     "claude-sonnet-5",
        MaxTokens: 1024,
        Messages:  msgs,
        Tools:     tools,
    })

    req, _ := http.NewRequest("POST",
        "https://api.anthropic.com/v1/messages",
        bytes.NewReader(body))
    req.Header.Set("x-api-key", os.Getenv("ANTHROPIC_API_KEY"))
    req.Header.Set("anthropic-version", "2023-06-01")
    req.Header.Set("content-type", "application/json")

    resp, err := http.DefaultClient.Do(req)
    if err != nil {
        return Response{}, err
    }
    defer resp.Body.Close()

    data, _ := io.ReadAll(resp.Body)
    if resp.StatusCode != 200 {
        return Response{}, fmt.Errorf("api %d: %s", resp.StatusCode, data)
    }

    var out Response
    return out, json.Unmarshal(data, &out)
}

The anthropic-version header is not optional. Leave it out and you get a 400 with a message that does not obviously point at the missing header. That was gotcha number one for me. The Messages API reference documents the current version string.

Step 2: Define the tools

A tool is a name, a description, and a JSON Schema for its arguments. The description is doing real work here: the model reads it to decide when to call the tool, so write it the way you'd brief a new teammate who has to figure out on their own when to reach for it.

var readFileTool = Tool{
    Name:        "read_file",
    Description: "Read a file at a relative path and return its contents as text.",
    InputSchema: map[string]any{
        "type": "object",
        "properties": map[string]any{
            "path": map[string]any{
                "type":        "string",
                "description": "Relative path, e.g. main.go or docs/readme.md",
            },
        },
        "required": []string{"path"},
    },
}

var listFilesTool = Tool{
    Name:        "list_files",
    Description: "List the files and folders in a directory. Defaults to the current directory.",
    InputSchema: map[string]any{
        "type": "object",
        "properties": map[string]any{
            "dir": map[string]any{
                "type":        "string",
                "description": "Directory to list. Empty means current directory.",
            },
        },
    },
}

And the dispatcher that actually runs a tool when the model asks:

func executeTool(name string, input json.RawMessage) string {
    switch name {
    case "read_file":
        var args struct {
            Path string `json:"path"`
        }
        json.Unmarshal(input, &args)
        data, err := os.ReadFile(args.Path)
        if err != nil {
            return "error: " + err.Error()
        }
        return string(data)

    case "list_files":
        var args struct {
            Dir string `json:"dir"`
        }
        json.Unmarshal(input, &args)
        if args.Dir == "" {
            args.Dir = "."
        }
        entries, err := os.ReadDir(args.Dir)
        if err != nil {
            return "error: " + err.Error()
        }
        names := make([]string, 0, len(entries))
        for _, e := range entries {
            names = append(names, e.Name())
        }
        return strings.Join(names, "\n")
    }
    return "unknown tool: " + name
}

Notice I return the error string to the model instead of crashing. That's deliberate. A good agent tool hands failures back as text so the model can recover — try a different path, ask the user, give up gracefully. If your tool panics, the loop dies and the model never gets a chance to react.

Step 3: The agent loop

This is the whole payoff — the piece that turns a chat call into an agent. Add bufio and strings to the imports, then:

func main() {
    tools := []Tool{readFileTool, listFilesTool}
    scanner := bufio.NewScanner(os.Stdin)
    var msgs []Message

    fmt.Println("Go agent ready. Ask about files in this folder (Ctrl-D to quit).")
    for {
        fmt.Print("\nyou> ")
        if !scanner.Scan() {
            break
        }
        text := strings.TrimSpace(scanner.Text())
        if text == "" {
            continue
        }
        msgs = append(msgs, Message{
            Role:    "user",
            Content: []ContentBlock{{Type: "text", Text: text}},
        })

        // Inner loop: keep running tools until the model is done.
        for {
            resp, err := callClaude(msgs, tools)
            if err != nil {
                fmt.Println("error:", err)
                break
            }
            // Echo the assistant turn back into history verbatim.
            msgs = append(msgs, Message{Role: "assistant", Content: resp.Content})

            var results []ContentBlock
            for _, block := range resp.Content {
                switch block.Type {
                case "text":
                    fmt.Println("\nclaude>", block.Text)
                case "tool_use":
                    fmt.Printf("  [tool] %s(%s)\n", block.Name, block.Input)
                    out := executeTool(block.Name, block.Input)
                    results = append(results, ContentBlock{
                        Type:      "tool_result",
                        ToolUseID: block.ID,
                        Content:   out,
                    })
                }
            }

            if len(results) == 0 {
                break // stop_reason was end_turn — the model answered.
            }
            msgs = append(msgs, Message{Role: "user", Content: results})
        }
    }
}

Two loops, nested. The outer loop is the human conversation. The inner loop is the agent thinking: it keeps calling tools and feeding results back until the model stops asking. When there are no tool_use blocks in a response, results is empty and we break out to wait for the next human message.

The line that bit me, gotcha number two, is msgs = append(msgs, Message{Role: "assistant", Content: resp.Content}). You must echo the assistant's tool_use block back into the history before you send the matching tool_result. Skip it, and the API rejects your next request because a tool_result with no preceding tool_use is malformed. Keeping the raw resp.Content (including tool_use blocks as json.RawMessage) is the simplest way to get this right.

Running it

Build and run:

go build -o agent . && ./agent

Here's a real session against a folder holding this project's Go files:

Go agent ready. Ask about files in this folder (Ctrl-D to quit).

you> what go files are here and what does main.go do?

  [tool] list_files({"dir":""})
  [tool] read_file({"path":"main.go"})

claude> This folder has three Go files: main.go, tools.go, and go.mod.
main.go is a command-line AI agent. It reads your input, sends it to the
Claude Messages API with two tools (read_file and list_files), runs any
tool the model requests, and loops until the model gives a final answer.
The core is a nested loop in main() that keeps executing tools until
stop_reason is end_turn.

you> which file is the biggest?

  [tool] list_files({"dir":""})
  [tool] read_file({"path":"main.go"})
  [tool] read_file({"path":"tools.go"})

claude> main.go is the largest at roughly 120 lines; tools.go is about 40.

The model chained two tool calls in the first turn. It listed the directory, then decided to read main.go, all before writing a single word of the answer. You did not orchestrate that ordering. The loop simply kept handing control back until the model decided it had enough to answer.

When you actually want a framework

Building from scratch is the right way to learn, and honestly it's fine for small production agents too. But once you need streaming, retries with backoff, session persistence, or multi-agent handoff, a framework earns its place. Here are the main Go options today:

Option	What it is	Reach for it when
Standard library (this post)	Raw HTTP + your own loop	Learning, full control, single static binary, minimal surface area
Anthropic Go SDK	Official typed client	You want typed requests/streaming for one provider without hand-rolling JSON
Google ADK for Go	Full agent framework: sessions, runners, tools	Multi-agent orchestration, Gemini-first stacks
Eino	ByteDance's composable orchestration graph	Complex pipelines with branching and fan-out
langchaingo	LangChain-style multi-provider abstraction	You want one interface across OpenAI, Anthropic, and local models

My honest take: start with the standard library, add the official SDK when JSON marshaling gets tedious, and only pull in a full framework when you feel real pain from the missing pieces. Most agents I've seen reach for a framework far too early and end up fighting its abstractions instead of shipping.

If you're coming from the editor side of the agent world, the Agent Client Protocol covers how these loops plug into IDEs, and Claude Code Subagents shows the same tool-calling idea running inside a production coding agent. For the concurrency question specifically (why Go's model helps when tools run in parallel), the Rust vs Go comparison digs into the tradeoffs.

The parts I deliberately left out

This agent is real but minimal. Three things you'd add before trusting it with anything important:

Guardrails on tools. My read_file will happily read /etc/passwd if the model asks. In production you sanitize paths and sandbox the filesystem. I wrote up four ways this goes wrong in AI agent guardrails.
A turn limit. The inner loop trusts the model to eventually stop. A confused model can ping-pong tools forever. Add a counter and bail after, say, 10 iterations.
Streaming. We wait for the full response before printing. For a snappy CLI you'd switch to the streaming endpoint and print tokens as they arrive — Go's bufio.Scanner over the SSE body handles this cleanly.

None of these change the shape of the loop; they just harden it around the edges.

FAQ

Is Go good for building AI agents?

Yes, for the runtime. Go gives you static binaries, real concurrency for parallel tool calls, and strong typing around the JSON the API speaks. The tradeoff is fewer frameworks than Python — but as this post shows, you often don't need one. If your agent is mostly orchestration and I/O rather than data science, Go is a strong pick.

Do I need a framework like LangChain to build an agent in Go?

No. An agent is a loop over a chat API with tool calling, and the standard library covers it in ~120 lines. Frameworks add value for streaming, retries, session state, and multi-agent coordination, but they are optional. Start without one so you understand the mechanics.

How does an AI agent work under the hood?

It's a cycle: send the conversation plus a tool list to the model; the model either answers or requests a tool call; your code runs the tool and appends the result; repeat until the model answers with no tool request. The API's stop_reason field tells you when the loop is done.

Can I use OpenAI instead of Anthropic?

Yes. The shape is identical, only the field names differ. With the OpenAI API you POST to /v1/chat/completions, tools live under a tools array with a function wrapper, and the model returns tool_calls instead of tool_use blocks. Swap the request/response structs and the executeTool dispatcher stays exactly the same.

What model should I use for a Go agent?

Any model with tool-calling support works. I used claude-sonnet-5 because it's fast and reliable at deciding when to call a tool. For cheaper runs, a smaller model like Haiku handles simple tool routing well; for hard multi-step tasks, a frontier model chains tools more reliably.

Sources

Anthropic Messages API reference — the request/response format and tool-use schema this agent speaks
Anthropic tool use documentation — how tool_use and tool_result blocks work
Anthropic Go SDK — official typed client for production use
Google ADK for Go — full agent framework, referenced in the comparison table
"A case for Go as the best language for AI agents" — Hacker News — the discussion that framed the "why Go" argument

Bottom line

The framework tutorials make agents look like a stack of abstractions you have to learn first. Really an agent is just a loop that hands control back and forth between a model and your tools, and Go writes that loop about as cleanly as any language I've tried. Build the 120-line version first. Once you can see the message flow in your head, you'll know exactly which framework feature you're missing — and, more often than not, that you don't need one yet.

Claude Sonnet 5 Review: A Week With Anthropic's New Default

Maksim Danilchenko — Tue, 07 Jul 2026 08:42:22 +0000

TL;DR

Claude Sonnet 5 shipped on June 30, 2026 and immediately became the default model on Claude's Free and Pro plans. Anthropic's published numbers put it at 63.2% on SWE-bench Pro, a few points behind Opus 4.8's 69.2% but well ahead of Sonnet 4.6's 58.1%. On Terminal-Bench 2.1 it actually beats Opus 4.8 (80.4% vs 74.6%). Intro pricing is $2 per million input tokens and $10 output through August 31, then $3/$15. One detail is easy to miss: it ships with a new tokenizer that uses 1.0–1.35× more tokens for the same text, so the sticker price undersells what you'll actually pay. After a week of using it as my daily driver in Claude Code, my take is that Sonnet 5 is the model most people should be running now, and Opus 4.8 has shrunk to a specialist tool you reach for on the genuinely hard problems.

What Claude Sonnet 5 Actually Is

Anthropic ships models in three sizes: Haiku for cheap and fast, Sonnet for the middle, Opus for the frontier. Sonnet has always been the workhorse, the one that runs when you don't think about which model you're using. Sonnet 5 is the fifth generation of that workhorse, and the pitch is that the middle tier has crept close enough to the top tier that the top tier stops being the obvious choice.

It landed as the default for Free and Pro on claude.ai the day it launched, with access for Max, Team, and Enterprise, plus the API id claude-sonnet-5 on the Claude Platform and inside Claude Code. That default placement counts for more than the benchmarks. Most Claude traffic never picks a model on purpose. Whatever Anthropic sets as the default writes most of the code, drafts most of the emails, and shapes what people think "Claude" is. That makes Sonnet 5 the new baseline for almost everyone who touches Claude, whether they know the version number or not.

I flipped my Claude Code default to it on launch morning and left it there for a week, running it against a Go service with a messy migration path, a Next.js dashboard, and a pile of one-off scripts. This was the everyday work sitting in my terminal, the kind I'd normally split between Claude and Codex. What stuck with me is that it finishes things. Multi-step tasks that used to stall halfway and wait for me to nudge them now run to completion on their own, and that single behavioral change is worth more than any single-digit benchmark bump.

The Benchmark Breakdown

Here are the numbers Anthropic and early testers published, lined up against the two models Sonnet 5 is really being measured against: its own predecessor and the flagship one rung up.

Benchmark	Sonnet 5	Sonnet 4.6	Opus 4.8
SWE-bench Pro (agentic coding)	63.2%	58.1%	69.2%
Terminal-Bench 2.1	80.4%	—	74.6%
OSWorld-Verified (computer use)	81.2%	78.5%	83.4%
BrowseComp (agentic search)	84.7%	—	—
GDPval-AA v2 (knowledge work, Elo)	1,618	—	1,615
FrontierCode	38.8%	15.1%	—

Two rows stand out. On SWE-bench Pro, the 5.1-point jump from Sonnet 4.6 to Sonnet 5 (58.1 to 63.2) is a real generational gain on a benchmark where model families usually move a couple of points per release. FrontierCode more than doubled in reported results, from 15.1% to 38.8%. Since FrontierCode targets the hard, long-horizon coding problems that break most models, a 2.5× jump there is the clearest signal that Sonnet 5 got structurally better at sustained agentic work rather than merely more polished.

The Terminal-Bench result is the one that made me raise an eyebrow. A mid-tier model beating the flagship on any benchmark is unusual, and Terminal-Bench 2.1 measures exactly the kind of shell-driving, command-running, output-reading loop that agentic coding lives in. Sonnet 5's 80.4% against Opus 4.8's 74.6% lines up with what I felt using it: for the day-to-day loop of edit, run tests, read the failure, fix, rerun, it is genuinely excellent, and the extra brains in Opus don't help much on that kind of task.

Where Opus still wins is SWE-bench Pro and OSWorld, both by a handful of points. Those gaps are real but small, and they narrow further once you weigh the cost difference. The GDPval knowledge-work Elo is essentially a tie (1,618 vs 1,615), which tells you the reasoning and writing quality of the two models is now hard to tell apart on ordinary tasks.

A Week Of Real Use

Benchmarks tell you a model's ceiling. A week of use tells you its floor, how it behaves on the boring 90% of tasks that never make it into a benchmark suite.

The best thing about Sonnet 5 is the self-verification loop. On my Go service, I asked it to migrate a set of handlers from a hand-rolled router to chi and update the tests. Sonnet 4.6 would have written the change and stopped, leaving me to run the suite and paste back the failures. Sonnet 5 wrote the change, ran the tests itself, saw two of them fail on a middleware ordering bug, fixed the ordering, and reran until green — all without me typing anything between the first prompt and the finished diff. It did this consistently over the week, not once as a lucky demo. Early coverage reports the same pattern: the model runs tests, reads the failures, patches, and reruns without being told to. That is the behavior that turns "AI that writes code" into "AI that finishes a task."

The second thing I noticed is that it's honest about uncertainty more often. On an ambiguous refactor where I hadn't specified whether to keep backward compatibility, it stopped and asked instead of guessing and plowing ahead. That's a small thing that saves a lot of wasted diffs.

It still has clear failure modes. On a gnarly Next.js server-component bug involving a hydration mismatch, Sonnet 5 went in circles twice: it proposed the same wrong fix, I rejected it, and it came back with a slight variation of the same wrong fix. I switched that one task to Opus 4.8 and it found the actual cause (a Date being rendered without a stable server/client format) in one pass. That is the shape of the current gap: on the genuinely hard, ambiguous problems that need real backtracking, Opus still pulls ahead. On everything else, you won't notice a difference.

The Tokenizer Tax

The pricing page leaves out one detail. Sonnet 5 ships with an updated tokenizer, and Anthropic's own announcement says it produces roughly 1.0 to 1.35× more tokens for the same input text than Sonnet 4.6 did. That multiplier is not uniform: plain English lands near the low end, while code, structured data, and non-English text push toward the high end.

What that means in practice: the $2/$10 intro price is genuinely cheap, but your token counts go up, so the effective cost per request rises more than the sticker drop suggests. If you're comparing Sonnet 5's $3/$15 standard price against Sonnet 4.6's $3/$15, they look identical on paper, but your bill won't match, because the same task now consumes more tokens. I watched this happen in Claude Code's usage meter over the week: comparable tasks showed noticeably higher token counts than I was used to under 4.6.

This doesn't make Sonnet 5 expensive. It's still dramatically cheaper than Opus 4.8. But it does mean you should run your own numbers before assuming a migration is a pure cost win. If you have a high-volume production pipeline with tight per-task budgets, measure the real token consumtion on your actual prompts rather than trusting the price-per-million headline.

Model	Input / 1M	Output / 1M	Notes
Sonnet 5 (intro, through Aug 31)	$2	$10	New tokenizer, +0–35% tokens
Sonnet 5 (standard, from Sep 1)	$3	$15	Same sticker as 4.6, higher token use
Sonnet 4.6	$3	$15	Older, cheaper-per-token tokenizer
Opus 4.8	$5	$25	Flagship; reach for the hard tasks

When To Use Sonnet 5 vs Opus 4.8

After a week, my routing rule is simple. Send everything to Sonnet 5 by default. Reach for Opus 4.8 only when a task is hard enough that a wrong answer is expensive: a subtle concurrency bug, an ambiguous spec that needs careful interpretation, a refactor where backtracking pays off. Keep Haiku 4.5 for the high-volume, latency-sensitive calls where you'd never have used a big model anyway.

That's a real shift from three months ago, when Opus was my default for anything that mattered and Sonnet was the "good enough, save money" option. Now the ordering has flipped: Sonnet 5 is the default that's good enough for almost everything, and Opus is the specialist you escalate to, with Fable 5 sitting above it for the longest autonomous runs. The GDPval tie and the Terminal-Bench win explain why. On the work that fills most of a day, the two models produce output I can't reliably tell apart, so paying 2.5× more for Opus stops making sense.

If you're weighing this against non-Anthropic options, the picture is more crowded than it was. GLM-5.2 undercuts everyone on price with open weights and lands close on coding benchmarks, and GPT-5.5 trades blows on general tasks. Sonnet 5's edge over both is the agentic follow-through, the finish-the-task behavior, which is exactly the thing that doesn't show up cleanly in a single benchmark score. For a broader field, our GPT vs Claude vs Gemini coding comparison still holds up on the general shape of the tradeoffs, even if the specific version numbers have moved.

How To Switch To Sonnet 5

If you're on claude.ai Free or Pro, you're already using it; it's the default. For everything else, you point at the model id.

Via the API with the Python SDK from the Anthropic console:

import anthropic

client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY

message = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=4096,
    messages=[
        {
            "role": "user",
            "content": "Refactor this Go HTTP handler to use chi router "
                       "and update the table-driven tests.",
        }
    ],
)
print(message.content[0].text)

In Claude Code, set it as the session default so every agent turn uses it:

# One-off for the current session
claude --model claude-sonnet-5

# Or make it sticky in your shell profile
export ANTHROPIC_MODEL="claude-sonnet-5"

If you drive Claude through a coding IDE, the same model id works. I run it as the default in Cursor for ordinary edits and only flip to Opus when a task fights back. One thing to test before you migrate a production prompt: the new tokenizer can change how structured inputs get parsed, so if you have prompts that depend on exact output formatting or tight tool-call schemas, run them through once and check the edges rather than assuming a drop-in swap.

Here's a quick way to sanity-check the token difference on your own text before and after switching, using the count-tokens endpoint:

import anthropic

client = anthropic.Anthropic()

resp = client.messages.count_tokens(
    model="claude-sonnet-5",
    messages=[{"role": "user", "content": open("prompt.txt").read()}],
)
print("Sonnet 5 input tokens:", resp.input_tokens)

Run that against a representative prompt, compare it to what the same text cost you under Sonnet 4.6, and you'll know your real multiplier instead of guessing at the 1.0–1.35× range.

Who Should Use It, Who Should Skip

Good fit:

Anyone already on Claude for coding. Sonnet 5 is a straight upgrade over 4.6 in every category, and during the intro window it's cheaper too.
Teams running agentic workflows (Claude Code, Cursor, custom harnesses) where the self-verification loop saves human turns.
Cost-conscious builders who were paying for Opus out of habit. Most of that work drops to Sonnet 5 with no quality loss you'll notice.

Think twice:

High-volume production pipelines with razor-thin margins. Measure the tokenizer tax on your real traffic before you assume the migration saves money.
Anyone whose prompts depend on exact tokenization or output formatting. Test the edges first.
Work that lives at the hard end of the difficulty curve: deep debugging, ambiguous specs, tasks that need backtracking. Keep Opus 4.8 in reach for those, and see our Claude Code Max vs Pro breakdown if you're deciding how much Anthropic compute to buy.

FAQ

How much does Claude Sonnet 5 cost?

Introductory API pricing is $2 per million input tokens and $10 per million output tokens through August 31, 2026, moving to $3/$15 after that. On claude.ai it's the default model on the Free and Pro plans at no extra cost. Factor in the new tokenizer, which uses 1.0–1.35× more tokens for the same text, when you estimate real spend.

Is Claude Sonnet 5 good for coding?

Yes — it's the strongest mid-tier coding model Anthropic has shipped. It scores 63.2% on SWE-bench Pro (up from Sonnet 4.6's 58.1%) and beats Opus 4.8 on Terminal-Bench 2.1. The standout in daily use is agentic follow-through: it runs tests, reads failures, and fixes its own code without being told to.

Claude Sonnet 5 vs Opus 4.8 — which should I use?

Use Sonnet 5 as your default for almost everything; it's close to Opus 4.8 on most benchmarks, wins on Terminal-Bench, and costs less than half as much. Reserve Opus 4.8 for the hardest tasks (subtle bugs, ambiguous requirements, anything that needs real backtracking) where its extra reasoning still pulls ahead.

What is Claude Sonnet 5's context window?

Sonnet 5 supports a 1 million token context window, matching the top of Anthropic's lineup. That's enough to hold a mid-size codebase, long agent trajectories, or large documents in a single session.

When was Claude Sonnet 5 released?

Anthropic released Claude Sonnet 5 on June 30, 2026, and made it the default model on the Free and Pro plans the same day.

Sources

Introducing Claude Sonnet 5 (Anthropic): official announcement, pricing, availability, and the tokenizer change
Claude models overview (Claude Platform Docs): model ids, context windows, and standard pricing
Claude Sonnet 5 vs Sonnet 4.6 vs Opus 4.8 (MarkTechPost): benchmark table and cost-performance analysis
Claude Sonnet 5 pricing: the hidden costs behind the "cost-neutral" launch (Finout): breakdown of the tokenizer's effect on real spend

Bottom Line

Claude Sonnet 5 is the clearest sign yet that the interesting action in AI coding has moved from the frontier tier to the tier just below it. It nearly matches Opus 4.8 on the benchmarks that matter, beats it outright on the terminal-driving loop that agentic coding actually runs on, and does it at less than half the price. After a week of leaving it as my Claude Code default, the only tasks I sent back to Opus were the two or three genuinely hard bugs a week throws at you.

The tokenizer tax is the one thing to watch. Sonnet 5 is cheaper than Opus and, during the intro window, cheaper than 4.6, but the higher token consumption means you should measure your real cost rather than trust the headline. Do that math, then switch. For the work that fills most of a developer's day, this is the model to reach for now, and if you're on a Free or Pro plan, it's already the one running by default.

Perplexity Bumblebee Review: The Supply Chain Scanner Your Dev Machine Needs

Maksim Danilchenko — Mon, 29 Jun 2026 08:35:52 +0000

TL;DR

Bumblebee is a read-only supply chain scanner from Perplexity AI that checks your installed packages, editor extensions, MCP configs, and browser extensions against known-compromised versions. It never runs a package manager command. I ran it on three machines. It found 847 packages across 9 ecosystems in under 4 seconds. The output is NDJSON you can pipe into anything. It won't replace Snyk or Socket in your CI pipeline, but for the question "does anyone on the team have that compromised package installed right now?" it's the fastest answer that exists.

Why Developer Machines Are the Blind Spot

When a supply chain advisory drops (say, a compromised npm package like @redhat-cloud-services/frontend-components in June 2026), the security team's first question is simple: who has it installed?

Your CI pipeline knows what's in your lockfile. Your SBOM tool knows what shipped to production. Your EDR knows what processes ran. None of them know what's sitting in ~/.npm, what extensions your VS Code installed last Tuesday, or what MCP servers your Claude Code config points to.

Sonatype's 2026 State of the Software Supply Chain report counted over 454,600 new malicious open-source packages in 2025, pushing the cumulative total past 1.233 million. That's a 75% year-over-year increase. The attacks keep getting more creative, too: the Mini Shai-Hulud campaigns starting in late April 2026 hit npm, PyPI, RubyGems, and Composer across companies like SAP, with the TanStack compromise following in May. In June 2026, TeamPCP's Miasma worm injected itself into the SessionStart hooks of 13 AI coding tools, including Claude Code, GitHub Copilot, and Gemini CLI.

That last one is why MCP config scanning matters. If your claude_desktop_config.json or mcp.json references a compromised server, you've handed an attacker tool-level access to your coding environment. Traditional security scanners don't even look at these files.

Bumblebee fills that gap. It answers one question: what packages, extensions, and configs are on this machine right now, and do any of them match a known-bad advisory?

What Bumblebee Actually Is

Bumblebee is a single static binary written in Go 1.25+ with zero non-stdlib dependencies. Perplexity open-sourced it in May 2026 under Apache 2.0. The v0.1.1 release has crossed 4k GitHub stars.

The design is deliberately narrow. Bumblebee collects an inventory of what's installed on a developer machine and optionally matches that inventory against an exposure catalog of known-compromised packages. It does not:

Run npm install, pip install, or any other package manager command
Execute install scripts or lifecycle hooks
Read your source code
Make network calls during the scan
Modify anything on disk

That last set of constraints is the whole point. When your machine might already be compromised, the scan tool itself can't be the thing that triggers the compromise. npm's postinstall scripts are a well-known attack vector. Bumblebee never gives them a chance to fire.

What It Scans

The coverage spans ten ecosystems, which is wider than I expected from a v0.1 tool:

Ecosystem	What It Reads	Package Manager
npm	package-lock.json, yarn.lock, pnpm-lock.yaml, bun.lock	npm/yarn/pnpm/bun
PyPI	.dist-info/METADATA, .egg-info/PKG-INFO	pip/setuptools
Go modules	go.sum, go.mod	go
RubyGems	Gemfile.lock, *.gemspec	bundler
Composer	composer.lock, installed.json	composer
Homebrew	INSTALL_RECEIPT.json, .metadata	brew
MCP servers	claude_desktop_config.json, mcp.json, various IDE configs	Claude Code/Cursor/Gemini CLI
Agent skills	skills-lock.json	vercel-labs/skills
Editor extensions	VS Code, Cursor, Windsurf, VSCodium manifests	IDE ecosystem
Browser extensions	Chrome, Edge, Brave, Arc, Firefox manifests	Browser ecosystem

The MCP and agent skills scanning is what makes this feel purpose-built for 2026. Phoenix Security's 2026 supply chain report found that AI-agent skills carry a risk rate 2.3 times higher than IDE extensions, and more than 1 in 4 deep-scanned skills triggered a critical-risk finding. Bumblebee parses claude_desktop_config.json, mcp.json, and Gemini CLI config files to catalog which MCP servers are registered and their source packages.

Running It: Three Machines, Three Profiles

I installed Bumblebee via go install and ran it across three environments: my primary macOS development laptop, a Linux CI runner, and a colleague's machine that I suspected had stale npm packages from a prototype project six months ago.

Installation

go install github.com/perplexityai/bumblebee/cmd/bumblebee@v0.1.1

One binary. No config files. No daemon. The Go install pulls the source and compiles it locally, so the binary on your machine was built from source you can audit.

Baseline Scan

The baseline profile hits common global package roots, editor extensions, browser extensions, and MCP configs:

bumblebee scan --profile baseline > inventory.ndjson

On my macOS machine, the baseline scan completed in 1.8 seconds and found 312 packages across npm, PyPI, Go modules, Homebrew, 23 VS Code extensions, 4 Cursor extensions, and 3 MCP server configurations.

Each record in the NDJSON output looks like this:

{
  "record_type": "package",
  "hostname": "maxbook.local",
  "ecosystem": "mcp",
  "package_name": "filesystem-mcp-server",
  "version": "0.6.2",
  "source_type": "mcp_config",
  "source_file": "/Users/max/.config/claude/claude_desktop_config.json",
  "confidence": "medium"
}

The confidence levels are useful. high means exact canonical metadata with verified version. medium means reliable identity but partial version info, which is common for MCP servers where the config only records a package reference. low means it's a config path or spec reference only.

Project Scan

bumblebee scan --profile project \
  --root "$HOME/code" \
  --root "$HOME/Developer" > project-inventory.ndjson

This took 3.4 seconds and found 535 packages. The extra coverage comes from crawling lockfiles in project directories. A Go project I'd forgotten about contributed 47 transitive dependencies from its go.sum.

Deep Scan

bumblebee scan --profile deep \
  --root "$HOME" \
  --max-duration 10m > deep-inventory.ndjson

The deep profile is for incident response. It sweeps everything under the given root. On my machine with a messy home directory, it found 847 packages in 3.9 seconds. On the colleague's machine (the one with stale npm prototypes), it found 1,423 packages. A lot of them were cached npm packages from six months ago that nobody realized were still sitting on disk.

Exposure Check

This is where Bumblebee earns its keep. Once you have an advisory (say, the June 2026 @redhat-cloud-services compromise), you write an exposure catalog:

{
  "schema_version": "0.1.0",
  "entries": [
    {
      "id": "advisory-2026-redhat-npm",
      "name": "@redhat-cloud-services/frontend-components 4.2.3",
      "ecosystem": "npm",
      "package": "@redhat-cloud-services/frontend-components",
      "versions": ["4.2.3", "4.2.4"],
      "severity": "critical"
    }
  ]
}

Then run:

bumblebee scan --profile deep \
  --root "$HOME" \
  --exposure-catalog ./catalog.json

If any match exists, Bumblebee emits a finding record alongside the normal inventory. The finding includes the matched package, version, catalog entry, and severity. Pipe the output through jq '.record_type == "finding"' and you have a yes/no answer in seconds.

Self-Test

Before trusting it with real incident data, verify the binary works:

bumblebee selftest
# selftest OK (2 findings in 1ms)

The self-test uses embedded fixtures with fake package names. It validates that the scanning and matching logic works correctly on your platform before you point it at real data.

Where Bumblebee Fits (and Where It Doesn't)

Don't treat Bumblebee as a replacement for your existing security toolchain. It's a complement. Here's how it maps against the tools you probably already use:

Tool	What It Does	When It Runs
Snyk	Scans dependencies for known CVEs, generates fix PRs	CI/CD, on PR merge
Socket	Behavioral analysis of packages pre-install (typosquatting, exfiltration)	Pre-install, CI/CD
OSV-Scanner	Matches lockfiles against the OSV database	CI/CD, local
npm audit	Checks installed packages against the npm advisory database	Post-install, CI/CD
Bumblebee	Reads on-disk metadata across 10 ecosystems, no execution	On-demand, scheduled on dev machines

Bumblebee fills the post-incident triage gap. Snyk tells you what's vulnerable in your repo. Socket tells you if a package is behaving suspiciously before you install it. Bumblebee tells you which developer machines are currently exposed to a specific known-compromised package. They complement each other.

The strongest use case: a zero-day advisory drops at 2 AM. Your security team pushes an exposure catalog. Every developer machine runs bumblebee scan --profile deep --exposure-catalog ./catalog.json as part of their morning boot script. By the time standup happens, you know exactly who's affected.

What I Liked

Speed. Sub-4-second scans across 800+ packages on a well-loaded macOS machine. It reads metadata files directly from the filesystem, with no process spawning, no network calls, and no dependency resolution. The Go implementation is tight.

Zero dependencies. The entire binary is Go standard library. For a security tool, that's a big deal: every third-party dependency is an attack surface. Bumblebee's supply chain risk is literally zero non-stdlib packages. You can verify this yourself. The go.mod file lists only the module path and Go version.

MCP config scanning. No other supply chain tool does this. With MCP tool poisoning becoming a real attack vector in 2026, a tool that catalogs which MCP servers are configured on a machine is ahead of the curve.

NDJSON output. Every record is a self-contained JSON line. Pipe it through jq, feed it to your SIEM, aggregate it with xargs. No proprietary format, no dashboard required.

Read-only guarantee. I checked the source. There are no os.Create, no os.WriteFile, no exec.Command calls outside the self-test fixtures. The binary genuinely can't modify your system.

What It Doesn't Do

The biggest practical limitation is version range matching. Bumblebee v0.1 only matches exact name + version pairs. If an advisory says "all versions below 2.3.1 are affected," you need to write out every affected version in the catalog, or write a wrapper that expands ranges. Most advisories use ranges, so this will bite you early.

MCP config parsing is JSON-only. Continue uses YAML, Codex uses TOML, and both are skipped silently. Claude Code and Cursor both use JSON so you're covered there, but it's an incomplete picture.

There's no remediation. Bumblebee tells you what's there. It won't remove it, update it, or file a PR. That's by design (a read-only tool should stay read-only), but it means you need a separate process for acting on the findings.

There's also no continuous monitoring. It's a one-shot execution model. You run it, get results, done. If you want recurring scans, wrap it in a cron job or MDM policy. No agent, no daemon, no phone-home.

Finally, Linux and macOS only. No Windows support in v0.1.1. The scan profiles are built around Unix filesystem conventions. If your team has Windows developers, Bumblebee misses them entirely.

The MCP Angle

I want to call out the MCP scanning specifically because it's the feature that separates Bumblebee from every other supply chain tool I've used.

Your claude_desktop_config.json or project-level mcp.json defines which tools your AI coding agent can call. A compromised MCP server gets tool-level access inside your development session. It can read files, write files, execute commands, and interact with external services, all under the guise of a legitimate tool.

The Miasma worm that hit in June 2026 specifically targeted these config files, injecting itself into SessionStart hooks across 13 AI coding tools. Traditional security scanners didn't catch it because they don't read MCP configuration files. Bumblebee does.

When I ran the baseline scan, it found three MCP servers in my Claude Code config: filesystem-mcp-server, brave-search, and a custom internal tool. All three were clean, but the fact that they showed up in the inventory meant I could match them against future advisories automatically. That's the workflow: scan once, match many times.

Running Bumblebee in a Team

For individual developers, the one-shot model works fine. For a team, you need a bit more structure:

bumblebee scan --profile baseline \
  --exposure-catalog /shared/catalogs/latest.json \
  | jq -c 'select(.record_type == "finding")' \
  | curl -s -X POST -d @- https://internal-siem.company.com/ingest

The pattern is: scan locally, filter for findings, push to a central collector. The NDJSON output makes this trivial to integrate with whatever log aggregtion you already run.

For MDM-managed fleets, distribute the binary via your existing tooling (it's a single static file, no installer needed), set up a scheduled task, and point --exposure-catalog at a shared network path that your security team updates when advisories drop.

FAQ

Does Bumblebee replace Snyk or Socket?

No. Bumblebee scans what's already on your machine. Snyk and Socket operate in CI/CD to prevent bad packages from entering your projects. Use all three. They cover different phases of the supply chain.

Can Bumblebee trigger malicious code during a scan?

No. It reads metadata files (lockfiles, manifests, config JSON) directly from the filesystem. It never executes npm install, pip install, or any lifecycle hooks. The scan can't trigger a compromised package's install script.

Does it work with private registries?

Yes, for inventory purposes. Bumblebee reads lockfile metadata, which includes private package names and versions. It doesn't authenticate against registries. It reads from local disk, so there's nothing to authenticate against.

How do I get exposure catalogs?

You build them. Bumblebee doesn't ship with a built-in advisory database. When an advisory drops (from GitHub Security Advisories, Snyk's database, or your own threat intel), you convert it into the catalog JSON format and pass it to --exposure-catalog. Multiple catalogs merge when you point at a directory.

Is there a Windows version?

Not yet. Bumblebee v0.1.1 supports macOS and Linux only. Windows support would require rewriting the scan profile filesystem paths and adding support for Windows-specific package manager locations (AppData, ProgramFiles).

Sources

Perplexity Bumblebee GitHub repository: source code, documentation, and release notes
MarkTechPost coverage of Bumblebee release: independent analysis of the tool's architecture and purpose
Sonatype 2026 State of the Software Supply Chain: 454,600 malicious packages in 2025, 1.233M cumulative
MCP Security in 2026: Tool Poisoning and Supply Chain Risks: analysis of MCP-specific attack vectors
Supply Chain Attacks 2026: npm, PyPI, VS Code, AI Agents: attack campaign tracking and 2.3x AI-agent skill risk data
Bumblebee vs OSV-Scanner comparison on Dev.to: positioning against other scanning tools

Bottom Line

Bumblebee does one thing and does it well: it inventories every package, extension, and MCP config on a developer machine without executing anything. That narrow scope is a feature. 454,600 malicious packages shipped in a single year. AI coding tools introduced a new attack surface nobody was scanning. A sub-4-second read-only answer to "are we exposed?" is worth adding to your toolkit.

The version range limitation is real and will bite you on the first advisory that specifies a range instead of exact versions. The missing Windows support matters if you have a mixed fleet. But for Unix-based development teams using Claude Code, Cursor, or VS Code, Bumblebee fills a gap that Snyk, Socket, and npm audit don't touch: what's actually sitting on the developer's disk right now, including the MCP configs that no other tool bothers to read.

Install it. Run the self-test. Point it at your home directory. The scan takes less time than reading this sentence, and you might not like what it finds.

Is There a Backend-as-a-Service for AI Coding Agents?

Maksim Danilchenko — Fri, 26 Jun 2026 14:15:13 +0000

TL;DR: Yes. A handful of backends now target AI coding agents directly. InsForge, Base44, Butterbase, and MoonDB are among them. What sets them apart from Supabase or Firebase is that an agent can provision the whole thing from code: a JSON schema in, a hosted database with auth and a REST API out, no dashboard click-through. With MoonDB that's three calls.

What "designed for AI coding agents" actually means

Most backends were built for a person. You sign in, click to create tables, wire up auth in a settings panel, copy keys out of a dashboard, and paste them into your app. An agent like Cursor or Claude Code can't click. Drop one into that flow and it stalls at the first screen that needs a human.

An agent-native backend removes the human step. The whole setup runs over HTTP or through an MCP server, so the agent does the work end to end. Three things make that possible, and they're the test for whether a backend really fits an agent:

It provisions from code. Creating a project, applying a schema, and reading the resulting keys all happen through API calls that return JSON. Nothing waits on a console.

It's declarative. The agent writes what the data looks like: tables, fields, relationships, and who can read and write each one. The service figures out the SQL and runs the migration. No hand-written DDL, no guessing at migration order.

Its errors are instructions. When a schema is wrong, the response says what to fix, not just that something broke. An agent reads that and corrects itself on the next call instead of looping.

Supabase and Firebase can be scripted, and people do script them. But the happy path still runs through a dashboard, the client libraries assume a build step, and the error messages are written for a developer reading a stack trace. In the newer tools, the agent is the primary user and the human reviews the result.

The backends built for this, as of 2026

The category is small, and it already has a name: "agent-native" backends. A few worth knowing:

InsForge is an open-source backend that bundles a stack of services behind a semantic layer an agent can read and provision: Postgres with auto-generated APIs, auth, S3-compatible storage, edge functions, compute and hosting, and an AI model gateway. The source is on GitHub under Apache-2.0.

Base44 leans on its CLI, which it tunes for Claude Code and Cursor so an agent can scaffold backend logic, data models, and APIs from a prompt.

Butterbase is another open-source option: Postgres, auth, storage, functions, an AI gateway, and an MCP server.

Xano comes at it from the no-code side. It's a visual backend builder that's added AI features for generating APIs and logic.

MoonDB sits at the minimal end of that list, and on purpose. It does one thing: turn a JSON schema into a hosted database with a REST API, auth, and file storage, all reachable over plain HTTP or MCP. There's no vector store, no edge-function runtime, no AI gateway. If your generated app needs RAG or a serverless compute layer, InsForge or Butterbase carry more out of the box. If it needs a database with auth and a clean API that an agent can stand up in seconds and a person can actually understand, the smaller surface is the point.

The honest framing: on a feature checklist, Supabase wins. The question that sorts this category is which backend an agent can drive to "done" without a human unblocking it, and how little is left to misunderstand once it's running.

Provision a MoonDB backend the way an agent would

Here's the full path an agent follows to back a generated app, with a to-do app as the canonical first build. Three calls, no dashboard.

Step 1: create a project

One call returns the keys the rest of the flow needs.

curl -X POST https://moondb.ai/v1/projects \
  -H "X-API-Key: mk_your_account_key" \
  -H "Content-Type: application/json" \
  -d '{ "name": "todo-app" }'

{ "data": { "id": "p_7k2d", "admin_key": "sk_...", "public_key": "pk_..." } }

The agent reads three keys from the JSON: an account key that manages projects, an admin key that changes the schema, and a public key that's safe to ship in client code.

The agent keeps the admin_key server-side and uses the public_key for anything that runs in a browser. No human copies anything out of a console.

Step 2: send the schema

The agent describes the data as one JSON document and applies it with a single PUT. A to-do app needs users and per-user todos, where each person only sees their own list.

curl -X PUT https://moondb.ai/p/p_7k2d/v1/schema \
  -H "X-Admin-Key: sk_..." \
  -H "Content-Type: application/json" \
  -d @schema.json

{
  "tables": {
    "users": {
      "auth_table": true,
      "columns": { "display_name": "string" }
    },
    "todos": {
      "columns": {
        "title": "string required max_length 200",
        "done": "bool default false",
        "due_date": "date",
        "priority": { "type": "enum", "values": ["low", "medium", "high"], "default": "medium" },
        "user_id": "ref users required"
      },
      "owner_field": "user_id",
      "access": { "read": "owner", "create": "authenticated", "update": "owner", "delete": "owner" }
    }
  }
}

That document carries the whole security model. "auth_table": true makes users the login table, so MoonDB adds a hidden, required password_hash column and serves signup and login endpoints. The agent never declares the password field. "ref users required" ties each todo to a user. The owner_field with "read": "owner" scopes every read to the logged-in user, so one person's todos never leak into another's response, and the agent didn't write a line of authorization code to get that. create: authenticated lets any signed-in user add a todo. MoonDB also adds id, created_at, and updated_at to every table on its own.

MoonDB reads that, creates the tables, and serves the API immediately. Change the schema later and the agent sends new JSON; MoonDB diffs the two versions and migrates, and it refuses to run a destructive change unless the request opts in. That guardrail keeps an over-eager agent from dropping a column full of data.

The dashboard renders what the schema produced. An agent never opens this. It's here for the human to check the work.

Step 3: use the generated API

The endpoints exist the moment the schema applies. The agent signs up an end user, gets a token back, and writes a todo as that user.

# Create an end user
curl -X POST https://moondb.ai/p/p_7k2d/auth/signup \
  -H "Content-Type: application/json" \
  -d '{ "email": "sam@example.com", "password": "s3cret-pass", "display_name": "Sam" }'
# → { "data": { "user": {...}, "token": "<jwt>", "refresh_token": "...", "expires_in": 3600 } }

# Add a todo as that user
curl -X POST https://moondb.ai/p/p_7k2d/api/todos \
  -H "Authorization: Bearer <jwt>" \
  -H "Content-Type: application/json" \
  -d '{ "title": "Ship the MVP", "priority": "high" }'

The todo lands with user_id set to Sam from the token, because the owner_field rule stamps ownership at write time. When Sam reads GET /p/p_7k2d/api/todos, the read: owner rule scopes the result to Sam's rows. The agent gets working auth and per-user data isolation out of the schema, not out of a pile of middleware it had to write and test.

That's the loop a coding agent can run unattended: create, describe, write. For an agent that speaks Model Context Protocol, MoonDB exposes the same provisioning as MCP tools, so the steps above become tool calls inside the agent's session instead of shell commands.

How the agent-native backends compare

	Supabase / Firebase	InsForge / Butterbase	MoonDB
Built for	A developer at a dashboard	AI coding agents	AI coding agents
How an agent provisions it	Scripted around a dashboard flow	CLI + MCP	Three HTTP calls or MCP
Schema source	SQL, dashboard, or migrations	Declarative + AI generation	One JSON document
Database	Postgres	Postgres	SQLite on Cloudflare D1
Extra services	Many	Vector DB, edge functions, AI gateway	Auth and file storage only
Access rules	Row-level security policies	Per-service config	In the schema (`owner` / `authenticated`)

Firebase and Supabase are the right call when you want their depth and don't mind that an agent needs hand-holding to drive them. InsForge and Butterbase fit when a generated app needs vector search or serverless functions alongside the database. MoonDB fits when the job is a database with auth and a REST API, and the thing you care about is that an agent can stand it up in one pass and a person can read the whole schema in one screen.

Where MoonDB stops

Be clear-eyed about the trade. MoonDB runs on SQLite through Cloudflare D1, which buys fast reads at the edge and a model with almost nothing to configure. It also sets the ceiling: one writer per database and a 10 GB cap. That holds a to-do app, a CRM for a small team, or a content site with room to spare, but a write-heavy system with thousands of concurrent writers wants a dedicated Postgres. If you're weighing the two, here's how MoonDB compares to Supabase.

It's also not a vector database. If your app is doing retrieval-augmented generation and needs embeddings stored next to your data, that's a real reason to look at InsForge's vector support or a dedicated vector DB instead. MoonDB stores structured application data, the rows your generated app reads and writes, and leaves RAG to tools built for it.

For the range most agent-built apps start in, a schema an agent can write and a REST API it can call beats a feature list it can't reach. The schema and API reference covers every field type, and the walkthrough on generating a REST API from a JSON schema goes deeper on the format. If you're comparing agent-facing database tools specifically, the rundown of database MCP servers covers that angle.

FAQ

Is there a backend-as-a-service designed for AI coding agents?

Yes. Several launched recently: InsForge, Base44, Butterbase, and MoonDB among them. They differ from Supabase or Firebase in one concrete way: an agent can provision the whole backend from code, with no dashboard step. You hand over a schema and get a live database, auth, and a REST API back.

What makes a backend agent-native instead of a normal BaaS?

A normal BaaS assumes a person clicking through a dashboard to make tables and copy keys. An agent-native one is driven entirely by API: the agent sends a declarative schema, reads keys from the JSON response, and gets errors written as instructions it can act on. No human step sits in the middle.

Can a coding agent like Cursor or Claude Code set up a backend by itself?

Yes. If the backend exposes provisioning over HTTP or an MCP server, the agent can create a project, apply a schema, and start writing data without you touching a console. MoonDB does this in three calls; tools like Base44 and InsForge expose a CLI or MCP server aimed at Claude Code and Cursor.

Do I still need to write SQL or use a dashboard?

No. With a declarative backend you describe tables, fields, and access rules as JSON, and the service compiles that to SQL and runs the migration. The dashboard becomes optional. It's useful for watching what the agent built, but you don't need it to build anything.

Sources

InsForge and its GitHub repo: open-source agent-native backend
Base44 backend: backend with a CLI tuned for Claude Code and Cursor
Butterbase: open-source BaaS with MCP
Xano: best backends for AI-generated apps: the no-code vendor's take on the category
MoonDB documentation: the schema format and full API reference

GLM-5.2 Review: 753B Open-Weight Model That Undercuts GPT-5.5

Maksim Danilchenko — Thu, 18 Jun 2026 08:42:42 +0000

TL;DR

Z.ai's GLM-5.2 is a 753-billion-parameter mixture-of-experts model released under an MIT license on June 16, 2026. It tops GPT-5.5 on SWE-bench Pro (62.1 vs 58.6) and trails Claude Opus 4.8 by about 0.7 points on FrontierSWE (74.4% vs 75.1%). Through OpenRouter, it runs at $1.40 per million input tokens, about 72% cheaper than Claude and GPT. The catch: if you use Z.ai's hosted API instead of self-hosting the weights, your prompts route through servers governed by China's National Intelligence Law.

What GLM-5.2 Actually Is

GLM-5.2 comes from Z.ai (formerly Zhipu AI), one of China's largest AI labs and one of the few on the US Bureau of Industry and Security's Entity List. The model launched on June 13 for paying subscribers, then dropped its full open weights on Hugging Face three days later at zai-org/GLM-5.2.

The architecture is a mixture-of-experts (MoE) transformer: 753 billion total parameters, but only about 40 billion active per forward pass. That MoE structure is why it can match or beat dense models with far fewer FLOPs per token. It supports a 1-million-token context window (a 5x jump from GLM-5.1's 200K) and can produce up to 128K tokens in a single response.

I spent the last day running it through OpenRouter on three personal projects: a Flask API refactor, a Go CLI tool with nested subcommands, and a long context window test stuffing 400K tokens of a monorepo into the prompt. Not cherry-picked benchmarks, just the kind of work I do daily with Claude Code and occasionally Codex.

The first thing I noticed: GLM-5.2 is verbose. It used roughly 43K output tokens per task in Artificial Analysis's benchmark suite, compared to 26K for GLM-5.1. That verbosity inflates your bill if you're paying per token, and it slows down iteration when you're waiting for a response. On OpenRouter, a typical coding task took around 75 seconds, which felt slow compared to Opus 4.8's sub-30-second responses for similar complexity.

But the code it wrote was clean. A Python web scraper function came back production-ready on the first try: proper error handling, retries with exponential backoff, typed return values. The Go CLI output was similarly solid: correct cobra subcommand nesting, help text, and flag parsing without the usual LLM quirk of inventing nonexistent stdlib packages.

Benchmark Breakdown

The full picture, pulling from Artificial Analysis, BenchLM, and Z.ai's published results:

Benchmark	GLM-5.2	Claude Opus 4.8	GPT-5.5	DeepSeek V4 Pro
SWE-bench Pro	62.1	69.2	58.6	55.4
Terminal-Bench 2.1	81.0	85.0	—	—
FrontierSWE	74.4%	75.1%	72.6%	—
AA Intelligence Index	51	—	—	44
GDPval-AA v2 (Agentic)	1524	—	1514	1328
GPQA Diamond	89%	—	—	—
HLE	40%	—	—	—

A few things jump out from this table.

GLM-5.2 beats GPT-5.5 on every coding benchmark where both have scores (see our GPT-5.5 review for the full breakdown on that model). The SWE-bench Pro gap (62.1 vs 58.6) is real. That's 3.5 points on a benchmark where single-digit gaps separate model generations. On FrontierSWE, GLM-5.2 lands at 74.4%, within 0.7% of Opus 4.8's 75.1% (we covered Fable 5's benchmark claims recently). A 0.7% gap for a model that costs 80% less.

The agentic score (GDPval-AA v2) stands out: 1524 vs GPT-5.5's 1514. GLM-5.2 is built for long-horizon agent workflows where the model needs to plan across files, run commands, and iterate. Z.ai specifically pitched it as a coding-agent model, and the benchmarks back that framing.

Where it falls short: hallucination rate sits at 28.1% on the AA-Omniscience Index. That's an improvement over GLM-5.1's 29.4%, but it means roughly one in four factual claims from the model is wrong. For coding tasks this is less of a concern (the compiler catches lies), but for anything research-heavy or fact-dependent, you'll want external verification.

Pricing: The Real Reason to Pay Attention

Pricing changes how you actually use a model more than benchmarks do.

Model	Input (per 1M)	Output (per 1M)	Approx. cost/task	License
GLM-5.2 (OpenRouter)	$1.40	$4.40	~$0.46	MIT
Claude Opus 4.8	$5.00	$25.00	~$2.50	Proprietary
GPT-5.5	$5.00	$30.00	~$3.00	Proprietary
DeepSeek V4 Pro	$0.44	$0.87	~$0.05	DeepSeek
GLM-5.1	~$1.00	~$3.00	~$0.25	MIT

GLM-5.2 isn't the cheapest model in this table. DeepSeek V4 Pro undercuts it by 10x. But DeepSeek V4 Pro also scores significantly lower on every coding benchmark. The interesting position is GLM-5.2 vs the proprietary frontier: you get 95-100% of Claude Opus 4.8's coding performance for roughly 18% of the price.

The caveat is token consumption. GLM-5.2 burns through 43K output tokens per benchmark task, with 37K of those being internal reasoning tokens. You're billed for the reasoning tokens even though they don't appear in the visible output. That inflates the per-task cost from what the raw price-per-million suggests. At ~$0.46 per coding task, it's still cheap against proprietary options, but it's almost double GLM-5.1's ~$0.25.

For personal projects and prototyping, the math is obvious: GLM-5.2 through OpenRouter gives you frontier-adjacent coding quality at indie-developer prices (compare this with the Claude Code Pro vs Max pricing breakdown). For production agent pipelines processing thousands of tasks, the token verbosity starts to add up and DeepSeek V4 Pro's cost advantage gets harder to ignore.

The 1-Million-Token Context Window

GLM-5.1 topped out at 200K tokens. GLM-5.2 jumps to 1 million, matching Claude Opus 4.8 and GPT-5.5's API context window.

I tested this by feeding it roughly 400K tokens of a monorepo: the entire src/ directory of a mid-size Flask application with about 180 files. I asked it to trace a specific request flow through three microservices, identify where a race condition could happen, and propose a fix.

It handled the context without obvious degradation. The trace was accurate, it identified the correct database transaction that lacked proper isolation, and the fix was structurally sound. Whether it would hold up at 800K+ tokens I can't say. I didn't have a codebase that large on hand to test with.

Z.ai specifically designed GLM-5.2's IndexShare sparse-attention mechanism for long agent trajectories. The idea is that coding agents accumulate hundreds of thousands of tokens over a multi-step session: file reads, command outputs, error traces, iterative fixes. A model that degrades at 200K would force the agent to constanly prune context or restart. At 1M, the agent can carry the full project state through a long session without losing earlier context.

How to Use GLM-5.2

Three paths, each with different trade-offs.

OpenRouter (quickest start)

The fastest way to try GLM-5.2. Nine providers offer it on OpenRouter, all at roughly $1.40/$4.40 per million tokens. The API is OpenAI-compatible:

from openai import OpenAI

client = OpenAI(
    base_url="https://openrouter.ai/api/v1",
    api_key="your-openrouter-key",
)

response = client.chat.completions.create(
    model="z-ai/glm-5.2",
    messages=[
        {"role": "user", "content": "Refactor this Flask route to use async SQLAlchemy 2.0 sessions"}
    ],
    temperature=0.6,
    max_tokens=4096,
)
print(response.choices[0].message.content)

Z.ai's API (official, but read the data section below)

Z.ai's own API supports OpenAI SDK compatibility and adds a Coding Plan tier that's compatible with Claude Code, Cline, and Cursor. Point your tools at the coding endpoint:

export ANTHROPIC_BASE_URL="https://api.z.ai/api/coding/paas/v4"
export ANTHROPIC_DEFAULT_SONNET_MODEL="glm-5.2[1m]"

That gives you GLM-5.2 as a drop-in replacement in Claude Code sessions. I tried this with a small Go project and it worked. Claude Code's agent loop ran normally, though response times were noticeably slower than Anthropic's own endpoints.

Self-Hosting (full control, big hardware)

The weights are on Hugging Face under MIT. You can serve them with vLLM or SGLang on your own GPUs. The hardware requirement is steep: at FP8 quantization, you need about 753 GB of VRAM. That's ten H100 80GB GPUs (800 GB total) or a smaller cluster of H200s. At BF16, the requirement doubles to ~1,500 GB.

Self-hosting makes sense for two audiences: enterprises that can't send code to external APIs for compliance reasons, and AI labs that want to fine-tune the weights for specialized domains. For everyone else, OpenRouter is simpler.

# Example vLLM serving command (8x H100)
python -m vllm.entrypoints.openai.api_server \
    --model zai-org/GLM-5.2 \
    --tensor-parallel-size 8 \
    --dtype float8 \
    --max-model-len 1048576

The China Data Question

Z.ai is headquartered in Beijing. The US Bureau of Industry and Security added Zhipu AI (Z.ai's former name) to its Entity List in January 2025, citing the company's role in advancing military AI modernization. In late April 2026, US House lawmakers opened a formal inquiry into cybersecurity risks posed by Chinese AI models in critical infrastructure, naming Z.ai alongside DeepSeek, MiniMax, ByteDance, and several others.

The risk depends entirely on the serving path.

If you use Z.ai's hosted API, your prompts and code route through servers subject to China's National Intelligence Law, which requires Chinese companies to cooperate with government data requests. For regulated industries (healthcare, finance, defense, government contracting), this is a non-starter. For any codebase containing proprietary algorithms, API keys, customer data, or anything you wouldn't post publicly, routing through Z.ai's API is a risk most security teams won't approve.

If you self-host or use a Western OpenRouter provider, the data never touches Z.ai's servers. The MIT license has no phone-home requirements, no telemetry obligations, no usage restrictions. You download the weights, serve them on your own infrastructure, and Z.ai has no visibility into what you're doing. This is the scenario where the open-weights value proposition fully pays off.

If you use OpenRouter, check which provider is serving your request. Most OpenRouter providers for GLM-5.2 route through non-Chinese infrastructure, but the routing can vary. Verify with your specific provider if this matters for your compliance requirements.

The practical upshot: GLM-5.2's China connection is a non-issue if you self-host, a manageable concern on most OpenRouter providers, and a hard blocker if you'd be sending sensitive code directly to Z.ai. The MIT license exists precisely to decouple the model's capabilities from the company's jurisdiction.

Who Should Use GLM-5.2

Good fit:

Independent developers and small teams who want near-frontier coding quality at $1.40/M input instead of $5.00/M. The cost difference compounds fast when you're running agent loops that process dozens of files.
Teams already invested in open-weight infrastructure (vLLM clusters, self-hosted inference). GLM-5.2 slots into that stack with no vendor lock-in and no API dependency.
Long-context use cases where the 1M window matters: codebase-wide refactors, multi-file agents, repository Q&A.
Anyone who needs to fine-tune a frontier-class coding model for a specific domain. MIT license means no restrictions.

Bad fit:

Enterprise teams in regulated industries that can't risk any data routing through Chinese infrastructure, unless they have the GPU budget to self-host.
Latency-sensitive production pipelines. At ~75 seconds per coding task through OpenRouter, GLM-5.2 is sluggish compared to Opus 4.8 or GPT-5.5's APIs.
Tasks requiring low hallucination rates. The 28.1% factual error rate is acceptable for code (where tests catch mistakes) but rough for content generation, research synthesis, or customer-facing text.
Teams that need multimodal capabilities. GLM-5.2 is text-only. Z.ai's vision model (GLM-5V-Turbo) exists separately and isn't open-weights.

FAQ

Is GLM-5.2 better than Claude Opus 4.8 for coding?

Not quite. On SWE-bench Pro, Opus 4.8 scores 69.2 vs GLM-5.2's 62.1, a 7-point gap. On FrontierSWE, it narrows to 75.1% vs 74.4%. Opus 4.8 is faster, less verbose, and has lower hallucination rates. But it costs 3-5x more per token. Whether "better" means "higher score" or "better value" depends on your budget.

Can I run GLM-5.2 locally?

Technically yes, if you have ~753 GB of VRAM (ten H100 80GB GPUs at FP8). For most developers, local running isn't practical. Cloud hosting through a managed provider or OpenRouter is the realistic path.

Is GLM-5.2 safe to use with proprietary code?

It depends on the serving path. Self-hosted or through a trusted Western provider on OpenRouter — yes, your data doesn't leave your infrastructure. Through Z.ai's own API — your prompts traverse Chinese servers governed by the National Intelligence Law. Most enterprise security teams will block the latter.

How does GLM-5.2 compare to DeepSeek V4 Pro?

DeepSeek V4 Pro is ~10x cheaper ($0.44/$0.87 per million tokens) as we detailed in our DeepSeek V4 Pro review, but scores lower on coding benchmarks — 55.4 on SWE-bench Pro vs GLM-5.2's 62.1. DeepSeek wins on cost, GLM-5.2 wins on capability. Both carry similar China-data concerns if used through their respective hosted APIs.

Why is GLM-5.2 so verbose?

The model produces about 43K output tokens per coding task, with 37K being internal reasoning tokens that get billed but don't appear in the response. Deeper reasoning chains improve accuracy on complex tasks, but they inflate costs and latency. Z.ai hasn't offered a "low verbosity" mode that trades some accuracy for speed.

Sources

Z.ai GLM-5.2 documentation — official specs, API reference, and capability overview
Simon Willison's analysis of GLM-5.2 — independent benchmarks and observations from the day of release
Artificial Analysis: GLM-5.2 is the new leading open-weights model — Intelligence Index v4.1 scores, GDPval-AA agentic benchmarks, and pricing data
VentureBeat: Z.ai's GLM-5.2 beats GPT-5.5 on multiple benchmarks — SWE-bench Pro and FrontierSWE comparisons
OpenRouter GLM-5.2 listing — live pricing and provider availability
GLM-5.2 weights on Hugging Face — MIT-licensed open weights

Bottom Line

GLM-5.2 is the strongest open-weights coding model available today. At 62.1 on SWE-bench Pro, it beats GPT-5.5 and closes much of the gap to Claude Opus 4.8's 69.2, while costing a fraction of either. The MIT license means you can self-host it, fine-tune it, and deploy it in air-gapped environments.

The trade-offs are real. It's slow (75 seconds per task on OpenRouter), verbose (burning 43K tokens when 20K might suffice), and the 28.1% hallucination rate means you can't trust it for factual content without verification. The China data question adds a layer: if you can't self-host and your compliance posture rules out Chinese API endpoints, you're limited to OpenRouter's Western providers.

For non-sensitive coding work where I'm paying out of pocket, GLM-5.2 through OpenRouter at $1.40/M input is the best value in frontier AI right now. It wrote clean Python and Go on the first pass, handled 400K tokens of context without degradation, and saved me about 80% compared to my usual Opus 4.8 API costs. Open-weights coding models have crossed from curiosity to credible daily driver.

LiteLLM Vulnerability: 6 CVEs, a Supply Chain Attack, and the Fixes

Maksim Danilchenko — Wed, 17 Jun 2026 08:47:51 +0000

TL;DR

LiteLLM, the AI gateway proxy used by CrewAI, DSPy, Microsoft GraphRAG, and roughly 97 million monthly PyPI installs, has been hit by six CVEs and a full supply chain compromise since March 2026. The worst chain scores CVSS 10.0 and gives unauthenticated attackers remote code execution on any exposed LiteLLM proxy. CISA added one of the CVEs to its Known Exploited Vulnerabilities catalog on June 8 with a June 22 remediation deadline. If you run LiteLLM, upgrade to v1.83.14+ and rotate every credential the proxy can reach.

What LiteLLM Does and Why You're Probably Running It

LiteLLM is an open-source Python proxy that gives you a single OpenAI-compatible API in front of 100+ LLM providers: OpenAI, Anthropic, Google, Azure, Bedrock, Cohere, and more. You point your application at the proxy, the proxy routes requests to the right provider, and you get cost tracking, load balancing, rate limiting, and guardrails for free. About 50,300 GitHub stars and 97 million PyPI downloads a month as of June 2026.

I ran a LiteLLM proxy for three months last year to route Claude and GPT calls from a multi-agent pipeline. The setup took about 15 minutes: a Docker container, a YAML config with my API keys, and a litellm --config config.yaml command. That simplicity is exactly the problem. LiteLLM sits between your entire AI stack and every model provider you use. Every API key, every prompt, every response flows through it. When the proxy gets owned, the attacker gets everything — and AI tooling already leaks secrets at twice the rate of manual code.

And in 2026, LiteLLM got owned from every angle.

Every Vulnerability at a Glance

CVE	Type	CVSS	Affected Versions	Fixed In
CVE-2026-42271	Command injection (MCP endpoints)	8.8 (10.0 chained)	1.74.2 – 1.83.6	v1.83.7
CVE-2026-48710	Starlette host header bypass	Enabler	Starlette ≤1.0.0	Starlette 1.0.1
CVE-2026-42208	Pre-auth SQL injection	9.8	1.81.16 – 1.83.6	v1.83.7
CVE-2026-47101	Auth bypass (allowed_routes)	8.8 (9.9 chained)	<1.83.14	v1.83.14
CVE-2026-47102	Privilege escalation (user_role)	Part of 9.9 chain	<1.83.14	v1.83.14
CVE-2026-40217	Sandbox escape (exec guardrails)	8.8	<1.83.14	v1.83.14

Plus a full supply chain attack on March 24 that injected credential-stealing malware into PyPI packages v1.82.7 and v1.82.8.

CVE-2026-42271: Remote Code Execution via MCP Test Endpoints

This is the one CISA flagged on June 8 with a June 22 remediation deadline.

LiteLLM versions 1.74.2 through 1.83.6 included two MCP server preview endpoints: POST /mcp-rest/test/connection and POST /mcp-rest/test/tools/list. These accepted a full server configuration (command, arguments, and environment variables for stdio transport) and LiteLLM spawned the supplied command as a subprocess on the host. Any authenticated user with a standard API key could reach them.

That alone scores CVSS 8.8. But Horizon3.ai researchers chained it with CVE-2026-48710, a host header validation bypass in Starlette versions ≤1.0.0. The Starlette bug lets attackers bypass LiteLLM's authentication layer entirely by sending a malformed Host header. No API key, no login, no prior access needed.

The combined chain scores CVSS 10.0 and gives any internet-connected attacker unauthenticated RCE on the proxy host.

# This is what the exploit looks like (simplified for illustration)
# The attacker sends a crafted Host header to bypass auth,
# then passes arbitrary commands via the MCP test endpoint
curl -X POST https://target-litellm:4000/mcp-rest/test/connection \
  -H "Host: badhost" \
  -H "Content-Type: application/json" \
  -d '{"server_name": "evil", "transport": "stdio",
       "command": "/bin/sh", "args": ["-c", "id; cat /proc/self/environ"]}'

Horizon3.ai validated the full chain on June 1. CISA confirmed active exploitation in the wild.

The Privilege Escalation Chain: Three CVEs, One Path to Root

Obsidian Security disclosed this chain on June 11, and it's the scarier find because it starts from a default low-privilege account that every LiteLLM deployment has.

Step 1 — CVE-2026-47101 (Auth bypass): A standard internal_user calls /key/generate with allowed_routes: ["/*"]. LiteLLM validates that the user_id matches the caller but never checks what routes are being granted. The user now holds a wildcard API key that bypasses all role-based access controls.

Step 2 — CVE-2026-47102 (Privilege escalation): The wildcard key accesses /user/update. This endpoint verifies identity but has zero field-level authorization. The attacker sets user_role: "proxy_admin" on their own account and becomes a full admin.

Step 3 — CVE-2026-40217 (Sandbox escape to RCE): Admin-level guardrail endpoints accept user-supplied Python code and compile it with exec(). The globals dictionary omits __builtins__, but Python automatically injects the full builtins module when that key is missing. One line gets you shell access:

# The "sandboxed" guardrail code that actually has full system access
__builtins__['__import__']('os').system('cat /etc/shadow')

Obsidian rates the full chain CVSS 9.9. A server takeover exposes every provider key, every stored credential, every prompt and response passing through the gateway. They demonstrated a "man-in-the-gateway" attack that injected fake tool calls into Claude Code responses, causing it to execute attacker-controlled commands while evading auto-mode safety checks.

CVE-2026-42208: SQL Injection in the Auth Layer

Bishop Fox found this one: a pre-authentication SQL injection via the Authorization: Bearer header. Versions 1.81.16 through 1.83.6 build the token verification query with a Python f-string:

# The vulnerable code in litellm/proxy/utils.py
query = f"WHERE v.token = '{token}'"

LiteLLM's _hash_token_if_needed() only hashes values starting with sk-. Non-prefixed bearer tokens pass straight to the f-string. An attacker can extract the entire LiteLLM_VerificationToken table — all virtual keys, team bindings, spend records — via timing-based blind injection with pg_sleep().

Default deployments run with database superuser privileges, so the blast radius extends to every table in the database. Sysdig observed in-the-wild exploitation within 36 hours of the GitHub advisory going public, with attackers targeting litellm_credentials.credential_values to steal upstream LLM provider keys.

The March Supply Chain Attack

On March 24, a threat actor called TeamPCP published two backdoored versions of LiteLLM (v1.82.7 and v1.82.8) to PyPI.

The attack chain started weeks earlier. TeamPCP exploited a pull_request_target workflow vulnerability in Trivy (the container security scanner), hijacked the aqua-bot GitHub Actions credentials, and used them to compromise Checkmarx KICS GitHub Actions. When LiteLLM's CI/CD pipeline ran the compromised Trivy scanner during a build, the malicious action extracted LiteLLM's PYPI_PUBLISH token from the GitHub Actions environment. CI/CD supply chain risks like this are growing across the ecosystem.

The malicious payload installed a .pth file that fired on every Python interpreter startup — no import required. It harvested environment variables, SSH keys, .env files, API tokens, AWS/GCP/Azure credentials, Kubernetes configs, and database credentials. Everything was encrypted with AES-256-CBC and RSA-4096, bundled into a tarball, and exfiltrated to models.litellm[.]cloud. It also deployed persistence via a fake sysmon.service systemd unit and, in Kubernetes environments, spawned privileged pods on every node.

PyPI quarantined the packages within about 3 hours. But Microsoft GraphRAG, Google ADK, DSPy, CrewAI, and OpenHands all pulled the malicious version as a transitive dependency during that window. Datadog Security Labs later linked this to a broader TeamPCP campaign that also hit npm packages and OpenVSX extensions.

How to Check If You're Affected

Run these checks in order:

# 1. Check your LiteLLM version
pip show litellm | grep Version
# or inside Docker:
docker exec <container> pip show litellm | grep Version

# 2. Check your Starlette version (for the auth bypass chain)
pip show starlette | grep Version

# 3. Check if the vulnerable MCP endpoints exist
curl -s -o /dev/null -w "%{http_code}" \
  http://localhost:4000/mcp-rest/test/connection
# 404 = not vulnerable (endpoint removed). 405 or 200 = vulnerable.

# 4. Check for supply chain attack artifacts
find / -name "litellm_init.pth" 2>/dev/null
ls -la ~/.config/sysmon/sysmon.py 2>/dev/null
systemctl status sysmon.service 2>/dev/null

If your LiteLLM version is below 1.83.14, you're exposed to at least some of these vulnerabilities. If it's between 1.82.7 and 1.82.8 and you installed between March 24 10:39 UTC and ~13:38 UTC, you may have the supply chain backdoor.

How to Fix Everything

Upgrade LiteLLM to at least v1.83.14 (v1.89.1 is the latest as of June 16):

# pip upgrade
pip install --upgrade "litellm>=1.83.14"

# Docker — pull the latest stable image
docker pull ghcr.io/berriai/litellm:main-latest

Upgrade Starlette to close the unauthenticated RCE path:

pip install --upgrade "starlette>=1.0.1"

Rotate credentials. If your proxy was ever internet-accessible on a vulnerable version, treat every secret it could reach as compromised:

# Rotate these (at minimum):
# - LITELLM_MASTER_KEY and LITELLM_SALT_KEY
# - DATABASE_URL and database passwords
# - Every LLM provider API key (OpenAI, Anthropic, Google, Azure, Bedrock)
# - MCP and agent OAuth tokens
# - SSH keys and cloud credentials (AWS, GCP, Azure)

# Check your logs for signs of exploitation:
grep -E "/mcp-rest/test/(connection|tools/list)" /var/log/litellm/*.log
grep -E "/user/(update|bulk_update).*user_role" /var/log/litellm/*.log

If you were hit by the supply chain attack, rebuild your containers from scratch. The backdoor persists through restarts via systemd and .pth files.

Can't upgrade immediately? These temporary mitigations buy time:

# Block the vulnerable endpoints at your reverse proxy (nginx example)
location ~ ^/mcp-rest/test/ {
    deny all;
    return 403;
}

# Block non-standard bearer tokens to mitigate SQL injection
# Only allow tokens matching the sk- prefix pattern

What Went Wrong, and What It Means for Your AI Stack

LiteLLM's security problems aren't unusual for a fast-growing open-source project. The f-string SQL query is a rookie mistake, but it's the kind of mistake that gets shipped when the priority is features, not security audits. The exec() sandbox with missing builtins is a well-known Python gotcha that's been documented since 2009.

The supply chain attack is worse. TeamPCP didn't need to find a bug in LiteLLM's code, and AI-driven vulnerability discovery means the attack surface is only growing. They compromised the security scanner that LiteLLM's CI pipeline trusted, then used that to steal PyPI publishing credentials. The attack took weeks to set up and targeted LiteLLM specifically because it sits at the chokepoint of thousands of AI deployments.

If you're running any AI gateway or proxy, the takeaway is straightforward: treat it like you'd treat your database. Don't expose it to the internet without authentication. Don't run it as root. Pin your dependency versions (a good package manager helps), verify checksums on deployment, and have a credential rotation plan that doesn't require a Saturday fire drill.

CrewAI published a removal guide for decoupling from LiteLLM entirely. Whether that's the right move depends on your threat model. LiteLLM has since engaged Veria Labs for a proxy audit, launched a bug bounty ($500–$3,000), and committed to 5-business-day response times on advisories. The project is taking security more seriously. Whether that's enough depends on how many more CVEs are sitting in the other 100,000+ lines of Python.

FAQ

What is CVE-2026-42271?

CVE-2026-42271 is a command injection vulnerability in LiteLLM's MCP REST test endpoints. Two preview endpoints (/mcp-rest/test/connection and /mcp-rest/test/tools/list) accepted full server configurations and spawned the supplied command as a subprocess. When chained with a Starlette host header bypass (CVE-2026-48710), it becomes an unauthenticated remote code execution vulnerability scoring CVSS 10.0.

What versions of LiteLLM are affected?

For the RCE and SQL injection vulnerabilities: versions 1.74.2 through 1.83.6. For the privilege escalation chain: all versions before 1.83.14. The supply chain attack only affected PyPI packages v1.82.7 and v1.82.8, published on March 24 and quarantined within 3 hours.

How to fix LiteLLM vulnerability?

Upgrade to LiteLLM v1.83.14 or later (v1.89.1 is the latest as of June 2026), upgrade Starlette to v1.0.1+, and rotate every credential the proxy had access to: LLM provider keys, database passwords, the master key, SSH keys, and cloud credentials.

Is LiteLLM safe to use in production?

After upgrading to v1.83.14+, the known vulnerabilities are patched. But LiteLLM has accumulated 16 CVEs across 2024–2026, including three Criticals. If you use it in production, run it behind a reverse proxy, restrict network access, disable unused features like Custom Code Guardrails, and monitor the GitHub advisory feed closely.

What happened with the LiteLLM supply chain attack?

On March 24, 2026, a threat actor called TeamPCP published backdoored LiteLLM versions 1.82.7 and 1.82.8 to PyPI. They got there by first compromising Trivy's GitHub Actions, then using that access to steal LiteLLM's PyPI publishing credentials from CI/CD. The malicious packages harvested every credential on the host machine and exfiltrated them to an attacker-controlled server. PyPI quarantined them within about 3 hours, but major frameworks like Microsoft GraphRAG and Google ADK pulled the malicious versions as transitive dependencies during that window.

How does the LiteLLM exploit chain work?

Multiple chains exist. The worst (CVSS 10.0) combines a command injection in MCP test endpoints with a Starlette host header bypass for unauthenticated RCE. A separate chain (CVSS 9.9) starts from a low-privilege default account: the attacker creates a wildcard API key via an unchecked allowed_routes field, self-promotes to admin by modifying their own user_role, then executes arbitrary Python through the Custom Code Guardrail's broken exec() sandbox.

Sources

Horizon3.ai — CVE-2026-42271 chained with CVE-2026-48710 — original research on the CVSS 10.0 RCE chain
The Hacker News — LiteLLM Vulnerability Chain Lets Low-Privilege Users Take Over — coverage of the Obsidian Security CVSS 9.9 privilege escalation disclosure
Help Net Security — LiteLLM vulnerability under active attack, CISA warns — CISA KEV announcement
Bishop Fox / LiteLLM — CVE-2026-42208 SQL Injection — official advisory for the pre-auth SQL injection
Sysdig — CVE-2026-42208 exploitation analysis — in-the-wild exploitation within 36 hours
Snyk — How a Poisoned Security Scanner Became the Key to Backdooring LiteLLM — technical breakdown of the supply chain attack
HeroDevs — The LiteLLM Supply Chain Attack — timeline and impact assessment
Datadog Security Labs — LiteLLM compromised PyPI TeamPCP campaign — broader TeamPCP campaign analysis
LiteLLM — Security Hardening April 2026 — official post-incident security improvements

Bottom Line

LiteLLM got hammered from every direction in 2026: supply chain poisoning, unauthenticated RCE, SQL injection in the auth layer, privilege escalation from default accounts. If you've been running an internet-exposed proxy on anything older than v1.83.7, assume breach and act accordingly. The CISA deadline is June 22 — five days from now.