DEV Community: Owen

Routing GLM-5.2, DeepSeek V4, MiniMax M3 & Kimi K2.6 Through One API (2026)

Owen — Tue, 23 Jun 2026 10:19:38 +0000

TL;DR — Put GLM-5.2, DeepSeek V4 (Pro and Flash), MiniMax M3, and Kimi K2.6 behind one ofox API key and route per task instead of paying one model's price for every job. Blended per-token cost at a 2:1 input-to-output mix ranges from $0.19/M (V4 Flash) to $2.40/M (GLM-5.2) — a 12.86x spread. A worked 1,000-job/day routing table below cuts a $4,205/mo all-GLM bill to $1,453 (-65.5%). The routing rule is short: budget/batch → V4 Flash, long-context (up to 1M tokens) → V4 Pro or GLM-5.2, reasoning/code → GLM-5.2 or Kimi K2.6, images → MiniMax M3 or Kimi K2.6. All four sit on the same OpenAI-compatible endpoint, so routing is a one-string change — Python and Node loops included.

The mistake teams make is picking one model and running everything through it. A batch summarization job and a hard reasoning task do not deserve the same per-token price. With one key across all four models, the cheapest tier costs 12.86x less than the most capable one — so the entire game is matching each job class to the cheapest model that clears its quality bar.

This is a how-to with reproducible cost math, not a "which router is best" roundup. Every number below comes from ofox's listed per-token rates verified June 23, 2026, and you can recompute each table from the spec sheet.

TL;DR: Which Model for Which Job?

One-line verdict: default your batch traffic to the cheapest tier and only escalate the jobs that need it. Here is the routing map by task shape.

Task shape	Route to	ofox model ID	Why
Budget / high-volume batch	DeepSeek V4 Flash	`deepseek/deepseek-v4-flash`	$0.19/M blended, 12.86x cheaper than GLM-5.2
Cost-sensitive general work	DeepSeek V4 Pro	`deepseek/deepseek-v4-pro`	$0.59/M blended, free cache reads, 1M context
Long-context (up to ~1M tokens)	V4 Pro or GLM-5.2	`deepseek/deepseek-v4-pro` / `z-ai/glm-5.2`	V4 Pro cheapest 1M input ($0.45/M); GLM-5.2 best reasoning at 1M
Hard reasoning / agentic coding	GLM-5.2 or Kimi K2.6	`z-ai/glm-5.2` / `moonshotai/kimi-k2.6`	Strongest reasoning tier; Kimi K2.6 multimodal alternative
Image input (vision tasks)	MiniMax M3 or Kimi K2.6	`minimax/minimax-m3` / `moonshotai/kimi-k2.6`	Only two of the four accept `image_url`; M3 is cheaper
Very long single output	DeepSeek V4 Pro/Flash	`deepseek/deepseek-v4-pro`	384K max output, highest of the four

The honest default for most 2026 teams: send the bulk of your traffic to deepseek/deepseek-v4-flash or deepseek/deepseek-v4-pro, escalate the genuinely hard reasoning to z-ai/glm-5.2, and send anything with an image to minimax/minimax-m3. That covers the realistic 90% of mixed workloads behind one key with no vendor migration.

Quick Specs Comparison

Verified against the ofox /v1/models catalog on June 23, 2026. Prices are per million tokens.

Spec	GLM-5.2	DeepSeek V4 Pro	DeepSeek V4 Flash	MiniMax M3	Kimi K2.6
ofox model ID	`z-ai/glm-5.2`	`deepseek/deepseek-v4-pro`	`deepseek/deepseek-v4-flash`	`minimax/minimax-m3`	`moonshotai/kimi-k2.6`
Context window	1,048,576	1,000,000	1,000,000	1,131,000	262,144
Max output	128,000	384,000	384,000	131,000	262,144
Input $/M	$1.40	$0.45	$0.14	$0.60	$0.95
Output $/M	$4.40	$0.88	$0.28	$2.40	$4.00
Cache read $/M	$0.26	~$0.00	~$0.00	$0.12	$0.16
Modality	text	text	text	text + image	text + image

Three structural facts drive every routing decision below:

DeepSeek V4 Flash is the price floor. At $0.14/$0.28 it is 12.86x cheaper blended than GLM-5.2. Anything that does not need top-tier reasoning starts here.
DeepSeek V4 cache reads are effectively free. Both V4 tiers bill cache reads at a rounding-to-zero rate, versus GLM-5.2's $0.26/M. On repeated-context workloads this is a large, often-overlooked saving.
Only MiniMax M3 and Kimi K2.6 take images. GLM-5.2 and both DeepSeek tiers are text-only. Vision tasks have exactly two valid routes, and MiniMax M3 is the cheaper of them.

Blended Cost: The Number That Drives Routing

A model's headline input price is half the story. What you pay depends on your input-to-output ratio. A coding agent reads a lot (large context) and writes a little (a diff) — roughly 2:1 input-to-output. Chat is closer to 1:1. Pure code generation from a short prompt is output-heavy, around 1:3.

Here is the blended cost per million tokens at the coding-typical 2:1 mix (two-thirds input, one-third output), and the multiplier against GLM-5.2 as the reasoning-tier anchor:

Model	Blended $/M (2:1)	vs GLM-5.2
DeepSeek V4 Flash	$0.187	12.86x cheaper
DeepSeek V4 Pro	$0.593	4.04x cheaper
MiniMax M3	$1.200	2.00x cheaper
Kimi K2.6	$1.967	1.22x cheaper
GLM-5.2	$2.400	1.00x (anchor)

Pull quote: The cheapest model on this list costs 12.86x less than the most capable one. That spread is the entire economic case for routing — not which model "wins," but which jobs can ride the cheap tier without anyone noticing.

The ranking shifts a little with workload shape. At 1:3 output-heavy (code generation), GLM-5.2 climbs to $3.65/M and Kimi K2.6 to $3.24/M, while V4 Flash stays at $0.245/M. Output-heavy work tilts even harder toward the DeepSeek tiers because their output token is the cheapest of the five. If you only remember one rule: the more your job writes, the more it pays to route off GLM-5.2 and Kimi K2.6.

If you want to stop estimating and measure these numbers on your own traffic, route all five models through one ofox key — pay-as-you-go, no monthly fee, same OpenAI SDK shape, and the A/B loop at the end of this post swaps models with a one-line string change.

Per-Task Cost: What One Agent Run Costs on Each Model

Routing decisions are easier to feel in per-run dollars than per-million-token rates. Take a representative agent run: 50,000 input tokens, 15,000 output tokens (read a chunk of a codebase, produce a change).

Model	Cost per run (50K in / 15K out)
DeepSeek V4 Flash	$0.0112
DeepSeek V4 Pro	$0.0357
MiniMax M3	$0.0660
Kimi K2.6	$0.1075
GLM-5.2	$0.1360

At 10,000 such runs a month, that is $112 on V4 Flash versus $1,360 on GLM-5.2 for the same work. If even half those runs are routine enough for the budget tier, the routing decision pays for itself many times over. The point is not that V4 Flash is always right — it is that paying GLM-5.2's price for a job V4 Flash could handle is pure waste.

The Routing Decision Matrix (Worked Example)

Here is the part most "use a router" articles skip: the actual daily math. Assume 1,000 mixed jobs per day with this realistic distribution:

Job class	Count/day	Tokens (in / out)	Routed to
Budget / batch	600	10K / 2K	DeepSeek V4 Flash
Long-context	250	300K / 8K	DeepSeek V4 Pro
Reasoning / code	100	40K / 12K	GLM-5.2
Multimodal (image)	50	16.5K / 3K	MiniMax M3

Run everything on GLM-5.2 (the one-model trap) versus routing each class to its cost-appropriate model:

Strategy	Daily cost	Monthly (×30)
All-GLM-5.2 baseline	$140.17	~$4,205
Routed	$48.42	~$1,453
Savings	$91.75/day	~$2,753/mo (-65.5%)

The breakdown of the routed total:

Job class	Model	Daily cost
Budget / batch (600)	V4 Flash	$1.18
Long-context (250)	V4 Pro	$35.51
Reasoning / code (100)	GLM-5.2	$10.88
Multimodal (50)	MiniMax M3	$0.85
Total		$48.42

The 600 batch jobs — 60% of volume — cost $1.18/day on V4 Flash. On GLM-5.2 the same 600 jobs would cost about $13.68/day — roughly 11.6× more. That single routing rule (cheap batch → V4 Flash) does most of the work. The long-context class is where the dollars actually concentrate, which is why the next section matters.

flowchart TD
    A[Incoming request] --> B{Needs image input?}
    B -->|Yes| C[minimax/minimax-m3]
    B -->|No| D{Hard reasoning<br/>or agentic coding?}
    D -->|Yes| E[z-ai/glm-5.2]
    D -->|No| F{Context > 200K<br/>tokens?}
    F -->|Yes| G[deepseek/deepseek-v4-pro<br/>free cache reads, 1M ctx]
    F -->|No| H[deepseek/deepseek-v4-flash<br/>cheapest tier]

Cache Reads: DeepSeek V4's Quiet Cost Advantage

The long-context class above is where caching changes the math. DeepSeek V4 Pro and Flash bill cache reads at effectively $0/M. GLM-5.2 bills them at $0.26/M, MiniMax M3 at $0.12/M, Kimi K2.6 at $0.16/M.

Take the 300K-input long-context job from the routing table (per-run cost includes 8K output), with 80% of the input served from cache (realistic for code-review loops where the same codebase context repeats across requests):

Model	No cache	80% input cache	Saving
DeepSeek V4 Pro	$0.1420	$0.0340	76.0%
GLM-5.2	$0.4552	$0.1816	60.1%

V4 Pro starts cheaper and saves a larger share, because its cache read rounds to zero while GLM-5.2 still pays $0.26/M on the cached portion. For any workload that re-sends the same long context — RAG over a fixed corpus, iterative code review, document Q&A — route to DeepSeek V4 Pro and the free cache read compounds. This is a routing input GLM-5.2's stronger reasoning does not always justify overriding.

Splitting the Reasoning Tier: GLM-5.2 vs Kimi K2.6

The routing matrix sends "hard reasoning / agentic coding" to GLM-5.2 or Kimi K2.6, and that "or" deserves a rule rather than a coin flip. Both are the expensive end of this lineup — GLM-5.2 at $1.40/$4.40, Kimi K2.6 at $0.95/$4.00 — and on a 2:1 mix Kimi K2.6 actually blends slightly cheaper ($1.97/M vs $2.40/M) because its input rate is lower. Three concrete factors decide the route:

Decision factor	Route to GLM-5.2	Route to Kimi K2.6
Context length needed	Up to 1,048,576 tokens	Caps at 262,144 — drop it for >256K jobs
Image input in the task	Not supported (text-only)	Supported (text + image)
Cheaper blended cost at 2:1	$2.40/M	$1.97/M (18% lower)
Max single output	128,000 tokens	262,144 tokens

The practical rule: if the reasoning job carries a large context (>256K tokens), GLM-5.2 is the only one of the two that fits — Kimi K2.6 will reject the input. If the context is comfortably under 256K and the job involves an image or wants the cheaper per-token rate, Kimi K2.6 is the better route. For most short-context agentic coding turns, Kimi K2.6's lower input price makes it the value pick inside the reasoning tier; reserve GLM-5.2 for the long-context reasoning that only its 1M window can hold. The Kimi K2.6 release guide covers its agentic behavior in more depth.

This is exactly why client-side routing beats locking to one model: the "best reasoning model" depends on the shape of the reasoning job, and a model string is the cheapest possible switch between them.

Latency and Throughput Are Routing Inputs Too

Cost is the loudest routing signal, but not the only one. Two operational notes that change real routing decisions:

Interactive vs batch. For a user-facing assistant where first-token latency is felt, the cheapest model is not automatically the right one — a slightly pricier model that returns faster can be worth it on the interactive surface, while overnight batch jobs should ride the cheapest tier regardless of speed. Route by surface, not just by price: interactive traffic tolerates a higher per-token cost, batch traffic does not.
Output ceiling as a hard constraint. If a single response must exceed 128,000 tokens — full-file rewrites, large structured exports — GLM-5.2 and MiniMax M3 cap out and the call truncates. Only the DeepSeek V4 tiers (384K) and Kimi K2.6 (262K) clear that bar in one call. This is a binary routing gate, not a cost trade-off: send oversized-output jobs to a model that can physically emit the tokens.

Both of these are decisions your pick_model function can encode as plain conditionals — surface type and expected output size are usually known at request time.

When NOT to Route (and What to Use Instead)

Routing is not free engineering. Three cases where a multi-model split is the wrong move:

Single developer, < 1,000 calls/day, all one task type. The routing logic and per-model quality testing cost more time than you save. Pick deepseek/deepseek-v4-pro as a strong, cheap default and move on. The $0.59/M blended cost is already low enough that micro-optimizing is not worth the branching code.
You actually need server-side automatic fusion. ofox routes by your model field — it does not auto-pick a model or fuse outputs. If you specifically want quality-based auto-selection or response fusion (the OpenRouter Auto / Sakana-style idea), that is a different product category. Use one of those tools, or read our honest review of whether OpenRouter is reliable before deciding the auto-router is worth the unpredictability.
Every job genuinely needs top-tier reasoning. If your traffic is 100% hard agentic coding with no budget-tier work, there is nothing to route — run GLM-5.2 (or Kimi K2.6) and skip the matrix. Routing only pays when your workload is mixed. For a pure two-model reasoning split, our Claude Code hybrid routing pattern covers that narrower case.

The routing payoff is proportional to how heterogeneous your traffic is. Homogeneous traffic → one model. Mixed traffic → the matrix above.

Try It via ofox: Route All Five in One Loop

All five models share https://api.ofox.ai/v1 and one ofox key. Routing is a client-side decision: you set the model field per request. Here is the routing function and an A/B loop in both Python and Node.

Python — route by task, then A/B the candidates

from openai import OpenAI

client = OpenAI(base_url="https://api.ofox.ai/v1", api_key="<OFOXAI_API_KEY>")

def pick_model(task):
    if task["has_image"]:         return "minimax/minimax-m3"        # only M3/Kimi take images
    if task["hard_reasoning"]:                                       # split the reasoning tier
        return "z-ai/glm-5.2" if task["context"] > 256_000 else "moonshotai/kimi-k2.6"
    if task["context"] > 200_000: return "deepseek/deepseek-v4-pro"  # free cache reads, 1M ctx
    return "deepseek/deepseek-v4-flash"                              # cheapest tier

def run(task, messages):
    model = pick_model(task)
    return client.chat.completions.create(model=model, messages=messages)

To compare candidates on your own traffic, loop over the model IDs with a fixed prompt — swap the string, keep everything else constant:

CANDIDATES = ["deepseek/deepseek-v4-flash", "deepseek/deepseek-v4-pro", "z-ai/glm-5.2"]
for model in CANDIDATES:
    r = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": "Refactor this function for readability: ..."}],
    )
    u = r.usage
    print(model, u.prompt_tokens, u.completion_tokens)  # log tokens to price each route

Node — same shape

import OpenAI from "openai";

const client = new OpenAI({ baseURL: "https://api.ofox.ai/v1", apiKey: process.env.OFOXAI_API_KEY });

const pickModel = (t) =>
  t.hasImage        ? "minimax/minimax-m3"
  : t.hardReasoning ? (t.context > 256000 ? "z-ai/glm-5.2" : "moonshotai/kimi-k2.6")
  : t.context > 200000 ? "deepseek/deepseek-v4-pro"
  : "deepseek/deepseek-v4-flash";

const r = await client.chat.completions.create({
  model: pickModel(task),
  messages: [{ role: "user", content: "Summarize this changelog: ..." }],
});

Multimodal only: attach a screenshot to MiniMax M3 or Kimi K2.6

GLM-5.2 and both DeepSeek tiers are text-only — the call below physically fails on them. Route image input to minimax/minimax-m3 or moonshotai/kimi-k2.6:

import base64

img = base64.b64encode(open("screenshot.png", "rb").read()).decode()
r = client.chat.completions.create(
    model="minimax/minimax-m3",   # or moonshotai/kimi-k2.6
    messages=[{"role": "user", "content": [
        {"type": "text", "text": "What error is shown in this screenshot?"},
        {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{img}"}},
    ]}],
)

That is the whole router: a pick_model function and one OpenAI client. No new SDK, no per-model API key, one billing line. Detail pages for each model are linked in the table — z-ai/glm-5.2, deepseek/deepseek-v4-pro, deepseek/deepseek-v4-flash, minimax/minimax-m3, and moonshotai/kimi-k2.6.

Alternatives

If a single-key, client-side router fits your workload, ofox is the simplest path: one OpenAI-compatible endpoint, one balance, all five model IDs. For other shapes:

ofox — one key, 100+ models, OpenAI-compatible. You control routing via the model field; billing and the endpoint are unified. Best when you want cost-predictable, deterministic routing you write yourself. See the OpenRouter alternatives breakdown for how it compares on markup and reliability.
OpenRouter — large catalog with an optional Auto server-side router that picks a model for you. Useful if you specifically want automatic selection and can tolerate less predictable routing and the platform's markup.
Direct provider APIs — calling DeepSeek, Zhipu (GLM), MiniMax, and Moonshot each directly gives you the rawest pricing but four keys, four SDKs, and four billing lines to reconcile. Worth it only at very high single-provider volume.
Self-hosting — GLM and DeepSeek publish open weights, so an air-gapped or fork-required deployment is possible. The economics only work at scale; see our GLM-5.2 self-host hardware cost analysis for the breakeven math against hosted per-token pricing.

For deeper per-model context, the GLM-5.2 access guide, GLM-5.2 vs GPT-5.5 cost breakdown, DeepSeek V4 Pro vs Flash comparison, DeepSeek V4 release guide, and MiniMax M3 vs GPT-5.5 coding benchmark each go one layer deeper than this routing overview.

FAQ

The frontmatter FAQ block above answers the most common routing questions (one-key routing, cheapest model, longest context, which models do vision, real savings, free cache reads, no server-side auto-router, max output, and how to A/B). Those answers mirror the tables in this post — the cost numbers, model IDs, and routing rules are consistent throughout.

Sources Checked for This Refresh

ofox /v1/models live API catalog — all five model IDs, context windows, max output, and per-token pricing (input / output / cache read) verified 2026-06-23
ofox llms-full.txt — OpenAI-compatible base_url https://api.ofox.ai/v1 and single-key-across-models confirmed (2026-06-23)
ofox model detail pages for z-ai/glm-5.2, deepseek/deepseek-v4-pro, deepseek/deepseek-v4-flash, minimax/minimax-m3, moonshotai/kimi-k2.6 — all returned HTTP 200 (2026-06-23)
OpenAI Python SDK (openai 2.43.0 on PyPI) and OpenAI Node SDK — SDK shape used in code examples (2026-06-23)
All cost tables are recomputable from the per-token rates in the Quick Specs table

Originally published on ofox.ai/blog.

Run GLM 5.2 Locally (2026): 2-bit on a 256GB Mac or 4090 box

Owen — Tue, 23 Jun 2026 10:14:37 +0000

Zhipu put the GLM 5.2 weights on HuggingFace under an MIT license, so the question stopped being "can I download a frontier coding model" and became "will it run on the machine I already own." For a single Mac Studio or a desktop with one GPU and a lot of RAM, the answer is a qualified yes. The qualifier is the quant.

What You Can Run Locally (and What You Can't)

This guide is about running GLM 5.2 on one machine you own, using quantized GGUF weights and llama.cpp, LM Studio, or Unsloth Studio. That is a different job from serving it to a team on a rack of H200s, which the GLM 5.2 self-host hardware and cost guide covers, and a different job again from calling the hosted API, which the GLM 5.2 access guide covers.

GLM 5.2 is a 753B-parameter model with a 1M-token context, released under MIT. At full BF16 precision the weights are ~1.5 TB, which does not fit any single desktop. Local inference means quantizing: trading some quality for a footprint that fits in your RAM. Here is the 30-second version of what fits where.

Your machine	Quant that fits	Disk / RAM needed	What to expect
Mac Studio M3 Ultra, 512 GB	4-bit UD-Q4_K_XL	~376-475 GB	Best local quality, mostly lossless, usable coding speed
Mac Studio M3 Ultra, 256 GB	2-bit UD-IQ2_M	~240 GB	Codes well, ~3-9 tok/s, the common local rig
Desktop + 4090 + 256 GB DDR5	2-bit UD-IQ2_M	~240 GB	Runs via offload, low single-digit tok/s
8x H200 or 4x H100 rack	FP8 / Q4	376-750 GB	Production scale, see the self-host guide
MacBook / 64-128 GB box	none	n/a	Use the hosted plan instead

The honest headline: a 256 GB Mac Studio running the 2-bit quant is the realistic "GLM 5.2 on my desk" setup. The 4-bit quant is the quality sweet spot, but it wants a 512 GB machine or heavy offload. Anything smaller than 256 GB is a hosted-API job, not a local one.

Decision Frame: When Local GLM 5.2 Is Worth It (and When NOT)

Run the quant locally for the right reasons. The wrong reason is saving money, because for almost everyone the hosted plan is cheaper.

When to run it locally

Offline or air-gapped work. No outbound traffic to api.z.ai is allowed, so the model has to live on your hardware.
Privacy on a single box. Your prompts and code never leave the machine, and one Mac Studio is the whole perimeter.
You already own the hardware. A 256 GB or 512 GB Mac Studio bought for video or ML work is sitting idle at night, and a local quant costs you nothing extra to run.
Tinkering and learning. You want to feel how a 753B MoE behaves, test sampling settings, or build against a local OpenAI-compatible endpoint with no rate limits.

When NOT to run it locally

You want it to be cheap and fast. The Z.ai Coding Plan is ~$30/month and runs at full speed. A 2-bit local quant at 3-9 tok/s cannot match that for the price of electricity alone. Read the access guide.
You need to serve more than one person. A single Mac Studio is a single-session machine. Two developers hammering it at once will each feel it crawl. That is the datacenter path.
Your machine is under 256 GB. There is no quant that makes GLM 5.2 fit a 128 GB box at quality worth using. Do not burn a weekend trying.
You need the full 1M context. Long-context KV cache does not fit on consumer hardware. Local tops out around 16K-64K in practice.

Stop rule

If you do not have at least 256 GB of unified memory or system RAM, stop here and use the hosted plan. No amount of quantization changes that floor.

System Requirements

flowchart TD
  A[How much memory?] -->|512 GB Mac| B[4-bit UD-Q4_K_XL<br/>best local quality]
  A -->|256 GB Mac or DDR5| C[2-bit UD-IQ2_M<br/>the common rig]
  A -->|under 256 GB| D[Use the hosted plan<br/>not a local job]
  B --> E[llama.cpp / LM Studio / Unsloth Studio]
  C --> E

Before you pull 240 GB of weights, confirm you have:

Memory. 256 GB minimum (unified memory on Apple silicon, or system DDR5 on a CUDA box). The 2-bit quant is ~240 GB, so on a 256 GB machine the headroom is genuinely tight: close other apps and leave macOS its share of unified memory, or you will hit swap. 512 GB to run 4-bit comfortably.
Disk. The quant plus headroom: ~240 GB free for 2-bit, ~376-475 GB for 4-bit. An SSD, not a spinning disk, or load times become painful.
A runner. llama.cpp built from a recent commit, LM Studio, or Unsloth Studio. The architecture (GLM MoE DSA) is new enough that an old llama.cpp build will fail to load the tensors.
The right repo. Community GGUF quants live at huggingface.co/unsloth/GLM-5.2-GGUF. The official zai-org/GLM-5.2 repo is BF16 only and is not what you want for local inference.

Step-by-Step: Run GLM 5.2 Locally

Step 1: Pull a GGUF quant

Download only the quant you need, not the whole repo. The --include filter keeps you from fetching 750 GB of shards you will not use.

# 2-bit for a 256 GB machine (~240 GB on disk)
hf download unsloth/GLM-5.2-GGUF \
  --local-dir ~/models/glm-5.2-gguf \
  --include "*UD-IQ2_M*"

You should end up with a set of GLM-5.2-UD-IQ2_M-0000X-of-0000Y.gguf shards in ~/models/glm-5.2-gguf. Swap the filter to *UD-Q4_K_XL* if you are on a 512 GB machine. Check the live "Files and versions" tab on HuggingFace for the exact shard names, since Unsloth revises quant labels as the dynamic quants improve.

Step 2: Run it with llama.cpp

This is the command-line path and the one with the most control. Build a recent llama.cpp first (Metal compiles automatically on Mac; add -DGGML_CUDA=ON on an Nvidia box).

# Build once
cmake -B build && cmake --build build --config Release -j

# Serve an OpenAI-compatible endpoint on port 8080
./build/bin/llama-server \
  --model ~/models/glm-5.2-gguf/GLM-5.2-UD-IQ2_M-00001-of-00006.gguf \
  --ctx-size 32768 \
  --n-gpu-layers 999 \
  --temp 1.0 --top-p 0.95 --min-p 0.01 \
  --host 0.0.0.0 --port 8080

Each flag earns its place:

--ctx-size 32768 sets a 32K window. Raising it eats memory fast on a 256 GB machine; start here and grow only if a request needs it.
--n-gpu-layers 999 offloads every layer it can to the GPU. On a Mac the unified memory makes this nearly free; on a 4090 it offloads the fraction that fits in 24 GB and leaves the rest on the CPU.
--temp 1.0 --top-p 0.95 --min-p 0.01 are Zhipu's recommended sampling defaults. Getting these wrong is the most common cause of "the local model is dumber than the hosted one."

Once it loads, llama-server logs the layer count and then prints server listening on http://0.0.0.0:8080. The first load takes a minute or two off an SSD.

Step 3: Or use a GUI (LM Studio / Unsloth Studio)

If you would rather not touch a build toolchain, two GUI apps load the same GGUF quants.

LM Studio runs the same GGUF quants from a desktop app. Search for unsloth/GLM-5.2-GGUF in the in-app model browser, pick the 2-bit or 4-bit quant, and it handles the download and serving, exposing the same OpenAI-compatible endpoint on a local port.

Unsloth Studio is a web UI with automatic memory offloading, installed in one line.

curl -fsSL https://unsloth.ai/install.sh | sh
unsloth studio -H 0.0.0.0 -p 8888

Both are the better choice if you want to swap quants and settings without re-typing a long llama.cpp command each time.

Step 4: Smoke test

Point any OpenAI client at the local port and confirm it answers.

curl -s http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5.2",
    "messages": [{"role":"user","content":"Reply with only the string OK."}],
    "max_tokens": 16
  }' | jq -r '.choices[0].message.content'

You should get OK back after a short pause. If the reply is garbled or loops, your sampling params are off, so re-check --temp 1.0 --top-p 0.95 --min-p 0.01 against the values in huggingface.co/zai-org/GLM-5.2/generation_config.json.

Real Tokens/sec: What to Expect by Tier

Generation speed on local hardware is bound by memory bandwidth, not raw compute, which is why a Mac Studio with 800 GB/s unified memory beats a DDR5 desktop whose RAM runs closer to 80-100 GB/s. These are the figures to plan around.

Setup	Quant	Realistic generation speed	Good for
Mac Studio M3 Ultra, 256 GB	2-bit UD-IQ2_M	~3-9 tok/s	Solo coding agent, one session
Mac Studio M3 Ultra, 512 GB	4-bit UD-Q4_K_XL	a few tok/s, higher quality	Solo work where correctness matters more than speed
Desktop, 4090 + 256 GB DDR5	2-bit UD-IQ2_M	low single digits	Tinkering, offline use
4x H100 / 8x H200 rack	Q4 / FP8	tens of tok/s per stream	Teams (see self-host guide)

The pattern: local GLM 5.2 is a single-stream, single-developer tool. The speed is fine for one coding agent thinking through a task. It is not fine for a shared endpoint, and no consumer quant changes that. If you need throughput for a team, the self-host hardware guide walks the vLLM and SGLang path on datacenter GPUs.

Common Errors During Local Setup (and Fixes)

Error	Likely cause	Fix
`tensor not found: blk.X.attn_q.weight`	llama.cpp build too old for GLM MoE DSA	Pull a recent llama.cpp commit and rebuild with `cmake --build build`
Process killed / swap thrash on load	Quant is bigger than free RAM	Drop to a smaller quant, or close other apps; 2-bit needs ~240 GB free, not just installed
Output is repetitive or incoherent	Sampling params not aligned to Zhipu defaults	Set `--temp 1.0 --top-p 0.95 --min-p 0.01`; do not leave top_k at a low default
Painfully slow generation on a 4090 box	Most layers running from DDR5, not VRAM	Expected on 24 GB VRAM; lower `--ctx-size`, or move to a 256 GB Mac for better bandwidth
`failed to allocate KV cache` at high ctx-size	Context window too large for remaining memory	Lower `--ctx-size`, or quantize the KV cache with `--cache-type-k q4_1 --cache-type-v q4_1`
Model "thinks" forever before answering	Thinking mode on for a task that does not need it	Disable it with `--chat-template-kwargs '{"enable_thinking":false}'`
Ollama pull only offers `glm-5.2:cloud`	No local Ollama tag exists yet	Use llama.cpp or LM Studio with the Unsloth GGUF instead

Team / Multi-Developer: When One Mac Isn't Enough

A single local machine serves one person. The moment a second developer points an agent at the same llama-server, both sessions slow to a crawl, because consumer hardware has no spare bandwidth to split. There is no clever flag that fixes this.

Two real options when local stops scaling:

Move to datacenter GPUs. An 8x H200 node serving FP8 handles many concurrent streams at tens of tokens per second each. That is a different cost and operations story, fully worked through in the self-host vLLM and cost guide, including the break-even math against the hosted plan.
Use a hosted endpoint and stop running metal. For most teams this wins on every axis except data residency.

The local quant is the right tool for one developer who wants the model on their own machine. It is the wrong tool for a shared service.

Advanced: Long Context and Thinking Mode

Two knobs are worth knowing once the basic setup runs.

KV cache quantization. The 1M context is real in the architecture but unreachable on a 256 GB box, because the KV cache alone would need hundreds of gigabytes. Quantizing it buys back room:

./build/bin/llama-server \
  --model ~/models/glm-5.2-gguf/GLM-5.2-UD-IQ2_M-00001-of-00006.gguf \
  --ctx-size 65536 \
  --cache-type-k q4_1 --cache-type-v q4_1 \
  --n-gpu-layers 999 --port 8080

This roughly halves KV cache memory, letting you push context further on the same hardware, at a small quality cost on very long inputs.

Thinking mode. GLM 5.2 has a reasoning mode that spends tokens thinking before it answers. For quick edits and short prompts it adds latency you may not want. Turn it off per request with --chat-template-kwargs '{"enable_thinking":false}' and leave it on for hard multi-step problems where the extra reasoning earns its keep.

When Local Is the Wrong Answer: Hosted and ofox Alternatives

If the 256 GB floor or the single-session speed rules local out, you do not have to give up GLM 5.2 at all. The same model is on the ofox catalog as z-ai/glm-5.2, priced at $1.40/M input and $4.40/M output, so you can run it hosted at full speed by changing only the base URL and model ID, with no rig to buy or babysit. You prototype against your local llama-server and then point the same client at the hosted model:

export OPENAI_BASE_URL="https://api.ofox.ai/v1"
export OPENAI_API_KEY="ofox-..."
export OPENAI_MODEL="z-ai/glm-5.2"   # the exact same model, now hosted

The hosted access guide covers the Z.ai Coding Plan route to the same model as well. And if you want a few other open-weights coding models behind that one OpenAI-compatible endpoint, ofox lists these day-one too:

Model	ofox model ID	Context	When to pick over GLM 5.2
DeepSeek V4 Pro	`deepseek/deepseek-v4-pro`	1M	You want a longer community track record and published SWE-bench Verified numbers
Kimi K2.6	`moonshotai/kimi-k2.6`	262K	You need independently benchmarked long context, not a 16K local ceiling
Qwen 3 Coder Next	`bailian/qwen3-coder-next`	256K	Multilingual codebases where local speed is too slow to iterate

For a price-and-quality read on GLM against a closed model before you commit to either a local rig or a hosted subscription, see the GLM 5.2 vs GPT-5.5 cost comparison.

Sources Checked for This Refresh

HuggingFace official model card, zai-org/GLM-5.2 (753B parameters, MIT license, 1M context), verified 2026-06-23: https://huggingface.co/zai-org/GLM-5.2
Unsloth GGUF community quants and per-quant memory table, verified 2026-06-23: https://huggingface.co/unsloth/GLM-5.2-GGUF
Unsloth GLM 5.2 run guide (quant sizes, sampling defaults, KV-cache flags, Unsloth Studio install): https://unsloth.ai/docs/models/glm-5.2
llama.cpp project: https://github.com/ggml-org/llama.cpp
LM Studio: https://lmstudio.ai
Companion ofox guides: self-host hardware and cost, hosted access, GLM 5.2 vs GPT-5.5 cost

The interesting shift is not that a frontier model runs locally, it is how little it now costs to find out. A 256 GB Mac Studio you already own and an afternoon of downloading is the whole experiment. The next thing to watch is FP4 and tighter dynamic quants: the day a good 4-bit drops under 200 GB, the local floor moves from a 256 GB Mac down to a 128 GB one, and a lot more desks qualify.

Originally published on ofox.ai/blog.

GLM-5.2 vs GPT-5.5 Cost: Per-Token Math at 10K/100K/1M Req/Day (2026)

Owen — Sun, 21 Jun 2026 13:08:21 +0000

GLM-5.2 vs GPT-5.5 Cost: Per-Token Math at 10K/100K/1M Req/Day (2026)

TL;DR — At ofox.io's listed pricing, GLM-5.2 costs $1.4 input / $4.4 output per million tokens; GPT-5.5 sits at $5 / $30. Blended at a 2:1 input-to-output ratio, that is $2.40 vs $13.33 per million tokens — a 5.56x cost ratio. At 100K requests per day on 3K-token prompts, you spend roughly $720/day on GLM-5.2 versus $4,000/day on GPT-5.5 — about $21,600 vs $120,000 per month. Prompt caching helps both but doesn't close the gap. Both models are on the same OpenAI-compatible endpoint at ofox.io so the comparison is a one-line model swap.

GPT-5.5's per-token cost is 5.56x GLM-5.2's at a typical coding mix — and 6.82x on pure output tokens. The question stopped being whether GLM-5.2 is "good enough"; it became which workload still earns the GPT-5.5 premium.

If you want to skip the math and just A/B both models on your own workload, ofox.io hosts both z-ai/glm-5.2 and openai/gpt-5.5 on the same key — pay-as-you-go, no monthly fee, and the same SDK shape as the OpenAI Python client. The full math below uses ofox's listed per-token rates verified June 21, 2026.

TL;DR: Which One Should You Pick?

Scenario	Pick	Why
Cost-sensitive batch coding agents	GLM-5.2	5.56x cheaper at 2:1 mix, same 1M context
Long-context refactor jobs (>500K input)	GLM-5.2	Same 1M context and 128K output cap; 3.57x cheaper input dominates input-heavy jobs
Output-heavy code generation pipelines	GLM-5.2	6.82x cheaper per output token
Codex CLI / Terminal-Bench-heavy agentic workflows	GPT-5.5	Integration depth and 82.7% Terminal-Bench 2.1
Latency-sensitive interactive pair programming	GPT-5.5	Tuned for first-token speed on short prompts
Azure-backed procurement / Microsoft compliance shop	GPT-5.5	ofox's GPT-5.5 line is Azure-backed
Air-gapped or fork-required deployment	GLM-5.2 self-host	MIT weights on Hugging Face

The honest verdict for most 2026 coding teams: route the cost-sensitive default traffic to z-ai/glm-5.2, keep openai/gpt-5.5 on the Codex CLI / interactive surface, escalate the hardest 10% to Claude. The two-model split below covers the realistic 80% of your traffic without a vendor migration.

What Each Model Ships on ofox

Both models live on api.ofox.io/v1 under the OpenAI-compatible protocol, and on the Anthropic-protocol endpoint for Claude Code drop-in use. The boring numbers, verified against the ofox model catalog on June 21, 2026:

Spec	GLM-5.2	GPT-5.5
Listed on ofox	June 16, 2026	April 24, 2026
ofox model ID	`z-ai/glm-5.2`	`openai/gpt-5.5`
Detail page	ofox.io/en/models/z-ai/glm-5.2	ofox.io/en/models/openai/gpt-5.5
Input price	$1.4 / M tokens	$5.00 / M tokens
Output price	$4.4 / M tokens	$30.00 / M tokens
Cache read price	$0.26 / M tokens	$0.50 / M tokens
Web search add-on	$0.01 / request	$0.01 / request
Context window	1,000,000 tokens	1,000,000 tokens (922K in / 128K out)
Maximum output	128,000 tokens	128,000 tokens
Provider backing	Z.ai (Zhipu)	Azure (OpenAI via Microsoft)
Weights	Open (MIT, Hugging Face zai-org)	Closed (API only)

Two things to call out from the spec sheet. First, the context windows and output ceilings are effectively identical — both list a 1M context and a 128K max-output cap, so neither model lets you emit a larger single-call patch than the other; on long refactor jobs the deciding factor is per-token cost, not output capacity. Second, GPT-5.5 on ofox is Azure-backed. That is the procurement story for shops already inside the Microsoft compliance perimeter; it does not change the listed rate card visible to most accounts but it does mean the upstream is Microsoft, not OpenAI direct.

For the full GLM-5.2 access path — pricing tiers, MIT weights timeline, Z.ai's own Coding Plan — see our GLM-5.2 access guide. For the GPT-5.5 coding benchmark picture against the other 2026 frontier models, see the MiniMax M3 vs GPT-5.5 SWE-Bench breakdown.

Real Per-Token Math: Three Workload Scenarios

Sticker pricing is straightforward. The interesting number is what the invoice looks like at your actual scale. We use three scenarios across the realistic volume range that teams hit in production.

Assumption block (held constant across all three):

3,000 tokens per request, split 2:1 input to output (2K in, 1K out)
30 days per month
No cache hits in the headline number (we add cache impact in the next section)
Web search add-on excluded

Light: 10K requests per day

Roughly the shape of a small team running a single coding agent at moderate intensity, or a side project at scale.

Daily input tokens: 10K × 2K = 20M
Daily output tokens: 10K × 1K = 10M

Model	Input cost / day	Output cost / day	Total / day	Total / month
GLM-5.2	20M × $1.4 = $28	10M × $4.4 = $44	$72	~$2,160
GPT-5.5	20M × $5.0 = $100	10M × $30 = $300	$400	~$12,000
Difference	—	—	$328/day	~$9,840/month

Mid: 100K requests per day

The shape of a 10-engineer team running coding agents full time, or a product feature that exposes the model to end-users at moderate concurrency.

Daily input tokens: 100K × 2K = 200M
Daily output tokens: 100K × 1K = 100M

Model	Input cost / day	Output cost / day	Total / day	Total / month
GLM-5.2	200M × $1.4 = $280	100M × $4.4 = $440	$720	~$21,600
GPT-5.5	200M × $5.0 = $1,000	100M × $30 = $3,000	$4,000	~$120,000
Difference	—	—	$3,280/day	~$98,400/month

Heavy: 1M requests per day

The shape of a production agent fleet, a developer-tooling SaaS at scale, or an internal platform exposed to a four-figure-engineer org.

Daily input tokens: 1M × 2K = 2B
Daily output tokens: 1M × 1K = 1B

Model	Input cost / day	Output cost / day	Total / day	Total / month
GLM-5.2	2B × $1.4 = $2,800	1B × $4.4 = $4,400	$7,200	~$216,000
GPT-5.5	2B × $5.0 = $10,000	1B × $30 = $30,000	$40,000	~$1,200,000
Difference	—	—	$32,800/day	~$984,000/month

The 5.56x ratio holds at every volume tier — only the absolute spend scales. At light volume that is a useful saving; at mid volume it pays for two senior engineers per month; at heavy volume it is the difference between a feature shipping and a feature being killed for unit-economics reasons.

These tables hold for the standard 2:1 input-to-output mix. The ratio drifts with workload shape: at 1:1 (chat-style turns) the cost ratio is 6.03x; at 1:3 output-heavy (code generation from a short prompt) the ratio is 6.51x; at 3:1 input-heavy (long-context summarization) the ratio narrows to 5.23x because GLM-5.2's per-input-token discount (3.57x cheaper input) is smaller than its per-output-token discount (6.82x cheaper output). Output-dominated workloads tilt further toward GLM-5.2; input-dominated workloads tilt less hard but still favor GLM at every realistic mix.

Cache Impact: How Far Does Prompt Caching Close the Gap?

Both models bill cache reads below the full input rate: GLM-5.2 at $0.26/M (an 81% input discount), GPT-5.5 at $0.50/M (a 90% input discount). Cache hit rates above 50% are realistic for code-review workloads where the codebase context repeats across requests. Here is what 50% input cache hit does to the blended cost.

At 50% input cache hit (half of input tokens served from cache, output unchanged):

Model	Uncached input ($/M)	Cached input ($/M)	Effective input ($/M)	Output ($/M)	Blended ($/M) at 2:1	Drop vs no cache
GLM-5.2	$1.40	$0.26	$0.83	$4.40	$2.02	−15.8%
GPT-5.5	$5.00	$0.50	$2.75	$30.00	$11.83	−11.2%

At 100% input cache hit (every input token cached):

Model	Input ($/M, all cached)	Output ($/M)	Blended ($/M) at 2:1	Drop vs no cache
GLM-5.2	$0.26	$4.40	$1.64	−31.7%
GPT-5.5	$0.50	$30.00	$10.33	−22.5%

Two reads on this. First, cache saves more absolute dollars on GPT-5.5 per cached token — you avoid $4.50 per cached million on GPT-5.5 versus $1.14 on GLM-5.2. If your CFO scores the cache program by raw dollars saved, GPT-5.5 wins. Second, cache saves a larger share of GLM-5.2's total bill — because input is a bigger fraction of GLM-5.2's blended cost, cutting input costs has a bigger proportional effect. At 100% input cache hit, GLM drops 31.7% of its blended bill; GPT-5.5 drops 22.5%.

The net result is that GLM-5.2 stays cheaper at every cache hit rate point. The cost ratio actually widens slightly as cache hit rate climbs — from 5.56x without cache to 5.86x at 50% input cache hit to 6.30x at 100% input cache hit. That sounds counterintuitive, but the math is straightforward: cache eats a larger share of GLM-5.2's blended bill than of GPT-5.5's, so GLM's bill shrinks faster in percentage terms. Prompt caching is a uniform discount on input only; it does not change the GPT-5.5 output rate, and output is where the absolute dollar gap lives.

When GLM-5.2 Wins (and When the Benchmark Gap Is Acceptable)

Five workloads where GLM-5.2 is the obviously correct routing decision:

Batch code review and async refactor sweeps. Overnight dependency upgrades, doc generation, batched lint fixes — work where total token spend dominates and individual-request latency does not matter. The 5.56x cost gap compounds across thousands of requests per night.
Long-context refactor jobs. GLM-5.2's 1M context lets you submit an entire mid-sized module in one prompt. Its 128K output cap is identical to GPT-5.5's, so very large rewrites still chunk on both models — but GLM-5.2 emits the same patches at 5.56x lower per-token cost, and its input is 3.57x cheaper, which dominates on input-heavy refactor passes.
Output-heavy code generation pipelines. Per-output-token cost is the differentiator at 6.82x. If your agent emits more code than it reads (test generation, scaffolding, codemod application), GLM-5.2 disproportionately wins.
High-cache-hit workloads. Code-review agents reusing the same codebase context, RAG pipelines with stable corpora — GLM-5.2's cache read at $0.26/M is half of GPT-5.5's $0.50/M, and the proportional cache benefit on GLM is larger.
Open-weight insurance. MIT-licensed weights mean if Z.ai changes hosted pricing or terms, you can fall back to self-hosting on the same model. GPT-5.5 has no on-prem path. Even if you never deploy the weights, the option value is real.

The honest qualifier: the benchmark gap to GPT-5.5 is real on Terminal-Bench-style agentic work. Z.ai had not published SWE-Bench Verified scores at GLM-5.2's launch, and independent third-party benchmark numbers were pending as of mid-June 2026. If your workload depends on the multi-step shell agentic loop that Terminal-Bench measures, GPT-5.5 still leads — for everything else, the cost case is decisive.

When GPT-5.5 Still Makes Sense

Three workloads where the 5.56x premium earns its keep:

Codex CLI is your primary surface. OpenAI's terminal agent is tuned against GPT-5.5 at the protocol level — file handles, shell history, multi-turn recovery from failed commands. The Terminal-Bench 2.1 score (82.7%) reflects integration depth as much as model capability. Swapping the model behind Codex is not a free move.
Latency-sensitive interactive coding. Pair-programming flows where every extra second of first-token latency hurts adoption. GPT-5.5 is tuned for short prompts and fast first-token; on a 5K-token interactive prompt, GPT-5.5 typically wins the latency comparison.
Azure-backed procurement. ofox's GPT-5.5 line is Azure-backed, which closes the procurement story without a new vendor review for shops already inside Microsoft compliance. The procurement cost of adding a new model vendor often exceeds the per-token savings for teams below a few hundred thousand tokens per day.

The fourth scenario is mixed-workload reasoning load — if your coding agent occasionally writes architecture summaries, postmortems, or research briefs, GPT-5.5's general reasoning ceiling is higher than GLM-5.2's. That said, for purely coding workloads, the cost case for GLM-5.2 dominates.

A/B Routing Pattern via ofox: One Key, One Endpoint, Two Models

Both z-ai/glm-5.2 and openai/gpt-5.5 are live on https://api.ofox.io/v1 under the OpenAI-compatible protocol. The model swap is a single string change. The smallest useful A/B harness:

Python — A/B both models in one loop

from openai import OpenAI
import os, time

client = OpenAI(base_url="https://api.ofox.io/v1", api_key=os.environ["OFOX_API_KEY"])

prompt = "Refactor this Python function to use async/await and return early on empty list: ..."

for model in ["z-ai/glm-5.2", "openai/gpt-5.5"]:
    t0 = time.time()
    resp = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
    )
    elapsed = time.time() - t0
    print(f"{model}: {elapsed:.1f}s, {resp.usage.total_tokens} tokens")
    print(resp.choices[0].message.content[:200])

That gives you raw latency, total token count, and side-by-side output on your own task. Run it across 20-30 representative cases from your real workload — that is the only honest input to a routing decision.

Node — same shape

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.ofox.io/v1",
  apiKey: process.env.OFOX_API_KEY,
});

const prompt = "Refactor this Python function to use async/await and return early on empty list: ...";

for (const model of ["z-ai/glm-5.2", "openai/gpt-5.5"]) {
  const t0 = Date.now();
  const resp = await client.chat.completions.create({
    model,
    messages: [{ role: "user", content: prompt }],
  });
  console.log(`${model}: ${(Date.now() - t0) / 1000}s, ${resp.usage.total_tokens} tokens`);
  console.log(resp.choices[0].message.content.slice(0, 200));
}

Production routing — single-line model swap

The same SDK call, the same key, the same billing line. To route the cost-sensitive half of your traffic to GLM-5.2 and keep the interactive half on GPT-5.5:

def pick_model(request_type: str) -> str:
    if request_type in {"batch_refactor", "code_review", "doc_generation"}:
        return "z-ai/glm-5.2"
    return "openai/gpt-5.5"

resp = client.chat.completions.create(
    model=pick_model(request_type),
    messages=messages,
)

No migration, no new key, no separate billing reconciliation. The model column on your invoice tells you what each request cost; the routing function is one place to tune the split. For the broader pattern of routing across the full ofox catalog — including Claude for escalations — see our $30 AI coding stack guide.

Sources & Pricing References

ofox.io model catalog: z-ai/glm-5.2 — input $1.4/M, output $4.4/M, cache $0.26/M, 1M context, 128K max output, listed June 16, 2026 (verified June 21, 2026)
ofox.io model catalog: openai/gpt-5.5 — input $5/M, output $30/M, cache $0.5/M, 1M context (922K in / 128K out), listed April 24, 2026, Azure-backed (verified June 21, 2026)
GLM-5.2 access guide — pricing tiers, MIT weights, Z.ai Coding Plan
MiniMax M3 vs GPT-5.5 SWE-Bench Pro coding benchmark — companion benchmark-led comparison
Vellum — GPT-5.5 reference — Terminal-Bench 2.1 score 82.7%, output token rate $30/M confirmed

At a 5.56x cost ratio that holds across volume tiers and a 6.82x gap on pure output tokens, the routing question is no longer "is GLM-5.2 good enough" — it is "which workload still justifies paying the GPT-5.5 premium," and "Codex CLI shop" is the cleanest honest answer.

Originally published on ofox.ai/blog.

Self-Host GLM 5.2 in 2026: Hardware, vLLM Setup, and Cost vs Cloud

Owen — Wed, 17 Jun 2026 04:06:26 +0000

Self-Host GLM 5.2 in 2026: Hardware, vLLM Setup, and Cost vs Cloud

Zhipu's GLM 5.2 represents a significant milestone for the open-weights community. The MIT-licensed model weights are now available on HuggingFace, making frontier-class coding capabilities accessible for self-hosted deployments. However, the 753B parameters present substantial hardware requirements that merit careful evaluation before committing to a self-hosted infrastructure investment.

What You Get When You Self-Host GLM 5.2

The capabilities available immediately include serving the model via vLLM on an 8-GPU H200 node. Storage requirements vary significantly by format:

FP8 quantization requires approximately 750 GB
BF16 format needs roughly 1.5 TB
Q4_K_M GGUF weights occupy around 376 GB
2-bit UD-IQ2_XXS quantization uses approximately 241 GB

Production deployments require either 8x H200 GPUs with 141GB memory each (for FP8) or 4x H100 80GB units when using GGUF quantization. For experimentation, a Mac Studio M3 Ultra with 256GB unified memory can run the most aggressive 2-bit quantization at 3–9 tokens per second.

Multiple inference engines achieve compatibility on day one: vLLM v0.23.0+, SGLang v0.5.13.post1+, Transformers v5.12+, KTransformers v0.6.1+, llama.cpp for GGUF formats, and xLLM v0.10.0+. The MIT license permits commercial use, modification, and redistribution without restriction.

Benchmark Performance Indicators

"GLM 5.2 trails Opus 4.8 on raw SWE-bench Pro (62.1 vs 69.2) but pulls ahead on Terminal-Bench 2.1's Best Reported Harness run (82.7 vs 78.9) and on agentic-math (AIME 99.2 vs 95.7)."

Third-party leaderboards show competitive positioning: DesignArena's Web Dev composite ranks GLM 5.2 first overall at Elo 1,360, ahead of Claude Fable 5 (1,350) and Claude Opus variants. The Code Arena Frontend slice places it second at Elo 1,595, behind Claude Fable 5 at 1,654 (which carries a "not currently being sampled" designation).

The performance gap relative to open-weights competitors is substantial: 30–50 Elo points separate GLM 5.2 from Qwen 3.7 Max, Kimi K2.6, and GLM 5.1 on the composite benchmark and 60+ points on the frontend slice.

When to Self-Host

Self-hosting becomes economically and operationally sensible in limited scenarios:

Valid self-host scenarios include:

Data residency requirements preventing code or prompts from leaving internal infrastructure
Custom fine-tuning needs on proprietary codebases without hosted API support
Air-gapped deployments in restricted-network environments
High sustained throughput exceeding 3,000 prompts daily

Self-hosting is the wrong choice when:

Operating as a solo developer or small team (hosted plans cost ~$30–80 monthly)
No existing vLLM or SGLang deployment in production
Requiring vendor-published SWE-bench Verified, LiveCodeBench, or Aider polyglot benchmarks
Peak load remains below 100 prompts daily with no compliance constraints

"Do not self-host" if leveraging hosted services costs less than one-tenth of the engineering overhead required for self-hosted infrastructure management.

Available Formats and Sources

The official repository on HuggingFace distributes BF16 and FP8 variants optimized for production inference. The community-maintained Unsloth repository provides GGUF quantizations supporting both llama.cpp and LM Studio. Ollama's current glm-5.2:cloud tag routes through hosted inference rather than enabling local execution—no quantized local variant exists on the official Ollama library yet.

Hardware Sizing Requirements

KV cache utilization at extended context lengths represents the primary constraint for hardware selection. For 256K context:

BF16 format demands 16x H100 or 8x H200 nodes; H200 sizing remains tight
FP8 requires 8x H200 comfortably or 8x H100 with constrained KV cache
Q4_K_M GGUF works on 4x H100 or 2x H200 units
Quantized 2-bit variants run on high-memory workstations with 256GB+ unified memory

Scaling to 1M context increases KV cache footprint by approximately 4x, necessitating FP8 quantization for production use. VRAM headroom of 20% above total model plus cache requirements prevents fragmentation-related out-of-memory errors during extended inference.

vLLM Production Setup

The primary production deployment path follows this sequence:

First, download the FP8 weights from HuggingFace to local storage (approximately 30–60 minutes on 10 GbE connectivity):

huggingface-cli download zai-org/GLM-5.2-FP8 \
  --local-dir /models/glm-5.2-fp8 \
  --local-dir-use-symlinks False

Launch the vLLM server with tensor parallelism across all available GPUs:

vllm serve "zai-org/GLM-5.2-FP8" \
  --tensor-parallel-size 8 \
  --max-model-len 262144 \
  --kv-cache-dtype fp8 \
  --enable-prefix-caching \
  --port 8000

Tensor parallelism at size 8 distributes the model across all H200 GPUs. Maximum model length begins at 256K tokens (262144) and scales upward after benchmarking actual KV cache behavior. FP8 KV cache reduces memory requirements by half compared to BF16. Prefix caching reuses computed KV for shared prompt prefixes—essential for coding agents executing repetitive system prompts.

Verification through a curl command confirms basic operation:

curl -s http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"zai-org/GLM-5.2-FP8","messages":[{"role":"user","content":"Reply OK"}],"max_tokens":16}' | jq

Expected response confirms operation within approximately one second after initial compilation overhead.

SGLang Alternative

SGLang offers superior throughput for workloads featuring heavy prompt reuse:

python -m sglang.launch_server \
  --model-path zai-org/GLM-5.2-FP8 \
  --tp 8 \
  --context-length 262144 \
  --kv-cache-dtype fp8_e4m3 \
  --enable-mixed-chunk \
  --port 30000

RadixAttention delivers approximately 3x throughput improvement versus vLLM 0.23 when agents reuse 100K+ tokens of shared system context. Implementation complexity increases slightly but remains manageable for teams with existing SGLang production experience.

Local Deployment via llama.cpp

For development, tinkering, or single-node air-gapped scenarios, llama.cpp with Unsloth GGUF quantizations represents the lowest-friction path:

huggingface-cli download unsloth/GLM-5.2-GGUF \
  GLM-5.2-Q4_K_M.gguf \
  --local-dir /models/glm-5.2-gguf

cmake -B llama.cpp/build -DGGML_CUDA=ON
cmake --build llama.cpp/build --config Release -j

./llama.cpp/build/bin/llama-server \
  --model /models/glm-5.2-gguf/GLM-5.2-Q4_K_M.gguf \
  --ctx-size 32768 \
  --n-gpu-layers 999 \
  --host 0.0.0.0 --port 8080

M3 Ultra Mac Studio deployment with 256GB unified memory achieves 3–9 tokens per second depending on context. Performance scales appropriately for solo development but remains insufficient for team-scale throughput.

Cost Analysis: Self-Hosted vs Hosted

The financial comparison reveals hosted solutions dominate for most organizations:

Deployment	Monthly Cost	Notes
Z.ai Pro Plan	~$30	Supports ~2,000 prompts weekly
Z.ai Max Plan	~$80	Supports ~8,000 prompts weekly
Cloud 8x H200 (24/7)	$21–36k	$30–50 per hour blended rate
Cloud 8x H200 (9–5)	$6–10k	200 hours monthly typical
Owned 8x H200	$3–5k	~$200k hardware amortized over 4 years
Owned M3 Ultra	~$50	One-time $8k; electricity $30 monthly

Break-even analysis demonstrates hosted services win when self-host requirements remain absent:

M3 Ultra advantages emerge above $30 monthly hosted spend if 3–9 tokens/sec suffices
Cloud H200 justifies against Max Plan only with 3,000+ daily prompts and 30%+ duty cycle
Owned H200 economics favor self-hosting above 10,000 daily prompts and existing datacenter capacity

"Hosted wins for 95% of teams." Self-hosting becomes advantageous only for organizations with compliance constraints, data residency mandates, or sustained throughput exceeding typical team workloads.

Common Setup Errors and Resolutions

CUDA out of memory during model load occurs when tensor parallelism remains too low or KV cache budget proves too generous. Increase tensor parallelism to match GPU count; reduce maximum model length to half intended value initially.

FP8 operations unsupported indicates Ampere-generation hardware (A100). FP8 E4M3 requires Hopper architecture (H100/H200). A100 users should utilize Q4_K_M GGUF via llama.cpp.

Model has tied_word_embeddings: false warning represents harmless vLLM auto-detection noise and remains safe to ignore for GLM 5.2.

504 connection reset on 500K+ token requests signals first-token latency exceeding default client timeouts. Increase client timeout to 600 seconds; limit concurrent sequences to four requests for vLLM.

IndexError in RadixAttention indicates SGLang tokenizer cache mismatch. Delete ~/.cache/sglang/ completely and restart for cache rebuild on next inference.

GGUF load failure with missing tensor references reveals llama.cpp version incompatibility with GLM MoE DSA architecture. Update llama.cpp to a build version matching or exceeding the GGUF publication date.

Inconsistent outputs versus Z.ai hosted suggests sampling parameter misalignment. Verify temperature (1.0), top_p (0.95), and unset top_k against official generation_config.json from the HuggingFace repository.

Observability Requirements

Production deployments require three critical metrics:

"Track tokens-per-second throughput separately at p50 and p95 percentiles" since individual 900K-context requests drag tail latencies by orders of magnitude.

Monitor KV cache utilization percentage via vLLM's /metrics endpoint. Sustained utilization crossing 90% threshold signals imminent throughput collapse.

Instrument per-request total token consumption at PR or session level to catch runaway token burning in coding agent loops before budget exhaustion.

Wire these metrics into existing observability infrastructure (Datadog, Honeycomb, Grafana). SGLang exposes equivalent metrics at /metrics_collect.

Managed Hosting Alternatives

For scenarios where self-host mathematics fail but Chinese-origin coding models remain preferred, several alternatives support OpenAI-compatible API patterns:

DeepSeek V4 Pro (deepseek/deepseek-v4-pro) offers 1M context and published SWE-bench Verified benchmarks—a specification missing from GLM 5.2's public table which reports only SWE-bench Pro.

Kimi K2.6 (moonshotai/kimi-k2.6) provides independently-benchmarked 262K context as verified capability.

Qwen 3 Coder Next (bailian/qwen3-coder-next) addresses multilingual codebases with Chinese, Japanese, and Korean language support.

These models share identical API wiring—only base URL and model identifier change. GLM 5.2 remains unlisted on the ofox catalog as of June 17, 2026, though eventual availability would require only a single string modification in client configuration.

Future Considerations

The most significant opportunity emerges from potential FP4 quantization community releases within the next 90 days. Should FP4 variants prove viable, production deployments could consolidate from 8x H200 to 4x H100 hardware, fundamentally altering self-host economics for the 5% of organizations currently justifying infrastructure investment.

Originally published on ofox.ai/blog.

Codex Weekly Limit Drained: 7 Fixes, Why It Burns, and a Drop-in API (2026)

Owen — Tue, 16 Jun 2026 10:28:25 +0000

Codex Weekly Limit Drained: 7 Fixes, Why It Burns, and a Drop-in API (2026)

TL;DR

Your Codex weekly limit can drain in three hours on a $20 Plus plan if you trigger a single long refactor, and the only team-wide reset OpenAI shipped was on June 11, 2026.

When the limit hits, you have two paths: wait for the rolling reset (Plus rolls every 5 hours, weekly caps reset weekly) or switch Codex CLI's OPENAI_BASE_URL to a pay-per-token API. This guide gives you a 30-second diagnostic, the exact tier math for Plus and Pro, and seven concrete fixes ranked from "you can still use ChatGPT today" to "drop Codex onto a metered API in 60 seconds."

Is Codex Down or Did Your Weekly Limit Drain? The 30-Second Diagnosis

Before you start switching providers, confirm which window blew up. Run this inside any open Codex CLI session:

codex /status

The output shows three numbers. Match them against this table:

Diagnosis	Context window	5-hour window	Weekly window	What to do
Weekly cap hit	Any	>10%	<5%	Plan-level — switch tier or move to API key
5-hour cap hit	Any	<5%	>20%	Wait for the next 5-hour rollover (max 5h)
Context overflow	<5%	>20%	>20%	Compact the conversation or split the task
Phantom limit (issue #19215)	>40%	>50%	>50%	Restart Codex CLI; if persists, switch base_url

If /status shows healthy capacity but you still see You've hit your usage limit. To get more access now, send a request to your admin or try again at 3:51 PM, you are hitting the #19215 phantom limit bug (reported April 23, 2026 on Codex CLI v0.124.0). Skip to fix #4.

Common Codex error wording → cause → fix

The exact string Codex prints is the fastest way to triage. Map yours to the right column:

Error wording (exact)	Likely cause	Fix to apply
`You've hit your usage limit. ... try again at HH:MM PM`	Genuine 5-hour or weekly cap	Fix #1 (wait) or fix #5/#6 (API path)
`You've hit your usage limit` but `/status` shows >50% remaining	Phantom-limit bug (#19215) on CLI v0.124.0	Fix #4 (clear sessions cache)
`request a free reset` button visible in CLI prompt	Banked reset still available (post June 11)	Fix #2 (claim the free reset)
`429 Too Many Requests` HTTP error in API mode	Direct API spend-burst throttle	Backoff, lower concurrency, or fix #6 (per-key budget)
`insufficient_quota` HTTP error in API mode	OpenAI account spend limit hit	Raise the dashboard spend cap or switch base_url
`You've reached your weekly limit` without reset offer	Banked reset already redeemed this launch	Fix #3 (tier upgrade) or fix #5/#6 (API)
`context_length_exceeded`	Conversation too long, not a tier cap	`codex /compact` or start a fresh session

When to Fix the Limit (and When to Switch to a Drop-in API Instead)

Not every drained limit deserves a workaround. Match your situation to the rule:

When to fix the limit on your existing plan. You hit the cap once this week. Your /status confirms a normal cap (not the phantom bug). The task you were running is wrap-up rather than a fresh multi-hour agent loop. The team-wide June 11 reset banking gives you one free reset per launch event, so you may already be sitting on a free reset.

When to switch to a token-billed API. You drain weekly limits more than twice a month. You run agent workflows that touch 300k+ input tokens in a single session. You need predictable per-task cost (e.g. quoting client work). You hit limits inside CI or shared dev environments where tiered ChatGPT auth doesn't make sense.

Stop rule. If your /status shows fresh remaining capacity and you only hit usage limit once today, do nothing. The rolling 5-hour window will reset, and switching providers for a one-off is over-engineering. Close this tab and write code.

Understanding Codex Rate Limits: Plans, Windows, and What Each Plan Actually Gives You

Codex CLI inherits the limits of whichever auth path you use. Three subscription tiers and the API path each have different ceilings.

Subscription-tier 5-hour message caps

ChatGPT subscription tiers gate Codex usage by messages in a rolling 5-hour window, not tokens. The published bands are wide because each "message" is weighted by model and task complexity.

Plan	Monthly cost	GPT-5.5 msgs / 5h	GPT-5.4 msgs / 5h	GPT-5.3-Codex msgs / 5h
Free	$0	n/a (Plus minimum for Codex)	n/a	n/a
Plus	$20	15-80	20-100	45-225
Pro 5x	$100	75-400	100-500	225-1,125
Pro 20x	$200	300-1,600	400-2,000	900-4,500
Business	per-seat	varies by seat allocation	—	—

Source: OpenAI Codex rate card and Codex pricing reference, figures verified June 15, 2026. Plus (15-80) and Pro 20x (300-1,600) corroborate against independent community reports; Pro 5x's 75-400 band is the rate-card figure only and has not been cross-verified by a third party at the time of writing.

The wide bands (e.g. "15-80") reflect that OpenAI dynamically adjusts caps based on aggregate load and individual usage patterns. The low end of the band is what you actually get on a busy day.

Weekly cap mechanics

A separate weekly cap stacks on top of the 5-hour window. The weekly cap resets on a rolling 7-day calendar from your first message of the week. The two windows are tracked independently — and this is where the math breaks.

A single multi-file agent loop on GPT-5.5 (input-heavy: ~250k tokens / output ~25k) burns roughly the equivalent of 30-40 messages in 5-hour accounting but counts as one "session" against the weekly window. Plus users running 2-3 heavy refactors in a row can drain the weekly cap while still showing 30%+ on the 5-hour gauge.

API-key path (no message cap)

If your OPENAI_API_KEY starts with sk- and is a normal API key, Codex CLI bypasses subscription tier caps entirely. You pay per token at the API rate card. There is no weekly message cap — only the spend limit you set on the dashboard. This is the cleanest fix for hitting weekly limits, and we cover the math below.

Why a single agent run can drain a weekly cap

The unintuitive part of Codex weekly limits is that they don't scale linearly with the message count you see in /status. OpenAI's Codex pricing reference documents the underlying credit math:

GPT-5.5 burns 125 credits per 1M input tokens, 12.50 cached, and 750 per 1M output
GPT-5.4 burns 62.50 / 6.25 / 375
GPT-5.4 mini burns 18.75 / 1.875 / 113

A single agent loop on GPT-5.5 that reads 30 files (~250k input tokens) and produces a 25k-token plan/diff burns roughly:

input:  250,000 × 125/1M = 31.25 credits
output:  25,000 × 750/1M = 18.75 credits
total:                     ≈ 50 credits per run

A Plus plan's weekly budget on GPT-5.5 averages around 250-300 credits, depending on aggregate platform load. Six of those agent loops drain the whole week — and the /status 5-hour gauge can still report 40%+ remaining because the 5-hour window is tracked in messages, not credits. This is why users see the apparent contradiction in issue #19215: healthy 5-hour gauge, exhausted weekly window.

The fix is structural: route heavy agent loops through an API key (no credit accounting, just dollar accounting) and keep your subscription for interactive chat.

How to Fix "You've Hit Your Usage Limit" (Solutions for Every Tier)

Below are seven fixes in order from least to most disruptive. Pick the first one that matches your situation and stop reading.

Fix #1 — Wait for the 5-hour rollover (Free / Plus, light usage)

If /status shows the 5-hour window at <5% remaining but the weekly window above 30%, you only need to wait. The exact rollover timestamp is shown in /status — typically within 3-5 hours of your last burst.

While you wait, the productive move is to switch your IDE to manual coding mode or use a non-Codex tool that doesn't share the cap (any IDE-side completion).

Fix #2 — Claim your free reset (June 11, 2026 changelog)

The Codex changelog entry for June 11, 2026 introduced "rate-limit reset banking" with one free reset granted at launch for Plus and Pro users. If you haven't redeemed yours, the option appears in the Codex CLI when you hit the cap, or via the codex /reset command if you are on CLI ≥ v0.135.

This is a one-time fix per launch event. If OpenAI ships another reset (the May 15, 2026 reset announced by Tibo on X was team-wide and didn't consume the banked one), check changelog announcements before burning your single banked reset.

Fix #3 — Upgrade the plan tier

If you are draining Plus weekly caps weekly, Pro 5x ($100) gives 5× the message budget. Pro 20x ($200) gives 20×. The break-even versus the API path is roughly:

Path	Monthly cost	What you get	Break-even vs API
Plus	$20	15-80 GPT-5.5 msgs / 5h	~3-5 heavy sessions / week
Pro 5x	$100	75-400 GPT-5.5 msgs / 5h	~100 light sessions or 25-30 heavy / month
Pro 20x	$200	300-1,600 GPT-5.5 msgs / 5h	~80 heavy / month
API key (direct)	$0 + spend	Pay per token, no cap	Linear with usage
API key (ofox, 15% off flagship)	$0 + spend	Pay per token, no cap	~$0.80 per heavy session

If your usage pattern is "occasional bursts of heavy work," the API path almost always wins. Subscriptions only make sense if your usage is steady and predictable.

Fix #4 — Phantom-limit workaround (issue #19215)

If /status shows healthy capacity but Codex still rejects requests with usage limit, you are hitting #19215. The bug surfaces on Codex CLI v0.124.0 (and was still active when this issue was filed). Working steps:

codex /quit
rm -rf ~/.codex/sessions/*
codex

Clearing the local session cache resolves it in most reported cases. If it persists, ship logs to OpenAI via the issue thread and switch to fix #5 (API key path) to keep working.

Fix #5 — Switch `OPENAI_BASE_URL` to OpenAI's direct API endpoint

The cleanest "I just need to keep coding" fix. Generate an OpenAI API key (the sk- kind, not the sk-proj- ChatGPT-linked one) at platform.openai.com/api-keys, then:

export OPENAI_API_KEY=sk-your-api-key
export OPENAI_BASE_URL=https://api.openai.com/v1
codex

Codex now hits the metered API instead of your ChatGPT subscription. Costs are per token. A typical small bug fix runs about $0.40 on GPT-5.5, and a multi-file refactor of ~300k input / 30k output sits around $2.40 — both well under what a single banked reset would cost if it were paid.

The trade-off: you lose the cloud features bundled with ChatGPT (Slack integration, GitHub reviews from inside the Codex dashboard). For pure CLI work, this is the highest-leverage fix.

Fix #6 — Switch `OPENAI_BASE_URL` to a drop-in OpenAI-compatible API

Same shape as fix #5 but routed through a provider that aggregates models, gives a unified key, and surfaces per-key budgets. The example below uses ofox, but the pattern works for any OpenAI-compatible endpoint.

export OPENAI_API_KEY=ofox-your-key-here
export OPENAI_BASE_URL=https://api.ofox.ai/v1
codex

Three reasons this beats hitting OpenAI directly when you've already drained your ChatGPT plan:

Cheaper flagship models. ofox lists openai/gpt-5.3-codex at $1.49 input / $11.90 output per million tokens (a 15% discount on OpenAI's $1.75 / $14.00 list). The same refactor that costs $0.95 on OpenAI direct costs about $0.80 here.
Drop-in model switching. openai/gpt-5.4-mini is $0.638 input / $3.83 output per million on ofox — cheap enough to use as your default for iterative work. Reserve gpt-5.3-codex for hard refactors.
Per-key budgets. ofox exposes a per-key monthly cap on the dashboard. Set it to $50/month and you cannot exceed that limit, regardless of how aggressive the agent loop gets.

The ofox Codex integration docs and the deeper-dive Codex CLI custom endpoint guide cover model-specific tuning.

Fix #7 — Multi-provider routing via `~/.codex/config.toml`

For teams or anyone who wants gpt-5.3-codex for refactors and a cheaper model for iteration, configure two providers and let Codex switch:

[model_providers.ofox]
name = "OfoxAI"
base_url = "https://api.ofox.ai/v1"
wire_api = "responses"

[model_providers.openai_direct]
name = "OpenAI Direct"
base_url = "https://api.openai.com/v1"
wire_api = "responses"

[profiles.fast]
model = "openai/gpt-5.4-mini"
model_provider = "ofox"

[profiles.heavy]
model = "openai/gpt-5.3-codex"
model_provider = "ofox"

Switch on demand with codex --profile heavy for hard work and codex --profile fast for iteration. The full schema lives in Codex CLI config.toml deep dive.

Codex Limit Incidents: Real Patterns from 2026

Real timeline of weekly-limit-related incidents this year. Tracking patterns is the only way to predict the next one.

Date	Event	Source	Impact
Apr 23, 2026	Issue #19215 filed — `/status` shows healthy capacity but Codex CLI rejects with "usage limit" on GPT-5.5, Business plan, CLI v0.124.0	openai/codex#19215	Affected an unknown share of CLI v0.124.0 users; workaround is local cache clear
Late Apr 2026	Codex hits 3M weekly active users; OpenAI announces team-wide rate-limit reset	Knightli debrief	One-shot reset for affected users
May 15, 2026	Tibo (OpenAI) posts on X: monitoring continues, manual reset shipped that evening	Knightli debrief above	Reset was active, not scheduled — no permanent cap raise
Jun 11, 2026	Changelog adds "rate-limit reset banking" — one free reset per user at launch	Codex changelog	First user-controlled reset mechanism; one-time
Jun 4, 2026	Bedrock model support — Codex can now route via Amazon-managed quota	Codex changelog	Indirect relief: Bedrock-side quotas instead of ChatGPT-tier

The pattern is clear: OpenAI has shipped tactical resets rather than permanent cap increases. Plus and Pro users hitting their cap twice in the same month should plan for the API path rather than a third reset.

When the Limit Won't Reset: Drop-in API Alternatives That Work Right Now

If you cannot wait, here are the options ranked by switching cost. The headline comparison:

Provider	base_url	gpt-5.3-codex input $/M	gpt-5.3-codex output $/M	Per-key budget	Switch time
ofox	`https://api.ofox.ai/v1`	$1.49	$11.90	Yes (dashboard)	~60s (2 env vars)
OpenAI direct	`https://api.openai.com/v1`	$1.75	$14.00	Account-level only	~60s (2 env vars)
Amazon Bedrock	(Bedrock proxy)	varies by region	varies by region	AWS account cap	10-30 min (IAM + region)
Switch coding tool	n/a (e.g. Claude Code)	n/a — different model	n/a	n/a	hours (workflow rewrite)

Source: pricing verified June 15, 2026 against ofox model catalog and Codex pricing reference.

Option A — ofox (OpenAI-compatible drop-in)

Pros: 15% discount on flagship OpenAI models, per-key spend caps, unified billing if you also use Claude/Gemini, Codex CLI listed as officially supported.

How to switch: Two env vars. Set OPENAI_BASE_URL=https://api.ofox.ai/v1 and OPENAI_API_KEY to your ofox key. Restart Codex. Total time: under 60 seconds.

API model IDs: openai/gpt-5.3-codex, openai/gpt-5.5, openai/gpt-5.4, openai/gpt-5.4-mini. The full catalog is at ofox model catalog.

Option B — OpenAI direct API

Pros: Same provider as your ChatGPT plan, no third-party trust questions, full model lineup.

Cons: List pricing (no discount), no per-key budgets without scripting.

How to switch: Get an sk- API key from platform.openai.com/api-keys, set OPENAI_BASE_URL=https://api.openai.com/v1. Restart Codex.

Option C — Amazon Bedrock (June 4, 2026 release)

Pros: Quota lives inside your AWS account, useful if you already pay for AWS.

Cons: AWS-only model catalog (currently a subset of OpenAI), region-specific availability, more complex auth than env vars.

How to switch: Configure Bedrock credentials, set OPENAI_BASE_URL to your Bedrock proxy endpoint. See the changelog for setup steps.

Option D — Switch coding tool entirely

If Codex CLI weekly limits keep biting, you can pivot to Claude Code (on claude-opus-4-8, the current Anthropic flagship as of June 2026 and live on the ofox marketplace) or any other coding agent. The friction is config rewrites — your Codex AGENTS.md, prompts, and harness habits don't transfer cleanly. Reserve this for "I've decided Codex is not for me," not "I just need to ship today."

For a deeper look at how the workflow translates, see the Codex real-world coding workflow writeup.

How to Monitor Codex Status and Cap Future Burn

Once you've stopped the bleeding, set yourself up to never hit the same wall again.

Check `/status` before every heavy task

Make it a reflex. Any task that will exceed 100k input tokens deserves a /status check first. If you are under 30% on either window, switch to API key auth for that session.

Subscribe to the OpenAI status page

status.openai.com shows API outages and degraded Codex behavior in near-real-time. Subscribe to email or RSS — you want to know about Codex degradation before you've spent 10 minutes assuming your config is broken.

Track Codex CLI version

The phantom-limit bug (#19215) was tied to CLI v0.124.0. Pin a known-good version with:

npm install -g @openai/[email protected]

Check the version in use with codex --version. Upgrade after at least one week on the new version to let regressions surface.

Set per-key spend caps when on API auth

If you switched to a metered API (fix #5/#6), set a hard cap. On OpenAI direct, this is "Spend limits" in the dashboard. On ofox, it's the per-key monthly limit on the keys page. A reasonable starting cap for a single-developer pattern: $50/month. Raise it once you've watched a month of actual burn.

Watch your daily spend with a 10-line shell hook

A pragmatic alternative to hoping the dashboard alerts arrive: log every Codex session's token count locally and add up the daily total. Drop this in ~/.codex/hooks/post-session.sh:

#!/bin/bash
# Append per-session token counts to ~/.codex/spend.log
LOG=~/.codex/spend.log
TS=$(date -u +%Y-%m-%dT%H:%M:%SZ)
echo "$TS in=$CODEX_INPUT_TOKENS out=$CODEX_OUTPUT_TOKENS model=$CODEX_MODEL" >> "$LOG"

Then read today's total with:

grep "^$(date -u +%Y-%m-%d)" ~/.codex/spend.log | \
  awk '{for(i=1;i<=NF;i++){if($i~/^in=/){gsub("in=","",$i);ins+=$i};if($i~/^out=/){gsub("out=","",$i);outs+=$i}}} \
       END{printf "today: %d input / %d output tokens\n", ins, outs}'

You will not get bill-accurate dollars (model pricing changes the math) but you will see "today already burned 800k tokens" before you start the next agent loop. The behavioral nudge alone tends to cut weekly burn by 20-30% in practice.

Default to a cheap model

The cheapest reliable model for iterative work is openai/gpt-5.4-mini at $0.638/M input on ofox. Set it as your Codex CLI default model and only switch to gpt-5.3-codex when you explicitly need flagship coding capability. A week of this pattern typically cuts spend by 50%.

For the broader pattern, see the custom model providers BYO setup guide.

Team-shared config (multi-developer setup)

If you are on a team of three or more developers all hitting the same weekly cap, individual ChatGPT subscriptions stop being the right shape. The economics flip:

5 devs × $20 Plus = $100/mo, but caps are individual and unpredictable
5 devs × shared API key with $50/mo cap = predictable $250/mo ceiling, pooled usage

The team-shared pattern most ofox customers run uses one API key per environment (dev, staging, ci) rather than per developer. The config.toml lives in your repo, env vars come from each developer's secret manager. Example committable repo config:

# .codex/config.toml — committed to repo
[model_providers.team]
name = "Team Shared (ofox)"
base_url = "https://api.ofox.ai/v1"
wire_api = "responses"
# Auth comes from each dev's $OPENAI_API_KEY (developer env, not repo)

[profiles.default]
model = "openai/gpt-5.4-mini"
model_provider = "team"

[profiles.heavy]
model = "openai/gpt-5.3-codex"
model_provider = "team"

Each developer sets their own OPENAI_API_KEY locally (pointed at a per-dev or per-team key in your secret manager). Spend monitoring then lives on one ofox dashboard rather than five individual ChatGPT accounts. CI pipelines use a separate key with a stricter per-run cap.

The differentiator over individual ChatGPT subscriptions: when one developer ships a refactor that costs $4 in tokens, the whole team sees it on one dashboard. When five separate ChatGPT subs each drain their weekly cap in the same week, you have no visibility and no shared budget.

A Note on Authenticity and Cost

Switching Codex CLI off your $20 ChatGPT plan onto an OpenAI-compatible API costs around $0.13 for a small bug fix on gpt-5.3-codex — less than the $20 subscription buys you on a single bad afternoon of phantom limits.

That is the real arithmetic. ChatGPT subscriptions are great if your usage is steady and within the cap. The moment your usage becomes burst-shaped — a hard refactor week, a tight CI deadline, a single agent loop that touches 30 files — the subscription cap becomes the wrong abstraction. Token billing pays for what you used and lets you ship.

Originally published on ofox.ai/blog.

MiniMax M3 vs Claude Opus 4.8: 59% vs 69% SWE-Bench, 10 Pricing, Pick (2026)

Owen — Sun, 14 Jun 2026 05:58:13 +0000

MiniMax M3 just landed at 59% on SWE-Bench Pro for one-tenth the price of Claude Opus 4.8 — but the headline that says "M3 beats GPT-5.5" quietly compares it to Anthropic's old flagship.

30-Second Verdict

Question	Answer
Higher SWE-Bench Pro score?	Claude Opus 4.8 (69.2% vs M3's 59.0%)
Cheaper per token?	MiniMax M3 (~10× less on both input and output)
Bigger context window?	Tied — both 1M tokens
Open-weight available today?	Neither, in practice (M3 weights delayed past the promised 10-day window)
Best for routine coding agents?	M3 — quality cost gap closes once you account for $/task
Best for hard multi-file diffs and audit work?	Opus 4.8 — the ~10-point benchmark gap is real

Verdict: if your workload is price-sensitive agent runs, pick MiniMax M3 via minimax/minimax-m3. If your workload is hard reasoning over multi-file PRs, pick anthropic/claude-opus-4.8. The clean way to find out is to swap a string and run both on the same prompt — code at the end of this post.

TL;DR: Which One Should You Pick?

A one-line decision table for the four scenarios that cover ~90% of real coding work:

Scenario	Pick	Why
Lint-fix loops, formatter agents, low-stakes refactors	MiniMax M3	10× cheaper per run; quality difference invisible on simple diffs
Agentic IDE plugins (Cursor, Windsurf, Cline)	MiniMax M3 by default, Opus 4.8 for "explain this bug"	M3 handles tool-loop volume; Opus handles the few prompts that need reasoning
Multi-file refactor where a wrong patch costs a debugging hour	Claude Opus 4.8	10-point SWE-Bench gap = noticeably fewer broken diffs on hard repos
1M-context whole-repo grep+patch	Test both	MSA is faster at long ctx; Opus is more accurate. A/B on your actual repo

The trap is treating this as one decision. Most teams want both models available, routed by task — and that's exactly what ofox's same-base_url swap is built for.

Quick Specs Comparison

All prices verified from the ofox catalog on 2026-06-13. Context and output limits from vendor docs.

Spec	MiniMax M3	Claude Opus 4.8
Model ID on ofox	`minimax/minimax-m3`	`anthropic/claude-opus-4.8`
Input price	$0.60/M tokens	$5.00/M tokens
Output price	$2.40/M tokens	$25.00/M tokens
Cached input price	$0.12/M tokens	$0.50/M tokens
Context window	1M tokens	1M tokens
Max output	131K tokens	128K tokens
Modalities (input)	Text + image + video	Text + image
Vendor SWE-Bench Pro	59.0%	69.2%
Released	2026-06-01	2026-05-28
Open weight?	Promised, weights delayed	No (closed)
Architecture	MiniMax Sparse Attention (MSA)	Dense transformer (Anthropic)

Two specs worth pausing on:

Input price ratio is 8.3×; output ratio is 10.4×. A typical coding agent emits 0.2–0.5 output tokens per input token, so the effective ratio sits between 9× and 10× depending on workload. Round to 10× for back-of-envelope.

Max output is effectively a tie. M3 ships 131K, Opus 4.8 ships 128K — the 3K gap doesn't change the operational shape. Both can emit a small file or a dozen unit tests in one call, and both will need chained calls past roughly 130K.

SWE-Bench Pro: The Number That Started the Story

SWE-Bench Pro is the hardest variant of the SWE-bench family — problems from actively-maintained repositories, multi-file diffs, no public ground-truth leakage. It's the closest thing the field has to a coding benchmark that resists memorization.

Here's where the three frontier models sat in early June 2026:

Model	SWE-Bench Pro	Released	Notes
Claude Opus 4.8	69.2%	2026-05-28	Anthropic-run, official
Claude Opus 4.7	64.3%	2026-04	What MiniMax compared M3 against
MiniMax M3	59.0%	2026-06-01	Vendor-run on own infra, Claude Code scaffolding
GPT-5.5	58.6%	2026-04-23	OpenAI-run
Gemini 3.1 Pro	< 58.6%	2026	Below GPT-5.5 per public leaderboards

The first sentence of MiniMax's June 1 launch announcement reads, essentially: "M3 beats GPT-5.5 and Gemini 3.1 Pro on SWE-Bench Pro at one-tenth the cost." That's correct as printed. What's left out: Anthropic had shipped Opus 4.8 four days earlier with a 69.2% score, and the MiniMax deck compared M3 against the older Opus 4.7 at 64.3%.

Independent verification status is the other footnote. MiniMax ran the eval on its own infrastructure, using Claude Code as the agentic scaffolding, with evaluation logic aligned to the official methodology. The official SWE-Bench Pro leaderboard had not added M3 as of this writing. Treat the 59.0% as a directional signal — it might land at 56% or 61% on a clean third-party run, and either still leaves the same shape: M3 is in the same league as GPT-5.5, one tier below Opus 4.8.

The honest one-line read: the M3 number is real, the marketing framing is selective.

Terminal-Bench 2.1 and Multimodal: Where M3 Closes the Gap

SWE-Bench Pro is one signal. On Terminal-Bench 2.1 — long-horizon terminal execution, the kind of thing a coding agent does when you ask it to "set up the dev environment and run the failing test" — MiniMax reports M3 at 66.0%. That's competitive with Opus 4.8 at similar ranges per Anthropic's release notes, and notably ahead of GPT-5.5. The reason: MSA's decoding speed at long context makes long tool-use loops cheaper to retry, so the agent can recover from more failures within a budget.

Native multimodality is the other pitch. M3 accepts image and video input. Opus 4.8 accepts image input but not video. In practical coding terms this matters for two things: pasting a screenshot of a stack trace, and feeding a short screencast of a UI bug. Both models handle the screenshot case; only M3 handles the screencast.

For 95% of coding work neither of these tips the decision — you're staring at text. They become decisive once you start building agents that actually look at the browser.

Pricing Math: What 1M Tokens Actually Cost

Vendor benchmarks are run on perfect infrastructure. Your bill is run on production traffic. Here are three realistic shapes:

Workload shape	Tokens	MiniMax M3 cost	Claude Opus 4.8 cost	Multiplier
Routine refactor agent (1M in + 200K out)	1.2M total	$1.08	$10.00	9.3×
Heavy code generation (500K in + 500K out)	1M total	$1.50	$15.00	10.0×
Whole-repo grep + patch (1M in + 50K out)	1.05M total	$0.72	$6.25	8.7×
Long-context audit with cache hit (1M cached + 50K out)	1.05M total	$0.24	$1.75	7.3×

Numbers use ofox's published rates verified on 2026-06-13: M3 $0.60/M input / $2.40/M output / $0.12/M cached; Opus 4.8 $5/M input / $25/M output / $0.50/M cached. Math is unit price × token count, no rounding.

The picture changes when you scale to a team. Pick a representative profile — five developers, 100 coding-agent runs per day each, 500K input and 100K output per run, 22 working days per month:

M3 per run: $0.30 + $0.24 = $0.54. Monthly: 5 × 100 × 22 × $0.54 = $5,940.
Opus 4.8 per run: $2.50 + $2.50 = $5.00. Monthly: 5 × 100 × 22 × $5.00 = $55,000.

A five-person engineering org running Opus on default routing burns through a small mortgage every month. The same team on M3-default routing with Opus called only for hard problems (say 10% of runs) pays roughly $11K instead. The price-performance argument for M3 isn't "cheap is fine"; it's that you can spend the saved $44K on running Opus more on the prompts that actually need it.

The "Open-Weight" Caveat: Where Are the Weights?

MiniMax's June 1 announcement positioned M3 as "the first and only open-weight model" combining frontier coding, 1M context, and native multimodality. The weights and technical report were scheduled for Hugging Face and GitHub "within roughly 10 days" of launch — call it the June 10–11 window.

As of June 13, 2026, the MiniMax-M3 GitHub repo still notes: "this model is not yet released — this repository exists so the community can share what they need next." The API is live and you can call M3 via providers including ofox, but you cannot self-host it today. The repo has been frozen on a placeholder for almost two weeks.

This is not a fatal point — vendors slip weight releases all the time, and "10 days" was a soft window, not a contract. But it changes the practical comparison. If you picked M3 specifically because the weights would land in your private cluster within two weeks, that bet has not paid off yet. For now, both MiniMax M3 and Claude Opus 4.8 are API-only from a deployment perspective; the open-weight axis isn't decisive in June 2026.

When the weights do ship, the math changes again. A self-hosted M3 cluster amortizes against your GPU lease, not per-token pricing — for sustained 24/7 workloads that's a fundamentally different cost curve from per-token Opus.

When to Pick MiniMax M3

Pick minimax/minimax-m3 if any of the following is true:

You're running coding agents at volume. Lint-fixer bots, formatter loops, codemod agents, "write the docstring" pipelines. These are dominated by token cost, not per-prompt quality, and M3's 10× pricing edge dwarfs the ~10-point quality gap.
You're paying for long-context input. Whole-repo prompts (1M tokens of code in, small diff out) are where MSA's decoding speed and M3's input pricing compound. A million cached tokens on M3 costs $0.12 versus $0.50 on Opus.
Video input is a hard requirement. Opus 4.8 accepts images but not video. If your agent needs to look at a 30-second screen recording of a UI bug, you have one option in this comparison.
You're hedging against the Opus 4.8 price tier. Even teams that prefer Opus 4.8 for primary work route routine prompts to a cheaper model. M3 is currently the strongest sub-$1/M coding option that also tops 1M context.
You'll switch if and when independent SWE-Bench Pro reruns come in lower. Treat the 59% as provisional. Build your stack so swapping minimax/minimax-m3 for the next cheap challenger is one config change away.

When to Pick Claude Opus 4.8

Pick anthropic/claude-opus-4.8 if any of the following is true:

A wrong patch costs more than a token bill. Production hotfixes, security-sensitive refactors, anything where you'd review the diff yourself before merging anyway. The ~10-point SWE-Bench Pro gap is concentrated on the hardest problems — not the median ones.
You're building reasoning-heavy agents. "Read this incident postmortem and propose three fixes." "Audit this OAuth flow and find the bug." Opus 4.8's reasoning gains over 4.7 are tangible per Anthropic's release notes and per independent reviews like Simon Willison's.
You're already in the Anthropic ecosystem. Claude Code, Anthropic's MCP tooling, dynamic workflows — all of these assume Anthropic-style tool semantics. M3 works with Claude Code (MiniMax themselves used it as scaffolding) but you'll hit edge cases on tool format expectations.
The "Fast mode" cost tier suits your shape. Opus 4.8 introduced a $10/M input / $50/M output Fast mode tier for latency-sensitive use cases. It's more expensive than the regular tier but less than calling Opus 4.7 and waiting longer.
Your eval harness already calibrates against Opus. If your team has a "would the senior reviewer accept this PR" eval suite that's been tuned against Opus outputs, switching models invalidates your eval until you re-baseline. That's real engineering cost, not vibes.

When NOT to Pick Either (and What to Use Instead)

A few scenarios where this whole comparison is the wrong question:

Sub-$0.10/M-token budget, simple refactors. Look at smaller models like Claude Haiku 4 or GPT-5.4 Mini. Spending $0.60/M on M3 when GPT-5.4 Mini at $0.10/M would do the same lint-fix is theater.
You need on-prem deployment today. Both M3 (weights not shipped) and Opus 4.8 (closed) are API-only. Self-host options for frontier coding today are Qwen 3.7 Max and the open Chinese model lineup.
You're optimizing for a strict latency SLA, not cost. Both M3 and Opus 4.8 are designed for quality, not p50 latency. Smaller faster models will beat both on TTFT.
You need to evaluate multiple frontier models at once. Build a comparison harness instead of picking one.

Try Both via ofox: A/B in 10 Lines of Code

The whole comparison reduces to a one-string change if you call both models through ofox's OpenAI-compatible endpoint. Same base_url, same SDK, just swap the model argument.

Python — A/B both models in one loop

from openai import OpenAI

client = OpenAI(api_key=OFOX_API_KEY, base_url="https://api.ofox.ai/v1")
PROMPT = "Refactor this function to remove duplication: ..."

for model in ["minimax/minimax-m3", "anthropic/claude-opus-4.8"]:
    resp = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": PROMPT}],
    )
    print(model, resp.usage.total_tokens, resp.choices[0].message.content[:120])

Run this and you get per-model token usage and the first 120 characters of each output for eyeball comparison. Plug the total_tokens numbers into the pricing math table above and you have a per-run cost on a real prompt rather than a vendor benchmark.

Node — same shape

import OpenAI from "openai";

const client = new OpenAI({ apiKey: process.env.OFOX_API_KEY, baseURL: "https://api.ofox.ai/v1" });
const prompt = "Refactor this function to remove duplication: ...";

for (const model of ["minimax/minimax-m3", "anthropic/claude-opus-4.8"]) {
  const r = await client.chat.completions.create({ model, messages: [{ role: "user", content: prompt }] });
  console.log(model, r.usage.total_tokens, r.choices[0].message.content.slice(0, 120));
}

Identical shape, identical endpoint, identical SDK call. The migration cost between models is one string.

For a multi-turn agent loop that includes tool calls, the same swap works — both models accept OpenAI-style tools arrays via ofox. You'll want to test the tool-call format on your specific tools because each provider's strict mode handling diverges at the edges, but the contract is the same.

Compatibility Quirks: What Differs Between the Two APIs

Same endpoint, same SDK call — but a few sharp edges worth knowing before you wire either model into production.

System prompt handling. Claude Opus 4.8 treats the system role as a strict system prompt with elevated trust. MiniMax M3 (via the OpenAI-compatible path) folds system into the conversation more loosely. If your agent depends on system-prompt-only constraints — "never call this tool unless asked," "always respond in JSON" — M3 follows them most of the time but is statistically more likely to drift on long tool loops. Workaround: repeat critical constraints in the first user message.

Tool-call format strictness. Opus 4.8 enforces tool argument schemas hard — it will refuse to call a tool if your parameters JSON Schema marks a field required and the model can't fill it. M3 is more lenient and will sometimes emit a tool call with a placeholder string. If your tool layer treats placeholders as valid, you'll silently execute wrong actions; if it validates strictly, you'll see more retry loops. The fix is the same either way: validate tool arguments on the server side, not just at the model.

Caching semantics. Both models offer cached input pricing, but Anthropic splits the bill into write and read. On Opus 4.8 you pay a one-time cache write at $6.25/M (5-minute TTL) or $10/M (1-hour TTL), then every subsequent cache read lands at $0.50/M. M3's cache on ofox is a single $0.12/M read rate with implicit TTL and no separate write surcharge. For workloads that hit the same long-context prompt many times per minute, M3 is dramatically cheaper at the cache read layer. For workloads where the cached portion stays warm for hours and write costs amortize across many reads, Opus 4.8's 1-hour tier is competitive on a per-token basis even before quality.

Streaming chunk shape. Both models stream OpenAI-compatible chunks, but Opus 4.8 emits more granular delta.thinking events when extended thinking is enabled. If your client parses thinking deltas separately from content deltas, that code works against Opus but no-ops against M3, which doesn't currently expose thinking deltas through the OpenAI-compatible route. Not a bug — just an unused field.

Rate limits at the provider edge. When you call both models through ofox, you share one rate limit envelope keyed to your API key — not two separate per-vendor quotas. That's the point of the gateway shape: M3 fallback when Opus is rate-limited, Opus fallback when M3 is, all without juggling two sets of credentials.

The whole MiniMax M3 vs Claude Opus 4.8 question collapses to one string swap on the same endpoint — which is the only sane way to pick a coding model in 2026.

Originally published on ofox.ai/blog.

DeepSeek V4 Pro Real Cost: 120x Cache Gap Behind Sticker

Owen — Fri, 12 Jun 2026 08:03:37 +0000

DeepSeek V4 Pro Real Cost: 120x Cache Gap Behind Sticker

TL;DR — DeepSeek V4 Pro's "$0.28/M" reputation is wrong on two counts. First, $0.28/M is the output price of V4 Flash, not Pro — Pro is $0.87/M output. Second, the published $0.435/M input price assumes a cache miss. Cache hits cost $0.003625/M, a 120x spread that quietly determines whether your bill matches the headline. To show why the published price isn't the bill, we ran V4 Pro, GPT-5.5, and Claude Sonnet 4.6 against the same refactoring task on api.ofox.ai. The mechanism is the lesson, not the leaderboard: V4 Pro finished in ~1,500 output tokens, GPT-5.5 ran into our 8,192-token cap on 2 of 3 runs, Sonnet 4.6 used ~3,800. Output verbosity × output sticker price compounds into an order-of-magnitude gap on a reasoning-heavy task like this one. The exact "Nx cheaper" multiplier depends on caps, quality gates, and sample size we explicitly didn't lock down — see the methodology limits below. Pricing verified at api-docs.deepseek.com/quick_start/pricing on 2026-06-12.

Try it yourself, with the 75% permanent discount built in. Run V4 Pro through ofox.ai on the same API key that already covers Claude, GPT, and Gemini — same V4 Pro price as the official endpoint, no Chinese phone number required, no separate account.

The three variables that decide your bill

A model's sticker price is the smallest input to your monthly invoice. Three variables matter more:

Cache miss rate. Cache-miss input on V4 Pro is $0.435/M. Cache-hit is $0.003625/M. That is not a 90% discount, that is a 120x discount. A workload that runs at 95% cache hit costs a different order of magnitude than one running at 60% hit. Most "DeepSeek is 50x cheaper than Claude" comparisons silently assume the high-hit case.
Thinking-mode output inflation. V4 Pro's thinking mode is on by default. On a single refactoring prompt in our benchmark, V4 Pro emitted 1,556 output tokens; GPT-5.5 emitted 8,192 (it hit our max_tokens cap) for the same prompt. Output is also where the real spend lives — $0.87/M on V4 Pro vs $0.435/M for cache-miss input. Verbose models burn the more expensive token.
Tokenizer drift. Different providers count the same English text slightly differently. For our 1,400-character prompt, DeepSeek counted 339 input tokens, OpenAI counted 331, Anthropic counted 396 — a ~20% spread between the lightest and heaviest counter. Small. But it's measurable and asymmetric: Anthropic tends to count more.

These three variables compound. A 60% cache hit rate with a verbose model and a heavy-counting tokenizer can turn a "12x cheaper" provider into "3x cheaper" on the actual invoice. None of that is misrepresentation by DeepSeek — it's what cache pricing always means, just at an unusually large spread.

What "$0.28/M" actually refers to

The reason $0.28/M shows up in conversations about V4 Pro pricing is that V4 Flash, the smaller sibling, lists its output at exactly that number. Once a price gets repeated on social media, it stops carrying the model name attached. Here's the full table, verified against the official DeepSeek docs on 2026-06-12:

Tier	Model	Input (cache-miss)	Input (cache-hit)	Output
Premium	`deepseek-v4-pro`	$0.435/M	$0.003625/M	$0.87/M
Budget	`deepseek-v4-flash`	$0.14/M	$0.0028/M	$0.28/M

Source: api-docs.deepseek.com/quick_start/pricing, accessed 2026-06-12. Both models advertise a 1M-token context window and a 384K max output. Pricing is in USD per 1M tokens.

The original V4 Pro launch listed regular prices of $1.74/M input cache-miss and $3.48/M output, with a 75% promotional discount running through May 31, 2026. In late May, the 75% discount was reportedly made permanent — DeepSeek's official docs no longer show a "promo until" footnote, and the community announcement thread on r/GithubCopilot (340 upvotes, 71 comments) summarizes the change:

DeepSeek just made the 1/4 discounted price for v4 Pro permanent. […] It increases the gap with the frontier (Sonnet/GPT 5.4) models to a 12 to 17x difference. And we are not even talking about the cache hit, where the difference is easily 60 to 80x cheaper.

The 75% discount being permanent matters because it locks in the cache-hit/miss spread at 120x. At the original sticker price the spread was the same ratio but in absolute terms looked different — at the new "permanent" price the spread is large enough that hit-rate behavior dominates your bill outright.

A one-task mechanism probe (not a leaderboard)

We ran one workload to make the mechanism visible — not to publish a Nx ranking. Three models, one task, three runs each, same single user message to all three. Same prompt body, temperature=0.2, max_tokens=8192. All three were called through api.ofox.ai/v1, an OpenAI-compatible router that exposes V4 Pro, GPT-5.5, and Sonnet 4.6 on the same key. Verbatim prompt:

You are a senior TypeScript engineer.

Refactor the following module into idiomatic TypeScript with these constraints:
- Replace any "any" types with precise generics.
- Extract pure helper functions.
- Add Vitest unit tests covering: happy path, empty input, malformed JSON, and a network timeout.
- Use AbortController for the timeout.
- Return a single fenced ```ts code block with the refactored module followed by a separate fenced ```ts block with the tests.
- Do not include prose commentary outside the code blocks.

```ts
// fetchAndAggregate.ts — original
export async function fetchAndAggregate(urls, opts) {
  const results = [];
  for (const u of urls) {
    try {
      const r = await fetch(u, opts);
      const j = await r.json();
      if (j && j.records) {
        for (const rec of j.records) {
          if (!rec.skip) results.push(rec);
        }
      }
    } catch (e) {
      console.log('failed', u, e);
    }
  }
  // group by category
  const out = {};
  for (const r of results) {
    const k = r.category || 'misc';
    if (!out[k]) out[k] = [];
    out[k].push(r);
  }
  // sort each group by ts desc
  for (const k of Object.keys(out)) {
    out[k].sort((a, b) => (b.ts || 0) - (a.ts || 0));
  }
  return out;
}
```

Paste that into any OpenAI-compatible client (point baseURL at https://api.ofox.ai/v1 or your provider of choice) with the parameters above and you'll re-run exactly what we ran.

Median of 3 runs, on 2026-06-12. The right way to read this table is output verbosity, not the dollar column — the dollar column is illustrative because GPT-5.5's outputs are capped at 8,192 (see methodology limits):

Model	Median wall-clock	Input tokens	Output tokens	`finish_reason`	Illustrative cost per run*
`deepseek/deepseek-v4-pro`	18.8 s	339	1,556	`stop` (3/3)	$0.0015
`openai/gpt-5.5`	117.4 s	331	8,192	`length` (2/3), `stop` at 7,961 (1/3)	$0.2474 (truncated)
`anthropic/claude-sonnet-4.6`	38.9 s	396	3,880	`stop` (3/3)	$0.0594

*Computed using each provider's published sticker price (DeepSeek $0.435/$0.87, OpenAI $5/$30, Anthropic $3/$15, USD per 1M tokens). The GPT-5.5 dollar number is a lower bound — its outputs were truncated by our cap on 2 of 3 runs, and a complete run would cost more.

What the data does say with confidence:

Verbosity is doing more of the work than per-token price. V4 Pro emitted 1,556 output tokens; Sonnet emitted 3,880; GPT-5.5 emitted 8,192+ before our cap stopped it. The model-behavior gap (5x verbosity between V4 Pro and GPT-5.5) compounds with the sticker gap (34x output price between V4 Pro and GPT-5.5). That's a mechanism statement and it doesn't depend on whether the GPT-5.5 cost number is exact.
None of the three runs reported cached tokens. Each call was a cold prompt. On a real Claude Code or OpenCode session, the system prompt and project context stay stable across turns and the cache fires. Our cold-start numbers are an upper bound on V4 Pro's per-call cost. If you sustain a 70% input cache hit, V4 Pro's effective input cost drops from $0.435/M to ~$0.13/M.
Tokenizer differences exist but are small here. DeepSeek counted 339 input tokens, OpenAI 331, Anthropic 396 for the same English prompt. Anthropic counts heaviest, but not by enough to change the conclusion. On longer prompts with code, the spread widens — worth eyeballing if you're doing nine-figure-token workloads.

Methodology limits (read this before you quote the numbers)

The numbers above are honest about what they are: a single mechanism probe, not a benchmark you can quote a clean "Nx cheaper" multiplier from. Four limits, in order of how much they would change the headline:

GPT-5.5 was truncated on 2 of 3 runs. Our max_tokens=8192 cap is the reason 2 of 3 GPT-5.5 runs report finish_reason: "length" — those outputs are unfinished. A fair version of this benchmark would set max_tokens high enough that no model is artificially cut off, or discard length-terminated runs from the cost average. Until that's redone, the GPT-5.5 cost column is a lower bound and the V4-Pro-vs-GPT-5.5 cost ratio is best stated as "an order of magnitude on this kind of task," not "Nx."
No quality gate. We did not run the generated code or its tests through Vitest. "V4 Pro is cheaper" only matters if the output is also correct — a model that ships syntactically valid code but with broken tests isn't actually cheaper. A proper rerun would extract each model's tsblocks, write them to disk, runvitest run, and only countfinish_reason=stop` + tests-pass runs into the cost comparison.
n=3 with no variance reported. Three runs are enough to spot a mechanism (cap hits, verbosity, latency) but not enough to claim a stable rate. A defensible rerun would be n≥10 with median and min/max.
One task. This is a TypeScript refactor with test scaffolding. The conclusion does not generalize to "V4 Pro is an order of magnitude cheaper on every task." On a one-shot prose generation, on tool-heavy agent loops, or on a Flash-suitable scaffold, the ranking changes — sometimes inverts. The honest framing: V4 Pro is cheaper on reasoning-heavy refactors where its smaller output footprint is what matters.

We're doing the proper rerun (max_tokens lifted, Vitest gate applied, n≥10, full per-run usage and finish_reason published) as a follow-up — link will land in this section when it ships. Until then: trust the mechanism, treat the multipliers as illustrative.

What real bills look like in the wild

Our benchmark is one task across three models. The community has been posting real monthly bills for weeks. Sampled from r/DeepSeek, r/opencode, r/Anthropic, r/LLMDevs, and r/GithubCopilot — bills are real, screenshots posted by users on their own accounts:

"I just spent $2 over two days with DS on OpenCode. I'd be so upset if I'd spent $265 with Claude for the same thing." — u/pepeperezcanyear on r/DeepSeek "Pricing is crazy" (1,027 upvotes, 149 comments).

"200M tokens total. roughly 70/30 split on prompt vs completion. came out under 35 bucks all in. […] for context, when we were on claude pro for similar workload the per-seat math was 6x that and we had to babysit context limits. when we tested gpt-5.5-codex on the same kind of work the per-token was 8-10x and the wall time was worse." — u/Fun_Walk_4965, r/DeepSeek, 189↑/132💬.

"With $3.88 & 690,003,591 tokens and 5 hours, Deepseek Pro & Flash combined, managed to reverse engineer Teamspeak's Licensing System […] In 5 hours of trial and error, debugging with Ghidra and x64dbg." — original post on r/DeepSeek, 365↑/48💬. Note the top reply: "Less than 1% of those tokens are output tokens" (u/—Spaci—, +23) — meaning the cache was doing real work on this run.

"Deepseek V4 current cost: 78.2m tokens for $1.14. What's yours?" — r/opencode, 361↑/55💬. A reply from u/Still-Notice8155 (+5): "mine roughly 85m/1$ using pro, it's insane with opus 4.6 like quality."

"65 million tokens for 7 dollars lol" — r/DeepSeek, 170↑/93💬. Top reply, with a screenshot: "You spent too much lmfao. 680 million for 14" (u/deleted-account69420, +55).

The thread on r/Anthropic — where you'd expect skepticism — is 232 upvotes and 123 comments deep, and the highest-rated reply (u/KaMaFour, +74) gets the warning exactly right:

PRICE PER MILLION TOKENS IS NOT A GOOD MEASURE OF THE COST BECAUSE IT DOESN'T TAKE INTO ACCOUNT THE VERBOSITY OF THE MODEL. 5x cheaper model per token can be as expensive if it uses 5x more tokens per task.

That's the cleanest version of the GPT-5.5 result above. Per-token price is half the story.

The counter-narrative: when V4 Pro isn't cheap

It's not all "$2 for two days." Worth giving the dissent a real airing. From r/LLMDevs "Token costs are actually unsustainable for multi-project work" (32↑/94💬), top reply from u/look (+11):

"I primarily use a mix of Mimo V2.5 Pro, GLM-5.1, Qwen 3.6 Plus, and Deepseek V4 Flash (don't waste your time with Pro — it's as expensive as US models in actual use) and my average blended token costs are under 5 cents per Mtok and still dropping."

The "Pro is as expensive as US models in actual use" claim is the inverse of our benchmark result. Both can be true. Pro's pricing assumes you don't blow through cache, and assumes the task benefits from thinking output enough to justify the higher per-token cost. If you're driving Pro on prompts where Flash would have answered correctly, you are paying the Pro tax — 3x more input, 3x more output — for output you didn't need. Our benchmark task was deliberately reasoning-heavy (refactor + write tests + reason about edge cases). On a one-shot "write a function that sums an array" prompt, the conclusion flips and Flash wins decisively. We covered that tradeoff in detail in our earlier piece on DeepSeek V4 Pro vs Flash: real cost-quality tradeoff.

The honest answer: on the reasoning-heavy refactor we tested, V4 Pro produced output an order of magnitude cheaper than GPT-5.5 and Sonnet 4.6 — most of that gap coming from output verbosity, not per-token price. On tasks Flash can handle, Pro is not cheap relative to Flash. And the exact multiplier we publish above is one mechanism probe, not a benchmark you can A/B procurement on (see methodology limits). Route by task type, not by default.

What to do with this

Three concrete moves, in order of how much they actually affect your bill:

Measure your cache hit rate before you trust any "Nx cheaper" claim. Pull a week of usage from your API dashboard. DeepSeek reports prompt_cache_hit_tokens and prompt_cache_miss_tokens per call. If your hit rate is 80%+, the published savings numbers apply roughly as-is. If you're at 60%, multiply the cache-miss cost by 0.4 and the cache-hit cost by 0.6 and recompute. The math in the cache-hit math primer walks through this step by step.
Route reasoning to Pro, scaffolding to Flash. On a one-shot CRUD scaffold, Flash produces output that's hard to distinguish from Pro on a blind read, and it's 3x cheaper at every tier. On a multi-file refactor with implicit invariants, Pro holds the constraints across the rewrite; Flash drifts. The DeepSeek API pricing guide has a per-task decision table.
Run cold-start prompts through a router that lets you A/B. Routing through a unified endpoint like ofox.ai's API means you can ship the same prompt to V4 Pro, GPT-5.5, and Sonnet 4.6 with a one-character model-ID change. We use it ourselves — the benchmark in this post took three runs per model on one key, all three models reachable through the same endpoint. If you're still budgeting against someone else's sticker price, that's the cheapest experiment to run before you commit.

The sticker price is a number on a tag. The bill is a receipt that unspools downward. Both are real; only one of them gets paid.

Methodology footnote. Benchmark task: TypeScript refactor + add Vitest tests. 3 runs per model, median reported, no Vitest gate applied. Routed through https://api.ofox.ai/v1 on 2026-06-12, temperature=0.2, max_tokens=8192. Finish reasons from raw runs: V4 Pro stop × 3; Sonnet 4.6 stop × 3; GPT-5.5 length × 2 (truncated at 8,192) + stop × 1 (at 7,961 tokens, hugging the cap). Cost computed using each provider's published sticker price (USD per 1M tokens): V4 Pro $0.435 input cache-miss / $0.87 output (verified at api-docs.deepseek.com/quick_start/pricing 2026-06-12); GPT-5.5 $5 input / $30 output (verified at ofox.ai/models/openai 2026-06-12); Sonnet 4.6 $3 input / $15 output (verified at ofox.ai/models/anthropic 2026-06-12). Cached-token columns were 0 across all runs (cold prompts). The prompt body and per-run breakdown are in the post so you can repeat the same probe on your own key. Known limits (see in-post section): GPT-5.5 cap hit, no quality gate, n=3, single task.

Originally published on ofox.ai/blog.

Claude Code Safe Mode: 5 Things Disabled + When to Use Over /clear (2026)

Owen — Thu, 11 Jun 2026 06:59:53 +0000

You upgrade to Claude Code v2.1.169. The first thing you notice is that your old /clear reflex no longer fixes everything. CLAUDE.md is stale, a plugin is shadowing a real command, your MCP server stopped responding, and a hook is silently rewriting every diff. Safe Mode is the kill switch you reach for when /clear only wipes the conversation but the problem lives in your config.

This guide covers the five things --safe-mode actually disables, exactly when to reach for it instead of /clear, and how to enable it in under 10 seconds on macOS, Windows, and WSL — with a triage workflow you can hand a teammate without explanation.

What You Can Do After Enabling Safe Mode (And What You Can't)

Before you run the flag, know which problems it solves and which it leaves on the floor.

Outcome	Safe Mode	`/clear`	Reinstall
Disable a broken CLAUDE.md	✅	❌	❌
Disable a misbehaving plugin	✅	❌	partial
Disable hooks rewriting tool output	✅	❌	❌
Disable a hung MCP server	✅	❌	partial
Hide bundled skills from the model	✅	❌	❌
Wipe runaway conversation context	❌	✅	❌
Reset API credentials	❌	❌	❌
Reset the model used	❌	❌	❌
Reset slash-command history	❌	❌	❌
Take less than 10 seconds	✅	✅	❌

Decision in one sentence: if /clear does not fix it within one or two turns, reach for --safe-mode before you start uninstalling things.

Decision Frame: When to Use Safe Mode (and When NOT)

When to use

Claude Code starts behaving differently after upgrading from v2.1.16x to 2.1.169 — the upgrade is the only variable, so cancel your customizations to confirm.
A coworker reports "it works for me, breaks for you" — strip your CLAUDE.md and plugins, see if the bug survives.
A plugin or MCP server installed in the last 24 hours starts blocking, throwing, or producing wrong tool output — isolate before you uninstall.
Hooks are silently mutating tool calls and you are not sure which one — turn them all off in one shot.
You inherit a workstation set up by someone else and want to see the "stock" CLI behavior before opting into anyone's config.

When NOT to use

You only need to wipe context — use /clear, do not relaunch.
You only need to swap models — set ANTHROPIC_MODEL or use the model picker, not safe mode.
You suspect a credential issue — safe mode does not touch ANTHROPIC_API_KEY or ANTHROPIC_BASE_URL; check those first.
You want to disable one plugin out of five — edit your settings or the plugin directly; safe mode is all-or-nothing.
You are mid-task with valuable conversation state — safe mode launches a fresh session, so any work-in-progress context is gone.

Stop rule

If safe mode reproduces the same bug as a normal launch, the problem is not in your customizations. Stop reading this guide, do not waste time pruning plugins, and start looking at the model, network, account, or upstream API.

What `--safe-mode` Disables: The Five Layers (Verbatim from v2.1.169)

The official v2.1.169 release notes describe the flag in one sentence: "start Claude Code with all customizations (CLAUDE.md, plugins, skills, hooks, MCP servers) disabled for troubleshooting." Each of those five layers maps to a distinct failure mode.

#	Layer	What gets disabled	Most common failure it fixes
1	CLAUDE.md	Project root + parent + `~/.claude/CLAUDE.md` global	Stale instructions overriding the user prompt; contradictory team rules
2	Plugins	All entries in your plugin directory, including marketplace plugins	Plugin shadowing a built-in command; plugin crashing on cold start
3	Skills	User-added skills under `~/.claude/skills/`. To also hide built-in / bundled skills, combine with `CLAUDE_CODE_DISABLE_BUNDLED_SKILLS=1`	Skill being invoked for the wrong intent; skill emitting bad tool calls
4	Hooks	Every hook event you have registered — across all ~30 documented types (full list in the hooks reference) including `PreToolUse`, `PostToolUse`, `UserPromptSubmit`, `Stop`, `Notification`, `SessionStart`, `SessionEnd`, `PreCompact`, `SubagentStart`/`Stop`, and the rest	Hook rewriting diffs or blocking tool calls silently
5	MCP servers	Every MCP server, whether from settings, `--mcp-config`, or IDE-typed configs	MCP server hung on startup; MCP tool list collision; auth loop

What it does not disable: your API key, the configured base URL (so your ofox endpoint stays live), the model name in ANTHROPIC_MODEL, your conversation history file, your slash-command keybindings, or the trust dialog for the current project.

Layer 1: CLAUDE.md — the silent system prompt extender

CLAUDE.md is loaded from up to three locations on every session start: the project root, every parent directory walked toward /, and your global ~/.claude/CLAUDE.md. Whatever you write there is injected ahead of every user prompt. When safe mode is on, the model behaves as if none of those files exist. This is the first layer to suspect when "the same prompt produces different output today than yesterday" — someone (or you) likely added a new rule in CLAUDE.md that pulls behavior in an unexpected direction.

Layer 2: Plugins — the third-party extension surface

Plugins live in ~/.claude/plugins/ (or your configured plugin directory) and can register slash commands, intercept tool calls, expose new MCP servers, or modify the prompt. Safe mode skips the plugin loader entirely, which means a plugin that crashes on cold start, shadows a built-in command like /clear, or holds a Windows file lock will not appear at all. Plugin issues are the second most common "weird behavior after update" cause after CLAUDE.md.

Layer 3: Skills — both user-added and (optionally) bundled

User-added skills under ~/.claude/skills/ are disabled by safe mode automatically. Bundled skills, workflows, and built-in slash commands are not disabled by safe mode alone — they require the separate CLAUDE_CODE_DISABLE_BUNDLED_SKILLS=1 env var or disableBundledSkills: true setting (also new in v2.1.169). For most triage you want only safe mode; reach for the bundled-skills flag when you suspect a built-in skill is being chosen over your own logic.

Layer 4: Hooks — the silent tool-call mutators

Hooks fire on dozens of events — PreToolUse, PostToolUse, UserPromptSubmit, Stop, Notification, SessionStart, SessionEnd, PreCompact, SubagentStart/Stop, and more — covering tool calls, session lifecycle, and async events. A hook can rewrite the model's tool input, block a tool from executing, swallow output, or just print extra chatter. They are the hardest layer to debug manually because their effect is silent and out-of-band. Safe mode disables every hook in one shot — usually the fastest way to confirm "is a hook eating my diffs?"

Layer 5: MCP servers — the longest-tail surface

MCP servers come from your settings file, --mcp-config flags, IDE-typed configs, and v2.1.169 enterprise-managed allowedMcpServers/deniedMcpServers policies. They can hang on cold start, collide on tool names, throw auth loops, or simply respond too slowly to be useful. Disabling all of them in one flag lets you ask "would this session work with zero MCP at all?" — a question that is otherwise expensive to answer because most users have between three and ten MCP servers configured.

System Requirements

Claude Code ≥ v2.1.169 (June 8, 2026). Verify with claude --version. If you see 2.1.168 or below, upgrade first: npm install -g @anthropic-ai/claude-code for the npm install, or run the official installer.
A terminal you can set env vars in: zsh, bash, fish, PowerShell, CMD, or WSL all work.
Existing API credentials: ANTHROPIC_API_KEY (or your ofox-style key under the same name) must still be configured. Safe mode disables customizations, not auth.
(Optional) Write access to your shell rc: if you want to persist the flag for a triage session, you will set CLAUDE_CODE_SAFE_MODE=1 in ~/.zshrc, ~/.bashrc, or PowerShell $PROFILE.

Step-by-Step: Enable Claude Code Safe Mode

Step 1: Confirm your version is v2.1.169 or newer

claude --version
# Expect: 2.1.169 (or higher)

If it returns an older version, run the upgrade matching how you installed it, then relaunch your terminal to make sure the new binary is on your PATH.

Step 2: Launch with the flag (one-shot)

macOS / Linux / WSL:

claude --safe-mode

Windows PowerShell:

claude --safe-mode

Windows CMD:

claude --safe-mode

The flag is the same everywhere; only the shell wrapping changes. The session that opens will skip every CLAUDE.md, plugin, skill, hook, and MCP server.

Step 3: (Alternative) Persist with the env var for a triage session

When you want every claude invocation in a shell to be safe — useful when you are debugging across multiple panes or repos:

macOS / Linux / WSL (bash, zsh):

export CLAUDE_CODE_SAFE_MODE=1
claude

Windows PowerShell:

$env:CLAUDE_CODE_SAFE_MODE = 1
claude

Windows CMD:

set CLAUDE_CODE_SAFE_MODE=1
claude

Verify the env var is live before you launch:

echo $CLAUDE_CODE_SAFE_MODE
# Expect: 1

Step 4: Confirm safe mode is active

Once in the session, type a prompt that should trigger a project rule from your CLAUDE.md. If the rule is ignored, safe mode is on. You can also list your loaded MCP servers — the list should be empty.

Step 5: Exit safe mode cleanly

For a one-shot launch, just /quit and start claude again without the flag.

For the env-var path, unset it before you relaunch:

# macOS / Linux / WSL
unset CLAUDE_CODE_SAFE_MODE

# PowerShell
Remove-Item Env:CLAUDE_CODE_SAFE_MODE

# CMD
set CLAUDE_CODE_SAFE_MODE=

Then check your rc files (~/.zshrc, ~/.bashrc, $PROFILE) for a stale export that would re-arm safe mode on the next shell. This is the single most common reason people report "Claude Code still ignores my CLAUDE.md" days after they thought they had turned safe mode off.

Step 6: The 3-step triage workflow

Once safe mode confirms the bug is in your customizations, narrow it down with a binary-search pattern instead of un-installing everything at once:

Relaunch normally, keep CLAUDE.md and plugins, manually disable hooks and MCP via settings. Reproduce? If yes, the culprit is in CLAUDE.md or a plugin — usually the larger and more recent layer.
Re-enable hooks, keep MCP off. Reproduce? If yes, the culprit is a hook. Disable hooks one at a time; v2.1.169 hooks are evaluated in load order, so disable the most recently added hook first.
Re-enable MCP servers one by one. The one that reintroduces the bug is your suspect. Restart the session between each MCP toggle — MCP servers cache their tool list at session start, so live toggling without a restart can give you a false negative.

You almost never need step 3 to finish — the bug usually surfaces by step 1 or 2. Step 3 only matters when you run many MCP servers, which is increasingly common in larger teams.

Step 7: Capture a clean repro for the bug tracker

After you have identified the offending layer, run safe mode one more time with the single offending file copied into place. This isolates the bug to one customization and gives you a tight repro to file. The repro should fit in two lines: "I added ~/.claude/plugins/foo/index.ts, with safe mode I get behavior A, with that plugin re-enabled I get behavior B." This is the format the Claude Code maintainers respond to fastest.

Common Errors During Safe Mode Setup (and Fixes)

Symptom	Root cause	Fix
`Unknown flag: --safe-mode`	Claude Code is on v2.1.168 or older	Upgrade to v2.1.169+, relaunch terminal so `PATH` picks up the new binary
Safe mode "doesn't seem to do anything" — CLAUDE.md still applies	Two `claude` binaries on `PATH`; older one wins	`which claude` (Mac/Linux) or `where claude` (Windows); remove or rename the older copy
`CLAUDE.md still loads` despite the flag	You confused `--safe-mode` with `/clear`; you ran `/clear` inside an already-loaded session	Quit the session entirely (`Ctrl-D` or `/quit`), then relaunch with the flag
MCP servers still appear in the slash-command list	Stale plugin cache on Windows (a known v2.1.169 fix for MCPB plugin cache)	Quit, delete the MCPB plugin cache directory, relaunch with `--safe-mode`
`claude -p` hangs forever on Windows after enabling safe mode	Unrelated regression already fixed in v2.1.169 (skill-scan stall)	Confirm you are on `>=2.1.169`; if so, file an issue with `--verbose` output
`CLAUDE_CODE_SAFE_MODE` set but env var has spaces around `=`	Bash treats `CLAUDE_CODE_SAFE_MODE =1` as a command, not an assignment	`export CLAUDE_CODE_SAFE_MODE=1` with no spaces — strict, no shortcuts
Safe mode disables your hook that does mandatory pre-commit signing — now your commits are unsigned	Working as designed; safe mode is for diagnosis, not for everyday work	Exit safe mode before any commit you intend to push; reserve safe mode for read-only triage
Enterprise managed `allowedMcpServers` list still seems "active"	Safe mode disables MCP servers, not the policy schema parser; policy text remains in settings	Treat the policy file as dormant in safe mode — it is parsed but no MCP servers run

Team / Multi-Developer Configuration

Safe mode is the cleanest reproducibility primitive your team has when a bug report comes in. Wire it into your runbook.

Shared triage alias

Drop a one-liner into your team's dotfiles repo so every engineer has the same entry point:

# ~/.zshrc or ~/.bashrc, committed to the team's dotfiles repo
alias claude-triage='CLAUDE_CODE_SAFE_MODE=1 claude'

When a teammate reports a Claude Code bug, the first request from the on-call is "run claude-triage, paste the repro." That is one command, no context, no version drift.

Triage matrix

Bug surface	Reporter runs	Expected if bug is in their config	Expected if bug is upstream
Tool output looks rewritten	`claude-triage` then repro	Bug disappears → hook is the culprit	Bug persists → model / API issue
Slash command shadowed	`claude-triage` then `/help`	Built-in command reappears → plugin is shadowing	Command still missing → CLI bug
CLAUDE.md rules ignored	`claude-triage` then ask a question only the rule would answer	Behavior identical → your CLAUDE.md was unreachable anyway (path issue)	Behavior changes only in safe mode → CLAUDE.md was loading but losing
MCP tool returns errors	`claude-triage` then attempt task	Task succeeds without MCP → server is broken	Task fails the same way → not an MCP issue

CI / scripts: don't ship safe mode

--safe-mode is a diagnostic flag, not a deploy target. Do not add it to your CI invocations of claude -p or claude agents — you will silently lose your team's CLAUDE.md, hooks, and any MCP integrations your CI relies on. If you suspect CI is hitting a customization bug, run safe mode locally to confirm, then patch the offending customization. A --safe-mode line in a CI YAML file is an outage waiting to happen — keep the flag where humans can see it.

Onboarding new hires

For day-one onboarding, give the new engineer a 5-minute exercise: run claude-triage, ask the same question they ask in normal mode, observe the difference. They walk away knowing what their team's customizations actually do — instead of treating CLAUDE.md as magic that just happens.

Advanced: Pair Safe Mode with `disableBundledSkills` and `/cd`

v2.1.169 shipped three related controls, and they compose well.

`--safe-mode` + `CLAUDE_CODE_DISABLE_BUNDLED_SKILLS=1`

--safe-mode disables user-added skills under ~/.claude/skills/. To also hide the built-in / bundled skills, workflows, and built-in slash commands from the model, set CLAUDE_CODE_DISABLE_BUNDLED_SKILLS=1 (or disableBundledSkills: true in your settings). Combined, the model sees a CLI with nothing on top — useful when you want to compare "stock model behavior" against your customized behavior.

CLAUDE_CODE_SAFE_MODE=1 CLAUDE_CODE_DISABLE_BUNDLED_SKILLS=1 claude

Safe mode + `/cd`

v2.1.169 also added /cd to switch the session's working directory without breaking the prompt cache mid-session. Combine the two when you triage across repos: launch with --safe-mode once, then /cd between repositories. You only pay the cold-start cost once and your triage stays on the same model context window.

Safe mode + an ofox-style upstream

If you route Claude Code through a multi-provider gateway like ofox — using ANTHROPIC_BASE_URL=https://api.ofox.ai/anthropic — safe mode keeps the routing intact while still neutralizing every local customization. That separation is the whole point: you can rule out local config without touching your upstream credentials. Safe mode is the one CLI flag that lets you ask "is it me or is it everything I added on top?" — and get an honest answer in under ten seconds.

For deeper config patterns once you are past triage, see the Claude Code ofox configuration guide, the Claude Code hooks, subagents, and skills guide, and the Claude Code safety guide for preventing accidental file deletion — all three lean on the customization layers safe mode disables.

Claude Fable 5 vs Opus 4.8 vs GPT-5.5: SWE-Bench, Pricing, When to Switch

Owen — Wed, 10 Jun 2026 02:08:07 +0000

Claude Fable 5 vs Opus 4.8 vs GPT-5.5: SWE-Bench, Pricing, When to Switch

TL;DR — Anthropic shipped Claude Fable 5 on June 9, 2026, its first publicly available Mythos-class model. It hits 95.0% on SWE-bench Verified and 80.3% on SWE-bench Pro — an 11-point lead over Opus 4.8 and 21.7 points clear of GPT-5.5. Pricing is $10/$50 per million tokens, exactly 2x Opus 4.8. GPT-5.5 still wins Terminal-Bench 2.1 (82.7% vs 80.5%), Opus 4.8 still owns long-context retrieval and price-performance, and the upgrade math turns on whether your bottleneck is capability or bill. Below: the real numbers, the cost-per-point math, and a decision tree you can apply today.

Fable 5 is the first publicly available model to clear 80% on SWE-bench Pro and 95% on Verified — but at $10/$50 per million tokens, the cost per SWE-bench point runs 72% higher than Opus 4.8.

What Each Model Actually Shipped

Three releases over seven weeks reset the top of the coding leaderboards.

GPT-5.5 dropped on April 23, 2026 as OpenAI's single flagship — no Standard/Pro split for capability, just two surfaces (GPT-5.5 and GPT-5.5 Pro) for cost and latency. The launch leaned on Codex CLI and computer use; "agentic coding" was the headline. GPT-5.5 Instant followed on May 5 as the default model in ChatGPT.

Claude Opus 4.8 landed on May 28, 2026 at the same $5/$25 price as 4.7. SWE-bench Pro jumped from 64.3% to 69.2%, OSWorld-Verified to 83.4%, and Artificial Analysis's independent GDPval-AA leaderboard put it 121 Elo points clear of GPT-5.5 on real economic work — using 35% fewer output tokens per task than 4.7. Same price, higher score, lower bill. We covered the full release in our Opus 4.8 review.

Claude Fable 5 shipped on June 9, 2026 — yesterday, as of this writing. It's Anthropic's first generally available model from the Mythos class, the family Anthropic previously held back because of cybersecurity capabilities Anthropic deemed too risky for broad release. Fable 5 is the Mythos model with three safety classifiers layered on top: when a query hits cybersecurity, biology/chemistry, or distillation patterns, the request automatically routes to Opus 4.8 instead. Pricing is $10/$50 — half of what Anthropic charged for Mythos Preview, but still 2x Opus 4.8.

The headline isn't that Anthropic shipped two models in two weeks. It's that the gap between capability leader and value leader widened — and they're now both Claude.

The SWE-Bench Picture, Side by Side

Coding benchmarks are noisy. SWE-bench Verified and SWE-bench Pro are the two that matter most for production decisions because they run against real GitHub issues end-to-end, with a maintainer-graded ground truth. Here's how the three line up:

Benchmark	Fable 5	Opus 4.8	GPT-5.5
SWE-bench Verified	95.0%	88.6%	—
SWE-bench Pro	80.3%	69.2%	58.6%
Terminal-Bench 2.1	80.5%	74.6%	82.7%
FrontierCode Diamond	Leader (5x GPT-5.5, 2x Opus)	—	—
Every Senior Engineer (/100)	91	63	62
GraphWalks BFS @ 1M tokens	—	68.1%	45.4%
OSWorld-Verified	—	83.4%	78.7%
GDPval-AA (Elo, real work)	—	1890	1769

Three things in that table are worth more than the headline numbers.

Every's Senior Engineer benchmark is the cleanest read on capability ceiling. Every runs it on the hardest coding problems they can write — the kind a senior engineer would take a working day to solve. Fable 5 at 91/100 lands in the range of the human engineers who've taken the test. Opus 4.8 at 63 and GPT-5.5 at 62 are essentially tied, and both sit in the "junior engineer with debugger" range. The 28-point gap between Fable 5 and Opus 4.8 on this test is the gap that justifies the price premium — if your work lives at that ceiling.

Terminal-Bench is the one place GPT-5.5 still wins, and the asterisk matters. GPT-5.5 hits 82.7% against Fable 5's 80.5% — close, but a real lead. The asterisk: GPT-5.5's score comes through Codex CLI, OpenAI's strongest agentic surface for terminal work. Fable 5's number is the model in a standard harness. On Codex CLI, GPT-5.5 has had two months to embed itself in real workflows; if your stack is already Codex-centric, "switch to Fable" isn't a free upgrade. We unpack this trade-off in Codex CLI configuration.

Long-context retrieval is a Claude-family lead that compounded. On the GraphWalks BFS benchmark at 1M tokens, Opus 4.8 hits 68.1% versus GPT-5.5's 45.4% — a 22.7-point spread that turns into "the agent actually remembers what happened on turn 12" in practice. Anthropic hasn't published Fable 5's GraphWalks score directly, but the long-context architecture is shared, so the gap to GPT-5.5 on million-token retrieval almost certainly persists.

Pricing, and What "Cost Per Benchmark Point" Actually Buys

Sticker pricing is straightforward. The interesting number is what each model returns per dollar.

Model	Input ($/M)	Output ($/M)	Blended (2:1)*	Per SWE-bench Pro point
Claude Fable 5	$10.00	$50.00	$23.33	~$0.62
Claude Opus 4.8	$5.00	$25.00	$11.67	~$0.36
GPT-5.5	$5.00	$30.00	$13.33	~$0.50

Blended assumes a 2:1 input-to-output token ratio typical of coding workloads (more context in than code out). ofox.ai routing applies the same per-token rates with no markup.

Cost per SWE-bench Pro point is the metric most teams should care about, because it's what your monthly invoice looks like when you scale agentic coding traffic. Fable 5's $0.62 is 72% more expensive per point than Opus 4.8's $0.36. GPT-5.5 sits between at $0.50 — losing on absolute capability to both Claudes, but cheaper per point than Fable 5.

Two adjustments push the math in Fable 5's favor before you write it off as a luxury:

Fable 5 finishes the same task in fewer turns. Anthropic's reported numbers, corroborated by independent runs, put Fable 5 at roughly 25–30% fewer turns than Opus 4.8 on agentic spreadsheet and codebase tasks. If your bottleneck is output token volume — common on long autonomous runs — that efficiency partially offsets the 2x rate card. Opus 4.8 already runs 35% fewer output tokens than 4.7; Fable 5 pushes that further.

The capability ceiling is real on the hardest 10–20%. If your team's escalation pattern today is "Opus 4.8 hands off to a human after three failed attempts," routing those handoffs to Fable 5 instead may finish the task without the human in the loop. The cost question stops being "which model is cheaper per token" and becomes "which model removes a senior engineer from the loop." That comparison usually pays out at Fable 5's price.

Test the routing math on your own workload. Through ofox.ai, one key gets you Opus 4.8 and GPT-5.5 today (Fable 5 rolling in), on a single OpenAI-compatible endpoint. Run the same prompts through all three, compare token counts and quality on your workload before committing to the upgrade.

When to Switch: A Decision Tree

The right question isn't "which model wins" — Fable 5 wins most benchmarks. The right question is "which model wins on my task and bill." Here is the routing logic that maps the published numbers to a defensible choice.

1. Your primary workload is long-horizon agentic coding (multi-hour runs, codebase-wide migrations). Use Fable 5. The Senior Engineer benchmark, the FrontierCode Diamond lead, and the 25–30% turn reduction all compound on long runs. The price premium is offset by fewer wasted turns and fewer human escalations. Best AI model for coding walks through the routing patterns that work at this scale.

2. Your primary workload is terminal-driven CLI work, ops automation, or you're already on Codex CLI. Use GPT-5.5. Terminal-Bench 2.1 is the only benchmark of the three GPT-5.5 leads on, and the gap on Codex-centric workflows is real — not benchmark noise. The 7-week head start on integration matters here.

3. Your primary workload is everything else — refactors, code review, daily agent loops at scale. Use Opus 4.8. Same $5/$25 pricing as Opus 4.7, top of the GDPval-AA real-work leaderboard, 35% fewer output tokens than the prior generation. For 80% of teams, this is the right answer in 2026 — and it stays the right answer until your workload pushes past the capability ceiling.

4. You need million-token context retrieval (legal review, codebase audits, long transcripts). Use Opus 4.8 (or Fable 5 if you can absorb the price). GPT-5.5's 45.4% on GraphWalks BFS at 1M tokens is the disqualifying number — it means the model is no longer reliably finding facts past the first ~200K tokens. The Claude family architecture is the only one that holds up at that scale today.

5. You're hitting refusals or routing to Opus 4.8 on Fable 5. Expected behavior, not a bug. Fable 5's three safety classifiers (cybersecurity, biology/chemistry, distillation attempts) trigger on ~5% of sessions per Anthropic, and the fallback is silent — the request runs on Opus 4.8 anyway. If your workload sits in any of those three areas (security research, biotech, model training pipelines), don't try to engineer around the classifier. Just call Opus 4.8 directly and skip the indirection.

The one routing pattern that doesn't survive the new numbers: "Opus is the daily driver, GPT-5.5 for math and long context." That logic was true through May. GraphWalks closed the long-context gap. Opus 4.8 closed the math gap (USAMO 2026 jumped from 69.3% on Opus 4.7 to 96.7% on 4.8). If you're routing math or long-context to GPT-5.5 today, you're paying more per output token for a worse result.

How to Access Through ofox.ai

The three models land on a single OpenAI-compatible endpoint, so the upgrade path from "use one model" to "test all three" is one base URL change.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.ofox.ai/v1",
    api_key="your-ofox-key",
)

# Claude Opus 4.8 — daily driver
opus = client.chat.completions.create(
    model="anthropic/claude-opus-4-8",
    messages=[{"role": "user", "content": "Audit this service for race conditions..."}],
)

# GPT-5.5 — terminal-heavy workflows
gpt = client.chat.completions.create(
    model="openai/gpt-5.5",
    messages=[{"role": "user", "content": "Write a shell script that..."}],
)

Opus 4.8 and GPT-5.5 are live on ofox.ai today at anthropic/claude-opus-4-8 and openai/gpt-5.5. Fable 5 is rolling into the aggregator now — check the model page or the changelog for the live ID. One key covers all three, and going through an aggregator makes the capability vs. cost question easier to answer empirically: same prompts, three models, one endpoint, real numbers on your traffic.

For Anthropic-native features (adaptive thinking, effort control on Opus 4.8), point the official Anthropic SDK at https://api.ofox.ai/anthropic instead. We walk through both protocols in Why use an LLM API gateway.

Verdict

Fable 5 is the new capability ceiling. Opus 4.8 is the new value floor. GPT-5.5 is the ecosystem play that still wins one important benchmark.

If you're shipping agentic coding to production in 2026, the migration path is no longer "pick one and go." Route Opus 4.8 by default, escalate the hardest 10–20% to Fable 5, and keep GPT-5.5 on Codex CLI workflows where it has the integration lead. The cost-per-point math justifies the routing complexity within the first few thousand requests.

The one thing that hasn't changed: independent leaderboards still beat vendor claims. Watch Artificial Analysis's GDPval-AA for Fable 5's real-work Elo when it lands. That's the number that will tell you whether the 2x price tag holds up against the 25–30% turn reduction outside the benchmark suite.

Originally published on ofox.ai/blog.

Apple's Third-Generation Foundation Models: A Developer's Read on WWDC 2026

Owen — Tue, 09 Jun 2026 06:07:04 +0000

Apple's Third-Generation Foundation Models: A Developer's Read on WWDC 2026

TL;DR — Apple shipped its third generation of foundation models on June 8, 2026, alongside a rebranded "Siri AI." Five models. The headline is a 20-billion-parameter sparse on-device model (AFM 3 Core Advanced) that activates only 1–4B parameters per prompt using a technique Apple Research calls Instruction-Following Pruning. The other headline — quieter, more consequential for developers — is that Apple's most capable cloud model, AFM 3 Cloud Pro, runs on NVIDIA GPUs hosted in Google Cloud, and is refined using outputs from Google's Gemini frontier models. Apple says the resulting model is theirs; Apple executives are careful to distinguish "trained using" Gemini from "is" Gemini. The Foundation Models framework, which exposes the on-device model to any Swift app, now accepts images. None of it works in the EU on iPhone/iPad or in mainland China at launch.

The Five-Model Lineup

Apple's research post names five distinct models. The naming is more disciplined than 2024's "AFM-on-device / AFM-server" pair, and it tracks how Apple wants you to think about the stack: two tiers on-device, three in Private Cloud Compute.

Model	Where it runs	Size	Active params	Job
AFM 3 Core	On-device	3B (dense)	3B	Lightweight text, routing, fast NLU
AFM 3 Core Advanced	On-device	20B (sparse)	1–4B per prompt	New Siri / dictation / TTS; image understanding
AFM 3 Cloud	Private Cloud Compute	undisclosed	—	Main cloud text / image-understanding model
ADM 3 Cloud	Private Cloud Compute	undisclosed	—	Image generation (Image Playground, Reframe, Extend, Cleanup)
AFM 3 Cloud Pro	NVIDIA GPUs in Google Cloud (Private Cloud Compute extension)	undisclosed	—	Complex reasoning, agentic tool use

Apple has not published parameter counts for any of the three cloud models. The on-device models are the only ones with disclosed sizes.

The 20B Sparse Model and Why It Matters

The most technically interesting model is AFM 3 Core Advanced. It's a 20-billion-parameter model that fits — and runs — on a phone, by never activating more than ~4B parameters at once.

The trick is Instruction-Following Pruning (IFP), originally published by Apple Research in a January 2025 paper. The idea: rather than treating sparsity as a static structural decision (set at training), let a small predictor read the prompt and dynamically choose which rows and columns of the feed-forward-network matrices to activate for that request. The paper's headline result: their 3B activated model "outperformed the 3B dense baseline by 5–8 absolute points on math and coding, while matching the performance of a 9B dense model." So the same active compute footprint as a 3B dense model bought roughly 9B-class quality.

What changes for the production model is the memory story: Apple stores the full model in flash (NAND), keeps a small set of "always-active shared experts" in DRAM, and pages routed experts into DRAM only when the predictor selects them. That's how 20B fits in an on-device model footprint without melting battery.

The blunt way to read this: Apple just gave the iPhone the first production-scale dynamic-sparse LLM that ships to consumers. It's not a mixture-of-experts model in the classic sense (no learned router selecting K-of-N experts per token), but it's a cousin — and the deployment hardening is a first.

What Apple does not claim: it does not benchmark AFM 3 Core Advanced against GPT-5.5, Claude Opus 4.8, Gemini 3.1 Pro, Qwen 3.7, or Llama 4. Every comparison is against Apple's own 2025 baseline. Treat the eval numbers below as evidence of generational progress, not as a competitive ranking.

What Apple's Human Evaluations Actually Show

Apple's evaluation methodology is side-by-side blind human preference vs. the previous AFM generation. The numbers, verbatim from the research post:

Eval	New model preference	2025 baseline preference
Text (AFM 3 Core, on-device)	45.6%	23.3%
Text (AFM 3 Cloud)	64.7%	8.7%
Image understanding (AFM 3 Core)	>61%	—
Image understanding (AFM 3 Cloud)	37.8%	9.6%
Dictation overall quality (AFM 3 Core Advanced)	44.7%	17.6%

Cloud Pro adds +10% relative preference over Cloud on text, +14% on math, and +14% on image understanding.

Mean Opinion Score for the new on-device TTS:

Voice	Current TTS	AFM 3 Core Advanced
General	3.87	4.15
Conversational	3.82	4.24

Two caveats matter when you cite these:

No third-party benchmarks. No MMLU, no SWE-bench, no GPQA. Apple's published numbers are preferences against the 2025 baseline only.
Side-by-side preference is loose for technical work. It captures "did the human like this answer better," which is informative for chat, weaker for code or reasoning.

The Gemini Question: What's Verified

The Apple–Google partnership produced two parallel storylines that have been hard to reconcile in coverage. Here's what each Apple executive actually said:

"The amount of the Google Assistant we use is none." — Craig Federighi, SVP Software Engineering

"All of these are custom builds for Apple Silicon, trained using proprietary data, and refined using outputs from Gemini frontier models." — Amar Subramanya, Apple AI VP

Reconciled: Apple is not running Gemini in production for Apple Intelligence. Apple is using Gemini's outputs as part of post-training (distillation-style refinement). For AFM 3 Cloud Pro specifically, multiple reports describe a deeper Google involvement — Gemini-derived training infrastructure, Apple-owned pre-training and post-training, NVIDIA inference. Apple has not contradicted that account but has chosen not to volunteer it on stage.

The honest summary: Gemini is a teacher signal, not the runtime model. That's a real and growing pattern in 2026 — frontier labs train teacher models, downstream players distill — and Apple is the largest distribution channel to publicly adopt it.

Private Cloud Compute, Now on NVIDIA in Google's Datacenter

Apple's Private Cloud Compute (PCC) launched in 2024 with a striking security architecture: Apple Silicon servers running attested, code-audited builds, with cryptographic guarantees that user data is unreachable even by Apple. The 2026 extension is the surprise: PCC now also runs on NVIDIA GPUs hosted inside Google Cloud, while Apple says the same data-handling guarantees still apply.

Two related details worth flagging:

Why Google's datacenter? Reporting suggests Apple tried to run the new Cloud Pro model on its own PCC hardware first, and the model was too slow. NVIDIA capacity on Google Cloud was the path that shipped.
Why none of this in the keynote? Apple's keynote mentions NVIDIA, not Google. Google appears only in the research post and in executive interviews afterward. The brand story Apple wants you to hear is "Apple models, NVIDIA hardware, Apple privacy." The full supply chain is more entangled.

For builders evaluating Apple's privacy claim, the engineering substance is the cryptographic attestation chain, not the geographic location of the GPUs. The substrate moving to NVIDIA-in-GCP doesn't break that — but it does mean the trust model now spans more vendors than the 2024 version.

The Foundation Models Framework: What 2026 Adds

This is the under-covered part of the announcement, and the one most directly relevant to developers.

The Foundation Models framework was introduced in 2025 as a Swift API that gives any third-party app direct access to Apple's ~3B on-device model — no API key, no network, no per-token cost. The 2026 update adds image input: developers can now pass images alongside text into the on-device model, enabling on-device visual tasks (caption a photo, extract structured data from a receipt, classify a UI element) without any cloud round-trip.

What the framework is good at:

Structured output (typed Swift values, not just text)
Tool calling / function calling
Privacy-sensitive embedded intelligence (notes summarization, on-device search, smart suggestions)
Offline reliability (no network dependency)

What it is not good at, by design:

General-knowledge Q&A (it's not a chatbot back-end)
Anything that requires fresh world knowledge
Workloads that need frontier-tier reasoning, long context, or multi-step agentic tool use

For an iOS app shipping in fall 2026, the realistic pattern is a hybrid: use the Foundation Models framework for fast, free, offline work; fall back to a cloud model for everything else. That fallback is where multi-provider gateways (including ofox.ai) get useful — you want OpenAI/Anthropic/Google/Qwen/DeepSeek behind one API so you can change providers without reshipping the app.

Who Can't Use This at Launch

The geography is unusually restrictive even by Apple AI standards:

🇪🇺 EU: Siri AI is not available on iPhone or iPad at launch. Mac, Apple Watch, and Vision Pro are included. Apple cites DMA compliance work.
🇨🇳 Mainland China: All of Apple Intelligence, including Siri AI, is unavailable pending regulatory approval.
Hardware floor: iPhone 16 family, iPhone 15 Pro / 15 Pro Max, iPad mini with A17 Pro, M1-or-later iPads, M1-or-later Macs, Apple Vision Pro. On Apple Watch, watchOS 27 runs on Series 10, Series 11, Ultra 2, Ultra 3, and SE 3 — and Watch-side Apple Intelligence additionally requires pairing with an iPhone 15 Pro / Pro Max or newer.
Launch cadence: Siri AI starts as a beta later in 2026 in English, with the 32 supported locales rolling in over time. The locales span English (US, UK, Australia, India), PFIGSCJK (Portuguese, French, Italian, German, Spanish, Chinese, Japanese, Korean), DNNSTV (Danish, Dutch, Norwegian, Swedish, Turkish, Vietnamese), and AFIHHMPRTU (Arabic, Finnish, Indonesian, Hebrew, Hindi, Malay, Polish, Russian, Thai, Ukrainian).

The EU/China gap means Apple Intelligence is now formally a partial product across geographies — the same hardware does materially different things depending on Apple ID region, and developer documentation will need to fork on capability availability.

What This Actually Changes for Builders

Three things to take away if you're shipping AI features in late 2026:

On-device LLMs cross a usability threshold. A 20B sparse model on a phone, with image input, free for app developers, is enough to handle a meaningful slice of in-app AI tasks — structured extraction, classification, embedded summarization, tool routing. Apps that previously paid for cloud calls to do this can stop.
Frontier work still belongs in the cloud. Cloud Pro exists for a reason. Long context, agentic loops, frontier reasoning, vision-language across many images — all still cheaper, more capable, or both via a cloud LLM. The build decision is now "what can't run on-device" rather than "how big a model do I need."
Multi-provider sourcing is the safer default. Apple now ships an on-device model partly distilled from Gemini, running cloud workloads on NVIDIA-in-GCP. Vendor coupling at the model layer is no longer optional even for Apple. If you're building a cross-platform product, picking a single model vendor at the application layer is the bet that's getting harder to justify.

The throughline: Apple just made on-device LLMs a baseline capability on iOS. The interesting work moves up the stack — to deciding when to use it, when to route past it, and how to do that without locking your app to any one vendor.

Sources Checked

Apple Machine Learning Research — Introducing the Third Generation of Apple's Foundation Models (model lineup, IFP, eval numbers verbatim)
Apple Newsroom — Apple unveils next generation of Apple Intelligence, Siri AI, and more (hardware list, language list, region availability)
9to5Mac — Federighi details Apple's collaboration with Google for Siri AI (Federighi quote)
CNBC — Apple partnering with Google and Nvidia for most advanced AI model (Subramanya quote, NVIDIA-in-GCP arrangement)
AppleInsider — Apple's new foundation models don't contain a drop of Gemini (independent read on the Gemini relationship)
MacRumors — Siri AI not available in EU/China initially (region restrictions)
arXiv 2501.02086 — Instruction-Following Pruning for Large Language Models (IFP technique, original Apple paper)
MarkTechPost — Apple Researchers Introduce IFPruning (third-party IFP explainer)

Originally published on ofox.ai/blog.

Claude API Error 529 Overloaded: 8 Fixes, When to Switch Providers, and How to Avoid It in 2026

Owen — Mon, 08 Jun 2026 09:43:38 +0000

Claude API Error 529 Overloaded: 8 Fixes, When to Switch Providers, and How to Avoid It in 2026

TL;DR. Claude API error 529 means Anthropic is temporarily overloaded, not that your code is wrong and not that your account is throttled. Four confirmed platform-wide incidents in 2026 already (March 2, March 18, March 19, June 2), the longest stretching past three hours. The retry-only playbook fails after roughly five minutes; what survives is a layered strategy — exponential backoff for the first 30 seconds, then automatic failover to a different Claude model or a different vendor entirely. The eight fixes below are sorted by tier and by how long they take to recover, and the closing section shows the ten-line unified-endpoint pattern that turns a three-hour outage into a two-second hop.

529 is Anthropic temporarily saying "no." 429 is Anthropic telling you you are saying too much. The fixes look similar for the first three retries and diverge completely after that.

Is Claude API Down Right Now? The 30-Second Diagnosis

Three checks, in order. If any of them confirms the issue is upstream, stop debugging your own code and move to the fixes section.

Step	What to check	Confirms 529 if
1	The error body	Contains `{"type":"error","error":{"type":"overloaded_error","message":"Overloaded"}}`
2	Anthropic status page at `status.claude.com`	Open incident on Claude API, claude.ai, or Claude Code
3	Your own request log over the last 5 minutes	529 rate jumped from <1% to >20% with no deploy on your side

If all three light up, this is a platform-wide overload, not a bug in your application. The next section tells you which fix to reach for based on how long you can wait.

When to Apply These Fixes (and When to Switch Models Instead)

This is the decision frame that keeps you from wasting an afternoon on the wrong layer of the stack.

When to retry in place (apply fixes 1-3):

The 529 rate just spiked and the status page is still green — you are early in the incident
Your workload is batch or asynchronous and can tolerate ~30 seconds of delay
You are on a free or low-tier plan where multi-provider routing is not yet justified

When to switch models or vendors (apply fixes 4-8):

Same claude-opus-4-8 request has returned 529 three times in a row across a 30-second window
The status page has confirmed an incident and the ETA is ">1 hour" or unstated
Your workload is user-facing and any visible latency above one second damages product experience
You are calling Claude from an agent loop (Claude Code, Codex, Cursor) where retries amplify into compound delay

Stop rule. If you have retried the same model four times in 60 seconds and three of the responses were 529, stop retrying that model. Every retry past that point is queuing behind every other client doing the same thing — you are making the storm worse. Switch.

The shortest version: retry buys you 30 seconds, failover buys you the rest of the day.

Understanding 529 vs 429: Anthropic's Two Limits

Half the production teams hitting 529 in 2026 walked in thinking they had a 429 problem and tuned the wrong knob. The two errors look superficially similar and need different fixes.

Code	Type	Cause	Whose problem	Long-term fix
429	`rate_limit_error`	Your account exceeded the per-minute or per-day quota for your tier	Yours	Smaller batches, request a tier upgrade, request acceleration limits raised
529	`overloaded_error`	Anthropic's platform is over capacity globally across all customers	Anthropic's	Multi-model fallback, multi-provider failover, retry with jitter

Anthropic also flags a third edge case in the official errors documentation: if your organization itself causes a sudden traffic spike, you can see 429 errors specifically because of acceleration limits that ramp protect Anthropic's own infrastructure. The fix there is to ramp up traffic gradually rather than treat the 429 as a quota problem.

The practical version: if you see 429 from a single account while everyone else is fine, it is yours to fix. If you see 529 (or 429s correlated with the status page), it is upstream and retry-only strategies will not save you.

How to Fix Claude API 529 Overloaded (Solutions for Every Tier)

The eight fixes are sorted from "ten seconds, works on free tier" to "production-grade failover." Apply the ones that match your tier; do not skip ahead to fix 8 if you have not done 1-3.

Free / Pro Tier (Solutions 1-3)

Fix 1 — Retry once after 2 seconds. Most 529 spikes are short. If your workload is interactive (you are pasting a prompt into a script), wait two seconds and retry once. Roughly 60% of 529s clear inside two seconds based on the March 19 incident pattern where the status page moved from open to monitoring inside an hour.

Fix 2 — Exponential backoff with jitter (Python). For any script that runs more than once, this is the floor. The jitter component matters: without it, every client that hit 529 at the same time retries at the same time, recreating the overload.

import time, random, httpx

def call_claude_with_backoff(payload, max_attempts=4):
    for attempt in range(max_attempts):
        try:
            r = httpx.post(
                "https://api.anthropic.com/v1/messages",
                json=payload,
                headers={"x-api-key": API_KEY, "anthropic-version": "2023-06-01"},
                timeout=30,
            )
            if r.status_code == 200:
                return r.json()
            if r.status_code not in (429, 500, 502, 503, 504, 529):
                r.raise_for_status()
        except httpx.RequestError:
            pass
        delay = (2 ** attempt) + random.uniform(0, 0.3 * (2 ** attempt))
        time.sleep(delay)
    raise RuntimeError("Claude returned 529 four times — switch provider")

The backoff sequence is 1s, 2s, 4s, 8s with up to 30% jitter on each. Four attempts cover roughly 15 seconds total — enough for a transient spike, not enough to blow a user-facing SLO.

Fix 3 — Stream instead of polling. If you are seeing 529s on a non-streaming call that takes more than 30 seconds, your problem is partly socket-level: idle connections drop and the SDK retries from scratch. Anthropic's docs explicitly recommend the streaming Messages API for requests over 10 minutes. Streaming holds the connection and reduces the surface area for 529-during-reconnect.

Paid / Team Tier (Solutions 4-6)

Fix 4 — Fallback to a sibling Claude model. Inside Anthropic, capacity is shared across models but not symmetrically. When claude-opus-4-8 is overloaded, claude-sonnet-4-6 often is not. The catch: model behavior differs, so do a regression pass on prompts before flipping the fallback. The pattern:

PRIMARY = "claude-opus-4-8"
FALLBACK = "claude-sonnet-4-6"

def call_with_model_fallback(payload):
    try:
        return call_claude_with_backoff({**payload, "model": PRIMARY})
    except RuntimeError:
        return call_claude_with_backoff({**payload, "model": FALLBACK})

This buys you a second pool of capacity and adds maybe 200ms to the failover. It does not survive a platform-wide 529 (March 18, March 19) because both pools are overloaded at once — but it covers the model-specific incidents (the May 22 and May 25 events on Opus 4.7 documented in our Opus 4.7 reliability fix guide).

Fix 5 — Move tolerable workloads to the Message Batches API. Per Anthropic's official Message Batches API documentation, the Batches API gives you two properties that matter for 529 exposure: a 24-hour processing window (most batches finish in under an hour, but you have up to 24h before expiry) and a 50% discount on all usage. The docs are explicit that batch processing speed "may be slowed down based on current demand and your request volume," so batches are not on a separate, 529-immune infrastructure — they can be delayed under the same platform pressure. What changes is the shape: a 529-driven delay never reaches a user-facing latency budget, and the half-price bill blunts the cost impact of any retries the system absorbs internally.

batch = client.messages.batches.create(
    requests=[
        {"custom_id": f"job-{i}", "params": {
            "model": "claude-opus-4-8",
            "max_tokens": 1024,
            "messages": [{"role": "user", "content": prompt}],
        }} for i, prompt in enumerate(prompts)
    ]
)

Anything you can defer to overnight summarization, bulk classification, or async report generation belongs here. You trade real-time response for a tolerant window and a halved bill — and a 529 storm during that window costs you wall-clock delay inside a 24h budget instead of a customer-facing outage.

If you are running Claude Code interactively rather than as a batch worker, note the separate --fallback-model CLI flag — per the official CLI reference, it enables automatic fallback when the default model is overloaded or unavailable, which does cover 529. Two limits, both spelled out in the docs: it takes effect in -p (print mode) and background sessions but is ignored in interactive sessions, and the fallback target is another Anthropic model that shares the same upstream capacity pool — useful for model-specific incidents, not the platform-wide March 18 / June 2 pattern. The current Claude Code settings reference does not list a fallbackModel entry in settings.json, so the CLI flag is the documented surface.

Fix 6 — Send to AWS Bedrock as a second region. Anthropic's direct API and the Claude on AWS Bedrock deployment are separate infrastructure pools. The March 18 outage hit the direct API; Bedrock was unaffected for the first 90 minutes. If you have AWS credentials and your compliance posture allows it, run Bedrock as the fallback path. The trade-off is request-ID complexity (you now have to track an AWS request ID and an Anthropic request ID, per the official docs), but the dual-region capacity is real.

Enterprise / Production (Solutions 7-8)

Fix 7 — Multi-provider failover through a unified endpoint. This is the only fix that survives a platform-wide outage like June 2 because it routes across vendors, not just across Anthropic models. Through ofox.ai's unified endpoint, the same OpenAI-compatible request hits anthropic/claude-opus-4.8 first, falls back to openai/gpt-5.5 on 529, then to bailian/qwen3.7-max if GPT also degrades — all in sub-200ms because the failover happens inside the gateway, not your application. Code in the next section.

Fix 8 — Circuit breaker around the entire Anthropic path. When 529 rate exceeds 20% over a five-minute rolling window, open a circuit breaker that stops calling Anthropic entirely and routes 100% of traffic to a secondary provider for ten minutes. This stops your retries from contributing to the storm and gives Anthropic's autoscaler room to recover. Implementation is a classic circuit breaker pattern — sliding-window counter, open/half-open/closed states, automatic reset.

Claude 529 Outage History: Real Incidents in 2026

Four confirmed platform-wide 529 events so far this year. Two of them lasted long enough that a retry-only strategy would have failed entirely.

Date (UTC)	Duration	Services affected	Public root cause
March 2, 2026	~2 hours	Global — Claude API, claude.ai, Claude Code	Not disclosed; correlated with Opus 4.7 launch traffic
March 18, 2026	3+ hours from 06:30 UTC	Claude Code 529 storm, persisted past status-page "monitoring"	Not disclosed — see GitHub issue #35704 (Max subscriber, 3h+ no recovery)
March 19, 2026	~53 minutes from 00:28 UTC	Elevated errors across claude.ai, platform.claude.com, Claude API, Claude Code	Authentication errors 23:59-00:30 UTC, moved to monitoring at 01:21 UTC
June 2, 2026	Global outage window	Cross-vendor outage observed at AI infra layer	Not disclosed

The pattern: status-page time-to-detect ran 10-30 minutes behind real user impact in three of the four events. The March 18 episode is the worst case study — the page said monitoring while real users were still seeing pure 529 for over two hours.

What this means for your retry budget: plan for a five-minute upper bound on retry, then route. Anything beyond five minutes of retries during one of these windows produces zero successful responses and burns through your error budget for the day.

Why 529 Happens: Anthropic's Capacity Architecture in 2026

A retry strategy you trust requires a mental model of what is on the other side of the wire. Three structural reasons explain why Claude 529 is more common in 2026 than it was in 2025 and why the pattern is not going away.

Shared capacity pools across models. Anthropic does not run a dedicated cluster per model name. claude-opus-4-8 and claude-sonnet-4-6 share infrastructure with weighted routing. When Opus 4.8 demand spikes — for example, the first hour after a major release — both models can return 529 simultaneously. This is why fix 4 (sibling model fallback) helps during model-specific incidents but does not survive a true platform-wide event.

Launch-week traffic shocks. The March 2 and June 2 incidents both correlated with new model releases. Anthropic's autoscaler is reactive, not predictive, so a release that triples baseline load in 30 minutes blows past the autoscale ramp. The pattern is predictable enough that you can pre-warm your fallback chain in the week of any major Anthropic announcement.

Speculative decoding and tokenizer changes. Newer Claude models use more output tokens per task even when the prompt is identical (~35% more for Opus 4.7 vs 4.6, documented in our Claude Max throttling postmortem). More output tokens means more GPU-seconds per request, which means the same QPS pressure now consumes more capacity. The math compounds: a tokenizer change effectively reduces platform capacity without anyone seeing a hardware downgrade.

The takeaway: 529 is a structural feature of running on a single provider's evolving infrastructure. Treating it as a transient bug to retry through misses the point — you need a routing strategy that assumes 529 will be normal in 2027 too.

When Claude 529 Won't Stop: Multi-Provider Failover via ofox

The honest version: no gateway prevents 529 errors. Those come from Anthropic. What a gateway does is collapse the failover from a multi-minute incident-response exercise into a sub-200ms request-time decision. You write the fallback chain once, in one place, and every downstream service inherits it automatically.

Python — failover in 10 lines via OpenAI SDK shape

from openai import OpenAI

client = OpenAI(
    api_key=OFOX_KEY,
    base_url="https://api.ofox.ai/v1",
)

FALLBACK_CHAIN = [
    "anthropic/claude-opus-4.8",   # primary
    "openai/gpt-5.5",              # cross-vendor failover
    "bailian/qwen3.7-max",         # second cross-vendor failover
]

def chat_with_failover(messages):
    for model in FALLBACK_CHAIN:
        try:
            return client.chat.completions.create(model=model, messages=messages)
        except Exception as e:
            if "overloaded" not in str(e).lower() and "529" not in str(e):
                raise
    raise RuntimeError("All three providers are overloaded — that almost never happens")

The pattern: same OpenAI SDK shape, swap one string per provider, no second SDK. The same client.chat.completions.create() call works against Claude, GPT, and Qwen because ofox terminates the call into the right provider transparently.

Node — same shape

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.OFOX_KEY,
  baseURL: "https://api.ofox.ai/v1",
});

const FALLBACK_CHAIN = [
  "anthropic/claude-opus-4.8",   // primary
  "openai/gpt-5.5",              // cross-vendor failover
  "bailian/qwen3.7-max",         // second cross-vendor failover
];

async function chatWithFailover(messages) {
  for (const model of FALLBACK_CHAIN) {
    try {
      return await client.chat.completions.create({ model, messages });
    } catch (e) {
      if (!/overloaded|529/i.test(String(e))) throw e;
    }
  }
  throw new Error("All three providers overloaded — escalate");
}

How the routing decision actually works

The decision logic at the gateway level is simpler than it looks. The platform watches three signals and picks per-request:

Signal	What it tells the router	Action on positive signal
HTTP 529 from primary	Anthropic is overloaded right now	Retry against next model in chain
Median latency on primary > 3× baseline	Primary is degraded but not failing	Send 30% of traffic to secondary; observe
Open status-page incident for primary	Sustained outage in progress	Send 100% of traffic to secondary until incident closes

This is the difference between "I retry until something works" and "the gateway routed me to a working model before I noticed." The application code does not change between a green day and an incident day.

For the broader case where the same pattern applies to cost optimization rather than just reliability, see the hybrid routing breakdown — same plumbing, optimizing for $/task instead of for uptime. For the broader unified-endpoint pattern, see the API aggregation explainer.

A production team's June 2 playbook (what actually shipped)

The pattern below is reconstructed from public GitHub issue threads and Reddit r/ClaudeAI posts during the June 2, 2026 outage. Names removed; the timeline and the specific decisions are real.

T+0:00 — A two-engineer team's agent-loop product starts logging 529 from anthropic/claude-opus-4-8 (the primary in their ofox routing chain). Initial retry rate of 12% jumps to 67% inside 90 seconds. The on-call pager fires on their internal SLO dashboard, not on the Anthropic status page (which was still green for another 18 minutes).

T+0:02 — The gateway routing config (already pinned to ofox, with a four-step fallback chain) starts shifting traffic. First failover hop to anthropic/claude-opus-4.7 returns the same 529s — capacity is shared inside Anthropic. Second hop to openai/gpt-5.5 succeeds. Median latency goes from 2.1s to 2.4s. Users see nothing.

T+0:17 — Anthropic's status page opens an incident at "investigating." The team's own dashboard shows 78% of traffic already on GPT-5.5; the remaining 22% are non-streaming Claude requests that succeeded before the routing tier flipped them.

T+1:48 — Incident moves to "monitoring." 529 rate on Claude drops below the 5% threshold and the gateway gradually shifts traffic back. Final tally for the engineering team: zero customer-facing errors, a small bill skew toward GPT-5.5 for two hours, and one Slack message that read "did you notice anything?"

The cost of the routing setup that absorbed this incident was roughly four hours of one-time configuration. The cost of not having it would have been a multi-hour outage on a user-facing product.

Anti-patterns we have watched fail in 2026

The most common retry mistakes — each one observed in a production system in the first half of 2026:

Infinite retry with linear backoff. A 1-second retry loop that never gives up turns your application into a denial-of-service amplifier against Anthropic's already-overloaded infrastructure. It also blows your bill on retried token spend.
No jitter on exponential backoff. Every client that hit 529 at the same time retries at the same instant, re-creating the overload spike at 2s, 4s, 8s. Always add ±30% random offset to backoff delays.
Catching 529 as a generic Exception. Hides the signal in your error logs and prevents the routing tier from acting on it. Match the type explicitly (overloaded_error or HTTP 529) and re-raise other errors.
Setting a fallback model that lives in the same capacity pool. claude-opus-4-8 fallback to claude-opus-4-7 survives model-specific incidents but not platform-wide ones. The fallback chain must cross vendors at least once.
Skipping the circuit breaker. During the March 18 three-hour event, teams without circuit breakers paid for tens of thousands of retried failed requests. The 529 itself is free, but the upstream token-count meter still incremented when the retry succeeded after the budget was already blown.

How to Monitor Claude Status and Get Alerts

Three layers of monitoring, in order of importance.

Status page subscription. Subscribe to email or Slack notifications at status.claude.com/subscribe. Useful for awareness, useless for detection — the page lags real incidents.

Your own 529 counter. A simple rolling-window counter on your error log is the leading indicator. Page yourself when the 529 rate on a five-minute window exceeds 5% — that is the threshold where retry-only strategies start failing. The May 2026 throttling backlash we documented in our Claude Max throttling postmortem was visible in user retry counters two weeks before Anthropic announced the May 6 reversal.

Gateway-level fallback metrics. If you route through ofox, the gateway exposes per-model success rates and median time-to-failover. When the anthropic/claude-opus-4-8 success rate dips below 95% for an hour, the gateway has already shifted traffic; the dashboard just tells you why your bill mix changed.

The fastest production teams in 2026 do not detect 529 storms. Their gateway already routed around them before the engineer woke up.

Alternatives That Work When Claude 529 Persists

If a 529 window stretches past 30 minutes and your fallback chain is exhausted, here are the realistic next moves, ranked by setup time.

ofox.ai — Single OpenAI-compatible endpoint covering Claude, GPT, Gemini, Qwen, DeepSeek, Kimi, Doubao, Zhipu, Mistral. 99.9% uptime, ~300ms median latency, 100+ models. Pre-write a fallback chain in any client SDK; no incident-time code changes. Best for teams that want one bill and one auth surface across vendors.
AWS Bedrock (Claude) — Separate Anthropic capacity pool with strong compliance story. Higher setup cost (AWS account, IAM, separate billing) and longer cold-start for new prompts but real value during direct-API incidents like March 18.
Google Vertex AI (Claude) — Same model, third capacity pool. Similar trade-off to Bedrock.
Direct provider rotation — Talk to GPT-5.5 or Gemini 3.1 Pro directly via their own SDKs. Lowest infra cost, highest per-task integration cost because you now own three SDKs and three retry strategies.
OpenRouter or LiteLLM — Open-source-ish alternative gateways. Slower per-request than ofox (300-500ms median) and the cost stack passes upstream margin to you, but worth knowing as a comparison.

Sources Checked for This Refresh

Anthropic official API errors documentation — https://platform.claude.com/docs/en/api/errors (verified 2026-06-08): defines 529 as overloaded_error and warns it occurs during high traffic across all users
Anthropic status page — https://status.claude.com (incidents from March 2, March 18, March 19, June 2, 2026)
GitHub issue #35704 — Claude Max subscriber 3h+ 529 storm on March 18, 2026 (https://github.com/anthropics/claude-code/issues/35704)
Anthropic Message Batches API documentation — https://platform.claude.com/docs/en/build-with-claude/batch-processing (verified 2026-06-08): 24-hour processing window, 50% discount on all usage, processing speed "may be slowed down based on current demand" (docs do not claim a separate capacity tier)
Claude Code official CLI reference — https://code.claude.com/docs/en/cli-reference (verified 2026-06-08): --fallback-model documented for "overloaded or not available" cases, takes effect in -p/background sessions, ignored interactively
Claude Code official settings reference — https://code.claude.com/docs/en/settings (verified 2026-06-08): no fallbackModel entry in the settings.json schema as of this date
ofox.ai unified endpoint — https://ofox.ai (verified 2026-06-08, 100+ models, 99.9% uptime, ~300ms median latency, OpenAI-compatible base URL https://api.ofox.ai/v1)
Reddit r/ClaudeAI threads correlated with the March 18 and June 2, 2026 incident windows (community-reported 529 spikes and recovery times used to cross-check the status-page timeline)

Originally published on ofox.ai/blog.

Install CC Switch: Manage Claude Code, Codex, Gemini CLI, OpenCode, OpenClaw & Hermes Agent from One App (2026)

Owen — Fri, 05 Jun 2026 08:42:07 +0000

What You Can Do After This Setup, In 30 Seconds

Question	Answer
What you'll have	One desktop app that switches API providers across Claude Code, Claude Desktop, Codex, Gemini CLI, OpenCode, OpenClaw, and Hermes Agent — without editing config files by hand
Time required	10 minutes (install + first provider + tray switch verified)
What you need	macOS 12+ / Windows 10+ / Linux (Ubuntu 22.04+, Debian 11+, Fedora 34+, Arch), Node.js 20 LTS for most CLIs (Gemini CLI and Hermes Agent floor), Node 22.19+ if you install OpenClaw, and at least one provider API key
Latest version (verified)	CC Switch v3.16.1, released 2026-06-01

If you're already on CC Switch and looking for a specific problem — config file not picked up, OAuth re-login loop, Codex wire_api mismatch — jump to Common Errors During Setup below. Otherwise, the next four sections walk through install → first provider → verify → multi-CLI in order.

System Requirements

CC Switch is a Tauri 2 desktop app — small (~10 MB installer), but it does need a graphical session and the right CLI runtimes underneath. Check each row before you start, because most "CC Switch isn't working" reports trace back to a missing CLI rather than the app itself.

Component	Minimum	Recommended	Verified version
CC Switch	v3.13.0 (Codex OAuth proxy floor)	v3.16.1	v3.16.1 (2026-06-01)
Node.js (for npm-installed CLIs)	20 LTS (22.19+ for OpenClaw)	22 LTS	22.x
Claude Code (`@anthropic-ai/claude-code`)	Node 18+	latest	2.1.163
Codex (`@openai/codex`)	Node 16+; 0.137.0 (wire_api responses-only floor)	latest	0.137.0
Gemini CLI (`@google/gemini-cli`)	Node 20+	latest	0.45.1
OpenCode (`opencode-ai`)	no engine floor declared	latest	1.16.0
OpenClaw (`openclaw`)	Node 22.19+	latest	2026.6.1
Hermes Agent (`hermes-agent`)	Node 20+	latest	0.15.2
OS (desktop)	macOS 12 / Windows 10 / Ubuntu 22.04 / Debian 11 / Fedora 34 / Arch	Latest stable	—

You don't have to install every CLI up front — CC Switch only manages the panels you actually have on disk. Start with the one or two you use most.

Step 1 — Install CC Switch

Pick the platform-native path. Don't mix Homebrew and manual download on macOS; auto-update only tracks one channel.

macOS (Homebrew)

brew install --cask cc-switch
# upgrade later with:
brew upgrade --cask cc-switch

The cask is signed and notarized by Apple — first launch opens directly without a Gatekeeper prompt.

macOS (manual `.dmg`)

# Download the latest DMG from the Releases page, then:
open CC-Switch-v3.16.1-macOS.dmg
# Drag CC Switch.app to /Applications

Windows

Download CC-Switch-v3.16.1-Windows.msi from github.com/farion1231/cc-switch/releases and double-click. A portable CC-Switch-v3.16.1-Windows-Portable.zip is available if your machine blocks MSI installs.

Linux (Debian / Ubuntu)

# x86_64 example
sudo dpkg -i CC-Switch-v3.16.1-Linux-x86_64.deb
# If apt complains about dependencies:
sudo apt-get install -f

Linux (Arch)

paru -S cc-switch-bin   # or: yay -S cc-switch-bin

Linux (universal AppImage)

chmod +x CC-Switch-v3.16.1-Linux-x86_64.AppImage
./CC-Switch-v3.16.1-Linux-x86_64.AppImage

Headless servers / CI

The Tauri 2 desktop app needs Wayland or X11. For headless boxes — build runners, remote sandboxes — use the Rust CLI fork SaladDay/cc-switch-cli instead. It reads the same WebDAV-synced config bundle, so you can author providers on a laptop and pull them down on the agent.

Verify the install

After first launch:

The CC Switch window opens to the Claude Code panel by default.
A CC Switch icon appears in the system tray (menu bar on macOS, system tray on Windows/Linux).
The top-left app switcher lists the seven managed targets — Claude Code, Claude Desktop, Codex, Gemini CLI, OpenCode, OpenClaw, Hermes Agent. Greyed-out entries are CLIs CC Switch couldn't detect on disk; that's expected if you haven't installed them yet.

If the tray icon is missing on Linux, your DE may not support StatusNotifierItem; install gnome-shell-extension-appindicator (GNOME) or equivalent and restart the session.

Step 2 — Install the CLIs You'll Manage

CC Switch doesn't install the CLIs themselves — it manages their config files. Install whichever you need:

# All seven, for reference. Pick what you actually use.
npm install -g @anthropic-ai/claude-code   # Claude Code (Node 18+)
npm install -g @openai/codex               # Codex (Node 16+)
npm install -g @google/gemini-cli          # Gemini CLI (Node 20+)
npm install -g opencode-ai                 # OpenCode (no engine floor)
npm install -g openclaw                    # OpenClaw (Node 22.19+)
npm install -g hermes-agent                # Hermes Agent (Node 20+)

Claude Desktop is the standalone Anthropic app — download from claude.ai/download rather than npm.

If npm install -g fails with EACCES or the new shim isn't on PATH, see our walkthrough on npm global install permission and PATH fixes for Codex CLI — the same fixes apply to every CLI on this list.

Step 3 — Add Your First Provider

Open CC Switch and click the + in the top-right corner. The Add Provider panel has two tabs:

App-specific Provider — only for the panel you're on right now
Universal Provider — shared across Claude Code, Codex, and Gemini CLI

For the first run, use App-specific:

From the Preset dropdown, pick the provider that issued your key. CC Switch ships with ~50 third-party presets across all panels: DeepSeek, Zhipu GLM, MiniMax, Kimi, Bailian (Alibaba Qwen), AWS Bedrock, NVIDIA NIM, OpenRouter, plus a long list of relay services. Most just need the API key — the endpoint URL and protocol fields are pre-filled.
Paste your API Key.
(Codex only) If the preset is marked "Needs Local Routing" — that's any chat-completions backend like DeepSeek or Zhipu — CC Switch turns on the Local Routing toggle for you, which proxies Codex's Responses API protocol down to the provider's /v1/chat/completions. Don't disable it; Codex 0.137.0 has dropped support for wire_api = "chat" at config-load time and only accepts responses.
Click Add.

What each preset actually writes

The preset list is the convenience layer. Underneath, CC Switch writes the CLI's native config file:

CLI	Config file	What CC Switch writes
Claude Code	`~/.claude/settings.json`	`env.ANTHROPIC_API_KEY` + `env.ANTHROPIC_BASE_URL`
Codex	`~/.codex/auth.json` + `~/.codex/config.toml`	`OPENAI_API_KEY` to `auth.json`; `model_provider`, `model`, `base_url`, `wire_api` to `config.toml`
Gemini CLI	`~/.gemini/.env` + `~/.gemini/settings.json`	`GEMINI_API_KEY` + `GOOGLE_GEMINI_BASE_URL` to `.env`
OpenCode	`~/.config/opencode/opencode.json`	provider block under `provider`
OpenClaw	`~/.openclaw/openclaw.json` (JSON5)	provider block under `models.providers`
Hermes Agent	`~/.hermes/config.yaml` + `~/.hermes/.env`	`custom_providers` entry + `.env` secrets; `model.provider` / `model.default` on switch

The principle is "minimal intrusion" — even if you uninstall CC Switch tomorrow, every CLI still works because their native config files are intact. CC Switch's own state in ~/.cc-switch/cc-switch.db is purely additive.

Step 4 — Switch and Verify

Once you have at least one provider per CLI:

From the app: click the provider card, then Enable.
From the system tray (the genuinely useful part): right-click the CC Switch icon → pick a CLI panel → click a provider name. The new config is written before the menu closes.

Then activate it on the CLI side:

CLI	Activation
Claude Code	Hot-reload — the next prompt picks up the new key with no restart
Codex	Quit and reopen the terminal
Gemini CLI	Hot-reload — `~/.gemini/.env` is re-read on each request
OpenCode	Quit and reopen the terminal
OpenClaw	Quit and reopen the terminal
Hermes Agent	Quit and reopen the terminal

Quick sanity test inside each CLI's session:

> Hello — which model am I talking to right now?

A reasonable model identification confirms both that the key is valid and that the routing is correct.

Common Errors During Setup (and Fixes)

Symptom	Root cause	Fix
Tray icon missing on Linux	DE doesn't support `StatusNotifierItem`	Install `gnome-shell-extension-appindicator` (or KDE equivalent); restart session
Switch succeeds but Codex still uses old key	Codex caches `config.toml` at startup	Quit the Codex process fully (not just the prompt) and restart
Codex preset connects, but every request returns `CHAT_WIRE_API_REMOVED_ERROR`	Preset's "Needs Local Routing" toggle was turned off after adding	Re-enable Local Routing on the provider card; Codex 0.137.0 rejects `wire_api = "chat"` at config-load time
Claude Code "first-run wizard" shows up every time you switch	Default onboarding flow re-triggers when `~/.claude/settings.json` changes	Settings → General → enable Skip Claude Code first-run confirmation (writes `skipIntroduction` to `~/.claude/settings.json`)
`Authentication failed (401/403)` on every preset	API key string is wrong or the key isn't authorized for the chosen model	Verify the key on the provider's own dashboard first; the Auto-Fetch Models button (download icon next to the model input) calls `/v1/models` and surfaces auth errors immediately
Can't delete the currently active provider	Last-config-standing guard	Switch to another provider first, then delete; if you really want to wipe the panel, hide that CLI in Settings instead
Gemini CLI ignores the provider	Node 18 detected, but `@google/gemini-cli` requires Node 20	`nvm install 20 && nvm alias default 20`, then reinstall the CLI
OpenClaw fails to install or start	Node < 22.19 (engines floor `>=22.19.0`)	`nvm install 22 && nvm alias default 22`, then reinstall
Hermes Agent fails to install	Node < 20 (engines floor `>=20.0.0`)	`nvm install 20 && nvm alias default 20`, then reinstall
WebDAV sync writes succeed but other devices don't see the new providers	Both devices need the same WebDAV root and the same custom config directory in Settings → Storage	Set the same `claudeConfigDir` / `codexConfigDir` etc. on every device
`~/.cc-switch/cc-switch.db` is locked	Two CC Switch instances open at once (common on macOS with iCloud sync)	Quit one; CC Switch holds an exclusive SQLite mutex on the file

If your CLI complains about its own config — model_not_found, EACCES, 429, 503 — that's the CLI's problem, not CC Switch's. For Codex specifically, the install-time command not found failure mode is covered in Codex command not found: 7 fixes after npm install -g. Each CLI has its own runtime-error runbook; CC Switch only manages the config file, not the request path.

Team / Multi-Developer Configuration

The pain CC Switch actually solves is teams that share third-party keys across multiple agents — not the solo dev with one CLI. Three patterns scale.

Pattern 1 — Universal Provider for a shared relay key

If your team uses a single OpenAI-compatible relay (one base URL, one shared key) across Claude Code, Codex, and Gemini CLI, the Universal Provider tab is the right tool. Add it once with the relay's endpoint and key, check the boxes for the three apps to sync to, and CC Switch writes the same key to each CLI's native config file. When the relay rotates the key, you update one row and click Save and Sync — every CLI on every laptop picks it up on next switch.

Pattern 2 — WebDAV-synced config bundle for laptop + CI

Set up a WebDAV server (Nextcloud, self-hosted, whatever your team already has) and point Settings → Storage → custom config directory at it. Authors create providers on their laptops; CI runners using SaladDay/cc-switch-cli pull the same SQLite snapshot. Don't sync ~/.cc-switch/cc-switch.db through a vanilla Dropbox/iCloud folder — those will happily overwrite the database mid-write if two devices push at the same time. WebDAV with proper locking is the safer transport for the database file specifically.

Pattern 3 — Per-machine override for an on-call engineer

If one engineer needs to test a different provider without affecting the rest of the team, the device-level ~/.cc-switch/settings.json overrides for claudeConfigDir, codexConfigDir, etc. point that engineer's CC Switch at a non-shared config directory. The team's WebDAV bundle stays untouched; the override evaporates when they revert the setting.

For all three, treat ~/.cc-switch/cc-switch.db like ~/.ssh/ — it's not encrypted at rest, so the file's protection is filesystem permissions plus whatever transport you sync it over.

Advanced — MCP, Skills, Prompts, and OAuth Reuse

Once switching providers is solved, the unified panels are where CC Switch starts pulling its weight.

Unified MCP across five CLIs

Open the MCP button on any panel. CC Switch reads the MCP server list from each CLI's native location — ~/.claude.json for Claude Code, [mcp_servers.*] blocks in ~/.codex/config.toml, mcpServers in ~/.gemini/settings.json, and the equivalents in OpenCode and Hermes — and lets you toggle each server's presence per CLI. Add an mcp-fetch server once, sync it to Claude Code + Codex + Gemini, and every agent has the same tool surface.

Prompts (CLAUDE.md / AGENTS.md / GEMINI.md)

The Prompts panel is a Markdown editor with cross-app sync. Edit one prompt in CC Switch, sync to ~/.claude/CLAUDE.md, ~/.codex/AGENTS.md, and ~/.gemini/GEMINI.md in one click. Backfill protection means edits made directly in the live files are pulled back into CC Switch before the next sync overwrites them — useful if a teammate edited CLAUDE.md directly.

Skills

The Skills panel installs skills from GitHub repos or ZIP files into each CLI's skills directory (~/.claude/skills/, ~/.config/opencode/skills/, etc.). Symlink mode is the default — one canonical copy lives in ~/.cc-switch/skills/ and the CLIs see it via symlink. Switch to file-copy mode if your CLI is sandboxed and doesn't follow symlinks.

Codex OAuth reverse proxy (since v3.13)

CC Switch added a path that reuses your ChatGPT account's Codex entitlement inside Claude Code. It shows up as a new Claude provider card type — not as a Codex preset — and routes Claude Code's requests through a local proxy fronting chatgpt.com and auth.openai.com. Useful if you already pay for ChatGPT Plus or Pro. Read the in-app risk notice before relying on it for production; it's an unofficial reuse path and could break the day OpenAI tightens auth.

Alternatives — When You Don't Want a Desktop App

Two paths cover the cases where the Tauri desktop app isn't a fit.

Single OpenAI-compatible gateway — If you just want one base URL that Claude Code, Codex, Cursor, Cline, and OpenClaw can all point at, OfoxAI exposes https://api.ofox.ai/v1 as a unified OpenAI-compatible endpoint. Most teams reach for this when they want to avoid editing per-CLI configs at all; see the Claude Code integration guide for the one-line setup. You can keep CC Switch for tray-level switching and use ofox as the underlying provider, or skip CC Switch entirely if a single endpoint is enough.
Headless / scripted environments — SaladDay/cc-switch-cli is a Rust CLI fork that does provider switching, MCP management, skills install, and local proxy from the terminal. Same WebDAV-compatible storage format as the desktop app. Best fit for build agents, remote dev containers, and tmux-only servers.
Browser UI — Laliet/cc-switch-web is a community web-based alternative covering Claude Code, Codex, and Gemini CLI. Less feature-complete than the desktop app today, but useful if you want a shared URL the whole team opens.

How to Monitor and Stay Current

Auto-updater — Settings → About → Check for updates. The desktop app auto-checks on launch.
Release feed — Subscribe to github.com/farion1231/cc-switch/releases. Recent shipped changes worth tracking: Claude Desktop as a first-class panel, the Codex OAuth reverse proxy (v3.13), unified Skills install from GitHub (added pre-v3.16), and per-app usage dashboards.
Backup hygiene — ~/.cc-switch/backups/ keeps the 10 most recent SQLite snapshots automatically. Before any major version upgrade, cp ~/.cc-switch/cc-switch.db ~/.cc-switch/cc-switch.db.bak.$(date +%F) — a manual snapshot survives the rolling window.

FAQ

The questions here mirror the People Also Ask cluster for cc switch claude code install, ccswitch install npm, and related queries — covering supported CLI scope, terminal-restart behavior, fork landscape, official-account coexistence, the Codex OAuth reverse-proxy mode, key storage, headless support, and the hand-editing comparison.

Originally published on ofox.ai/blog.