DEV Community: Dylan Brown

How to migrate from the OpenAI API to an OpenAI-compatible gateway in 5 minutes

Dylan Brown — Tue, 21 Jul 2026 10:37:51 +0000

I maintain a couple of side projects that all talk to LLMs through the OpenAI SDK. Last month I moved them from api.openai.com to an OpenAI-compatible gateway (APIsRouter) so I could hit Claude, Gemini, DeepSeek and a few Chinese models (GLM, Kimi, Qwen) from the same client. The whole migration took less time than writing this post, so here is the exact diff.

What actually changes

One thing: the base URL. If your code uses the official OpenAI SDK, you are already done after this:

# Python
from openai import OpenAI

client = OpenAI(
    api_key="sk-...",                      # key from the gateway console
    base_url="https://api.apisrouter.com/v1",
)

resp = client.chat.completions.create(
    model="gpt-5.5",
    messages=[{"role": "user", "content": "hello"}],
)

// Node / TypeScript
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.APISROUTER_API_KEY,
  baseURL: "https://api.apisrouter.com/v1",
});

No SDK swap, no request shape changes. Streaming, tool calling and JSON mode go through unchanged because the gateway speaks the same wire protocol.

If you cannot touch the code

Most frameworks read the standard env vars, so this also works:

export OPENAI_API_KEY="sk-..."
export OPENAI_BASE_URL="https://api.apisrouter.com/v1"

That covers LangChain, LiteLLM, aider, Open WebUI, Cline and most agent frameworks without editing a line.

The part that surprised me: cross-vendor models in the same client

The model id is the router. Same client object, different vendors:

client.chat.completions.create(model="claude-sonnet-4-6", messages=msgs)
client.chat.completions.create(model="gemini-3.1-pro-preview", messages=msgs)
client.chat.completions.create(model="deepseek-v4-pro", messages=msgs)
client.chat.completions.create(model="glm-5.2", messages=msgs)

If you have ever wanted to A/B two vendors inside one eval script without juggling three SDKs and three billing dashboards, this is the appeal. GET /v1/models lists everything your key can use.

Gotchas I hit

Model ids are not always identical to the vendor's. Check the gateway's model list first instead of assuming. My eval script had gpt-5.5-turbo hardcoded from an old run and got a clean 404 with a readable error body, which made it easy to catch.
Anthropic-native clients are a separate path. Claude Code and other tools that speak the Anthropic Messages API use the same base URL but the /v1/messages route. Worth knowing it exists; nothing to configure if you stay on the OpenAI protocol.
Billing is prepaid. You top up first and usage draws down the balance. I prefer that failure mode over a surprise invoice, but if you expect postpaid invoicing, plan around it.

When this is not the right move

If you only ever call one vendor and you need that vendor's newest beta endpoints on day zero, staying direct is simpler. A gateway earns its place when you are comparing models, need a fallback vendor, or want Chinese models (GLM, Kimi, Qwen) that are annoying to buy access to from outside China.

Full endpoint reference and the current model catalog are on the docs page.

Happy to answer questions about the setup in the comments.

How much does Claude Code actually cost per session? I did the math

Dylan Brown — Mon, 13 Jul 2026 16:20:52 +0000

Last month I tracked Claude Code and Codex pass rates for 95 days. The question I got most in response was not about quality at all. It was "what does this actually cost you?"

Fair question. Claude Code is an agentic tool, not a chat window. One request from you can trigger dozens of model calls, and every call re-sends the entire growing conversation as input tokens. Input volume, not output, drives the bill. Output is usually a rounding error.

So I sat down and did the session math properly. Here it is.

Where the tokens actually go

System prompt and tool definitions: roughly 15 to 20K tokens, re-sent with every single model call.
CLAUDE.md, rules files, MCP schemas: loaded at session start, carried in every subsequent call.
File reads: a 500-line source file is 5 to 7K tokens, and it stays in the window after being read once.
Tool results: test output, grep hits, terminal logs all get appended and re-billed as input on every later turn.
The agentic loop itself: a single "fix this failing test" can produce 10 to 40 model calls before Claude Code reports back.

Two mechanics soften this. Prompt caching bills cache reads at about a tenth of the normal input rate, and /compact summarizes history to shrink the window. Both help a lot. Neither changes the fundamental rule: everything sitting in the window is paid for on every call that includes it.

This is why the same prompt costs wildly different amounts in different repos. A question in a fresh session with a lean CLAUDE.md is cheap. The same question 30 turns deep, after a dozen large file reads, rides on top of a six-figure token window.

Realistic session cost math

Model it turn by turn instead of guessing. A typical mid-size session runs about 40 model calls. Once a few files and some test output have accumulated, the full window averages around 90K tokens per call. Forty calls at 90K is 3.6M input tokens. Output stays small, about 40K tokens total.

At Anthropic direct list rates, before caching discounts:

Session profile	Input tokens	Output tokens	Sonnet 4.6 ($3/$15)	Opus 4.7 ($5/$25)
Light (10 calls, small repo)	~0.6M	~12K	$1.98	$3.30
Typical (40 calls, medium repo)	~3.6M	~40K	$11.40	$19.00
Heavy day (3 long sessions)	~12M	~120K	$37.80	$63.00

Treat these as ceilings. Prompt caching discounts the repeated prefix heavily, so cached sessions usually land lower. But the shape matches what people report: occasional afternoons cost a few dollars, and running Opus as a daily driver on the API can clear several hundred dollars a month.

Plan vs API break-even

Anthropic sells Claude Code two ways: flat-rate plans with usage caps enforced in rolling windows, or metered API billing with no cap.

The break-even is plain arithmetic. If your typical session costs about $11 on Sonnet at direct rates, a $20 plan pays for itself at roughly two sessions a month, provided you stay under its caps. The catch is the caps themselves. Heavy agentic use is exactly the pattern that trips rolling limits, and a flat rate stops being cheap the moment you are locked out mid-session with a failing build.

Many heavy users end up hybrid: a plan for interactive work, an API key for CI jobs and overflow.

The six levers that actually cut the bill

Keep CLAUDE.md lean. Every line is a recurring charge on every model call in every session.
Fresh session per task, /compact when it drifts. A 150K-token window bills 150K on each call.
Scope reads. Point it at specific files instead of letting it crawl. Exclude build output and vendored directories.
Disable MCP servers you are not using. Their tool schemas ride along in the system prompt on every call.
Route by model. Opus for architecture and gnarly debugging, Sonnet as the default, cheap models for boilerplate, test scaffolding, and commit messages.
Lower the per-token rate itself by changing where requests are billed.

The last two compound. Routing a cheap model through a discounted endpoint can turn a $10 session into well under a dollar for routine work.

Full disclosure on lever six: I operate keaiapi, an OpenAI-compatible gateway with pay-as-you-go billing and no subscription. Global models are priced 20% below official list. The routing ladder I actually use:

Model	Input $/1M	Output $/1M	Use it for
claude-opus-4-7	$4.00	$20.00	Architecture, hard debugging
claude-sonnet-4-6	$2.40	$12.00	Default coding driver
glm-5	$0.514	$2.314	Routine edits, agentic sub-tasks
deepseek-v4-pro	$0.3915	$0.783	Boilerplate, tests, commit messages

Repointing Claude Code takes one settings change

Claude Code reads ANTHROPIC_BASE_URL on startup, so switching where tokens are billed is a config change, not a workflow change. Model IDs stay the same, tool execution stays local, and rollback is deleting one line.

// ~/.claude/settings.json
{
  "env": {
    "ANTHROPIC_BASE_URL": "https://api.keaiapi.com",
    "ANTHROPIC_AUTH_TOKEN": "sk-keaiapi-...",
    "ANTHROPIC_MODEL": "claude-sonnet-4-6",
    "ANTHROPIC_DEFAULT_OPUS_MODEL": "claude-opus-4-7"
  }
}

Verify the endpoint with a one-off curl before pointing your daily driver at it:

curl https://api.keaiapi.com/v1/chat/completions \
  -H "Authorization: Bearer sk-keaiapi-..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-4-6",
    "messages": [{"role": "user", "content": "Reply with one word: ok"}],
    "max_tokens": 16
  }'

Run /cost inside a session to see what you are actually burning. If your numbers look very different from my table, I would genuinely like to hear them: repo size, session length, and which model you run as the driver.

The full breakdown with the plan comparison table and FAQ lives here.

I tracked Claude Code and Codex pass-rates for 95 days — what "getting dumber" actually looks like

Dylan Brown — Sat, 30 May 2026 05:04:45 +0000

Every few weeks a thread blows up: "Is Claude Code getting worse?" Someone swears Opus got lazy after an update; someone else says it's placebo. The arguments are always vibes — nobody posts numbers.

So I built a tracker. For ~95 days it's logged the daily SWE-Bench-Pro pass rate for Claude Code and Codex — the % of real coding tasks each agent completes unassisted — and plotted them as candlesticks (open = yesterday, close = today, wick = the 90% confidence interval for that day's sample). Same idea as a stock K-line, except the "price" is how often the agent actually solves the task.

Here's what the data says — and it's more interesting than "it got dumber."

Claude Code: a real step up, then a recent slide

Plotting per-model-version baselines (median of the first 14 days after each release) makes the story obvious:

Opus 4.6 era — baseline ~54%
Opus 4.7 era — baseline ~65%

That 4.6 → 4.7 jump is a genuine +11 percentage point step. Not placebo — the model got materially better at finishing tasks, and it held ~65% steady for a month.

Then the last ~7 days: today's pass rate is ~52%, well below the 65% baseline and past the significance threshold (p < 0.05). So the "Claude Code feels worse lately" crowd isn't imagining it — there's a real, recent drift below the current model's own established baseline. Whether it's a quantization change, a routing tweak, or load — the number moved, and it moved past noise.

The nuance most threads miss: Claude Code is both "much better than 6 months ago" and "drifting down this week." Both are true. Vibes can't hold two facts at once; data can.

Codex: three versions, basically flat

Now the part nobody expects. Across three Codex releases:

gpt-5.3-codex — ~58%
gpt-5.4-xhigh — ~54%
gpt-5.5-xhigh — ~56%

Three "major" version bumps, and the pass rate just oscillates in a 54–58% band. No step change. The releases didn't move the benchmark needle the way Opus 4.7 did. If you've felt like "new Codex doesn't feel smarter" — the data agrees: it's been flat.

Why candlesticks (and a fixed 0–100 axis)

Two design choices that matter if you want to read drift honestly:

Fixed 0–100% y-axis. Auto-scaling per time window makes a 5pp dip look catastrophic because the view zooms in. A 5pp drop should look like a 5pp drop whether you're comparing 30 days or 90, Claude or Codex.
Per-era baselines, not one flat line. A single baseline across model versions lies about the older model. Each release gets its own dashed reference, so you can see the step, not just the absolute level.

The live, daily-updating version (red/green toggle for CN vs Western convention, daily/weekly K, 30/90/all windows per agent) is here: Drift K-Line tracker →

What this means if you ship with these agents

Don't trust a single bad day. One red candle is inside the noise band. A week below baseline is signal. Watch the baseline line, not the last datapoint.
"Newer version" ≠ "smarter." Codex's flat line is the proof. Benchmark before you migrate a workflow to a new release.
Capability drifts. Your costs shouldn't have to. If an agent quietly drops 13pp, the last thing you want is to also be locked into one vendor's pricing while you wait it out.

Author note: I build keaiapi, a pay-as-you-go aggregator that routes Claude, GPT, Gemini, DeepSeek and 20+ models through one OpenAI-compatible endpoint — so when a model drifts, you can switch the one you point at without rewriting code or eating a subscription. The tracker above is a free tool we run; no signup needed to read it. Methodology notes are on the tracker page.

Building an Autonomous AI Agent That Writes Novels — Architecture of a 10-Agent Pipeline

Dylan Brown — Fri, 27 Mar 2026 16:00:03 +0000

AI-generated fiction has a consistency problem. Ask any LLM to write chapter 1 of a novel and it'll do a decent job. Ask it to write chapter 30 and it has no idea what happened in the first 29.

I built InkOS to solve this. It's an open-source CLI AI agent that writes, audits, and revises novels autonomously — using a pipeline of 10 specialized AI agents with persistent state tracking across the entire book.

This post walks through the architecture and the specific engineering problems it solves.

The Problem

Most AI writing tools work like this: you give the model a prompt, it generates text, you copy it, repeat. There's no memory between chapters. After 20+ chapters, you run into:

Continuity breaks — characters remember things they never witnessed, weapons reappear after being lost, relationships reset
Context bloat — injecting all previous state into each prompt hits token limits, causes 400 errors, costs $200/chapter in API calls
Hook accumulation — the model plants plot hooks but never resolves them. After 30 chapters you have 40+ dangling threads
AI voice — every paragraph uses the same words ("delve", "tapestry", "testament", "intricate"), sentence structure is monotonous, and there's excessive summarization

The Architecture: 10 Agents in Sequence

Instead of one model doing everything, InkOS splits the work across 10 specialized agents:

Radar → Planner → Composer → Architect → Writer → Observer → Reflector → Normalizer → Auditor → Reviser

Each agent has exactly one job:

Agent	What it does
Radar	Scans platform trends and reader preferences (pluggable, skippable)
Planner	Reads author intent + current focus + memory retrieval, produces chapter intent with must-keep/must-avoid lists
Composer	Selects relevant context from truth files by relevance, compiles rule stack
Architect	Plans chapter structure: outline, scene beats, pacing targets
Writer	Produces prose from composed context (length-governed, dialogue-driven)
Observer	Over-extracts 9 categories of facts from the chapter text
Reflector	Outputs Zod-validated JSON deltas for state updates
Normalizer	Single-pass compress/expand to hit the target word count band
Auditor	Validates draft against 7 truth files across 33 dimensions
Reviser	Auto-fixes critical issues, flags others for human review

If the audit fails, the pipeline loops back: revise → re-audit until all critical issues are resolved.

State Management: 7 Truth Files

Every book maintains 7 canonical truth files as the single source of truth:

current_state.md — character locations, relationships, knowledge, emotional arcs
particle_ledger.md — resource accounting: items, money, stats with quantities
pending_hooks.md — open plot threads, foreshadowing, unresolved conflicts
chapter_summaries.md — per-chapter summaries with state changes
subplot_board.md — A/B/C subplot line status tracking
emotional_arcs.md — per-character emotion tracking and growth
character_matrix.md — interaction matrix, encounter records, information boundaries

The Auditor checks every draft against these files. If a character "remembers" something they never witnessed, or pulls a weapon they lost two chapters ago — the auditor catches it.

Since v0.6, truth files are stored as Zod-validated JSON (story/state/*.json). The Reflector outputs JSON deltas — not full markdown rewrites. Corrupted data is rejected, not propagated.

Solving Context Bloat: SQLite Temporal Memory

On Node 22+, InkOS uses a SQLite temporal memory database (story/memory.db). Instead of injecting all 7 truth files into every prompt (which blows up after 20 chapters), the Composer agent does relevance-based retrieval — pulling only the facts, hooks, and summaries that matter for the current chapter.

This was the single biggest improvement in v0.6. Before: context bloat caused 400 errors and made each chapter cost $200+ in API calls. After: selective retrieval keeps context lean regardless of book length.

Hook Governance

One of the hardest problems in long-form AI fiction: the model loves planting hooks but never pays them off. After 30 chapters you'd have 40+ open threads, none resolving.

The Planner agent now generates a hookAgenda — scheduling which hooks to advance and which to resolve in each chapter. analyzeHookHealth audits hook debt, evaluateHookAdmission blocks duplicate hooks, and new mention semantics prevents fake advancement (where the model references a hook without actually progressing it).

De-AI-ification

Every genre profile includes a fatigue word list. For LitRPG: "delve", "tapestry", "testament", "intricate", "pivotal". The Auditor flags these automatically.

But detection alone isn't enough — InkOS bakes de-AI-ification into the Writer agent's prompts at the source: banned sentence patterns, style fingerprint injection, dialogue-driven scene guidance. revise --mode anti-detect runs dedicated anti-detection rewriting on existing chapters.

You can also clone any author's style: inkos style analyze reference.txt extracts a statistical fingerprint (sentence length distribution, word frequency, rhythm profiles), and inkos style import injects it into all future chapters.

Genre Support

10 English-native genre profiles, each with dedicated pacing rules, audit dimensions, and fatigue word lists:

LitRPG, Progression Fantasy, Isekai, Cultivation, System Apocalypse, Dungeon Core, Romantasy, Sci-Fi, Tower Climber, Cozy Fantasy — plus 5 Chinese web novel genres.

Getting Started

npm i -g @actalk/inkos
inkos book create --title "The Last Delver" --genre litrpg
inkos write next

One command writes a full chapter: draft → audit → auto-revise. Run inkos up for daemon mode that writes chapters on a schedule.

Works with Claude, GPT-4, or any OpenAI-compatible API including local models. Multi-model routing lets you put Claude on the Writer and GPT-4o on the Auditor.

InkOS is also published as an OpenClaw skill — install with clawhub install inkos and any compatible agent can invoke it.

GitHub: github.com/Narcooo/inkos (2.4k stars, MIT license)

npm: npm i -g @actalk/inkos

Would love feedback from anyone working on multi-agent systems, long-context state management, or creative AI. What continuity problems have you run into with long-form AI generation?