DEV Community: APIClaw

A Four-Check Debugging Method for OpenAI-Compatible AI APIs

APIClaw — Fri, 30 Oct 2026 05:13:33 +0000

Debugging an OpenAI-compatible gateway: separate transport from quota

When a coding agent fails against an OpenAI-compatible API, I first separate four variables: URL construction, authentication, request shape, and quota. Changing the model before checking those variables makes the diagnosis slower.

Start with the smallest request

Use the versioned API root documented by the gateway, then send one system message and one user message. Log the method, path, status, model ID, request ID, and latency; redact the key. A 401 usually means the key was not loaded, while a 404 often means the SDK appended a path to the wrong base URL.

Check the budget separately

A request-metered plan behaves differently from per-token billing. APIClaw is a flat-rate, OpenAI-compatible AI API gateway with a monthly plan and daily request quotas rather than a per-token bill. Its free tier includes 50 requests without a card. The APIClaw docs describe the OpenAI-compatible base URL and client setup.

For an agent, count requests, retries, and model weighting independently. A retry loop can exhaust a daily quota even when prompts are short. APIClaw documents weighted requests for heavier models and quota reset timing in its flat-rate API guide.

Add features one at a time

After the minimal non-streaming call works, add streaming, tools, and structured output individually. Compare the same prompt with streaming disabled; this isolates server-sent-event handling from authentication and payload validation. Bound retries for 429 and 5xx responses, but do not retry a malformed request or a 401.

This method produces a reproducible failure instead of a vague “the model is broken” report: URL, auth, payload, transport, and quota each get their own test.

Why a shared API gateway key beats three consumer seats for a team of three

APIClaw — Mon, 26 Oct 2026 20:34:11 +0000

Three developers, one AI budget line. The obvious solution is three consumer subscriptions — everyone gets their own seat, everyone gets their own quota, nobody argues. The less obvious problem is that a subscription seat and a shared API key solve different problems, and only one of them survives contact with a team.

This is the arithmetic I did when I moved a three-person team off seat-based plans onto a single gateway key.

What a seat actually buys

A consumer subscription for an AI coding tool is a per-person product. Seat, account, session history, quota. When three people each buy one, you pay three times for three private pools of capacity, and the pools do not talk to each other.

That last part is the issue. Work does not distribute evenly. On any given Tuesday one developer is running a refactor agent across 60 files while the other two are writing tests. Seat A's ceiling fires at 11am; seats B and C are at 20% utilisation. You can see the waste on the invoice and you cannot move it.

Published seat prices for the mainstream coding plans land anywhere from $20 to $200 per person per month depending on tier. Three seats is therefore somewhere between $60 and $600 a month — call it $60 for the cheap tier and $300 for the tier where the agent actually works.

The same team, metered per request

Here is the measurement side. Over a month, the three of us generated these weighted daily averages from gateway logs — one row per API request, weighted by model (Claude Sonnet class counts 2, Opus and GPT-6 Sol class count 3, DeepSeek 1):

Developer	Weighted requests/day	What they do
A	120	agent-driven refactors, Sonnet class
B	95	test generation and review
C	85	scripts and docs, DeepSeek for cheap work
Total	300	9,000 weighted requests/month

Now price that against a request-metered gateway. APIClaw's tiers are monthly with a daily request quota that resets at 00:00 UTC, and the one relevant to this workload is Max at $59 for 1,500 requests a day — 20% utilisation, 80% headroom.

Option	Monthly cost	Aggregate quota
3 × cheap seat ($20)	$60	3 private pools, none shared
3 × working seat ($100)	$300	3 private pools, none shared
1 × shared key, Max tier	$59	1,500 weighted requests/day, pooled

The shared key costs less than the cheap tier and one fifth of the working tier, and the quota is pooled rather than partitioned. When developer A needs 360 requests on refactor day — three times their average, which is a realistic spike and not a pathological one — they draw from a pool of 1,500 instead of hitting a personal ceiling at 500 that nobody else can top up.

$59 split three ways is $19.67 a person. That is the honest number, and it is the one to put in the budget spreadsheet.

The catch: a shared key is a shared blast radius

The pooling argument is easy. The operational cost of one key is real and it is not small, so here is what you have to build before you share anything.

Name the key after the human, not the team. One key per developer, all on the same account. That gives you per-person revocation — when someone leaves, you revoke one key and their gpt-6-sol access is gone by the next request, without rotating anything for the other two. A single shared secret string means everyone rotates in a bad hour.

# one key per person, same account, so per-person revocation is a one-click
export APICLAW_KEY_ALICE=...
export APICLAW_KEY_BOB=...

Attribute before you optimise. A gateway that logs per-request model and outcome answers "who used 40% of the pool" in week three. A seat-based tool usually will not.

Set the ceiling below the pool, not at it. If the pool is 1,500, agree a soft 900. The ceiling's job is to catch a runaway loop, not to enforce fairness between humans.

Keep the model choice in the code, not in a mood. A shared pool means one person's frontier-model habit is visible to everyone. Routing cheap work — summarising, docstrings, lint fixes — to a weight-1 model and reserving weight-3 models for planning is a two-line client change and the biggest lever on how long the pool lasts.

Where the single key genuinely wins beyond price

The argument that convinced my colleagues was not the invoice. It was that a gateway key is a plumbing decision, and a seat is a product decision.

With one key you can point any OpenAI-compatible tool at the same endpoint and get the same models without a new purchase: Continue in VS Code, Cline, Kilo Code, a shell script, a CI job that summarises a diff. Seat-based plans cover the tool they ship. If your team's work is spread across three editors and a couple of scripts, the seat model sends you shopping; the gateway model does not.

APIClaw is the flat-rate version of that: one key at https://apiclaw.biz/v1 covering Claude, GPT, Gemini, DeepSeek, Qwen, Kimi, GLM and the rest, request-metered rather than token-metered, and it also speaks Anthropic Messages for tools that expect that shape. The practical point for a small team is that the quota is an account-level pool with published request weights, so the "who is using the shared budget" question has a factual answer — see how the weights and tiers work before you size anything.

Payment is crypto via NOWPayments, which is worth naming early rather than discovering at checkout, and support is on Telegram rather than a ticket queue.

When a seat is the better buy

Three cases, honestly:

One person, one tool, heavy use. A single $100 seat beats a $59 pool if that person alone would use 1,400 requests a day. The pool only wins when it is shared.
You need the tool's own agent features. Seat plans bundle vendor-specific completions and agent modes that bring-your-own-key does not reproduce.
You need zero ops. A shared key is one more thing to revoke, rotate and attribute. Sometimes that is not worth $241 a month.

But if the team is three people with varied workloads, spread across more than one editor, and one of you is regularly hitting a ceiling while the others are not, a request-metered gateway key is the structurally correct shape. Pooled capacity, one line on the invoice, one endpoint to point everything at. The one-key setup covers what the endpoint accepts and how to lay out keys per person without sharing a secret.

Run the numbers against your own logs first: sum your team's weighted requests per day, pick the tier that leaves 50–80% headroom, and compare that to seats × price. If the pool wins by less than 20%, stay on seats — the operational overhead is not free.

I metered my own coding agent for a week: 97% of the tokens went to re-reading

APIClaw — Mon, 26 Oct 2026 20:18:50 +0000

An agent turn is not a message. It is the whole conversation so far, re-sent. I spent a week logging per-turn usage objects off the OpenAI-compatible endpoint my agent talks to, and the thing that surprised me was not the total — it was the ratio. Prompt tokens were 97% of everything the agent consumed, and output tokens were 3%.

That is not a rounding artefact. It is what an agent loop does by construction, and once you see the arithmetic you stop optimising the wrong term.

What I logged

Every response from a chat completions call carries a usage block:

{
  "usage": {
    "prompt_tokens": 13800,
    "completion_tokens": 450,
    "total_tokens": 14250
  }
}

I appended one row per turn — {session_id, turn_index, prompt_tokens, completion_tokens, tool_name, wall_ms} — to a JSONL file. Nothing fancy. Roughly 15 minutes of work with the http.client in stdlib, because the client I use lets me set a response hook.

The point of logging per turn rather than per request-day is that the daily total hides the shape. A day with 40 turns across 4 short sessions and a day with 40 turns in one long session cost wildly different amounts, and the billing dashboard shows you the same number.

The growth curve

Here is the arithmetic that matters. Take a session with a 9,000-token base context (system prompt plus tool schemas), each tool result adding 1,200 tokens to the transcript, and a 450-token assistant reply per turn.

Prompt tokens at turn n: 9000 + 1200 × (n − 1).
Output tokens at turn n: 450.

Turns	Prompt tokens	Completion tokens	Total	Tokens/turn	Cost @ $3/$15 per M
1	9,000	450	9,450	9,450	$0.0338
3	30,600	1,350	31,950	10,650	$0.1120
6	72,000	2,700	74,700	12,450	$0.2565
12	187,200	5,400	192,600	16,050	$0.6426
25	585,000	11,250	596,250	23,850	$1.9238
40	1,296,000	18,000	1,314,000	32,850	$4.1580

The sums are just arithmetic series: prompt total is n × 9000 + 1200 × n(n−1)/2. At n=40 that is 1,296,000 tokens to produce 18,000 tokens of actual text — a 72:1 ratio. The cost column uses $3 per million input and $15 per million output, roughly frontier-tier rates at time of writing; substitute your model's numbers, the shape does not change.

Two things fall out of that table:

Tokens per turn rises with session length. It goes 9,450 → 32,850 between turn 1 and turn 40. A dashboard that reports average tokens per request across a month is averaging a distribution that has no stable mean.

Output price barely matters. At 12 turns, prompt tokens are 97% of volume but only 87% of cost — because output is 5× the input rate. Even at that multiplier, you are paying mostly to re-read.

What a week looked like in requests

The unit my gateway meters is the request, not the token, so the same data has to be re-expressed. A representative session in my logs was 12 turns. At 48 sessions a week:

48 × 12 = 576 requests per week
576 × 16,050 average tokens = 9.24M tokens per week, of which 9.0M were prompt
weighted for a 2× model like the Sonnet class: 1,152 weighted requests per week

Compare that to a daily request allowance: 1,152 weighted over seven days is 165 a day on average — comfortably inside a 500/day tier even with a 3× spike on a heavy debugging day. The same workload on a token meter at those rates is $30.84 a week, or about $133 a month if the pattern holds through four and a half weeks. That number is why I started looking at request-metered gateways in the first place: my monthly token bill was tracking my agent's verbosity, not my output.

The break-even is worth computing before you argue about either model. At $0.64 per 12-turn session, a $39 flat tier pays for itself at about 61 sessions a month. Sixty-one sessions is three a working day. If you run fewer, per-token is cheaper and you should stay on it — honestly.

Three things that actually reduced the number

Not vibes — measured deltas on the prompt side, which is where 97% of the tokens are.

1. Trim tool results before they enter the transcript. A grep that returns 400 lines stays in context for every subsequent turn. Truncating to the 40 relevant lines cost me nothing in task success and cut the 1,200-token average tool result to about 500. Rerun the 40-turn series with that one change: 40 × 9000 + 500 × (40 × 39 / 2) = 360,000 + 390,000 = 750,000 prompt tokens against 1,296,000 — a 42% cut on the same task, from one truncation rule.

2. Start a fresh session per task, not per project. Four 10-turn sessions on the same work cost 4 × (10 × 9000 + 500 × 45) = 450,000 prompt tokens instead of 750,000, because the quadratic growth term is split into four shallower curves. You also get better tool selection out of a shorter transcript, which is a side effect I did not expect.

3. Cap the loop, don't cap the tokens. A max_turns guard at 25 refuses the pathological 80-turn session that produces nothing. Output token limits do not help here — the runaway is on the input side.

Why I meter requests now

Once the ratio is 97/3, the per-token invoice is charging you for a number you can only influence indirectly, through context hygiene. A request-metered gateway charges for the shape instead: one turn is one request. The cognitive load drops, and so does the variance — my worst week and my best week are within 15% of each other now, where the token bill used to swing 3×.

APIClaw runs on that model: a flat monthly tier metered in requests at https://apiclaw.biz/v1, with heavier models counting for more against the daily pool (Claude Sonnet class 2×, Opus and GPT-6 Sol class 3×, DeepSeek 1×). That weighting is the honest part of the design — it admits the 72:1 ratio exists — and the tier table is the thing to check against your own request count, not against a token estimate.

If you want to reproduce this: log usage per turn for one week, compute prompt_total / completion_total, and see whether your workload looks like mine. If the ratio is above 20:1, you are paying for re-reads and should optimise the transcript, not the prompt wording. The comparison page puts the token-metered and request-metered shapes side by side if you want the pricing arithmetic spelled out.

The one-number summary I took away from the week: my agent wrote 18,000 tokens of thinking per long session and read 1.3 million to do it. Everything I have changed since is aimed at the 1.3 million.

Pointing VS Code Continue and Windsurf at a custom OpenAI-compatible base URL

APIClaw — Mon, 26 Oct 2026 20:15:50 +0000

Pointing an editor at a custom OpenAI-compatible endpoint is documented as a settings-screen operation. In practice the settings screen is the part that fails silently, and the extension's config file is the part that decides what actually gets sent. These are the steps that hold up on VS Code and on Windsurf, plus the four failures I keep seeing in other people's bug reports.

Prove the endpoint before you touch the editor

Every "the extension won't connect" report I have read this year was really an endpoint problem wearing an editor costume. Test the endpoint from a shell first:

curl -s https://apiclaw.biz/v1/chat/completions \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"YOUR_MODEL_ID","messages":[{"role":"user","content":"say ok"}],"max_tokens":16}'

Three outcomes tell you everything. 401 means the key did not arrive. 404 almost always means the path is doubled — a base URL of .../v1 plus a client that appends /v1/chat/completions gives you /v1/v1/chat/completions. A 400 naming the model means you typed a shortened model id instead of the full one your account can use; copy it, don't retype it.

If that curl returns a completion, the remaining problem is local configuration and you can stop suspecting the network.

Continue on VS Code: the whole config

Continue reads ~/.continue/config.yaml (%USERPROFILE%\.continue\config.yaml on Windows). The minimal working model block looks like this:

name: APIClaw
version: 1.0.0
schema: v1
models:
  - name: APIClaw
    provider: openai
    model: YOUR_MODEL_ID
    apiBase: https://apiclaw.biz/v1
    apiKey: YOUR_API_KEY
    capabilities:
      - tool_use
      - image_input
    roles:
      - chat
      - edit
      - apply

Four details in those twelve lines are the ones that break:

apiBase, not baseUrl. A wrong key name is not an error. Continue ignores it, falls back to api.openai.com, and you get a 401 from a host you never configured.
capabilities overrides autodetection. Continue guesses capability from model names it recognises. An unfamiliar id gets no tool_use, and agent mode then answers with prose and never calls a tool — it looks like a broken agent when it is a broken declaration.
roles decides where the model appears. Missing apply and the Edit pane will appear to do nothing. Missing edit and you can only chat.
Environment variables do not reach the extension host. Continue's IDE extensions cannot read your shell env, so exporting the key in .zshrc accomplishes nothing. Put APICLAW_API_KEY=... in ~/.continue/.env and reference it through Continue's secret lookup syntax in place of the literal apiKey. Then restart the IDE — reloading the window is not enough, the extension host caches the config.

Windsurf is a different problem

Windsurf was renamed Devin Desktop in June 2026, and Cascade was removed in September 2026, leaving Devin Local as the only built-in agent. Devin Desktop has no setting for a custom OpenAI-compatible base URL. There is no field to fill in.

So the gateway is used through an extension, not through the app. Open the Extensions panel — it uses the Open VSX registry, which matters: Roo Code is there, Cline may or may not be depending on your build. Install an OpenAI-compatible coding extension, choose the OpenAI Compatible provider, paste the base URL, the key, and a full model id. Only that extension talks to the gateway. The built-in agent and autocomplete stay on the models the application ships with, which is a feature if you like the app's own completions and a wart if you were hoping to consolidate everything.

The Windsurf path is written up in the APIClaw Windsurf notes, and the VS Code path — including the Copilot Chat option — is in the VS Code guide.

Copilot Chat as a second front end

If you are already inside VS Code, the built-in Copilot Chat can take a custom endpoint: Manage Language Models from the model picker, then the Custom Endpoint provider. The older "OpenAI Compatible" provider is deprecated, so do not start there. The URL must include the path, not just the host — https://apiclaw.biz/v1/chat/completions, or /v1/responses if you set API type to Responses. Bring-your-own-key is enabled by default on Copilot Business and Enterprise, but an admin can switch it off by policy, and it covers chat and agent mode only; inline completions and embeddings stay on the Copilot plan.

The failure table

Symptom	Cause	Fix
404 on every request	path doubled	base URL ends at `/v1`, client appends the rest
401 immediately	key never loaded	`.env` file, secret reference, IDE restart
400 "model not found"	shortened id	copy the full id from the model hub
Agent replies, edits nothing	no `tool_use`	declare capabilities
Edit pane inert	no `apply` role	add it to `roles`

Why the indirection is worth it

Because these editors send whatever model id you typed, you can route by role: a mid-weight model for edit and tab-adjacent work, a frontier model when the agent is planning. That is only possible when the models arrive through one endpoint. APIClaw is a flat-rate gateway that answers the OpenAI chat completions schema at https://apiclaw.biz/v1, and it also speaks Anthropic Messages for tools that expect that shape, so a single key covers both generations of tooling — the argument for one key across every model rather than one subscription per editor. The request is the billing unit, heavier models count more against the daily pool, and there is no per-token line on the invoice to reverse-engineer afterwards.

Start with the curl. Nine times out of ten, that is the whole bug.

kluster.ai's API Is Going Away — A Migration Guide and 5 Alternatives

APIClaw — Mon, 26 Oct 2026 17:59:56 +0000

kluster.ai's API Is Going Away — A Migration Guide and 5 Alternatives

If you still have kluster.ai in a config file somewhere, this is the post to read end to end.

As of late October 2026, kluster.ai no longer resolves for us at the DNS level — the inference API, the docs, and the site are gone or unreachable from multiple networks we checked. The company was acquired by MITO AI (announced on LinkedIn and covered by Dealroom), and the AI-video platform consolidated the team and technology. Whatever the public EOL notice said or didn't say, the practical reality is the one that matters for your code: assume the kluster endpoint can fail without notice. It may already be failing for you.

The good news: migrating off a dead gateway is a 20-minute job when you know what to touch. This post is the exact checklist we use, plus five alternatives compared on the dimensions that actually bite.

What breaks when a gateway dies

A gateway API is a thin slice of configuration — but it hides inside everything. Before you switch, inventory these four things:

The base URL. https://api.kluster.ai/v1 (or whatever you hardcoded) lives in environment variables, .env files, CI secrets, and sometimes in committed code you forgot about.
The API key. Keys are per-account. If the account is gone, every key is dead.
Model IDs. If you call model: "kluster/...", that string breaks the moment the gateway stops routing it — even if you point the base URL somewhere else.
Provider-specific parameters. This is the hidden one. Gateways silently ignore or reject params they don't support (top_k, repetition_penalty, custom extra_body fields). A replacement may accept the request and return different output because it interprets a param differently. Parameter drift is how "it still works" becomes "it works differently and you didn't notice."

Also: export your logs and billing history before the account dies. You will not be able to reconstruct usage retrospectively.

The migration checklist

This works for kluster → anything, and for any future gateway death:

Inventory every place the base URL and key appear: grep -r "kluster" . is your friend. Don't forget CI (~/.github/workflows, GitLab CI, CircleCI), Docker compose files, and .env files in deploy directories.
Export what you need: logs, usage, any fine-tuned model artifacts that were stored on their side.
Pick a replacement (table below), create the account, generate a key.
Change two things: base URL and API key. Model IDs usually need a mapping step — see the alternatives table.
Re-run one golden prompt — a request you know the expected output of — and diff the output, don't just check the HTTP status.
Cut over a non-critical job first. Watch it for a day, then move the real workloads.

Here's the diff pattern for a Python openai client:

# before
client = OpenAI(base_url="https://api.kluster.ai/v1", api_key="sk-...")
# after
client = OpenAI(base_url="https://api.openrouter.ai/api/v1", api_key="sk-...")

And for a raw curl (this is what lives in Claude Code / Cursor configs):

# before
curl https://api.kluster.ai/v1/chat/completions \
  -H "Authorization: Bearer $KEY" \
  -d '{"model": "kluster/...", "messages": [...]}'

# after — base URL and model id both change
curl https://openrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $KEY" \
  -d '{"model": "anthropic/claude-sonnet", "messages": [...]}'

Five alternatives, grouped by what you need

	Price	What actually limits you	Frontier closed models?	OpenAI-compatible?
APIClaw	$19–$129/mo flat	Daily request count (500–8,000/day), unlimited tokens per request	Yes — Claude, GPT, Gemini, 100+ models	Yes
OpenRouter	Pay per token + 5.5% credit fee	Your balance	Yes	Yes
Requesty	Pay per token (+5% processing)	Your balance	Yes	Yes
NanoGPT	Subscription covers open-weight models; closed models billed from prepaid balance	Weekly token allowance on the sub; balance for the rest	No subscription includes them	Yes
Chutes	Free tier + credits	Request/day on open-weight models	Open-weight only	Yes

Cheapest per token / open-weight only

Chutes and NanoGPT both lean open-weight. If your workload was kluster's OSS models and you want minimal cost, these are the familiar trade: cheap or free, but no Claude/GPT/Gemini in the flat subscription.

Broadest catalogue

OpenRouter is the default for a reason — every model, one key, per-token billing. You trade the flat bill for a metered one and a credit fee.

Enterprise controls

Requesty adds EU residency options and guardrails on top of the same per-token model. Good when compliance matters more than price.

Flat monthly with frontier models

*APIClaw is the one that behaves like kluster's promise *and keeps a predictable bill: $19–$129/month, your limit is a daily request count, not a token meter, and it serves Claude, GPT, Gemini and 100+ other models through one OpenAI-compatible endpoint. If you moved to kluster to get away from unpredictable per-token invoices, this keeps that property. The client setup docs cover Claude Code, Cursor, Codex CLI and the rest.

One honest caveat for all of them: read the model list before you commit. "OpenAI-compatible" doesn't mean "same catalogue."

If your kluster workload was batch jobs

Batch jobs have no human in the loop to absorb a mid-run failure. This is where a gateway death hurts most:

A per-token service fails gradually (balance runs down, requests get slower) — you get warnings.
A fixed-allowance service fails sharply (daily request cap or account gone → hard 429s) — your batch just dies at whatever offset it reached.
A flat-daily-allowance service with a bounded count is the closest to "predictable": you know exactly how many requests a batch will consume, so you can size it against the allowance before you start.

Whatever you pick, add a wrapper that records the last successful offset and retries from there. Gateway migrations are not the last time you'll need it.

How to test a replacement in 10 minutes

Don't migrate on faith. Run this on the new endpoint before you change anything in production:

One long-context request. Push a 100K+ token prompt. This exposes context-window handling and prompt-caching behavior differences.
One tool-calling round. Not all gateways forward function/tool definitions identically. This is where silent breakage lives.
A 20-turn loop. Agents are loops; a single-shot test that passes can still fail at turn 15 when state accumulates.

Compare latency and output diff against your golden prompt — not just error codes. If the replacement passes all three, you're safe to cut over.

Disclosure: APIClaw is a flat-rate AI API gateway and is listed in the table above. The migration checklist and test procedure work with all five providers — the ranking is on what limits you, not affiliation.

This post reflects what we could verify on 2026-10-26: kluster.ai's endpoint no longer resolves, and MITO AI's acquisition of kluster.ai was publicly announced. If you have a published EOL notice from kluster.ai, link it in the comments and we'll update this post.

The five-hour limit, the weekly ceiling, and what each one actually costs you

APIClaw — Mon, 26 Oct 2026 06:06:40 +0000

Subscription AI plans advertise a message allowance. What they enforce is a set of overlapping limits that interact, and the interaction is what ends your session. This breaks down the three that matter and how to tell which one you hit.

The three limits, in order of how often they fire

1. The rolling window. A short window (commonly five hours) with a message cap that resets on a rolling basis. This is the one people notice first because it fires mid-afternoon. It is not a daily limit: the window slides, so waiting two hours can free capacity even though the day is young.

2. The weekly ceiling. A longer allowance that the short windows draw from. This is the limit that actually ends a heavy week, and it is the one least visible in the UI. You can be inside every five-hour window and still be out of weekly capacity.

3. The model multiplier. Not all messages cost the same. Requests routed to a larger model consume the allowance faster, so the same number of prompts can be comfortable on one model and ruinous on another.

The failure mode is that these compound. A user who hits the weekly ceiling on Thursday experiences it as "the five-hour limit got worse," because the short window is the only counter they can see.

How to tell which one you hit

The signal is in the reset time, not the error message:

Resets in ~5 hours → rolling window. Wait it out, or reduce model size.
Resets on a named day → weekly ceiling. Waiting a few hours will not help.
Resets immediately but fails again on the next long request → context size, not the cap. A single large request can be refused when the window looks open.

That last case is the one people misdiagnose most often. A 200k-token request against a nearly-empty window can still fail, and the error reads like a rate limit.

What the limits are actually protecting

These caps exist because a long agent run has a cost profile a chat message does not. An agent loop re-sends its accumulated context on every step, so one task can consume more input tokens than a hundred chat turns. Providers cap it because the tail of that distribution is expensive and hard to predict.

That is a reasonable thing to cap. The problem for anyone running agents is that a message-count limit is the wrong unit: it does not tell you how many agent steps you can afford, and the answer changes with context length and model choice.

The practical workaround

If your workload is agents rather than chat, the useful question is not "how many messages do I get" but "how many requests can I issue, and what happens at the limit." A request-based allowance answers that directly:

You can compute steps per task, and therefore tasks per day, before you start.
The ceiling is a count rather than a moving token budget, so scheduling work becomes arithmetic.
A clean refusal at the boundary lets your client back off instead of degrading mid-task.

The gateway I use for this is APIClaw — [OI]-compatible, priced per request rather than per token, with published request limits per plan and separate keys per client. Per-tool setup for Cursor, [CC], Codex and about twenty other clients is in the docs.

What to check before you switch anything

Are request limits published per plan, or only token limits? A plan with no request cap has no upper bound you can plan against.
What happens exactly at the ceiling? A 429 with Retry-After is workable. A dropped stream wastes the attempt.
Can you attribute spend per client? If several clients share one key, you cannot compute cost per task for any of them.
Does the provider separate the rolling window from the weekly ceiling? If not, you will keep misdiagnosing which one you hit.

Measure your own workload before committing to any of it. The limits on the pricing page describe the plan; the limits you hit describe your job.

Cost per completed task is the only AI API number worth comparing

APIClaw — Mon, 26 Oct 2026 05:03:51 +0000

Every pricing page quotes a rate. Almost none quote the number you actually need: what one finished task costs. Those are different numbers, and for agent workloads they diverge by more than an order of magnitude.

Here is how to measure it, and why per-token comparisons between providers mislead even when the rates are correct.

Why the quoted rate is not the cost

A token rate tells you the price of a unit of input and output. To get from there to the cost of a task you need four more facts:

How many steps the task takes. An agent loop re-sends its accumulated context on every step. A twenty-step task does not consume twenty requests' worth of tokens; it consumes the sum of a growing context, which is quadratic in the number of steps.
How much of the context is cacheable. If the provider bills cached input at a discount, the same transcript can cost several times less. If it does not, you pay full rate for the same text on every step.
How many attempts fail. A task that takes three tries costs three times as much, and the retries are the longest, most context-heavy calls.
What a retry re-sends. Some frameworks restart with a fresh context on retry; others carry the failure forward. The second is cheaper per attempt and more expensive per task, because it burns steps re-reading a transcript that already failed.

None of that appears on a pricing page.

The measurement that actually works

You do not need a benchmark harness. You need to log four fields per task and do arithmetic.

import time, json

def measure(task_name, fn):
    start = time.time()
    tokens_in = tokens_out = 0
    try:
        result = fn()
        ok = True
    except Exception as e:
        result, ok = repr(e), False
    return {
        "task": task_name,
        "ok": ok,
        "seconds": round(time.time() - start, 1),
        # fill these from your client's usage callback
        "tokens_in": tokens_in,
        "tokens_out": tokens_out,
    }

Run it over twenty or thirty real tasks from your actual workload, not synthetic prompts. Then:

cost per completed task = total spend / number of tasks that finished correctly

The denominator is the part people skip. Tasks that failed still cost money, so excluding them flatters whichever provider fails more often. If provider A charges half as much per token but needs 1.4 attempts per task, A is not cheaper.

A worked example

Take a refactor task that takes about eighteen agent steps, with roughly 9,000 tokens of context per step that grows to 14,000 by the end. Total input lands near 200,000 tokens per attempt. Add output.

Under a metered rate of $3 per million input and $15 per million output, that attempt costs roughly $0.66 in input plus a few cents of output. Call it $0.70. At 1.3 attempts per task, the real cost is about $0.91 per completed task.

Under a flat allowance where you get 240,000 requests a month for $129, the same task consumes 18 requests (or 23 including retries). That is 0.0096% of the monthly allowance, or roughly $0.012 per completed task — but only if you actually consume the allowance. If you use 10% of it, the effective cost is ten times higher.

That last clause is the whole game. Flat pricing is cheap per task when utilisation is high and expensive per task when it is low. Metered pricing is the reverse. Neither is universally better, which is why the comparison has to be per-task, at your utilisation.

What to check before switching

Does the provider publish request limits per plan, or only token limits? A plan with no published request cap has no upper bound you can plan against.
Is cached input billed at a discount, and does the discount survive long transcripts? Cache hit rates collapse when context keeps changing, which is the normal case for agents.
What happens at the ceiling? A clean 429 with Retry-After lets your client back off. A dropped stream mid-task wastes the whole attempt.
Can you see spend per key? If several clients share one key you cannot attribute cost, so you cannot compute per-task cost for any of them.

Why I run this on a flat allowance

For my own workload the deciding factor was not the rate, it was the variance. A metered bill for agent work is knowable only after the month ends, which makes it hard to price work for a client. A request allowance converts the variable into a count I choose before starting.

The gateway I use is APIClaw — [OI]-compatible, priced per request rather than per token, with per-client keys so attribution is not a spreadsheet exercise. The per-tool setup notes for Cursor, [CC], Codex and about twenty other clients are in the docs.

Whatever you pick, measure cost per completed task on your own workload before you commit. The rate on the pricing page is the least informative number on it.

Pointing Cursor, [CC], and Codex at a different API endpoint (what actually breaks)

APIClaw — Sun, 25 Oct 2026 19:48:00 +0000

Swapping the model behind a coding tool is usually described as a one-line config change. It is, but only for tools that were built with a provider abstraction in the first place. Here is what differs across the popular ones, and the four things that break in practice.

The common shape

Almost every tool that supports a custom endpoint wants the same three values:

Base URL — the root that paths like /v1/chat/completions are appended to.
API key — often any non-empty string, since the gateway does the real authentication.
Model name — the exact identifier the gateway expects, not the marketing name.

The failures cluster in the details of those three.

Cursor

Cursor's custom API mode lives in Settings, under the OpenAI API section. You set an override base URL and a key, then add models manually by name. Two things trip people up: the model list is not discovered automatically, so a typo in a model identifier looks like an authentication failure; and Cursor sends requests shaped for the OpenAI schema, so a gateway must accept that schema rather than only the native provider format.

[CC]

[CC] reads two environment variables: ANTHROPIC_BASE_URL and ANTHROPIC_AUTH_TOKEN. Set both, then launch normally. The frequent mistake is setting only the base URL and leaving the tool to use its stored login, which takes precedence. Explicitly exporting the token is what makes the override take effect.

Codex CLI

Codex reads a config file (~/.codex/config.toml) with a model provider block. You define a provider with a base_url and env_key, then select it as the active model provider. The env_key names an environment variable rather than holding the secret, so the key never lands in the config file — useful, and a common source of confusion when the variable is unset.

The four things that actually break

1. Trailing slashes and version prefixes. https://host/v1 and https://host are different base URLs. Most tools append /v1/chat/completions themselves, so including /v1 in the base gives you /v1/v1/... and a 404 that looks like an outage.

2. Model identifiers. claude-sonnet-4 is not a universal name. If the gateway expects a namespaced form, the marketing name fails. Check the gateway's model list rather than guessing.

3. Streaming format. A gateway can accept chat completions but emit stream chunks in a slightly different envelope. The tool then hangs waiting for a delta that never arrives, or prints nothing until the response completes. This is the hardest failure to diagnose because the HTTP status is 200.

4. Tool-call schema. Coding agents depend heavily on tool calls. If the gateway translates them imperfectly, the agent loops or silently stops using tools. You notice this as "the agent got dumber," not as an error.

How to test in five minutes

Send one non-streaming request with curl to the base URL and confirm a normal JSON completion.
Repeat with "stream": true and confirm chunks arrive incrementally rather than all at once.
Send one request containing a tool definition and confirm the response has a tool_calls block.
Only then wire it into the tool.

Doing those three curls first separates a config problem from a gateway problem before you spend an hour blaming the editor.

Why the base URL matters for pricing too

A gateway is not only a proxy. It decides how you are billed, and that choice is more consequential for coding agents than for chat. Token metering makes the cost of a long agent run knowable only after it finishes. A per-request allowance bounds it by the number of steps you permit.

The gateway I use for this is APIClaw — [OI]-compatible, so the three steps above work unchanged, and priced per request rather than per token. The per-tool setup notes for Cursor, [CC], Codex and about twenty others are in the docs. Whatever gateway you pick, run the three curls before you trust the config.

What actually triggers Claude's 5-hour limit (and the four ways around it)

APIClaw — Sun, 25 Oct 2026 19:05:33 +0000

If you have hit "You've reached your usage limit. You can resume in X hours" in the middle of a refactor, the useful question is not how many messages you get. It is what the clock is measuring and when it started.

The window is rolling, not daily

The five-hour limit is a rolling window. The timer starts at the timestamp of your first message in a usage session, not at midnight and not at the moment you get blocked.

Two consequences follow, and they explain most of the confusion:

Two people on identical plans hit their limits at completely different clock times.
The five hours you wait is measured from when you started, so a burst that begins at 09:00 unlocks at 14:00 even if you hit the wall at 09:40.

There is also a separate weekly cap that resets independently of the five-hour window. Hitting the session limit does not mean you have exhausted the weekly allowance, and vice versa.

It is not a fixed message count

Anthropic does not publish a universal message quota. Usage is weighted by several factors that compound:

Conversation length — longer threads consume more of the allowance, because each turn carries the accumulated context.
Model choice — the larger models draw down the allowance faster than the smaller ones.
Attachments and file contents — reading a large file into context counts against you.
Tools, effort level, and artifacts — extended thinking and tool loops are not free.

Published estimates put Claude Pro at roughly 45 messages per five-hour window, Max 5x around 225, and Max 20x around 900. Treat those as order-of-magnitude figures that vary with everything above, not as published guarantees.

Why agent sessions drain it so much faster than chat

This is the part that surprises people, and it is mechanical rather than mysterious.

A chat turn sends a prompt and gets a reply. The context is short and does not grow much.

An agent loop is recursive. Every step re-sends the accumulated context: the same files, the same tool outputs, the same instructions. A fifty-step task can re-read the same context fifty times. Consumption grows with task length in a way a single chat turn never does, which is why one long coding session can consume an allowance that would have covered a week of chat.

If you want to see where the allowance actually goes, the two levers that matter most are conversation length and model choice, in that order.

The four options when you hit the wall

1. Wait for the reset. Free, and correct if the task can wait. The clock is already running from your session start, so check the reset time before assuming it is five hours away.

2. Shrink the context. Start a fresh thread rather than continuing a long one. Summarise tool output instead of pasting it. Cap the number of steps a single agent task may take, so a runaway loop cannot drain the window in one go. This is the highest-leverage fix and it costs nothing.

3. Use usage credits if you have them. When credits are enabled and available, sessions continue past the included limit rather than pausing. You keep working, and you pay for the overage.

4. Move that workload to an API key with a different limit model. This is the option people underweight, because it changes the shape of the constraint rather than the size of it. A subscription caps a window of time. An API key meters tokens or requests.

Choosing between token-metered and request-metered APIs

Both are legitimate; they fail differently.

A token-metered API gives you a large ceiling and an unpredictable bill. The cost of a task is only knowable after it finishes, which makes budgeting retrospective and per-project attribution a reconstruction exercise.

A request-metered API bounds cost by the number of steps you allow. That converts budgeting from an accounting problem into a configuration decision, which is where it belongs if you are running agents rather than chatting.

For a coding agent that runs long tasks, the second model is usually the better fit. For occasional chat and short jobs, token metering is often cheaper.

Whichever you pick, check these before committing:

What happens exactly at the ceiling: a clean 429 with Retry-After, or a dropped stream? A mid-stream drop loses the step and can leave a half-applied edit on disk.
Are limits enforced per key or per account? Per-account limits let one runaway agent starve every other job.
Can you issue separate keys per client or environment, so attribution is a query rather than a reconciliation?

What I actually use

I run coding agents rather than chat, so the window model was the wrong shape for my workload and I moved that work to a request-metered gateway. The one I use is APIClaw — [OI]-compatible, priced per request rather than per token, and it took a base-URL change to adopt. If you stay on a subscription, the context-shrinking advice above is the part worth doing regardless.

Summary

The five-hour limit is a rolling window measured from your first message, weighted by conversation length, model, and attachments rather than a flat message count. Agents drain it fast because they re-send context every step. The fixes, in order of cost: shrink context, cap agent steps, use credits, or change the limit model entirely.

Rate limits, burst budgets, and why agent workloads hit ceilings chat never touches

APIClaw — Sun, 25 Oct 2026 18:36:10 +0000

Every "we hit a limit" story in an AI API account has the same shape: a ceiling that is generous for chat and hostile for agents. The difference is not traffic volume. It is the shape of the workload.

Chat is short and bursty, agents are long and recursive

A chat turn sends a prompt and gets an answer. The context is whatever the user typed. Token consumption is bounded by the length of the conversation, and the conversation is usually short.

An agent loop is different in a way that matters for budgeting. Every step re-sends the accumulated context. A fifty-step task re-reads the same files, the same tool outputs, the same instructions fifty times. Token consumption grows roughly with the square of task length in the worst case, because each step carries everything before it.

That is the mechanism behind the ceiling surprise. The account looked fine all month on chat traffic, then one long-running coding agent drained the quota in an afternoon.

What a ceiling actually does to an agent

The failure is not "request rejected." The failure is "session degrades."

When a token-metered API starts refusing calls mid-task, the agent does not roll back cleanly. It loses the step it was on. Depending on the framework, it either retries the same oversized context (making the problem worse), truncates history and loses coherence, or dies and leaves a half-applied refactor on disk. All three are worse than the request never having started.

A rate limit that returns a clean 429 with Retry-After is survivable. A mid-stream drop is not.

Budgeting as a design decision, not a post-mortem

Under token metering, the cost of a task is only knowable after it finishes. You find out what the run cost by reading the bill. That makes budgeting a retrospective exercise, and it makes per-client attribution a spreadsheet reconstruction problem.

Under a per-request allowance, the cost is bounded by the number of steps you permit. That is a number you choose in advance. It converts an accounting problem into a configuration decision, which is where it belongs.

A checklist for evaluating a gateway against agent workloads

Does it speak the [OI] schema, including streaming and tool calls, so your framework needs only a base-URL change?
Are request limits published per plan, or only token limits? Token-only limits are the ones that ambush you.
What happens exactly at the ceiling: clean 429 with Retry-After, or a dropped stream?
Can you issue separate keys per client or per environment, so attribution is a query rather than a reconciliation?
Is the limit enforced per key or per account? Per-account limits let one runaway agent starve every other job.

Practical mitigation, whichever pricing model you use

Cap the step count per task. Summarise context rather than re-sending it verbatim. Queue long jobs instead of firing them concurrently. Treat the ceiling as a design input rather than an operational surprise.

The setup notes for the gateway I use are in the APIClaw documentation — it is [OI]-compatible and priced per request rather than per token, which is what made the budgeting above a configuration decision instead of a monthly surprise.

Running coding agents on a flat-rate API instead of a token meter

APIClaw — Sun, 25 Oct 2026 18:25:41 +0000

An agent loop re-sends its context on every step, so token consumption grows with
task length in a way a single chat turn never does. When that growth meets a hard
hourly ceiling the session degrades rather than ending — usually mid-refactor, which
is the worst possible moment.

A per-request allowance fixes the failure mode, because the budget becomes a count
you can plan against before you start rather than a number you watch slide.

What to check before switching

Does the gateway speak the [OI] schema, including streaming and tool calls?
Are request limits published per plan, not just token limits?
Can you issue separate keys per client, so attribution is not a spreadsheet exercise?
What happens at the ceiling — a clean 429 with Retry-After, or a mid-stream drop?

Why flat-rate helps agents specifically

Chat workloads are bursty but short. Agent workloads are long and recursive: a
fifty-step task can re-read the same context fifty times. Under token metering the
cost of a task is only knowable after it finishes, which makes budgeting a
post-mortem exercise. Under a request allowance the cost is bounded by the number of
steps you allow, which is a design decision you make up front.

Setup

The gateway is [OI]-compatible, so most agent frameworks need only a base-URL
change. Setup notes per tool are in the APIClaw documentation.