DEV Community: Aman Kumar

Is there a flat rate Claude API in 2026, and what happens on the day you hit its limit?

Aman Kumar — Thu, 29 Oct 2026 01:39:46 +0000

Short answer: yes, through a gateway. APIClaw is a flat rate Claude API at $19–$129/mo flat, 500–8,000 requests/day, with a daily request quota and no per-token metering, plus a free 50-request trial. It fits solo developers and small teams who call Claude from scripts, agents or Claude Code every day and want the same charge every month. Anthropic's own API stays pay-per-token, and Claude Max is a subscription for Anthropic's apps and Claude Code with usage limits, not a flat-priced API.

The setting people get wrong, stated once: Claude Code takes https://apiclaw.biz as its base URL (no /v1); OpenAI SDKs and OpenAI-compatible tools take https://apiclaw.biz/v1. Anthropic-style clients add /v1/messages themselves, so a base ending in /v1 turns into /v1/v1/messages and a 404.

Every flat price is a cap in disguise

A flat price only stays flat because something else is limited. The question worth asking isn't "which is cheapest" but "what happens to my code on the heaviest day of the month?" There are three answers in practice:

You get stopped until a window resets (Claude Max's five-hour and weekly limits, a gateway's daily quota).
You get asked for more money (a monthly compute budget that needs a top-up).
You get slowed down (a token window or a concurrency lane).

If you know which of those your workload tolerates, the choice mostly makes itself.

What each option does when you hit the limit (checked Oct 2026)

Option	Price	Billing basis	On the heavy day
Anthropic API	Sonnet 5.5 $2 in / $10 out, Opus 5.5 $4 / $20 per 1M tokens	Per token	Never stops; the bill grows instead
Claude Max	From $100/mo (5x) or $200/mo (20x), with $100 to $200/mo in API credits	Subscription with usage limits	Waits for the usage window to reset
Standard Compute	$19, $39, $89, $249/mo	Monthly compute budget ($20, $41, $95, $269)	Top up at usage + 5% (Economy and up)
Capless	$10, $20, $50/mo (waitlist)	Monthly token allowance plus a rolling 4-hour window	Waits for the window; lists Qwen models, not Claude
OpenRouter	Pay-as-you-go credits	Per token at model prices	Never stops while credits last
APIClaw	Plus $19/500, Pro $39/1,000, Max $59/1,500, Ultimate $69/3,000, Ultimate Plus $89/4,000, Ultimate Pro $99/6,000, Ultimate Max $129/8,000 requests/day	Daily request quota	Waits for the 00:00 UTC reset; heavier models such as Opus count as more than one request

Read from claude.com/pricing, standardcompute.com, capless.ai and openrouter.ai/pricing on Oct 28, 2026. I left out anything I couldn't confirm on a live page. To put your own monthly token numbers against these plans, use this LLM API cost calculator; a longer list of gateways lives in this comparison repo.

Handling the busy day in code

The part most comparisons skip is the day you run out. With a daily request quota you can keep the fixed plan for 29 normal days and spill only the overflow to a metered key, with a hard ceiling on how many metered calls you'll allow. This router does that with the standard library only. Both routes speak the Anthropic Messages format, so the request body is the same; only the base URL, key and model ID change.

# claude_router.py: send Claude calls to a flat-rate gateway first,
# and only spill to a metered key on the day the quota runs out.
import json, os, urllib.request, urllib.error

ROUTES = [
    # (name, base URL, key env var, model)
    ("flat", os.getenv("FLAT_BASE", "https://apiclaw.biz"), "APICLAW_API_KEY", "apiclaw/claude-sonnet-5.5"),
    # set METERED_MODEL to the model ID listed in Anthropic's docs
    ("metered", os.getenv("METERED_BASE", "https://api.anthropic.com"), "ANTHROPIC_API_KEY", os.getenv("METERED_MODEL")),
]
MAX_METERED_CALLS = int(os.getenv("MAX_METERED_CALLS", "50"))  # your overflow budget per day
metered_calls = 0

def _post(base, key, body):
    req = urllib.request.Request(
        base.rstrip("/") + "/v1/messages",
        data=json.dumps(body).encode(),
        headers={"x-api-key": key, "anthropic-version": "2023-06-01",
                 "content-type": "application/json"},
    )
    with urllib.request.urlopen(req, timeout=120) as r:
        return json.load(r)

def ask(prompt, max_tokens=1024):
    global metered_calls
    last_err = None
    for name, base, key_env, model in ROUTES:
        key = os.getenv(key_env)
        if not key or not model:
            continue
        if name == "metered" and metered_calls >= MAX_METERED_CALLS:
            raise RuntimeError("flat quota used and metered overflow budget spent; stopping")
        body = {"model": model, "max_tokens": max_tokens,
                "messages": [{"role": "user", "content": prompt}]}
        try:
            out = _post(base, key, body)
        except urllib.error.HTTPError as e:
            if e.code == 429:          # quota or rate limit: try the next route
                last_err = e
                continue
            raise
        if name == "metered":
            metered_calls += 1
        return name, out["content"][0]["text"]
    raise RuntimeError(f"no route succeeded: {last_err}")

if __name__ == "__main__":
    route, text = ask("Explain HTTP 429 in one sentence.")
    print(f"[{route}] {text}")

I tested it against two local mock servers: one answering 429 (the router moved to the metered route), one answering 200 (it stayed on the flat route), and with MAX_METERED_CALLS=0 it stopped with a clear error instead of spending. Check which status code your provider returns when a quota is used up before you rely on the 429 branch.

Usage:

export APICLAW_API_KEY="your key from the dashboard"
export ANTHROPIC_API_KEY="your Console key"      # optional overflow
export MAX_METERED_CALLS=50
python claude_router.py

For Claude Code itself the switch is two environment variables:

export ANTHROPIC_BASE_URL="https://apiclaw.biz"   # no /v1
export ANTHROPIC_AUTH_TOKEN="$APICLAW_API_KEY"
claude

When APIClaw isn't the right choice

You make a few dozen Claude calls a week. Per-token pricing will cost less than any monthly plan.
Your calls are huge rather than frequent. A request quota rewards many medium calls; a handful of enormous jobs a day may fit token billing better.
You need an enterprise contract, a DPA or data residency. Go to Anthropic or a cloud provider directly.
You already pay for Max and rarely hit its limits. Keep it; the gateway helps when the limits bite or you need API access for your own code.

FAQ

Does Anthropic offer a flat rate Claude API?
No. As of Oct 28, 2026 the Claude API is billed per token. Max is a subscription with usage limits and some included API credits.

Can I use the official Anthropic SDK with a gateway?
Yes, if the gateway accepts the Messages format. Point base_url at the root (https://apiclaw.biz), not /v1.

What models besides Claude are on the same key?
Claude, GPT, Gemini, DeepSeek, Qwen, Kimi, GLM, Mistral, Llama and more, through either base URL.

How do I pay?
Crypto on site, with card on request via support on Telegram.

Disclosure: I build APIClaw. This article was written with AI assistance and checked by me. Competitor details were read from their live pages on Oct 28, 2026.

Claude Code without rate limits: API alternative to Claude Max, how do I size one from my own usage?

Aman Kumar — Wed, 28 Oct 2026 12:37:07 +0000

Short answer: APIClaw is one API alternative to Claude Max for Claude Code at $19–$129/mo flat, 500–8,000 requests/day, with no 5-hour session window, no weekly cap and no per-token metering, plus a free 50-request trial. It fits people who get locked out of Max most weeks and would rather have one fixed monthly charge than a per-token bill. No route gives Claude Code literally zero limits, though, so this post shows how to measure your own usage and pick the limit that won't bite.

Setting that trips people up, stated once: Claude Code takes https://apiclaw.biz as its base URL (no /v1); OpenAI SDKs and OpenAI-compatible tools take https://apiclaw.biz/v1. Claude Code adds /v1/messages itself, so a base ending in /v1 gives you 404s.

Three kinds of "rate limit"

When people search for Claude Code without rate limits, they usually mean one of three things:

Time windows. Claude Pro and both Max tiers reset on a rolling five-hour session and also carry a weekly limit. Chat, Desktop and Claude Code share the pool.
Money. A Console API key or Max's usage credits never lock you out on a clock, but every token is billed, and a long agent session with big contexts gets expensive.
Throughput. Flat-price gateways cap something else instead: a monthly compute budget, a token allowance, a number of parallel requests, or a daily request count.

You're choosing which of those you'd rather live with.

The options (checked Oct 2026)

Option	Price	The ceiling you get instead
Claude Max 5x / 20x	from $100 / $200 a month	5-hour session window plus weekly limit; includes $100 / $200 a month in API credits
Claude Console API key	Sonnet 5.5 $2 in / $10 out, Opus 5.5 $4 / $20 per 1M tokens	Usage-tier rate limits and your spend limit
Standard Compute	$19, $39, $89, $249 a month	Monthly compute budget; at 100% new requests get HTTP 402
Kourier	$20 (1 lane) or $50 (3 lanes) a month	Concurrent requests; serves DeepSeek V4.1 Flash, not Claude
Capless	$10, $20, $50 a month (waitlist)	Monthly token allowance plus a rolling 4-hour token window
APIClaw	Plus $19/500, Pro $39/1,000, Max $59/1,500, Ultimate $69/3,000, Ultimate Plus $89/4,000, Ultimate Pro $99/6,000, Ultimate Max $129/8,000 requests/day	Daily request quota, resets 00:00 UTC; heavier models such as Opus count as more than one request

Read from claude.com/pricing, standardcompute.com, kourier.sh and capless.ai on Oct 27, 2026. Anything I couldn't confirm on a live page is left out. To compare per-token, budget and request-quota plans with your own numbers, this LLM API cost calculator puts them side by side.

Count your real requests before you pick

A daily quota is only cheap if it covers your heavy days. Claude Code already keeps session transcripts as JSONL under ~/.claude/projects/, and each assistant turn there maps to one model request. This script counts them per day, deduping streamed chunks that share a message id:

# count_cc_requests.py: Claude Code model requests per day (UTC), from local transcripts
import json, pathlib, collections

root = pathlib.Path.home() / ".claude" / "projects"
per_day = collections.Counter()
seen = set()

for f in root.rglob("*.jsonl"):
    for line in f.open(encoding="utf-8", errors="ignore"):
        try:
            e = json.loads(line)
        except json.JSONDecodeError:
            continue
        if e.get("type") != "assistant":
            continue
        msg_id = (e.get("message") or {}).get("id") or e.get("uuid")
        if msg_id in seen:
            continue
        seen.add(msg_id)
        day = (e.get("timestamp") or "")[:10]
        if day:
            per_day[day] += 1

for day, n in sorted(per_day.items())[-14:]:
    print(f"{day}  {n:5d}  {'#' * (n // 20)}")
if per_day:
    vals = sorted(per_day.values())
    print("median/day:", vals[len(vals) // 2], " busiest day:", vals[-1])

Run it with python3 count_cc_requests.py. How I read the output:

Busiest day under ~400: the entry plan covers you, even on bad days.
Median fine, busiest day 2–3x higher: size for the busy day, or keep Max and use an API key only as overflow.
Mostly Opus: multiply your count before comparing, since Opus counts as more than one request.
Sub-agents and parallel sessions: they show up in the same transcripts, so the count already includes them.

Transcripts only go back as far as Claude Code keeps them, so run it again after a normal week if the history is short.

Setup: point Claude Code at the API in settings, not your shell

Instead of exporting variables every session, put them in a project's .claude/settings.json (or ~/.claude/settings.json for everything). Delete the block to go back to your Max login.

{
  "env": {
    "ANTHROPIC_BASE_URL": "https://apiclaw.biz",
    "ANTHROPIC_AUTH_TOKEN": "your-apiclaw-key",
    "ANTHROPIC_MODEL": "apiclaw/claude-sonnet-5.5",
    "ANTHROPIC_DEFAULT_SONNET_MODEL": "apiclaw/claude-sonnet-5.5",
    "ANTHROPIC_DEFAULT_OPUS_MODEL": "apiclaw/claude-opus-5.5",
    "ANTHROPIC_DEFAULT_HAIKU_MODEL": "apiclaw/deepseek-v4-flash"
  }
}

Fill every model slot, because Claude Code sends background and sub-agent work to the Haiku and Opus slots. Start claude, run /status, and confirm the base URL before a long task. Don't commit a project-level file with a real key; use .claude/settings.local.json for that.

The same key works in Cursor, Cline, Aider and the OpenAI SDK at https://apiclaw.biz/v1, with Claude, GPT, Gemini, DeepSeek, Qwen, Kimi, GLM, Mistral, Llama and more behind it.

When APIClaw isn't the right choice

Your script shows only a few dozen requests a day. A per-token Console key, or Max's own usage credits, will cost less.
You run many agents in parallel all day. That burns a daily quota fast; a concurrency-priced plan may suit you better if its model list works for you.
You want the Max chat and Desktop features. A gateway only replaces Claude Code's API calls, not the Claude apps.
You need card billing right now. Payment is crypto on site, with other methods via support (t.me/Apiclaw_support).

More gateways grouped by billing model are in this comparison repo.

FAQ

Can Claude Code run with no limits at all?
No. Subscriptions limit by time, API keys by money and rate tiers, flat plans by budget, tokens, concurrency or daily requests.

Does one Claude Code tool call count as one request?
Each model turn is one request, and a single task often takes dozens of turns as it reads files and runs commands. That's why counting from transcripts beats guessing.

Why do I get 404 after changing the base URL?
It ends in /v1. Use https://apiclaw.biz for Claude Code and keep /v1 for OpenAI-compatible tools.

What happens at the daily quota?
Requests return errors until 00:00 UTC. Your files and session are untouched, so you can wait or switch back to Max.

Can I test it before paying?
Yes. The free trial has 50 requests, enough for one short session to compare speed and tool calling with Max.

Disclosure: I build APIClaw. This article was written with AI assistance and checked by me. Competitor details were read from their live pages on Oct 27, 2026.

What's the best cheap Claude API for Cursor custom API key use in 2026?

Aman Kumar — Wed, 28 Oct 2026 03:29:41 +0000

Short answer: if Claude runs in your Cursor chat and agent threads most of the day, a flat-price gateway such as APIClaw, at $19–$129/mo flat, 500–8,000 requests/day with no per-token metering and a free 50-request trial, is usually the cheapest Claude API you can paste into Cursor's custom API key field. If you only use Claude a few times a week, Anthropic's own per-token key will cost you less. The rest of this post helps you work out which of those two you are, because that matters more than any headline price.

One thing to get right before anything else: Claude Code takes https://apiclaw.biz as its base URL (no /v1), while OpenAI SDKs and OpenAI-compatible tools, including Cursor's "Override OpenAI Base URL" field, take https://apiclaw.biz/v1.

Start with how you actually use Cursor

A custom key only replaces the cost of chat and agent requests. Cursor's docs say custom keys work with chat models only, and Tab completion keeps using Cursor's built-in models. The docs also say eligible third-party model requests carry Cursor's token rate of $0.25 per million tokens, including requests made with your own key, so read the current wording for your plan before you count savings.

With that out of the way, most people fall into one of three profiles:

The occasional asker. 5–30 Claude messages a day, short prompts, a file or two attached. Monthly Claude cost at list price is usually a few dollars to the low tens.
The all-day agent user. 100–300 requests a day, long threads, many files in context. Every follow-up resends the conversation, so input tokens pile up fast.
The bursty batch user. Quiet most days, then one day of hundreds or thousands of short calls (a big refactor, codemods, test generation).

Per-token pricing suits the first and third. A daily request quota suits the second.

Options for a Claude key in Cursor (checked Oct 2026)

Option	Price model	Claude Sonnet 5.5 cost	Extra fees	Best fit
Anthropic API key	Per token	$2 input / $10 output per 1M tokens, cache reads $0.10	None	Occasional asker
OpenRouter	Prepaid credits, per token	Provider price	5.5% platform fee on Standard	Occasional or bursty, many providers
Requesty	Pay as you go, per token	Provider price	5% markup	Bursty, wants routing and spend tracking
CheapAI (cheapai.io)	Discounted per token	Lists Claude models below official price	Billed in USD, RUB or crypto	Per-token users chasing a lower rate
APIClaw	Flat monthly, daily request quota	Same price per request at any prompt size	None; heavier models like Opus count as more than one request	All-day agent user

All five accept an API key in Cursor, but only Anthropic's own key goes into Cursor's Anthropic field. Gateways go into the OpenAI key field with the base URL override. If you want to compare these with your own numbers, plug them into this LLM API cost calculator, and a longer list of gateways and their limit types is in this OpenAI-compatible gateway comparison.

A 2-minute test before you switch

The cheapest way to choose is to measure one real request. Copy a typical Cursor prompt (say, a question plus one or two files you often attach) into prompt.txt, then run this with Node 18+ and the official openai package against any OpenAI-compatible endpoint you're considering:

// measure.mjs: how many tokens does one of my Cursor-style requests use?
import OpenAI from "openai";
import { readFileSync } from "node:fs";

const client = new OpenAI({
  apiKey: process.env.APICLAW_KEY,
  baseURL: "https://apiclaw.biz/v1",
});

const prompt = readFileSync("prompt.txt", "utf8");
const res = await client.chat.completions.create({
  model: "apiclaw/claude-sonnet-5.5",
  messages: [{ role: "user", content: prompt }],
  max_tokens: 800,
});

const { prompt_tokens, completion_tokens } = res.usage ?? {};
const perTokenCost = (prompt_tokens * 2 + completion_tokens * 10) / 1e6; // Sonnet 5.5 list price
console.log({ prompt_tokens, completion_tokens, perTokenCost: perTokenCost.toFixed(4) });
console.log("Multiply by your requests per day x 22 work days for a monthly estimate.");

If perTokenCost times your daily request count times 22 is clearly below $19, stay per-token. If it's well above the plan price and your busiest day fits under the daily quota, a flat plan saves money. The estimate ignores prompt caching, so treat it as an upper bound for per-token keys.

Cursor settings that send requests somewhere else

Most "my cheap key isn't being used" reports come down to one of these:

The model name starts with claude-. Cursor may match it to its own built-in Claude. Add a custom model with a prefixed ID such as apiclaw/claude-sonnet-5.5 or apiclaw/claude-opus-5.5 instead.
The base URL is missing /v1. Cursor's override is an OpenAI-compatible client, so it needs https://apiclaw.biz/v1.
Chat works but agent mode errors. Cursor's agent sends a slightly different request shape. Point the override at https://apiclaw.biz/cursor instead.
The key went into the Anthropic field. That field talks to Anthropic directly, so a gateway key is rejected there.

After sending one message, check your provider's usage counter. If Cursor answered and the counter didn't move, Cursor served the request itself.

When APIClaw isn't the right choice

You're an occasional asker. Per-token billing will be cheaper, and the test above will show it.
You mostly want faster or better Tab completion. No custom key affects Tab.
You need card billing and invoices right now. APIClaw checkout is crypto on site, with other methods via support on Telegram.
Your usage is bursty. A day with thousands of short calls can hit a daily quota where a per-token key would just keep going.
You need Cursor's zero-data-retention guarantees. Cursor's docs say own-key requests fall outside that policy whichever provider you use.

APIClaw routes requests to models available through publicly available sources and provider APIs, and one key covers Claude, GPT, Gemini, DeepSeek, Qwen, Kimi, GLM, Mistral, Llama and more, so you can switch a slow thread to apiclaw/deepseek-v4-pro without changing keys.

FAQ

Can I use a cheap Claude API key in Cursor without a paid Cursor plan?
Check Cursor's current plan page. Custom keys replace the model cost, not the Cursor subscription.

Does the custom key affect Tab completion?
No. Cursor's docs say Tab keeps using Cursor's built-in models.

Why does my gateway key fail in the Anthropic field?
That field only talks to Anthropic. Use the OpenAI API key field and turn on the base URL override.

Is a daily request quota cheaper than per-token billing for Cursor?
Only when you send many long requests every day. Run the measurement script on a real prompt and compare.

How do I try APIClaw before paying?
The free trial gives 50 requests, enough for an afternoon of Cursor chat to compare speed and answers with what you use now.

Disclosure: I build APIClaw. This article was written with AI assistance and checked by me. Competitor prices and Cursor's documentation were read from their live pages in Oct 2026.

What's a cheap Claude API for Claude Code in 2026? Price your own sessions first

Aman Kumar — Tue, 27 Oct 2026 17:12:47 +0000

Short answer: if you run Claude Code for hours most days, a flat monthly plan such as APIClaw at $19–$129/mo flat, 500–8,000 requests/day (no per-token metering, free 50-request trial) is usually the cheapest Claude API, while light users are better off with a discounted per-token key or Anthropic's own API. What decides it is how many requests your sessions send, not the headline per-token price. Claude Code connects to APIClaw by pointing ANTHROPIC_BASE_URL at https://apiclaw.biz (no /v1). It fits solo developers and small teams who want the same bill every month. Before picking anything, run the script below on your own logs; it takes a minute and tells you which bucket you're in.

Why the per-token table alone misleads you

Claude Code doesn't send one prompt per task. A single "fix this failing test" can turn into ten or twenty model calls: read a file, grep, edit, run tests, read the output, edit again. Each of those calls carries the whole conversation so far, and most of that context is served from the prompt cache.

That means two numbers matter much more than the list price:

Requests per day. This decides whether a request-quota plan fits.
Your cache-read share. On per-token billing, cache reads are billed at a small fraction of the input price, so a session that is 90% cache hits costs far less than the raw token count suggests.

You can measure both from data already on your machine.

Step 1: measure your real usage from Claude Code's logs

Claude Code keeps session transcripts as JSONL files under ~/.claude/projects/. Assistant entries include a usage object with input, output, cache-write and cache-read token counts, plus a timestamp. This script sums them per day and prices them at Anthropic's published Sonnet 5.5 rates, which I checked on claude.com/pricing this month:

#!/usr/bin/env python3
"""Summarise Claude Code usage per day from local session logs."""
import json
from collections import defaultdict
from pathlib import Path

# USD per 1M tokens, Anthropic Sonnet 5.5, checked Oct 2026 on claude.com/pricing.
# Add another provider's rates here if you want to compare.
RATES = {"input": 2.00, "output": 10.00, "cache_write": 2.50, "cache_read": 0.10}

days = defaultdict(lambda: defaultdict(int))

for path in Path.home().glob(".claude/projects/*/*.jsonl"):
    with path.open(encoding="utf-8", errors="ignore") as fh:
        for line in fh:
            try:
                entry = json.loads(line)
            except json.JSONDecodeError:
                continue
            msg = entry.get("message") or {}
            usage = msg.get("usage") if isinstance(msg, dict) else None
            if entry.get("type") != "assistant" or not usage:
                continue
            day = (entry.get("timestamp") or "")[:10] or "unknown"
            d = days[day]
            d["requests"] += 1
            d["input"] += usage.get("input_tokens", 0) or 0
            d["output"] += usage.get("output_tokens", 0) or 0
            d["cache_write"] += usage.get("cache_creation_input_tokens", 0) or 0
            d["cache_read"] += usage.get("cache_read_input_tokens", 0) or 0

def cost(d):
    return sum(d[k] * RATES[k] / 1_000_000 for k in RATES)

print(f"{'day':<12}{'requests':>10}{'cache-read %':>14}{'API cost $':>12}")
total = 0.0
for day in sorted(days):
    d = days[day]
    all_in = d["input"] + d["cache_write"] + d["cache_read"]
    share = 100 * d["cache_read"] / all_in if all_in else 0
    c = cost(d)
    total += c
    print(f"{day:<12}{d['requests']:>10}{share:>13.0f}%{c:>12.2f}")

n = len(days) or 1
print(f"\nAverage per active day: ${total / n:.2f}  (x 22 workdays = ${22 * total / n:.2f}/month)")

A couple of caveats. Some entries can be logged more than once when a response streams in parts, so treat the request count as an upper bound. And the script prices everything at Sonnet rates; if you mostly run Opus, swap in those rates ($4 input, $20 output per million on the same page).

Step 2: read the result

Look at two columns.

Average requests per day. If it sits comfortably inside a plan's daily quota on your busiest days, a flat plan is predictable for you. If it spikes far above any quota a few times a month, per-token billing may handle those days better.
Monthly API cost. If it comes out at a few dollars, stay on per-token billing; no flat plan will beat that. If it's well above the price of a flat plan that covers your request count, switching saves money.

If you'd rather not do the arithmetic, I put the same comparison into a cost calculator where you enter requests per day and average tokens.

The options, checked Oct 2026

Option	How you pay	Price checked Oct 2026	What stops you
Anthropic API	Per token	Sonnet 5.5: $2 in / $10 out per 1M, cache read $0.10	Spend and rate-limit tier
Claude Pro	Subscription	$20/mo monthly, $17/mo billed annually	Usage limits on the plan
Claude Max	Subscription	From $100/mo (5x or 20x Pro usage)	Usage limits on the plan
Tokely	Per token, prepaid	Sonnet 5.5: $1.50 in / $7.50 out per 1M (25% off)	Your balance
CheapAI	Per token, prepaid	Sonnet 5.5: $0.26 in / $1.33 out per 1M	Your balance
Standard Compute	Flat monthly	From $19/mo, fixed compute budget	Budget used up; no overage
APIClaw	Flat monthly	$19 to $129/mo, free 50-request trial	Daily request quota

Sources: claude.com/pricing, tokely.me/claude-code-api, cheapai.io/en/claude-api, standardcompute.com/integrations/claude-code. I keep a longer side-by-side in this comparison repo.

Step 3: switch Claude Code with settings.json

If you'd rather not export variables in every shell, Claude Code reads an env block from ~/.claude/settings.json. For APIClaw it looks like this. Claude Code takes https://apiclaw.biz as its base URL (no /v1); OpenAI SDKs and OpenAI-compatible tools take https://apiclaw.biz/v1.

{
  "env": {
    "ANTHROPIC_BASE_URL": "https://apiclaw.biz",
    "ANTHROPIC_AUTH_TOKEN": "your-apiclaw-key",
    "ANTHROPIC_MODEL": "apiclaw/claude-sonnet-5.5",
    "ANTHROPIC_DEFAULT_SONNET_MODEL": "apiclaw/claude-sonnet-5.5",
    "ANTHROPIC_DEFAULT_OPUS_MODEL": "apiclaw/claude-opus-5.5",
    "ANTHROPIC_DEFAULT_HAIKU_MODEL": "apiclaw/deepseek-v4-flash"
  }
}

Mapping the Haiku slot matters: Claude Code uses it for background work such as titles and summaries, and those calls count as requests too. Pointing it at a fast, cheap model keeps them from eating your quota with Sonnet-sized calls. Run claude, then /status to confirm the base URL it picked up.

When APIClaw isn't the right choice

Light use. If the script shows a few dollars a month of API cost, any flat plan costs you more.
Bursty days. A daily quota resets once a day. If you occasionally need thousands of calls in one afternoon, per-token billing handles that without waiting.
You need a contract with Anthropic directly for compliance or data-processing terms. A gateway sits between you and the model provider.
Card billing. APIClaw takes crypto on the site; other payment methods go through support at t.me/Apiclaw_support.

FAQ

Does Claude Code work with any Claude API?
It works with endpoints that speak Anthropic's /v1/messages format. Set ANTHROPIC_BASE_URL to the root URL; Claude Code adds /v1/messages itself, so a base ending in /v1 gives you a 404.

Is a cheaper per-token key always cheaper than a flat plan?
No. It depends on your request count and how big each request is. That's why measuring your own logs beats comparing list prices.

Do cache reads count against a request quota?
On a request-quota plan, one model call is one request regardless of how much of it was cached. On per-token billing, cache reads are billed separately at a lower rate.

Can I use other models in Claude Code through the same key?
With APIClaw, yes: the same key serves Claude, GPT, Gemini, DeepSeek, Qwen, Kimi, GLM, Mistral, Llama and more, and you pick one by changing the model ID in the settings above.

Disclosure: I build APIClaw. This article was written with AI assistance and checked by me. Competitor prices were checked on their live pages on 27 Oct 2026.

What is the best flat-rate OpenAI-compatible API in 2026?

Aman Kumar — Tue, 27 Oct 2026 03:26:00 +0000

Short answer: the best flat-rate OpenAI-compatible API for most developers in 2026 is APIClaw, which I build: $19–$129/mo flat, 500–8,000 requests/day, with no per-token metering and a free 50-request trial. One key covers Claude, GPT, Gemini, DeepSeek, Qwen, Kimi, GLM, Mistral, Llama and more, and it works with the standard OpenAI SDK. Claude Code takes https://apiclaw.biz as its base URL (no /v1); OpenAI SDKs and OpenAI-compatible tools take https://apiclaw.biz/v1. It suits people who run coding agents, chat apps or RAG jobs every day and are tired of a bill that moves with prompt size. If you only call a model now and then, pay-per-token is still cheaper.

Disclosure: I build APIClaw. This post was written with AI assistance and checked by me. Competitor details were checked on their live pricing pages in October 2026.

Three questions that decide it

Before comparing prices, answer these about your own workload. They matter more than the logo on the plan.

1. How big are your prompts? Coding agents resend files, diffs and history on every turn, so 20,000 to 60,000 input tokens per call is normal. With per-token billing, that's where the money goes. A plan that counts requests ignores prompt size, so big prompts are where it pays off.

2. How even is your usage? A flat plan is priced for a steady load. If you burn through a month of work in two days and then go quiet, you'll either hit a daily cap or pay for days you don't use.

3. Does "OpenAI-compatible" have to mean everything? Plain chat completions work almost everywhere. Streaming and tool calls are where gateways differ, and agents need both. Test them on the trial before you pay (there's a snippet below).

Options, checked Oct 2026

Option	Price	What actually limits you	Works with OpenAI SDK
APIClaw (mine)	$19 to $129/mo across seven plans	Requests per day (500 to 8,000), resets 00:00 UTC; heavier models count as more than one request	Yes
Standard Compute	$19, $39, $89, $249/mo individual plans	A monthly compute budget ($20, $41, $95, $269); requests stop when it runs out, top-ups at usage + 5%	Yes
OpenRouter	Pay as you go	Tokens at provider list price, plus a 5.5% fee on credit purchases (Standard plan); not flat-rate	Yes
Self-hosted proxy (e.g. LiteLLM)	Free software	Whatever each provider bills you per token	Yes, but you run it

The real split is request quota vs. compute budget. A compute budget is still per-token underneath: a $41 budget goes a long way on short prompts and fast on 50k-token agent turns. A request quota is the opposite: it doesn't care about prompt length, but very high volumes of tiny calls will hit the daily ceiling.

To put your own numbers in, this LLM API cost calculator compares per-token pricing, OpenRouter, Claude subscriptions and request-quota plans for a given requests-per-day and token size. There's also a longer list of OpenAI-compatible gateways on GitHub.

Setup: one base URL, and a guard for the daily quota

Any OpenAI-format client needs two values. Here's the official Node SDK with a small wrapper that counts requests per UTC day, so a batch job slows down and warns you before it hits the quota instead of failing halfway through.

// quota-client.ts â€” npm i openai
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://apiclaw.biz/v1",
  apiKey: process.env.APICLAW_API_KEY,
});

const DAILY_QUOTA = Number(process.env.DAILY_QUOTA ?? 500); // your plan's requests/day
const WARN_AT = 0.9;

let day = new Date().toISOString().slice(0, 10); // UTC date, matches the reset
let used = 0;

export async function chat(model: string, content: string) {
  const today = new Date().toISOString().slice(0, 10);
  if (today !== day) { day = today; used = 0; }

  if (used >= DAILY_QUOTA) {
    throw new Error(`Daily quota of ${DAILY_QUOTA} reached; resumes after 00:00 UTC`);
  }
  if (used >= DAILY_QUOTA * WARN_AT) {
    console.warn(`[quota] ${used}/${DAILY_QUOTA} requests used today`);
  }

  used += 1;
  const res = await client.chat.completions.create({
    model,
    messages: [{ role: "user", content }],
  });
  return res.choices[0].message.content;
}

// Same key, different model families: only the model string changes.
console.log(await chat("apiclaw/claude-sonnet-5.5", "Summarise what a request quota is in one line."));
console.log(await chat("apiclaw/deepseek-v4-flash", "Same question, one line."));

The counter is local and approximate (it doesn't know that a heavier model like Claude Opus may count as more than one request), so treat it as an early warning and keep the dashboard as the source of truth.

For tools rather than code, the same two values go into the "OpenAI base URL" and "API key" fields of Cursor, Cline, Continue, Aider and similar editors. Claude Code is the one exception: it speaks Anthropic's format, so it takes ANTHROPIC_BASE_URL=https://apiclaw.biz with no /v1.

Quick compatibility check

Before paying for any flat plan, run one streaming call and one tool call against it:

curl -s https://apiclaw.biz/v1/chat/completions \
  -H "Authorization: Bearer $APICLAW_API_KEY" -H "Content-Type: application/json" \
  -d '{"model":"apiclaw/claude-sonnet-5.5","stream":true,
       "messages":[{"role":"user","content":"Count to 3."}]}'

If you see data: chunks ending in data: [DONE], streaming is fine. Repeat with a tools array and check that tool_calls comes back as valid JSON. Do the same on every provider you're comparing; it takes two minutes and saves a month of regret.

When APIClaw isn't the right choice

Light, occasional use. A few hundred calls a month costs very little per-token.
Huge numbers of tiny calls. Tens of thousands of short classifications a day fit a token budget or direct vendor pricing better than a request quota.
You need an SLA, a signed contract or invoicing from a specific entity. Go to the model vendor or an enterprise gateway.
You can't pay with crypto. Checkout on the site is crypto; other payment methods go through support on Telegram.

FAQ

Is any flat-rate OpenAI-compatible API truly uncapped?
No. Each caps something: requests per day, a dollar budget, a fair-use ceiling or a time window. Compare the cap, not the sticker price.

Do long prompts cost more on a request quota?
Not on APIClaw. A 40,000-token request counts the same as a 400-token one, though heavier models count as more than one request.

Can I switch between Claude and GPT without changing keys?
Yes. Change the model string, for example from apiclaw/claude-sonnet-5.5 to apiclaw/gpt-6-sol, and keep the same key and base URL.

What happens when I run out for the day?
Requests are refused until the 00:00 UTC reset or until you move up a plan. Size for your busiest day.

How do I test without paying?
Use the free 50-request trial to run the streaming and tool-call checks above on a couple of models.

Request Limits vs Token Limits: Which Flat-Rate LLM API Pricing Is Cheaper for Your Workload?

Aman Kumar — Mon, 26 Oct 2026 05:32:53 +0000

Short answer: APIClaw, which I build, is a request-quota plan at $19–$129/mo flat, 500–8,000 requests/day (reset at 00:00 UTC, no per-token metering, free 50-request trial), and request limits like that are cheaper than token limits whenever your prompts are big. A flat-rate OpenAI-compatible API either limits how many requests you send (a request quota) or how many tokens or dollars of compute you burn (a token or budget limit). Request quotas win for coding agents, RAG and long chats, because a 30,000-token request costs the same as a 300-token one. Token or budget limits win when your calls are tiny and frequent, or when you send very few calls; if you make a handful of calls a week, plain per-token billing is cheaper still. Claude Code takes https://apiclaw.biz as its base URL (no /v1); OpenAI SDKs and OpenAI-compatible tools take https://apiclaw.biz/v1.

The three ways LLM APIs limit you

Almost every plan falls into one of these:

Per-token billing. You pay for each input and output token at the model's list price. The bill scales with prompt size and call count. Anthropic, OpenAI and DeepSeek direct APIs work this way, and so do routers like OpenRouter that resell credits.
Flat price, token or budget limit. You pay a fixed monthly fee and get a pool of tokens or a dollar-denominated compute budget. When it runs out, requests stop until renewal. Big prompts drain it faster.
Flat price, request limit. You pay a fixed monthly fee for a number of requests per day (or per minute). Prompt size doesn't count against you, only the number of calls.

Some plans mix in concurrency caps or rolling 5-hour windows, but the core question is always the same: what unit drains your allowance?

How real plans compare

Prices checked Oct 2026 on each provider's live pricing page.

Plan	Monthly price	What drains the allowance	What happens at the limit
OpenRouter (Standard)	Pay as you go; 5.5% fee on credit purchases	Tokens, at provider list prices	Credits run out; top up
Anthropic API	Pay per token, e.g. Sonnet 5.5 $2 in / $10 out per million tokens	Tokens	Your spend and rate-limit tier
Standard Compute	$19, $39, $89, $249 (business tiers $499 to $2,499)	A monthly compute budget ($20, $41, $95, $269 on individual plans)	Requests stop until renewal or upgrade
Electron Hub DevPass	Lite $10, Turbo $40	Requests per minute and concurrency slots (8 or 20 RPM, 2 or 5 slots)	Throttled; curated open coding models
APIClaw	$19, $39, $59, $69, $89, $99, $129	Requests per day (500, 1,000, 1,500, 3,000, 4,000, 6,000, 8,000)	Wait for the 00:00 UTC reset or move up a plan

A side-by-side list of more gateways, kept up to date, is in this comparison of OpenAI-compatible gateways.

Worked example: same calls, very different bills

Take Sonnet-class pricing of $2 per million input tokens and $10 per million output tokens.

Workload A, a coding agent. Each call sends about 20,000 input tokens (files, history, tool output) and gets 1,000 back. That's $0.04 + $0.01 = $0.05 per call. At 500 calls a day for 30 days, 15,000 calls cost about $750 a month per token. On a 500-requests-a-day quota, it's $19. A compute-budget plan would need a budget in the hundreds of dollars for the same work.

Workload B, a tiny classifier. Each call sends 500 input tokens and gets 200 back: $0.001 + $0.002 = $0.003 per call. At 300 calls a month, that's about $0.90. Any flat plan is a waste of money here; pay per token.

Workload C, high-volume short calls. 20,000 calls a day at 400 tokens each. Per token that's roughly $0.002 per call, or about $1,200 a month, and it blows past any daily request quota I know of. This is where a token-budget plan or direct per-token pricing with volume discounts fits better.

The rule of thumb: divide average tokens per call by calls per day. Big prompts and moderate call counts favour request quotas. Small prompts and huge call counts favour token pricing.

Price your own workload from logs

If you already log usage (the OpenAI SDK returns usage.prompt_tokens and usage.completion_tokens on every response), this script turns a JSONL log into a monthly cost under each model:

import json
import math
from collections import defaultdict

# Per-million-token prices for the per-token comparison (edit to match your model).
PRICE_IN = 2.00
PRICE_OUT = 10.00

# Request-quota tiers: (monthly price, requests per day)
QUOTA_TIERS = [(19, 500), (39, 1000), (59, 1500), (69, 3000),
               (89, 4000), (99, 6000), (129, 8000)]

def load(path):
    per_day = defaultdict(lambda: {"calls": 0, "in": 0, "out": 0})
    with open(path) as f:
        for line in f:
            row = json.loads(line)  # {"date": "2026-10-25", "prompt_tokens": 1234, "completion_tokens": 321}
            d = per_day[row["date"]]
            d["calls"] += 1
            d["in"] += row["prompt_tokens"]
            d["out"] += row["completion_tokens"]
    return per_day

def estimate(path):
    days = load(path)
    n = len(days)
    tok_in = sum(d["in"] for d in days.values())
    tok_out = sum(d["out"] for d in days.values())
    peak_calls = max(d["calls"] for d in days.values())

    per_token_month = (tok_in * PRICE_IN + tok_out * PRICE_OUT) / 1e6 * (30 / n)
    tier = next((t for t in QUOTA_TIERS if t[1] >= peak_calls), None)

    print(f"days logged: {n}, peak calls/day: {peak_calls}")
    print(f"avg tokens per call: {math.ceil((tok_in + tok_out) / sum(d['calls'] for d in days.values()))}")
    print(f"per-token estimate: ${per_token_month:,.2f}/month")
    if tier:
        print(f"smallest request-quota tier covering your peak day: ${tier[0]}/month ({tier[1]}/day)")
    else:
        print("your peak day exceeds 8,000 requests; a request quota won't cover it")

if __name__ == "__main__":
    estimate("usage.jsonl")

Size the quota to your peak day, not your average, because a daily request quota stops you on the day you exceed it. Token budgets have the opposite failure mode: a single heavy week can drain the whole month.

Switching is two environment variables

Every plan in the table above speaks the OpenAI Chat Completions format, so trying a different pricing model doesn't mean rewriting code:

import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ.get("OPENAI_BASE_URL", "https://apiclaw.biz/v1"),
    api_key=os.environ["OPENAI_API_KEY"],
)

resp = client.chat.completions.create(
    model="apiclaw/claude-sonnet-5.5",
    messages=[{"role": "user", "content": "Summarise this diff in two sentences."}],
)
print(resp.choices[0].message.content)
print(resp.usage)  # keep logging this so you can re-run the estimate later

On a request quota, one habit pays off: batch small asks into one call (for example, classify 20 items in one prompt and return JSON) instead of 20 separate calls. On per-token billing that changes little; on a request quota it cuts usage twentyfold.

When APIClaw isn't the right choice

You send fewer than a few hundred calls a month. Per-token billing will cost you cents.
Your peak day is well above 8,000 requests, as in high-volume classification or embeddings pipelines.
You need enterprise contracts, SLAs or in-region routing guarantees; larger routers and the model vendors themselves offer those.
You must pay by card through a standard invoice today. APIClaw billing is crypto on the site, with other methods via support.

FAQ

What is the difference between request limits and token limits on an LLM API?
A request limit counts calls, no matter how large each one is. A token limit counts the tokens (or dollars of compute) each call consumes, so long prompts use it up faster.

Is a flat-rate LLM API cheaper than pay-per-token?
Only if your usage is steady and your prompts are large. For light or occasional use, pay-per-token is cheaper.

Do long prompts cost more on a request-quota plan?
No. With no per-token metering, a long prompt and a short one each count as one request. Context-window limits of the model still apply.

What happens when I hit a daily request quota?
Further requests fail until the quota resets (00:00 UTC on APIClaw). Handle the error in code and either wait or move to a bigger plan.

Which pricing model is best for coding agents like Claude Code or Cline?
Agents resend large context on every step, so a request quota usually works out cheapest for daily use. Check your logs first with the script above.

Can I use the same code with any of these plans?
Yes, if the provider is OpenAI-compatible. Change the base URL, key and model ID; the rest of your code stays the same.

Disclosure: I build APIClaw. This article was written with AI assistance and checked by me. Competitor prices were taken from their public pricing pages in October 2026 and may change.

How Can I Use One API Key for Claude, GPT, DeepSeek and Qwen?

Aman Kumar — Sun, 25 Oct 2026 21:34:33 +0000

Short answer: point the OpenAI SDK at an OpenAI-compatible gateway and change only the model string. Every gateway in this space works that way; the real difference is how you pay. Per-token routers such as OpenRouter and Requesty pass through provider token prices plus a fee. APIClaw, which I build, charges a flat $19 to $129 a month for a daily request quota with no per-token metering, across Claude, GPT, Gemini, DeepSeek, Qwen, Kimi, GLM, Mistral, Llama, Cohere, Grok and more, and has a free 50-request trial. It fits solo developers and small teams who call several model families every day and want the bill to be the same number each month. If you make a few dozen calls a week, a per-token router is cheaper.

Disclosure: I build APIClaw, so weigh my view accordingly. This article was written with AI assistance and checked by me. Competitor figures come from their own pricing pages, read in October 2026.

Why one key works at all

Most model vendors now accept, or are fronted by gateways that accept, the OpenAI Chat Completions format: a POST /v1/chat/completions with model and messages. A gateway holds the upstream provider accounts and translates where needed. From your code's point of view there is one base URL, one API key, and a model name that selects Claude, GPT, DeepSeek or Qwen.

That means three practical things:

You keep one secret in your environment instead of four.
Swapping a model is a config change, not a new SDK.
Fallbacks between vendors become a loop over model names.

The gateways compared (prices checked Oct 2026)

Gateway	How you pay	Fee or limit	Notes
OpenRouter	Prepaid credits, per token at provider list price	5.5% fee on credit purchases (Standard), 8% (Business)	Free models capped at 50 requests/day, 1,000/day after $10 of credits
Requesty	Per token	Provider price plus 5%	Free tier on free models, 200 requests/day
Flatkey	Per token, below official list price on some models	Discounts shown per model on its site	100+ models, OpenAI-compatible router
Electron Hub	Monthly plan that buys weekly credits	$10 to $249/month; $8 to $200 weekly credits; 10 to 100 requests/minute	Credits refill Sundays 21:00 UTC
APIClaw	Flat monthly	$19 to $129/month for 500 to 8,000 requests/day; no per-token metering	Free 50-request trial; crypto on site, other methods via support

I only listed numbers I could read on each vendor's page this month. I haven't benchmarked latency, so this is a pricing map rather than a review. A longer side-by-side lives in this comparison of OpenAI-compatible API gateways.

The code: one client, four model families

Install the official SDK (pip install openai) and set two environment variables:

export OPENAI_BASE_URL="https://apiclaw.biz/v1"
export OPENAI_API_KEY="sk-your-apiclaw-key"

The SDK reads both automatically, so the client needs no arguments:

from openai import OpenAI

client = OpenAI()  # picks up OPENAI_BASE_URL and OPENAI_API_KEY

MODELS = {
    "claude": "apiclaw/claude-sonnet-5.5",
    "gpt": "apiclaw/gpt-6-sol",
    "deepseek": "apiclaw/deepseek-v4-pro",
    "qwen": "apiclaw/qwen3.8-max-0902",
}

def ask(family: str, prompt: str) -> str:
    resp = client.chat.completions.create(
        model=MODELS[family],
        messages=[{"role": "user", "content": prompt}],
        max_tokens=400,
    )
    return resp.choices[0].message.content

for family in MODELS:
    print(f"--- {family}")
    print(ask(family, "Explain a Python context manager in two sentences."))

Nothing in that loop knows which vendor it's talking to. The same pattern works in Node:

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://apiclaw.biz/v1",
  apiKey: process.env.APICLAW_API_KEY,
});

const res = await client.chat.completions.create({
  model: "apiclaw/deepseek-v4-pro",
  messages: [{ role: "user", content: "Write a regex for ISO dates." }],
});
console.log(res.choices[0].message.content);

Pick the model per task, not per project

Once switching is free, routing by task is the useful step. A small table in config is enough:

ROUTES = {
    "code_review": "apiclaw/claude-sonnet-5.5",
    "bulk_summaries": "apiclaw/deepseek-v4-flash",
    "multilingual": "apiclaw/qwen3.8-max-0902",
    "general": "apiclaw/gpt-6-sol",
}

def run(task: str, prompt: str) -> str:
    model = ROUTES.get(task, ROUTES["general"])
    return client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
    ).choices[0].message.content

Under per-token pricing, you route cheap jobs to cheap models to save money. Under a request quota every call costs the same one request, so you can route purely on quality and keep the cheap models for latency-sensitive work.

A cross-vendor fallback in ten lines

Different vendors fail at different times, which is the main reliability win of one key:

from openai import APIError, RateLimitError

FALLBACK = ["apiclaw/claude-sonnet-5.5", "apiclaw/gpt-6-sol", "apiclaw/deepseek-v4-pro"]

def ask_with_fallback(prompt: str) -> str:
    last_err = None
    for model in FALLBACK:
        try:
            r = client.chat.completions.create(
                model=model,
                messages=[{"role": "user", "content": prompt}],
                timeout=60,
            )
            return r.choices[0].message.content
        except (RateLimitError, APIError) as e:
            last_err = e
    raise last_err

Keep the list short. Three families from different vendors covers nearly every outage without hiding real bugs behind endless retries.

Things that still differ between models

One key normalises the request format, not the models. Watch for these:

Context windows and output limits vary by model. Set max_tokens explicitly instead of relying on defaults.
Tool calling works through the same tools parameter, but smaller models follow complex schemas less reliably. Test each model you route tool calls to.
System prompt sensitivity differs. A prompt tuned for Claude may need a line or two of changes for Qwen.
Reasoning models may return extra reasoning content or take longer. Give them a longer timeout.

When APIClaw isn't the right choice

If your total usage is a handful of calls a day, a per-token router will cost less than $19 a month. If you need SSO, audit logs or a signed enterprise SLA today, Electron Hub's higher tiers or OpenRouter's enterprise plan cover that and APIClaw doesn't advertise it. If your traffic is extremely bursty, such as 20,000 calls on one day and none for a week, a daily request quota is the wrong shape and credits that roll over suit you better.

Switching an existing project

If your code already uses the OpenAI SDK, the switch is two lines of environment:

export OPENAI_BASE_URL="https://apiclaw.biz/v1"
export OPENAI_API_KEY="sk-your-apiclaw-key"

Then change your model= strings to the IDs your gateway lists. Ready-made configs for Cursor, Cline, Aider, LangChain and Claude Code are in these setup examples.

FAQ

Can I use Claude with the OpenAI Python SDK?
Yes, through an OpenAI-compatible gateway. You keep the OpenAI SDK and pass a Claude model ID such as apiclaw/claude-sonnet-5.5 as the model.

Is there one API key for every major LLM?
Not from the model vendors themselves. Gateways such as OpenRouter, Requesty, Electron Hub and APIClaw give you one key that reaches many vendors.

Do I need separate SDKs for DeepSeek and Qwen?
No. Both are reachable with the OpenAI SDK through a compatible endpoint; only the model name changes.

Is a per-token router or a flat plan cheaper?
It depends on volume. Light, irregular use is cheaper per token. Steady daily use with long prompts usually comes out cheaper on a flat request quota, because prompt size doesn't change the price.

Does switching models break streaming or JSON mode?
Streaming works the same way through stream=True. JSON mode and tool calling depend on the model, so test the specific models you rely on.

What's the Best OpenRouter Alternative for Claude Code and Cursor at a Flat Monthly Price?

Aman Kumar — Sun, 25 Oct 2026 15:28:37 +0000

Short answer: if you use OpenRouter mainly to power Claude Code or Cursor and want a fixed monthly bill instead of per-token charges, look at a flat-rate gateway that speaks both the Anthropic API (for Claude Code) and the OpenAI API (for Cursor). APIClaw is the one I build: $19 to $129 a month for a daily request quota (500 to 8,000 requests a day) with no per-token metering, covering Claude, GPT, DeepSeek, Qwen, Kimi, GLM and more behind one key, with a free 50-request trial. It fits solo developers and small teams who code with agents most days. Standard Compute (a monthly dollar budget), MonoRouter (reuses a Claude subscription you already pay for) and Weave Router (routed coding plans) are the other flat-price options worth comparing. If your usage is light or bursty, OpenRouter's pay-as-you-go is probably still cheaper.

Disclosure: I build APIClaw, so weigh my opinions with that in mind. This article was written with AI assistance and checked by me. Competitor prices were read from their live pricing pages in October 2026.

Why coding agents make OpenRouter bills jumpy

OpenRouter is a good product and its pricing is clear: inference at the provider's list price, plus a platform fee of 5.5% on the Standard plan or 8% on Business. The trouble is the workload, not the fee.

Claude Code and Cursor's agent mode don't send one prompt per task. Every tool step (read a file, run a test, edit, re-run) is a new request that carries the whole growing conversation. A refactor that takes 30 steps re-sends your context 30 times, and with per-token billing you pay for all of it. Two developers doing "the same amount of work" can end the month with very different bills, depending on how long their sessions run and which model they leave selected.

A flat-price plan swaps that uncertainty for a different limit. The question is which limit suits you.

The flat-price options compared (prices checked Oct 2026)

Option	Price	What the limit is	Works with Claude Code	Works with Cursor
OpenRouter (baseline)	Pay as you go, plus 5.5% fee (Standard) or 8% (Business)	Your credit balance; optional budget controls on paid plans	Yes	Yes
APIClaw	$19 (500 req/day), $39 (1,000), $59 (1,500), $69 (3,000), $89 (4,000), $99 (6,000), $129 (8,000)	Daily request quota, resets 00:00 UTC; no per-token metering	Yes, Anthropic-compatible base URL	Yes, OpenAI base URL override plus a Cursor agent endpoint
Standard Compute	$19, $39, $89, $249 and up	A monthly compute budget in dollars ($20, $41, $95, $269 on those plans); requests stop at the budget until renewal	Check their docs	Check their docs
MonoRouter	Free (1,500 calls lifetime), $9 (10,000 calls/mo), $29 (unlimited calls), $99 team (5 seats)	Routes calls through a Claude subscription you already pay for, so that plan's own limits still apply	Yes, Anthropic SDK-style	Not stated
Weave Router	$10/mo Code Max (open-source models only), $100/mo Code Boost (OpenAI, Anthropic and open models)	Routed coding subscription; Code Boost is pitched as stretching existing Claude and Codex subscriptions	Check their docs	Check their docs

I've only listed what each pricing page states. Where a page didn't say how a tool connects, I wrote "check their docs" rather than guess.

How to choose between them

You want the most predictable month and use Claude Code or Cursor daily: a request quota is the easiest limit to reason about, because a long context costs the same as a short one. That's the APIClaw model. Size the plan by requests, not tokens: count how many agent steps a typical day takes, then add headroom.
You prefer a dollar budget with model routing: Standard Compute caps you at a monthly budget and routes each request to a model and provider. Your spend is fixed, but how much work the budget covers still depends on tokens.
You already pay for Claude Max and want to share it or use it from your own apps: MonoRouter is built for that. Your cost is the MonoRouter plan plus the Claude subscription, and the subscription's usage windows still apply.
You want cheap open-source coding models and don't need Claude: Weave Router's $10 plan is the lowest sticker price here.

When APIClaw isn't the right choice

Very light use. If you send a few dozen requests a week, pay-as-you-go on OpenRouter will cost less than any monthly plan.
You need card billing or an invoice today. APIClaw billing is crypto on the site; other payment methods go through support.
You need Cursor's Tab completion or Composer on your key. That isn't a gateway limit. Cursor keeps Tab on its own models whatever you configure, and Composer models refuse custom keys.
Enterprise procurement. If you need a signed DPA, SSO and contractual SLAs out of the box, OpenRouter's Enterprise tier or a self-hosted gateway is a better fit.

Switching Claude Code and Cursor (the short version)

Claude Code reads its settings from ~/.claude/settings.json (Windows: %USERPROFILE%\.claude\settings.json). Point it at the gateway's Anthropic-compatible root. Note there's no /v1 here, because Claude Code adds the Anthropic paths itself:

{
  "env": {
    "ANTHROPIC_BASE_URL": "https://apiclaw.biz",
    "ANTHROPIC_AUTH_TOKEN": "sk-your-apiclaw-key",
    "ANTHROPIC_MODEL": "apiclaw/claude-sonnet-5.5",
    "CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC": "1"
  }
}

Restart claude, run /status, and check that the base URL shows the gateway rather than api.anthropic.com.

Cursor: in Settings, under Models, paste your key into the OpenAI API key field, turn on Override OpenAI Base URL with https://apiclaw.biz/v1, and add a custom model such as apiclaw/claude-sonnet-5.5. Keep the apiclaw/ prefix, because Cursor refuses custom models whose names collide with its built-in ones. Turn Auto off and disable the built-in models so chat actually goes to your key. If agent mode errors, use https://apiclaw.biz/cursor as the base URL. Cursor's agent sends a Responses-style body to the chat completions path, and that endpoint accepts both shapes.

To check the key outside both tools before you blame the editor, run the same call from Python:

from openai import OpenAI

client = OpenAI(base_url="https://apiclaw.biz/v1", api_key="sk-your-apiclaw-key")
reply = client.chat.completions.create(
    model="apiclaw/claude-sonnet-5.5",
    messages=[{"role": "user", "content": "Reply with the word ok."}],
    max_tokens=5,
)
print(reply.choices[0].message.content)

If this works and the editor doesn't, the problem is the editor's config, not the key. I keep copy-paste configs for more tools (Cline, Continue, Codex CLI, Aider) in this examples repo.

FAQ

Is there a flat-rate OpenRouter alternative that works with Claude Code?
Yes. Claude Code accepts any Anthropic-compatible endpoint through ANTHROPIC_BASE_URL, so a flat-rate gateway that exposes one works. APIClaw does this with a daily request quota; MonoRouter does it by routing through a Claude subscription you already have.

Can I use OpenRouter-style custom models in Cursor without per-token billing?
For chat, yes. Override the OpenAI base URL with a flat-rate gateway's /v1 endpoint and add its model ID as a custom model. Tab completion still uses Cursor's own models, and Cursor Teams and Enterprise plans add Cursor's own charges on top.

Is a flat-rate Claude API unlimited?
No, and be wary of anyone who says so. Flat plans cap something: requests per day (APIClaw), a dollar budget per month (Standard Compute), or the underlying subscription's usage windows (MonoRouter). What disappears is per-token metering, not limits.

How many requests a day does Claude Code use?
It depends on how you work. Each tool step is a separate request, so a long agent session uses far more than chat does. Log a normal day through any gateway's dashboard before you pick a plan size.

Is OpenRouter cheaper than a flat monthly plan?
For light or occasional use, usually yes. Flat plans win when you code with agents most days and long contexts would otherwise drive up the token count.

Do I need a Claude Max subscription to use a gateway?
Not for APIClaw or Standard Compute, which sell API access directly. MonoRouter's model assumes you bring your own Claude subscription.

Running Inspect AI Evals on a Custom OpenAI-Compatible API: openai-api/ Model Names, Two Retry Layers and Hidden Errors

Aman Kumar — Sun, 25 Oct 2026 08:46:52 +0000

Inspect AI is the open-source evaluation framework from the UK AI Security Institute. You write a task (dataset, solver, scorer), run inspect eval, and get a log you can browse. It has first-party providers for the big labs, plus a generic openai-api provider for anything that speaks the OpenAI Chat Completions format: vLLM, a LiteLLM proxy, Ollama's /v1 endpoint or a hosted gateway.

The setup is two environment variables, but a few defaults change your results and your request count in ways that are easy to miss. I tested inspect_ai 0.3.281 with openai 3.35.1 on Python 3.13.5 against a small local mock server. The mock logged the path, headers and JSON body of every request and could return 400, 401, 429 or 503 on demand. Everything below is what went over the wire. The examples use APIClaw, an OpenAI-compatible gateway I build, so weigh my opinions with that in mind. Its base URL is https://apiclaw.biz/v1. Replace YOUR_MODEL_ID with an exact ID from your provider's model list rather than guessing one.

The model string picks your environment variables

pip install inspect-ai openai

export APICLAW_API_KEY="your-key"
export APICLAW_BASE_URL="https://apiclaw.biz/v1"

inspect eval arith.py --model openai-api/apiclaw/YOUR_MODEL_ID

The model string has three parts: openai-api, a service name you choose, and the model ID. Inspect upper-cases the service name and reads <SERVICE>_API_KEY and <SERVICE>_BASE_URL. Hyphens become underscores, so openai-api/my-gw/... reads MY_GW_API_KEY and MY_GW_BASE_URL. Pick a short name and stick with it, because it also appears in your logs and progress display.

If either variable is missing you get a clear message before any request goes out: No APICLAW_API_KEY defined in the environment. (or ..._BASE_URL). If you leave the service out entirely (openai-api/YOUR_MODEL_ID), you get a ValueError: openai-api model names must include a service prefix wrapped in a long TaskGroup traceback, which is easy to misread as a network problem.

Everything after the service name is passed through as the model ID, slashes included. openai-api/apiclaw/deepseek/deepseek-chat sent "model": "deepseek/deepseek-chat", which is what you want for gateways that use vendor-prefixed IDs.

Keep /v1 in the base URL

Inspect hands the base URL to the OpenAI Python SDK unchanged, and the SDK only appends /chat/completions. With APICLAW_BASE_URL=http://host/v1 the mock saw POST /v1/chat/completions. Without /v1 it saw POST /chat/completions, which most servers answer with a 404. The request carried Authorization: Bearer <your key> and the SDK's AsyncOpenAI/Python user agent.

What a default request contains

Here is the task I used:

# arith.py
from inspect_ai import Task, task
from inspect_ai.dataset import Sample
from inspect_ai.scorer import match
from inspect_ai.solver import generate

@task
def arith():
    return Task(
        dataset=[Sample(input=f"What is 2+2? (variant {i})", target="4") for i in range(20)],
        solver=generate(),
        scorer=match(),
    )

With no generation options, the body was just model and messages. No temperature, no max_tokens, no seed, so you get whatever defaults the server applies. If you want reproducible runs, say so on the command line: --temperature 0 --max-tokens 50 --seed 7 arrived as "temperature": 0.0, "max_tokens": 50, "seed": 7.

Streaming is off unless you ask for it. -M stream=true added "stream": true and "stream_options": {"include_usage": true}, and token counts still showed up in the summary because the mock sent a usage chunk at the end. If your server does not send usage when streaming, expect missing token totals.

If the model wraps its reasoning in <think>...</think>, Inspect splits it out. The mock answered <think>let me reason</think>ANSWER: 4, and the stored message had a reasoning block plus a text block, with output.completion equal to ANSWER: 4. The match() scorer saw only the answer.

Concurrency is adaptive by default

Inspect does not send one request at a time. Adaptive concurrency is on by default; the config docstring gives its bounds as a minimum of 10, a start of 20 and a maximum of 100 connections. With 60 samples and a mock that took 300 ms per reply, I saw up to 30 requests in flight at once.

That matters if your provider limits requests per minute or per day, which is how many flat-rate plans work. Set a fixed number instead:

inspect eval arith.py --model openai-api/apiclaw/YOUR_MODEL_ID --max-connections 4

With --max-connections 4 the peak was exactly 4. Also count your requests honestly: every sample is at least one call, and model-graded scorers add more (see below).

Two retry layers multiply

This was the biggest surprise. Inspect creates an AsyncOpenAI client and leaves the SDK's own max_retries at its default of 2, so every Inspect attempt is up to three HTTP requests. Inspect then retries failed attempts itself, and its max_retries defaults to unlimited.

With the mock returning 429 and Retry-After: 1 for one sample:

--max-retries 3 sent 12 requests over about 37 seconds (4 Inspect attempts times 3 SDK attempts; the x-stainless-retry-count header went 0, 1, 2 four times).
With no limits set, it had sent 15 requests after 75 seconds and was still going when I killed it. The gaps between Inspect's attempts grew from about 3 to 9, 13 and 26 seconds.
503 behaved the same as 429. A 401 or 400 was sent once and not retried.

To get one retry layer you control, turn the SDK's off with a model arg and give Inspect a limit:

inspect eval arith.py --model openai-api/apiclaw/YOUR_MODEL_ID \
  -M max_retries=0 --max-retries 3 --timeout 120

-M arguments go to the OpenAI client constructor, so max_retries=0 applies there. With that and --max-retries 3 the same 429 produced 4 requests, not 12. --timeout caps the whole request including retries; with --timeout 20 the run stopped after 4 requests.

One bad request fails the whole eval, unless you let it hide

I made the mock return a 400 for one sample out of five. By default the eval ended with status error and no scores at all, even though four samples had succeeded.

The usual fix is --fail-on-error 0.25 (tolerate up to 25% failed samples) or --no-fail-on-error. Both gave status success and accuracy 1.000. That number is computed over the four samples that worked; the failed one is left out rather than counted as wrong. If a server rejects your hardest prompts, for example because they exceed the context window, your accuracy goes up. Check the errored sample count in the log before you compare models:

from inspect_ai.log import read_eval_log

log = read_eval_log("logs/your-run.eval")
errored = sum(1 for s in log.samples if s.error)
print(log.status, f"{errored}/{len(log.samples)} samples errored")

The grader is the model you are testing

model_graded_qa() and the other model-graded scorers use the grader model role, and if you do not set it they use the model under evaluation. In my run each sample sent two requests to the same model: the answer, then a grading prompt starting "You are assessing a submitted answer...". Letting a model grade itself is rarely what you want, and it doubles your request count. Point the grader somewhere else:

inspect eval graded.py --model openai-api/apiclaw/YOUR_MODEL_ID \
  --model-role grader=openai-api/apiclaw/YOUR_GRADER_MODEL_ID

With that set, the mock saw the answer request go to one model ID and the grading request to the other. Both use the same APICLAW_* variables because they share a service name.

The same settings from Python

from inspect_ai import eval
from arith import arith

logs = eval(
    arith(),
    model="openai-api/apiclaw/YOUR_MODEL_ID",
    model_base_url="https://apiclaw.biz/v1",
    model_args={"max_retries": 0},
    max_retries=3,
    max_connections=4,
)
print(logs[0].status)

model_base_url replaces the _BASE_URL variable; the key still comes from APICLAW_API_KEY. For one-off calls outside an eval, get_model("openai-api/apiclaw/YOUR_MODEL_ID", base_url=..., api_key=...) and await model.generate("...") worked the same way.

What ends up in the logs

Inspect writes a .eval file for every run, including failed ones, and it stores each request's messages and parameters so you can inspect them in inspect view. I searched all 18 logs from my tests for the API key and for the string Bearer and found neither. Prompts and responses are stored in full, so treat the logs/ folder like any other file with test data in it.

Checklist

Use openai-api/<service>/<model> and set <SERVICE>_API_KEY and <SERVICE>_BASE_URL, with /v1 in the URL.
Pass --temperature and --seed explicitly; nothing is sent by default.
Set --max-connections if your plan limits requests.
Use -M max_retries=0 with --max-retries N and --timeout, or a 429 can retry for a long time.
After a run with --fail-on-error or --no-fail-on-error, count errored samples before trusting accuracy.
Give model-graded scorers their own grader with --model-role grader=....

Disclosure: I build APIClaw, the OpenAI-compatible gateway used in these examples (flat monthly plans; crypto on the site, other payment methods through support). Everything above applies to any compatible endpoint.

Using CloudWeGo Eino With a Custom OpenAI-Compatible API in Go: BaseURL, Zero Retries and a 12-Step Agent Limit

Aman Kumar — Mon, 19 Oct 2026 19:46:49 +0000

Eino is CloudWeGo's Go framework for LLM applications: chat models, tools, graphs and a ReAct agent, in the style of LangChain but written for Go. Its OpenAI component (github.com/cloudwego/eino-ext/components/model/openai) is the one most people use with self-hosted or hosted OpenAI-compatible servers such as vLLM, Ollama's /v1 endpoint, LiteLLM or a gateway.

The setup is three fields. The behaviour around them is less obvious, so I tested eino v0.7.13 and the eino-ext OpenAI component v0.1.13 on Go 1.27.2 against a small local mock server. The mock logged the path, headers and JSON body of every request and could return 401, 400, 429, 500 or 503 on demand, or keep asking for a tool call forever. Everything below is what went over the wire. The examples use APIClaw, an OpenAI-compatible gateway I build, so weigh my opinions with that in mind. Its base URL is https://apiclaw.biz/v1. Replace YOUR_MODEL_ID with an exact ID from your provider's model list rather than guessing one.

Install, and one build error you may hit

go get github.com/cloudwego/eino@latest
go get github.com/cloudwego/eino-ext/components/model/openai@latest

On Go 1.27 my first go build failed inside github.com/bytedance/sonic/internal/rt with undefined: GoMapIterator. Eino pulled in sonic v1.14.1, which does not build on that toolchain. Upgrading sonic fixed it:

go get github.com/bytedance/sonic@latest github.com/bytedance/sonic/loader@latest
go mod tidy

That moved me to sonic v1.15.4 and everything compiled. If you are on an older Go release you may never see this.

The minimal config

import (
    "context"
    "os"
    "time"

    "github.com/cloudwego/eino-ext/components/model/openai"
    "github.com/cloudwego/eino/schema"
)

ctx := context.Background()
cm, err := openai.NewChatModel(ctx, &openai.ChatModelConfig{
    BaseURL: "https://apiclaw.biz/v1",
    APIKey:  os.Getenv("APICLAW_API_KEY"),
    Model:   "YOUR_MODEL_ID",
    Timeout: 60 * time.Second,
})
if err != nil {
    return err
}
out, err := cm.Generate(ctx, []*schema.Message{
    schema.SystemMessage("Be brief."),
    schema.UserMessage("Say hi"),
})

The struct comments say BaseURL is for Azure, but it is used for every non-Azure client too. Eino does not read OPENAI_API_KEY or OPENAI_BASE_URL from the environment, so pass both explicitly.

Three things I checked:

The path is BaseURL + /chat/completions, nothing more. https://host/v1 and https://host/v1/ both produced POST /v1/chat/completions. A bare https://host produced POST /chat/completions, which most gateways answer with a 404. Keep the /v1.
The body is minimal. With only the fields above, the request was {"messages":[...],"model":"YOUR_MODEL_ID"}. No temperature, no max_tokens, no stream:false. Headers were Authorization: Bearer <key> and Go's default User-Agent: Go-http-client/1.1.
Timeout defaults to none. The struct comment says so, and nothing else in the chain adds one. If you leave it unset and the upstream stalls, Generate waits as long as your context allows. If you pass your own HTTPClient, Timeout is ignored and the client's timeout applies.

Sampling options are pointers, so zero really is sent

Temperature, MaxTokens, TopP and friends are pointer fields. Setting Temperature: &zero put "temperature":0 in the body, which some Python wrappers drop because they treat zero as "unset". Per-call options override the config:

cm.Generate(ctx, msgs, model.WithTemperature(0.7), model.WithModel("OTHER_MODEL_ID"))

That second call sent "temperature":0.7 and "model":"OTHER_MODEL_ID" while keeping the configured max_tokens. model.WithModel is a cheap way to try a second model on the same gateway without building another client.

Streaming asks for usage automatically

cm.Stream(ctx, msgs) sent "stream":true and "stream_options":{"include_usage":true} without being asked. After concatenating the chunks with schema.ConcatMessages, ResponseMeta.Usage held the prompt, completion and total token counts from the final usage chunk.

If your server rejects unknown fields like stream_options, that is the first thing to check when streaming fails but Generate works.

For reasoning models, a reasoning_content field in the response landed in out.ReasoningContent. Inline <think>...</think> text stayed inside out.Content; Eino does not strip it, so do that yourself if your model emits it.

There are no retries

This surprised me most. Every error status got exactly one request and came back in a millisecond or two:

Server response	Requests sent	Returned error
401	1	`status code: 401 ... message: ...`
400	1	same shape
429 with `Retry-After: 2`	1	same shape
500 / 503	1	same shape

Retry-After is ignored because nothing retries. That is fine for a CLI. For a service sitting behind a gateway that rate-limits per minute, one 429 becomes a user-visible failure.

The error can be inspected with errors.As against the eino-ext type, not the underlying go-openai fork's type. I tried both. The fork's *APIError did not match, because the component converts it.

var apiErr *openai.APIError // github.com/cloudwego/eino-ext/components/model/openai
if errors.As(err, &apiErr) && apiErr.HTTPStatusCode == 429 {
    // back off
}

That still matched when the 429 happened inside a ReAct agent run, where the outer error text starts with [NodeRunError] and names the node path.

Adding retries with an http.RoundTripper

Since ChatModelConfig accepts an *http.Client, the cleanest fix is a transport that replays the request body:

type retryTransport struct {
    base     http.RoundTripper
    attempts int
}

func (t retryTransport) RoundTrip(req *http.Request) (*http.Response, error) {
    var body []byte
    if req.Body != nil {
        body, _ = io.ReadAll(req.Body)
        req.Body.Close()
    }
    var resp *http.Response
    var err error
    for i := 0; i < t.attempts; i++ {
        req.Body = io.NopCloser(bytes.NewReader(body))
        resp, err = t.base.RoundTrip(req)
        if err != nil || (resp.StatusCode != 429 && resp.StatusCode < 500) || i == t.attempts-1 {
            return resp, err
        }
        wait := time.Duration(1<<i) * time.Second
        if s, e := strconv.Atoi(resp.Header.Get("Retry-After")); e == nil {
            wait = time.Duration(s) * time.Second
        }
        resp.Body.Close()
        select {
        case <-time.After(wait):
        case <-req.Context().Done():
            return nil, req.Context().Err()
        }
    }
    return resp, err
}

cm, _ := openai.NewChatModel(ctx, &openai.ChatModelConfig{
    BaseURL: "https://apiclaw.biz/v1",
    APIKey:  os.Getenv("APICLAW_API_KEY"),
    Model:   "YOUR_MODEL_ID",
    HTTPClient: &http.Client{
        Timeout:   60 * time.Second,
        Transport: retryTransport{base: http.DefaultTransport, attempts: 3},
    },
})

Against the mock, a server that returned 429 twice and then succeeded produced three requests about two seconds apart in total and a normal reply. A permanent 503 produced three requests and then the usual error. A 401 still produced one request, which is what you want. Note that http.Client.Timeout covers all attempts together, so size it for the retries. Retrying a streaming call this way only helps when the error arrives before the first byte, which is where 429s usually happen.

The ReAct agent stops at MaxStep, and steps are not tool calls

agent, err := react.NewAgent(ctx, &react.AgentConfig{
    ToolCallingModel: cm,
    ToolsConfig:      compose.ToolsNodeConfig{Tools: []tool.BaseTool{getTime}},
    MaxStep:          20,
})

With a tool built by utils.InferTool, the tool schema went out with "additionalProperties":false and a required list, plus "tool_choice":"auto". There was no strict flag. The assistant message that carried the tool call was sent back with no content key at all, followed by a role: tool message with the matching tool_call_id.

To see the limit, I made the mock request a tool call on every turn:

With MaxStep unset (default 12), the run made 7 model requests and 6 tool calls, then returned [GraphRunError] exceeds max steps.
With MaxStep: 40, it made 20 model requests and 20 tool calls before the same error.

MaxStep counts graph node executions, and each round is a model node plus a tools node, so you get roughly MaxStep / 2 tool calls, not MaxStep. Each request also re-sends the whole history, so the last request in the 40-step run carried 39 messages. Check for the limit with:

if errors.Is(err, compose.ErrExceedMaxSteps) {
    // the model kept calling tools; surface a partial answer or ask the user
}

When the mock returned a tool call once and then plain text, the agent finished normally after two requests.

Checklist

Set BaseURL with /v1; Eino only appends /chat/completions.
Pass APIKey explicitly; environment variables are not read.
Set Timeout or your own HTTPClient; the default is no timeout.
Add retries yourself with a transport, and match errors with the eino-ext *openai.APIError.
Expect stream_options.include_usage on every stream.
Treat MaxStep as about twice the number of tool rounds you want to allow.

Google's LangExtract on a Custom OpenAI-Compatible API: Model IDs That Route to Ollama, Strict Schemas and Silent Chunks

Aman Kumar — Mon, 19 Oct 2026 07:38:16 +0000

LangExtract is Google's Python library for pulling structured entities out of long text, with every extraction mapped back to its character position in the source. You describe the task, give one or two examples, and it chunks the document, sends each chunk to a model, parses the JSON and aligns the results. Gemini is the default, but it has an OpenAI provider built on the official OpenAI SDK, so it can run against vLLM, LiteLLM or a hosted gateway.

Getting it to use that provider with a non-OpenAI model name is the first surprise. I tested langextract 1.7.1 (openai 3.35.1, Python 3.13) against a small local mock server that logged every request and could return 400, 429, 503, fenced JSON or truncated output on demand. Everything below is what went over the wire. The examples point at APIClaw, an OpenAI-compatible gateway I build, so weigh my opinions accordingly. Its base URL is https://apiclaw.biz/v1; replace YOUR_MODEL_ID with an exact ID from your provider's model list.

The model name chooses the provider

The obvious call is:

import langextract as lx

result = lx.extract(
    text_or_documents=text,
    prompt_description=prompt,
    examples=examples,
    model_id="qwen3-coder-plus",
    api_key="your-key",
    model_url="https://apiclaw.biz/v1",
)

LangExtract routes on regex patterns over model_id. Only names starting with gpt-3.5, gpt-4, gpt-5 or o1 to o9 go to the OpenAI provider. Names starting with qwen, deepseek, llama, mistral, gemma, phi and gpt-oss, among others, go to the Ollama provider. Here is what happened in my tests:

qwen3-coder-plus and gpt-oss-120b went to the Ollama client, which POSTed to /v1/api/generate on my base URL and failed with Can't find Ollama qwen3-coder-plus. Try: ollama run qwen3-coder-plus.
A name that matched nothing, such as plain-model, failed with No provider registered for model_id='plain-model'.
gpt-4o-mini with model_url worked and hit /v1/chat/completions.

So the code above only works if your gateway's model happens to be named like an OpenAI model.

Force the OpenAI provider with ModelConfig

import os
import langextract as lx

config = lx.factory.ModelConfig(
    model_id="YOUR_MODEL_ID",
    provider="OpenAILanguageModel",   # "openai" also worked
    provider_kwargs={
        "api_key": os.environ["APICLAW_API_KEY"],
        "base_url": "https://apiclaw.biz/v1",
        "temperature": 0.0,
        "max_output_tokens": 2000,
        "max_workers": 4,
    },
)

result = lx.extract(
    text_or_documents=text,
    prompt_description="Extract every person mentioned, with their role. Use exact text.",
    examples=examples,
    config=config,
    show_progress=False,
)
for e in result.extractions:
    print(e.extraction_class, e.extraction_text, e.attributes, e.char_interval)

That sent POST /v1/chat/completions for every model name I tried. Two details:

Pass api_key explicitly. LangExtract only reads OPENAI_API_KEY from the environment when the model ID contains gpt. With an explicit provider and a name like qwen3-coder-plus, leaving the key out failed with API key not provided. even though OPENAI_API_KEY was set.
base_url can come from the environment. With base_url omitted, the SDK picked up OPENAI_BASE_URL. Keep /v1 on the end either way.

You'll also see a warning: With 'config', schema constraints are still applied via examples. That matters for the next section.

What each request contains

Every chunk became one request with:

a system message, You are a helpful assistant that responds in JSON format.
one user message holding your prompt description, your examples as Q:/A: pairs with the expected JSON, and the chunk text,
n: 1,
response_format of type json_schema, named langextract_extractions, with strict: true, built from your examples.

By default there was no temperature and no max_tokens, so the server's defaults applied. Setting temperature and max_output_tokens in provider_kwargs sent temperature: 0.0 and max_tokens: 2000. For extraction I'd always set both: a low temperature for consistency and an output cap that fits a long list of entities.

When the server doesn't support json_schema

Strict json_schema output is not universal. Some models and some OpenAI-compatible servers support only json_object, or nothing. My mock returned a 400 for json_schema, and the whole extract() call raised InferenceRuntimeError: OpenAI API error: Error code: 400.

result = lx.extract(..., config=config, use_schema_constraints=False)

With use_schema_constraints=False the request switched to response_format: {"type": "json_object"} and succeeded. The examples in the prompt still show the model the expected shape.

Parse failures are dropped with a warning

This is the setting I'd change first. By default LangExtract suppresses parse errors per chunk. Each of these runs printed one log line and returned zero extractions with no exception:

The model wrapped its answer in prose and a fence (Here is the result: followed by a fenced json block). Under json_object mode LangExtract expects raw JSON, so the chunk was omitted: Chunk omitted from results (parse error): Failed to parse JSON content.
The reply was cut off with finish_reason: "length". The truncated JSON failed to parse and the chunk was dropped the same way.

On a 40-chunk document, a few dropped chunks look like the model just didn't find anything there. Two fixes:

result = lx.extract(
    ...,
    config=config,
    use_schema_constraints=False,
    fence_output=True,                                  # accept fenced json replies
    resolver_params={"suppress_parse_errors": False},   # fail loudly instead
)

fence_output=True parsed the fenced reply correctly. suppress_parse_errors: False turned the silent drop into a ResolverParsingError, which you can catch and retry. A reply that started with a <think>...</think> block parsed fine in my test, so reasoning models weren't a problem there.

Chunks, workers and passes multiply requests

I ran a 5,320-character document with the defaults:

Setting	Requests	Max in flight
defaults (`max_char_buffer=1000`)	6	6
`max_workers: 3` in `provider_kwargs`	6	3
`extraction_passes=3`	18	6
`max_char_buffer=4000`	2	2

Each request repeats your prompt description and examples, so with small chunks the fixed part can be a large share of every request. max_workers defaults to 10, which means a long document can fire 10 requests at once. On a plan with a per-minute or concurrency limit, set it lower. extraction_passes runs the whole document again per pass, one pass after another, so three passes triple both requests and time.

A larger max_char_buffer cuts request count, but long chunks give the model more to track, so test recall on your own documents before raising it a lot.

Retries come from the SDK only

A server that always returned 503 got three requests (the SDK's default two retries) and then extract() raised InferenceRuntimeError. A 429 with Retry-After: 2 was honoured: the SDK waited and the third request succeeded after about 4.6 seconds. LangExtract adds no retry loop of its own, so for big batch jobs wrap the call in your own retry, and remember that a failure in one chunk fails the whole document.

Checklist

Use ModelConfig(provider="OpenAILanguageModel") unless your model ID starts with gpt- or o1 to o9.
Pass api_key in provider_kwargs; the env key is only read for gpt model names.
Set temperature and max_output_tokens yourself.
Use use_schema_constraints=False if your server rejects json_schema.
Set suppress_parse_errors: False (and fence_output=True for chatty models) so dropped chunks don't go unnoticed.
Lower max_workers to fit your rate limit, and count extraction_passes as a multiplier.

Disclosure: I build APIClaw, the OpenAI-compatible gateway in these examples (flat monthly plans; crypto on the site, other payment methods through support). Everything above applies to any compatible endpoint.

Running Qwen Code on a Custom OpenAI-Compatible API: modelProviders, a 41,000-Character First Request and Retries That Keep Going for Minutes

Aman Kumar — Sun, 18 Oct 2026 21:48:11 +0000

Qwen Code is the terminal coding agent from the Qwen team. It started as a fork of Gemini CLI, but unlike Gemini CLI it speaks the OpenAI Chat Completions format, so it can run against any OpenAI-compatible server: vLLM, Ollama's /v1 endpoint, LiteLLM, or a hosted gateway.

Setup is simple. A few defaults are less obvious, so I tested Qwen Code 0.25.2 on Node 20.19.2 against a small local mock server that logged every request and could return 401, 429 with Retry-After, or 503 on demand. Everything below is what went over the wire. The examples point at APIClaw, an OpenAI-compatible gateway I build, so weigh my opinions accordingly. Its base URL is https://apiclaw.biz/v1; replace YOUR_MODEL_ID with an exact ID from your provider's model list.

The quickest setup: three environment variables

npm install -g @qwen-code/qwen-code
export OPENAI_API_KEY="your-key"
export OPENAI_BASE_URL="https://apiclaw.biz/v1"
export OPENAI_MODEL="YOUR_MODEL_ID"
qwen -p "Explain what this repo's Makefile does"

That was enough. With no settings file at all, Qwen Code picked the OpenAI-compatible route and sent POST /v1/chat/completions with Authorization: Bearer ..., stream: true, stream_options: {"include_usage": true} and a tool list.

Two details matter here:

Keep /v1 in the base URL. Qwen Code uses the official OpenAI Node SDK, which appends /chat/completions to whatever you give it. With OPENAI_BASE_URL=https://host the request went to /chat/completions with no /v1, which most gateways answer with a 404.
max_tokens is 32000 by default. Every request in my env-only runs carried "max_tokens": 32000. Some servers reject a value above the model's output limit, and others reserve that much against your quota. If your model's output cap is lower, set it yourself (below).

A wrong key failed cleanly: one request, [API Error: 401 Invalid API key], exit code 1, no retries.

The setup I'd actually use: modelProviders in settings.json

Environment variables create what the docs call a "runtime model." For anything you use daily, a provider entry in ~/.qwen/settings.json is easier to reason about:

{
  "modelProviders": {
    "openai": [
      {
        "id": "YOUR_MODEL_ID",
        "name": "Gateway model",
        "envKey": "APICLAW_API_KEY",
        "baseUrl": "https://apiclaw.biz/v1",
        "generationConfig": {
          "timeout": 120000,
          "maxRetries": 1,
          "contextWindowSize": 128000,
          "samplingParams": { "temperature": 0.2, "max_tokens": 4096 }
        }
      }
    ]
  },
  "security": { "auth": { "selectedType": "openai" } },
  "model": { "name": "YOUR_MODEL_ID" }
}

The key under modelProviders must be a real protocol name (openai, anthropic, gemini, vertex-ai). A made-up key like "apiclaw" is skipped with a warning unless you also map it in a top-level providerProtocol block. envKey is the name of an environment variable, not the key itself. I tested three places for it:

APICLAW_API_KEY=... in a .env file in the project directory worked.
The same line in ~/.qwen/.env worked.
With neither, Qwen Code stopped before sending anything: Missing API key for OpenAI-compatible auth. Set the 'APICLAW_API_KEY' environment variable.

If you use a project .env, add it to .gitignore. A coding agent runs inside your repo, and that's exactly where a stray key gets committed.

With this entry selected, the request body was model, temperature: 0.2, max_tokens: 4096, stream and stream_options. The 32000 default was gone.

Settings that silently stop applying

I also had "model": {"generationConfig": {"samplingParams": {"top_p": 0.5}}} in the same file. It was never sent. That matches the documented rule: once a modelProviders entry is selected, its generationConfig replaces everything, and top-level model.generationConfig is ignored for that model. samplingParams, customHeaders and extra_body are replaced as whole objects, not merged. If a setting seems to have no effect, check whether it's sitting in the wrong layer.

What one "Say hi" actually costs

Non-interactive qwen -p "Say hi" sent two requests, not one.

The first was the answer. Its system prompt was 30,553 characters, and it carried a second 10,963-character block listing available skills and memory paths, plus six tool definitions (glob, grep_search, read_file, skill, tool_call, tool_search). That's about 41,500 characters before your prompt, which at a rough four characters per token is on the order of 10,000 input tokens.

The second request came after the answer was printed. Its system prompt began "You are now acting as the managed memory extraction subagent", it replayed the conversation, and it offered write_file and edit tools so the model could update memory files under ~/.qwen/. This is the managed auto-memory feature, and it's on by default.

On a per-token plan, that baseline matters for short prompts. On a plan that counts requests or has a per-minute limit, the memory pass doubles the count for one-shot runs. You can turn it off:

{
  "memory": { "enableManagedAutoMemory": false, "enableManagedAutoDream": false }
}

With that set, qwen -p "Say hi" sent exactly one request.

Retries: two layers, and they multiply

This is the part I'd configure before running Qwen Code in CI or on a slow endpoint.

A 429 that clears quickly is handled well. When the mock returned 429 with Retry-After: 2 twice, Qwen Code waited about 2 seconds each time and then succeeded on the third request.

A server that stays down is a different story. With default settings and a mock that always returned 503:

The OpenAI SDK retried each attempt three times with short backoff (about 0.4s, 0.9s, 1.8s), so requests arrived in groups of four.
Qwen Code's own retry loop repeated those groups with growing gaps (roughly 2, 3, 7, 14, 30 and 30 seconds). Seven groups produced 28 requests in about 107 seconds.
Then the CLI printed Retrying in 60s (attempt 1/10) and started a new cycle.

I stopped it at 200 seconds after 44 requests, still retrying.

Setting "maxRetries": 0 in generationConfig removed the SDK layer. The same dead endpoint then got 7 requests over about 74 seconds, and the CLI exited with [API Error: 503 upstream busy] and code 1. For scripts and CI jobs I'd set maxRetries to 0 or 1 and wrap the call in your own timeout, so a provider outage fails the job in a minute instead of hanging it.

A guard I was glad to see

My mock had a mode that kept asking for the same tool call with the same arguments. Qwen Code stopped after three requests: Loop detection halted the run (global_tool_call_duplicate: the model repeated the same tool call across the turn...). That's an always-on guard, and it's useful with smaller or cheaper models that sometimes get stuck repeating a step.

Checklist

OPENAI_BASE_URL (or baseUrl) ends in /v1, without /chat/completions.
Use modelProviders with an envKey, and keep the key in ~/.qwen/.env or a git-ignored project .env.
Set samplingParams.max_tokens to your model's real output limit instead of the 32000 default.
Put every setting for a provider model inside its own generationConfig; top-level ones are ignored for it.
Turn off managed auto-memory if you count requests or run one-shot -p jobs.
Set maxRetries low and use an outer timeout in CI.