DEV Community: build996

OpenClaw showed the right primary model while every turn went to a dead one

build996 — Mon, 14 Sep 2026 02:08:05 +0000

I switched a local OpenClaw gateway from NVIDIA's free endpoint to Groq. Set the new model as primary, restarted, sent a message. It hung for 120 seconds and timed out.

So I checked the config the way you would: openclaw config get on the model, then the JSON file on disk. Both showed the Groq model as primary. Correct, every time I looked. And yet the turns were going to moonshotai/kimi-k3 — an NVIDIA-hosted model that had already been pulled, which is why nothing ever answered.

The value that mattered was somewhere else:

"agents": {
  "defaults": {
    "model": { "primary": "custom-groq/openai/gpt-oss-120b" },
    "modelPolicy": {
      "allow": ["custom-integrate-api-nvidia-com/moonshotai/kimi-k3", "..."]
    }
  }
}

modelPolicy.allow is a hard allowlist, and it wins over primary. When the primary isn't on the list, the gateway doesn't fail or warn — it quietly picks something that is allowed. Every list entry was an old NVIDIA model, so it picked a dead one, and the failure surfaced as a timeout, three layers away from the cause.

What cost the hours is that every "what is my model?" check reported the primary, and the primary was fine. The resolved value and the dispatched value were two different things, and only one of them was visible.

The check that would have found it in ten seconds:

$ grep -n "kimi-k3" ~/.openclaw/openclaw.json

When a model change doesn't seem to take effect, grep the config for the old model id instead of reading back the new one. Any hit that isn't the line you just edited is the thing overriding you. The fix itself is one line: put the new primary on the allow list.

One related trap from the same afternoon: agents.defaults.model accepts only primary. Adding a fallback array fails schema validation and the gateway exits with code 78, so a list of fallbacks is not a way around the allowlist either.

My upload verified byte-exact, then the host re-encoded it three seconds later

build996 — Fri, 11 Sep 2026 09:21:13 +0000

I deployed a PHP app with about 1,500 book covers to a free host, and my deploy script did the sensible thing: upload, read each file back, compare. Every file matched. The deploy passed.

The covers were a different size the next time I looked.

Not corrupted, not missing — re-encoded. The host runs uploaded images back through GD and writes the result over your file. You can see it in the JPEG comment:

$ curl -s https://yourhost/covers/1.jpg | strings | grep -i creator
CREATOR: gd-jpeg v1.0 (using IJG JPEG v62), quality = 85

The part that actually cost me time is that it is asynchronous. Right after the transfer the file is byte-identical to what I sent:

$ curl -s -o a.jpg https://yourhost/covers/1.jpg && sleep 5 \
  && curl -s -o b.jpg https://yourhost/covers/1.jpg && cmp a.jpg b.jpg
a.jpg b.jpg differ: byte 3, line 1

At t+2s, cmp was silent. By t+5s it wasn't. So a deploy that verifies immediately — which is the only sane time to verify — passes, and is wrong seconds later. Nothing in the panel or the docs mentions this.

Sizes, measured on the same set of files:

format	change after rewrite
JPEG	+24%
PNG	+9%
GIF	−6%

Dimensions are preserved. 25 MB of covers became about 30 MB, which matters on a free plan where the disk quota is the binding constraint and the panel's usage figures lag by a long way anyway.

It is worth saying that this is probably not a compression feature gone wrong. Bytes appended after the JPEG EOI marker get deleted, which is exactly what you would do to kill polyglot payloads — a JPEG that is also a PHP file stops being one. Read that way it is a reasonable defence that nobody told you about.

Two things I would take away. First, if your deploy verification runs at t+0, it is not verifying the state your visitors get; sleep first, or re-check later. Second, this is a property of the platform, not the brand — the two hosts I tested that sit on iFastNet both do it, and the two on ZETTA leave files completely alone, which is a useful reminder that four free-host brands can be two actual companies.

The four hosts, which brand runs on which platform, and the FTP and MySQL traps on each: https://toolfreebie.com/tools/free-php-hosting-tested/

Googlebot read my sitemap and fetched zero articles. Bingbot fetched 64.

build996 — Mon, 07 Sep 2026 15:41:39 +0000

My site has one page in Google's index and 79 in the sitemap. I assumed the content was being crawled and judged. The access log says it was never crawled at all.

Two days of raw log, every Googlebot request:

66.249.72.1 - [04/Sep/2026:19:57:09] "GET /robots.txt HTTP/1.1" 200 4951
66.249.72.8 - [04/Sep/2026:19:57:10] "GET /post-sitemap.xml HTTP/1.1" 200 2793
66.249.72.2 - [04/Sep/2026:23:04:52] "GET /category-sitemap.xml HTTP/1.1" 200 5267

Three requests. It read robots.txt, read both sitemaps, and fetched zero articles.

Same window, same server, other crawlers:

YandexBot      154 requests
bingbot        139 requests   (64 distinct article URLs)
ClaudeBot        7 requests
Googlebot        3 requests   (0 article URLs)
PerplexityBot    1 request
GPTBot           0 requests

Bingbot pulled 64 different articles plus cover images and the WP REST endpoints. Same URLs, same robots.txt, same response times.

That rules out most of what I would otherwise have spent a week on. Not robots.txt, Googlebot fetched it and it disallows nothing relevant. Not the sitemap, it fetched that too and got a 200 with 79 URLs in it. Not the content, it has never seen the content. Not server errors, everything returned 200. It knows exactly which URLs exist and is choosing not to spend requests on them.

If you have shell access, this is the whole check:

grep -i googlebot access.log   | grep -oE '"GET [^ ]+' | sort | uniq -c | sort -rn

If the only lines that come back are robots.txt and your sitemaps, no amount of on-page work will move anything, because the crawler never reaches the pages you are editing. Search Console will not tell you this either — it reports "URL is unknown to Google", which reads like a submission problem rather than a budget decision.

Worth knowing before you rewrite titles for the fourth time.

Groq's free tier bills the tokens you ask for, not the ones you get

build996 — Thu, 03 Sep 2026 02:21:50 +0000

I lost an afternoon to a 413 that made no sense. A twenty-token prompt, a model with a 131k context window, and Groq answering Request too large. The request was not too large. The number I had declared was.

Here is the whole mechanism, plus a sweep of all fourteen models Groq advertises on the free tier, because the failure modes are not uniform and the documentation does not separate them.

The rule

Groq's free (on_demand) tier gives you 8,000 tokens per minute on the chat models. That budget is charged against max_tokens — the ceiling you declare — not against what the model actually generates.

Two requests to openai/gpt-oss-120b, same key, seconds apart:

prompt 4,078 tokens, max_tokens 16    -> 200 OK
prompt    20 tokens, max_tokens 8192  -> 413

The second one never produced a token. The error:

Request too large for model `openai/gpt-oss-120b` ... on tokens per minute (TPM):
Limit 8000, Requested 8271

8,271 = a 20-token prompt plus the 8,192 I said I might want. The prompt was almost free. The option on 8,192 output tokens cost the entire minute.

This is the opposite of how most people reason about max_tokens. It is not a safety cap that costs nothing if unused. On this tier it is a reservation, billed up front.

It is a rolling window, not a per-model allowance

My first pass made qwen/qwen3.8-27b look exempt — it passed the same request that 413'd on gpt-oss-120b. It is not exempt. A later sweep caught it:

Request too large for model `qwen/qwen3.8-27b` ... Limit 8000, Requested 8212

The first probe simply landed in a minute that still had headroom. You can watch the budget drain inside one window:

Rate limit reached ... Limit 8000, Used 4373, Requested 6278

Used is the giveaway. This is one shared time-window budget, and a passing call proves nothing about the next one. If you are writing a compatibility matrix, do not mark a model "works" off a single green result.

The sweep

Fourteen model IDs from /models, two requests each — one trivial, one with max_tokens: 8192:

Model	Trivial request	`max_tokens: 8192`
`openai/gpt-oss-120b`	200 (478 ms)	413 — Requested 8271
`openai/gpt-oss-safeguard-20b`	200 (308 ms)	413 — Requested 8271
`qwen/qwen3.8-27b`	200 (324 ms)	413 — Requested 8212
`qwen/qwen3.6-27b`	200 (330 ms)	413 — Requested 8210
`openai/gpt-oss-20b`	200 (308 ms)	200
`groq/compound-mini`	200 (897 ms)	200
`meta-llama/llama-prompt-guard-2-22m`	200 (311 ms)	400 — cap is 512
`meta-llama/llama-prompt-guard-2-86m`	200 (330 ms)	400 — cap is 512
`allam-2-7b`	200 (298 ms)	400 — cap is 4096
`whisper-large-v3`	400	—
`whisper-large-v3-turbo`	400	—
`canopylabs/orpheus-v1-english`	400	—
`canopylabs/orpheus-arabic-saudi`	400	—
`groq/compound`	429	—

Nine of fourteen answer a chat request at all. Four of those nine break at max_tokens: 8192. Two pass it.

Three separate failures wearing the same clothes

The table above collapses three unrelated problems that all look like "the model didn't work":

400 because it is not a chat model. /models returns speech-to-text (whisper-*), text-to-speech (orpheus-*) and classifier models (llama-prompt-guard-2-*) in one flat list with no modality field. Nothing in the response tells you which endpoint a given ID belongs to. If you enumerate /models and pipe the IDs into chat/completions, four of them 400 and you will assume your request shape is wrong. It isn't; the model just doesn't do chat.

400 because the model has a lower ceiling than you assumed. The prompt-guard classifiers cap max_tokens at 512. allam-2-7b caps at 4096. These are model properties, not tier properties, and they produce a different error than the TPM one:

`max_tokens` must be less than or equal to `512`,
the maximum value for `max_tokens` is less than the `context_window` for this model

413 because you declared too much. The TPM case above.

Only the third one is about the free tier. The first two would fail on a paid plan too.

`groq/compound` is a router, and it tells you

groq/compound returned 429 on a trivial request. What is interesting is the body: the error names openai/gpt-oss-120b, not groq/compound. It is a routing layer over that model, sharing its budget. That is the only place I found the relationship stated — it is not in the model list, and it means "switch to compound to get around the gpt-oss limit" does not work.

groq/compound-mini behaves differently: it passed max_tokens: 8192 where gpt-oss-120b failed, which fits a quota shaped for fewer, larger calls rather than many small ones.

Why this specifically breaks agents

For a single completion, none of this matters much — you set max_tokens to what you need and move on.

An agent loop is the bad case. Tool schemas go in every request, so the prompt is never small. Frameworks default maxTokens high because a truncated tool call is worse than a slow one. Multiply by twenty turns and you are not near the TPM ceiling, you are through it, on a tier whose headline number is throughput.

If you point an agent framework at Groq's free tier, declare a small ceiling explicitly — roughly 1,500 output tokens and a context window well under the model's real one — so the harness trims the request instead of collecting a 413. This costs you nothing when responses are short, which in an agent loop they usually are.

Worth knowing: at least one harness surfaces that 413 as "context overflow", which sends you hunting for an oversized system prompt that does not exist. If you see a context error on a request you know is small, check the status code before you believe the message.

For contrast, this is not how everyone counts

Alibaba's Model Studio publishes TPM too, and its rate-limit page is explicit that the number "Includes input and output tokens" — actual tokens, both directions. The ceilings are also a different order of magnitude: qwen3.5-plus runs 15,000 RPM and 5,000,000 TPM; qwen-turbo 600 RPM and the same 5,000,000 TPM.

So "8,000 TPM" and "5,000,000 TPM" are not 625x apart in practice — they are measuring different things, and only one of them charges you for tokens that never existed. When you compare free tiers, check what the meter counts before you compare the numbers on it.

Reproducing it

One request is enough to see the mechanism:

curl -s https://api.groq.com/openai/v1/chat/completions \
  -H "Authorization: Bearer $GROQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"openai/gpt-oss-120b",
       "messages":[{"role":"user","content":"hi"}],
       "max_tokens":8192}'

Then drop max_tokens to 16 and send the same thing. One 413, one 200, same prompt.

Run it twice in the same minute and you will also see the Used field move, which is the part that makes single-probe conclusions unsafe.

I keep the full sweep and the numbers behind the table in a benchmark of free AI API tiers that I re-run rather than copy from docs — these limits move, and the ones in this post are from 2026-09-01.

A /models list with no modality field, and five entries that can't chat

build996 — Wed, 02 Sep 2026 11:34:28 +0000

If you build a model picker by reading GET /models and dropping every id into a chat completion, you will ship a picker where a third of the options are broken.

I tried it against Groq. The endpoint advertises 14 models. Sending each one the same minimal chat request:

answered a minimal request : 9/14
failed outright            : 5/14

The five failures are not outages. They are models that were never chat models:

whisper-large-v3              400
whisper-large-v3-turbo        400
canopylabs/orpheus-v1-english 400
canopylabs/orpheus-arabic-saudi 400
groq/compound                 429

Two speech-to-text, two text-to-speech, and one router that was rate limited at the time. They all sit in the same flat list as the chat models, with no field saying which modality they are. The only way to know is to recognise the names.

Two smaller things from the same sweep:

The guard models (llama-prompt-guard-2-22m, -86m) and allam-2-7b answer chat requests but reject max_tokens: 8192 with a different error — max_tokens must be less than or equal to 512 for the guards, 4096 for allam. So per-model output ceilings are real and undiscoverable from the list.

And groq/compound turns out to be a router. Its 429 named openai/gpt-oss-120b rather than itself, which is the only place the routing is visible from the outside.

None of this is documented in the list response. If you enumerate models programmatically, probe each one once and cache what actually answers, rather than trusting the catalogue.

A rate limit that counts the tokens you asked for, not the ones you got

build996 — Wed, 02 Sep 2026 11:34:27 +0000

Spent a while assuming a 413 meant my prompt was too long. It wasn't. The prompt was 20 tokens.

Groq's free tier caps you at 8,000 tokens per minute, and that budget is charged against the max_tokens you declare, not the number the model actually generates. So this fails:

prompt: 20 tokens, max_tokens: 8192
-> 413  "on tokens per minute (TPM): Limit 8000, Requested 8271"

Nothing was generated. The request was rejected on the ceiling I asked for.

The same model with max_tokens: 16 and a 4,078-token prompt returns 200 without complaint. Prompt size was never the problem.

I swept the 14 models Groq advertises. Four of them answered a minimal request in ~300ms and then returned 413 on the identical request with max_tokens: 8192:

openai/gpt-oss-120b          478ms   ->  413  (Requested 8271)
qwen/qwen3.8-27b             324ms   ->  413  (Requested 8212)
qwen/qwen3.6-27b             330ms   ->  413  (Requested 8210)
openai/gpt-oss-safeguard-20b 308ms   ->  413  (Requested 8271)

Two more passed the same request, which is the second thing worth knowing: it is a rolling window shared across models, not a per-model constant. A model that passes one probe can 413 a minute later — one of mine came back Limit 8000, Used 4373, Requested 6278. Never conclude a model is exempt from a single passing call.

The practical consequence is for agents. Tool schemas already eat the prompt, and most harnesses declare a generous max_tokens by default. Set it to what you actually need — 1,500 works — or every call dies before the model sees it.

Worth checking whether your provider bills the declared ceiling or the real output. Alibaba's docs, for contrast, say plainly that their TPM "includes input and output tokens."

Tokens per second told me nothing about agent performance

build996 — Wed, 02 Sep 2026 09:26:07 +0000

I benchmarked a dozen free AI APIs on generation throughput, then plugged the same models into an agent harness and gave them real tasks. The rankings barely overlapped.

Two examples from the same platform:

Gemma 4 31B — 50.9 tok/s, second fastest I measured. Inside the agent: hung until the 900-second timeout. Never finished.
DeepSeek V4 Flash — 27.3 tok/s, the slowest of the group. Inside the agent: completed the task correctly.

And Nemotron Super 120B, which swung between 27 and 100 tok/s across runs of the streaming benchmark, was the fastest model in the agent loop at 43 seconds.

It makes sense once you look at what each number measures. Tokens per second is typing speed. Agent wall-clock time is decision quality — how many tool calls it makes, how many wrong turns it takes, whether it re-reads a file it already understood. A verbose fast model loses to a terse slow one.

So if you're picking a model for agent work, throughput leaderboards are close to useless. Twenty minutes with your actual task will tell you more.

Method and full numbers: https://toolfreebie.com/free-ai-models-agent-test/

OpenRouter served NVIDIA's own model faster than NVIDIA did

build996 — Wed, 02 Sep 2026 09:26:06 +0000

While benchmarking free AI APIs I ran the identical model — nemotron-3-super-120b, NVIDIA's own — on two endpoints:

NVIDIA's own API: 7.6 s to first token, 36.7 tok/s
OpenRouter's free tier: 2.7 s to first token, 45.7 tok/s

Same weights. Same request. Different serving infrastructure.

I'd assumed the first-party endpoint would be the fastest path to a vendor's own model. It wasn't, at least not on the free tier that day. Free capacity is shared and unevenly provisioned, and a reseller with spare headroom can beat the source.

Two practical takeaways:

If a specific model matters to you, test every endpoint that serves it rather than assuming the vendor's own is best.
Treat any single measurement as a snapshot. Re-running the same request on NVIDIA days apart gave anywhere from 27 to 100 tok/s. The swing is itself the finding.

Cross-platform numbers for a dozen free APIs: https://toolfreebie.com/free-ai-api-speed-test/

One curl shows which of 14 providers actually serves your Hugging Face model

build996 — Fri, 28 Aug 2026 16:09:48 +0000

When you call a model through Hugging Face's Inference Providers router, someone else runs it. Which someone is not obvious from the model ID, and it changes.

The routing table is public. No token:

curl -s https://router.huggingface.co/v1/models

As of 2026-08-28 that returns 135 models across 14 providers. Counted by how many models each one backs:

featherless-ai  66     cohere      12
novita          65     baseten     11
deepinfra       62     scaleway     8
zai-org         19     ovhcloud     7
together        17     publicai     7
nscale          17     groq         3
fireworks-ai    14     cerebras     2

The long tail is the interesting bit. openai/gpt-oss-120b is served by 11 of the 14: baseten, cerebras, deepinfra, featherless-ai, fireworks-ai, groq, novita, nscale, ovhcloud, scaleway and together. zai-org/GLM-5.2 by 8. moonshotai/Kimi-K3 by 5.

So for a popular open-weight model you are not picking a provider, you are accepting a routing decision — and providers differ enormously in latency and context handling for the same weights. If you have measured gpt-oss-120b on Groq and are getting different numbers through the router, that is why: you may not be on Groq.

To pin one, pull the provider list per model:

curl -s https://router.huggingface.co/v1/models \
| jq -r '.data[] | select(.id=="openai/gpt-oss-120b") | .providers[].provider'

Then address that provider explicitly rather than letting the router choose.

Worth noting Groq backs only 3 models here and Cerebras only 2, despite both being the usual answer to "which is fastest". Their catalogs are small by design; the router's bulk comes from featherless-ai, novita and deepinfra, which carry sixty-plus each.

I benchmarked the fast providers against each other here: Groq vs Cerebras vs Gemini speed

SambaNova serves its model list without a key, but the inference now needs a card

build996 — Fri, 28 Aug 2026 16:09:10 +0000

Almost every inference provider puts /v1/models behind auth. Groq returns 403 without a key, Together 401, DeepSeek 401, Cerebras 403.

SambaNova just answers:

curl -s https://api.sambanova.ai/v1/models

No key, no account, no header. Today it returns exactly 7 models:

DeepSeek-V3.1                  131072
DeepSeek-V3.2                   32768
Meta-Llama-3.3-70B-Instruct    131072
MiniMax-M2.7                   196608
MiniMax-M3                    1048576
gemma-4-31B-it                 131072
gpt-oss-120b                   131072

Seven is a small number, and that's the useful part. Provider comparisons tend to quote catalog sizes in the hundreds, which mostly measures how many old checkpoints someone left listed. This is the whole menu.

Two things in there worth noticing. MiniMax-M3 carries a 1,048,576-token context window, which is a million tokens on a provider whose pitch is throughput rather than context. And DeepSeek-V3.2 is listed at 32,768 — a quarter of V3.1's 131,072. Newer version, smaller window, so if you bump the model string assuming context goes up or stays flat, that one will bite.

Now the correction, because I got this wrong when I first looked at it: a public catalog is not a free tier. I read the open /v1/models endpoint as a signal that SambaNova was still the no-card option it used to be. It isn't. Actual inference calls now come back with:

402 PAYMENT_METHOD_REQUIRED

on a zero balance, and the pricing page lists per-token rates for all seven models with no free row anywhere. The standing free tier — 20 requests/day, 200K tokens/day, no card — is gone.

So the endpoint is still worth knowing about, just for a narrower reason than I thought: it lets you diff the model IDs your config hardcodes against the live catalog, in CI, with no secret to provision. Most providers make you burn a key for that. It just won't tell you whether you can afford to call any of them.

The corrected write-up, with what it costs now: SambaNova Cloud: the free tier is gone

GitHub Models returns 410 with a JSON body, which is worse than a timeout

build996 — Fri, 28 Aug 2026 16:08:32 +0000

GitHub Models was fully retired on 2026-07-30. That part is well covered. What is less obvious is how it fails now, and it is the annoying kind.

The host still resolves. One command, no token:

curl -i https://models.github.ai/catalog/models

Today that answers HTTP 410 Gone with this body:

{"error":{"code":"github_models_retirement_brownout",
          "message":"GitHub Models is temporarily unavailable as part of a scheduled retirement brownout."}}

Two things worth noticing.

It is a well-formed JSON response. Not a connection refused, not a DNS failure, not an HTML error page. Any client that wraps the call in a try/except around response.json() parses this successfully. If your code then does data.get("choices", []) and moves on, the failure is silent — you get empty output and no exception, forever.

The message still says "temporarily". That is leftover brownout copy from the July 16 and July 23 rehearsals, and it is now a month stale. A human reading a log line will reasonably conclude it is a transient outage and retry. It will never succeed.

So the audit is: grep your repos for models.github.ai and models.inference.ai.azure.com, then check whether the calling code inspects response.status_code at all. The base URL being dead is easy to fix. Code that treats a 410 as an empty result is the thing that has been quietly returning nothing since July.

For what it's worth, nothing replaces the specific thing GitHub Models did — a frontier closed model, free, authenticated with a token you already had in CI. Free tiers now are open-weight only.

I wrote up the migration options and current free limits here: GitHub Models is retired: what to use instead

Five of OpenRouter's free models take a million tokens of context

build996 — Fri, 28 Aug 2026 14:55:25 +0000

I went looking for the cheapest way to throw a large codebase at a model and found this in OpenRouter's public catalog — no key needed to check:

curl -s https://openrouter.ai/api/v1/models \
| jq -r '.data[]
  | select((.pricing.prompt|tonumber)==0 and (.pricing.completion|tonumber)==0)
  | select(.context_length >= 1000000)
  | "\(.context_length)\t\(.id)"' | sort -rn

As of 2026-08-28, five text models come back at $0 per token with a million-token window or more:

1048576  thinkingmachines/inkling
1048576  thinkingmachines/inkling-small
1048576  minimax/minimax-m3
1000000  nvidia/nemotron-3-ultra-550b-a55b
1000000  nvidia/nemotron-3.5-lightning

(Two Google Lyria audio previews also return $0 at 1M; I've left them out since they aren't chat models.)

For scale: 1,048,576 tokens is roughly 4 MB of source. Nemotron 3 Ultra is a 550B-parameter MoE. Free.

The catch is throughput, not context. Free models on OpenRouter run at 20 requests/minute and 50 requests/day until the account has purchased $10 in credits at some point, after which the daily cap goes to 1,000. So this is a "read something enormous a few times a day" budget, not a "serve users" budget — which happens to be exactly the shape of a one-off repo analysis or a nightly summarization job.

Worth knowing that 119 paid models in the same catalog also clear 1M, so if you outgrow the request cap the migration is a model ID, not an architecture.

Current free-model list and how the limits work: OpenRouter free models