Groq's free tier bills the tokens you ask for, not the ones you get

I lost an afternoon to a 413 that made no sense. A twenty-token prompt, a model with a 131k context window, and Groq answering Request too large. The request was not too large. The number I had declared was.

Here is the whole mechanism, plus a sweep of all fourteen models Groq advertises on the free tier, because the failure modes are not uniform and the documentation does not separate them.

The rule

Groq's free (on_demand) tier gives you 8,000 tokens per minute on the chat models. That budget is charged against max_tokens — the ceiling you declare — not against what the model actually generates.

Two requests to openai/gpt-oss-120b, same key, seconds apart:

prompt 4,078 tokens, max_tokens 16    -> 200 OK
prompt    20 tokens, max_tokens 8192  -> 413

The second one never produced a token. The error:

Request too large for model `openai/gpt-oss-120b` ... on tokens per minute (TPM):
Limit 8000, Requested 8271

8,271 = a 20-token prompt plus the 8,192 I said I might want. The prompt was almost free. The option on 8,192 output tokens cost the entire minute.

This is the opposite of how most people reason about max_tokens. It is not a safety cap that costs nothing if unused. On this tier it is a reservation, billed up front.

It is a rolling window, not a per-model allowance

My first pass made qwen/qwen3.8-27b look exempt — it passed the same request that 413'd on gpt-oss-120b. It is not exempt. A later sweep caught it:

Request too large for model `qwen/qwen3.8-27b` ... Limit 8000, Requested 8212

The first probe simply landed in a minute that still had headroom. You can watch the budget drain inside one window:

Rate limit reached ... Limit 8000, Used 4373, Requested 6278

Used is the giveaway. This is one shared time-window budget, and a passing call proves nothing about the next one. If you are writing a compatibility matrix, do not mark a model "works" off a single green result.

The sweep

Fourteen model IDs from /models, two requests each — one trivial, one with max_tokens: 8192:

Model	Trivial request	max_tokens: 8192
openai/gpt-oss-120b	200 (478 ms)	413 — Requested 8271
openai/gpt-oss-safeguard-20b	200 (308 ms)	413 — Requested 8271
qwen/qwen3.8-27b	200 (324 ms)	413 — Requested 8212
qwen/qwen3.6-27b	200 (330 ms)	413 — Requested 8210
openai/gpt-oss-20b	200 (308 ms)	200
groq/compound-mini	200 (897 ms)	200
meta-llama/llama-prompt-guard-2-22m	200 (311 ms)	400 — cap is 512
meta-llama/llama-prompt-guard-2-86m	200 (330 ms)	400 — cap is 512
allam-2-7b	200 (298 ms)	400 — cap is 4096
whisper-large-v3	400	—
whisper-large-v3-turbo	400	—
canopylabs/orpheus-v1-english	400	—
canopylabs/orpheus-arabic-saudi	400	—
groq/compound	429	—

Nine of fourteen answer a chat request at all. Four of those nine break at max_tokens: 8192. Two pass it.

Three separate failures wearing the same clothes

The table above collapses three unrelated problems that all look like "the model didn't work":

400 because it is not a chat model. /models returns speech-to-text (whisper-*), text-to-speech (orpheus-*) and classifier models (llama-prompt-guard-2-*) in one flat list with no modality field. Nothing in the response tells you which endpoint a given ID belongs to. If you enumerate /models and pipe the IDs into chat/completions, four of them 400 and you will assume your request shape is wrong. It isn't; the model just doesn't do chat.

400 because the model has a lower ceiling than you assumed. The prompt-guard classifiers cap max_tokens at 512. allam-2-7b caps at 4096. These are model properties, not tier properties, and they produce a different error than the TPM one:

`max_tokens` must be less than or equal to `512`,
the maximum value for `max_tokens` is less than the `context_window` for this model

413 because you declared too much. The TPM case above.

Only the third one is about the free tier. The first two would fail on a paid plan too.

groq/compound is a router, and it tells you

groq/compound returned 429 on a trivial request. What is interesting is the body: the error names openai/gpt-oss-120b, not groq/compound. It is a routing layer over that model, sharing its budget. That is the only place I found the relationship stated — it is not in the model list, and it means "switch to compound to get around the gpt-oss limit" does not work.

groq/compound-mini behaves differently: it passed max_tokens: 8192 where gpt-oss-120b failed, which fits a quota shaped for fewer, larger calls rather than many small ones.

Why this specifically breaks agents

For a single completion, none of this matters much — you set max_tokens to what you need and move on.

An agent loop is the bad case. Tool schemas go in every request, so the prompt is never small. Frameworks default maxTokens high because a truncated tool call is worse than a slow one. Multiply by twenty turns and you are not near the TPM ceiling, you are through it, on a tier whose headline number is throughput.

If you point an agent framework at Groq's free tier, declare a small ceiling explicitly — roughly 1,500 output tokens and a context window well under the model's real one — so the harness trims the request instead of collecting a 413. This costs you nothing when responses are short, which in an agent loop they usually are.

Worth knowing: at least one harness surfaces that 413 as "context overflow", which sends you hunting for an oversized system prompt that does not exist. If you see a context error on a request you know is small, check the status code before you believe the message.

For contrast, this is not how everyone counts

Alibaba's Model Studio publishes TPM too, and its rate-limit page is explicit that the number "Includes input and output tokens" — actual tokens, both directions. The ceilings are also a different order of magnitude: qwen3.5-plus runs 15,000 RPM and 5,000,000 TPM; qwen-turbo 600 RPM and the same 5,000,000 TPM.

So "8,000 TPM" and "5,000,000 TPM" are not 625x apart in practice — they are measuring different things, and only one of them charges you for tokens that never existed. When you compare free tiers, check what the meter counts before you compare the numbers on it.

Reproducing it

One request is enough to see the mechanism:

curl -s https://api.groq.com/openai/v1/chat/completions \
  -H "Authorization: Bearer $GROQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"openai/gpt-oss-120b",
       "messages":[{"role":"user","content":"hi"}],
       "max_tokens":8192}'

Then drop max_tokens to 16 and send the same thing. One 413, one 200, same prompt.

Run it twice in the same minute and you will also see the Used field move, which is the part that makes single-probe conclusions unsafe.

I keep the full sweep and the numbers behind the table in a benchmark of free AI API tiers that I re-run rather than copy from docs — these limits move, and the ones in this post are from 2026-09-01.