GPT-4o's Max Output Tokens, and What Happens When You Hit It

GPT-4o will not return more than 16,384 tokens in one response, however much of its 128,000-token window is free. That cap is separate from the context window, it is enforced by truncation rather than by an error, and the truncation happens mid-token-stream with no regard for whether you were in the middle of a JSON object.

The cap, per snapshot

OpenAI lists a max output tokens figure per model snapshot on its models reference. For GPT-4o the value changed between snapshots, which is the detail that catches teams pinning an old one:

• gpt-4o-2024-05-13 — 4,096 output tokens.
• gpt-4o-2024-08-06 and later snapshots — 16,384 output tokens.

The number is a property of the model, not of your account or your tier, and there is no parameter or header that raises it. It is also genuinely independent of the context window: GPT-4o will not produce a 17,000-token response even when 120,000 tokens of its window are free, because the two limits answer different questions. The window asks how much the model can attend to at once; the output cap asks how long a single uninterrupted generation is allowed to run. Both are checked, and the tighter one binds — the interaction is worked through in what the 128K window buys you.

If you pinned the May snapshot for reproducibility — a reasonable thing to do, see pinning a model snapshot — you have a quarter of the output headroom of the alias, and a request that works against gpt-4o can truncate against your pin. This is the single most common version of “it works in the playground and not in production” for long outputs.

Per-snapshot output caps are exactly the kind of figure that moves when a new snapshot ships. Treat the two numbers above as the documented values at the time of writing and read the current value off the models page for the exact string you send in model.

max_tokens, max_completion_tokens and the default

The parameter that limits a response is max_completion_tokens. It replaced max_tokens in the Chat Completions API, and the rename was not cosmetic: with the arrival of reasoning models a “completion” can contain tokens that are billed and counted but never returned to you, so a parameter called max_tokens would have had two plausible meanings. max_tokens is still accepted for non-reasoning models and is documented as deprecated; the o-series rejects it.

curl https://api.openai.com/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-2024-08-06",
    "max_completion_tokens": 2000,
    "messages": [
      {"role": "user", "content": "Summarise this contract as JSON."}
    ]
  }'

If you set nothing, the response is bounded only by the model’s own cap and by what is left of the context window — the default is not a small safe number, it is “as much as the model is willing to produce”. That is fine for chat and expensive for a runaway generation, which is why the default value of max_tokens is worth knowing rather than assuming.

What a truncated response looks like

There is no error. The model generates until it reaches your limit or the model cap, the response comes back with HTTP 200, and the only signal is one field. Ask for structured output with a low cap and you get something like this:

{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "model": "gpt-4o-2024-08-06",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "{\"parties\": [{\"name\": \"Acme Ltd\", \"role\": \"suppl"
      },
      "finish_reason": "length"
    }
  ],
  "usage": {
    "prompt_tokens": 1834,
    "completion_tokens": 32,
    "total_tokens": 1866
  }
}

Note what is and is not true of that payload. The HTTP status is 200. The content is a string, and it is valid as a string — it is simply not valid JSON, because it stops inside a value. The closing braces do not exist and never will; the model was interrupted, not asked to wrap up. And finish_reason is "length", which is the only thing in the response that tells you any of this.

The same applies to Structured Outputs. A response schema guarantees that what the model produces conforms, not that it completes: hit the cap and you get a prefix of a conforming object, which is not a conforming object. This is why schema-constrained output does not remove the need to check the finish reason.

Detecting it instead of parsing it

The wrong shape for this check is a try around JSON.parse. A parse failure tells you the string was not JSON; it does not tell you why, and truncation and refusal and a malformed schema all arrive as the same exception. Check the reason first:

const choice = res.choices[0];

if (choice.finish_reason === "length") {
  // Truncated by max_completion_tokens or the model cap.
  // Do not attempt to parse. Retry with a larger budget,
  // or with the work split.
  throw new TruncatedOutput(res.usage.completion_tokens);
}

if (choice.message.refusal) {
  // Distinct field, distinct problem — the model declined.
  throw new Refused(choice.message.refusal);
}

const data = JSON.parse(choice.message.content);

finish_reason takes a small set of documented values — stop, length, tool_calls, content_filter and the legacy function_call — and they are enumerated in the finish_reason reference. In a stream, the field arrives on the final content chunk rather than on the first, which is why a streaming client that forgets to read it will happily hand a truncated string downstream; see the shape of a streaming chunk.

Four ways out

1. Raise the cap, if you have room. Check usage.completion_tokens against the limit you set. If they are equal, your own max_completion_tokens was the binding constraint and raising it is the whole fix.
2. Check which cap you hit. If completion tokens landed on 16,384 (or 4,096 on the old snapshot) rather than on your number, the model cap bound and raising your parameter changes nothing. Move to a newer snapshot or split the work.
3. Split the output, not the input. Ask for one section per call, or one array element per call, and assemble. A job that produces 40,000 tokens of structured data is three or four requests by construction, and each one is independently retryable.
4. Ask for less. A schema with shorter field names, enums instead of free text, and no prose commentary can cut output length by a large fraction without losing information — and output tokens are the expensive ones.

Continuation — sending the truncated prefix back and asking the model to carry on — works for prose and is fragile for structured data, because the model must both resume mid-token and re-derive the open bracket state. If you need it, prefill the assistant turn with the partial output rather than describing it in a user message.

Related

• GPT-4o's Context Window: What 128K Actually Buys You