Prefilling Claude's Response: Steer Output Without JSON Mode

Most people fight Claude's output with longer system prompts. "Return only JSON." "Do not add any preamble." "Answer with a single word." Then Claude replies Sure! Here is the JSON you requested: and your json.loads throws. You add another sentence. It happens again on 3% of calls, which is exactly the tail that pages you at 2am.

There's a lever that ends this class of bug, and it isn't structured output mode: prefilling Claude's response. You put words in Claude's mouth by seeding the assistant turn, and the model continues from your tokens instead of deciding how to open. It's one of the highest-leverage, least-used primitives in the Anthropic Messages API.

TL;DR

• Prefilling Claude's response = you append a final message with role: "assistant" and partial content; Claude continues from it. The API returns only the continuation, so you concatenate your prefill back on.
• Prefill { to force JSON, prefill a label to force a classification, prefill <result> to force a tag — no constrained decoding, no schema, no extra latency.
• The returned completion does not include your prefill text or any stop-sequence text — a common parsing bug.
• Two hard constraints: the prefill cannot end in trailing whitespace (API error), and prefilling is incompatible with extended thinking.
• It's cheaper and lower-latency than tool-forced structured output, but it doesn't guarantee a grammar the way constrained decoding does — pair it with a stop sequence and a validator.

What does prefilling Claude's response actually do?

Prefilling exploits the fact that a chat model is still just next-token prediction over a running sequence. The Messages API normally ends with a user turn and Claude generates the whole assistant turn. If instead you make the last message an assistant message with partial content, that content becomes the committed prefix of Claude's turn. Decoding starts at the token after your prefix.

So if the last assistant token you supply is {, Claude's first generated token is conditioned on a sequence that already opened a JSON object. The probability mass on Sure! Here is collapses — there's no natural continuation of { that starts with an apology. You didn't ban the preamble with instructions; you made it structurally impossible.

import json
import anthropic

client = anthropic.Anthropic()

def extract_invoice(text: str) -> dict:
    resp = client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=512,
        system="Extract invoice fields. Output a single JSON object.",
        messages=[
            {"role": "user", "content": text},
            # The prefill: Claude continues FROM this token.
            {"role": "assistant", "content": "{"},
        ],
        stop_sequences=["\n\n"],  # optional guard
    )
    # The API returns only what Claude generated AFTER "{".
    # You must prepend the prefill yourself.
    raw = "{" + resp.content[0].text
    return json.loads(raw)

Note the two easy-to-miss details: the response text starts after {, so you prepend it, and stop_sequences text (if hit) is also excluded from the output.

How do I force JSON without structured output mode?

Prefill an opening { (or [). That single token does most of the work of a schema instruction. It's not a grammar — Claude can still emit invalid JSON on a hard input — but in practice it removes the entire "conversational wrapper" failure mode, which is where the vast majority of parse errors come from.

Why reach for this instead of tool-forced structured output? Three reasons:

1. Latency and tokens. Constrained decoding and tool schemas add input tokens (the schema) and, depending on the backend, per-token masking overhead. A one-character prefill adds one token.
2. You want prose, not a struct. If you need a Markdown table or a <summary> block, there's no JSON schema to force — but a prefill of | Field | or <summary> pins the format.
3. Composability. Prefill stacks with everything: streaming, prompt caching, multi-turn. A forced-tool call is a different response shape you have to special-case.

The tradeoff is real: prefill steers, constrained decoding guarantees. If you have a strict downstream contract and can't tolerate any invalid output, use tool use with an input schema and validate. If you're mostly fighting preamble and format drift, prefill is faster and simpler. I lean prefill for anything human-adjacent and tool-forced schemas for machine-to-machine contracts.

How do I get a single-token classification out of Claude?

Prefill the start of the answer and cap generation. For a sentiment classifier with labels positive / negative / neutral, don't ask for "just the label" and hope — prefill the assistant turn and stop early:

resp = client.messages.create(
    model="claude-haiku-4-5",
    max_tokens=1,
    system="Classify sentiment. Reply with exactly one of: positive, negative, neutral.",
    messages=[
        {"role": "user", "content": review},
        {"role": "assistant", "content": "The sentiment is "},
    ],
)
label = resp.content[0].text.strip()  # "positive"

The prefill The sentiment is forces Claude past any hedging directly into the label token. On Haiku this is a fast, cheap classifier — one input pass, a handful of output tokens. For higher reliability, prefill a tighter frame like Label: and constrain the label set in the system prompt. If you need calibrated confidence rather than a hard label, this is the point where tool use or logprob-based scoring earns its extra cost; prefill alone gives you the label, not a probability.

Why does my prefill throw a trailing-whitespace error?

Because the Anthropic API rejects any assistant message whose content ends in whitespace. This trips people constantly, because the natural prefill is exactly "The sentiment is " with a trailing space — which errors.

You have two options. Either drop the trailing space and accept that Claude's first token may or may not include a leading space (usually it does, because of how the tokenizer merges word boundaries), or move the whitespace into the model's job by ending the prefill on a non-space character like "The sentiment is:". I default to ending prefills on punctuation or a structural character ({, :, >) precisely to sidestep this.

# ❌ API error: assistant content ends in whitespace
{"role": "assistant", "content": "Answer: "}

# ✅ ends on a non-space char
{"role": "assistant", "content": "Answer:"}

Why can't I prefill when extended thinking is on?

Because extended thinking injects a reasoning block that the model must generate itself before the visible answer, and prefilling the assistant turn would collide with that internal structure. The API disallows the combination — you can't hand Claude a partial answer and ask it to think first, since the prefill would sit where the thinking block belongs.

This is the single biggest architectural gotcha. Teams build a prefill-based extractor, then flip on extended thinking for a hard subset of inputs, and the whole path 400s. If you need both reasoning and a pinned format, drop the prefill and force format the other way: ask for a <answer>...</answer> tag in the prompt and parse it, or use tool use, both of which coexist with thinking. Pick one lever per call.

Does the response include my prefill or the stop sequence?

No, and both omissions are a parsing bug waiting to happen. The completion text you get back is only the tokens Claude generated after your prefix. If you prefilled {, the { is not in resp.content[0].text — you prepend it. And if generation halted on a stop_sequences entry, stop_reason is "stop_sequence" and the matched string is not in the output either.

So the correct reconstruction is: prefill + response_text, and if you relied on a closing token being the stop sequence (say you stopped on }), you re-append it yourself. Log stop_reason in production. If you see "max_tokens" where you expected "stop_sequence" or "end_turn", your output is truncated mid-structure and any json.loads is a coin flip.

When should I not use prefilling?

When you need a hard guarantee, when extended thinking is on, or when the "format" is genuinely a typed object with nested constraints. Prefill is a steering primitive with excellent cost characteristics, not a validator. It dramatically shrinks the failure surface — it does not eliminate it. The production pattern that actually holds up: prefill to pin the opening, set a stop sequence to pin the close, validate the reconstructed string, and on validation failure retry once with a tightened prefill or fall back to tool-forced structured output. That combination gets you most of constrained decoding's reliability at a fraction of the token and latency cost.

Direct answer: what is prefilling Claude's response and why use it?

Prefilling Claude's response means ending your Messages API call with an assistant-role message containing partial content, so Claude continues from your tokens instead of choosing how to open its turn. You use it to force JSON by prefilling {, to force a classification by prefilling The answer is, or to force a tagged block by prefilling <result> — killing preamble and format drift at their source with a single extra token, no schema, and no added latency. Remember the API returns only the continuation (prepend your prefill and re-append any stop sequence), never end the prefill in whitespace, and never combine it with extended thinking. It's a steering lever, not a grammar guarantee, so pair it with a stop sequence and a validator for anything that feeds a strict downstream contract.