Assistant Prefill: write the start of the model's reply and the format stops being a request

A chat API call is not a question. It is a transcript: a system turn, a user turn, and then an empty assistant turn that the model fills in.

Assistant prefill — also called response priming — means you write the first few characters of that assistant turn yourself. The model, which cannot tell your tokens from its own, simply continues.

That is the whole technique. One extra message in the array.

The problem it solves

Ask a chat model for JSON and you get this:

Sure! Here's the extracted data as JSON:

json
{
"order_id": "A-1042",
"customer": "Priya Nair",
"total": 129.5,
"status": "shipped"
}

Let me know if you need any other fields!

javascript

The content is perfect. The envelope is a disaster. JSON.parse throws, your regex grabs the wrong line, and the markdown fence has to be stripped by hand.

That greeting is not a bug. Chat models are trained on helpful-assistant conversations, so for an empty assistant turn the highest-probability opening genuinely is "Sure! Here's..." followed by a fence and a friendly sign-off. It is what makes the model pleasant to talk to. It only becomes a bug when a program is the reader.

Asking versus prefilling

You can add "respond with JSON only, no preamble" to the prompt. That helps. It is also a request the sampler is free to ignore, and one stray "Certainly!" in a thousand calls is a production incident when a parser is downstream.

Prefill does not compete with the habit. It forecloses it:

const messages = [
  { role: "user",      content: `Extract order_id, customer, total, status as JSON.\n\n${doc}` },
  { role: "assistant", content: '{\n  "order_id": "' }   // <- the prefill
];

There is no prefill parameter. You append one more message with role assistant as the last entry. Assistant messages elsewhere in the array are ordinary history — that is how few-shot examples work. Only a trailing one is a prefill, because only the last turn is the one being generated.

The output:

{
  "order_id": "A-1042",
  "customer": "Priya Nair",
  "total": 129.5,
  "status": "shipped"
}

Why it works: continuation, not restart

The model is autoregressive. It conditions on every token in the context, including the ones you wrote into the assistant turn, and it has no way to know you wrote them. So it does not re-plan an answer — it finishes the sentence you started.

Land the prefill mid-array and it emits the next element. Land it mid-string, as above, and the very first generated token is the order ID itself, because the only sensible continuation of an unterminated JSON string is its contents.

Formats are self-reinforcing. Once a response starts with {, the likely continuation is a JSON key. Once it starts with -, another bullet. Once it starts with SELECT, SQL rather than prose about SQL. So the entire fight over output format is really a fight over the first token — and prefill wins it by fiat.

This is why a one-character prefill routinely beats three paragraphs of formatting instructions.

Pick the shortest prefix with one plausible continuation

Task	Prefill
JSON object	{
Classification label	Label:
Bulleted list	-
SQL	SELECT
Tag-delimited answer	<answer>
A letter, not a chat reply	Dear Ms Alvarez,

Longer is not better. Longer starts asserting content, and that is the sharp edge: anything you prefill is asserted as true whether or not it is. Prefill "status": " for a record with no status and the model will invent one rather than break the syntax you started. It cannot contradict you, only continue you.

Scaffold the shape. Leave every value to the model.

Three rules that travel with it

Pair it with a stop sequence. Prefill controls how the turn starts; a stop sequence controls where it ends. With Label: alone, the model writes Label: negative and then keeps going with an explanation. Add \n as a stop sequence and the response is exactly Label: negative.

Re-attach the prefill before parsing. Most APIs return only the generated text — the prefill you sent is not echoed back:

const PREFILL = "Label:";
const res  = await call({ messages, prefill: PREFILL, stop: ["\n"] });
const full = PREFILL + res.text;   // "Label: negative"
//            ^^^^^^^ the API returned only " negative"

Never end a prefill with whitespace. The join between your text and the first generated token is a real token boundary. "Label: " plus " negative" gives you "Label: negative" — double space, exact match fails. One line fixes it: prefill.replace(/\s+$/, "").

And one security note: because the model continues whatever you wrote as if it had written it, the prefill is the most direct steering lever the API exposes. It is developer-owned text, on the same footing as the system prompt. Never let an end user supply it.

A nudge, not a guarantee

Prefill shifts probabilities. It does not restrict the sampler's vocabulary. The model can still close the JSON early, emit a trailing comma, or drift back into prose over a long generation.

Grammar-constrained decoding is the stronger thing: JSON mode, structured outputs and strict tool schemas mask the logits so an invalid token literally cannot be sampled. Prefill can never make that promise. Keep the try/catch either way.

Where it stands today

Worth being precise here, because it changed.

On current Claude models — Opus 4.6 and later, Sonnet 4.6 and later, and Fable 5 — a trailing assistant turn returns a 400. Prefill was removed as an affordance. Assistant messages elsewhere in the array are unaffected.

Its jobs were split up and made stronger:

response = client.messages.create(
    model="claude-opus-5",
    max_tokens=1024,
    output_config={"format": {"type": "json_schema", "schema": ORDER_SCHEMA}},
    messages=[{"role": "user", "content": f"Extract the order fields.\n\n{doc}"}],
)

A schema for JSON, a tool with an enum for classification labels, and a one-line system instruction for "respond directly, no preamble". That is a genuine upgrade — the schema is enforced rather than nudged, and there is no prefill to re-attach.

Meanwhile the mechanism itself is alive and native anywhere you control the raw prompt string, which is most local and self-hosted runtimes. There, "prefill" is simply the prompt ending mid-assistant-turn — which is what it always was.

Learn it regardless. It is the clearest demonstration I know of why the first token owns the whole response.

The interactive version

I built a page where you can watch this run: a deterministic decoder that emits the assistant turn token by token, choosing each step by actually scanning the text so far — is a quote open, how deep is the brace nesting, which schema keys already appear, is a markdown fence open. Editing the prefill genuinely changes the decode path rather than selecting a canned answer.

Four tasks, 14 real validators (JSON.parse, exact-label match, bullet-line parsing, SQL shape). Baseline 4/14, prefill 14/14 — and the four the baseline passes are, in every case, the content check. The model knew the answer every time. It lost on the envelope.

There is a live stop-sequence field too, so you can break the exact-match check by clearing it, and a trailing-space warning that shows the double-space bug in action.

https://dev48v.infy.uk/prompt/day59-assistant-prefill.html

Day 59 of PromptFromZero — one prompt-engineering technique a day, built from scratch, no API key needed.