Most people paste everything into one message: the persona, the rules, the format, and the actual question, all mashed together. But the model has two separate channels, and knowing the difference is the whole trick. The user turn is the question that changes every time; the system prompt is the standing instruction that persists across every turn. I built a live playground that assembles a system prompt from parts and shows the same question answered differently under it. Here's what I learned building it.
Two channels, not one string
The user message is the changing request. The system message is the behaviour you set once — and it applies to every question that follows. That separation is the point: the same user turn yields a completely different answer depending on the system prompt sitting above it. A bare model free-associates and will answer things it shouldn't; a role-only prompt is on-brand but shapeless; a full spec is structured, on-brand, and safely refuses out-of-scope asks. It's prompt-construction, not fine-tuning — the behaviour lives in an editable string you can read, diff and toggle.
Five blocks, in a fixed order
A good system prompt layers the same components, and I built each as a switch you can flip on or off:
- ROLE — who the model is ("You are a senior support engineer for…"). Sets voice and default assumptions.
- RULES — the explicit must / must-not list. This is where behaviour actually gets pinned down.
- OUTPUT FORMAT — how to shape the reply: JSON, numbered steps, one sentence. Turns prose into structure.
- FEW-SHOT — one or two anchors that show the wanted behaviour rather than describe it.
- GUARDRAILS — what to refuse, and how. The block that flips a risky answer into a safe decline.
Only the switched-on blocks appear, in a fixed canonical order, and the whole string is sent once in the system channel — never mixed into the user's message.
Watching a guardrail bite
The demo runs a documented, deterministic mock reply (no API key, no live call) that reacts to exactly the blocks you turned on — so you can see the cause and effect. Ask a risky question under a bare model and it answers. Add the guardrail block and the same question flips to a refusal. Add the format block and prose becomes numbered steps. Add the role and a flat answer becomes on-brand. The message structure panel shows it as three turns:
system: ROLE + RULES + FORMAT + GUARDRAILS (set once, persists)
user: "the same question, every time" (changes each turn)
assistant: answer shaped by whatever system sat above it
Flip the system prompt, keep the user turn identical, and the assistant turn changes — that's the entire demonstration.
Behaviour, not content
The thing that finally clicked: the system prompt configures behaviour, and the user turn carries content. This is distinct from the reasoning techniques — chain-of-thought and the rest are about how the model works through a problem. The system prompt is about where and how you set the model up before any reasoning even begins. Get the role, rules, format and guardrails right in that standing instruction, and every user turn inherits them for free; get them wrong, and no amount of clever phrasing in the question will save the answer.
There's a second reason to keep them separate that only shows up at scale: the system prompt is the same for every request, so it's the part you can cache, version, and A/B test independently of user input. When a refusal leaks or a format drifts, you have one editable string to diff — not a thousand tangled messages. And because it's declarative text rather than weights, changing behaviour is a code review, not a training run.
The practical upshot: stop stuffing rules into the question. Put the persona, the hard constraints, the output shape and the refusals in the system channel where they belong, and let the user turn be nothing but the ask. Then the same question answers consistently, on-brand, and safely — every single turn. Build one from parts and watch the same question answer differently under each:
Top comments (0)