A system prompt is the part of an AI coding assistant's context that never scrolls away. It sits at the top of every request, shaping how the model interprets everything that follows, and most teams either skip writing one entirely or write one once and never revisit it. Both are mistakes. A deliberately built system prompt is one of the cheapest, highest-leverage levers for keeping an assistant from wandering off into unrelated refactors, inventing conventions your codebase doesn't use, or ignoring constraints you assumed were obvious.
Here's a concrete process for building one that actually holds up across a working session, not just a demo.
Step 1: Separate Identity From Instructions
Start by writing two distinct sections instead of one paragraph of vague guidance. The first defines what the assistant is for on this specific codebase: a backend service in a particular language, following particular conventions, with particular things it should never touch. The second defines how it should behave when uncertain: ask before assuming, flag ambiguous requirements instead of guessing at them, prefer smaller diffs over rewrites.
Mixing these together produces a prompt that reads well but fails to constrain behavior, because "be helpful and careful" doesn't tell the model what careful means in your specific context. "Never modify files under /migrations without explicit confirmation" does.
Step 2: Name the Conventions Explicitly, Don't Assume They're Obvious
Every codebase has unwritten rules that an experienced human contributor picks up by osmosis over a few weeks: which error-handling pattern is standard, whether the team prefers composition over inheritance, how tests are structured, which utility functions already exist so the assistant doesn't reinvent them. None of this is obvious to a model seeing your repository for the first time in a session.
Writing these conventions into the system prompt, even briefly, measurably reduces the number of suggestions that technically work but don't match how the rest of the codebase is written. "Use the existing ApiError class for all thrown errors instead of raw exceptions" is a one-line addition that saves a recurring category of review comments. This is the same instinct behind prompt engineering generally: being specific costs almost nothing and pays off every single time the model would otherwise have to guess.
Step 3: Define Boundaries as Explicitly as Capabilities
Most system prompts describe what the assistant should do and skip what it shouldn't. Boundaries matter just as much, particularly around anything with real consequences: database migrations, authentication logic, anything touching production configuration, third-party API keys. Stating plainly that the assistant should propose changes to these areas but never apply them directly gives you a consistent safety net that doesn't depend on remembering to double-check every single response.
Step 4: Keep It Stable, Update It Deliberately
A system prompt that changes every session loses its main advantage, which is establishing a consistent baseline the model can rely on across a working session. Treat it more like a configuration file than a scratch note: version it alongside the code, update it when conventions genuinely change, and resist the urge to tack on one-off instructions that belong in the specific task prompt instead.
Photo by Jimmy Liao on Pexels
Step 5: Test It Against Edge Cases, Not Just the Happy Path
Once a draft system prompt exists, deliberately probe it with requests designed to reveal gaps: ask for a change in an area you explicitly excluded and see if the assistant respects the boundary, ask an ambiguous question and see if it asks for clarification or guesses. A system prompt that only gets tested against straightforward requests will look fine right up until the first time someone asks something it wasn't built to handle.
Step 6: Keep the System Prompt Separate From Task Context
It's tempting to fold everything into one large prompt, but system-level instructions and task-specific context serve different purposes and age differently. The system prompt should describe stable facts about the codebase and team preferences. The task-specific context, the actual files and instructions for the request at hand, should be assembled fresh every time, which is the same discipline covered in the guide on managing an AI coding assistant's context window and token budget. Conflating the two makes both harder to maintain and wastes tokens re-explaining stable facts inside every task-specific request.
Step 7: Write It for the Team, Not Just Yourself
A system prompt that lives in one engineer's personal tooling config helps that one engineer. A system prompt checked into the repository and referenced by the whole team means everyone gets consistent assistant behavior, and it becomes something the team can improve collaboratively instead of quietly diverging into five different personal setups that each work slightly differently. Treat changes to it the way you'd treat changes to a linter configuration: reviewed, discussed, and applied consistently.
A Reasonable Starting Template
A workable structure to start from: one paragraph on what the codebase is and its primary language and framework, one short list of hard boundaries, one short list of house conventions worth naming explicitly, and one sentence on how to handle uncertainty. That's usually under two hundred words, and it's enough to noticeably change how an assistant behaves compared to no system prompt at all, or one that's several paragraphs of generic "be a helpful expert programmer" filler that says nothing specific to your actual codebase. For guidance on what official documentation from major assistant providers recommends here, Anthropic's own documentation is a useful reference point on how system-level instructions are meant to be used.
Step 8: Give It a Owner, Not Just an Author
A system prompt written once by whoever set up the project and never revisited tends to drift out of sync with how the codebase actually evolves. Naming an explicit owner, or at least a review cadence tied to something concrete like a quarterly cleanup or a major dependency upgrade, keeps it from silently going stale. A system prompt that references a testing framework the team stopped using two versions ago actively confuses the assistant rather than helping it, which is worse than having no guidance on that topic at all.
A Common Mistake: Treating It Like a Wishlist
A frequent failure pattern is writing a system prompt that describes an aspirational version of the codebase rather than the actual one. "Follow strict TypeScript conventions" in a prompt attached to a codebase that's only partially migrated to strict mode produces confusing, inconsistent suggestions, because the assistant is being told something that isn't uniformly true. A system prompt should describe the codebase as it actually is right now, including its inconsistencies, rather than the state the team wishes it were in. If there's a genuine migration in progress, saying so explicitly, "some files use strict mode, most don't yet, check the file before assuming," is far more useful than a blanket instruction that's only true in parts of the codebase.
Why This Matters More Than It Seems
Most complaints about AI coding assistants "not understanding the codebase" trace back to a missing or vague system prompt more often than to a genuine model limitation. The model isn't failing to understand your conventions. It was never told what they are. A well-built system prompt closes that gap cheaply, and it compounds: every session benefits from the same investment instead of re-explaining the same context in every task prompt from scratch.
If you're looking to go deeper on the related discipline of managing what actually goes into an assistant's working context beyond the system prompt itself, the folks who wrote this up cover the token-budget side of the same problem in more detail than fits here.
Top comments (0)