My first system prompt was this:
You are a helpful and friendly AI assistant. You are knowledgeable about many topics
and always try to give thorough, well-explained answers. You are patient and
understanding. You never refuse to help with reasonable questions. You always
maintain a positive and encouraging tone in all your responses.
It did nothing. The model behaved exactly the same with it as without it. I'd written five sentences that described what an LLM already does by default.
This article is about writing system prompts that actually change how the model behaves — with real examples from the app we've been building.
What a system prompt actually is
When you call the Gemini API, your request has three parts:
- System prompt — instructions the model reads before anything else
- Conversation history — the previous messages
- User message — what the user just typed
The system prompt is processed first, before any user input. The model uses it to understand its role, constraints, and expected behaviour for the entire conversation.
Think of it as the briefing you give an employee before their first day. The more specific and well-structured the briefing, the less they have to guess.
What breaks a system prompt
Too vague
You are a helpful assistant.
This tells the model nothing it doesn't already know. It'll just do what it would do anyway.
Describing what the model IS instead of what it should DO
You are an expert travel planner who knows everything about destinations worldwide.
Describing the model's identity doesn't constrain its behaviour. The model can still say anything.
Too long with no structure
A wall of text is hard for the model to parse. Instructions buried in paragraph 4 often get ignored or deprioritised. Important rules need to stand out.
Contradictory instructions
Be concise. Always give thorough, detailed answers with examples.
When instructions conflict, the model picks one — and you won't know which.
What actually works
Good system prompts have clear building blocks. Not every prompt needs all of them, but knowing what each one does helps you write deliberately.
Persona — who the model is in this context
Not a description of the model's general nature, but a specific role with scope:
You are a database assistant. You help users query business data
using natural language.
Persona alone isn't enough, but it sets the frame for everything that follows.
Constraints — what the model will not do
This is where system prompts earn their keep. Hard limits that override the model's defaults:
Only write SELECT queries — no INSERT, UPDATE, DELETE, DROP, or DDL.
Answer ONLY based on the provided context. If the context lacks enough
information, say so clearly.
If the user asks about any destination outside India, politely decline.
Constraints are the most powerful thing in a system prompt. They make the model predictable. Without them, the model will try to be helpful in ways you didn't intend.
Output format — how to structure the response
ALWAYS return query results as a full markdown table showing ALL rows
and ALL columns — do NOT summarize, abbreviate, or omit rows.
Answer only YES or NO, nothing else.
Output format instructions are surprisingly effective. When you tell the model exactly what shape the response should be, it follows it consistently.
Few-shot examples — show the model exactly what you want
Instructions tell the model what to do. Examples show it. When format consistency matters, one or two examples in the system prompt are more reliable than a paragraph of instructions.
You are a helpful assistant for a Java backend engineer learning AI development.
Be concise and practical.
Example:
User: How do I check if a string is empty in Java?
Assistant: Use `str == null || str.isEmpty()`. Prefer `str.isBlank()` in Java 11+ to also catch whitespace-only strings.
The model picks up the response style — length, tone, format — from the example and applies it consistently. One example is usually enough. Two if the format is unusual or strict.
Sequence — the order things should happen
For tool-calling agents, the order of operations matters:
For DATABASE questions, ALWAYS follow this sequence:
1. Call listTables to see what tables are available
2. Call getTableSchema for every table you need — never guess column names
3. Write a safe SELECT query and call executeQuery
Without this, the model might try to write a query without checking the schema first — and guess column names that don't exist.
Real examples from this app
Here's how these principles look in practice, using the actual system prompts from the app we've been building.
The simplest case — lean and specific
public static final String CHAT_AI_SYSTEM =
"You are a helpful assistant for a Java backend engineer learning AI development. " +
"Be concise and practical.";
Two sentences. But they're specific. The model knows the audience (Java backend engineers learning AI) and the output style (concise and practical). It won't give long theoretical explanations when a code snippet would do.
Single-purpose validator — extremely tight
public static final String INDIA_VALIDATION_SYSTEM =
"You are a geography validator. Answer only YES or NO, nothing else.";
This prompt is used to check whether a destination is in India before the travel agent runs. The output constraint (only YES or NO, nothing else) is absolute. No explanation, no uncertainty — just the answer the code needs to branch on.
RAG assistant — context-bound
public static final String RAG_SYSTEM =
"You are a helpful assistant. Answer ONLY based on the provided context. " +
"If the context lacks enough information, say so clearly.";
The key instruction is ONLY based on the provided context. Without this, the model uses its general knowledge to fill in gaps — which means it might answer with information that isn't in your documents. For a RAG system, that's a hallucination problem.
Complex agent — structured with rules
public static final String DATABASE_AGENT_SYSTEM =
"You are a database and knowledge assistant. " +
"You have five tools: listTables, getTableSchema, executeQuery, askDocuments, ingestDocument.\n" +
"For DATABASE questions, ALWAYS follow this sequence:\n" +
"1. Call listTables to see what tables are available\n" +
"2. Call getTableSchema for every table you need — never guess column names\n" +
"3. Write a safe SELECT query and call executeQuery\n" +
"DATABASE RULES:\n" +
"- Only write SELECT queries — no INSERT, UPDATE, DELETE, DROP, or DDL\n" +
"- Always check the schema before querying — column names must come from getTableSchema, not guesses\n" +
"- ALWAYS return query results as a full markdown table showing ALL rows and ALL columns\n" +
"- If the user asks something the data cannot answer, say so honestly";
This prompt is longer — but it's structured. The sequence is numbered. The rules are bulleted. Important words are capitalised (ALWAYS, ONLY). The model can scan this prompt and find what applies to the current situation.
Dynamic system prompts
System prompts don't have to be static strings. In Spring AI you can build the system prompt at call time and inject runtime context — current date, user preferences, session data — before sending the request.
@PostMapping("/chat-ai")
public String chat(@RequestBody Map<String, String> request) {
String userId = request.get("userId");
String systemPrompt = String.format(
"You are a helpful assistant for a Java backend engineer learning AI development. " +
"Be concise and practical. Today's date is %s.",
LocalDate.now()
);
return chatClient.prompt()
.system(systemPrompt) // ← override the default system prompt per request
.user(request.get("message"))
.advisors(a -> a.param("chat_memory_conversation_id", request.get("conversationId")))
.call()
.content();
}
.system(systemPrompt) on the prompt overrides the defaultSystem(...) you set in the ChatClient builder for that one call. The builder default is still used for everything else.
This is also how production applications inject per-user context: user tier, preferences, retrieved memory facts, or feature flags — all assembled into the system prompt at request time rather than stored as a static constant.
Keep the static parts in Prompts.java as constants and string-format in the runtime values. That way the static structure stays readable and the dynamic parts are clearly marked.
How to test if your prompt is working
Write test cases the same way you'd write unit tests. For each instruction in your prompt, write a user message that should trigger it:
| Instruction | Test input | Expected behaviour |
|---|---|---|
| India-only | "Plan a trip to Paris" | Politely declines |
| SELECT only | "Delete all orders" | Refuses |
| YES or NO only | "Is Mumbai in India?" | Returns YES
|
| Concise | "What is Spring Boot?" | Short answer, no long essay |
If the model doesn't behave as expected on your test inputs, the instruction is either too vague, missing, or buried where the model deprioritises it. Move critical rules earlier in the prompt and make them more explicit.
Practical rules I follow
Lead with constraints, not with personality. The model's personality is fine by default. Your constraints are what matters.
Use ALL CAPS for non-negotiable rules. ALWAYS, NEVER, ONLY. It works.
Number sequences. If order matters, number the steps. Bullet points suggest optional items. Numbered lists suggest mandatory sequence.
Be specific about output format. Don't say "give a structured answer." Say "return a markdown table with all rows and columns."
Keep it under 300 tokens where possible. Every token in your system prompt runs on every API call. A bloated system prompt costs money and attention.
What's next
System prompts shape how the model thinks. Next: structured output — making the model return JSON instead of prose, so your backend can parse and act on the response programmatically.
Have a system prompt that finally started working after you changed something specific? Drop it in the comments.
Sham Prakash K — Backend Engineer, 4+ years in Java, Spring Boot, and distributed systems. Building AI backend infrastructure. Writing about what I actually learned, mistakes included.
Top comments (0)