Advanced Prompt Engineering: Techniques, Patterns & Production Systems
Master prompt engineering from fundamentals to production systems — chain-of-thought, few-shot, RAG prompting, agent prompts, and evaluation frameworks.
Introduction
Prompt engineering is the discipline of designing inputs to elicit optimal outputs from LLMs. This guide covers production-grade techniques — not just tips and tricks.
Prompt Engineering Maturity Model
| Level | Approach | Example |
|---|---|---|
| 1 | Ad-hoc prompting | "Write a summary" |
| 2 | Structured templates | "Summarize in 3 bullets: {text}" |
| 3 | Few-shot + CoT | Examples + "Think step by step" |
| 4 | RAG-integrated | Context-aware with citations |
| 5 | Agent prompts | Multi-step with tool use |
| 6 | Self-improving | Prompt optimization loops |
Part 1: Foundational Techniques
Zero-Shot Prompting
# prompting/zero_shot.py
from dataclasses import dataclass
from typing import Optional
@dataclass
class PromptTemplate:
system: str
user: str
variables: list[str]
def format(self, **kwargs) -> dict:
return {
"system": self.system,
"user": self.user.format(**kwargs)
}
# Basic templates
SUMMARIZE = PromptTemplate(
system="You are a concise summarizer.",
user="Summarize the following text in {num_sentences} sentences:\n\n{text}",
variables=["num_sentences", "text"]
)
EXTRACT_ENTITIES = PromptTemplate(
system="You extract structured entities from text.",
user="Extract {entity_types} from this text:\n\n{text}\n\nOutput as JSON.",
variables=["entity_types", "text"]
)
CLASSIFY = PromptTemplate(
system="You are a precise classifier.",
user="Classify this text into ONE category: {categories}\n\nText: {text}\n\nCategory:",
variables=["categories", "text"]
)
Few-Shot Prompting
# prompting/few_shot.py
from typing import List, Dict
def build_few_shot_prompt(
task_description: str,
examples: List[Dict[str, str]],
test_input: str,
output_format: str = ""
) -> str:
"""Build a few-shot prompt with consistent formatting."""
parts = [task_description, ""]
for i, ex in enumerate(examples, 1):
parts.append(f"Example {i}:")
parts.append(f"Input: {ex['input']}")
parts.append(f"Output: {ex['output']}")
parts.append("")
parts.append(f"Input: {test_input}")
if output_format:
parts.append(f"Output ({output_format}):")
else:
parts.append("Output:")
return "\n".join(parts)
# Example: SQL generation
SQL_EXAMPLES = [
{
"input": "Find all users who signed up last month",
"output": "SELECT * FROM users WHERE signup_date >= date('now', '-1 month')"
},
{
"input": "Count orders per customer",
"output": "SELECT customer_id, COUNT(*) as order_count FROM orders GROUP BY customer_id"
},
{
"input": "Get products with price > 100",
"output": "SELECT * FROM products WHERE price > 100"
}
]
prompt = build_few_shot_prompt(
task_description="Convert natural language to SQLite SQL.",
examples=SQL_EXAMPLES,
test_input="Show me the top 5 customers by total spend",
output_format="SQL only, no markdown"
)
Chain-of-Thought (CoT)
# prompting/cot.py
COT_PROMPT = """Solve this problem step by step.
Problem: {problem}
Think through this carefully:
1. First, identify what is being asked
2. Break down the problem into sub-problems
3. Solve each sub-problem
4. Combine results for final answer
Show your reasoning at each step.
Step 1:"""
COT_PROMPT_STRUCTURED = """{problem}
Let's think step by step.
{scratchpad_start}"""
# Self-consistency: multiple CoT paths, majority vote
async def self_consistency_cot(llm, problem: str, n: int = 5) -> str:
"""Run CoT multiple times, return majority answer."""
import asyncio
from collections import Counter
async def single_cot():
response = await llm.ainvoke(COT_PROMPT.format(problem=problem))
# Extract final answer (last number or conclusion)
return extract_answer(response)
results = await asyncio.gather(*[single_cot() for _ in range(n)])
counter = Counter(results)
return counter.most_common(1)[0][0]
def extract_answer(text: str) -> str:
# Extract final answer from CoT output
import re
# Look for "Answer: X" or final number
answer_match = re.search(r"(?:Answer|Final answer|Result):\s*(.+)", text, re.IGNORECASE)
if answer_match:
return answer_match.group(1).strip()
# Fallback: last sentence
return text.strip().split('\n')[-1]
Part 2: Advanced Patterns
Tree of Thoughts (ToT)
# prompting/tot.py
from dataclasses import dataclass
from typing import List, Optional
import asyncio
@dataclass
class Thought:
content: str
score: float = 0.0
parent: Optional['Thought'] = None
children: List['Thought'] = None
def __post_init__(self):
if self.children is None:
self.children = []
class TreeOfThoughts:
def __init__(self, llm, branching_factor: int = 3, max_depth: int = 3):
self.llm = llm
self.branching_factor = branching_factor
self.max_depth = max_depth
async def solve(self, problem: str) -> str:
# Generate initial thoughts
root = Thought(content=problem)
current_level = [root]
for depth in range(self.max_depth):
next_level = []
for thought in current_level:
# Generate branches
branches = await self._generate_branches(thought.content)
for branch in branches[:self.branching_factor]:
child = Thought(content=branch, parent=thought)
thought.children.append(child)
next_level.append(child)
# Evaluate and prune
scored = await self._evaluate_thoughts(next_level, problem)
scored.sort(key=lambda x: x.score, reverse=True)
current_level = scored[:self.branching_factor]
# Get best path
best = current_level[0]
path = []
while best:
path.append(best.content)
best = best.parent
return " → ".join(reversed(path))
async def _generate_branches(self, thought: str) -> List[str]:
prompt = f"""Given this partial solution: {thought}
Generate {self.branching_factor} different next steps or approaches.
Each should be a distinct reasoning direction.
Return as numbered list."""
response = await self.llm.ainvoke(prompt)
# Parse numbered list
return [line.split('. ', 1)[1] for line in response.split('\n') if '. ' in line]
async def _evaluate_thoughts(self, thoughts: List[Thought], problem: str) -> List[Thought]:
for thought in thoughts:
prompt = f"""Problem: {problem}
Proposed solution path: {thought.content}
Rate the promise of this approach on a scale of 0-10.
Consider: correctness, completeness, feasibility.
Return only a number."""
response = await self.llm.ainvoke(prompt)
try:
thought.score = float(response.strip())
except:
thought.score = 0.0
return thoughts
ReAct (Reasoning + Acting)
# prompting/react.py
REACT_TEMPLATE = """You are an agent that can use tools to solve problems.
Available tools:
{tools}
Format:
Question: {question}
Thought: I need to...
Action: tool_name
Action Input: {{"arg": "value"}}
Observation: [tool result]
... (repeat as needed)
Thought: I now know the answer
Final Answer: {answer}
Begin!
Question: {question}
Thought:"""
class ReActPromptBuilder:
def __init__(self, tools: list[dict]):
self.tools = tools
def build(self, question: str) -> str:
tool_desc = "\n".join(f"- {t['name']}: {t['description']}" for t in self.tools)
return REACT_TEMPLATE.format(tools=tool_desc, question=question)
def parse_response(self, response: str) -> dict:
import re
# Check for final answer
final_match = re.search(r"Final Answer:\s*(.+)", response, re.DOTALL)
if final_match:
return {"type": "final", "content": final_match.group(1).strip()}
# Check for action
action_match = re.search(r"Action:\s*(\w+)", response)
input_match = re.search(r"Action Input:\s*(\{.+\})", response)
if action_match and input_match:
import json
return {
"type": "action",
"tool": action_match.group(1),
"args": json.loads(input_match.group(1))
}
return {"type": "thought", "content": response}
RAG Prompting with Citations
# prompting/rag_prompts.py
RAG_PROMPT = """You are a precise assistant that answers questions using ONLY the provided context.
## Context Documents
{context}
## Instructions
1. Answer ONLY from the provided context
2. If context is insufficient, say "I don't have enough information to answer this"
3. Cite sources inline using [doc_N] format where N is the document number
4. Be concise but complete
5. If asked for a list, use bullet points
## Question
{question}
## Answer:"""
RAG_PROMPT_STRUCTURED = """{context}
## Question
{question}
## Output Format (JSON)
{{
"answer": "Your answer with [doc_N] citations",
"citations": [
{{"doc_id": "doc_1", "quote": "exact quote from doc", "relevance": "high"}}
],
"confidence": 0.95,
"insufficient_context": false
}}"""
def format_rag_context(docs: list, max_chars: int = 8000) -> str:
"""Format retrieved documents for RAG prompt."""
parts = []
total = 0
for i, doc in enumerate(docs, 1):
content = doc.get('content', '')[:2000] # Limit per doc
source = doc.get('metadata', {}).get('source', 'Unknown')
part = f"[doc_{i}] Source: {source}\n{content}"
if total + len(part) > max_chars:
break
parts.append(part)
total += len(part)
return "\n\n---\n\n".join(parts)
Part 3: Agent System Prompts
Supervisor Agent
# prompting/agent_prompts.py
SUPERVISOR_PROMPT = """You are a Supervisor Agent managing a team of specialist agents.
## Your Role
- Decompose complex user requests into subtasks
- Delegate subtasks to appropriate specialists
- Synthesize specialist outputs into coherent responses
- Track progress and handle failures
## Specialist Agents Available
{specialists}
## Delegation Protocol
When delegating, use this format:
DELEGATE: {{"agent": "agent_name", "task": "specific task description", "context": {{}}}}
## Response Format
1. If task is complete: Provide final synthesized answer
2. If delegation needed: Output DELEGATE command only
3. If clarification needed: Ask specific question
## Current Request
{request}
## Previous Delegation Results
{history}
Your response:"""
### Specialist Agent (Researcher)
python
RESEARCHER_PROMPT = """You are a Research Specialist Agent.
Your Capabilities
- Web search for current information
- Document lookup and summarization
- Fact verification and cross-referencing
- Technical concept explanation
Tools Available
{tools}
Instructions
- Break down the research task into specific queries
- Execute searches in parallel when possible
- Synthesize findings with citations
- Flag uncertainties or conflicting sources
- Return structured results
Task
{task}
Context
{context}
Your response (use DELEGATE format for tool calls):"""
### Code Generation Agent
python
CODER_PROMPT = """You are a Senior Software Engineer Agent.
Your Capabilities
- Write production-quality code in multiple languages
- Debug and fix existing code
- Write tests and documentation
- Refactor for performance and clarity
- Architecture and design decisions
Principles
- Write clean, readable, well-documented code
- Follow language idioms and best practices
- Include error handling and edge cases
- Write tests for non-trivial logic
- Explain design decisions
Tools Available
{tools}
Task
{task}
Context (existing code, requirements, constraints)
{context}
Output Format
Provide code in markdown blocks with language tags.
Include brief explanation of approach.
If using tools, use DELEGATE format.
Your response:"""
## Part 4: Prompt Optimization
### Automated Prompt Optimization
python
prompting/optimization.py
from dataclasses import dataclass
from typing import List, Callable
import asyncio
@dataclass
class PromptCandidate:
prompt: str
score: float = 0.0
metadata: dict = None
class PromptOptimizer:
def init(
self,
llm,
evaluator: Callable[[str, str], float], # (prompt, expected) -> score
population_size: int = 10,
generations: int = 5
):
self.llm = llm
self.evaluator = evaluator
self.population_size = population_size
self.generations = generations
async def optimize(
self,
base_prompt: str,
test_cases: List[dict], # {"input": "", "expected": ""}
mutation_rate: float = 0.3
) -> PromptCandidate:
# Initialize population
population = await self._initialize_population(base_prompt)
for gen in range(self.generations):
# Evaluate
scored = await self._evaluate_population(population, test_cases)
scored.sort(key=lambda x: x.score, reverse=True)
# Select top performers
elite = scored[:self.population_size // 2]
# Generate next generation
population = elite.copy()
while len(population) < self.population_size:
parent = elite[0] # Best parent
if random.random() < mutation_rate:
child = await self._mutate(parent.prompt)
else:
child = await self._crossover(elite[0].prompt, elite[1].prompt)
population.append(PromptCandidate(prompt=child))
return scored[0]
async def _initialize_population(self, base: str) -> List[PromptCandidate]:
population = [PromptCandidate(prompt=base)]
# Generate variations
variations = [
"Add more explicit instructions",
"Add few-shot examples",
"Add chain-of-thought prompting",
"Add output format specification",
"Add role/persona definition",
"Add constraint specifications",
"Add step-by-step reasoning",
"Add self-verification step",
"Add edge case handling",
"Add few-shot with diverse examples"
]
for var in variations[:self.population_size - 1]:
prompt = await self._apply_variation(base, var)
population.append(PromptCandidate(prompt=prompt))
return population
async def _mutate(self, prompt: str) -> str:
mutations = [
"Add 'Think step by step' to the prompt",
"Add 'Be concise' instruction",
"Add output format specification",
"Add 'Verify your answer before responding'",
"Add role definition at the start",
"Add constraint: no markdown unless asked",
"Add 'If uncertain, say so explicitly'",
"Restructure into clear sections"
]
return await self._apply_variation(prompt, random.choice(mutations))
async def _crossover(self, p1: str, p2: str) -> str:
# Combine best parts of two prompts
prompt = f"""Combine the best elements of these two prompts:
Prompt 1:
{p1}
Prompt 2:
{p2}
Create a single improved prompt that incorporates the strengths of both."""
return await self.llm.ainvoke(prompt)
async def _apply_variation(self, prompt: str, variation: str) -> str:
prompt_text = f"""Improve this prompt by: {variation}
Original prompt:
{prompt}
Return only the improved prompt."""
return await self.llm.ainvoke(prompt_text)
### DSPy-Style Optimization
python
prompting/dspy_style.py
from dataclasses import dataclass
from typing import List, Any
@dataclass
class Example:
inputs: dict
outputs: dict
class PromptModule:
def init(self, prompt_template: str, llm):
self.template = prompt_template
self.llm = llm
self.demos: List[Example] = []
def add_demo(self, inputs: dict, outputs: dict):
self.demos.append(Example(inputs=inputs, outputs=outputs))
def __call__(self, **inputs) -> Any:
# Format prompt with demos
demo_text = ""
for demo in self.demos:
demo_text += f"Input: {demo.inputs}\nOutput: {demo.outputs}\n\n"
full_prompt = self.template.format(demos=demo_text, **inputs)
return self.llm.invoke(full_prompt)
def compile(self, trainset: List[Example], metric: Callable):
# Bootstrap demos from trainset using metric
scored = [(metric(ex.outputs, self(**ex.inputs)), ex) for ex in trainset]
scored.sort(reverse=True)
self.demos = [ex for _, ex in scored[:3]] # Top 3
## Part 5: Evaluation Frameworks
### Prompt Evaluation Metrics
python
prompting/evaluation.py
from dataclasses import dataclass
from typing import List, Callable
import asyncio
@dataclass
class PromptMetrics:
accuracy: float
consistency: float # Self-consistency across runs
instruction_following: float
format_compliance: float
latency_ms: float
token_count: int
cost_usd: float
class PromptEvaluator:
def init(self, llm, judge_llm=None):
self.llm = llm
self.judge = judge_llm or llm
async def evaluate(
self,
prompt: str,
test_cases: List[dict],
n_runs: int = 3
) -> PromptMetrics:
all_outputs = []
latencies = []
token_counts = []
for tc in test_cases:
case_outputs = []
for _ in range(n_runs):
start = time.time()
output = await self.llm.ainvoke(prompt.format(**tc["input"]))
latency = (time.time() - start) * 1000
case_outputs.append(output)
latencies.append(latency)
token_counts.append(len(output.split()))
all_outputs.append(case_outputs)
# Accuracy via LLM judge
accuracy = await self._judge_accuracy(test_cases, all_outputs)
# Consistency: agreement across runs
consistency = self._compute_consistency(all_outputs)
# Instruction following
instruction_following = await self._judge_instruction_following(
prompt, test_cases, all_outputs
)
# Format compliance
format_compliance = self._check_format_compliance(all_outputs, prompt)
return PromptMetrics(
accuracy=accuracy,
consistency=consistency,
instruction_following=instruction_following,
format_compliance=format_compliance,
latency_ms=sum(latencies) / len(latencies),
token_count=sum(token_counts) // len(token_counts),
cost_usd=self._estimate_cost(token_counts)
)
async def _judge_accuracy(self, test_cases, all_outputs) -> float:
scores = []
for tc, outputs in zip(test_cases, all_outputs):
# Use first output for evaluation
output = outputs[0]
prompt = f"""Rate the accuracy of this response (0-1).
Expected: {tc.get('expected', 'N/A')}
Actual: {output}
Consider: factual correctness, completeness, relevance.
Return only a number between 0 and 1."""
response = await self.judge.ainvoke(prompt)
try:
scores.append(float(response.strip()))
except:
scores.append(0.5)
return sum(scores) / len(scores) if scores else 0
def _compute_consistency(self, all_outputs) -> float:
# Semantic similarity between runs (simplified)
agreements = 0
total = 0
for outputs in all_outputs:
for i in range(len(outputs)):
for j in range(i + 1, len(outputs)):
# Simple word overlap similarity
words_i = set(outputs[i].lower().split())
words_j = set(outputs[j].lower().split())
if words_i and words_j:
overlap = len(words_i & words_j) / len(words_i | words_j)
agreements += overlap
total += 1
return agreements / total if total > 0 else 1.0
### A/B Testing Framework
python
prompting/ab_test.py
from dataclasses import dataclass
from typing import List
import random
@dataclass
class ABTestResult:
variant_a: PromptMetrics
variant_b: PromptMetrics
winner: str # "A", "B", or "tie"
confidence: float
sample_size: int
class ABTester:
def init(self, evaluator: PromptEvaluator):
self.evaluator = evaluator
async def run_test(
self,
prompt_a: str,
prompt_b: str,
test_cases: List[dict],
min_samples: int = 50
) -> ABTestResult:
# Randomly assign test cases to variants
random.shuffle(test_cases)
split = len(test_cases) // 2
cases_a = test_cases[:split]
cases_b = test_cases[split:]
# Evaluate both
metrics_a = await self.evaluator.evaluate(prompt_a, cases_a)
metrics_b = await self.evaluator.evaluate(prompt_b, cases_b)
# Statistical significance (simplified)
winner = "A" if metrics_a.accuracy > metrics_b.accuracy else "B"
if abs(metrics_a.accuracy - metrics_b.accuracy) < 0.02:
winner = "tie"
return ABTestResult(
variant_a=metrics_a,
variant_b=metrics_b,
winner=winner,
confidence=self._compute_confidence(metrics_a, metrics_b),
sample_size=len(test_cases)
)
## Part 6: Production Prompt Management
### Prompt Versioning & Registry
python
prompting/registry.py
import sqlite3
import json
from dataclasses import dataclass
from datetime import datetime
from typing import Optional
@dataclass
class PromptVersion:
id: str
name: str
version: int
prompt: str
metadata: dict
created_at: str
parent_id: Optional[str] = None
class PromptRegistry:
def init(self, db_path: str = "prompts.db"):
self.conn = sqlite3.connect(db_path)
self._init_db()
def _init_db(self):
self.conn.execute("""
CREATE TABLE IF NOT EXISTS prompts (
id TEXT PRIMARY KEY,
name TEXT,
version INTEGER,
prompt TEXT,
metadata TEXT,
created_at TEXT,
parent_id TEXT
)
""")
self.conn.execute("""
CREATE INDEX IF NOT EXISTS idx_name_version ON prompts(name, version)
""")
self.conn.commit()
def save(self, prompt: PromptVersion):
self.conn.execute("""
INSERT INTO prompts (id, name, version, prompt, metadata, created_at, parent_id)
VALUES (?, ?, ?, ?, ?, ?, ?)
""", (
prompt.id, prompt.name, prompt.version,
prompt.prompt, json.dumps(prompt.metadata),
prompt.created_at, prompt.parent_id
))
self.conn.commit()
def get_latest(self, name: str) -> Optional[PromptVersion]:
cursor = self.conn.execute(
"SELECT * FROM prompts WHERE name = ? ORDER BY version DESC LIMIT 1",
(name,)
)
row = cursor.fetchone()
return self._row_to_prompt(row) if row else None
def get_version(self, name: str, version: int) -> Optional[PromptVersion]:
cursor = self.conn.execute(
"SELECT * FROM prompts WHERE name = ? AND version = ?",
(name, version)
)
row = cursor.fetchone()
return self._row_to_prompt(row) if row else None
def get_history(self, name: str) -> List[PromptVersion]:
cursor = self.conn.execute(
"SELECT * FROM prompts WHERE name = ? ORDER BY version",
(name,)
)
return [self._row_to_prompt(r) for r in cursor.fetchall()]
def _row_to_prompt(self, row) -> PromptVersion:
return PromptVersion(
id=row[0], name=row[1], version=row[2],
prompt=row[3], metadata=json.loads(row[4]),
created_at=row[5], parent_id=row[6]
)
### Prompt Deployment with Feature Flags
python
prompting/deployment.py
from dataclasses import dataclass
from typing: Dict, Optional
import random
@dataclass
class PromptConfig:
name: str
version: int
traffic_percentage: float = 100.0 # 0-100
conditions: Dict[str, any] = None # e.g., {"user_tier": "premium"}
class PromptDeployment:
def init(self, registry: PromptRegistry):
self.registry = registry
self.configs: Dict[str, PromptConfig] = {}
def register(self, config: PromptConfig):
self.configs[config.name] = config
def get_prompt(self, name: str, context: dict = None) -> str:
config = self.configs.get(name)
if not config:
# Fallback to latest
prompt = self.registry.get_latest(name)
return prompt.prompt if prompt else ""
# Check conditions
if config.conditions and context:
for key, value in config.conditions.items():
if context.get(key) != value:
# Condition not met, get previous version
prev = self.registry.get_version(name, config.version - 1)
return prev.prompt if prev else ""
# Check traffic percentage
if config.traffic_percentage < 100:
if random.random() * 100 > config.traffic_percentage:
prev = self.registry.get_version(name, config.version - 1)
return prev.prompt if prev else ""
prompt = self.registry.get_version(name, config.version)
return prompt.prompt if prompt else ""
## Part 7: Specialized Prompt Patterns
### Structured Output Enforcement
python
prompting/structured_output.py
from pydantic import BaseModel, Field
from typing import List, Optional, Literal
import instructor
class StructuredPrompts:
@staticmethod
def json_schema_prompt(schema: dict, examples: List[dict] = None) -> str:
"""Generate prompt that enforces JSON schema output."""
schema_str = json.dumps(schema, indent=2)
example_str = ""
if examples:
example_str = "\n\nExamples:\n"
for i, ex in enumerate(examples, 1):
example_str += f"Example {i}:\n{json.dumps(ex, indent=2)}\n\n"
return f"""Output ONLY valid JSON matching this schema:
{schema_str}
Rules:
- No markdown, no explanations, no extra text
- All required fields must be present
- Types must match exactly
-
{example_str}"""
@staticmethod
def constrained_generation_prompt(constraints: List[str]) -> str:
"""Generate prompt with hard constraints."""
constraint_text = "\n".join(f"- {c}" for c in constraints)
return f"""You must follow these constraints STRICTLY:
{constraint_text}
Violating any constraint makes the output invalid.
Think carefully before each token."""
### Multilingual Prompting
python
prompting/multilingual.py
MULTILINGUAL_TEMPLATE = """You are a multilingual assistant. Respond in {target_language}.
Task
{task}
Context
{context}
Constraints
- Respond ONLY in {target_language}
- Maintain cultural appropriateness
- Use region-appropriate terminology
- {additional_constraints}
Output
"""
Language-specific adjustments
LANGUAGE_ADJUSTMENTS = {
"ja": "Use polite forms (desu/masu). Avoid direct translations.",
"zh": "Use Simplified Chinese. Prefer formal written style.",
"de": "Use formal 'Sie' form. Compound nouns appropriately.",
"es": "Use formal 'usted'. Consider regional variations.",
"fr": "Use formal 'vous'. Follow French typography rules.",
"ar": "Use Modern Standard Arabic. RTL-aware formatting.",
}
## Decision Framework
plaintext
PROMPT PATTERN SELECTION:
Simple, deterministic task?
├─→ Yes → Zero-shot with clear template
└─→ No
├─→ Need specific format/style?
│ ├─→ Yes → Few-shot with examples
│ └─→ No
│ ├─→ Complex reasoning needed?
│ │ ├─→ Yes → Chain-of-Thought or Tree of Thoughts
│ │ └─→ No
│ │
│ ├─→ Need external info?
│ │ └─→ Yes → RAG prompting with citations
│ │
│ ├─→ Multi-step with tools?
│ │ └─→ Yes → ReAct or Agent system prompt
│ │
│ └─→ High-stakes / production?
│ └─→ Yes → Optimized + evaluated + versioned
## Production Checklist
### Prompt Quality
- [ ] Clear role/persona definition
- [ ] Explicit task description
- [ ] Specific output format requirements
- [ ] Constraints and guardrails
- [ ] Few-shot examples (3-5 diverse cases)
- [ ] Chain-of-thought for reasoning tasks
- [ ] Edge case handling instructions
- [ ] Self-verification step
### Evaluation
- [ ] Golden test set (50+ cases)
- [ ] Automated metrics (accuracy, format, consistency)
- [ ] LLM judge for subjective quality
- [ ] A/B testing framework
- [ ] Regression detection in CI/CD
### Management
- [ ] Version control for all prompts
- [ ] Prompt registry with metadata
- [ ] Feature flags for gradual rollout
- [ ] Canary deployment support
- [ ] Rollback capability (< 1 min)
- [ ] Usage analytics per prompt version
### Security
- [ ] Input sanitization
- [ ] Injection attempt detection
- [ ] PII handling instructions
- [ ] Output validation
- [ ] Rate limiting awareness
## Resources
- [Prompt Engineering Guide (OpenAI)](https://platform.openai.com/docs/guides/prompt-engineering)
- [Chain-of-Thought Paper](https://arxiv.org/abs/2201.11903)
- [Tree of Thoughts Paper](https://arxiv.org/abs/2305.10601)
- [ReAct Paper](https://arxiv.org/abs/2210.03629)
- [DSPy Framework](https://dspy-docs.vercel.app/)
- [Instructor Library](https://python.useinstructor.com/)
- [Prompt Engineering Patterns](https://github.com/dair-ai/Prompt-Engineering-Guide)
---
*Prompt engineering is not magic — it's systematic communication with a probabilistic system. Treat prompts as code: version, test, evaluate, and deploy with the same rigor.*
Top comments (0)