GEPA Explained — From Paper to Working Code in 10 Minutes

If you've ever thought "I wish I could improve my LLM pipeline without burning GPU budget on fine-tuning," GEPA is what you've been waiting for.

TL;DR

• 🧬 GEPA = Genetic-Pareto Evolutionary Prompt Adaptation
• 🏆 ICLR 2026 Oral paper from UC Berkeley Sky Computing Lab
• ⚡ Outperforms GRPO/PPO on compound AI tasks — zero GPU required
• 🔧 Works with any black-box LLM (GPT-4o, Claude, Gemini, etc.)
• 📦 Ships as dspy.GEPA — drop-in optimizer for DSPy pipelines

The Problem

Picture this: you've built a multi-step RAG pipeline — retrieve → reason → format. The retriever is good, but the final answer quality is mediocre. You want to improve it.

Your options:

1. Fine-tune with GRPO/PPO: requires GPU access, thousands of training steps, and only updates one model in the chain
2. Hand-tune prompts manually: not systematic, doesn't scale to multi-component systems
3. Use MIPROv2 or similar APO: better, but hill-climbing only — no population search, no compound system awareness

GEPA introduces option 4: evolve a population of prompts using LLM-generated critiques, with Pareto-optimal selection across accuracy, cost, and latency. It's genetic algorithms, but where the mutation operator is an LLM reflecting on its own failures.

How It Works

GEPA runs three steps in a loop for N generations:

Step 1 — Evaluate: Run each prompt in the population on your training examples. Score them with your metric (F1, exact match, whatever).

Step 2 — Reflect: For each prompt that made mistakes, ask an LLM: "Here are the examples this prompt failed on. Why? What pattern do you see?"

def reflect(prompt, failures):
    return llm(f"""
    Prompt: {prompt}

    These examples failed:
    {failures}

    Analyze: (1) failure pattern, (2) specific fix needed.
    """)

Step 3 — Evolve: Use the critique to generate an improved prompt variant:

def evolve(prompt, critique):
    return llm(f"""
    Improve this prompt using the critique.

    Current: {prompt}
    Critique: {critique}

    Improved version:
    """)

Then apply Pareto selection to keep the best prompts on the accuracy-cost-latency frontier. Repeat for N generations.

For compound AI systems (multi-step pipelines), GEPA adds credit assignment — it traces the full execution and figures out which component's prompt is responsible for the failure, then applies targeted reflection to that component.

Show Me The Code

Option A: DSPy (Recommended)

pip install dspy-ai

import dspy
from dspy.teleprompt import GEPA

# Configure your model
lm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=lm)

# Your pipeline (works with any existing DSPy module)
class AnswerPipeline(dspy.Module):
    def __init__(self):
        self.step1 = dspy.ChainOfThought("question -> search_query")
        self.step2 = dspy.ChainOfThought("question, context -> answer")

    def forward(self, question, retriever):
        q = self.step1(question=question)
        ctx = retriever(q.search_query)
        return self.step2(question=question, context=ctx)

# Metric
def f1_score(example, pred, trace=None):
    p_toks = set(pred.answer.lower().split())
    g_toks = set(example.answer.lower().split())
    if not p_toks or not g_toks:
        return 0.0
    precision = len(p_toks & g_toks) / len(p_toks)
    recall    = len(p_toks & g_toks) / len(g_toks)
    denom = precision + recall
    return 2 * precision * recall / denom if denom > 0 else 0.0

# GEPA optimization — this is the magic line
optimizer = GEPA(
    metric=f1_score,
    num_threads=8,
    num_candidates=10,    # population size per generation
    num_iterations=5,     # number of generations
    max_bootstrapped_demos=3,
)

# Compile (optimize)
optimized = optimizer.compile(AnswerPipeline(), trainset=train_examples)

# Use it
result = optimized(question="What is the tallest mountain?", retriever=my_retriever)
print(result.answer)

# Inspect what changed
print(optimized.step1.signature.instructions)
print(optimized.step2.signature.instructions)

Option B: Minimal from-scratch implementation

from openai import OpenAI

client = OpenAI()

class GEPAMinimal:
    def __init__(self, base_prompt, metric_fn, train_data,
                 population_size=6, generations=4):
        self.population = [base_prompt] * population_size
        self.metric = metric_fn
        self.data = train_data
        self.pop_size = population_size
        self.generations = generations

    def _evaluate(self, prompt):
        scores = []
        failures = []
        for ex in self.data:
            pred = self._run_prompt(prompt, ex["input"])
            score = self.metric(ex["expected"], pred)
            scores.append(score)
            if score < 0.5:
                failures.append({
                    "input": ex["input"],
                    "predicted": pred,
                    "expected": ex["expected"]
                })
        return sum(scores) / len(scores), failures

    def _run_prompt(self, prompt, user_input):
        resp = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[
                {"role": "system", "content": prompt},
                {"role": "user", "content": user_input}
            ]
        )
        return resp.choices[0].message.content

    def _reflect(self, prompt, failures):
        if not failures:
            return None
        resp = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[{"role": "user", "content": f"""
Analyze why this system prompt produces wrong answers.

SYSTEM PROMPT:
{prompt}

FAILURES (input | predicted | expected):
{chr(10).join(f'- {f["input"]} | {f["predicted"]} | {f["expected"]}' for f in failures[:5])}

Output: specific diagnosis + actionable fix."""}]
        )
        return resp.choices[0].message.content

    def _mutate(self, prompt, reflection):
        resp = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[{"role": "user", "content": f"""
Rewrite this system prompt to fix the identified problem.

ORIGINAL:
{prompt}

DIAGNOSIS:
{reflection}

IMPROVED PROMPT (different from original, must address the diagnosis):"""}]
        )
        return resp.choices[0].message.content

    def optimize(self):
        for gen in range(self.generations):
            scored = []
            for p in self.population:
                score, failures = self._evaluate(p)
                scored.append((score, failures, p))

            scored.sort(reverse=True, key=lambda x: x[0])
            best_score = scored[0][0]
            print(f"Gen {gen+1}/{self.generations} | Best: {best_score:.3f}")

            new_pop = [scored[0][2]]  # elitism: keep best
            for score, failures, prompt in scored[:self.pop_size//2]:
                reflection = self._reflect(prompt, failures)
                if reflection:
                    mutant = self._mutate(prompt, reflection)
                    new_pop.append(mutant)
                new_pop.append(prompt)

            self.population = new_pop[:self.pop_size]

        final_scored = [(self._evaluate(p)[0], p) for p in self.population]
        return max(final_scored, key=lambda x: x[0])[1]


# Usage
optimizer = GEPAMinimal(
    base_prompt="Answer the user's question concisely.",
    metric_fn=lambda expected, pred: int(expected.lower() in pred.lower()),
    train_data=[
        {"input": "Capital of France?", "expected": "Paris"},
        {"input": "Largest planet?", "expected": "Jupiter"},
    ]
)

best_prompt = optimizer.optimize()
print("Optimized prompt:", best_prompt)

Benchmark Results

The paper (ICLR 2026 Oral) shows GEPA's key wins:

Scenario	GRPO	MIPROv2	GEPA
Single LLM, reasoning	Strong	Decent	Competitive
Multi-step pipeline (RAG)	Limited	Decent	Best
Black-box API model	Can't apply	Works	Works
GPU cost	High	None	None
Interpretable result	No	Yes	Yes

The headline: on compound AI systems (multi-step pipelines), GEPA beats GRPO even though it never touches model weights. That's the key result — and it's why ICLR reviewers gave it an Oral slot.

Gotchas & Limitations

Things to watch out for before you run this in production:

1. Prompt length creep: Each generation tends to make prompts longer. After 10 generations, your system prompt might be 3x the original length. Set a length budget.

# Add a length penalty to your metric
def metric_with_length_penalty(example, pred, trace=None):
    base_score = f1_score(example, pred)
    prompt_tokens = len(pred._prompt.split()) if hasattr(pred, '_prompt') else 0
    length_penalty = max(0, (prompt_tokens - 200) / 1000)
    return base_score - length_penalty

2. You need labeled examples: GEPA requires ~50-500 training examples with ground truth labels. Zero-shot or few-shot settings don't give it enough signal to evolve meaningfully.

3. The reflection LLM quality matters: Use a strong model (GPT-4o, not GPT-3.5) for reflection, even if your inference model is smaller. The mutation operator is only as good as the reflecting model.

4. Not a replacement for domain knowledge injection: If your model genuinely doesn't know something (it's not in pre-training), GEPA can't evolve a prompt to make it know. Use RAG for knowledge, GEPA for skill.

5. Re-optimize when model changes: If GPT-4o gets upgraded to GPT-5, your GEPA-optimized prompts might not transfer perfectly. But re-optimization is fast (hours not days) since the evolved prompts serve as a good starting population.

Try It Today

pip install dspy-ai
# Then follow the DSPy quickstart at:
# https://dspy.ai/api/optimizers/GEPA/overview/

---

What's your experience with prompt optimization vs. fine-tuning? Have you tried DSPy-style compiled pipelines? Drop a comment — I'm especially curious whether anyone has tried GEPA on production agentic systems.

Sources

• arXiv:2507.19457 — GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning
• GitHub: gepa-ai/gepa
• DSPy GEPA Optimizer Docs
• ICLR 2026 Oral Presentation
• Hugging Face Paper Page