DSPy for Prompt Engineers: Build Your First Modular LLM Program (OpenAI & Llama)

No More Prompt Tinkering: 3 Steps to a Self-Improving JSONL Q/A Agent with DSPy

Part 1 – Hello DSPy: Building Your Mental Model

Too often, LLM frameworks baffle you with black-box “magic.” You read a tutorial, copy some code, and it works; until it doesn't. In this opening chapter, my goal is to not just show you how to write a Signature or a Module but to give you a clear mental model of what is happening at each stage.

Towards the end of this tutorial, you should start looking at DSPy not as "another library", but a compiler for your prompts: Allowing you to write maintainable LLM programs in a declarative way.

The Core analogy: DSPy as a Compiler

Let's start by thinking of DSPy as a mini programming language ( or DSL ) for LLM calls.

• Prompt engineering is like writing SQL inline in your code. It's quick and dirty, but brittle.
• DSPy is like using an ORM (Object-Relational Mapper) or a query builder: you still define what you want, but you do it through typed objects and methods. The framework does the heavy lifting of transforming your code into underlying SQL query.

In DSPy:

1. Signature = your function signature (types and names of inputs/outputs).
2. Module = your function body. But instead of raw code, it calls dspy.Predict or other primitives.
3. Compiler = the part of DSPy that reads your Signature+Module and generates the actual prompt text.
4. Adapter = the plugin that sends that prompt to OpenAI, a local Llama, or any LLM you configure
5. Parser = the piece that takes the model’s raw text reply and maps it back into your declared output fields, enforcing types.

Signatures: Your Contract with the Model

A Signature answers the question: “What inputs can I give, and what outputs can I expect?”
It’s critical for:

• Clarity: you (and your teammates) see the I/O schema at a glance.
• Safety: DSPy can validate or cast types.
• Evolution: if you later swap to a structured-output adapter, you already have the schema.

from dspy import Signature

# Guarantees: Takes a `question` string, and return an `answer` string.
sig = Signature("question:str -> answer:str")

You can think of Signatures like function type hints in TypeScript or Python’s def foo(x: int) -> str. But they're more than docs as they drive DSPy’s code generation and parsing.

Modules: Encapsulating Prompt Logic

A Module is just a Python class with a forward method. Inside that, you call DSPy primitives (like Predict) instead of openai.ChatCompletion.create(...).

from dspy import Module, Predict

class SimpleQA(Module):
    def forward(self, question: str):
        # This single line replaces 5–10 lines
        # of prompt assembly and API calls
        return Predict(sig, question=question)

These are really helpful, as they are:

• Composable: You can call one Module from another.
• Testable: You can write unit tests asserting SimpleQA("X") contains a substring.
• Inspectable: You can ask DSPy to show you the exact prompt it generated.

Backend Agnosticism: Swap with One Line

Once your Module is defined, you can flip backends without touching it. DSPy’s configure is your only change.

Local Llama Example

You need to have an ollama instance running and serving a model. I chose gemma3n:e2b as gemma models are designed for efficient execution on low-resource devices.

You can replace it with any ollama model running on your system. ollama_chat/<model>

import dspy
# ─── CONFIGURE BACKEND ──────────────────────────────────────────
# Swap between "openai" or "local-llama" here.
lm = dspy.LM('ollama_chat/gemma3n:e2b', api_base='http://localhost:11434', api_key='')
dspy.configure(lm=lm)

OpenAi Example

import dspy
# ─── CONFIGURE BACKEND ──────────────────────────────────────────
# Swap between "openai" or "local-llama" here.
lm = dspy.LM("openai/gpt-4o-mini", api_key="YOUR_OPENAI_API_KEY")
dspy.configure(lm=lm)

Mental Model: Your Module declares what you want. The Adapter determines where and how to execute it. They're separate concerns.

Full Code Example

#!/usr/bin/env python3
import sys
import os
import dspy

# ─── CONFIGURE BACKEND ──────────────────────────────────────────
# Swap between "openai" or "local-llama" here.
lm = dspy.LM('ollama_chat/gemma3n:e2b', api_base='http://localhost:11434', api_key='')
dspy.configure(lm=lm)

sig = dspy.Signature("question:str -> answer:str")

class SimpleQA(dspy.Module):
    def __init__(self):
        super().__init__()
        self.predictor = dspy.Predict(sig)

    def forward(self, question: str):
        return self.predictor(question=question)

if __name__ == "__main__":
    q = sys.argv[1] if len(sys.argv) > 1 else "What is the " \
                                             "capital of France?"
    print(q)
    qa = SimpleQA()
    # Execute
    result = qa(question=q)
    print("=== Prediction ===")
    print(result)  # Prediction(answer="Paris"), with metadata
    # Show the actual prompt for intuition:
    print("=== Generated Prompt ===")
    dspy.inspect_history(n=1)
    print("========================\n")

1. Inspect the prompt—you see exactly what you’re asking the model.
2. Run it—you get back a typed Prediction object, not just raw text.

What is the capital of France?
=== Prediction ===
Prediction(
    answer='Paris'
)
=== Generated Prompt ===




[2025-07-05T14:50:51.853070]

System message:

Your input fields are:
1. `question` (str):
Your output fields are:
1. `answer` (str):
All interactions will be structured in the following way, with the appropriate values filled in.

[[ ## question ## ]]
{question}

[[ ## answer ## ]]
{answer}

[[ ## completed ## ]]
In adhering to this structure, your objective is: 
        Given the fields `question`, produce the fields `answer`.


User message:

[[ ## question ## ]]
What is the capital of France?

Respond with the corresponding output fields, starting with the field `[[ ## answer ## ]]`, and then ending with the marker for `[[ ## completed ## ]]`.


Response:

[[ ## question ## ]]
What is the capital of France?
[[ ## answer ## ]]
Paris
[[ ## completed ## ]]





========================

Revisiting Intuition

Signature → Prompt: Every field in your Signature becomes a placeholder in the template. Visualize these as slots in an email template.
Module → Pipeline: A Module is more than a function; it encapsulates the entire lifecycle—compile, adapt, execute, parse.
Adapter → Backend: Think of adapters like database drivers: you can switch from SQLite to Postgres by changing one line of config.
Prediction → Value Object: The model’s reply arrives as a structured object you can introspect, test, and log easily.

Pitfalls / Advice

1. Overthinking Signatures: Keep them simple at first. You can always extend later.
2. Ignoring inspect_history(): If you don’t look at the generated prompt, you’re blind to DSPy’s “magic.” Always start your tutorials by printing it.
3. Tight coupling to one backend: Resist the urge to sprinkle openai calls in your code. Trust DSPy’s adapters—they save you from rewrite hell.
4. Skipping type validation: Use Signatures to catch simple errors (e.g., passing an int where a str was needed).

In Part 2, we’ll wire in a realistic JSONL knowledge base, build a retrieval module, and compose it into a two-stage pipeline. This will demonstrate DSPy’s true power in building maintainable, real-world AI applications.