Debugging LLM Models: A Step-by-Step Guide

#engineering #oxlo #ai

I shipped an internal tool last quarter that automates the worst part of prompt engineering: figuring out why a model hallucinates, ignores instructions, or returns malformed JSON. In this guide, I will walk you through building the same LLM debugger so you can catch errors before they reach users.

What you'll need

Python 3.10 or newer, an Oxlo.ai API key from https://portal.oxlo.ai, and the OpenAI SDK installed with pip install openai.

Step 1: Scaffold the debugger client and capture a failing call

First, wire up the OpenAI SDK to hit Oxlo.ai. I use Llama 3.3 70B as the target model and deliberately send a vague prompt that asks for JSON but does not provide a schema. This gives us a realistic failure to diagnose.

import json
from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

def buggy_call():
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Give me info about the user. Return JSON."},
        ],
    )
    return response.choices[0].message.content

raw_output = buggy_call()
print("RAW OUTPUT:\n", raw_output)

Step 2: Define the debugger system prompt

The debugger is a separate model call that acts like a senior engineer reviewing a bug ticket. I use Qwen 3 32B because its reasoning is strong for agent workflows. Store the prompt in a constant so you can iterate without touching business logic.

DEBUGGER_PROMPT = """You are a prompt debugging engine. Analyze the provided prompt, the expected behavior, and the actual model output.

Identify exactly one of these failure modes:
1. Instruction ambiguity
2. Missing formatting constraints
3. Context overflow or truncation
4. Reasoning shortcut or hallucination

Then provide:
- diagnosis: one sentence describing the root cause
- fix: a rewritten system prompt or user message that resolves the issue
- confidence: low, medium, or high

Respond in JSON with keys: failure_mode, diagnosis, fix, confidence."""

Step 3: Build the analysis pipeline

Now pipe the raw failure into the debugger. I package the original prompt, the expected result, and the actual output into a single user message. Qwen 3 32B handles the long context well, which matters when you are debugging multi-turn conversations later.

def analyze_failure(original_prompt, actual_output, expected_schema):
    user_message = f"""Original system prompt:
{original_prompt}

User request:
Give me info about the user. Return JSON.

Expected behavior:
Return valid JSON matching this schema:
{expected_schema}

Actual model output:
{actual_output}

Diagnose the failure and provide the fix."""

    response = client.chat.completions.create(
        model="qwen-3-32b",
        messages=[
            {"role": "system", "content": DEBUGGER_PROMPT},
            {"role": "user", "content": user_message},
        ],
    )
    return response.choices[0].message.content

expected_schema = '{"name": "string", "age": "integer"}'
diagnosis = analyze_failure("You are a helpful assistant.", raw_output, expected_schema)
print("DIAGNOSIS:\n", diagnosis)

Step 4: Implement the auto-fix loop

A diagnosis is only useful if we verify it. I parse the suggested fix, swap it into a fresh call, and compare the new output against the schema. If it still fails, we log the incident for manual review. This closes the loop.

def test_fix(diagnosis_json):
    parsed = json.loads(diagnosis_json)
    fixed_prompt = parsed["fix"]

    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": fixed_prompt},
            {"role": "user", "content": "Give me info about the user."},
        ],
    )
    new_output = response.choices[0].message.content

    # naive schema check
    try:
        parsed_json = json.loads(new_output)
        assert "name" in parsed_json and "age" in parsed_json
        print("FIX VERIFIED. Output:", new_output)
        return True
    except Exception:
        print("FIX FAILED. Output:", new_output)
        return False

test_fix(diagnosis)

Run it

Here is the full script entry point. When I ran this against Oxlo.ai last week, the debugger correctly flagged instruction ambiguity and injected a strict JSON schema constraint. The corrected prompt produced valid JSON on the first retry.

if __name__ == "__main__":
    print("=== Capturing buggy call ===")
    raw = buggy_call()

    print("\n=== Analyzing failure ===")
    diag = analyze_failure("You are a helpful assistant.", raw, expected_schema)
    print(diag)

    print("\n=== Testing fix ===")
    try:
        test_fix(diag)
    except Exception as e:
        print("Unhandled error:", e)

Example output:

=== Capturing buggy call ===
RAW OUTPUT:
 Sure, here is the info: Name is Alice and she is 30 years old.

=== Analyzing failure ===
{
  "failure_mode": "Missing formatting constraints",
  "diagnosis": "The prompt asks for JSON but provides no schema or formatting rules, so the model returns plain text.",
  "fix": "You are a helpful assistant. Always respond with valid JSON matching the schema {\"name\": \"string\", \"age\": \"integer\"}. Do not add markdown code blocks or explanatory text.",
  "confidence": "high"
}

=== Testing fix ===
FIX VERIFIED. Output: {"name": "Alice", "age": 30}

Wrap-up

This debugger runs on Oxlo.ai's request-based pricing, so iterating on long prompts or running batch regression tests does not scale in cost with input length. See https://oxlo.ai/pricing for details. Next, wire this into a pytest suite that runs on every pull request, or extend the pipeline to diff outputs across multiple models like DeepSeek V3.2 and Kimi K2.6 to catch model-specific regressions.

DEV Community