We are building a lightweight troubleshooting agent that ingests error traces or bad outputs from LLM pipelines and returns a root-cause diagnosis plus a corrected code snippet. It helps anyone running production prompts on Oxlo.ai or migrating from token-based providers who needs to debug fast without switching contexts.
What you'll need
- An Oxlo.ai API key from https://portal.oxlo.ai
- Python 3.10 or newer
- The OpenAI SDK:
pip install openai
Step 1: Initialize the Oxlo.ai client
We start by importing the OpenAI SDK and pointing it at Oxlo.ai's flat per-request endpoint. This gives us predictable costs while we iterate on prompts and test fixes against long error traces.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
Step 2: Craft the system prompt
I treat the system prompt as a runbook. It forces the model to return a JSON object with four fields so I can parse it downstream without regex hacks.
SYSTEM_PROMPT = """You are an LLM reliability engineer. A user will paste an error trace, a malformed model response, or a misbehaving prompt.
Analyze the issue and return a single JSON object with these keys:
- symptom: one-sentence description of what went wrong.
- root_cause: explain why it failed.
- fix: concrete steps to resolve it.
- corrected_code: a runnable Python snippet that fixes the issue.
Rules:
- Do not wrap the JSON in markdown fences.
- If the input is an API error, map it to the likely client-side cause.
- If the input is garbled output, suggest parameter or prompt changes."""
Step 3: Build the diagnose function
This function sends the raw error text to Oxlo.ai and attempts to parse the structured response. I default to Llama 3.3 70B for general reasoning, but the flat per-request pricing means I can swap in Qwen 3 32B or DeepSeek V3.2 for a second opinion without worrying about token costs on long traces.
import json
def diagnose(issue_text: str, model: str = "llama-3.3-70b"):
response = client.chat.completions.create(
model=model,
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": issue_text},
],
)
raw = response.choices[0].message.content
try:
return json.loads(raw)
except json.JSONDecodeError:
return {
"symptom": "parse error",
"root_cause": "Model did not return valid JSON.",
"fix": "Retry with qwen-3-32b or deepseek-v3.2.",
"corrected_code": "",
}
Step 4: Add resilience for timeouts and bad parses
Production agents need to handle transient failures and bad parses gracefully. This wrapper catches both client errors and JSON decode issues, then retries once on Oxlo.ai with no cold start.
def diagnose_with_retry(issue_text: str, primary: str = "llama-3.3-70b", fallback: str = "deepseek-v3.2"):
try:
result = diagnose(issue_text, model=primary)
if result.get("symptom") == "parse error":
raise ValueError("Bad JSON from primary model")
return result
except Exception:
response = client.chat.completions.create(
model=fallback,
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": issue_text},
],
)
raw = response.choices[0].message.content
try:
return json.loads(raw)
except json.JSONDecodeError:
return {
"symptom": "persistent parse error",
"root_cause": "Both models returned unstructured text.",
"fix": "Add stricter schema validation or shorten the input trace.",
"corrected_code": "",
}
Step 5: Wire up a CLI runner
Finally, I add a small entrypoint that reads a hardcoded test case so we can verify the agent end-to-end before plugging it into a monitoring pipeline.
if __name__ == "__main__":
sample_issue = """
Traceback (most recent call last):
File "pipeline.py", line 42, in extract_entities
data = json.loads(response.choices[0].message.content)
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes
"""
report = diagnose_with_retry(sample_issue)
print(json.dumps(report, indent=2))
Run it
Save the file as troubleshoot_agent.py, export your key, and execute:
export OXLO_API_KEY="sk-oxlo.ai-..."
python troubleshoot_agent.py
When I ran this against the JSON decode sample, the agent returned:
{
"symptom": "json.decoder.JSONDecodeError when parsing model output.",
"root_cause": "The LLM returned malformed JSON, likely because the prompt did not explicitly request JSON-only output or the temperature was too high.",
"fix": "Set a lower temperature (0.0-0.2) and add an instruction in the system prompt to emit raw JSON without markdown fences.",
"corrected_code": "import json\nfrom openai import OpenAI\n\nclient = OpenAI(base_url='https://api.oxlo.ai/v1', api_key='YOUR_OXLO_API_KEY')\n\nresponse = client.chat.completions.create(\n model='llama-3.3-70b',\n messages=[\n {'role': 'system', 'content': 'Return only valid JSON. No markdown.'},\n {'role': 'user', 'content': 'Extract entities from: Apple is headquartered in Cupertino.'}\n ]\n)\n\ndata = json.loads(response.choices[0].message.content)"
}
Wrap-up and next steps
The flat per-request pricing on Oxlo.ai makes this agent cheap to keep in a dev loop, even when you feed it thousand-token stack traces. For pricing details, see https://oxlo.ai/pricing.
Next, wire this agent into a LangChain callback handler so it automatically triggers on exceptions, or add a Slack slash command so your team can paste traces directly from production alerts.
Top comments (0)