LLM explainability is usually discussed in the abstract. Today we are building a concrete audit tool that captures a model's raw output and then generates a structured reasoning report, so you can ship applications where every answer is inspectable. We will use Oxlo.ai because its per-request pricing means running a two-pass pipeline, a generator plus an analyzer, stays predictable even when the context grows.
What you'll need
- Python 3.10 or newer.
- An Oxlo.ai API key from https://portal.oxlo.ai.
- The OpenAI SDK:
pip install openai.
Step 1: Bootstrap the Oxlo.ai client
I start every project by verifying the connection. This script pings Oxlo.ai with a trivial prompt to confirm the key and base URL are working. I use Llama 3.3 70B as the workhorse because it is reliable for general-purpose tasks.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Say 'Connection OK' and nothing else."},
],
)
print(response.choices[0].message.content)Step 2: Capture the raw response
Next, I wrap the call in a function that accepts a user question and returns the assistant's text. I keep the system prompt neutral so the model does not add extra embellishments that could confuse the audit pass.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
def generate_answer(question: str) -> str:
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": "Answer concisely and accurately."},
{"role": "user", "content": question},
],
)
return response.choices[0].message.content
if __name__ == "__main__":
print(generate_answer("What caused the 1929 stock market crash?"))Step 3: Design the audit prompt
This is the core of the tool. The second pass does not guess the answer again. It treats the question and the first answer as evidence, then produces a JSON object with four fields: reasoning_steps, confidence, potential_hallucinations, and source_reliability. I use Qwen 3 32B for this pass because it handles structured multilingual reasoning well.
SYSTEM_PROMPT = """
You are an explainability engine. Your job is to audit a question-answer pair produced by an LLM.
Output strictly valid JSON with no markdown formatting. Use this schema:
{
"reasoning_steps": [list of logical steps the LLM likely took],
"confidence": "high" | "medium" | "low",
"potential_hallucinations": [list of claims that appear unverified or speculative],
"source_reliability": "high" | "medium" | "low"
}
Be critical. If the answer contains predictions, unsourced statistics, or vague generalizations, flag them.
"""Step 4: Build the explainer pipeline
Now I wire the two passes together. The explainer function takes the original question and the raw answer, injects them into a user message template, and sends everything to the audit model. I parse the JSON response and return a native Python dict.
import json
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """
You are an explainability engine. Your job is to audit a question-answer pair produced by an LLM.
Output strictly valid JSON with no markdown formatting. Use this schema:
{
"reasoning_steps": [list of logical steps the LLM likely took],
"confidence": "high" | "medium" | "low",
"potential_hallucinations": [list of claims that appear unverified or speculative],
"source_reliability": "high" | "medium" | "low"
}
Be critical. If the answer contains predictions, unsourced statistics, or vague generalizations, flag them.
"""
def audit_answer(question: str, answer: str) -> dict:
user_message = f"Question: {question}\n\nAnswer: {answer}\n\nAudit this answer."
response = client.chat.completions.create(
model="qwen-3-32b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user_message},
],
)
raw = response.choices[0].message.content
return json.loads(raw)Step 5: Wrap it in a CLI
Finally, I add a small interactive loop so you can type a question, see the model's answer, and immediately see the explainability breakdown. I pretty-print the JSON with indentation.
import json
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """
You are an explainability engine. Your job is to audit a question-answer pair produced by an LLM.
Output strictly valid JSON with no markdown formatting. Use this schema:
{
"reasoning_steps": [list of logical steps the LLM likely took],
"confidence": "high" | "medium" | "low",
"potential_hallucinations": [list of claims that appear unverified or speculative],
"source_reliability": "high" | "medium" | "low"
}
Be critical. If the answer contains predictions, unsourced statistics, or vague generalizations, flag them.
"""
def generate_answer(question: str) -> str:
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": "Answer concisely and accurately."},
{"role": "user", "content": question},
],
)
return response.choices[0].message.content
def audit_answer(question: str, answer: str) -> dict:
user_message = f"Question: {question}\n\nAnswer: {answer}\n\nAudit this answer."
response = client.chat.completions.create(
model="qwen-3-32b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user_message},
],
)
raw = response.choices[0].message.content
return json.loads(raw)
if __name__ == "__main__":
question = input("Ask something: ")
print("\n--- RAW ANSWER ---\n")
answer = generate_answer(question)
print(answer)
print("\n--- EXPLAINABILITY REPORT ---\n")
report = audit_answer(question, answer)
print(json.dumps(report, indent=2))Run it
I ran the script and asked a question that mixes historical fact with a request for prediction, which usually exposes weak reasoning.
$ python explainability_debugger.py
Ask something: Explain the causes of the 1929 stock market crash and predict when the next one will happen.
--- RAW ANSWER ---
The 1929 crash was caused by speculative buying, margin debt, and a lack of regulatory oversight. The next crash could happen within the next few years due to rising interest rates.
--- EXPLAINABILITY REPORT ---
{
"reasoning_steps": [
"Identified historical factors: speculation, margin debt, regulation.",
"Shifted from historical analysis to future prediction without data.",
"Used vague temporal phrasing: 'within the next few years'."
],
"confidence": "medium",
"potential_hallucinations": [
"Claim that the next crash will happen within a few years due to rising interest rates is speculative and unsourced."
],
"source_reliability": "low"
}Wrap-up
That is the entire pipeline. Two concrete next steps: wire this into FastAPI middleware so every production response gets audited automatically, or swap the audit model to DeepSeek R1 671B on Oxlo.ai when you need deeper reasoning on complex coding outputs. Because Oxlo.ai charges a flat rate per request, adding this audit pass does not scale in cost with token count, which makes long-context quality assurance practical.
Top comments (0)