We are building a lightweight interpretability agent that reverse-engineers why an LLM produced a specific output. It helps engineers debug prompt failures and audit model behavior without access to internal weights. The entire pipeline runs against Oxlo.ai's API using standard OpenAI SDK calls.
What you'll need
- Python 3.10 or newer
- An Oxlo.ai API key from https://portal.oxlo.ai
- The OpenAI SDK:
pip install openai
Step 1: Configure the Oxlo.ai client
Create a single client instance pointing at Oxlo.ai. We will reuse this client to call both the target model and the interpreter model.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Say ok"},
],
max_tokens=5,
)
print(response.choices[0].message.content)Step 2: Create the interpreter prompt
The interpreter prompt instructs the model to act as a mechanistic interpretability analyst. It must decompose a given response into latent assumptions, surface claims, and confidence markers.
INTERPRETER_PROMPT = """You are an LLM interpretability analyst. Your job is to examine a user query and a target LLM response, then produce a structured interpretability report.
For every analysis, provide:
1. Latent Assumptions: unstated premises the target model likely used.
2. Surface Claims: explicit factual assertions and their apparent sources.
3. Alternative Paths: what the target model could have answered but did not.
4. Confidence Score: a rating from 1 to 10 for each surface claim.
5. Attribution Map: mark each claim as Inferred, Retrieved, or Hallucinated.
Be concise. Use bullet points. Do not add fluff."""Step 3: Build the analysis function
This function takes a user query and a target response, then asks the interpreter model to audit it. I use qwen-3-32b here because it handles multilingual reasoning and agentic decomposition well on Oxlo.ai.
def analyze_response(user_query: str, target_response: str) -> str:
payload = f"""User Query:
{user_query}
Target LLM Response:
{target_response}
Produce the interpretability report now."""
response = client.chat.completions.create(
model="qwen-3-32b",
messages=[
{"role": "system", "content": INTERPRETER_PROMPT},
{"role": "user", "content": payload},
],
temperature=0.2,
max_tokens=1024,
)
return response.choices[0].message.contentStep 4: Add contrastive probing
To surface what the target model avoided, we ask the interpreter to generate contrastive continuations. This reveals the decision boundary of the original output.
def contrastive_probe(user_query: str, target_response: str) -> str:
probe_prompt = f"""User Query:
{user_query}
Target Response:
{target_response}
Generate exactly two alternative responses the model could have given. Label them:
- ALT A: a safer, more conservative answer.
- ALT B: a more speculative or creative answer.
Then explain in one sentence why the original response was chosen over each alternative."""
response = client.chat.completions.create(
model="qwen-3-32b",
messages=[
{"role": "system", "content": INTERPRETER_PROMPT},
{"role": "user", "content": probe_prompt},
],
temperature=0.7,
max_tokens=1024,
)
return response.choices[0].message.contentStep 5: Score attribution and confidence
We extract structured scores by forcing the interpreter to return JSON. This lets us programmatically flag high-risk claims.
import json
def score_attribution(user_query: str, target_response: str) -> dict:
scoring_prompt = f"""User Query:
{user_query}
Target Response:
{target_response}
Return ONLY a JSON object with this exact shape:
{{
"overall_confidence": ,
"claims": [
{{"text": "...", "type": "Inferred|Retrieved|Hallucinated", "confidence": }}
]
}}
Do not include markdown fences or explanation."""
response = client.chat.completions.create(
model="qwen-3-32b",
messages=[
{"role": "system", "content": INTERPRETER_PROMPT},
{"role": "user", "content": scoring_prompt},
],
temperature=0.1,
max_tokens=512,
response_format={"type": "json_object"},
)
raw = response.choices[0].message.content
return json.loads(raw)Step 6: Wrap in a CLI audit loop
We combine everything into a script that generates a target response with one model, then audits it with the interpreter.
def generate_target(query: str) -> str:
resp = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "user", "content": query},
],
temperature=0.5,
max_tokens=512,
)
return resp.choices[0].message.content
if __name__ == "__main__":
query = "Why did the Roman Empire fall?"
print("=== TARGET RESPONSE ===")
target = generate_target(query)
print(target)
print("\n=== INTERPRETABILITY REPORT ===")
print(analyze_response(query, target))
print("\n=== CONTRASTIVE PROBE ===")
print(contrastive_probe(query, target))
print("\n=== STRUCTURED SCORES ===")
scores = score_attribution(query, target)
print(json.dumps(scores, indent=2))Run it
Set your key and run the script.
export OXLO_API_KEY="your-key-here"
python interpretability_agent.pyExample output (abbreviated for brevity):
=== TARGET RESPONSE ===
The Roman Empire fell due to a combination of internal instability...
=== INTERPRETABILITY REPORT ===
- Latent Assumptions: the model assumes "fall" refers to the Western Empire in 476 CE.
- Surface Claims: economic overextension (confidence 7), barbarian invasions (confidence 8).
- Alternative Paths: could have focused on the Eastern Empire's continuity.
=== CONTRASTIVE PROBE ===
- ALT A: list only military causes.
- ALT B: argue climate change was primary.
=== STRUCTURED SCORES ===
{
"overall_confidence": 7,
"claims": [
{"text": "economic overextension", "type": "Inferred", "confidence": 7}
]
}Next steps
Hook the scoring output into a pytest suite to regress confidence scores across prompt versions. Or swap in deepseek-r1-671b as the interpreter on Oxlo.ai to test whether a reasoning-heavy model surfaces different latent assumptions than qwen-3-32b.
Top comments (0)