We are going to build an LLM interpretability probe that dissects a model's chain-of-thought reasoning using contrastive analysis, input attribution, and self-consistency checks. This tool helps engineers audit agentic pipelines for hidden biases or fragile logic before they reach production. Because the probe issues multiple long-context requests per audit, Oxlo.ai's flat per-request pricing keeps costs predictable regardless of how verbose the model gets.
What you'll need
- Python 3.10 or newer
- The OpenAI SDK:
pip install openai - An Oxlo.ai API key from https://portal.oxlo.ai
Every snippet below calls Oxlo.ai through the OpenAI-compatible endpoint at https://api.oxlo.ai/v1.
Step 1: Set Up the Oxlo.ai Client and Benchmark Task
I start by importing the SDK and defining a deliberately ambiguous reasoning task. The bat-and-ball puzzle is a classic test because most models carry a latent bias toward the intuitive but wrong answer.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
BENCHMARK_PROMPT = (
"A bat and a ball cost $11 total. "
"The bat costs $10 more than the ball. "
"How much does the ball cost? Think step by step."
)
Step 2: Generate the Base Reasoning Trace
Next I generate the initial chain-of-thought response that the rest of the pipeline will dissect. I keep the solver system prompt minimal so the model focuses on the puzzle rather than formatting.
SOLVER_PROMPT = "You are a careful reasoning assistant. Think step by step before giving your final answer."
solver_response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SOLVER_PROMPT},
{"role": "user", "content": BENCHMARK_PROMPT},
],
)
base_answer = solver_response.choices[0].message.content
print(base_answer)
Step 3: Define the Interpretability Agent and Extract Structure
Now I define the interpretability agent. Its job is to inspect the solver's output and return structured findings. Here is the agent's system prompt.
AGENT_SYSTEM_PROMPT = (
"You are an LLM interpretability analyst. "
"Follow the user's instructions exactly. "
"Always output strictly valid JSON with no markdown formatting."
)
Using that system prompt, I ask the agent to extract the explicit reasoning, latent assumptions, and a robustness score.
reasoning_prompt = (
f"User question: {BENCHMARK_PROMPT}\n\n"
f"Model answer: {base_answer}\n\n"
"Extract the model's explicit reasoning, latent assumptions, and robustness. "
"Output JSON with keys: quoted_reasoning, latent_assumptions, confidence."
)
reasoning_response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": AGENT_SYSTEM_PROMPT},
{"role": "user", "content": reasoning_prompt},
],
)
print(reasoning_response.choices[0].message.content)
Step 4: Run a Contrastive Probe
Contrastive analysis surfaces what the model rejected. I prompt the agent to propose the strongest alternative answer and explain why the solver likely discarded it.
contrastive_prompt = (
f"User question: {BENCHMARK_PROMPT}\n\n"
f"Model answer: {base_answer}\n\n"
"What is the strongest alternative answer and why was it rejected? "
"Output JSON with keys: alternative_answer, rejection_reason."
)
contrastive_response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": AGENT_SYSTEM_PROMPT},
{"role": "user", "content": contrastive_prompt},
],
)
print(contrastive_response.choices[0].message.content)
Step 5: Extract Salient Input Attribution
Without access to internal attention weights, I simulate input attribution by asking the agent to quote the exact words from the prompt that most influenced the solver's conclusion.
attribution_prompt = (
f"User question: {BENCHMARK_PROMPT}\n\n"
f"Model answer: {base_answer}\n\n"
"Which exact phrases in the question most influenced the answer? "
"Output JSON with keys: salient_phrases, influence_explanation."
)
attribution_response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": AGENT_SYSTEM_PROMPT},
{"role": "user", "content": attribution_prompt},
],
)
print(attribution_response.choices[0].message.content)
Step 6: Measure Self-Consistency
Finally, I sample the solver three times at a higher temperature to check for variance. If the reasoning path or final answer flips across samples, the underlying logic is unstable.
samples = []
for _ in range(3):
resp = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SOLVER_PROMPT},
{"role": "user", "content": BENCHMARK_PROMPT},
],
temperature=0.8,
)
samples.append(resp.choices[0].message.content)
for idx, text in enumerate(samples, 1):
print(f"--- Sample {idx} ---")
print(text[:400] + "...\n")
Run It
The full script below wires all six steps into a single audit. Running it against Llama 3.3 70B produces a JSON report that you can log or diff across prompt versions.
from openai import OpenAI
import json
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
BENCHMARK_PROMPT = (
"A bat and a ball cost $11 total. "
"The bat costs $10 more than the ball. "
"How much does the ball cost? Think step by step."
)
SOLVER_PROMPT = "You are a careful reasoning assistant. Think step by step before giving your final answer."
AGENT_SYSTEM_PROMPT = (
"You are an LLM interpretability analyst. "
"Follow the user's instructions exactly. "
"Always output strictly valid JSON with no markdown formatting."
)
# 1. Solve
solver_resp = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SOLVER_PROMPT},
{"role": "user", "content": BENCHMARK_PROMPT},
],
)
base_answer = solver_resp.choices[0].message.content
# 2. Structured reasoning extraction
reasoning_prompt = (
f"User question: {BENCHMARK_PROMPT}\n\n"
f"Model answer: {base_answer}\n\n"
"Extract the model's explicit reasoning, latent assumptions, and robustness. "
"Output JSON with keys: quoted_reasoning, latent_assumptions, confidence."
)
reasoning_resp = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": AGENT_SYSTEM_PROMPT},
{"role": "user", "content": reasoning_prompt},
],
)
# 3. Contrastive probe
contrastive_prompt = (
f"User question: {BENCHMARK_PROMPT}\n\n"
f"Model answer: {base_answer}\n\n"
"What is the strongest alternative answer and why was it rejected? "
"Output JSON with keys: alternative_answer, rejection_reason."
)
contrastive_resp = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": AGENT_SYSTEM_PROMPT},
{"role": "user", "content": contrastive_prompt},
],
)
# 4. Attribution
attribution_prompt = (
f"User question: {BENCHMARK_PROMPT}\n\n"
f"Model answer: {base_answer}\n\n"
"Which exact phrases in the question most influenced the answer? "
"Output JSON with keys: salient_phrases, influence_explanation."
)
attribution_resp = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": AGENT_SYSTEM_PROMPT},
{"role": "user", "content": attribution_prompt},
],
)
# 5. Consistency
samples = []
for _ in range(3):
s = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SOLVER_PROMPT},
{"role": "user", "content": BENCHMARK_PROMPT},
],
temperature=0.8,
)
samples.append(s.choices[0].message.content)
report = {
"base_answer": base_answer,
"structured_reasoning": reasoning_resp.choices[0].message.content,
"contrastive_analysis": contrastive_resp.choices[0].message.content,
"input_attribution": attribution_resp.choices[0].message.content,
"consistency_samples": samples,
}
print(json.dumps(report, indent=2))
Typical output looks like this. The structured reasoning block usually flags the latent assumption that "more than" implies simple addition, while the consistency samples reveal whether the model corrects itself on rerolls.
{
"base_answer": "Let x be the cost of the ball...",
"structured_reasoning": "{\"quoted_reasoning\": \"Let x be the cost of the ball...\", \"latent_assumptions\": \"The model assumes 'more than' maps directly to addition without verifying the total constraint.\", \"confidence\": 0.6}",
"contrastive_analysis": "{\"alternative_answer\": \"$1\", \"rejection_reason\": \"The model rejected $1 because it did not check that the bat would then cost $11, breaking the total.\"}",
"input_attribution": "{\"salient_phrases\": [\"$10 more\", \"$11 total\"], \"influence_explanation\": \"The numbers triggered an intuitive arithmetic shortcut.\"}",
"consistency_samples": [
"Let x be the cost of the ball...",
"The ball costs $0.50 because...",
"If the ball is $1, the bat is $11..."
]
}
Wrap-Up
You can schedule this probe in CI to regression-test your prompts every time you ship a new system prompt. You can also point it at qwen-3-32b or kimi-k2.6 on Oxlo.ai to compare how different architectures handle the same ambiguous input. For pricing details on running these multi-request audits at scale, see https://oxlo.ai/pricing.
Top comments (0)