I built a black-box interpretability scanner that treats any LLM as a target and extracts attribution hypotheses through contrastive prompting. It runs multiple perturbations of a prompt, compares the outputs, and uses a strong reasoning model to explain which input features drove the response. This is useful for engineers auditing model behavior without access to internal weights or activations.
What You'll Need
- Python 3.10 or newer
- The OpenAI SDK:
pip install openai - An Oxlo.ai API key from https://portal.oxlo.ai
Step 1: Capture the Baseline Response
First I need a target prompt and a baseline completion from the model I want to interpret. I use Llama 3.3 70B on Oxlo.ai as the target because it is a reliable general-purpose model, and I store the response for later comparison.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
TARGET_MODEL = "llama-3.3-70b"
BASELINE_PROMPT = "Explain why the sky appears blue during daytime without using the word 'scattering'."
response = client.chat.completions.create(
model=TARGET_MODEL,
messages=[
{"role": "user", "content": BASELINE_PROMPT},
],
temperature=0.2,
)
baseline_text = response.choices[0].message.content
print("Baseline response:")
print(baseline_text)Step 2: Generate Contrastive Perturbations
Black-box interpretability relies on counterfactuals. I generate perturbed versions of the baseline prompt, such as negation and entity substitution, then run each through the same target model. Because Oxlo.ai charges a flat rate per request, running ten perturbations against a long prompt does not balloon in cost the way token-based pricing would.
import json
def perturb_prompt(original):
perturbations = [
{"label": "negation", "prompt": original.replace("without", "with")},
{"label": "entity_swap", "prompt": original.replace("sky", "ocean").replace("blue", "red")},
{"label": "instruction_reversal", "prompt": "Explain why the sky appears blue during daytime. You must use the word 'scattering'."},
]
return perturbations
perturbations = perturb_prompt(BASELINE_PROMPT)
results = []
for p in perturbations:
resp = client.chat.completions.create(
model=TARGET_MODEL,
messages=[{"role": "user", "content": p["prompt"]}],
temperature=0.2,
)
results.append({
"label": p["label"],
"prompt": p["prompt"],
"response": resp.choices[0].message.content
})
print(json.dumps(results, indent=2))Step 3: Run Attribution Analysis
Now I need an analyzer to compare the baseline against the perturbations. I use Kimi K2.6 on Oxlo.ai for advanced reasoning. The analyzer identifies which input constraints drove specific phrases in the output. This simulates attribution patching at the API layer.
SYSTEM_PROMPT = """You are a mechanistic interpretability analyst. Your job is to examine how input prompt changes affect model outputs.
You will receive:
1. A BASELINE prompt and response
2. A list of PERTURBATIONS (changed prompts and their responses)
For each perturbation, explain:
- Which specific words or constraints changed in the input
- How the output changed as a result
- What this reveals about what the model was attending to when it produced the baseline
Be concise and technical. Respond in raw JSON with keys: perturbation_label, changed_tokens, output_delta, attribution_hypothesis.
"""ANALYZER_MODEL = "kimi-k2.6"
analysis_input = f"""
BASELINE PROMPT: {BASELINE_PROMPT}
BASELINE RESPONSE: {baseline_text}
PERTURBATIONS:
{json.dumps(results, indent=2)}
"""
response = client.chat.completions.create(
model=ANALYZER_MODEL,
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": analysis_input},
],
response_format={"type": "json_object"},
)
attribution_report = json.loads(response.choices[0].message.content)
print(json.dumps(attribution_report, indent=2))Step 4: Extract a Reasoning Trace
To approximate internal feature activation, I force the target model to emit its chain of thought before the final answer. I then analyze that trace with a second pass. DeepSeek V3.2 on Oxlo.ai handles this reasoning-heavy step well, and it is available on the free tier for initial experiments.
COT_PROMPT = BASELINE_PROMPT + "\n\nBefore answering, explain your step-by-step reasoning inside <thinking> tags."
cot_response = client.chat.completions.create(
model="deepseek-v3.2",
messages=[{"role": "user", "content": COT_PROMPT}],
temperature=0.2,
)
reasoning_text = cot_response.choices[0].message.content
print(reasoning_text)Step 5: Compile the Structured Interpretability Report
Finally, I synthesize the baseline, contrastive results, attribution hypotheses, and reasoning trace into one JSON document. I use Llama 3.3 70B again to keep formatting consistent and fast.
FINAL_SYSTEM_PROMPT = """You are a report compiler. Combine the provided interpretability artifacts into a single valid JSON object with this exact schema:
{
"baseline_summary": string,
"contrastive_findings": array,
"attribution_hypotheses": array,
"reasoning_trace_present": boolean,
"recommended_followup": string
}
Do not include markdown formatting. Output raw JSON only.
"""
artifacts = f"""
BASELINE: {baseline_text}
CONTRASTIVE RESULTS: {json.dumps(results)}
ATTRIBUTION REPORT: {json.dumps(attribution_report)}
REASONING TRACE: {reasoning_text}
"""
final = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": FINAL_SYSTEM_PROMPT},
{"role": "user", "content": artifacts},
],
response_format={"type": "json_object"},
)
report = json.loads(final.choices[0].message.content)
print(json.dumps(report, indent=2))Run It
Save the script as interpretability_scanner.py, set your Oxlo.ai API key, and run it against any prompt you need to debug. Here is a complete end-to-end invocation using Qwen 3 32B as the target for a multilingual reasoning task.
import json
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
TARGET = "qwen-3-32b"
ANALYZER = "kimi-k2.6"
user_message = "If a train travels 60 km in 30 minutes, how far will it travel in 2 hours? Explain your reasoning."
# Baseline
base = client.chat.completions.create(
model=TARGET,
messages=[{"role": "user", "content": user_message}],
temperature=0.2,
).choices[0].message.content
# Perturbation: changed numbers
perturbed = "If a train travels 120 km in 60 minutes, how far will it travel in 3 hours? Explain your reasoning."
pert = client.chat.completions.create(
model=TARGET,
messages=[{"role": "user", "content": perturbed}],
temperature=0.2,
).choices[0].message.content
# Analyze
hypothesis = client.chat.completions.create(
model=ANALYZER,
messages=[
{"role": "system", "content": "You compare two model outputs and state which input tokens most influenced the reasoning path. Return JSON."},
{"role": "user", "content": f"Baseline output: {base}\n\nPerturbed output: {pert}"},
],
response_format={"type": "json_object"},
).choices[0].message.content
print(json.dumps({
"baseline": base,
"perturbed_output": pert,
"analysis": json.loads(hypothesis)
}, indent=2))Example output:
{
"baseline": "The train travels 60 km in 30 minutes...",
"perturbed_output": "The train travels 120 km in 60 minutes...",
"analysis": {
"changed_tokens": ["60 km", "30 minutes", "2 hours", "120 km", "60 minutes", "3 hours"],
"output_delta": "The final distance scaled proportionally...",
"attribution_hypothesis": "The model locked onto the numerical ratio and time-unit conversion rather than surface wording."
}
}Wrap Up and Next Steps
This scanner gives you a reproducible black-box interpretability workflow using nothing but API calls. Because Oxlo.ai uses flat per-request pricing, you can scale the perturbation grid to twenty or thirty variants without worrying about long-context token charges.
Two concrete next steps: integrate logprobs to measure token-level probability shifts when they are supported by your provider, and automate the perturbation generator with a synonym graph so you can run unsupervised attribution sweeps across entire datasets.
Top comments (0)