Overview
We are building a lightweight interpretability pipeline that treats an LLM as a subject to be analyzed, not just queried. The script captures token-level logprobs from an Oxlo.ai model, then uses a second model to generate a readable report explaining uncertainty and likely reasoning paths. This helps developers debug prompt sensitivity without managing local infrastructure.
What you'll need
- Python 3.10 or newer
- The OpenAI SDK:
pip install openai - An Oxlo.ai API key from https://portal.oxlo.ai
Step 1: Configure the Oxlo.ai client
Import the SDK and point it at the Oxlo.ai base URL. Using the OpenAI-compatible client means the rest of the code is portable, but we will run everything against Oxlo.ai.
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key=os.getenv("OXLO_API_KEY", "YOUR_OXLO_API_KEY")
)Step 2: Generate a target response with logprobs
We send an intentionally ambiguous prompt to DeepSeek V3.2 and request logprobs so we can inspect the probability mass at every token position.
import json
TARGET_SYSTEM_PROMPT = "You are a concise assistant."
TARGET_PROMPT = (
"Explain why the following statement might be controversial: "
"'AI alignment is already solved.'"
)
target_response = client.chat.completions.create(
model="deepseek-v3.2",
messages=[
{"role": "system", "content": TARGET_SYSTEM_PROMPT},
{"role": "user", "content": TARGET_PROMPT},
],
logprobs=True,
top_logprobs=5,
max_tokens=200,
)
target_text = target_response.choices[0].message.content
logprobs_data = target_response.choices[0].logprobs.content
print("Generated text:\n", target_text)
print("Token count:", len(logprobs_data))Step 3: Normalize the probability data
The API returns logprobs as negative floats. We convert them to percentages and keep the top alternative tokens so the analyst model has clean, structured input.
import math
tokens = []
for entry in logprobs_data:
token_info = {
"token": entry.token,
"probability": round(math.exp(entry.logprob) * 100, 2),
"top_alternatives": [
{
"token": alt.token,
"probability": round(math.exp(alt.logprob) * 100, 2),
}
for alt in entry.top_logprobs
],
}
tokens.append(token_info)
print(json.dumps(tokens[:3], indent=2))Step 4: Build the interpretability analyst prompt
We define a strict system prompt for the analyst and assemble a JSON payload containing the original prompt, the generated text, and the full token map.
ANALYST_SYSTEM_PROMPT = """You are an LLM interpretability analyst. Your job is to inspect a model's output and its per-token probability distribution, then produce a concise technical report.
For each section of the response, report:
1. High-certainty phrases (probability > 90%) that indicate memorized or default patterns.
2. Low-certainty transitions (probability < 60%) where the model was ambivalent.
3. The top alternative tokens at uncertain positions and what they imply about the model's latent reasoning.
4. A one-paragraph hypothesis about what features of the prompt caused the observed uncertainty.
Be precise. Do not speculate beyond the data. Format your report in Markdown."""
payload = {
"original_prompt": TARGET_PROMPT,
"generated_text": target_text,
"token_analysis": tokens,
}
user_message = (
"Analyze the following generation data and produce your report.\n\n"
f"
```json\n{json.dumps(payload, indent=2)}\n```
"
)Step 5: Generate the interpretability report
We call Kimi K2.6 to synthesize the report. Because Oxlo.ai uses flat per-request pricing, passing a large JSON context of token probabilities does not inflate cost; you can see details at https://oxlo.ai/pricing.
report_response = client.chat.completions.create(
model="kimi-k2.6",
messages=[
{"role": "system", "content": ANALYST_SYSTEM_PROMPT},
{"role": "user", "content": user_message},
],
temperature=0.2,
max_tokens=1500,
)
report = report_response.choices[0].message.content
print(report)Run it
Save the full script as interpretability_debugger.py, set your OXLO_API_KEY, and execute it. The first call generates text and probabilities, and the second call produces the analysis.
$ python interpretability_debugger.pyExample output:
Generated text:
The claim that "AI alignment is already solved" is controversial because it dismisses ongoing technical and ethical challenges. Researchers continue to debate reward hacking, distributional shift, and oversight. Declaring the problem solved risks complacency in safety-critical systems.
Token count: 56
[
{
"token": " The",
"probability": 92.15,
"top_alternatives": [
{"token": "This", "probability": 5.43},
{"token": "In", "probability": 1.02}
]
},
{
"token": " claim",
"probability": 97.88,
"top_alternatives": [
{"token": " statement", "probability": 1.10},
{"token": " idea", "probability": 0.55}
]
},
{
"token": " that",
"probability": 99.12,
"top_alternatives": [
{"token": "\"", "probability": 0.30},
{"token": " is", "probability": 0.20}
]
}
]
--- Interpretability Report ---
**High-certainty phrases**
- The opening phrase "The claim that" shows probabilities above 92%, indicating a standard argumentative framing pattern.
- Punctuation marks and common conjunctions consistently exceed 98%, reflecting strong syntactic confidence.
**Low-certainty transitions**
- At token position 18, "already" is selected with 58% probability. The top alternative is "mostly" (26%), suggesting the model weighed degrees of completion before accepting the prompt's exact wording.
- At position 42, "solved" carries 61% probability versus "addressed" (29%). This signals semantic tension between absolute and partial resolution.
**Alternative token implications**
- The presence of "mostly" as an alternative implies the model briefly considered softening the stance, likely because the prompt itself is provocative and the model sought a neutral analytical register.
**Hypothesis**
- The prompt's use of scare quotes around a bold assertion created a meta-linguistic cue. The model defaulted to neutral analysis but showed uncertainty when choosing evaluative adverbs, likely because no explicit stance was provided in the prompt. This pattern suggests high sensitivity to framing devices in the input.
Wrap-up and next steps
You now have a working pipeline that turns raw logprobs into readable interpretability signals. A practical extension is to automate A/B testing across models: run the same prompt through Llama 3.3 70B and Qwen 3 32B on Oxlo.ai, then compare where each model hesitates. Because Oxlo.ai charges a flat rate per request, these multi-model experiments stay predictable even when you pass large token maps as context. Another next step is to feed the report back into the prompt as a critique loop, letting the model rewrite its own output to reduce unwanted uncertainty.
Top comments (0)