Today we are building a self-explaining security review agent that exposes its chain-of-thought reasoning and token-level confidence. This gives developers a practical window into why an LLM makes a specific decision, which is essential before putting any automated system into production.
What you'll need
- Python 3.10 or newer
- The OpenAI SDK installed with
pip install openai - An Oxlo.ai API key from https://portal.oxlo.ai
I am running this on Oxlo.ai because its request-based pricing keeps the cost flat even when we pass long code snippets and detailed reasoning traces. For current plan details, see https://oxlo.ai/pricing.
Step 1: Configure the Oxlo.ai client
Start by importing the SDK and pointing it at Oxlo.ai. This is a literal drop-in replacement for any OpenAI-compatible script.
from openai import OpenAI
import os
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key=os.environ.get("OXLO_API_KEY", "YOUR_OXLO_API_KEY"),
)Step 2: Define the reasoning agent prompt
To make the model's reasoning inspectable, I force it to separate its internal monologue from its final verdict using strict XML tags.
REASONING_SYSTEM_PROMPT = """You are a security review assistant. Your job is to analyze a user-supplied code snippet and decide if it is Safe, Suspicious, or Vulnerable.
Follow these rules exactly:
1. Think step by step about the code inside tags.
2. State your final verdict inside tags using exactly one word: Safe, Suspicious, or Vulnerable.
3. Be concise but thorough.
"""Step 3: Generate a decision with logprobs
We call Kimi K2.6 through Oxlo.ai with logprobs enabled. The logprobs content will let us inspect the probability mass around the verdict token.
def get_reasoning_and_verdict(user_code: str):
response = client.chat.completions.create(
model="kimi-k2.6",
messages=[
{"role": "system", "content": REASONING_SYSTEM_PROMPT},
{"role": "user", "content": f"Analyze this code:\n\n
```python\n{user_code}\n```
"},
],
logprobs=True,
top_logprobs=5,
temperature=0.2,
max_tokens=800,
)
return response.choices[0].message.content, response.choices[0].logprobs
code_snippet = "user_input = input('Enter command: ')\neval(user_input)"
raw_output, raw_logprobs = get_reasoning_and_verdict(code_snippet)
print(raw_output)Step 4: Parse the trace and extract confidence
Next we pull out the reasoning block and locate the log probability assigned to the verdict token. This number is the core interpretability signal.
import re
def parse_explanation(content: str, logprobs_data):
reasoning_match = re.search(r"(.*?)", content, re.DOTALL)
verdict_match = re.search(r"(.*?)", content, re.DOTALL)
reasoning = reasoning_match.group(1).strip() if reasoning_match else "No reasoning found"
verdict = verdict_match.group(1).strip() if verdict_match else "Unknown"
verdict_token = None
token_prob = None
if logprobs_data and logprobs_data.content and verdict:
for token_info in logprobs_data.content:
if verdict in token_info.token or token_info.token.strip() == verdict.split()[0]:
verdict_token = token_info.token
token_prob = token_info.logprob
break
return reasoning, verdict, verdict_token, token_prob
reasoning, verdict, v_token, v_prob = parse_explanation(raw_output, raw_logprobs)
print(f"Verdict: {verdict}")
print(f"Token: {v_token}, logprob: {v_prob}")Step 5: Audit into a human-readable report
Finally, we feed the raw reasoning and the confidence score into Llama 3.3 70B. This second pass acts as an explainability layer that translates the chain of thought into plain English and flags any logical gaps.
EXPLAINER_PROMPT = """You are an interpretability analyst. Given a model's internal reasoning chain, its final verdict, and the token-level log probability for that verdict, produce a short report that explains:
1. Why the model likely chose this verdict.
2. How confident the model was (convert the log probability to a percentage).
3. Any assumptions or jumps in logic that a human should double-check.
Keep the report under 150 words.
"""
def explain_decision(reasoning: str, verdict: str, logprob: float):
prob_pct = round(100 * (2.718 ** logprob), 2) if logprob is not None else "unknown"
audit_input = f"""Reasoning chain:
{reasoning}
Final verdict: {verdict}
Verdict token log probability: {logprob} (~{prob_pct}% confidence)
"""
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": EXPLAINER_PROMPT},
{"role": "user", "content": audit_input},
],
temperature=0.3,
max_tokens=300,
)
return response.choices[0].message.content
report = explain_decision(reasoning, verdict, v_prob)
print(report)Run it
Here is the complete script. Save it as explain_agent.py, set your OXLO_API_KEY environment variable, and run it.
from openai import OpenAI
import os
import re
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key=os.environ.get("OXLO_API_KEY", "YOUR_OXLO_API_KEY"),
)
REASONING_SYSTEM_PROMPT = """You are a security review assistant. Your job is to analyze a user-supplied code snippet and decide if it is Safe, Suspicious, or Vulnerable.
Follow these rules exactly:
1. Think step by step about the code inside tags.
2. State your final verdict inside tags using exactly one word: Safe, Suspicious, or Vulnerable.
3. Be concise but thorough.
"""
EXPLAINER_PROMPT = """You are an interpretability analyst. Given a model's internal reasoning chain, its final verdict, and the token-level log probability for that verdict, produce a short report that explains:
1. Why the model likely chose this verdict.
2. How confident the model was (convert the log probability to a percentage).
3. Any assumptions or jumps in logic that a human should double-check.
Keep the report under 150 words.
"""
def get_reasoning_and_verdict(user_code: str):
response = client.chat.completions.create(
model="kimi-k2.6",
messages=[
{"role": "system", "content": REASONING_SYSTEM_PROMPT},
{"role": "user", "content": f"Analyze this code:\n\n
```python\n{user_code}\n```
"},
],
logprobs=True,
top_logprobs=5,
temperature=0.2,
max_tokens=800,
)
return response.choices[0].message.content, response.choices[0].logprobs
def parse_explanation(content: str, logprobs_data):
reasoning_match = re.search(r"(.*?)", content, re.DOTALL)
verdict_match = re.search(r"(.*?)", content, re.DOTALL)
reasoning = reasoning_match.group(1).strip() if reasoning_match else "No reasoning found"
verdict = verdict_match.group(1).strip() if verdict_match else "Unknown"
verdict_token = None
token_prob = None
if logprobs_data and logprobs_data.content and verdict:
for token_info in logprobs_data.content:
if verdict in token_info.token or token_info.token.strip() == verdict.split()[0]:
verdict_token = token_info.token
token_prob = token_info.logprob
break
return reasoning, verdict, verdict_token, token_prob
def explain_decision(reasoning: str, verdict: str, logprob: float):
prob_pct = round(100 * (2.718 ** logprob), 2) if logprob is not None else "unknown"
audit_input = f"""Reasoning chain:
{reasoning}
Final verdict: {verdict}
Verdict token log probability: {logprob} (~{prob_pct}% confidence)
"""
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": EXPLAINER_PROMPT},
{"role": "user", "content": audit_input},
],
temperature=0.3,
max_tokens=300,
)
return response.choices[0].message.content
if __name__ == "__main__":
code_snippet = "user_input = input('Enter command: ')\neval(user_input)"
raw_output, raw_logprobs = get_reasoning_and_verdict(code_snippet)
reasoning, verdict, v_token, v_prob = parse_explanation(raw_output, raw_logprobs)
report = explain_decision(reasoning, verdict, v_prob)
print("=== RAW OUTPUT ===")
print(raw_output)
print("\n=== PARSED VERDICT ===")
print(f"{verdict} (token: {v_token}, logprob: {v_prob})")
print("\n=== INTERPRETABILITY REPORT ===")
print(report)Example output:
=== RAW OUTPUT ===
The code reads untrusted user input via input() and passes it directly to eval(). This allows arbitrary code execution, which is a critical security vulnerability.
Vulnerable
=== PARSED VERDICT ===
Vulnerable (token: Vulnerable, logprob: -0.0423)
=== INTERPRETABILITY REPORT ===
The model flagged the direct use of eval() on raw user input as a critical vulnerability. Its confidence is high at approximately 95.8%. The reasoning is sound, though it assumes the input source is truly untrusted. If this runs in a sandboxed environment, a human reviewer might downgrade the severity.
Next steps
Swap in deepseek-v3.2 or qwen-3-32b for the reasoning step to compare how different models justify the same verdict. You could also extend this into a small web service that renders token probabilities as a heatmap over the reasoning text.
Top comments (0)