DEV Community

shashank ms
shashank ms

Posted on

Debugging LLM Inference Performance: Best Practices and Techniques

I recently shipped a small internal agent that chews through our Oxlo.ai request logs and tells us exactly why a particular call was slow. It saves us from guessing whether the bottleneck is prompt bloat, model size, or greedy generation parameters. In this walkthrough, I will show you how to build the same thing in under fifty lines of Python.

What you'll need

Before we start, grab the following:

Step 1: Capture a slow baseline

We need something to debug. I will fire a deliberately inefficient request at Oxlo.ai and wrap it in a timer so we have ground truth.

import time
from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

start = time.perf_counter()
response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain quantum computing in exhaustive detail, covering every major algorithm and physical implementation."},
    ],
    max_tokens=4000,
    temperature=0.9,
)
elapsed = time.perf_counter() - start

log_entry = {
    "model": "llama-3.3-70b",
    "request_messages": [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain quantum computing in exhaustive detail, covering every major algorithm and physical implementation."},
    ],
    "max_tokens": 4000,
    "temperature": 0.9,
    "total_duration_sec": round(elapsed, 2),
    "prompt_tokens": response.usage.prompt_tokens,
    "completion_tokens": response.usage.completion_tokens,
}

print(f"duration={log_entry['total_duration_sec']}s prompt_tokens={log_entry['prompt_tokens']} completion_tokens={log_entry['completion_tokens']}")

Step 2: Serialize the log

The debugger needs a compact JSON view. I will format the log, truncating long strings so we do not flood the context window.

import json

def serialize_log(entry):
    messages = []
    for m in entry["request_messages"]:
        text = m["content"]
        preview = text if len(text) < 180 else text[:180] + " ... [truncated]"
        messages.append({"role": m["role"], "content_length": len(text), "preview": preview})
    return json.dumps({
        "model": entry["model"],
        "prompt_tokens": entry["prompt_tokens"],
        "completion_tokens": entry["completion_tokens"],
        "max_tokens": entry["max_tokens"],
        "temperature": entry["temperature"],
        "total_duration_sec": entry["total_duration_sec"],
        "messages": messages,
    }, indent=2)

log_text = serialize_log(log_entry)
print(log_text)

Step 3: Define the system prompt

This prompt tells the model how to reason like an inference engineer.

SYSTEM_PROMPT = """You are an LLM inference performance engineer. Analyze the request log below and explain why the call was slow. Then list concrete fixes.

Rules:
- Identify the single biggest bottleneck first.
- If prompt_tokens is high, suggest trimming system prompts or removing redundant context.
- If max_tokens is high but completion_tokens is low, suggest lowering max_tokens to reduce preallocation overhead.
- If temperature is above 0.5 for a factual task, suggest lowering it.
- If the model is large and the output is small, suggest a smaller Oxlo.ai model such as deepseek-v3.2 or qwen-3-32b.
- Keep the diagnosis under 120 words. Use bullet points.
"""

Step 4: Run the diagnostic

Now I send the serialized log to qwen-3-32b, which excels at reasoning and agent workflows.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

response = client.chat.completions.create(
    model="qwen-3-32b",
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": f"Diagnose this slow request log:\n

```json\n{log_text}\n```

"},
    ],
    temperature=0.2,
    max_tokens=500,
)

diagnosis = response.choices[0].message.content
print(diagnosis)

Step 5: Stream the diagnosis

In production, I want to see the analysis as it arrives. Oxlo.ai supports streaming, so I will switch to chunk-based output.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

stream = client.chat.completions.create(
    model="qwen-3-32b",
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": f"Diagnose this slow request log:\n

```json\n{log_text}\n```

"},
    ],
    temperature=0.2,
    max_tokens=500,
    stream=True,
)

print("Diagnosis: ", end="", flush=True)
for chunk in stream:
    content = chunk.choices[0].delta.content
    if content:
        print(content, end="", flush=True)
print()

Run it

Here is the complete script. Save it as debug_agent.py and run it.

import time
import json
from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

SYSTEM_PROMPT = """You are an LLM inference performance engineer. Analyze the request log below and explain why the call was slow. Then list concrete fixes.

Rules:
- Identify the single biggest bottleneck first.
- If prompt_tokens is high, suggest trimming system prompts or removing redundant context.
- If max_tokens is high but completion_tokens is low, suggest lowering max_tokens to reduce preallocation overhead.
- If temperature is above 0.5 for a factual task, suggest lowering it.
- If the model is large and the output is small, suggest a smaller Oxlo.ai model such as deepseek-v3.2 or qwen-3-32b.
- Keep the diagnosis under 120 words. Use bullet points.
"""

def capture_baseline():
    start = time.perf_counter()
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Explain quantum computing in exhaustive detail, covering every major algorithm and physical implementation."},
        ],
        max_tokens=4000,
        temperature=0.9,
    )
    elapsed = time.perf_counter() - start
    return {
        "model": "llama-3.3-70b",
        "request_messages": [
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Explain quantum computing in exhaustive detail, covering every major algorithm and physical implementation."},
        ],
        "max_tokens": 4000,
        "temperature": 0.9,
        "total_duration_sec": round(elapsed, 2),
        "prompt_tokens": response.usage.prompt_tokens,
        "completion_tokens": response.usage.completion_tokens,
    }

def serialize_log(entry):
    messages = []
    for m in entry["request_messages"]:
        text = m["content"]
        preview = text if len(text) < 180 else text[:180] + " ... [truncated]"
        messages.append({"role": m["role"], "content_length": len(text), "preview": preview})
    return json.dumps({
        "model": entry["model"],
        "prompt_tokens": entry["prompt_tokens"],
        "completion_tokens": entry["completion_tokens"],
        "max_tokens": entry["max_tokens"],
        "temperature": entry["temperature"],
        "total_duration_sec": entry["total_duration_sec"],
        "messages": messages,
    }, indent=2)

def diagnose(entry):
    log_text = serialize_log(entry)
    stream = client.chat.completions.create(
        model="qwen-3-32b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": f"Diagnose this slow request log:\n

```json\n{log_text}\n```

"},
        ],
        temperature=0.2,
        max_tokens=500,
        stream=True,
    )
    print("Diagnosis: ", end="", flush=True)
    for chunk in stream:
        content = chunk.choices[0].delta.content
        if content:
            print(content, end="", flush=True)
    print()

if __name__ == "__main__":
    log_entry = capture_baseline()
    print(f"Captured baseline: {log_entry['total_duration_sec']}s\n")
    diagnose(log_entry)

When I run this, the output looks something like the following.

Captured baseline: 4.82s

Diagnosis: 
- Bottleneck: max_tokens is set to 4000 but completion_tokens is only 312. The high limit forces unnecessary KV-cache allocation and extends time-to-last-token.
- Fix: Lower max_tokens to 512 for this task. On Oxlo.ai this costs the same per request regardless of token count, but latency will drop sharply.
- Fix: Reduce temperature from 0.9 to 0.3. The task is factual, so high temperature adds non-deterministic sampling overhead without benefit.
- Fix: If output length stays small, try deepseek-v3.2 or qwen-3-32b instead of llama-3.3-70b for faster turnaround.

Wrap-up

That is the whole system. A few ideas for what to build next:

  • Pipe the agent into a Slack bot so it triggers automatically when a request exceeds your SLO.
  • Add a regression test that re-runs the same prompt against deepseek-v3.2 and llama-3.3-70b to confirm the speedup before you merge the fix.

Top comments (0)