DEV Community

Cover image for Stop Guessing Where Your LLM Prompt Tokens Go: prompt-flamegraph
Julien
Julien

Posted on

Stop Guessing Where Your LLM Prompt Tokens Go: prompt-flamegraph

#python #llm #promptengineering #opensource

Stop Guessing Where Your LLM Prompt Tokens Go: prompt-flamegraph

LLM APIs are cheap until they are not.

If you build anything serious with tools, RAG, or multi-turn chat, your prompts quickly become huge black boxes. You know you are paying per token, but you do not know which part of the context is eating the budget.

Is it the 10 retrieved documents? The 50-message chat history? The 20 tools you declared just in case?

I built prompt-flamegraph to make this visible.

What it does

prompt-flamegraph takes a structured prompt — system prompt, tools, RAG context, chat history — and renders it as an interactive HTML flamegraph.

You get:

  • A visual breakdown of token usage by category.
  • Cost estimation if you provide the price per token.
  • Waste detection: duplicate text, oversized RAG, long history, too many tools.
  • Diff view between two prompt versions.
  • Terminal, SVG, and Markdown exports.
  • Zero required dependencies, no server, no telemetry.

Install 

pip install prompt-flamegraph
Enter fullscreen mode Exit fullscreen mode

Optional extras:

pip install prompt-flamegraph[tiktoken]   # accurate OpenAI-style counts
pip install prompt-flamegraph[rich]       # prettier terminal output
Enter fullscreen mode Exit fullscreen mode

Quick start 

from prompt_flamegraph import profile_prompt

prompt = {
    "system_prompt": "You are a helpful coding assistant.",
    "tools": ["..."],
    "rag_context": {"doc_1": "..."},
    "chat_history": ["..."],
}

profile_prompt(prompt, output="context.html")
Enter fullscreen mode Exit fullscreen mode

Open context.html in your browser.

CLI 

# HTML flamegraph with cost estimation
prompt-flamegraph prompt.json -o context.html --cost 1.5e-6

# Terminal bar chart
prompt-flamegraph prompt.json --terminal

# Diff two prompt versions
prompt-flamegraph v1.json --diff v2.json -o diff.html

# SVG or Markdown
prompt-flamegraph prompt.json --format svg -o context.svg
Enter fullscreen mode Exit fullscreen mode

Detect waste before paying 

from prompt_flamegraph import build_tree, detect_waste

prompt = {
    "system_prompt": "You are a helpful coding assistant.",
    "tools": ["read_file", "write_file", "run_command", "search_web", "send_email", "create_ticket"],
    "rag_context": {
        "doc_1.py": "def helper():\n    return 'value'\n",
        "doc_2.py": "def helper():\n    return 'value'\n",
        "doc_3.py": "def helper():\n    return 'value'\n",
    },
    "chat_history": ["Hi!"] * 10,
}

tree = build_tree(prompt, name="prompt")
report = detect_waste(tree)

print(f"Wasted: {report.wasted_tokens} / {report.total_tokens} tokens ({report.waste_ratio:.1%})")
for finding in report.findings:
    print(f"- {finding.kind}: {finding.message}")
Enter fullscreen mode Exit fullscreen mode

Output:

Wasted: 26 / 88 tokens (29.5%)
- duplicate: 3× duplicate text ('def helper():     return 'value' ') — keep only one
- duplicate: 5× duplicate text ('Hi!') — keep only one
- too_many_tools: 6 tools defined — only declare the ones the model actually calls
Enter fullscreen mode Exit fullscreen mode

What the report looks like

prompt-flamegraph report

The report shows:

  • total tokens,
  • estimated cost,
  • waste findings,
  • an interactive flamegraph you can explore.

Going further: prompt-optimizer

If you want an automated cleanup step, I also built prompt-optimizer on top of prompt-flamegraph.

It reads a prompt, prints a waste report, and writes an optimized version.

pip install -e .
prompt-optimizer optimize examples/sample_prompt.json -o prompt.optimized.json
Enter fullscreen mode Exit fullscreen mode

It removes duplicate RAG chunks, trims long history, and suggests fewer tools.

Links

Feedback welcome

If you try it, open an issue or a PR. I built this to scratch my own itch and I would love to see it useful for others.

Top comments (0)