Stop Guessing Where Your LLM Prompt Tokens Go: prompt-flamegraph

Julien — Wed, 09 Sep 2026 14:27:51 +0000

Stop Guessing Where Your LLM Prompt Tokens Go: prompt-flamegraph

LLM APIs are cheap until they are not.

If you build anything serious with tools, RAG, or multi-turn chat, your prompts quickly become huge black boxes. You know you are paying per token, but you do not know which part of the context is eating the budget.

Is it the 10 retrieved documents? The 50-message chat history? The 20 tools you declared just in case?

I built prompt-flamegraph to make this visible.

What it does

prompt-flamegraph takes a structured prompt — system prompt, tools, RAG context, chat history — and renders it as an interactive HTML flamegraph.

You get:

A visual breakdown of token usage by category.
Cost estimation if you provide the price per token.
Waste detection: duplicate text, oversized RAG, long history, too many tools.
Diff view between two prompt versions.
Terminal, SVG, and Markdown exports.
Zero required dependencies, no server, no telemetry.

Install

pip install prompt-flamegraph

Optional extras:

pip install prompt-flamegraph[tiktoken]   # accurate OpenAI-style counts
pip install prompt-flamegraph[rich]       # prettier terminal output

Quick start

from prompt_flamegraph import profile_prompt

prompt = {
    "system_prompt": "You are a helpful coding assistant.",
    "tools": ["..."],
    "rag_context": {"doc_1": "..."},
    "chat_history": ["..."],
}

profile_prompt(prompt, output="context.html")

Open context.html in your browser.

CLI

# HTML flamegraph with cost estimation
prompt-flamegraph prompt.json -o context.html --cost 1.5e-6

# Terminal bar chart
prompt-flamegraph prompt.json --terminal

# Diff two prompt versions
prompt-flamegraph v1.json --diff v2.json -o diff.html

# SVG or Markdown
prompt-flamegraph prompt.json --format svg -o context.svg

Detect waste before paying

from prompt_flamegraph import build_tree, detect_waste

prompt = {
    "system_prompt": "You are a helpful coding assistant.",
    "tools": ["read_file", "write_file", "run_command", "search_web", "send_email", "create_ticket"],
    "rag_context": {
        "doc_1.py": "def helper():\n    return 'value'\n",
        "doc_2.py": "def helper():\n    return 'value'\n",
        "doc_3.py": "def helper():\n    return 'value'\n",
    },
    "chat_history": ["Hi!"] * 10,
}

tree = build_tree(prompt, name="prompt")
report = detect_waste(tree)

print(f"Wasted: {report.wasted_tokens} / {report.total_tokens} tokens ({report.waste_ratio:.1%})")
for finding in report.findings:
    print(f"- {finding.kind}: {finding.message}")

Output:

Wasted: 26 / 88 tokens (29.5%)
- duplicate: 3× duplicate text ('def helper():     return 'value' ') — keep only one
- duplicate: 5× duplicate text ('Hi!') — keep only one
- too_many_tools: 6 tools defined — only declare the ones the model actually calls

What the report looks like

The report shows:

total tokens,
estimated cost,
waste findings,
an interactive flamegraph you can explore.

Going further: prompt-optimizer

If you want an automated cleanup step, I also built prompt-optimizer on top of prompt-flamegraph.

It reads a prompt, prints a waste report, and writes an optimized version.

pip install -e .
prompt-optimizer optimize examples/sample_prompt.json -o prompt.optimized.json

It removes duplicate RAG chunks, trims long history, and suggests fewer tools.

Feedback welcome

If you try it, open an issue or a PR. I built this to scratch my own itch and I would love to see it useful for others.

DEV Community: Julien

Stop Guessing Where Your LLM Prompt Tokens Go: prompt-flamegraph

Stop Guessing Where Your LLM Prompt Tokens Go: prompt-flamegraph

What it does

Install

Quick start

CLI

Detect waste before paying

What the report looks like

Going further: prompt-optimizer

Links

Feedback welcome