DEV Community

Chris Yao
Chris Yao

Posted on

Your AI Conversations Are Getting Long — I Built a Tool to Distill, Compress, and Score Every Prompt (Open Source)

#python #ai #productivity #opensource

Every developer using AI coding tools has sessions that spiral. What starts as "add a login page" becomes a 100-turn conversation where 80% of the turns are noise — follow-up corrections, restated requirements, tangential debugging. The 20 turns that actually drove the implementation are buried.

I built reprompt to fix this. It started as a prompt scorer, evolved into a compression engine, and is now a conversation distillation tool. Everything runs locally, no LLM needed.

Conversation Distillation: 6 Signals, No LLM

reprompt distill --last extracts the important turns from your most recent AI session using 6 rule-based signals:

  1. Position — first and last turns carry more weight (they frame the task and conclude it)
  2. Length — substantive turns are longer than "yes" and "try again"
  3. Tool trigger — turns that invoke tool calls (file edits, test runs) drive actual work
  4. Error recovery — turns after failures carry debugging context
  5. Semantic shift — topic changes indicate new decisions or pivots
  6. Uniqueness — non-repetitive turns carry more information

Each turn gets a 0-1 importance score. Default threshold is 0.3 — tune it up for tighter distillation.


reprompt distill extracting 12 important turns from a 47-turn session

reprompt distill --last
# Shows the ~20% of turns that mattered

reprompt distill --last 5 --summary
# Compressed summaries of your last 5 sessions
Enter fullscreen mode Exit fullscreen mode

This is useful for:

  • Handoffs: "What happened in this session?" in 30 seconds instead of scrolling 100 turns
  • Context management: Feed the distilled turns into a new session instead of the full history
  • Self-review: See which of your turns actually drove progress

Prompt Compression: 4 Layers, 15-30% Token Savings

reprompt compress removes filler from your prompts before you send them:

$ reprompt compress "I would like you to please help me refactor the authentication
middleware so that it properly handles expired tokens. The current implementation
doesn't seem to work correctly when a session times out."

Compressed (saved 34%):
"Refactor the authentication middleware to handle expired tokens.
Current implementation fails on session timeout."
Enter fullscreen mode Exit fullscreen mode

Four layers:

  1. Character normalization — curly quotes, zero-width chars, NFKC
  2. Phrase simplification — "in order to" → "to", "make sure that" → "ensure" (43 Chinese + 51 English rules)
  3. Filler deletion — "I would like you to please help me" → "" (jieba-aware for Chinese)
  4. Structure cleanup — markdown artifacts, emoji noise, LLM output formatting

Based on research from LLMLingua (Microsoft), CompactPrompt, and stopwords-iso.

Prompt Scoring: 4 NLP Papers, 0-100

The original feature. reprompt score rates prompts against research:

$ reprompt score "fix the bug"
# 18/100 — no file, no line, no error message, no expected behavior

$ reprompt score "fix the NPE in auth.service.ts:47 — token is null
when session expires, should throw AuthException not return 200"
# 87/100 — file path, line number, error type, expected behavior
Enter fullscreen mode Exit fullscreen mode

Based on Google 2512.14982 (repetition), Stanford 2307.03172 (position bias), SPELL EMNLP 2023 (perplexity), and the Prompt Report 2406.06608 (taxonomy). Deterministic NLP — same input, same output.

After six weeks of tracking with reprompt digest, my debug prompts went from averaging 31/100 to 48. Not from reading advice — from seeing a number after each session.

Privacy Exposure

reprompt privacy maps where your prompts actually went:

Privacy Exposure
────────────────────────────────────
  Sent to cloud:       892 (85%)
  Local only:          155 (15%)
  Training risk:       412 prompts (opt-out/unknown policy)
Enter fullscreen mode Exit fullscreen mode

85% of my prompts hit cloud services. 412 went to tools with opt-out training policies. Those prompts contain file paths, function names, production error messages — a map of the codebase.

What I Actually Use Daily

  1. reprompt scan discovers session files from all my tools (zero config)
  2. reprompt distill --last after long sessions — what actually happened?
  3. reprompt digest --quiet as a session-end hook — one-line trend
  4. reprompt insights --source claude-code — per-tool pattern analysis
  5. reprompt privacy monthly — where did my prompts go?

Each command now suggests the next one to try. New users get guided through the analysis workflow.

Technical Details

  • Python 3.10+, scikit-learn (TF-IDF + K-means), SQLite for storage
  • 1237 tests, strict mypy, ruff for lint/format
  • 8 adapters: Claude Code, Cursor, Aider, Gemini CLI, Cline, OpenClaw (auto-scan) + ChatGPT, Claude.ai (manual import)
  • Pluggable embedding backends: TF-IDF (default), Ollama, sentence-transformers, OpenAI
  • Zero config defaults, optional TOML customization
  • Everything runs locally — no data leaves your machine
  • MCP server for Claude Code / Continue.dev / Zed integration
  • --source filter on all commands for multi-tool users

Try It 

pipx install reprompt-cli
reprompt demo              # built-in sample data
reprompt scan              # scan your actual sessions
reprompt distill --last    # what mattered in your last session?
reprompt compress "your verbose prompt here"
reprompt score "your prompt"
Enter fullscreen mode Exit fullscreen mode

MIT licensed. v1.3.1 Production/Stable.

GitHub: https://github.com/reprompt-dev/reprompt
PyPI: https://pypi.org/project/reprompt-cli/

What's the longest AI conversation you've had? And how many of those turns actually mattered? I'd bet it's fewer than you think.

Top comments (0)