Built a proxy, so my AI coding sessions stop forgetting everything after 20 turns

If you've used Claude or GPT for a long coding session, you know the drill. Context fills up, it summarizes, and a few turns later it's suggesting a library you already rejected, or forgetting why you picked postgres over MySQL in the first place.

I built TokenMizer to fix that. It's a local proxy that sits between your app and whatever LLM you're using. Instead of dumping raw conversation history back at the model, it builds a small graph of what actually happened in the session, tasks, decisions with the reasoning behind them, files touched, errors hit.

Using it is a one line change:

from openai import OpenAI

client = OpenAI(api_key="your-key", base_url="http://localhost:8000/v1")

response = client.chat.completions.create(
    model="claude-sonnet-4-6",
    messages=[{"role": "user", "content": "Let's build an auth service"}],
    extra_body={"session_id": "my-project"},
)

When context hits around 85%, it auto-checkpoints. I ran a 40-turn session that resumed the next day in 233 tokens instead of re-explaining the whole project from scratch. Decisions carry state too, active, superseded, invalidated, archived, so "why did we switch from React to Next.js" is still answerable weeks later instead of buried in a chat log nobody's scrolling back through.

There's also a file intelligence layer. Drop in a CSV, PDF, or Excel file, and it gives the model a schema and sample instead of burning your entire token budget on raw rows, 99%+ savings on large files.

Works with Claude, GPT, Gemini, Grok, DeepSeek, and Ollama out of the box. There's a Claude Code plugin and an MCP server if you want it wired straight into your editor.

pip install "tokenmizer[anthropic,cache]"
tokenmizer serve

Repo's here: https://github.com/Shweta-Mishra-ai/tokenmizer

Still actively building this, would love to know if this is a problem you've hit too, and how you're dealing with it right now.

Comments

[Max Quimby]

The decision-provenance piece is the part I'd dig into most — most "memory" proxies just compress transcript, but tracking decisions as first-class objects with an active/superseded/invalidated lifecycle is where the real leverage is. "Why did we switch from React to Next.js" being answerable weeks later is a genuinely different capability than summarization.

The failure mode I'd watch for: stale active decisions. It's easy to record "we chose Postgres," much harder to reliably flip it to superseded when the human quietly changes their mind three turns later without announcing it. So my question — how does supersession get triggered? Is the model inferring "this contradicts an earlier decision," or is it explicit? Inferred supersession is powerful but that's also where I'd expect silent corruption to creep in, because a missed flip means the graph confidently hands back a decision that's no longer true.

The 40-turn → 233-token resume number is striking. Would love to see how it holds up on a session with a lot of back-and-forth reversals rather than clean forward progress.

[Shweta Mishra]

So it's not the model inferring anything — that's actually the part I avoided. Every new decision node runs through a deterministic topic classifier the moment it's added: keyword/bigram match against a taxonomy of ~20 tech categories (db, auth, frameworks, deployment, etc), zero LLM calls, sub-millisecond. If the new decision's topic overlaps with an existing active decision's topic, the old one flips to superseded right there. History stays in the graph, nothing gets deleted.
So "actually use MySQL instead" three turns later with zero signal words still gets caught, because it lands in the same database bucket as the earlier Postgres decision. No contradiction-spotting required.
There's also an LLM extraction pass that looks for explicit phrasing like "switching from X to Y" — but that's just there to grab a clean old→new label pair for the transition record, it's not what triggers the flip.
Honest gap, and you're right to poke at it: the taxonomy is finite. If someone pivots on something outside the ~20 known categories, it falls back to a fuzzy word-overlap check (0.6 threshold), and that's noticeably less reliable. That's where I'd actually expect a missed flip, not in the explicit vs inferred split.
On the reversal-heavy session number — fair, I haven't run that specific stress test yet. Current numbers are from cleaner forward-progress sessions. Adding a reversal-heavy case to the benchmark suite is a good idea, noted.