I Built a Memory for My Friend

I was excited for the HacktoberFest Weekend Challenge, but there was one problem: I didn't have a friend to build for.

So I messaged my GPT for some good projects. I picked one, developed the idea with GPT, and then passed it over to Claude to actually build. A few prompts later, broke me hit the free-tier rate limit.

And suddenly, my "friend" couldn't continue. If I switched to another AI, I had to explain everything again: what we were building, why we made certain decisions, what we'd already tried, what had failed, and what I had explicitly asked it not to change. And when Claude became available again, I'd have to bring all of that context back.

I didn't need another AI.
I needed my AI friend to remember.

So I built Baton.

What I lost when the chat ended

I wasn't really losing a conversation. I was losing the project's state:

• Decisions, and why — why we chose one approach over another
• Dead ends — approaches we'd already tried and abandoned
• Rules — constraints and conventions that shouldn't be changed
• The half-finished task — and the bug I'd deliberately postponed

A new assistant doesn't know any of that.Pasting the entire transcript is slow and noisy. The new AI can cheerfully retry the failed idea, or "improve" the one thing I explicitly said to leave alone. Existing tools mostly move the transcript between assistants.

I wanted to move the state.

The idea

conversation
      ↓
local open-weight model
      ↓
structured project state
      ↓
handoff for any AI

Baton reads a conversation and fills a fixed schema instead of producing another generic summary:

{
  "goal": "...",
  "architecture": [],
  "decisions": [{"what": "", "why": ""}],
  "constraints": [],
  "user_preferences": [],
  "files": [{"path": "", "note": ""}],
  "recent_changes": [],
  "failed_attempts": [{"what": "", "why_failed": ""}],
  "known_bugs": [],
  "open_tasks": [],
  "do_not_change": [],
  "current_state": "...",
  "next_action": "..."
}

The split between decisions, failed_attempts and do_not_change is the heart of it.

Those are exactly the things a new AI can get wrong if it only sees a transcript. If a later message reverses an earlier decision, only the latest truth is kept. Baton then renders a handoff tuned for the target: XML sections for Claude, Markdown for Gemini and GPT, a compact format for small local models, or an AGENTS.md.

Every handoff ends with rules for the receiving AI:

• Treat do_not_change as binding.
• Don't repeat failed attempts.
• Ask instead of guessing.

The goal isn't to make another summary. It's to make the next AI feel like it was there.

Why open-weight and local is the point

Your project history contains your architecture, your mistakes, and sometimes your secrets. It seems wrong to send that to yet another service just to keep your own context.

So Baton is designed around local extraction using an open-weight model:

• Default: Gemma 4 E4B (Q4_K_M) through llama.cpp, downloaded from Hugging Face on first start.
• Ollama, LM Studio, llamafile or any OpenAI-compatible server can also be used.
• The model is instructed to produce the JSON schema, and the output is repaired and validated before reaching the UI.
• Long chats are processed in chunks, with the running state fed back into subsequent extraction.

A hosted open-weight API is supported too, but the trade-off is simple: it stops being local-first.

Multi-agent mode for small models

A 4B model asked to fill 13 fields at once can miss things. So Baton has an optional multi-agent mode: four specialists each own a subset of the fields:

• Archivist
• Rules Keeper
• Decision Analyst
• Work Tracker

A Reconciler then fixes misfiled and duplicate items. There are also two safeguards:

• If the Reconciler drops more than 30% of items, its output is discarded.
• A deterministic check removes any filename that never appears in the conversation or git history.

You can also assign larger models to individual jobs using BATON_AGENT_MODELS.

[ADD YOUR ACTUAL EVALUATION RESULT HERE.]

Using it

1. Run bash scripts/run.sh to start llama.cpp, wait for the model, and open Baton. Docker is also supported with docker compose up -d --build.
2. Paste a conversation, or capture it with the browser extension.
3. Inspect the memory panel: decisions, constraints, failed attempts, do-not-change rules and open tasks.
4. Pick a target and generate the handoff.
5. Paste it into a fresh AI conversation and ask what the next task is and what must not be changed.

There's also Git awareness: point Baton at a local repository and recent commits and uncommitted changes can be included, so "recent changes" can come from repository facts rather than memory.

What's inside

Baton is intentionally small:

• Plain Python with no pip dependencies
• One static HTML page
• A Manifest V3 browser extension
• Support for Claude, ChatGPT and Gemini
• 24 tests against mock model servers
• MIT licensed

Originality and credit

The space already has good tools.

Context-Bridge, for example, compresses transcripts into structured blocks through cloud providers. Other tools move raw sessions between coding agents or summarize conversations using your own API key.

Baton shares the broad problem, but not their code.

It's an independent implementation, and NOTICE.md lists related projects.

What I find most interesting about Baton's approach is the combination of:

• the decision / failed-attempt / do-not-change split
• local-first extraction
• grounding checks against the conversation and git history

Honest limitations

I built Baton because I needed it, but that doesn't mean every part of it is solved.

• I haven't run a rigorous benchmark across models; scripts/eval_models.py provides a rough keyword-overlap score rather than a full extraction-quality benchmark.
• The browser extension's page selectors are fragile and will need maintenance when the target sites change.
• The hosted demo is static and loads a reference state. Live analysis requires a local model.
• Small models still miss items. The memory panel should be checked before handing the state to another AI.

What's next

This is only the beginning.

I'd like to add:

• Importers for Claude Code and Codex session files
• In-browser inference with WebGPU, so there is no local server at all
• A verification step where the receiving AI restates the recovered state for the user to confirm
• Better evaluation of extraction quality across different model sizes

And maybe, by the end of this challenge, my AI friends won't need me to introduce them to the project every time we switch between them. For now, Baton is my attempt to make project context portable.

Try it

• Repo: : https://github.com/SachchitNesarikar/hacktober-26