How I Stopped Wasting Unnecessary Tokens: Building a Local Memory Scaffold for AI Agents

Francis — Wed, 03 Jun 2026 11:13:04 +0000

Every developer experimenting with the new wave of terminal-based AI Coding Agents (like Claude Code, GitHub Copilot CLI, or Cline) eventually hits the exact same wall: The Token Tax and Memory Decay.
You fire up a new session, and the agent spend minutes mindlessly rescanning your entire repository just to understand the layout. It works great for 10 minutes. Then, as your session grows, it hits a "compact" event or overflows its context window. Suddenly, the AI forgets crucial architectural constraints, repeats a bug you just told it to fix 5 minutes ago, or loses track of what it was supposed to do next.
Feeding entire codebases into an LLM over and over isn't just expensive; it’s architecturally fragile. This is the "Lost in the Middle" problem in action.
To solve this for my own workflow, I built Memory Anchor — a lightweight, local context scaffolding tool engineered specifically to give AI agents a persistent, compact, cross-session memory.
🛑 The Core Pain Points
When managing long-lived agent sessions, we usually battle three systemic inefficiencies:
Token Bleeding (The Cold Start): AI agents re-reading massive source files or directory skeletons on every session boot just to get their bearings.
Context Drift (Memory Decay): The agent forgetting repository-specific design patterns or "lessons learned" halfway through a task, repeating the same mistakes.
State Disconnection: When you close the terminal and come back tomorrow, the agent has no idea where it left off on the project's TODO board.
⚓ The Solution: Memory Anchor
Memory Anchor operates via automated lifecycle hooks that orchestrate a tight loop between your source code, Git state, and the AI's instructions. Instead of passing raw, unorganized code dumps, it wraps your repo in a predictable Local Context Scaffold.
It establishes three lightweight metadata anchors directly within your repository:
🧠 The Ballast (ballast.md): Long-term memory. It persists repository-specific design constraints, architectural decisions, and "lessons learned" across sessions so your AI never repeats the same mistakes.
📅 The Manifest (manifest.md): A cross-session TODO/DONE board. It synchronizes the project state so agents can seamlessly resume work exactly where you left off.
🗺️ The Chart (chart.md): A compact map of your project, containing the directory skeleton and precise export signatures rather than full implementations.
🔄 How It Works (The Lifecycle)
Memory Anchor hooks into your agent's lifecycle (sessionStart and sessionEnd) to manage the state automatically:
Plaintext
[Start Session] ──> sessionStart Hook ──> Aggregates Ballast + Manifest ──> Injects Compact Payload
│
[AI Coding]
│
[End Session] <── sessionEnd Hook <── Incremental Git Diff Analysis <───────────────────┘
sessionStart: Loads the pre-compiled Chart, Ballast, and Manifest to inject a highly optimized, compact context payload. This slashes token spend on re-reading the repo and eliminates cold-start latency.
sessionEnd: Analyzes your Git diffs incrementally, prompts or records the changes into the Manifest/Ballast, and Refreshes the Chart slices so the next session starts instantly with up-to-date memory.
🛠️ Getting Started in 30 Seconds
Memory Anchor is built with Node.js and TypeScript, exposing a clean, declarative CLI.
Bash

Install the global CLI tool

npm install -g memory-anchor

Initialize the local context scaffold

anchor init
Running anchor init scaffolds the control directory (./.memoryanchor/) and sets up ./AGENTS.md which houses your AI behavior rules.
If you are using specific environments like GitHub Copilot or Claude Code, you can easily inject custom configurations:
Bash
anchor init-copilot # Points Copilot instructions to your agent workflow
anchor init-claude # Establishes automated settings hooks for Claude Code
🏗️ Technical Implementation & Challenges
Building a lightweight orchestration tool meant keeping the footprint minimal and the execution blazing fast:
Incremental Git Diff Tracking: The sessionEnd hook doesn't blindly rebuild your project map. It performs an incremental analysis on Git changes, ensuring that updating the chart.md slices happens in milliseconds.
Agent-Agnostic Scaffolding: Whether it's Copilot reading .github/copilot-instructions.md or Claude Code executing terminal hooks via standard I/O streams, Memory Anchor standardizes the structural prompt payload so any LLM can digest it.
Purely Local & Privacy-First: No external servers, no cloud databases. Your project's context, memory, and rules live exactly where they belong: inside your .git tracked repository.
🔮 Open Source & Feedback
Memory Anchor is completely open-source and freshly published. It has completely transformed how I pair-program with terminal agents, bringing my token costs down significantly while keeping the agent laser-focused.
I'd love to get your feedback:
How do you currently handle long-term memory or custom constraints with your AI tools?
What IDE or CLI Agent would you like to see a native init- command for next?
If you want to save your token budget and stop explaining the same architectural rules to your AI over and over, give it a spin!

DEV Community: Francis

How I Stopped Wasting Unnecessary Tokens: Building a Local Memory Scaffold for AI Agents

Install the global CLI tool

Initialize the local context scaffold