Svetlana Perekrestova

Posted on Jun 13 • Originally published at sperva.hashnode.dev

I Built a Local Control Plane for My Coding Agents

#ai #opensource #tutorial #productivity

I have been playing with my local AI setup for a while, and for most of that time it was hard to explain without opening five terminals and pointing at folders.

The tools were fine. That was not the issue.

I used Copilot. I used Claude Code. I used Codex. I tried OpenCode. Sometimes I wanted Claude because I already had the subscription. Sometimes I wanted Codex because I liked the workflow. Sometimes I wanted OpenCode because switching models through OpenRouter was easier there.

The annoying part was everything around the agents.

Every tool had its own home directory, its own settings format, its own MCP config, its own skills folder, its own memory story. So whenever I added a useful MCP or wrote a good rule, I had to ask the stupid question again: where does this tool expect it?

After enough of that, the setup stopped feeling like "a powerful local AI stack" and started feeling like duplicated dotfiles with chat windows attached.

The original mess

This was the rough shape before I cleaned it up:

My first workaround was Markdown.

I would ask one agent to plan a feature, save the plan into a .md file, and then hand that file to another agent to implement. It was better than relying on chat history. At least the plan existed outside one session.

But it did not really solve the problem.

The second agent would read the plan, miss some assumption, make a different tradeoff, and produce something slightly different. Then I would go back to the first agent and ask for review, and it would be confused because the implementation was not the thing it had planned.

So yes, files helped. They were not shared context.

What I wanted was much less dramatic:

write high-level rules once
keep permission rules in one place
stop copying skills by hand
stop re-declaring the same MCPs
have memory survive across projects and sessions
keep my home directory readable

That last one sounds cosmetic until you live with the opposite. A home directory full of half-overlapping AI folders is not a setup. It is a junk drawer.

The memory rabbit hole

I started with memory because that was the pain I could name.

Claude-specific memory was useful, but it was still Claude-specific. AgentMemory through MCP worked, but it felt heavier than I needed. Too many moving parts, too many features I was probably never going to touch.

Headroom was the first thing that made the shape click for me.

The useful bit was not only "memory". It was the split between two jobs I had been mentally mixing together:

compaction and routing: the model request goes through a local proxy before it reaches the provider
memory: long-lived facts and decisions live in a local store and are accessed through MCP

Those should not be the same thing.

In my setup, Headroom's proxy listens on 127.0.0.1:8787. Model traffic can go through that. The memory MCP talks to a shared SQLite database. Same project, related pieces, but different responsibilities.

Once I separated those two ideas, the rest of the architecture became easier to reason about.

The current folder layout

The center is now ~/.ai.

Not because ~/.ai is magic. I just needed one place that I could point tools at.

Most of the old-looking dotfolders are now symlinks. The tools can still find what they expect, but the real structure is under ~/.ai.

My shell makes that explicit:

export OPENCODE_CONFIG_DIR="$HOME/.ai/opencode"
export CLAUDE_CONFIG_DIR="$HOME/.ai/claude"
export CODEX_HOME="$HOME/.ai/codex"
export ANTHROPIC_BASE_URL="http://127.0.0.1:8787"
export OPENAI_BASE_URL="http://127.0.0.1:8787/v1"

Small Codex footnote: I have both ~/.ai/codex and ~/.ai/codex-desktop. That is intentional. The desktop app and the CLI/runtime state are not exactly the same thing. I still want both under the same roof.

This is not a perfect abstraction. It is just a directory structure I can remember.

OpenCode became my main entrance

OpenCode is now the place I usually start from when I want a normal CLI coding session.

The reason is provider switching. With OpenRouter behind it, I can move between models without rebuilding the rest of the setup. Some tasks need a stronger model. Some tasks are fine on something cheaper. Some days a provider is just slow.

The exact model in the config is not the point. The shape is:

{
  "instructions": [
    "~/.ai/instructions.md",
    "~/.ai/rules.md",
    "~/.ai/security.md"
  ],
  "provider": {
    "openrouter": {
      "options": {
        "baseURL": "http://127.0.0.1:8787/v1"
      }
    }
  }
}

OpenCode reads the shared instruction files from ~/.ai. Its OpenRouter traffic goes through Headroom.

The MCP list lives there too. Mine currently includes Headroom, Headroom memory, Firecrawl, Tavily, Deepgram docs, Sentrux, and Context7. That list will change. The important part is that I now treat MCPs as part of my local agent environment, not as random plugins installed for one tool and forgotten.

Runtime flow

This is the part I wish I had drawn earlier:

There are two paths.

Model calls go through the proxy. Memory goes through the memory MCP.

I keep repeating this because it prevents a very specific kind of debugging pain. If a model call behaves strangely, I look at the proxy path. If an agent forgot something, I look at the memory path. Different questions, different places to inspect.

Memory rules that are boring on purpose

The shared memory is a local SQLite database exposed through headroom-memory.

I do not want it to save everything. That would just turn memory into another log pile. I want it to remember things that are expensive to rediscover:

project decisions
architecture notes
workflow preferences
recurring bugs
lessons from previous sessions

For my own workflow I use two commands:

/remember when something should survive future sessions
/recall when an agent needs context from earlier work

Examples of memories that are useful:

"This project uses build scripts instead of editing Info.plist directly."
"Do not trust SourceKit diagnostics until the actual iOS build runs."
"The VPN project stores memory under the shared Headroom DB."
"This repo prefers feature branches and conventional commits."

I also have a hard rule against saving secrets, credentials, raw logs, or private personal data. Memory should make future work easier. It should not become a liability.

Permissions are not decoration

I used to think of permissions as a safety checkbox. Now I think of them as part of the agent environment.

If Claude Code, Codex, and OpenCode disagree about what they can read or run, then switching agents changes the behavior of the task. That is exactly the kind of invisible difference that causes trouble later.

So I moved the common policy into shared files:

~/.ai/rules.md for coding and workflow standards
~/.ai/security.md for denied commands and paths
~/.ai/instructions.md for memory behavior and cross-agent notes

The security file is plain on purpose:

## denied paths
- ~/.ssh/**
- ~/.aws/**
- ~/.kube/**

## denied commands
- rm -rf /
- rm -rf ~
- git push --force *
- git reset --hard *

Each tool still has its own config format. I am not trying to erase that. I just want the policies to rhyme closely enough that I do not have to remember which agent is more dangerous today.

What is still rough

Headroom memory needs a UI.

The CLI is fine for listing memories, and I like CLIs more than most people probably should. Still, I want a small browser view where I can inspect memories, edit wording, merge duplicates, and delete things that should not be there.

Some configuration also refuses to become shared, and that is fine. Claude Code has Claude-specific plugin settings. Codex has desktop runtime state. OpenCode has provider and TUI settings.

I stopped trying to force those into one shape.

The useful boundary is simple: shared things go into the local AI home, tool-specific things stay with the tool.

Local models, for now

I also tried to make local coding models part of this story.

My MacBook is an older Apple Silicon machine with 16 GB of memory. It can run small coding models, around the 5B range, but I do not get enough value from them yet.

For the tasks I actually care about, I still need stronger reasoning, better codebase navigation, and more reliable instruction following than this hardware gives me comfortably.

So my setup is not "offline AI". It is local orchestration with remote model execution.

That is less romantic, but more accurate.

Where I landed

Right now the setup gives me the thing I was missing:

one local AI directory
shared rules
shared security policy
deduplicated skills
MCPs treated as local infrastructure
model traffic routed through a local proxy
memory shared across agents, projects, and sessions
fewer random dotfolders in $HOME

The part I care about most is continuity.

I can plan in one tool and implement in another without pretending the chat transcript is enough. I can use OpenCode when I want model flexibility. I can still use Claude Code when the subscription workflow is more convenient. I can use Codex without treating it as a separate universe.

It is not one agent to replace the others.

It is a local setup where the agents share enough context that switching between them does not feel like starting over.