So I build this tool called Roam. It saves my coding agent sessions (OpenCode, Claude Code, Codex, whatever I'm using that day) into a private git repo, so when I switch from my PC to my laptop I can just pick up where I left off. That's the whole idea.
Two weeks ago my sync just stopped. I went and checked the repo behind it. 22 GB. GitHub was timing out on every single push.
And see, I don't have 22 GB of conversations. Nobody does. So where did it come from? That's what this post is about, and how I fixed it.
How I designed it (and why it looked fine)
Every project had one diary file, LOG.md. Each agent session was a section inside it, and every message in the session had a heading like this:
### [3/57] user
Meaning message 3 of 57. Every time Roam captured the session again (when the agent goes idle, or every 10 minutes), it rewrote that whole section with the latest transcript.
Now, git doesn't store "the lines that changed". It stores the whole file version every time. Usually that's fine, files are small, git compresses, nobody cares.
Where it went wrong
Two things stacked on top of each other.
First, the file was big. One of my long running projects had a diary of around 9 MB.
Second, every capture changed every heading. When message 58 comes in, [3/57] becomes [3/58]. And [4/57] becomes [4/58]. Every heading in the session changes, every single time. So even git's compression can't save you, because the whole 9 MB file is "different" on every save.
This is what I actually measured on my relay:
| Size of one project's diary | ~9 MB |
| Full copies of it stored in git | 1,425 |
| Copies stored in one single day | 296 |
| Total repo size | 22 GB |
Then GitHub started rejecting pushes with timeouts (HTTP 408). And the worst part? My tool kept saying "saved", because the local commit worked. The push didn't. So sync was silently dead for two days before I even noticed. That one's on me.
Fix 1: stop renumbering
That "of 57" part was honestly just decoration. Nobody needed it. Remove it and a message heading never changes once it's written:
### [3] user
Tiny change. But now old lines are byte for byte the same between captures, which is exactly what git wants.
Fix 2: append only
Instead of rewriting one big file, every session gets its own folder now:
sessions/<session-id>/
head.md # title, keywords, summary (small, can change)
state.json # what has been captured so far
c-000001.md # chunk 1: messages 1-12
c-000002.md # chunk 2: messages 13-20
...
A capture only writes a new chunk with the messages that are actually new. Old chunks are never touched again, so git stores each message once. That's it. When a session hits 64 chunks they get merged into one so the folder doesn't explode. And the readers rebuild the same diary as before, so search and resume didn't need to change at all.
I wrote a test for this: capture one growing session 15 times and measure what git actually stores. Before the fix it was more than 10x the size of the session. After, under 2x, and most of that is just git's own bookkeeping.
Fix 3: if sync fails, say so
The whole 22 GB thing went unnoticed because a failed push looked exactly like a successful one. So now:
- a stale git lock (left behind by a killed process) gets detected and cleared once it's clearly abandoned
- a failed sync gets recorded and reported, and
roam captureexits with an error - big backlogs get pushed in batches, so one giant push can't keep timing out forever
Then I made the same mistake again, in memory
Funny part. Later I found basically the same bug in search. To show results, my search index kept small strings like section titles and keywords, cut out of each diary file. And in JavaScript, a small string cut out of a big one can keep the whole big string alive in memory. So those tiny titles were quietly holding every diary file in RAM. Building the index for my 4.8 GB history went past 4 GB of memory.
Copying those strings properly, and reading old diaries one file at a time instead of as one giant string, brought it down to 1.2 GB in 78 seconds. Search now takes 0.6 s for a rare word and around 5 s for a really common one. It used to take 47 s.
What I'd tell myself two weeks ago
- Git is a great database for stuff you only append, and a terrible one for files you keep rewriting. If your tool rewrites a growing file every save, check what git actually stores, not how big the file looks.
- Decoration that changes is expensive. A running total in every heading cost me 20+ GB. Like, actually.
- "Saved" has to mean pushed. If your sync tool says success on a local commit, it will fail silently one day and you'll find out from a 22 GB repo.
- Measure before you claim. My first landing page said search takes milliseconds. It took 47 seconds. So now every number I put out comes from a script anyone can run.
Roam is free to try on Windows: https://roam-rho-gold.vercel.app (docs and releases are here: https://github.com/DeepxD-code/roam-releases). If you've used git as a database and run into the same thing, tell me how you dealt with it, I'd really like to know.
I wrote this with help from an AI assistant. Every number is from my own measurements.
Top comments (0)