DEV Community: Gil Neto

The index was lying and the eval knew

Gil Neto — Wed, 02 Sep 2026 07:10:05 +0000

Six weeks after I fixed capture with hooks, my second brain was still confidently wrong. The nightly test had been saying so for a month. Nobody read the number.

Six weeks ago I wrote that the way to stop an LLM agent from losing your context is hooks, structure and economics, not prompts. Capture cheaply, route once a night, enforce with scripts, cap what the agent reads. I still think that's right. This is the post about what it didn't cover.

The problem: capture never lost a fact again, and the agent was still wrong about a lot of things. It answered "what's going on with X" from an index that hadn't matched the underlying notes for weeks, and it did that with total confidence, because the rule said "read only the indexes" and it followed the rule. A nightly test I'd built in July had been reporting the decay the whole time. I had stopped reading it.

The TL;DR: the cap without a fallback produced amnesia. Three second-order failures did the damage - one fact written into six files, no way to read below the index, and diary files living in the config folder. And the test that measured all of this was itself rotting, which is a separate lesson I'd rather have learned some other way.

What held

Credit where it's due, the previous post promised things and some of them delivered.

The inbox held. Every fact I stated in any session, in any repo, got its one timestamped line, and the nightly batch routed it. In six weeks I couldn't find a single "I told you this" that had actually evaporated. The stop hook committed every turn. The batch ran every night, on a laptop that woke itself around 2am to do it. The morning report was there when I opened the lid.

So the mechanism from post #2 works. The part I'd underwritten is what happens to a fact after it's captured - and it turns out a fact that is filed but wrong is worse than one that is missing, because the missing one at least makes the agent say "I don't know".

What the eval saw before I did

In July I set up a golden set: thirty questions about my own vault with the answers I'd accept, graded nightly by a model that reads only the index files, exactly the way an interactive session would. First run, 19th of July: 17 of 30 passed, and the index-only tier missed 1 question out of 18.

Six weeks later, 28th of August: 9 of 30. Index-tier misses had gone from 1 to 6 out of 18.

That's a straight line down, and it was printed at the bottom of a report I read most mornings. I read the routed lines, I read the flags, and I skimmed past the eval because it was a number that never asked me for anything. A test nobody reads is a test that doesn't exist, and I'd built the second kind while congratulating myself on building the first.

When I finally sat down with it, about half of the decay was the vault and about half was the test. Some golden answers were July's truth - a project that was "next" in July was parked by August, and the grader was marking the correct new answer as a miss. That's its own failure: I'd written questions with a shelf life and no expiry date on them. The other half was real, and the real half is what the rest of this post is about.

Autopsy 1 - copies

Here is the path one fact took through the vault. I say something in a session. It gets an inbox line. The batch writes it into a session file, a change-log entry, a dated status block at the top of the project note, a row in that note's "recent sessions" block, a row in the projects index and a row in the master index. Six places. Each one hand-written by a model, each one paraphrased slightly differently, each one drifting on its own clock as later sessions updated some of them and not others.

I had caps. Ten recent sessions, sixty words per project row. The caps bounded the size of each index. They did nothing about the multiplicity of the fact. Bounded size, unbounded copies. The change log alone reached 214 KB, one entry per session, every entry a copy of a session file that already existed.

The tell, in hindsight, was the status blocks. Project notes had grown into stacks of "Status - 21 Aug", "Status - 17 Aug", "Status - 13 Aug", newest on top, each one a paragraph. The current state of the project was whichever block a reader happened to reach first. My largest project note was 59 KB; the largest note in the vault, a dossier, was 98 KB. Nobody, human or model, reads 98 KB to answer "where does this stand".

Autopsy 2 - no rung under the index

"For broad questions, read only the index files, no fan-out." I defended that rule in the last post and I'd defend it again. It makes the read cost of a session predictable. What it also does, if you stop there, is make everything below the cap invisible.

A fact that fell off an index - because a row got rewritten, because a cap bit, because the batch judged it less important than the newest thing - was gone as far as the agent was concerned. It wasn't gone from the vault. It was sitting in the owner note, correct, and the agent had no permitted way to reach it. So it did what a model does with a question it can't answer from what it's allowed to read: it answered anyway, from the nearest thing, in a confident voice.

The design had named this gap on day one. The roadmap literally said "search rung under the indexes - deferred until baseline". Deferred until baseline is where good ideas go to be forgotten. It stayed deferred for five weeks while the eval reported the consequences of not having it.

Autopsy 3 - config in the data's bed

This one is embarrassing in a different way, because it's a category error rather than a trade-off.

Two files that change every time I talk - a corpus of my own writing that a style skill learns from, and the thread index a reflective skill maintains - lived inside the agent's config tree, next to the skills that read them. Reasonable place for them, if you squint. But I also had a hook whose job was "you changed your instructions, run the sanity check before continuing", and it decided what counted as an instruction by looking at where the file lived.

So every time the diary index was updated, the agent's config was "dirty". The hook fired at the end of the turn, blocked it and demanded a config review. Over four days in mid-August it blocked thirty-plus turns. Worse, it fired inside the headless nightly runs, which don't have a human to say "no, skip it". Five cron runs each spent about sixty turns running a config sanity check on itself, unsupervised, one of which edited my instructions file. Roughly three hundred night turns in a month, spent by a hook that was working exactly as written.

The fix was one move: data out of the config tree. The hook stopped firing because nothing it watched changed anymore. Nothing about the hook's logic was wrong. Its scope was wrong, and scope isn't a thing a prompt or a hook can see.

The fix

None of the three failures needed a new idea. They needed the old idea applied one level further down.

One home per fact; links everywhere else. Every project note now opens with a ## Now block - at most fifteen lines, rewritten never appended, every volatile line carrying an "as of" date. Under it, ## Never drift: the three to six invariants the agent kept getting wrong about that project, so the correction lives next to the thing it corrects instead of in a memory file in another tree. Then ## Decisions, dated, append-only. The stacked status blocks moved, verbatim, into a history file beside the note. 59 KB became 41 KB; the 98 KB dossier became 8 KB with a diary file next to it. Nothing deleted, everything findable, one place that is current.

Indexes are generated. The task index, the project registry, the personal-sphere index and the child lists in every hub note are now produced by a script from frontmatter plus the first line of each note's ## Now block. The model can't drift a row, because it can't write a row. It edits the owner note. The only prose left in an index is a banner, and the banner has a word cap the linter checks. The change log is frozen; git and the session files were already the log.

A rung under the index. A keyword search over every note - SQLite full-text search, no embeddings, a hundred and fifty lines of Python, because at six hundred notes keyword recall is the documented sweet spot and I didn't want a vector database in the loop. The router rule got one clause longer: if the indexes don't hold the answer, run one search, read the one owner note it points at, answer with file and line. Still nothing, say so. Cost: one script call and one file read. The "I don't know" is now a real outcome instead of the thing that got papered over.

A linter for everything checkable. Frontmatter enums, required fields, links that resolve, caps, language-by-path, whether a generated index is stale, whether a task file has grown into a dossier, whether a ## Now block has a first line short enough to be an index row. It runs before and after the night's routing, and an error in a file the night touched reverts that file to the pre-night commit and flags it. The model routes; the linter decides pass or fail.

Eval questions that expire. Each golden question can carry an expiry date. Expired is its own grade, never a miss. And the tiers are graded separately now - answerable from the index, answerable with one search, deep - so a search-tier miss can't hide behind an index-tier pass or the other way round.

  before                                after

  fact ──▶ inbox ──▶ session file       fact ──▶ inbox ──▶ owner note (## Now)
                  ├▶ change log                             │
                  ├▶ status block                   generated from it, nightly
                  ├▶ recent-sessions row                    ▼
                  ├▶ project-index row            index rows · hub lists · search index
                  └▶ master-index row                       │
                                                  linter: pass / fail
        six copies, six clocks                              │
                                                  eval: index / search / deep

The night is one script. Ten stages, each logged with a result and a duration, each with an exit code: lint, pull, route, regenerate, banner, lint again, eval, report, explore, commit. Every prompt the night uses is a file in a folder, not a string inside a shell script, so changing one is a diff. The report the batch writes is assembled by the script from git's own view of what changed, not from the model's memory of what it did.

And the report proves the routing. This was the last thing I added, the same evening, after asking the obvious question: the linter proves the night was well-formed and the eval proves recall, but nothing proved a routed fact actually landed anywhere. So now it does, without a model in the loop. For every inbox line the night removed, a script searches git's diff for the added line that shares its words and reports the file and line it landed in. Below a threshold it prints NO ECHO, which is the one line in the morning report I now read first. The same stage proposes up to two golden questions from what it routed, with an expiry date, into a candidates file. I promote them with one word. The test set grows from the facts that actually moved, instead of from what I thought mattered in July.

The numbers

Before is August, measured from the transcripts and the git log. After is the 29th of August, the day the changes landed. That was one day in, so the first version of this post carried a single data point and a promise to come back. Five nights of history have run since. The last column is that follow-up, and it is less flattering than the first one, which is the point of measuring at all.

	before	day one	five nights in
Eval, all tiers	17/30 in July, 9/30 by late August	26/30	22, 28, 21, 24 — out of 32
Eval, index tier	1 miss of 18, then 6 of 18	0 of 13	3, 3, 4, 4 misses of 16
Eval, search tier	didn't exist	3 misses of 17	2, 3, 1, 0 misses of 16
Task index	4.8 KB in June, 21 KB by August, hand-written	8.1 KB, generated	—
Places one fact was written	6	1, plus links	—
Largest living note	98 KB	8 KB, diary alongside	—
Turns blocked by the config hook	30+ in four days	0	—
Night turns spent on the wrong job	~300 in August	0	—
Memory files restating vault facts	10 of 39	0 of 30	—
Lint errors on the first run	51	0	0 every night since

I said at the time that the 26/30 was the number I trusted least, because I had fixed the rotten golden answers in the same week, and one day of data cannot separate a better vault from a friendlier test.

Five nights later that caution was right. The all-tiers score went 26, then 22, 28, 21, 24. It did not hold at 26 and it did not decay back to 9 either. It wanders in the low-to-mid twenties out of thirty-two, which is roughly a two-thirds pass rate and nothing like the "fixed" the day-one number implied.

The tier split is more useful than the headline. The search tier improved steadily and reached zero misses. The index tier did not: it sits at three or four misses out of sixteen and has not moved. That is a real finding rather than noise. Retrieval by search is working; the generated index files still cannot answer questions they ought to own. The number that made me feel good was the aggregate, and the aggregate was hiding the tier that is actually broken.

The honest version

What's still broken, because a post like this is worth nothing without it:

The ## Now blocks are still written by a model. The generator can't drift a row, but the line it generates from is a paraphrase. What's new is that there's exactly one such line per fact, dated, and a monthly audit that spot-checks five of them against the newest session on that project. That's a check, not a guarantee.
One day of after-numbers. Everything in the right column of that table was measured hours after the change. The previous version of this system also looked great on day one. The eval runs nightly and the lint history is a file now; if the line bends down again I'll say so.
The eval is thirty-three questions, mostly written by the person being graded. Small, and biased toward what I thought mattered in July. The candidates mechanism above is the fix, and it has run for exactly zero nights as I write this.
The night still depends on one laptop being asleep in the right room. A missed night used to surface at my next session start, not before. Now a failed or stale run shows as a desktop notification at login, which is the whole fix and took twenty minutes. I'd built the fancy part first and the boring part last, which is a habit worth naming.
Staleness was invisible until I wrote this paragraph. A "current state" block dated six weeks ago is true and useless, and nothing flagged it. Now any active note whose block has no date newer than thirty days gets a stale marker in every generated row, and the linter says so. The first run flagged four notes. All four were exactly as stale as it said.
One person, one vault, one provider. Same caveat as last time. I put most of my life in this thing on purpose, with my own provider settings, and I don't claim my answer to the privacy question is yours.

The thing I'd want someone to take from this, one clause longer than last time: put the model where judgment belongs, determinism where correctness belongs, and a test where the guarantee is claimed - then read the test. A guarantee you don't measure is a hope, and a measurement you don't read is a number.

I'm a founder-engineer in Lisbon. I build B2B utilities end-to-end, and the tooling that keeps me honest while doing it. Open to fractional CTO and consultancy work (not full-time - my days are spoken for). Reach me at gilneto8.work@gmail.com or via gil-neto.com.

My agent had two rules and obeyed neither

Gil Neto — Wed, 19 Aug 2026 11:24:00 +0000

Both rules were well written. They contradicted each other and I hadn't noticed, because writing a rule and making a decision are not the same job.

I have a rule file that tells my agent to write tersely. Drop the articles, drop the filler, drop the pleasantries. Fragments are fine. It's precise about what to cut and it has examples.

I have a second rule file that tells it not to write convoluted sentences. One idea per sentence, break at around twenty words, no stacked clauses, gloss the jargon on first use. Also precise, also with examples.

Both files are good. Neither is wrong. And for a couple of months the agent obeyed whichever one it happened to weigh more that session, which meant I got compressed-and-unreadable on one day and readable-and-padded on the other, from the same setup, with no change in between. I kept editing the rules. The rules were never the problem.

The TL;DR: I had specified both behaviours completely and decided nothing. Specification describes what the thing does. Decision picks one option and kills the others. A document can be a hundred percent specified and zero percent decided, and from the outside those two look identical - which is exactly why nobody catches it until something has to execute.

What actually broke

Go back to the two rules. Read them side by side and the conflict is obvious in about four seconds. Terse says cut words. Readable says add the connective tissue back. These don't go well together.

Neither file said. Each one described its own behaviour thoroughly and stopped at its own edge. The question "what happens when these two pull in opposite directions" wasn't answered badly, it was never asked, because writing each file felt like finishing a job.

That's the shape of the failure and it generalizes past agent config. The parts of a spec that feel finished are the parts where one person wrote one thing and nobody argued. The parts where two things touch are where a decision has to happen, and a decision costs something that writing a spec doesn't: you have to give something up, on the record, and be wrong about it later in public.

So it gets deferred. Not maliciously. It gets deferred by being written around.

Specifying and deciding are different jobs

The distinction I'd draw, having got this wrong for a while:

Specification answers what does this do. It's descriptive, it's additive, and you can do it well without ever facing a trade-off. Two people can specify two conflicting behaviours in the same document and both be doing good work.

Decision answers which of these live options survives, and what do we lose by picking it. It's subtractive. It's uncomfortable. It leaves a scar in the document, in the form of an option that used to be there and isn't.

The tell that a decision happened is that something got killed. If you read a spec end to end and nothing was ever ruled out, no alternative is named as rejected, no cost is written down, then it isn't a spec that made hard calls. It's a spec that described a wish.

  what should happen

  ┌─────────────┐    ┌──────────────────────┐    ┌──────────┐
  │ open        │───▶│ DECIDE               │───▶│ specify  │
  │ question    │    │  resolve it, OR      │    │ the      │
  │             │    │  rule it out of scope│    │ winner   │
  └─────────────┘    └──────────────────────┘    └──────────┘

  what usually happens

  ┌─────────────┐                               ┌──────────┐
  │ open        │──────────────────────────────▶│ specify  │
  │ question    │      (skipped, silently)      │ both     │
  └─────────────┘                               └──────────┘
                                                      │
                                                      ▼
                                          looks complete, executes
                                          differently every time

Three tells

Ways I now spot an undecided spec from the outside, without having written it:

1. A quantity with no source. A weight, a threshold, a score component named to two decimal places and computed by nothing. The number is precise because precision was easy. Where the number comes from is the decision, and it's missing. Anything of the form "this factor counts for 25%" with no paragraph anywhere explaining what produces the 25% is a decision that was postponed by being written down.

2. Version-churn asymmetry. Look at revision counts across a document set. The ones revised over and over are usually the operational ones, and that churn is healthy - people are using them and correcting them. Now find the one or two documents that define what the thing fundamentally is, and check how many times those were touched. If the identity documents have the lowest churn in the set, that's not stability. Nobody argued about them, which usually means nobody had to commit to them.

3. Vocabulary collisions. The same word carrying two meanings in two sections, and no glossary entry, because a glossary entry would force somebody to choose. I hit this in my own vault with the word "phase", used for three unrelated things across two projects. Every one of those usages was clear locally. Together they were a trap, and the trap only sprang when something tried to act on all three at once.

Why agents make this worse

Here's the part that's new, and the reason I bothered writing this instead of filing it as a lesson - a human handed an undecided spec usually stalls. They read it, feel the ambiguity, and go and ask someone. The stall is annoying and it is also the safety mechanism. The question gets escalated to whoever can actually decide it.

An agent doesn't stall, it picks. It picks quickly, it picks plausibly, and it doesn't flag that it picked, because from where it sits there was nothing to flag: two instructions, both valid, one has to go first. Then it picks differently next session, because nothing pinned it.

So you get non-determinism exactly where you thought you had a rule. And you get it at speed, which is the new abyss. My terse-versus-readable oscillation was harmless, it just made me squint at my own output for a while. The same failure inside a build pipeline produces a lot of confident, internally consistent, mutually incompatible code, fast, all of it correctly implementing a spec that had no decisions.

AI accelerates what's decided. What's undecided, it improvises, at the same speed. That trade is fine when you know which half you're in. Most specs I've read, including several of mine, don't say.

What I do now

So I made two changes, none of them taking a lot of time.

The decision comes before the spec, and it's a separate artifact. Before anything gets specified, every open question is either resolved or explicitly marked out of scope. Out of scope is a real answer and it's often the right one - what's not allowed is leaving it unmarked, because unmarked reads as decided to everything downstream.

The resolution gets written into the thing it resolves, with the reason and a stop sign. When I finally settled the terse-versus-readable fight, I didn't write a third rule file. I wrote the decision into both existing files: which one wins, on which axis, why, the date, my name on it, and a line saying not to reopen it. Six months from now some version of me or some audit pass is going to look at those two rules, see the tension, and start helpfully resolving it again. The stop sign is there for that.

The second half is the part that gets underrated every time. A decision that isn't attached to the thing it constrains isn't a decision, it's a memory, and it lasts about as long as one.

Where I got this, and what I built out of it

I didn't arrive at any of this on my own. The thing that named it for me was Matt Pocock's wayfinder skill.

Wayfinder treats a plan as a map instead of a document. The unknowns are marked as fog of war, the open questions are decision tickets you have to actually close, and "not yet specified" is a first-class state on the map rather than a gap you skim past. That last part is the whole idea - most planning formats have no way to say this is undecided out loud, so undecided and decided end up looking the same on the page, which is the failure this entire post is about.

What I already had was the other half. I'd been running a spec-driven flow for a while - constitution, then spec, then plan, then tasks, then tickets, with traceability from any artifact back to the thing above it. It works well, and it has one condition nobody had written down: it assumes the decisions are already made. Feed it something unsettled and it will specify the ambiguity beautifully. Every question stated with precision, none of them answered.

So the two compose rather than compete. Wayfinder clears the fog and produces settled decisions. Spec-flow turns settled decisions into artifacts. I glued them together into one command I call midas - scout the ground first, refine what survives - and put the gate between the halves:

  /midas
    │
    ├─ scout        chart the idea, close every decision ticket,
    │               then accept or kill it
    │
    │               ▼ verdict gate - nothing touches a repo before 
    |                 this
    │
    └─ spec-flow    constitution → spec → plan → tasks → tickets

Two things about that gate that I'd defend. Everything before it is free to throw away, so the map lives outside any repo and abandoning an idea costs nothing and leaves nothing behind. And creating the repo is the ceremony that marks commitment, which turns the exit ramp into a structural fact instead of a note in a document that everyone ignores.

One honest caveat about wayfinder that isn't a criticism of it: it's built for a team with a tracker, and it assumes concurrency and handoff that a solo operator running eight hours a week does not have. Pointed at a one-person project it's coordination machinery against a problem that isn't there, and paying that cost anyway is how a process gets quietly abandoned. So the ceremony is tiered in my version, and the tier is worked out after charting, because before you've charted the thing nobody knows how foggy it is. The tool wasn't too heavy - my default consumption of it was just wrong.

The 'state of the art'

What's still broken, because a post like this is worth nothing without it:

Decisions rot and nothing tells you. The terse-versus-readable call was right right now. If my needs change it'll be quietly wrong and it will keep executing, with a stop sign on it that I put there. I don't have a good answer for this. Dating the decision and naming who made it is the whole mitigation, and it moves nothing.
You can't decide everything up front, and pretending otherwise is the same disease. Some things genuinely have to be discovered by building. The move isn't to resolve every question before writing a line, it's to mark which ones you deliberately left open, so the agent doesn't get to quietly close them for you.
This one I can't enforce outside the model. My last post was an argument for putting guarantees in hooks rather than in instructions, and I stand by it. This doesn't fit. No script can look at a document and tell whether a human made a hard call or wrote around one. So it's a manual gate, which means it's a gate I'll skip on a bad week, and I know that about myself.
I've run this on small things, not big ones. midas has gone through a handful of small features on Kelaro and Koa, the two products I actually ship, and it did what it says on the tin. What I haven't done is put a large build through it end to end, so I can tell you the gate works and I can't yet tell you what it costs at scale. The diagnosis is the part I'd defend hardest, because I got to it by being burned by it rather than by designing it.
One person, one setup. I've run this on my own config and on my own products. I have no idea what the decision-before-spec gate looks like when six people are writing the spec and the person who has to decide is in a different timezone. I'd guess it's harder in a way that changes the answer.

The thing I'd want someone to take from this: when a spec feels finished, check what it gave up. If the answer is nothing, it isn't finished, it's just written.

I'm a founder-engineer in Lisbon. I build B2B utilities end-to-end, and lately I spend a lot of time on the question of what an agent should and shouldn't be allowed to decide for me. Open to fractional CTO and consultancy work (not full-time - my days are spoken for). Reach me at gilneto8.work@gmail.com or via gil-neto.com.

Running a second brain on an LLM agent that forgets

Gil Neto — Fri, 17 Jul 2026 16:27:04 +0000

I keep my life in a folder of markdown files - projects, tasks, finances, ideas, the personal stuff. An LLM agent is the frontend to it. I talk, it answers from the notes and writes back to them. That part works now. It didn't for a long time, and the reason it didn't is the interesting part.

The TL;DR: I spent months writing better and better instructions telling the agent to save my context, and it kept losing things anyway. The trick is not in instructions - it's in hooks, in structure, in economics. Enforcement has to live outside the model, the data has to be shaped so that capture is cheap and routing is separate, and the whole thing has to cost little enough that you don't quietly stop running it.

What actually broke

The setup was the obvious one. A long instructions file, loaded every session, that said: when I tell you something new, write it down; at the end of the session, update the indexes and commit.

It worked most of the time. That's the problem.

An agent that follows an instruction 90% of the time doesn't fail 10% of the time - it fails silently 10% of the time, and you find out three weeks later when you ask what the state of something was and get a confident answer built on notes that stopped being updated in April. There's no exception, no red build, no failed job. Just a fact you told it once, that no longer exists anywhere, and no signal that it's gone.

Two things caused it, and both are structural rather than moral:

Instructions compete with the task. The session is about a bug in a payment webhook. The instruction file says "capture new facts as they arrive." The model is doing the bug. Something has to lose, and it's never the thing the user is visibly waiting for.
The expensive step is at the end. "Update the indexes and commit" is a real chunk of work, and it lands exactly when the session is winding down, when context is longest and everyone - me included - wants to be done. The most skippable step is scheduled at the most skippable moment.

You cannot fix either of those by writing the instruction more emphatically. I tried. Instructions aren't a contract. They're a suggestion with good intentions.

Hooks: enforcement outside the model

The first fix was to stop asking.

Most agent harnesses can run your code on lifecycle events - session start, session end, before or after a tool call, when the agent stops. That's a place to put things that must happen, that isn't the model's judgment.

So the rule "the vault must never be left with uncommitted work" stopped being a line in an instructions file and became a stop hook: when the agent finishes a turn, a shell script checks whether the vault is dirty and commits it. Not "please remember to commit". A script, on an event, whether or not the model was thinking about it.

The distinction that matters:

Judgment goes to the model. What does this new fact mean? Which note does it belong in? Is this a task or an idea? That's genuinely fuzzy work and the model is good at it.
Guarantees go to the harness. Did the commit happen? Did today's batch run? Is there unrouted input? That's a checkable condition and the model should never be the thing that checks it.

Every time I've been tempted to solve a reliability problem with a firmer instruction, the right answer has been a hook. Every time.

The corollary is the honest one: if it can't be enforced outside the model, don't design as if it will happen. Treat model-side steps as best-effort, and make sure the thing that catches the failure is somewhere else.

The shape at a glance

  INTERACTIVE (all day, cheap)          BATCH (03:00, once, expensive)
  ┌──────────────────────────┐          ┌────────────────────────────┐
  │  session (any project)   │          │  headless agent, on cron   │
  │                          │          │                            │
  │  I state a fact ─────────┼──┐       │  read whole day's inbox    │
  │                          │  │       │  route each line ──────────┼──┐
  │  reads: INDEX files only │  │       │  rewrite indexes (capped)  │  │
  │  (bounded, no fan-out)   │  │       │  quality + link pass       │  │
  └──────────┬───────────────┘  │       │  write the morning report  │  │
             │                  │       └──────────┬─────────────────┘  │
       ┌─────┴─────┐            │                  │                    │
       │ STOP HOOK │            ▼                  ▼                    ▼
       │  commits  │      ┌───────────┐     ┌────────────┐      ┌─────────────┐
       │ (no ask)  │      │ _inbox.md │     │ INDEX files│◄─────┤ notes (the  │
       └───────────┘      │ (queue)   │     │(projection)│      │  real data) │
                          └───────────┘     └────────────┘      └─────────────┘

The only thing an interactive session may do is append one line and let a hook commit. Everything that requires judgment across the whole picture happens once, at night, when nobody is waiting.

Structure: capture and routing are two different jobs

The second fix was noticing that "save this" is actually two jobs with completely different cost profiles, and I'd been welding them together.

Capture is: I just said something that matters. Write it down now, before the thought is gone. It needs to be atomic, cheap, and never blocked on anything. It happens mid-session, while the model is busy with something else, so it has to cost almost nothing - one append, one line, no reading, no thinking, no decisions.

Routing is: figure out what that line means, which of the fifty notes it belongs in, whether it changes a project's status, whether it makes a task obsolete, then rewrite the indexes. That's expensive, it needs the whole picture, and it is absolutely not something you want to do while the human is waiting.

So: an inbox file. One append-only list. Anything I state that the system should know gets one timestamped line appended, in the same turn, in any session, regardless of what that session is even about. Format's fixed, the line is dumb on purpose, and the timestamp has seconds in it so ordering is never ambiguous.

Nothing else happens at capture time. The inbox is allowed to be messy - it's a queue, not a record. The invariant is only this: an empty inbox means everything was routed. A non-empty one is a visible, checkable backlog rather than a fact that quietly evaporated.

That one split fixed more than any prompt engineering I did in six months. Capture became so cheap that it stopped competing with the task in front of it, which was the whole reason it was being skipped.

The same instinct applies to the instructions themselves. I had procedures spread across several files, described in prose, slightly differently in each place - which is how you get an agent improvising a slightly different save procedure every time. Now there's one canonical written procedure per job, in one place, and everything else points at it. The prose files say when; the procedure says how, once.

Structure, part two: indexes are projections, not documents

The other structural piece: how does the agent know what's going on without reading everything?

Naively, you let it search the vault. That's slow, it's expensive, and it's non-deterministic in a bad way - two sessions ask the same question and read different subsets of files, so they give different answers.

Instead there are a handful of index files - active projects, all open tasks, personal state - and those are the only entry points for any "what's going on?" question. Three properties make them work:

They're projections, not documents. The notes are the source of truth. An index is a derived view, and it gets fully rewritten by the nightly job, never appended to. Anything appended-to grows without bound and drifts from its source; anything rewritten is either correct or obviously broken.
They have hard caps. Ten recent sessions. Sixty words per project row. The caps are written into the file itself, so the thing generating them reads its own constraint. Detail lives in the notes it points at.
Reading them is the rule, not an option. "For broad questions, read only these files - no fan-out." A bounded, predictable read cost per session, no matter how big the vault gets.

The caps are the part people push back on, and they're the part that makes it work. An index without a cap becomes a second copy of the vault within a month, and then you have two sources of truth, which is one more than the maximum useful number.

Economics: the reason it runs at all

Here's the thing nobody tells you about agent systems: the token budget is an architectural constraint, not a billing detail.

Routing the inbox properly - reading it, deciding where each line goes, updating the notes, rewriting four indexes, committing - is expensive. Doing that at the end of every interactive session means every session pays for it, and I have a lot of small sessions. The cost lands on the wrong side of the trade: I'm paying premium interactive tokens, at the moment I'm most impatient, for work that has no deadline.

So it moved. Interactive sessions only capture. The expensive routing runs once a night, on a cron, in a headless agent process, over the whole day's inbox at once.

That change did several things at once, which is usually the sign you've found the right seam:

The batch sees the whole day, so it routes better than fourteen separate sessions each seeing one line. It can tell that a task created at 11:00 was made obsolete by a decision at 16:00.
The expensive work happens when nobody's waiting, so it can afford to be thorough - a quality pass, a link check, a contradiction check between indexes.
My interactive sessions got cheaper and faster, because the last thing before "done" is now one file append instead of a full sync.
It produces a report. I read it in the morning: what got routed, what changed, anything it wants me to look at.

And then the one I want to defend because it sounds absurd and isn't: the laptop wakes itself up at 02:28 to run it.

The batch is at 03:00. The laptop is suspended at 03:00, because I'm asleep. Systemd timers with Persistent=true are helpful about this - they run the missed job at the next boot instead. Which meant the whole overnight batch fired the moment I opened the lid at 9am: my machine busy, my agent busy, and the day's first hour of token allowance spent on last night's chores before I'd had coffee.

The fix is an RTC alarm - WakeSystem=true on a system timer, which arms the hardware clock to wake the box from suspend. It wakes at 02:28, the jobs run, a guard checks it's still the middle of the night and nobody's touched the machine, and it suspends again. I wake up to a finished report and a full budget.

This is the least glamorous part of the system and it's the one that changed my mornings. The economics aren't a footnote to the architecture. They're the reason it survives contact with a real week - a system you keep flinching away from because of what it costs you at 9am is a system you'll stop running by March.

The same bet, three times

I noticed while writing this that the vault isn't where I first made this decision. It's just where I made it most recently. The two products I'm actually developing on the same sentence.

Kelaro and Koa - never silently wrong about money. Kelaro is an accounting product; Koa is the engine underneath it that turns a bank statement PDF into a ledger. The whole market is currently sprinting toward vision-LLM extraction: hand the page to a model, get structured data back, no templates to maintain. It's genuinely seductive. It also puts a non-deterministic system in charge of arithmetic, which is a category of decision I'm not willing to make on someone else's books - the failure isn't a model that says "I don't know", it's a model that returns a plausible total. FinGround (ACL 2026 Industry Track) is blunt about the failure modes it set out to fix in financial LLM systems: fabricated metrics, invented citations, arithmetic errors.

Koa uses fixed templates, forces the balances to reconcile, and refuses - loudly - when it can't. It would rather reject a statement than guess at a number. Around it, Kelaro's pipeline runs on durable workflows: retries, timeouts, operator-visible state. The model isn't in the arithmetic anywhere. That's not purism. It's that "never silently wrong about money" is worth more than the convenience of skipping template work.

Augur - evidence you can check, not evidence you're asked to trust. Augur is an EU AI Act evidence product: the deliverable is a bundle you can hand a reviewer. The interesting design constraint is the same one. The judgment - does this system's documentation actually satisfy the obligation? - is human work, and I'm not pretending otherwise. But what comes out the end has to be checkable: a fixed record structure, a signed bundle, a verification anyone can run without taking my word for anything. A regulator does not want a confident summary. They want the artifact and the ability to check it themselves.

Which is the same sentence as this post. Put the model where judgment belongs, and determinism where correctness belongs. Koa's model doesn't get to invent a number. Augur's doesn't get to be the reason you believe the evidence. The vault's doesn't get to decide whether the commit happened. In every case the LLM does the genuinely hard, fuzzy, valuable part - and something dumb, deterministic and checkable stands behind it making sure the failure mode is loud instead of silent.

I didn't plan the symmetry. I noticed it, which is a slightly embarrassing thing to admit and a fairly good argument that it's real.

The honest version

What's still broken, because a post like this is worthless without it:

The inbox has lag. Facts captured today aren't reflected in the indexes until tomorrow's batch. Sessions have to account for it - read the raw inbox as well as the index, treat it as "true but not yet filed". It's a real cost of the batching decision, and I pay it knowingly.
The routing is still the model's judgment. It's better at it overnight with the full day in view, but it's not correct by construction. So there's a weekly drift check: another job that audits the indexes against the notes and reports contradictions. Enforcement outside the model, again, one level up.
Nothing here proves the model didn't misunderstand a fact at capture time. Garbage in, faithfully routed garbage out. The system guarantees nothing is lost. It doesn't guarantee it was right.
It did this to me while I was writing this post. Part of the nightly batch is a research pass that surfaces things worth reading. It handed me a tidy claim - the 2026 research shows vision-LLM extraction confidently fabricating figures that pass review - with four sources attached. Every URL was real, which is the part that makes this dangerous. But when I actually opened them: one paper genuinely supports it. One is about something else entirely, and its headline result arguably cuts against my framing. Two are vendor marketing from companies selling the fix, one of which does no original research and one of which admits its benchmark contradicts the public ones. The claim was a confident synthesis that none of its sources quite made. I nearly published it, in this post, about this exact failure mode. I checked because the post told me to.
It's a single user's system. I have no idea what any of this looks like with two people writing into the same vault, and I'm suspicious of anyone who claims to.

I put most of my life in this thing - the work, the money, the personal history - as context the agent reads before it answers. That's a decision with an obvious privacy dimension, and each person has to make it for themselves, on their own terms, with their own provider settings. I've made mine deliberately. I'm not going to pretend my answer generalises to yours.

But the architecture does generalise, and it's the part worth stealing:

Capture cheaply and atomically. Route expensively and rarely. Enforce with hooks, not adjectives. Cap the things the agent reads. Let the batch see the whole day. And put the model where judgment belongs, not where guarantees belong.

None of that is about prompts. I wish someone had told me that a year ago - I'd have written considerably fewer instructions and considerably more shell scripts.

I'm a founder-engineer in Lisbon. I build B2B utilities end-to-end - and, apparently, the tooling that keeps me sane while doing it. Open to fractional CTO and consultancy work (not full-time - my days are spoken for). Reach me at gilneto8.work@gmail.com or via gil-neto.com.

Running 3 SaaS products on a single Hetzner CCX23

Gil Neto — Tue, 12 May 2026 15:09:02 +0000

I run three SaaS products on a single Hetzner CCX23. The box costs about €30 a month, hosts a closed-beta accounting product, a Phase 0 GRC landing, and a deterministic PDF extraction engine, and absorbs all the inbound mail, outbound transactional email, and analytics for both customer-facing products. This is the boring, opinionated version of how it fits.

The TL;DR: a small dedicated-CPU VPS is enormously capable when you treat it like a single multi-tenant runtime instead of an ad-hoc bag of services. Below is what's actually running, where the seams are, and the signals that tell me when this stops being clever and starts being a liability.

Why a single VPS

Three reasons.

Cost discipline. Pre-revenue, my infra bill is a tax I'm choosing to pay before I know whether the product will return it. €30 a month is a tax I can ignore.

Operational surface area. One box means one set of firewall rules, one TLS story, one set of backups, one Docker daemon, one OS to patch. Each additional box doubles every operational rule I have to keep in my head — and I run this alone.

Co-location wins. Two products that share a database server, a reverse proxy, an SMTP relay, and an analytics endpoint also share their incident response. If anything goes wrong, I'm SSHed into the one place that matters.

The cost of that decision is a single, well-known failure mode: this box is the blast radius. I'll come back to that.

The box

Hetzner CCX23, EU region.

4 dedicated vCPU cores (AMD EPYC)
16 GB RAM
160 GB NVMe
20 TB monthly traffic included

For €30 or so. "Dedicated CPU" is the important part — the cheaper shared-CPU Hetzner tiers have noisy-neighbour problems that show up exactly when you're trying to demo a product to a customer. The CCX line is steady-state CPU you can plan against. If you're considering this setup, do not start with a CX or CPX shared-CPU instance. The savings are not real once you factor in stalls during workflow runs.

The stack at a glance

                       DNS (Cloudflare / Hetzner)
                                │
                                ▼
                       ┌──────────────────┐
                       │  Nginx (host)    │  TLS, vhost routing
                       └────────┬─────────┘
                                │
        ┌───────────────────────┼───────────────────────┐
        ▼                       ▼                       ▼
 ┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
 │  Kelaro stack   │    │  Augur stack    │    │  Koa stack      │
 │                 │    │                 │    │                 │
 │  • Next.js app  │    │  • Fastify API  │    │  • FastAPI eng. │
 │  • Temporal     │    │  • Umami        │    │  • Next.js UI   │
 │  • Postgres 16  │    │  • Postgres 16  │    │  • Postgres 16  │
 └─────────────────┘    └─────────────────┘    └─────────────────┘
                                │
                                ▼
                       ┌──────────────────┐
                       │  External SaaS   │
                       │  • Brevo (SMTP)  │
                       │  • ImprovMX (in) │
                       └──────────────────┘

Each product is its own Docker Compose stack — its own app container(s), its own Postgres 16, its own internal network. Nginx on the host is the single shared piece, handling TLS termination and routing each vhost to the right stack. Outbound transactional mail and inbound forwarding are the only external dependencies, and they're shared SaaS — not infrastructure I own.

The shape of each stack is intentionally the same. Same Postgres version, same base image, same compose layout, same .env.example pattern, same deploy story. The products differ in what they do; they don't differ in how they're put together. That consistency is the dial I tune fastest — every time I touch a piece in one stack, I make sure it slots cleanly into the others, because the next product is going to want exactly the same thing.

Routing layer: Nginx on the host

I run Nginx directly on the host rather than as a container. Two reasons:

TLS termination is a host-level concern. I want one place that knows about certificates, and I want it to come up before any Docker service tries to bind a port.
Zero-downtime config reloads. nginx -s reload is a battle-tested SIGHUP path. I trust it more than I trust orchestrating a containerized Nginx restart while two products are taking traffic.

Each vhost (kelaro.io, app.kelaro.io, tryaugur.eu) is a one-screen file in /etc/nginx/sites-available/. Certificates are Let's Encrypt via certbot --nginx, on a renewal cron.

kelaro.io lives behind Cloudflare DNS; tryaugur.eu uses Hetzner DNS. That's a wart — eventually I'll consolidate to one DNS provider — but right now the cost of leaving it is zero. The cost of migrating either is one nervous Saturday morning.

Persistence: one Postgres per product, same shape everywhere

Each product runs its own Postgres 16 container, bound to 127.0.0.1 on its own port, inside its own Docker network. Three products, three databases, three independent backup targets. The same image, the same postgresql.conf baseline, the same role-and-privileges pattern.

The instinct when you're cost-optimising is to merge them into a single Postgres with multiple logical databases. I don't, for three reasons:

Blast radius per product. A schema migration that's slower than I expected on Kelaro should not be a thing that touches Augur's lead capture or Koa's extraction state. Containerised isolation gives me a clean answer to "did anything else break?" — no.
Backups are per-product. Each Postgres has its own pg_dump cron and its own restore drill. When I rehearse a restore, I rehearse one product, not a whole-box recovery. The smaller the surface, the more honest the rehearsal.
Migration to managed is a no-op. The day Kelaro hits a customer count that demands managed Postgres, I lift one container out and point the connection string at RDS or Neon. The other two stacks don't move. Co-tenancy at the database level would make that lift a multi-day project; co-tenancy at the box level keeps it a Saturday.

Postgres is the resource that scales least gracefully, so I monitor it more aggressively than anything else: connection count, table bloat for the workflow tables, replication slot growth, and the usual disk and CPU metrics. The graphs are per-product, which means I can see exactly which product is misbehaving without having to filter a noisy global view.

This per-product-Postgres pattern is the heart of how the box stays legible. The temptation when you're running multiple products solo is to centralise everything in the name of efficiency. The lesson, twice learned, is that operational legibility beats efficiency at this stage. A small machine running uniformly-shaped stacks is easier to reason about, easier to debug, and dramatically easier to scale out from than a "clever" shared substrate.

Workflows: Temporal, where it earns its keep

Kelaro's PDF extraction pipeline runs through Temporal. Bank statements come in, get queued as workflow executions, Temporal handles retries, timeouts, durable state, and the operator UI gives me visibility into anything that misbehaves. Augur doesn't need workflows yet — Temporal is dormant from Augur's perspective.

Two things I tell people considering Temporal at small scale:

It's worth it well below 100 requests per second if your work is multi-step, externally fallible, and operator-visible. The cost is a Temporal Server you don't fully exercise — at this scale, a few hundred MB of RAM and a couple of Postgres schemas living inside the Kelaro stack. Cheap.
It is overkill for fire-and-forget jobs. If all you have is "send an email when a thing happens", BullMQ on a small Redis is dramatically less infrastructure.

Koa — the extraction engine — is a Python/FastAPI service running in its own stack with its own Postgres, behind a clean HTTP boundary. Kelaro's workflows call into it; Koa never knows it's part of a workflow. That separation is deliberate: Koa is going to outlive any one product that integrates with it.

Mail: ImprovMX inbound, Brevo outbound

I don't run my own mail server. Two SaaS handle it:

ImprovMX forwards *@tryaugur.eu to my real inbox. Free tier, dead simple, one MX record. There is no scenario in which I want to be on call for an inbound SMTP daemon.
Brevo handles transactional outbound — sign-up confirmations, billing receipts, password resets. Better deliverability than my VPS could earn from scratch (a fresh VPS IP is a deliverability cold start I have no patience for) and a working SPF/DKIM/DMARC alignment out of the box.

For Kelaro, I also use Brevo SMTP. For Augur, mail is currently inbound-only — outbound waits until there's an actual app. Until there's an app, there's nothing to send.

Analytics: self-hosted Umami

Augur targets EU-regulated buyers. Putting Google Analytics on its landing page would be a self-own. I run Umami inside Augur's stack — same Docker Compose, same Postgres pattern, same backup discipline — privacy-clean, cookie-banner-free, GDPR-easy. The analytics endpoint is internal to the stack; nothing leaves the box.

The trade-off is honest: I run my own analytics service. It's a thing I have to update, back up, and reason about. The win for Augur is that I get to truthfully say "we don't ship your visitors to a third-party tracker." For a GRC product, that's a sales line. Across the rest of the portfolio, I'm migrating in the same direction — uniform analytics, uniform privacy posture, no per-product carve-outs.

Backups and observability

This is the part most "I run my own VPS" posts skip and then regret.

Hetzner backups for the whole VM, enabled. Daily, rolling, included for a small fee. Recovery is "click a button"; it has saved me already.
Postgres logical backups via pg_dump to S3-compatible storage off-box. Daily, with 30-day retention. The whole-VM snapshot is for "the box is gone"; the logical backup is for "one database is corrupt and I noticed three days later."
Healthchecks.io pings for every cron job that's supposed to run. If a backup didn't happen, I get a Telegram message before I deploy any new code.
Structured logging to host syslog, with a daily logrotate. Eventually this needs to be Loki or similar; for now journalctl is enough.

The thing I do not yet have, and should: a real uptime monitor hitting each product's public endpoint every minute, with a status page. Until then, "Kelaro is down" gets reported to me by a pilot user — which is fine at pilot scale, and absolutely not fine the day there's a paying customer who hasn't agreed to be a beta tester.

Blast radius

Everything on one box means the box is one blast radius. Three failure modes I plan around:

Disk fills. A runaway Postgres table or a runaway Docker log can fill 160 GB faster than I'd like. Mitigations: logrotate configured aggressively, Postgres autovacuum tuned, disk usage alerts at 70 and 85 percent.
A workflow goes hot and starves the rest. Temporal can run workers that consume more CPU than I budgeted. Mitigations: Temporal task queues have explicit worker counts; CPU is partitioned implicitly via Docker cpus: limits when I remember to set them. (I do not always remember.)
The host goes down. This is the unmitigated one. There is no failover. The bet is that the products are early enough that an hour of downtime per quarter is acceptable in exchange for €30/month. The day that bet stops being acceptable is the day I split.

The point of being honest about the blast radius is that it tells you exactly when this architecture breaks: the moment a customer is paying enough that "an hour of downtime" is a refund conversation.

When this stops being clever

There is a list pinned above my desk. When any of the following becomes true, the box gets split:

Kelaro hits paying customers. Then it gets its own VPS, with its own backup strategy. Augur stays here.
CPU steady-state above 60%. Headroom matters more than savings; once I'm consistently above 60% I have no margin to absorb a deploy.
Postgres connections regularly above 60% of max_connections. Same logic.
Either product needs sub-100ms p99 globally. A single EU box doesn't serve from US-east; I'd be looking at a CDN or a multi-region split.
A regulated buyer asks for SOC 2 or ISO 27001. Compliance does not love "everything on one box." Some controls are dramatically easier on a managed platform.

None of these is true today. The day one of them is, the migration will be one Postgres database and one set of containers — which is the whole point of treating the box as a multi-tenant runtime in the first place.

The honest version

I am not running this stack because it is the best possible stack. I am running it because three pre-revenue products on a €30/month box is the right answer to a question that nobody asked me until I asked myself: what is the smallest infrastructure I can ship through?

Six months from now, the answer will be different. The point is not to defend this stack forever. The point is to stay cheap, fast, and operationally legible until the products earn the right to be more complicated.

If you're running something similar, or about to, I'd love to hear what your "when to split" list looks like. Mine is pinned above the desk because every founder-engineer I know secretly suspects they're under-engineering and would benefit, mostly, from leaving things alone.

I'm a founder-engineer in Lisbon. I build B2B utilities end-to-end on this box. Open to senior IC, staff, fractional CTO, or contract — remote-first, EU timezones. Reach me at gilneto8.work@gmail.com or via gil-neto.com.