DEV Community: Victor García

Your files, and the two questions a folder can't answer

Victor García — Thu, 20 Aug 2026 18:13:50 +0000

A folder is a very old idea and a very good one. You put a thing somewhere, and later you look in that somewhere and the thing is there.

What a folder has never been able to tell you is anything about what's inside it. Two questions in particular, and they're the two that cost you real time:

Did I already save this somewhere else?
What did this look like before I changed it?

Every file manager lets you answer those by hand. You go looking. You compare. You give up and keep both. This post is about the two screens that answer them for you.

THE ORDINARY PART, FIRST

This is a real working drive — a small fireworks company, eight departments, a few loose files that never found a home. Nothing here needs explaining, which is the point. You can drag things around, make folders, upload, star, throw away.

The storage counter in the corner says 97.6 MB used. It's one number, always visible, and it counts what's actually on the disk.

The one thing worth noticing: the sunset photo shows a preview and the documents don't. Previews are made ahead of time, when the file lands, not while you wait for the folder to open. So a folder full of images opens as fast as a folder full of anything else.

QUESTION ONE: DID I ALREADY SAVE THIS?

Here's the whole feature in one screen. Five groups. 380 KB you can get back.

Look at the first one. The same photo of a sunset at the Malvarrosa beach lives in three places:

/drive/Shows 2026/Malvarrosa sunset - site scouting.jpg
/drive/BoomClaw/Malvarrosa sunset - site scouting.jpg
/Photos/2026/07/beach-sunset-01.jpg

Two of them share a name. The third one doesn't. It's the same photo — byte for byte, the same photo — sitting in the camera roll under the name the camera gave it, and no amount of looking through folders was ever going to connect it to the other two.

That's the part that matters. The grouping isn't done by name, or by size, or by date. It reads the contents and gives each file a fingerprint; identical contents mean an identical fingerprint, whatever you called it or wherever you dropped it.

And then it doesn't do anything. Every copy has its own Keep this button, and until you press one, nothing moves. That's deliberate. A deduplicator that decides for you is a deduplicator that eventually deletes the copy you needed — the one on the desktop you were about to email, or the one your accountant has a link to. The system is allowed to notice. You do the deciding.

Two honest limits:

It catches identical files, not similar ones. Re-export that sunset at a smaller size and it's a different file with a different fingerprint. Same picture to you; not to the fingerprint.
380 KB is not going to change your life. On this drive the duplicates are small. The value isn't the space — it's knowing that the three copies exist, so you stop wondering which one is current.

QUESTION TWO: WHAT DID IT LOOK LIKE BEFORE?

This is a product spec for a firework shell, and it has been through three revisions since June.

v1, 14 June, 191 bytes — the first draft.
v2, 28 June, 367 bytes — after the first test firing went wrong.
v3, 11 July, 436 bytes, labelled "Before the safety review".

Each one has three buttons: download it, put it back, throw it away.

Two things are going on here, and they're different. v1 and v2 were taken automatically — nobody asked for them. When you save over a file, the version that was there a second ago gets kept first, and then your new one takes its place. You don't have to remember to do anything, which is the only reason it works; the backup you have to remember to make is the backup you don't have.

v3 was taken on purpose. Someone was about to hand the document to a safety reviewer, pressed + Snapshot, and typed "Before the safety review". That's the difference between a version history and a useful version history: automatic snapshots tell you when, and a note tells you why.

You can see where this goes. The reviewer comes back and says the minimum distance figure is wrong. You change it. Three weeks later somebody asks what the original measurement was, and the answer is one click away instead of a conversation.

The honest gap: you can't compare two versions side by side yet. You can download an old one and open it, or restore it and look, but there's no diff view. For a short document that's fine. For a long one it's the first thing you'll want, and it isn't there.

WHY THESE TWO AND NOT TWENTY

Both of these exist for the same reason, and it isn't tidiness.

Every file you keep is a small bet that you'll be able to find it and trust it later. Duplicates break the trust — three copies and no idea which one is current. A missing history breaks it too — you changed something, you can't remember what it said, so now you're not sure the current version is right either.

Neither problem shows up on the day you create it. Both show up eighteen months later, when you need the thing and you're standing in front of four files with similar names.

WHAT'S NOT HERE

Said plainly, because a features list that only lists wins isn't information:

No side-by-side compare between versions, as above.
Version history is a Pro feature. Snapshots take disk space, and there's a policy screen behind the gear icon to cap how many are kept per file — but it's not in the free tier.
Near-duplicate detection doesn't exist. Only exact matches. The resized copy of your own photo is invisible to it.
The duplicate list doesn't suggest which one to keep. It shows you the paths and the dates and stops. Deliberate, but it does mean you read five lines instead of pressing one button.

THE TAKEAWAY

Folders answer "where". They were never going to answer "again?" or "before?".

Those two questions are what turn a drive full of files into a drive you can rely on — and they're the two you can't answer by looking harder, because the answer lives across folders and across time. So the system answers them, shows its work, and leaves the decision with you.

Everything on your own hardware, as always. The fingerprints, the snapshots and the previews are all computed at home; nothing about your files goes anywhere.

Next up: your photos. Type what you half-remember about one — the colour of a door, a city, someone in a hi-vis vest — and see if it finds it.

Photograph a receipt, and your inventory fills itself in

Victor García — Thu, 06 Aug 2026 08:57:20 +0000

Everyone who has ever started an inventory has abandoned it in week two. Not because it isn't useful — because keeping it current is a chore that pays off later and costs you now.

So the interesting question isn't what the module can store. It's how little you have to do to keep it true.

THE ANSWER TO "WHAT DO I HAVE?"

Four numbers, then two charts that answer the two questions people actually ask: what is it worth and where is it.

The ring is value by category, with the total in the middle and the same data as a ranked list beside it — 54% of the value is one van, 16% is the firing system. The bars are value by location, so you can see that most of what you own is sitting in the rigging van and not in the warehouse you thought.

The amber card counts what's below its minimum. Two items. That's the entire low-stock feature: you set a floor on the things you don't want to run out of, and this number is how you find out.

THE PART THAT DOES THE WORK FOR YOU

Underneath is a queue, and it's the reason the inventory stays current:

Four items proposed, each with its price, each with a tag saying where it came from — RECEIPT. You didn't type any of them.

You photographed a receipt, or pressed a button on an order confirmation in your mail. A vision model reads it and pulls out each product, how many, the unit price, a category, the reference code, the shop and the date. The model runs on your own machine — the photo doesn't go to anyone's cloud, because there isn't one to go to.

And then it stops. It proposes; it doesn't add. You get Approve, an edit pencil, and a reject cross, and you can fix the price before you accept it. That's deliberate: a small model reading a crumpled receipt will get a line wrong now and then, and the person who should catch it is you, in the two seconds it takes to glance at a list — not in six months when the numbers don't add up.

Two small mercies that came out of using it:

Photograph the same receipt twice and nothing duplicates. Each proposed line gets a fingerprint, so re-analysing an email you already processed adds nothing.
It works out your warranty. If what you bought looks like electronics, an appliance or a tool, it sets the expiry three years out — the EU legal minimum — from the purchase date it read off the paper. There's a panel that warns you when there are fewer than 90 days left, for things you only ever photographed a receipt for.

THE CATALOGUE

One row per thing, and the columns are the questions: what is it, what kind, where is it, how many, what's it worth, when does the warranty end, and what state is it in.

Quantities read 9.000/4.000 — how many you have over the minimum you set. When the first number drops below the second it goes red, which is how "Mortar rack (12 tubes)" is showing 3.000/4.000 in amber up there. No configuration, no alert rules to write. A number and a floor.

Filters across the top for category, status, location and tag, and a search box. Services sit in the same list as physical things, with a dash where the quantity would be, because "Fireworks show (rigging + firing)" is something you sell but never stock.

NOTHING IS EVER DELETED

Every arrival, sale, loan, transfer and recount is a line here, and the quantity on the catalogue page is the sum of them. Not a number someone typed over — the sum.

That means the history survives your mistakes. Correct a movement and you don't lose the original: a counter-movement appears next to it, and both stay. The button that does it says so on the tooltip, in as many words: the ledger is never deleted.

There's an Undo on every row for exactly that, and a Receive document button in the toolbar, which is the same receipt-reading trick pointed at a delivery note.

AND A SCANNER THAT ASKS FIRST

The round button in the corner opens the camera, but not straight into a scan. It asks what you're doing first, with four options:

Locate — open whatever this is, or the bin it's in
Count — adjust the quantity, for a stocktake
Move to bin — transfer it somewhere else
Check in / out — return a loan, or lend it

Which sounds like a small thing until you've used a scanner that guesses. Same barcode, four completely different intentions, and no way to undo the wrong one gracefully. Stick a QR label on a box, scan it, pick Locate, and you get a list of what's inside without opening it.

WHAT IT DOESN'T DO YET

No printable QR labels for items yet — only for locations. Boxes and shelves, yes; individual things, not yet.
No mobile stocktake mode. You can count with the scanner one item at a time, but there's no "walk the shelf and tick things off" flow.
The loan reminder button is disabled. You can lend something with a return date and see it go overdue, but the nudge isn't wired up yet.
Serial numbers and batches aren't there. Right now ten identical things are a quantity of ten, not ten tracked units.

THE TAKEAWAY

An inventory is only worth having if it's true, and it's only true if updating it is nearly free. Everything above is in service of that: a camera instead of a keyboard, a queue you approve instead of a form you fill in, a floor number instead of an alert rule, and a history that survives being wrong.

The best inventory feature is the one that means you don't have to do the inventory.

Next up: your photos. Type what you half-remember about one — the colour of a door, a city, someone's face — and see if it finds it.

Three ways to look at your money. Pick the one that's you.

Victor García — Tue, 04 Aug 2026 15:38:18 +0000

Most money software makes you a promise it can't keep: that one screen can serve a person tracking their spending, a freelancer chasing an unpaid invoice, and a company that has to file a VAT return. You end up with a tool that's too heavy for the first and too thin for the last.

So the first thing you choose in Micelclaw isn't a report. It's who you are today.

Three lenses over exactly the same data. Nothing is duplicated, nothing is synced, and you never have to visit the one that isn't yours.

PERSONAL — WHERE DID IT ACTUALLY GO

This is the lens for a question everyone has and almost nobody can answer on demand: am I better off than I was in January, and what am I spending it on?

Five numbers across the top, and each one has a subtitle telling you what it's made of, because a figure without its definition is just decoration. Net worth is everything you own minus everything you owe. Available balance is only the accounts you can actually spend from — the sum on the left doesn't include money that's owed to you, which is the single most common way people fool themselves.

The savings rate says 24%, and underneath, in smaller type, you save 6560,86 €. A percentage tells you how you're doing; the euros tell you what it bought. Both, always.

Then three charts that answer three different questions:

Net worth over time — the shape of the last twelve months. Steps up are money arriving; the long flat stretches are the interesting part.
Expense by category — a ring, and next to it the same thing as a ranked list with percentages and amounts. Salaries 51%, Utilities 14%, Insurance 10%. You can flip the whole panel to Income with one click and see where it comes from instead.
Cash flow — money in and money out, month by month, with the running balance drawn over the top.

Below that, your accounts with their real balances, and the most recent movements. Notice that some of those movements have a small grey label to the right — Corporate tax payable, Social security payable. That's the system telling you which account each one landed in, without you having to ask.

INVOICING — WHO OWES YOU, AND WHO ARE YOU AVOIDING

The freelancer's lens. It exists because "how's business?" is really four questions, and they have four different answers.

Invoiced is what you've billed. Collected is what actually arrived. Pending is what's owed to you and still in date. Overdue is the one to look at on a Monday morning. Four numbers, four colours, and the difference between the first two is the whole reason freelancers run out of money while "doing well".

The same row repeats underneath for the other direction — what you've been billed, what you've paid, what you owe, what's late. And then the two figures that matter at the end of a quarter: the result for the period, spelled out as 29.222,33 € invoiced − 7541,98 € expenses, and the VAT you'll owe, spelled out as charged 5113,30 € − paid 1112,48 €. No black boxes. Every headline number shows its arithmetic.

Further down there's an ageing breakdown — how old your unpaid invoices are, in buckets, with the percentages — and your top customers as bars, so you can see at a glance whether one client is quietly becoming your whole business.

The invoice list itself is deliberately boring, which is the point:

Number, customer, issued, due, total, status. Sent, Paid, Overdue, Draft. A draft has no number yet — it shows a dash — because numbering an invoice you might delete is how you end up with gaps you have to explain later.

Click a customer and you get their whole file on one page:

Tax ID, currency, payment terms, email, address, and every document you've exchanged with them, each with its status and amount. The same list holds customers and suppliers, filtered by a chip at the top, because in real life the same company is often both. And there's an Open in CRM button, since a customer here is the same contact as everywhere else on your server — not a second copy that drifts.

ACCOUNTING — THE PART YOU HAND TO SOMEONE ELSE

The third lens is the one most people will open twice a year, and that's fine. It's there so that when your accountant asks for something, the answer is a click and not an evening.

Four figures, a profit-and-loss summary, and four shortcuts to the things an accountant actually names: the journal, the chart of accounts, the tax forms. And in the corner, quietly, a line that says The books balance.

That line is doing a lot of work. Underneath all three lenses is real double-entry bookkeeping — the same method your accountant uses, with a Spanish chart of accounts, VAT rates and withholding built in. You never see any of it from the Personal lens. You just get a sentence confirming that everything you did this month added up.

If you do want to look, the trial balance is right there:

And here's a detail worth pausing on, because it's the kind of thing that separates a real ledger from a spreadsheet with ambitions. Look at the account codes: 400 appears five times, once per supplier. 430 appears once per customer. That's not a bug. In proper bookkeeping those codes are categories, not identities — every customer gets their own receivable account, all filed under 430. It means you can ask "how much does Marina Costa Brava owe me?" and "how much is owed to me in total?" and both are one query.

ONE BOX, SEVERAL COMPANIES

There's a small dropdown next to the module name that says Principal. That's the profile switcher.

If you run more than one thing — a company and your own freelance work, or two businesses — they live in the same box and one click apart, with completely separate books, invoices, customers and tax numbers. Nothing leaks between them, and you don't need a second server or a second subscription.

WHAT IT DOESN'T DO YET

Being straight about the edges:

No electronic invoicing yet. Spain is moving towards mandatory structured e-invoices (Facturae, and reporting straight to the tax agency). We generate PDFs, not signed XML. That's the biggest gap and we know it.
The tax forms are calculations, not filings. It works out your quarterly VAT and your withholdings and shows you the numbers. It does not submit anything, and you can't yet save a snapshot of a form as you filed it.
The quarterly income-tax instalment is a simple version. It applies the flat rate to income minus expenses. It's not a personal tax return and doesn't pretend to be.
Bank connections are import, not sync. You bring in a CSV or an OFX file from your bank and match it against your books. There's no live feed.

THE TAKEAWAY

The reason this took three lenses instead of one is that the alternative is a screen full of things you have to learn to ignore.

A person shouldn't have to know what a chart of accounts is to find out where their money went. A company shouldn't have to lose the general ledger because the interface was simplified for someone else. So: same data underneath, three doors in, and the door you don't need stays shut.

You should be able to answer "how am I doing?" without first learning accounting.

Next up: the same idea, pointed at your things instead of your money — and the bit where you photograph a receipt and your inventory fills itself in.

A 45% cut, and two things I was wrong about

Victor García — Tue, 04 Aug 2026 09:07:14 +0000

Atlas is the agent that handles files, photos, office documents and projects. It runs on gemma-4-12b with an 80,000-token context window, locally, on the same box as everything else. One afternoon it stopped fitting.

Not "gave worse answers." Stopped fitting. The tool schemas of the apps it had been given were, on their own, larger than the room the model had to think in.

Back in March I wrote #07 — Your AI agent is wasting 90% of its context, about how much of an agent's window goes to describing tools it will never call. This is the sequel, four months later, with real measurements instead of estimates. It's also a retraction: two of the things I was most confident about turned out to be wrong, and a bug I'd diagnosed turned out not to exist.

THE MEASUREMENT THAT STARTED IT

Our agents reach the backend through a facade that exposes every domain as a typed tool (#13 has the architecture). tools/list returns the catalogue. Here's what it weighed:

tools/list → 508,782 bytes

Half a megabyte of JSON schema, for 409 tools. The interesting question isn't "that's a lot" — it's what exactly is in there.

So I counted. Two fields dominated:

AUTH_TOKEN_FIELD: z.string().describe("…")
CONVERSATION_ID_FIELD: z.string().describe("…")

Every tool declares both, because the facade needs the caller's identity and the conversation it belongs to. Their descriptions, multiplied by 409 tools, came to roughly 133,000 tokens of the catalogue. Descriptions of two fields that the model never fills in — since #14, a host-side hook injects the token before the call ever reaches the server.

The second offender was a boilerplate paragraph in every tool description — 211 characters explaining how to pass the auth token — repeated 408 times. Once is instruction. 408 times is furniture.

THE FIX WAS SUBTRACTION

T1: empty the .describe() on both identity fields. They stay declared — this matters, and I'll come back to it — but they carry no prose.

T3: strip the boilerplate from 408 descriptions and put it exactly once, in the agent's TOOLS.md, where it belongs:

- Token: the auth_token of every claw_* tool is injected AUTOMATICALLY (host-side identity hook).
  DON'T fill it in yourself — if you do, it's ignored.
- Params: build each tool's parameters ONLY from the user's CURRENT request;
  do NOT reuse or copy the ones from a previous call.

Result, verified live:

	before	after
`tools/list`	508,782 B	278,533 B
boilerplate occurrences	408	0
`auth_token` still declared	409/409	409/409
capability lost	—	none

−45.3%, and every tool still executes. The whole change is deletion.

Why the fields stay declared

The MCP SDK validates tool arguments with a zod object in strip mode: the handler receives the parsed args, and a field that isn't declared is discarded before it arrives. If we removed auth_token from the schema to save the last few bytes, the host-side injection would write into a field that the SDK then throws away — and every agent would stop authenticating at the same moment.

That's why the remaining optimisation is deferred rather than shipped. Removing $schema and omitting the two identity fields from the announced schema (while keeping them in the parsed one) is worth roughly 25,000 tokens of context per agent — and requires bridging past the SDK's convenience wrapper. It's a separate change with its own verification, because the failure mode is "everything breaks at once."

THE FIRST THING I WAS WRONG ABOUT

Here's what I believed when I started: 408 tools were leaking into the model's context.

The arithmetic seemed to support it. Half a megabyte of catalogue, divide by ~3.6 characters per token, and you get a number that looks catastrophic.

It's a false positive, and the error is embarrassing once you see it. Dense JSON doesn't tokenise at 3.6 characters per token — it's closer to 1.3, because every brace, quote and colon is its own token. At that ratio, 408 tools would be ~360,000 tokens, which is more than the context window of the models involved. The number was impossible, and I'd written it down anyway.

What was actually happening: the model received about 65 tools. The runtime's per-agent deny-list was working correctly the whole time.

The way to check is not arithmetic. It's the trace:

~/.openclaw/agents/<agent>/sessions/<id>.trajectory.jsonl
  → event: context.compiled
  → data.tools = the EXACT array sent to the model

That file has the answer, exactly, with no estimation. I had spent a day reasoning about a number I could have read.

THE SECOND THING I WAS WRONG ABOUT

I had also assumed the facade's tools/list was filtering by identity — that when an agent asked for the catalogue, it got its catalogue.

It doesn't. tools/list returns everything. The per-agent trimming is done 100% by the runtime, via a deny-list of tool names (claw-os__<tool>). The facade is not the enforcement point; it never was.

This matters because it changes where a fix has to live. Any per-agent trimming we want has to be expressed as entries in that deny-list — which is exactly what the next part does.

CORE AND FULL: THE MACHINERY WAS ALREADY THERE

Every capability in our registry can be tagged tier: 'core' | 'full'. The plumbing to compute a per-agent deny-list from those tiers existed too. It had been built months earlier and then left almost unused: only two domains tagged anything as full.

So the work wasn't building a mechanism. It was making a judgement call, 123 times: for each advanced capability, is this everyday or is this rare?

domain	capabilities	tagged `full`	everyday core
finance	70	48	22
inventory	51	33	18
projects	40	24	16
files	31	18	13

The criterion: core is CRUD plus the essential reads. Full is the advanced and the rare. Per-domain, core is 57–65% lighter.

And then a detail that pays for the whole exercise: the tiers had been inert. The filter only applies to domains exposed as one-tool-per-action, and those domains hadn't been migrated yet. Tags existed, meant nothing, and nobody noticed — including a docker.inspect_container that had been tagged full specifically to keep a raw dump out of core mode. Migrating the domains activated a security decision that had been sitting dormant, without writing another line for it.

The number that reordered the plan

My intuition was: migrate the infrastructure domains to per-action tools, and while we're at it, split core from full. Measured, it's the other way round.

	tools	catalogue bytes	vs before
Before (6 fat tools)	6	32,445	—
Core (the new default)	50	32,938	+1.5%
Full (asked for on demand)	88	48,161	+48%

The six infrastructure domains hold 88 capabilities between them. Migrating them to one tool per action with full mode as the default would have grown the catalogue by 48% — a straight regression. In core mode, 50 tools cost the same as the 6 fat ones did, and the model no longer has to guess its way through a 34-value action enum.

For the infrastructure agent — gemma, 80k window — that's +1,083 tokens instead of +12,793.

So the split isn't an optional optimisation layered on the migration. It's what makes the migration survivable. The comment now sits in the code where the default lives:

 * The six INFRASTRUCTURE manuals are the exception, and the reason is measured […]
 * In core they're 49 tools and cost the SAME as the six fat tools before (−0.1%) —
 * with the bonus that the 34-action enum the model had to guess through disappears.

One default, one place

That default used to live copied in three separate files: the visibility calculation, the TOOLS.md generator, and the context-budget estimator. Now there's one function, with the reason attached:

 * Effective mode of ONE skill. Single source: the default lived copied in three
 * places (visibility, TOOLS.md and context budget) and with copies it's a matter
 * of time before the catalogue says one thing and the deny-list says another.

An agent whose catalogue and whose deny-list disagree is a bug that presents as "the model hallucinated a tool."

ASKING FOR MORE, TEMPORARILY

Core mode raises an obvious question: what happens when an agent genuinely needs an advanced capability?

It asks. There's a capability for it (skills.elevate, approval level 2 — you get a card), and on approval the app moves to full mode and the deny-list is re-synced, so the complete tools enter the agent's context on its next turn.

The elevation is ephemeral, and that decision is worth more than the feature:

// The elevation is ephemeral, "until the session closes" […] with NO new table
// and no new cron.
// Gotcha: a Core restart loses this registry → the app stays 'full'
// (which is the system DEFAULT, harmless).

An in-memory registry keyed by conversation, reverted when the conversation is deleted. No migration, no scheduled job, and a failure mode that degrades to the pre-existing default instead of to something novel. Verified against the database and the runtime: core (33 tools denied) → elevate → full (0 denied).

THE BAR THAT MADE IT VISIBLE

None of the above is usable without an answer to "how full is this agent right now?" So: an endpoint, and a bar in the dashboard, showing base + system prompt + apps + free space against the model's window.

The most important thing about it is what it refuses to do:

// It's ALL ESTIMATES: there's no tokenizer, so we use the ratios already established
// […] (prose /4) and the JSON tool schema (/1.3, denser).
// Fail-soft by construction: a missing workspace file counts 0; if the model's window
// can't be resolved, model_window stays null (it is NOT invented).

const CHARS_PER_TOKEN_PROSE = 4;
const CHARS_PER_TOKEN_SCHEMA = 1.3;
export const CONTEXT_BUDGET_BASE_TOKENS = 4000;

And the base:

 * Calibrated estimate of OpenClaw's "base" prompt + its NATIVE tool definitions […]
 * NOT measurable server-side: that text is injected by the runtime binary, not in any
 * file Core reads. Exposed as a CONSTANT and marked source:'estimate'.

In the dashboard that segment is drawn striped, so it looks like an estimate. A number that can't be measured shouldn't render identically to one that can — that's the difference between a budget and a guess with a progress bar.

The first agent we pointed it at was Francis, our coordinator, with nine apps in full mode:

91,000 / 80,000 tokens  ·  −11,000 over budget

Red. The offenders were finance (27k), projects (15k) and files (13k) — the exact three domains where core mode is worth the most.

That reading was taken before the catalogue cut landed. Pointing the same bar at the same agent today:

78,000 / 80,000 tokens  ·  97%  ·  free: 2,300

Still uncomfortably close to the ceiling — nine apps is a lot — but it now fits, which it didn't. The apps that remain expensive are the ones still defaulting to full mode: photos (15k), mail (8.6k), calendar (7.6k).

THE BUG THAT DIDN'T EXIST

One more retraction, and this is my favourite.

While profiling, I noticed that gemma reported usage.input roughly double the previous call's usage, over and over. I diagnosed a phantom retry: something was silently re-sending the prompt, and we were paying twice for every turn.

Then I checked the correlation instead of trusting the story. Across a snapshot of turns:

ratio ≈ 2.0 ⇔ there was a tool call: 11 out of 11
no tool call: ratio 1.0, every time

There's no phantom retry. That's the normal shape of a tool turn: call one emits the tool call, call two reads the result and answers. usage.input is the sum of both. The system was working exactly as designed and I'd written a bug report about it.

The reframe is better than the bug would have been: since every prompt token is prefilled twice on a tool turn — and the local runtime doesn't reuse its KV cache between the two calls (cacheRead: 0) — every token we removed from the catalogue is worth double. The 45% cut isn't 45% of one prefill, it's 45% of two.

Which points at the next lever, and it isn't ours: getting the local runtime to reuse the KV prefix between those two calls. Same prompt, same tools, back to back.

WHAT I'D DO DIFFERENTLY

Read the trace before doing the arithmetic. Both of my wrong conclusions came from estimating something that was written down exactly, in a file I already had. Estimation is for things you can't observe. The tool array sent to the model is observable.

Check correlation before writing the bug report. "usage doubles" was a real observation with a wrong explanation. Eleven data points took ten minutes and dissolved a bug I'd have spent days chasing.

Look for mechanisms that exist and are inert before building new ones. The tier system was fully built and doing nothing. If I'd designed a fresh one, I'd have shipped a second mechanism and left the dormant one to confuse the next person.

Make estimates look like estimates. The striped segment is four lines of CSS and it's the reason nobody will ever quote the base number as a measurement.

HONEST CAVEATS

The 45% cut is verified live. The tiers are verified against the database and the runtime. The elevation cycle is verified at the database and runtime level — not end-to-end in a real chat conversation, which is a different thing and I'm not going to claim it. Non-infrastructure apps still default to full; only the six infrastructure ones default to core. And there's no measurement here in euros or in latency: this post is about bytes and estimated tokens, which is what we could actually measure.

THE TAKEAWAY

The engineering was subtraction: two empty .describe() calls and one paragraph moved to where it belonged, for 45% of the catalogue.

The expensive part was retracting two confident conclusions and one bug that didn't exist — all three of which came from reasoning about numbers I could have read directly.

If a number matters enough to act on, find where the system writes it down.

Next up: the seven agents from #15 now fit in the window. What happens when you ask them to work on the same thing at the same time?

Seven agents per user: delegation, and every way it broke

Victor García — Wed, 29 Jul 2026 13:42:55 +0000

Here's a bug report we got from our own system, and it took us longer to understand than we'd like to admit:

"Ask Francis to generate a voice recording. He says he'll delegate it to Dali. Then nothing happens."

Nothing happened. No error, no timeout, no failed tool call. The coordinator agent said it was handing the job to the specialist, and then the conversation just… stopped. Dali never spoke.

We built a seven-agent topology because one agent with sixty tools is worse than seven agents with ten each. That part turned out to be right. What we underestimated is that delegation is the whole product. Every interesting failure in the last four months has been at the seam between two agents, not inside one.

This is a tour of that seam, including the parts still broken.

THE TOPOLOGY

Every user gets seven agents, provisioned atomically when their account is created. They are not shared. They are not a pool. Here's the actual configuration, straight out of auto-provision.service.ts:

Agent	Role	Model	Scopes
Francis	Coordinator — notes, calendar, mail, contacts, search, diary, bookmarks	`claude-sonnet-4-6`	`notes:* events:* emails:* contacts:* diary:* bookmarks:* search:read graph:read`
Atlas	Productivity — files, photos, office docs, projects, diagrams	`claude-sonnet-4-6`	`files:* photos:* search:read`
Sentinel	Infrastructure — storage, network, reverse proxy, DNS, VPN, Docker	`deepseek-chat`	`storage:* hal:* proxy:* dns:* vpn:* docker:*`
Dali	Creative — image generation, media, voice (TTS/STT), web browsing	`claude-sonnet-4-6`	`audio:* media:* photos:read files:read`
Ledger	Money — bank accounts, transactions, budgets, CRM	`deepseek-chat`	`finance:* crm:* search:read`
Darwin	Intelligence — patterns, preferences, cross-domain search	`claude-sonnet-4-6`	`search:read`
HASS	Smart home — IoT devices, automations, scenes, sensors	`deepseek-chat`	`ha:*`

Three things worth pointing out, because they're the parts people get wrong when they copy this shape.

The models are deliberately different. Sonnet for the agents that need judgment (coordination, creative work, pattern-finding); DeepSeek for the agents that mostly run deterministic infrastructure and finance operations. This isn't cost optimization dressed up as architecture — Sentinel restarting a container doesn't need the same reasoning as Francis deciding which specialist should handle a request.

The topology is flat. Depth-1, siblings only, no nested sub-agents. Francis is the chief but not a bottleneck: you can talk to any agent directly from the chat. The coordinator exists to route the requests where you didn't say who should handle it.

Two of them are nearly empty now, and we left it that way. Darwin ships with exactly one skill (claw-search). There's a comment in the config explaining why: claw-graph and claw-insights were retired in June because the knowledge graph and the derived insights turned out to be human features of the dashboard, consumed through REST, not something an agent needed to poke at. Ledger is in the same shape. The honest version of a multi-agent design is that some agents shrink after you watch what people actually ask for.

There's also a hidden eighth agent, crawl, dedicated to the browser extension. It's marked hidden: true and never shows up in the agents module — it exists so that the extension's chat doesn't compete with Francis for attention in the user's real browser session.

Namespacing, or: why every agent has a double dash in its name

Agents are per-user, and the runtime that executes them is shared. So the identifier of an agent is {prefix}--{name} — paco--atlas, admin--francis. The prefix is the local part of the user's email, normalized to [a-z0-9-], with a numeric suffix on collision, and it lives in users.agent_prefix as UNIQUE NOT NULL.

Each agent-user pair gets its own workspace directory:

~/.openclaw/workspaces/paco--atlas/
├── SOUL.md      # personality (atlas = methodical, dali = creative...)
├── USER.md      # user profile, propagated from the chief on bootstrap
├── TOOLS.md     # the catalog of skills this agent can reach
├── MEMORY.md    # persistent memory
└── skills/      # physical copies of the SKILL.md files it's allowed to read

The display name stays "Atlas". The namespacing is invisible to the user — right up until it isn't, which we'll get to.

FAILURE 1: THE CHILD THAT INHERITED THE WRONG TOOLS

The runtime gives an agent two ways to start work somewhere else. sessions_spawn creates a child. sessions_send hands a message to another agent's session and waits for the reply.

We used sessions_spawn first, because "spawn a Dali to do the audio" reads like exactly what you want. It isn't. A spawned child inherits the parent's tools, not the target agent's. So sessions_spawn(agentId: "paco--dali") produces something that is named Dali, is addressed as Dali, and does not have claw_audio, because Francis doesn't have claw_audio.

The delegation guide now says it in a box, in capitals, at the top:

sessions_spawn creates a subagent that INHERITS YOUR tools, not the specialist's.
If you do sessions_spawn(agentId: "<PREFIX>--dali", ...), that child "Dali" ends up
WITHOUT claw_audio (because you don't have it) and the task fails.
sessions_spawn is only for YOUR OWN sub-tasks (a worker of yours running in parallel
with your same tools). To delegate to a specialist, ALWAYS sessions_send.

Scenario	Tool	Why
Ask Dali for audio, Atlas for files, Sentinel for infra	`sessions_send`	The specialist runs as itself, with its tools (synchronous, you wait for the reply)
Run a sub-task of your own in parallel	`sessions_spawn`	It's your worker; inheriting your tools is correct here

The second-order lesson is more useful than the fix: when two operations differ only in whose capabilities they carry, a model will pick the wrong one. The names don't encode the difference. Only the documentation does, and documentation is a prompt, and prompts get compacted.

FAILURE 2: AN OPTIONAL PARAMETER WITH NO DESCRIPTION

This is the one that produced the bug report at the top, and it is still open.

sessions_spawn takes an agentId. In the runtime's schema, that parameter is declared as:

agentId: Type.Optional(Type.String())

Optional. And — this is the part that matters — with no description. Meanwhile taskName and thread both have descriptions. The tool's own summary is one line: "Spawn subagent or ACP session."

Put yourself in the model's position. You get a tool with several described parameters and one undescribed optional one. You fill in what's described: task, taskName, label, model, runtime, sandbox, mode. You omit the one nobody explained. And because it's optional, the runtime doesn't complain — it spawns you. Francis delegates to Francis, does the work badly or not at all, and reports back that the specialist has been notified.

We verified this against a real trace (session ea2c4895): the call carried seven parameters and no agentId.

It's the same class of bug as another one we hit in the MCP facade — a parameter the model can't fill because the schema never told it the parameter was load-bearing. A tool call is a contract between a schema and a language model, and an undocumented optional field is a contract clause written in invisible ink.

This one is still in our open debt list. The fix we've scoped is a before_tool_call hook that rejects sessions_spawn without an agentId and returns the specialist map in the error, so the model can retry correctly in the same turn — the same hook mechanism we already use for agent identity. We haven't shipped it yet, and until we do, delegation via spawn is unreliable. Saying otherwise would be more comfortable and less true.

FAILURE 3: THE PARENT THAT WAITED FOREVER

When a parent delegates and yields, it's waiting for an announce-back from the child. That notification is best-effort. So:

Children can abort (timeout, reset, bug) and the notification to the parent is best-effort. If Core/Gateway restarts while a parent is yielded, the children's announce-back is lost and the parent hangs forever — there's no sweeper to detect it.

A conversation that is neither finished nor running, with no error anywhere, is the worst possible state: nothing to retry, nothing to report, nothing in the logs.

The detector we built reads the last 8 KB of each parent's session JSONL looking for a sessions_yield not followed by an assistant message, and writes a row into stuck_yield_sessions (migration 0187, idempotent via a unique key on (user_id, session_key)). Two design details earned their keep:

A 3-minute grace window (STUCK_GRACE_MS) before flagging anything. The comment in the code explains why: "so we don't false-positive during legitimately long delegations (Atlas with thinking + tool calls can take ~30 s before the first assistant text)." Without it, the detector's main output is noise about work that's going fine.
Auto-resolution. If the parent un-yields on its own, the row clears without anyone intervening. A recovery mechanism that requires a human to close its own tickets is a second job, not a fix.

Recovery is a POST /managed-agents/:id/stuck-yield/resume that injects a synthetic message into the parent, plus a button in the dashboard.

FAILURE 4: THE CONVERSATIONS NOBODY COULD SEE

The runtime runs children internally and never emits WS events for their messages to Core. Result: in the dashboard's Conversations tab you see the chief ("I delegated to Atlas") but you don't see Atlas's conversation. To audit a council or a multi-agent workflow you have to open the raw JSONL in the workspace.

This is an observability gap that becomes a correctness gap. If you can't see what the delegated agent did, you can't tell "Dali refused" from "Dali was never asked" from "Dali did it and the reply got lost" — which are three completely different bugs that present identically.

The mirror tails each subagent's session JSONL with byte-offset watermarks (migration 0188, dedup index 0189, 1 MB read cap per pass) and writes the messages into agent_conversations in two modes: type='delegation' for child runs, and type='webchat' for the parent's reply to the announce-back — which, as the code notes, "happens inside the Gateway after chat-bridge.callGateway has already returned", and was therefore invisible to Core by construction.

FAILURE 5: THE POLICY WITH NOWHERE TO PLUG IN

We built a DelegationPolicy. Per-agent defaults, retry with backoff, an in-memory circuit breaker, storage for all of it. It was good work. We reverted it the same day, with migration 0180_drop_delegation_policy.sql.

The reason is in the doc, and it is the most useful paragraph in this post:

Delegation between agents happens via sessions_spawn, which is a native LLM tool inside the runtime — not an RPC that Core can intercept. With that API, the DelegationPolicy we built has no insertion point:
✅ Storage · ✅ Per-agent defaults · ✅ withRetry helper with backoff · ✅ In-memory CircuitBreaker
❌ Nobody invokes it — Francis still delegates via sessions_spawn directly.
Keeping it alive was infrastructure without a consumer.

Four components, all working, all correct, all unreachable. We had built a governance layer for a call path that never passes through us.

We wrote down the three conditions that would unblock it — the runtime exposing sessions.spawn as an RPC, changing the skill's contract to an HTTP delegate_to against Core, or building our own spawning runtime ("several sprints, high risk") — and deleted the code. The conditions are the valuable artifact. The code was the expensive way to discover them.

FAILURE 6: THE ALLOW-LIST THAT ALLOWED TOO MUCH

Agent-to-agent messaging is gated by an allow-list. Ours was a flat list of wildcards, one per user: paco--*, pepito--*, admin--*, ten entries. And the guard's formula was matchesAllow(requester) && matchesAllow(target).

Read that twice. Both sides match the list. paco--francis matches. pepito--atlas matches. So paco--francis → pepito--atlas is allowed. One user's coordinator could send work to another user's specialist, and everything downstream — per-user workspaces, per-user scopes, isolated memory — would be doing its job correctly on the wrong person's data.

What was actually preventing it, as we noted at the time, is that the models don't know other users' agent IDs. That is not a security boundary. That is a fact about vocabulary.

It's now closed deny-by-default in the identity plugin (crossUserSendBlock): an agent can only send to agents sharing its own prefix. Legitimate cross-user delegation, when we build it, will be an explicit grant — not the absence of a check.

WHAT I'D DO DIFFERENTLY

Find the insertion point before writing the policy. The DelegationPolicy revert cost us a day and taught us something we could have learned in twenty minutes by asking "what function will call this?" before writing the first line. When you're building governance for a system you don't fully own, the first deliverable isn't the policy — it's the proof that the policy can be enforced.

Treat tool schemas as prompts, because they are. Two of the six failures above (the spawn/send confusion, the undescribed agentId) are the model correctly following an under-specified contract. We spent weeks looking at model behavior when the bug was in a type declaration. The parameter descriptions in a tool schema are not documentation for humans reading the code; they are the only instructions the model receives at the moment it has to decide.

Build the observability at the same time as the delegation, not after. We shipped delegation, then spent a month unable to diagnose it because the child conversations were invisible. The mirror should have been part of the first version. A multi-agent system where you can only see one agent isn't a multi-agent system, it's a black box with a friendly name.

Assume best-effort notifications will be lost, and write the sweeper first. The stuck-yield detector isn't clever; it's 8 KB of tail-read and a grace window. The mistake was shipping the yield before shipping the thing that notices when the yield never ends.

ONE MORE THING: THE AGENTS SHARE A BRAIN

A rule from the delegation guide that surprises everyone, including us:

While you wait, do NOTHING else: no sessions_history, no process, no exec, no
sessions_yield, no re-sending the task, no trying it yourself with Python/shell.
sessions_send already waits; the result comes back on its own. On this team the
agents SHARE an AI model: if you keep acting (e.g. polling their history), you hog
the model and the specialist CAN'T work → the task hangs. Stay still.

An agent that anxiously checks on its colleague's progress prevents its colleague from making progress. Seven agents, one model instance, and the failure mode is that helpfulness starves the thing it's trying to help.

THE TAKEAWAY

The topology was the easy decision. Seven specialists and a coordinator, flat, per-user, with their own scopes and their own workspaces — that shape has held up for four months without a single revision.

Every hard problem lived in the handoff: a child with the wrong tools, a parameter the model couldn't see, a parent waiting on a message that got lost in a restart, a conversation nobody could read, a policy with nowhere to attach, and an allow-list held together by the fact that nobody knew the right names.

If you're building multi-agent anything: the agents are not the system. The seams are the system.

Next up: #16 — fitting seven agents into a model with an 80k context window, which is a story about a 45% cut, two conclusions I was wrong about, and one bug that turned out not to exist.

Deterministic agent identity: a before_tool_call hook fills the token the model kept getting wrong

Victor García — Sun, 26 Jul 2026 10:42:27 +0000

Francis is the main agent — the one you talk to in the dashboard. Dalí is the creative one, the sub-agent you hand an audio or an image task to. One day we asked Francis to have Dalí generate a short audio clip, and nothing came back. No error in the dashboard, no red toast, no timeout the user could see. Just silence.

We logged the bug as "Dalí doesn't answer," and it sat in the tracker for a while looking like a flaky model or a bad prompt — the kind of thing you're tempted to blame on a small local model having a bad day.

It was neither. It was an authentication hole with a very specific shape, and closing it meant taking a 64-character secret out of the language model's hands entirely.

The problem the model couldn't see

Our agents don't talk to Core with curl. They call typed tools — claw_notes, claw_calendar, claw_audio — served by an MCP facade that runs host-side at 127.0.0.1:7250 (that's the previous post's subject). Every one of those tool calls has to carry an auth_token argument. Core takes the token, resolves it to a userId plus a set of scopes, and dispatches the call through its full pipeline via app.inject() — auth, scope check, semantic-scope check, approval, handler, audit. The facade reimplements none of that security; the token is the single thing that lets the pipeline know who is asking.

So the whole design rests on one question: where does that token come from?

Before this fix, it came from the prompt. Two bridges injected it — ws/chat-bridge.ts for webchat and ws/voice-bridge.ts for voice. When you typed to Francis in the dashboard, the chat bridge dropped Francis's token into the prompt, and Francis was instructed to copy it into every claw_* call. That works — mostly — for a main agent talking through a bridge.

A subagent spawned with sessions_spawn passes through neither bridge. When Francis delegates to Dalí, there is no chat bridge and no voice bridge in the path. So Dalí received no token at all. Every claw_* call it made failed auth, silently, and the delegated work just stopped. That was "Dalí doesn't answer": not a model refusing, a model that was never handed the credential it needed to do anything.

And even the main agent, on the happy path, was fragile. We were asking a language model to reproduce a 64-hex-character secret, verbatim, on every single call. A capable model does it — most of the time. But a secret the model has to retype is a secret the model will eventually get wrong, and when it improvised a tool it hadn't been shown properly, it improvised the token too.

Fact	Value
Real runtime token	64 hex characters
Copied correctly on a capable model	6 / 6 (Qwen3.6-35B IQ2_M, ~27K-token context)
Value emitted when it improvised a wrong tool	42 characters — not in the database
Subagent success rate before the fix	0% (no token was ever injected)

The 6/6 number is the trap. On rails, a strong model looks perfectly reliable, and you conclude the design is fine. Then the same model, one wrong turn later, emits a 42-character string that matches nothing in the database — and the subagents, which you can't watch as closely, get it wrong 100% of the time because they get nothing at all. Fragile by construction.

Why not just a header

The obvious fix is a header. MCP servers usually authenticate with a static header — so put the token there, stop asking the model to carry it, done.

It doesn't work here. In OpenClaw, the mcp.servers config is global. The headers are static and shared across every agent — there is no per-agent identity per session. If we hard-coded a token into the server's headers, every agent on the box would authenticate as the same identity. That destroys the entire point of per-agent scopes: Dalí is supposed to have audio:*/media:* and nothing else; Francis has its full grant. A shared header can't express "this call is Dalí, that call is Francis."

That global-static constraint is exactly why ADR-10 put identity in a tool argument in the first place. ADR-11 doesn't undo that decision — the facade still dispatches Authorization: Bearer <auth_token> through app.inject(). All that changes is who fills the argument. Instead of the model, it's the host.

The hook

OpenClaw lets a native plugin register a before_tool_call hook — a function that runs on the host, sees every tool call before it executes, and can rewrite its params. We wrote one: openclaw-plugins/claw-identity/index.js, plain ESM JavaScript, no build step. The core of it is about five lines.

// The core of openclaw-plugins/claw-identity/index.js — the before_tool_call hook
api.on("before_tool_call", async (event, ctx) => {
  const toolName = event?.toolName ?? "";
  if (!toolName.startsWith("claw-os__")) return;      // only the MCP facade tools
  const token = loadTokenMap(mapPath)[ctx?.agentId];  // map keyed by ctx.agentId
  if (!token) return;                                  // no entry -> don't touch the call
  return { params: { ...event.params, auth_token: token } }; // override host-side
}, { priority: 100 });

That's the whole idea. For any tool whose name starts with the facade prefix claw-os__, the hook looks up the calling agent by ctx.agentId in a token map and rewrites params.auth_token to that agent's token. Whatever value the model put there is overwritten. If there's no map entry for the agent, the hook returns undefined and leaves the call untouched — it never breaks a call it doesn't recognize.

The model never sees the token, never types it, never hallucinates it. And because the key is ctx.agentId, a subagent spawned via sessions_spawn gets its own token injected host-side, with its own scopes — the exact case that used to fail silently. Determinism, keyed on who is actually running the tool.

The per-agent token and the map

Where does the map come from? Core mints it. syncUserRuntimeTokens(userId) in core/src/services/agent-runtime-token.service.ts creates exactly one canonical agent_tokens row per agent, with name='runtime'. The database stores only the sha256 hash; the plaintext exists only in an on-disk map.

That map is ~/.config/micelclaw/agent-tokens.json, and its shape is deliberately dull:

{
  "paco--francis": "…64 hex chars…",
  "paco--dali":    "…64 hex chars…"
}

The keys are exactly the ctx.agentId values the hook looks up — <prefix>--<agent>, one per user-agent. Writing that file is where we were careful, because the plugin reads it live:

// Atomic map write — core/src/services/agent-runtime-token.service.ts
async function writeMapAtomic(map) {
  const path = runtimeTokenMapPath();               // ~/.config/micelclaw/agent-tokens.json
  await mkdir(configDir(), { recursive: true, mode: 0o700 });
  const tmp = `${path}.tmp`;
  await writeFile(tmp, JSON.stringify(map, null, 2), { mode: 0o600 });
  await rename(tmp, path);                           // atomic swap: the plugin never reads a half file
}

mkdir with mode 0o700, the file itself 0o600, and a temp-file-plus-rename so the swap is atomic. The plugin reads through an mtime cache (loadTokenMap): it only re-reads from disk when Core actually rewrites the file, and on any read error it falls back to the last-known map and never throws. That mtime-triggered re-read is exactly why the write has to be atomic — if we wrote the file in place, the plugin could catch it half-written and hand out a truncated token.

The scopes matter as much as the token. The desired scope set is derived from the agent's live role grant:

deriveScopesFromSkills(skillScopeDomains(skills), roleScopesFor(agentDbId), perms)

roleScopesFor returns the agent's broadest base token — so Francis keeps its full grant (~17 scopes) and Dalí keeps audio:*/media:*. Nothing is downgraded, nothing is escalated: the runtime token carries exactly the identity the agent already had. And it is re-minted only when the scopes actually change — a scopesEqual guard keeps it stable otherwise, so we're not churning credentials on every sync.

Here's the whole thing in four pieces:

Piece	File	Role
The hook	`openclaw-plugins/claw-identity/index.js`	`before_tool_call` → injects `auth_token`, priority 100
Token service	`core/src/services/agent-runtime-token.service.ts`	Mints one `runtime` token per agent, writes the map
Registration	`core/src/services/openclaw-bootstrap.service.ts`	Idempotent `configPatch` at boot + on regenerate-tools
The facade	`core/src/mcp/claw-os-mcp-server.ts`	Serves `claw_*` at `127.0.0.1:7250`, dispatches via `app.inject()`

Rollout and verification

The hook only helps if the plugin actually loads. Registration is an idempotent configPatch — plugins.load.paths plus plugins.entries.claw-identity.enabled — done by ensureClawIdentityPluginRegistered() in openclaw-bootstrap.service.ts, both at boot and from regenerate-tools. The plugin then loads on the next Gateway reload.

We didn't trust that it worked until the logs said so. Two lines in the Gateway log (/tmp/openclaw/openclaw-YYYY-MM-DD.log) tell the whole story:

[claw-identity] plugin cargado; mapa de tokens = /home/victor/.config/micelclaw/agent-tokens.json
[claw-identity] before_tool_call tool=claw-os__claw_notes agent=paco--francis token=ok

The first line says the plugin loaded and resolved its map. The second says the hook fired for a real MCP tool call, for the right agent, and injected a token that Core accepted (token=ok).

Then the end-to-end proof — the one that actually closed "Dalí doesn't answer." We sent Dalí a direct message: "genera un audio que diga Hola caracola." The log showed before_tool_call tool=claw-os__claw_audio agent=paco--dali token=ok. Dalí called the real claw_audio tool — not a shell workaround, the actual tool — and an .ogg file landed in Drive with a real file_id. The subagent that had never received a token now authenticates like any other agent.

The gotchas

None of this was clean the first time, and the credibility of a fix like this is in the sharp edges we hit getting there:

plugins.entries.<id> rejects env. The entry schema only accepts enabled, hooks, and config. A configPatch with an env key returns INVALID_REQUEST: Unrecognized key: "env" — and the whole patch is dropped, not just the bad key. So the plugin resolves the map path by its default (~/.config/micelclaw/agent-tokens.json) rather than reading it from the entry's env.
The MCP SDK strips undeclared params. Any tool-call argument not declared in the tool's inputSchema disappears silently. We host-inject an internal _conversation_id (so Core can route L2 approval cards back to the chat they came from) — and it must be declared in the schema, or the SDK drops it on the floor and the routing quietly breaks. Same class of trap as the token itself.
The plugin needs a local SDK symlink in dev. index.js imports openclaw/plugin-sdk/plugin-entry; from its own directory, Node needs a local node_modules/openclaw symlink pointing at /usr/lib/node_modules/openclaw (gitignored). A packaged deploy installs the plugin with its own dependencies; dev has to fake it.
Boot registration can lose the WS race. If the Gateway WebSocket isn't up yet when Core tries to register the plugin, the configPatch is dropped and retried. That's why POST /managed-agents/regenerate-tools re-ensures registration once the WS is reliably up — it's the belt to the boot-time suspenders.
tsx watch doesn't hot-reload reliably on WSL. Code changes need a full Core restart to take effect, and the plugin itself only loads on a Gateway reload. Edit-and-refresh does not apply here; you verify against the real, restarted runtime or you verify nothing.

Where it grew

Worth being honest about how this file aged. It started as ~40 lines that did one thing: fill auth_token. The current index.js is 327 lines, because the hook turned out to be the natural place to enforce anything that has to be true host-side, keyed by agent identity.

It now also redirects sessions_spawn→sessions_send for cross-agent delegation; enforces a deny-by-default cross-user sessions_send guard (crossUserSendBlock); injects a wb_<prefix> boardId into the five board-scoped workboard tools; ensures a dedicated delegation session per (conversation × specialist); and injects the _conversation_id mentioned above. Each of those is a place where the model shouldn't be trusted to get identity right, and the hook is already sitting on the one chokepoint every tool call passes through — so it kept accreting responsibilities.

That's powerful and it's a smell at the same time. What began as "the auth_token hook" is really "the per-agent host-side identity layer" now, and the name on the tin no longer describes it.

What I'd do differently

Ship identity out of the prompt from day one. Having the model copy its own token "works" on a strong model and fails silently for subagents and weak ones. The 6/6 copy rate lulled us into thinking the design was sound; the host-side hook should have been the first design, not the fix that arrived after "Dalí doesn't answer." If a component has to reproduce a secret to function, that's the bug, no matter how reliable the reproduction looks in a demo.

Decide up front whether the hook is single-purpose or the identity chokepoint. It has since absorbed delegation redirection, a cross-user deny guard, workboard board-scoping, and conversation-id routing. That's a coherent set of responsibilities — but nobody decided it should be one file, it just grew from ~40 to ~330 lines one commit at a time. I'd name and document it as "the per-agent host-side identity layer" from the start, so its scope is a choice rather than an accident.

Finish the cleanup. Two follow-ups are still open in the source doc. The chat and voice bridges still inject the identity token into the prompt as a fallback — redundant now that the hook overwrites it — and auth_token is still a declared field on the tool input schema. Once the hook is proven in production, both should go: drop auth_token from the schema, and slim the now-pointless token out of the bridge prompts. Leaving a fallback in place feels safe, but a fallback the model can still fill is a value the model can still get wrong.

The takeaway

If your model has to copy a secret, it will eventually get it wrong — and the calls you never see, the ones subagents make, will get it wrong 100% of the time. Move identity out of the prompt and into a host-side before_tool_call hook: deterministic, per-agent, each agent keeps its own scopes, and the model can't hallucinate what it never handles.

Next up: #15 — Multi-agent topology: 7 agents per user, delegation, and the failure modes.

The MCP facade: how agents talk to the backend without curl

Victor García — Thu, 23 Jul 2026 06:09:03 +0000

An agent skill runs curl http://127.0.0.1:7200/api/v1/notes from inside its Docker sandbox. It fails with exit code 7 — "couldn't connect" — before authentication even runs, because the sandbox is launched with network: none. There is no loopback to reach. There is no network at all.

Meanwhile, the SKILL.md files were confidently telling the model to authenticate with a $CLAW_API_KEY environment variable that didn't exist inside the sandbox in the first place.

So the picture was: our agents had been instructed to talk to a backend they could never reach, using a token that wasn't there. Every write to a note, every calendar lookup, every "remember this" — all of it was a curl into a wall. This post is about the MCP facade that replaced that model, and the one finding that made the whole design fall out cleanly.

# inside the agent's Docker sandbox (network: none)
$ curl http://127.0.0.1:7200/api/v1/notes
# exit code 7 — couldn't connect, before auth even runs

The fix we almost built

The first instinct was to punch a hole through the wall without taking it down. Keep network: none entirely — the isolation is a feature, not an accident — and hand the sandbox a filesystem socket instead of a network address. Bind-mount a Unix socket into the container and the agent could do curl --unix-socket /run/claw.sock http://core/api/v1/notes: no TCP, no network namespace, just a file descriptor mounted into the container. Call it option C.

Option C was sound. A bind-mounted socket genuinely sidesteps the network: none wall that killed exit-7 curl, because a Unix socket isn't a network — it's a file. It would have worked.

And then we found something that made it irrelevant.

The finding that changed everything

The Model Context Protocol has a client and a server. We had been assuming the MCP client — the thing that actually issues the tool call — would live wherever the agent's code lives: inside the sandbox.

It doesn't. The MCP client runs host-side, in the OpenClaw runtime process, not inside the agent's Docker container. When a model emits a tool call, the runtime — outside the sandbox — is what dispatches it.

That single fact rewrites the problem. A tool call never touches the sandbox's network namespace, because it never originates inside the sandbox. The network: none wall — the exact wall that killed curl with exit 7 — is simply not in the path. There is nothing to tunnel through, no socket to bind-mount, no bridge to build.

Option C became obsolete overnight. It would have solved a real problem — reach Core without a network — but host-side MCP solved the same problem more cleanly, with no second listener to run and secure. We shelved it before writing a line of it.

The shape was now obvious. Core would run its own MCP server — the facade — and expose each domain of the API as a typed tool.

Facade:  core/src/mcp/claw-os-mcp-server.ts
         streamable-HTTP MCP server on 127.0.0.1:7250
         (Core itself is on 127.0.0.1:7200)

Each domain gets one tool named claw_<domain> with an action enum. claw_notes takes action: create | list | get | update | delete. claw_calendar, claw_contacts, claw_finance, and the rest follow the same mould. The model never learns a URL. It learns a vocabulary of typed tools.

A translator, not a second backend

Here was the decision that mattered most, and the one we kept coming back to: the facade is a thin translator, not a reimplementation.

The tempting version of this project is a second backend — a service that knows how to create a note, check a scope, run an approval, sign an audit record. That version is a disaster. It's a duplicate of the real Core that drifts out of sync the first time someone changes a validation rule and forgets the copy.

We refused to write it. Every facade tool call does exactly three things:

Look up the capability registry → (method, url, body, query).
Build an internal HTTP request carrying Authorization: Bearer <auth_token> plus x-claw-capability: <domain>.<action>.
Dispatch that request through the full Fastify pipeline via app.inject(): auth → scope-validation → semantic-scope → approval-check → handler → audit.

app.inject() is Fastify's in-process request injector — it runs a request through every plugin and middleware the real HTTP server runs, without a socket. So a claw_notes call and a human's POST /api/v1/notes walk the same hallway. The facade adds the Bearer prefix, names the capability, and gets out of the way.

// what the agent actually calls (facade tool, host-side on :7250)
claw_notes {
  "action": "create",
  "params": { "title": "...", "content": "...", "tags": ["..."] },
  "auth_token": "<per-conversation session token>"  // bare value, NO "Bearer"
}

// what the facade builds internally and dispatches via app.inject()
Authorization: Bearer <auth_token>   // the facade adds the "Bearer"
x-claw-capability: notes.create
// → auth → scope-validation → semantic-scope → approval-check → handler → audit

Because every call routes through the real pipeline, everything the pipeline already enforces stays enforced, with nothing rewritten:

Preserved for free	Where it lives
Deny-by-default scopes	`scope-validation.ts`
L0–L3 approval levels	`approval-check.ts`
Ed25519 audit signing	audit stage of the pipeline
Tier gating (Free/Pro/Plus)	the handlers
CRUD hooks (embed / heat / extract)	post-write pipeline
Credits accounting	the handlers

The agent's identity travels in the auth_token argument; auth.ts resolves the fixed userId from it, which is what makes impersonation impossible — a model cannot claim to be someone else by editing a field, because the token is the identity.

The capability registry

The (method, url, body, query) lookup in step 1 comes from a declarative registry: one file per domain under core/src/mcp/registry/domains/*.ts. Each Capability maps a (domain, action, params) triple to exactly one internal HTTP request.

// registry entry: (domain, action, params) -> one internal request
// core/src/mcp/registry/domains/notes.ts
{ domain: 'notes', action: 'create', method: 'POST', url: '/notes',
  scope: 'write:notes', approval: 'L0' }

This registry is the single source of truth, and it earns its keep by feeding two consumers at once. The same declarations generate:

The facade's tool catalog (what the model sees as callable tools).
The GET /managed-agents/available-tools endpoint that powers dynamic Tool Access (what an admin toggles per agent in the dashboard).

One file, two surfaces, no chance of them disagreeing. Add a capability and it shows up in both the model's vocabulary and the permission UI in the same commit.

Security had to come first

There's an uncomfortable truth buried in this whole design. For as long as the agents lived behind network: none, a lot of latent vulnerabilities didn't matter — they were contained by a wall that happened to block everything. The moment we connected agents host-side, that wall came down, and those vulnerabilities became live.

So the actual first phase of this project (Fase 0) wasn't the facade at all. It was closing three holes before exposing anything:

Guard	What it closed
C1	`POST/PATCH /agent-tokens` now returns `403` when `keyScope === 'agent'` — an agent can't mint or widen its own token (no self-escalation).
C2	The same `403` on `requireAdmin` routes — an agent token can never reach an admin surface.
C3	`scope-validation` flipped to deny-by-default — an unmapped resource is denied, not allowed.

C3 is the one that changes the posture of the whole system. Before, "we didn't map this resource" meant "anything goes." After, it means "no." Sensitive domains had to be re-exposed deliberately and carefully on top of that floor — the kind of thing that, done in the wrong order, quietly ships a hole.

Identity: the model stopped copying the token

The first version of the auth model (ADR-10) made the model responsible for copying its own auth_token — a 64-hex secret — into every single claw_* call. On a capable model, on rails, this works. But it's fragile by construction: the moment the model improvises a wrong tool call, it also mangles or hallucinates the token, and worse, subagents got no token at all. A child spawned via sessions_spawn doesn't pass through the chat bridge that injected the token, so delegated work silently failed. This is the real reason behind the months-old complaint that "Dali doesn't answer" when you delegate to her.

ADR-11 fixed it by taking the model out of the loop entirely. A native OpenClaw plugin, openclaw-plugins/claw-identity, registers a before_tool_call hook. For any tool named claw-os__*, the hook overwrites params.auth_token with the correct per-agent token, resolved host-side from ctx.agentId against a token map at ~/.config/micelclaw/agent-tokens.json (chmod 600). The model's value, if it bothered to supply one, is ignored.

Deterministic, and — crucially — it covers subagents. Any agent, main or spawned, gets its own token injected host-side. It can't hallucinate a token it never has to write. (That hook is a whole story of its own; it's next week's post.)

Skills stopped being API docs

Once the mechanics — URL, headers, method, request shape — moved into the typed tool and its schema, the SKILL.md files had nothing left to document. So they became something better.

Each skill shrank from a full API reference to a ~25–45 line judgment guide: not how to call the endpoint (the schema captures that) but when and how — the taste and context a schema can't encode. "Prefer list with a tag filter before get." "Don't create a duplicate contact; search first." The skill teaches the decision, the tool carries the call.

Auditing the real surface

One rule we set early: never mirror the capability registry to describe the API. Audit the real REST surface, because the registry is a deliberate subset of it, and pretending otherwise hides what's exposed.

So on 2026-07-03 we cross-audited both:

Surface	Count
REST endpoints in `routes/`	1456
Capabilities across code domains	424
Code domains	31
REST surface deliberately not exposed to agents	~60%

That ~60% is intentional, not a backlog. It's admin, config, binary, and UI-only endpoints that agents have no business touching. The audit's value is precisely in making the gap explicit — you can only reason about what an agent can do if you've measured what the surface actually is, endpoint by endpoint, rather than trusting the shorter list to speak for the longer one.

The gotchas that cost us

Wiring typed tools into a model turned out to have sharp edges that had nothing to do with the design and everything to do with the plumbing.

The MCP SDK silently strips undeclared args. Any argument not declared in a tool's inputSchema is dropped before it reaches the facade. Since the identity hook injects auth_token (and _conversation_id) host-side, those fields must still be declared in the schema — otherwise the SDK strips the hook's injected value and it dies silently on the way in. The thing you inject must be a thing you declared.

A z.record schema sent a weak model into an 84-call loop. Our first cut emitted the tool's params as z.record → JSON Schema properties: {} — no declared fields. A small local model (Francis, a 35B-A3B at IQ2_M) was asked for "contacts starting with a" and made 84 calls with params: {}, completely unable to construct params: { search: "a" } because the schema gave it nothing to fill in. We fixed it in tool-builder.ts by generating params as the union of every action's .shape, .partial().passthrough() — real, named fields the model can reason about.

One .email() took down every Groq chat. zod v4's .email() emits a regex with lookaheads. Groq compiles every tool schema with an RE2 engine, and RE2 has no lookarounds — so it returned a 400 and rejected the entire chat, for every agent that happened to carry the files domain. One field, one lookahead, whole conversation dead. Fixed with a generic lookaround-stripping sanitizer in tool-builder.ts (the real validation still runs against the original zod, so nothing loosens).

And a preset gotcha: bundle-mcp — the tool key that hands the agent the facade at all — must be in tools.sandbox.tools.alsoAllow, or the sandbox prunes it and the agent sees none of the facade tools. It's in the coding, messaging, and full presets for exactly this reason.

For completeness, a handful of skills were not migrated and still curl, on purpose: claw-pdf-tools (binary endpoints), claw-bind (lives under /api/v1/auth, uses a JWT/system token), and the four claw-app-* meta-skills (app authoring, an admin flow).

What I'd do differently

I'd ship the Fase 0 hardening as the foundation, before wiring any agent host-side. Connecting agents lit up vulnerabilities that network: none had been quietly containing. Deny-by-default and the C1/C2/C3 guards should have been the first commit, not a prerequisite we scrambled to close. Ordering security after exposure is how you ship a window.

I'd inject identity host-side from day one (ADR-11), not make the model copy a 64-hex token (ADR-10). The model-copies-the-token design was fragile against weak models and broke subagents entirely — sessions_spawn children got no token, which is the whole reason delegated work silently failed. A before_tool_call hook is both more robust and simpler. We took the long way there.

I'd declare the params schema shape up front. The empty-properties z.record schema sent a small model into a loop it could not escape — 84 calls, all params: {}. It read like a model failure and was a schema-shape bug. Give the model named fields and it fills them.

I'd sanitize tool schemas for RE2 from the start. A single .email() in one domain 400'd every Groq chat for every agent carrying that domain. A one-line validator emitting a lookahead took down whole conversations. Cheap to prevent, expensive to diagnose.

The takeaway

The agent never learned an API. It learned a vocabulary of typed tools — and every call still walks the same auth → scope → approval → audit hallway a human's request does. That's the entire trick: the facade is a translator, not a second backend. It reuses the whole pipeline via app.inject() and rewrites nothing. The AI can only touch what you allow, and everything it touches passes the exact permission checks and audit log as if you'd done it yourself — because, mechanically, it did.

Next up: #14 — Deterministic agent identity: injecting the per-agent token with a before_tool_call hook

Building a process manager for 40 services on 8GB of RAM

Victor García — Wed, 22 Jul 2026 07:56:55 +0000

At some point during development, I ran docker ps and counted 16 containers. Then there were the systemd services — PostgreSQL, SSH, the core server itself. Plus internal workers: the job scheduler, the async queue, the WebSocket server, the OpenClaw gateway. And Ollama, sitting quietly with a 4B model eating 5.4GB of VRAM.

40 processes. On a machine with 8GB of RAM.

Not all of them need to run simultaneously. The mail server needs to be always on. The PDF converter only needs to run when someone opens a document. Jellyfin only needs to run in the evenings. The voice STT model can sleep unless someone is talking.

This post is about the process manager that orchestrates all of that — a unified view of everything running on the system, a RAM budget engine that decides what fits, and lifecycle policies that start and stop services based on actual usage.

The unified view

The first problem was visibility. Docker containers report stats through the Docker API. Systemd services report through systemctl. Internal Node.js workers (scheduler, queue, WebSocket) only exist in the Claw Core process. Ollama has its own API for model management. Four different systems, four different interfaces.

We unified them into a single table with a consistent model:

interface ClawProcess {
  id: string;           // 'docker:claw-whisper', 'systemd:postgresql', 'internal:scheduler'
  name: string;         // 'Speech-to-Text (Whisper)', 'PostgreSQL', 'Job Scheduler'
  type: 'docker' | 'systemd' | 'internal';
  status: 'running' | 'stopped' | 'failed' | 'starting';
  cpu_percent: number | null;
  memory_bytes: number | null;
  gpu_percent: number | null;
  vram_bytes: number | null;
  uptime_seconds: number | null;
  restartable: boolean;
  stoppable: boolean;
  has_logs: boolean;
}

The id format type:name makes every process globally addressable. docker:claw-mailu-imap is the IMAP container. systemd:postgresql is the database. internal:async-queue is the background job processor. One namespace, one table, one API.

Three collectors feed the table:

Collector	Source	Method
Docker	Docker Engine API	`GET /containers/json` via Unix socket
systemd	systemctl	`list-units --type=service --output=json`
Internal	Claw Core process	In-memory state from scheduler, queue, WS server

Plus a special case: Ollama. It's a systemd service, but it also has loaded models that consume VRAM independently. The process manager queries Ollama's API (GET /api/ps) to show which models are loaded, their parameter count, and VRAM usage. You can unload a model directly from the process table to free VRAM.

The summary bar

The top of the process manager shows system-wide stats at a glance:

PROCESSES 16/40    CPU 0.5%    MEMORY 3205 MB    GPU 4%    VRAM 5579 / 16311 MB

16 out of 40 registered processes are currently running. Total CPU across all processes. Total memory. GPU utilization and VRAM usage (critical when you're running AI models locally).

The VRAM indicator is particularly useful. When Ollama loads qwen3.5:4b, it consumes 5.4GB of VRAM. If someone tries to load a second model and there isn't enough VRAM, the process manager shows exactly why. No more mysterious "out of memory" errors — the dashboard tells you what's using what.

Lifecycle policies: not everything needs to be always-on

The real power isn't the table view — it's deciding what runs and when. Each managed service has a lifecycle policy:

Policy	When it runs	Example services
Always-on	24/7, never auto-stopped	PostgreSQL, Claw Core, mail server, WireGuard, Tailscale
On-demand	Starts when the user navigates to the module, stops after idle timeout	ONLYOFFICE, Stirling PDF, Portainer, SolidInvoice
Scheduled	Runs during configured time windows or intervals	Jellyfin (08:00-01:00), RSS fetcher (every 60 min)
Triggered	Starts on a specific event, stops when the task completes	yt-dlp (download requested), claw-vision (photo uploaded)

In a real deployment, the breakdown looks like this: 5 always-on services, 22 on-demand, 8 scheduled. The 22 on-demand services are the key — they represent ~70% of total potential memory usage, but most of the time only 2-3 of them are actually running.

When you open the Office module in the dashboard, the system starts ONLYOFFICE (if it's not already running). The loading state shows a progress spinner. After 30 minutes of no document activity, ONLYOFFICE stops automatically. You never think about it.

Each service's policy is configurable in Settings → Services. Click on a service to expand its configuration: policy selector (always/scheduled/on-demand/disabled), schedule windows, timeout settings, RAM limits. Presets cover common patterns ("Diurnal" for services that run during working hours, "Periodic fetch" for services that sync on intervals).

The RAM budget engine

Here's the core problem: 40 services can't all run on 8GB of RAM. Something has to decide.

The RAM budget engine works with hardware profiles:

Profile	RAM	Budget for services	Max concurrent on-demand
Lite	≤ 6 GB	~1.5 GB	2
Standard	8-14 GB	~3.3 GB	4
Performance	16+ GB	~8 GB	8

The profile is auto-detected from system RAM and determines the budget — how much memory is available for managed Docker services (excluding always-on core services like PostgreSQL and Claw Core, which are non-negotiable).

Before starting any on-demand service, the engine checks:

Can this service fit in the remaining budget?
├── Yes → Start it
└── No → Is there an idle on-demand service we can evict?
    ├── Yes → Stop the longest-idle service, then start the new one
    └── No → Tell the user: "Not enough memory. Stop X to start Y."

Eviction targets the longest-idle on-demand service. Always-on services are never evicted — the system won't kill your mail server to make room for a PDF converter. The user always has the final say: the engine suggests what to evict, but it doesn't force it.

Under memory pressure (free RAM drops below 10%), the engine gets more aggressive: it progressively stops idle on-demand services until pressure is relieved, notifying the user via WebSocket for each one.

The RAM budget view in Settings shows this live:

RAM Budget: 260 / 3335 MB (8%)
├── wg-easy: 11 MB
├── mailu: 57 MB
├── wyoming-whisper: 177 MB
└── wyoming-piper: 15 MB

260MB used out of a 3,335MB budget. Room for several more on-demand services before the budget gets tight.

Drain guards: stopping safely

You can't just docker stop a service and hope for the best. ONLYOFFICE might have a document open. Qbittorrent might be mid-download. The terminal might have an active SSH session.

Drain guards check safety before stopping:

Service	Guard	What it checks
ONLYOFFICE	Document sessions	Are there open documents? → Auto-save first
Qbittorrent	Active downloads	Is anything downloading? → Wait or warn
Terminal	SSH sessions	Are there active sessions? → Warn user
Jellyfin	Active streams	Is anyone watching? → Warn user

If a drain guard blocks a stop operation, the user gets a notification: "ONLYOFFICE has 2 documents open. Save and close them, or force stop." Force stop is always available — but the system tries to prevent data loss by default.

Ollama: the special case

Ollama deserves its own section because it's neither a simple Docker container nor a systemd service — it's both. The systemd service runs the Ollama daemon. The models loaded into it consume VRAM independently of the daemon's RAM usage.

The process manager handles Ollama differently:

Shows the Ollama daemon as a systemd process with its own CPU/RAM stats
Below it, shows loaded models in an expandable section: model name, parameter count, RAM usage, VRAM usage
Each model has an "Unload" button that calls DELETE /api/generate to free VRAM
The summary bar includes VRAM as a separate metric from RAM

In our current setup, qwen3.5:4b sits loaded with 5.43GB of VRAM. When we need to load a different model (say, for a vision task), the process manager shows the VRAM impact before loading. If VRAM is full, you can unload the current model first.

The Ollama client in Claw Core (the priority queue singleton from post 2) manages model loading automatically — it loads models on demand and unloads them based on idle time. The process manager gives you visibility into what it's doing and manual override when you need it.

The log panel

Every process with has_logs: true gets a log viewer. Click the logs icon on any process row and a side panel opens with the last 200 lines, auto-scrolling, with a filter input for grep-style searching.

Docker logs come from the Docker API (GET /containers/{id}/logs). Systemd logs come from journalctl -u {service}. Internal workers log to pino (the Fastify logger) and their output is captured from the running process.

The filter is surprisingly useful for debugging. When Mailu's IMAP container is acting up, filtering for "error" or "failed" in the log panel is faster than SSH-ing into the server and running docker logs | grep.

Approval integration

Process lifecycle operations integrate with the approval system (post 11):

Operation	Approval level
View processes, logs	Level 0 (auto)
Start a service	Level 0 (auto)
Restart a service	Level 2 (confirm)
Stop a service	Level 2 (confirm)

Starting is unrestricted because the RAM budget engine already gates it — if there's no budget, it won't start. Restart and stop require confirmation because they affect running services. The agent can restart a crashed container, but only after asking "claw-mailu-imap is failing, should I restart it?"

What the numbers look like in practice

On our development machine (8GB RAM, RTX 2060):

Category	Count	Typical RAM
Always-on (core)	5	~400 MB
Always-on (mail)	6	~400 MB
Always-on (network)	3	~50 MB
Always-on (AI)	3	~200 MB
On-demand (active)	2-3	~300-500 MB
Systemd	3	~100 MB
Internal workers	5	~50 MB

Total running at any given time: ~15-20 processes, ~1.5-2 GB. Well within the Standard profile's 3.3GB budget. The other 20+ on-demand services wait in Docker's stopped state, consuming zero RAM.

Peak usage (everything on-demand activated simultaneously): ~4.5 GB. This never actually happens — it would require opening every module in the dashboard at the same time. The lifecycle policies ensure only what's needed is running.

What I'd do differently

I'd build the RAM budget engine before adding any managed services. We added services first (mail, PDF, office, media) and the budget engine later. During the gap, the machine would occasionally OOM because too many containers were running. The budget engine should have been the foundation, not an afterthought.

I'd add a "service groups" concept. Right now, Mailu is 6 separate containers (IMAP, SMTP, antispam, admin, Redis, frontend). They start and stop together, but the process table shows them individually. A group like "Mail Server" that expands to show its 6 components would reduce visual noise from 40 rows to ~20.

I'd expose the process table via the agent. Currently, the agent can restart and stop services via the HAL API, but it can't see the process table the way the dashboard shows it. A claw-processes skill with ?format=compact would let the agent answer "what's using all my RAM?" with actual data instead of guessing.

The takeaway

A self-hosted personal OS isn't one service — it's 40 services pretending to be one. The user should never think about which containers are running, how much RAM they're using, or when to start and stop them. That's the process manager's job.

Three components make it work: a unified process table (Docker + systemd + internal + Ollama, one namespace), a RAM budget engine (hardware profiles, eviction logic, pressure detection), and lifecycle policies (always-on, on-demand, scheduled, triggered).

The result: 40 services on 8GB of RAM, with the user seeing a single coherent system. The process manager is the plumbing that makes "personal OS" feel like an OS instead of a collection of Docker containers.

Next up: InsightFace face recognition on a mini-PC — how we built a face detection pipeline that clusters people across your photo library using a 2B model and a Python sidecar, all running on CPU.

Designing an AI approval system: when should your agent ask for permission?

Victor García — Tue, 07 Apr 2026 10:38:11 +0000

An AI agent that can only read data is safe but useless. An AI agent that can send emails, delete files, format disks, and configure VPNs is useful but terrifying. The entire value of a personal AI operating system comes from the agent acting on your behalf — and the entire risk comes from the same thing.

We needed a system that says "yes" fast to everyday operations and "are you sure?" to dangerous ones. Not a blanket confirmation on everything (that just trains the user to click "approve" without reading). Not unrestricted access either (one prompt injection away from rm -rf /).

This post is about the 4-level approval system we built, the dual-layer architecture (shell + API), and the surprisingly difficult design decision of where to draw the line between "just do it" and "ask me first."

The two attack surfaces

An AI agent in our system can cause damage in two completely different ways:

Shell execution. The agent uses the runtime's exec tool to run commands on the host machine. This is raw power — curl, ls, grep, but also rm, dd, python3 -c 'import os; os.system("...")'. The attack surface is the entire operating system.

API operations. The agent calls our REST API via curl. The curl command itself is harmless — it's the endpoint that's dangerous. POST /storage/pools creates a RAID array. DELETE /files/:id removes a file. POST /emails/send sends an email you can't unsend. The business logic is the attack surface.

These need separate control mechanisms because they have different risk profiles and different mitigation strategies.

Layer 1: Shell control

By default, the agent can only execute a small set of safe binaries:

Safe bins: curl, jq, cat, echo, date, wc, head, tail, grep

That's it. The agent talks to the outside world through curl to our REST API. Everything else — file manipulation, package installation, network commands, scripting — is blocked at the runtime level.

There's an "Unrestricted Shell Mode" toggle in Settings → Security. It's deliberately scary: the toggle is marked in red, requires the user's password (not just a click), and shows a warning explaining that this allows the agent to execute any command on the system.

Even in unrestricted mode, destructive commands (rm, dd, mkfs, fdisk) always require per-operation confirmation from the user. Full freedom doesn't mean no guardrails — it means the agent can attempt anything, but the user decides on dangerous operations.

The key design principle: most users never enable unrestricted mode. The agent does everything it needs through the API. Shell access is a power-user feature for people who know what they're doing.

Layer 2: Operation approvals

This is where it gets interesting. Every API operation has an approval level:

Level	Name	What happens
0	Auto	Execute immediately, no record. Reads, searches, listings.
1	Logged	Execute immediately, log to audit trail. Creates, updates, non-destructive writes.
2	Confirm	Pause and ask the user "Are you sure?" before executing. Sends, deletes, infrastructure changes.
3	Secure	Pause, ask for confirmation AND a numeric PIN. Format disk, delete volume, system reset.

The approval level is checked by a Fastify middleware (preHandler) that runs before the route handler. If the request comes from an agent token and the operation requires Level 2+, the middleware returns a 202 Accepted with an approval_id instead of executing the operation. The agent then asks the user for confirmation through whatever channel they're chatting on — Telegram, WhatsApp, or the web dashboard.

How it looks in practice

Level 0 — Auto (reading notes):

User: "What notes do I have about the project?"
Agent: GET /notes?search=project&format=compact
→ Executes immediately. User sees results.

No friction. Reads are always auto.

Level 1 — Logged (creating a note):

User: "Save a note about today's meeting decisions"
Agent: POST /notes {title: "Meeting decisions", content: "..."}
→ Executes immediately. Logged to audit trail.

Still no friction — the user asked for it. But the audit trail records that agent francis created note abc123 at 14:32. If something goes wrong, there's accountability.

Level 2 — Confirm (sending an email):

User: "Send Ana the budget update"
Agent: POST /emails/send {to: "ana@techcorp.com", subject: "Budget Q3", ...}
→ 202 Accepted {approval_id: "req_xyz"}

Agent: "I'm about to send this email to Ana García:
        Subject: Budget Q3
        [Preview of the body]
        Should I send it?"

User: "Yes"
Agent: POST /approvals/req_xyz/approve
→ Email sent.

One confirmation step. The agent shows what it's about to do. The user says yes or no. This catches the common case where the agent misunderstood the intent — "I said Ana, not María" — without making every email a five-step process.

Level 3 — Secure (deleting a volume):

User: "Delete the old backup volume"
Agent: POST /storage/volumes/vol_old/delete
→ 202 Accepted {approval_id: "req_abc", level: 3}

Agent: "⚠️ This will permanently delete volume vol_old (2.3TB).
        This cannot be undone.
        Please confirm with your security PIN."

User: "4829"
Agent: POST /approvals/req_abc/approve {pin: "4829"}
→ Volume deleted.

Two-factor: the user confirms AND enters their PIN. The PIN is a 4-6 digit numeric code set during initial setup, stored hashed with bcrypt. In messaging channels, the PIN message is deleted from chat history after verification (when the channel supports it).

The approval lifecycle

Every Level 2+ operation creates an approval_request record:

CREATE TABLE approval_requests (
    id              UUID PRIMARY KEY,
    user_id         UUID NOT NULL,
    requested_by    VARCHAR(100) NOT NULL,    -- 'agent:francis'
    operation       VARCHAR(200) NOT NULL,    -- 'POST /emails/send'
    level           SMALLINT NOT NULL,        -- 2 or 3
    summary         TEXT NOT NULL,            -- "Send email to Ana: Budget Q3"
    params          JSONB,                    -- Request body snapshot
    status          VARCHAR(20) DEFAULT 'pending',
    resolved_at     TIMESTAMPTZ,
    pin_verified    BOOLEAN DEFAULT false,
    expires_at      TIMESTAMPTZ NOT NULL,
    created_at      TIMESTAMPTZ NOT NULL
);

The lifecycle is:

Agent triggers operation
    ↓
Middleware: level >= 2? → Create approval_request (status: pending)
    ↓
Notify user via WebSocket (Dash) + Gateway RPC (Telegram/WhatsApp)
    ↓
User approves, rejects, or ignores
    ↓
├── Approved → Execute operation, status: approved
├── Rejected → Return error to agent, status: rejected
└── Timeout (30min) → Auto-expire, status: expired

Three timeout stages prevent approvals from hanging forever:

Stage	Default	What happens
Reminder	5 minutes	Send a reminder to the user
Escalation	15 minutes	Notify the system owner (if different from the user)
Expiry	30 minutes	Auto-reject the request

All three are configurable in Settings → Security → Approval Timeouts.

What's configurable and what isn't

The default levels are sensible but not everyone agrees on what's "dangerous." A power user who sends 50 emails a day wants email sending at Level 1 (logged, no confirmation). A cautious user wants it at Level 2 (confirm every one).

Settings → Security shows a table of operations with dropdown selectors:

Operation                    Default    Your Level
─────────────────────────────────────────────────
Create note                  Logged     [1 - Logged ▼]
Send email                   Confirm    [2 - Confirm ▼]
Delete files (permanent)     Confirm    [2 - Confirm ▼]
Enable VPN                   Confirm    [2 - Confirm ▼]
Add VPN peer                 Confirm    [2 - Confirm ▼]
Delete volume                Secure     [3 - Secure  ▼]
Format disk                  Secure     [3 - Secure  ▼]
System reset                 Secure     [3 - Secure  ▼]

Two constraints prevent dangerous configurations:

Level 3 operations can't go below Level 2. You can downgrade "Delete volume" from Secure (3) to Confirm (2), but not to Logged (1) or Auto (0). Destructive, irreversible operations always require at least one confirmation.
Level 0 operations can't be upgraded. Read operations are always auto. Making GET /notes require confirmation would break the system — the agent would need approval to answer "what notes do I have?"

Changing approval levels is itself a Level 2 operation — the system asks for confirmation before letting you change the security settings.

Why agents can't approve their own requests

This sounds obvious but it's the most important security decision in the system: an agent API key cannot approve an approval request. Only JWT tokens (human login via Dash) or system tokens can call POST /approvals/:id/approve.

If the agent could approve its own requests, a prompt injection attack could chain: trigger the operation → intercept the approval request → approve it. The human-in-the-loop is only meaningful if the human is the one doing the approving.

In messaging channels (Telegram, WhatsApp), the approval flows through the agent — the user says "yes" in the chat, and the agent calls the approve endpoint. But the approve endpoint verifies that the approval was triggered by a user message, not by the agent itself. The requested_by field records which agent requested the operation, and the same agent cannot resolve it.

The middleware: 15 lines that matter

The approval check is a Fastify preHandler that runs on every route with an assigned level:

// Simplified — the real version handles edge cases
async function approvalMiddleware(req, reply) {
  // Skip for human (JWT) requests — the Dash IS the confirmation
  if (req.authType === 'jwt') return;

  const level = getOperationLevel(req.method, req.url, req.userId);

  if (level <= 1) return; // Auto or Logged — proceed

  // Level 2 or 3: create approval request and pause
  const approval = await createApprovalRequest({
    userId: req.userId,
    requestedBy: `agent:${req.apiKeyName}`,
    operation: `${req.method} ${req.url}`,
    level,
    summary: buildSummary(req),
    params: req.body,
  });

  // Notify user
  await notifyApproval(approval);

  // Return 202 — the agent knows to ask the user
  reply.code(202).send({
    data: { approval_id: approval.id, level, summary: approval.summary },
    hint: level === 3 
      ? "Ask user to confirm with PIN" 
      : "Ask user to confirm"
  });
}

The hint field in the 202 response tells the agent's skill what kind of confirmation to request. Level 2: ask for a yes/no. Level 3: ask for the PIN.

Human requests from the Dash skip the middleware entirely. When you click "Send" in the email composer, you ARE the confirmation. The approval system only gates agent-initiated operations.

The skill: teaching the agent to ask

The claw-approvals skill teaches the agent how to handle the approval flow:

## Approval Protocol

When you receive a 202 response with an approval_id:
1. Show the user what you're about to do (use the summary field)
2. For Level 2: ask "Should I proceed?"
3. For Level 3: ask "Please confirm with your security PIN"
4. On "yes" or PIN: POST /approvals/{id}/approve (with pin if Level 3)
5. On "no" or "cancel": POST /approvals/{id}/reject
6. Never approve your own requests — always wait for user input

The skill also handles /pending (show pending approvals), /history (show past approvals), and edge cases like expired requests.

What I learned about where to draw the line

The hardest part wasn't building the system — it was deciding which operations go at which level. Some were obvious:

Read anything → Level 0. No debate.
Format disk → Level 3. No debate.

The middle ground is where every conversation happened:

Email sending: Level 1 or Level 2? We went with Level 2 (Confirm) as default. Email is irreversible — you can't unsend it. A misunderstood intent ("send Ana the budget" when you meant "draft Ana the budget") has real consequences. But we made it configurable because power users find the confirmation annoying.

Creating a note: Level 0 or Level 1? We went with Level 1 (Logged). Creating a note is harmless — but logging it means the audit trail shows everything the agent did. If the agent starts creating garbage notes due to a bug, the log tells you when it started. Zero friction, full accountability.

VPN operations: Level 2. Enabling or disabling VPN changes network topology. Adding a peer grants network access to a device. These aren't destructive (you can undo them) but they affect security posture. Confirmation is appropriate.

Deleting files: Level 2, not Level 3. Files have soft delete with restore capability. "Delete" doesn't mean "gone forever" — it means "moved to trash." If deletion were permanent (bypassing soft delete), it would be Level 3.

The general heuristic: reversible actions need less approval than irreversible ones. Creating is Level 1 (you can delete it). Sending is Level 2 (you can't unsend it). Destroying is Level 3 (it's gone forever).

The dual-layer interaction

The two layers (shell + API) work together:

User: "Check my disk space"

Agent: exec "df -h"
→ Layer 1: "df" is in safe bins → Execute immediately

Agent: GET /storage/overview?format=compact
→ Layer 2: Level 0 (read) → Execute immediately

Agent: "You have 450GB free on /data. The backup volume is 78% full."

No approvals needed — both layers allowed the operations silently.

User: "Set up a RAID1 mirror with the two new drives"

Agent: POST /storage/pools {name: "mirror1", level: "raid1", devices: ["sda", "sdb"]}
→ Layer 2: Level 2 (Confirm) → 202 Accepted

Agent: "I'm about to create a RAID1 pool using sda and sdb. 
        This will erase all data on both drives. Confirm?"

User: "Yes"
→ Pool created.

And in unrestricted shell mode:

User: "Install htop on the server"

Agent: exec "apt-get install htop"
→ Layer 1: Not in safe bins + unrestricted mode → Ask for confirmation

Agent: "I want to run: apt-get install htop
        This will install a package on your system. OK?"

User: "Go ahead"
→ Layer 1: Approved → Execute

Both layers enforce independently. An operation that passes Layer 1 (shell) can still be blocked by Layer 2 (API). An operation that bypasses Layer 2 (because it's a direct shell command) is still caught by Layer 1.

What I'd do differently

I'd add batch approvals from the start. When the agent needs to send 15 emails from a mail merge, asking for 15 individual confirmations is absurd. A "batch approve" mechanism ("Send these 15 emails? Here's the list.") should have been in v1. It's now on the backlog.

I'd make the approval history more visible. The audit trail exists (GET /approvals/history), and there's an "Approvals History" section in the sidebar, but it should be more prominent. A weekly summary — "Your agent executed 340 operations this week: 280 auto, 55 logged, 5 confirmed" — would build trust and help users understand their agent's activity.

I'd reconsider the messaging channel PIN flow. Sending a PIN via Telegram is not ideal — it's visible in chat history even if the agent tries to delete the message. For Level 3 operations, maybe the system should redirect to the Dash where a secure input modal exists, rather than accepting PINs in plain text through messaging.

What's next: the sandbox

There's a missing piece between "restricted mode" (curl only) and "unrestricted mode" (everything, with confirmations). What if the agent could have a place to go completely wild — install packages, run services, break things — without any risk to your real system?

We're planning sandbox environments: Docker containers that the agent can create from Settings → Security. Not one — as many as you need. Each sandbox is an isolated machine with full root access. apt install, pip install, systemctl, custom scripts, databases, web servers — anything goes. Zero approval, zero restrictions, zero risk to the host.

The workflow we're designing around:

Sandbox "deploy-test"     → experimenting with nginx + certbot config
Sandbox "ml-pipeline"     → building a data processing pipeline with pandas
Sandbox "new-skill"       → developing and testing a new agent skill

Each one: independent, disposable, unrestricted.

The agent knows which sandbox it's working in. Commands routed to a sandbox go to that container. Commands on the real system go through normal approval layers. The two worlds don't touch — no shared volumes, no network bridge to internal services, no mount to /data.

The key is the promote-to-production flow. Once you've got something working in the sandbox — a configuration, a script, a service setup — you tell the agent "promote this to production." At that point, and only at that point, normal approval rules kick in. The agent needs Level 2 confirmation to copy files to the host, Level 2 to install a package on the real system, Level 3 to modify infrastructure. The sandbox is the drafting table; production is the real thing.

If a sandbox gets trashed, nuke it and spawn a fresh one in seconds. The sandboxes are cheap — a base Debian image with internet access and a persistent volume for the workspace. Multiple sandboxes can run simultaneously for different experiments without interfering with each other or with the host.

This isn't implemented yet, and there are open design questions: should sandboxes have read-only access to the real API (for testing skills against real data)? Should there be resource limits per sandbox (CPU, RAM, disk)? What's the UX for promoting — file-by-file or snapshot the whole container? We'd love input from anyone who's built agent sandboxing — this is genuinely uncharted territory for personal AI systems.

The takeaway

An AI approval system needs two properties: it must be fast for safe operations (no friction on reads, minimal friction on writes) and deliberate for dangerous ones (explicit confirmation, PIN for irreversible actions, timeout for stale requests).

Four levels handle this: auto (reads), logged (writes), confirm (irreversible), secure (destructive + PIN). Two layers: shell control for host-level commands, API control for business operations. One principle: agents cannot approve their own requests.

The system processes ~95% of operations at Level 0 or 1 — invisible to the user. The 5% that require confirmation are the operations where a mistake actually matters: sending an email to the wrong person, deleting a volume, configuring network access. Those 5% are where trust is built or broken.

Next up: building a process manager — how we manage Docker containers, systemd services, and Ollama models from a single dashboard with auto-start, health monitoring, and graceful shutdown.

PII-aware routing: how to use cloud AI and keep your sensitive data local

Victor García — Fri, 27 Mar 2026 14:23:21 +0000

Here's the tension at the heart of every personal AI system: cloud models are better at reasoning, but your data is private. A self-hosted system can run everything locally — but a 2B parameter model on a mini-PC isn't going to draft a nuanced email response or analyze a complex financial situation the way a frontier model can.

The naive solutions are both bad. "Send everything to the cloud" means your diary entries, medical notes, and financial records pass through someone else's servers. "Run everything locally" means accepting worse reasoning on tasks where model quality actually matters.

We built a third option: a PII-aware routing layer that classifies every piece of data by sensitivity, routes it to the right model, and pseudonymizes anything sensitive that needs cloud reasoning power.

The classification: four levels, zero LLM calls

Every record in the system gets a sensitivity level. The classification is entirely deterministic — regex patterns and domain rules. No LLM in the classification loop, because sending data to an LLM to decide if the data is too sensitive to send to an LLM is a circular problem.

Level	What it means	Example domains
`low`	Public or low-risk data	Events, bookmarks
`normal`	Common personal data	Notes, contacts, files, diary
`high`	Sensitive personal data	Emails, financial transactions
`critical`	Never leaves the device	Medical/health data

Each domain has a default sensitivity level. Events are low — knowing you have a meeting at 3pm isn't particularly sensitive. Emails are high — they contain names, addresses, business context, and sometimes confidential information. Health entries are critical — always local, no exceptions.

But domains are just the baseline. The classifier also scans content for PII patterns that override the default:

Email addresses     → elevate to high minimum
Phone numbers       → elevate to high minimum
Credit card numbers → elevate to high minimum
IBAN codes          → elevate to high minimum
SSN / DNI / NIE     → elevate to high minimum
Medical terminology → elevate to critical

A note titled "Grocery list" stays at normal. A note containing "Dr. García prescribed 20mg omeprazole" gets elevated to critical because the regex matched medical terminology. The content drives the classification, not just the domain.

This is deliberately conservative. The regex patterns over-match — "Dr." triggers medical detection even if it's "Dr. Pepper." False positives mean data gets routed locally when it could have gone to the cloud. False negatives mean sensitive data leaks. Over-matching is the correct failure mode.

The routing decision

Once classified, the router decides where each piece of data goes:

low / normal  → Cloud LLM — best reasoning
high          → Cloud LLM WITH pseudonymization — good reasoning, protected data
critical      → Local model only (Ollama) — or skip if Ollama unavailable

The decision isn't binary "local vs cloud." There's a middle path: pseudonymize the sensitive parts, send to the cloud for reasoning, and de-pseudonymize the response before the user sees it.

This matters because most tasks involving sensitive data don't need the sensitive parts for reasoning. "Summarize this email thread" needs the content structure and topic — not the actual names and email addresses. "What's the sentiment of this diary entry?" needs the emotional content — not the specific people mentioned.

The pseudonymizer

When a high sensitivity record needs cloud processing, the pseudonymizer replaces PII with consistent tokens:

Entity type	Pseudonym format	Example
Person	`Person_XXXX`	"Ana García" → `Person_A3F2`
Email	`email_XXXX@example.com`	"ana@techcorp.com" → `email_7B1C@example.com`
Phone	`+00-XXXX-0000`	"+34 612 345 678" → `+00-E5D9-0000`
Organization	`Org_XXXX`	"TechCorp" → `Org_4C8A`
Location	`Location_XXXX`	"Calle Sagasta 15" → `Location_B2E1`

Three properties make this work:

Consistency. The same value always produces the same pseudonym (SHA-256 of the original value, truncated). "Ana García" is always Person_A3F2, in every record, in every session. This means the cloud model can reason about relationships: "Person_A3F2 sent 3 emails to Person_B7D1 about Org_4C8A" preserves the structure even though the names are hidden.

Reversibility. The pseudonym_map table stores every mapping. When the cloud model's response comes back, the system replaces all pseudonyms with real values before storing or displaying the result. The user never sees Person_A3F2 — they see "Ana García."

Persistence. Mappings survive across sessions. If "Ana García" was pseudonymized yesterday and appears again today, she gets the same pseudonym. This means the cloud model can build consistent context across multiple interactions without ever learning the real name.

The detection itself uses regex — no LLM call. It's the same NER-lite approach as the sensitivity classifier: pattern matching for emails, phones, card numbers, and named entity patterns for persons and organizations. Not perfect, but fast and deterministic.

What this looks like in practice

Scenario 1: Calendar event (low sensitivity)

User asks: "What's on my calendar tomorrow?"

The system fetches tomorrow's events. Events are low sensitivity. The full data — titles, locations, attendees — goes straight to the cloud model. No pseudonymization needed. The model reasons about the schedule and responds with a natural summary.

Cost: one cloud API call. Privacy: no sensitive data exposed.

Scenario 2: Email analysis (high sensitivity)

User asks: "Summarize the email thread about the partnership."

The email thread is high sensitivity (default for emails). Before sending to the cloud model:

Original: "Ana García <ana@techcorp.com> wrote: Hi Paco, 
regarding the TechCorp partnership with NexaTech..."

Pseudonymized: "Person_A3F2 <email_7B1C@example.com> wrote: 
Hi Person_0D4E, regarding the Org_4C8A partnership with Org_9F3B..."

The cloud model receives the pseudonymized version. It can still analyze the thread structure, identify that Person_A3F2 is negotiating with Person_0D4E, and summarize the key points. The reasoning quality is nearly identical — the model doesn't need to know the real names to understand the negotiation dynamics.

The response comes back with pseudonyms:

"Person_A3F2 proposed a revenue-sharing model with Org_9F3B. 
Person_0D4E agreed in principle but requested..."

The system de-pseudonymizes:

"Ana García proposed a revenue-sharing model with NexaTech. 
Paco agreed in principle but requested..."

Cost: one cloud API call + ~2ms pseudonymization. Privacy: no real names or emails left the device.

Scenario 3: Health data (critical sensitivity)

User asks: "What medications am I taking?"

Health entries are critical. They never leave the device, period. The system routes to the local Ollama model. If Ollama is unavailable, the query fails gracefully — it does NOT fall back to the cloud.

The local model's response might be less polished, but for medical data retrieval, the task is usually simple: find the records and list them. A 2B model handles that fine.

Cost: one local model call. Privacy: absolute — zero data exposure.

Scenario 4: Note with accidental PII (elevated sensitivity)

User creates a note: "Meeting with Dr. López about the lab results. Blood pressure 140/90."

The note's domain is normal, but the content contains medical terminology ("Dr.", "lab results", "blood pressure"). The classifier elevates it to critical. From this point on, this note is treated like health data — local only.

The user didn't tag it as medical. They didn't configure anything. The system caught it automatically. Conservative false positives are the design choice: if a note mentions "Dr. Pepper," it gets elevated too. That's a minor inconvenience (one note processed locally instead of on the cloud) with zero privacy risk.

The audit trail

Every routing decision is logged:

Field	What it records
`domain`	Which data domain (notes, emails, etc.)
`record_id`	Which specific record
`sensitivity`	Classified sensitivity level
`action`	What happened: `sent_pseudonymized`, `sent_plain`, `blocked`
`destination`	Where it went: `embeddings`, `contextual_retrieval`, `sleep_time`

The pii_routing_log table creates a complete audit of what data was exposed to which processing pipeline. If you ever need to answer "did my medical data ever touch a cloud service?", the answer is in the log.

This is also how we verify the system works correctly. The log shows every routing decision. If a critical record ever appears with action sent_plain and a cloud destination, that's a bug — and the log caught it.

Where routing applies

PII-aware routing isn't just for chat interactions. It applies everywhere the system sends data to an LLM:

Embeddings. When generating semantic embeddings, the text is classified before being sent to the embedding model. If you're using a cloud embedding API (future option), high and critical records get embedded locally via Ollama instead.

Contextual retrieval. The HyDE pipeline (generating hypothetical answers for better search) uses LLM calls. If the search touches sensitive domains, those calls route through the pseudonymizer.

Sleep-time compute. The background intelligence jobs process records during idle periods. The enrichment job (re-extracting entities from hot records) respects the same routing rules — a critical record only gets re-extracted if Ollama is available.

Entity extraction. When the CRUD hooks pipeline sends text to the LLM for entity extraction, the same classification applies. A health-related note gets extracted locally.

The routing layer sits between every LLM consumer in the system and the actual model call. It's middleware — invisible to the features that use it, enforced consistently everywhere.

The multi-agent dimension

With a multi-agent topology (7 agents in our system), PII routing gets another layer: agent scoping.

Each agent has a scoped token that defines what domains it can access:

Francis (main):     notes:*, events:*, emails:*, contacts:*, diary:*
Sentinel (infra):   storage:*, hal:*, network:*
Dalí (creative):    photos:read, files:read
Ledger (finance):   finance:*, crm:*
Darwin (analytics): graph:*, insights:*

Sentinel can't access emails. Dalí can't read the diary. This is enforced at the API level — even if a prompt injection tricks Dalí into requesting diary entries, the scoped token blocks it.

Combined with PII routing, this creates defense in depth:

Agent scoping prevents access to domains the agent shouldn't touch
Sensitivity classification catches PII regardless of domain
Pseudonymization protects data that needs cloud processing
Audit logging records everything for verification

A prompt injection attack would need to bypass all four layers to exfiltrate sensitive data. The scoping blocks the API call. The classification catches the content. The pseudonymizer strips the PII. The audit log records the attempt.

What we explicitly don't do

We don't use ML for classification. A fine-tuned classifier could be more accurate than regex patterns. But it would need to see the data to classify it — which means sending potentially sensitive data to a model before deciding if it's safe to send to a model. Regex is dumber but has zero data exposure during classification.

We don't redact — we pseudonymize. Redaction ([REDACTED]) destroys information the cloud model needs for reasoning. Pseudonymization preserves structure ("Person_A sent an email to Person_B") while hiding identity. The cloud model can still reason about relationships, quantities, and patterns.

We don't let the user override critical. You can change a record's sensitivity from normal to high manually. You cannot downgrade critical to anything else. Health data stays local regardless of user preferences. This is a deliberate paternalistic choice — the privacy risk of accidentally exposing medical data outweighs the convenience of sending it to a better model.

We don't route based on the LLM provider's privacy policy. Whether provider A's privacy policy is better than provider B's is irrelevant. The system treats all cloud LLMs identically: external services that should never see critical data and should only see high data in pseudonymized form. Trust the math, not the terms of service.

What I'd do differently

I'd add per-field sensitivity, not just per-record. Currently, a contact record is normal even though the phones field is arguably more sensitive than the company field. Per-field classification would let us pseudonymize just the phone number while sending the company name to the cloud. More precise, but also more complex — the pseudonymizer would need to understand JSON field structure.

I'd build a sensitivity dashboard earlier. The pii_routing_log has all the data, but there's no UI for it yet. A dashboard showing "this week: 450 records processed, 380 sent plain, 65 pseudonymized, 5 blocked" would build user trust and make the privacy system tangible.

I'd make the regex patterns configurable. Different users have different sensitivity needs. A doctor might want "aspirin" to be flagged as medical. A pharmacist might want it treated as normal. The current patterns are one-size-fits-all, which means they're too aggressive for some users and not aggressive enough for others.

The takeaway

The privacy problem in personal AI isn't "local vs cloud." It's "which data goes where." Most of your data is fine to send to a cloud model — your calendar events and bookmark titles aren't secrets. Some data needs protection but can still benefit from cloud reasoning — pseudonymize it and send the structure without the identity. A small fraction of data should never leave your device — route it locally and accept the quality trade-off.

Three components: a regex classifier (zero LLM calls, deterministic), a SHA-256 pseudonymizer (consistent, reversible, persistent), and a routing table (domain defaults + content elevation). No ML, no fine-tuning, no privacy policy trust assumptions.

The system processes your medical notes with a 2B local model and your calendar queries with a cloud model. It knows the difference because a regex matched "blood pressure" — not because it asked an AI what's sensitive.

Next up: designing an AI approval system — when should your agent ask for permission, and how do you build a confirmation workflow that doesn't slow everything down?

Sleep-time compute for personal data: what your AI should do while you sleep

Victor García — Thu, 26 Mar 2026 12:01:52 +0000

Your personal AI assistant sits idle most of the day. You send it a message, it responds, then it waits. For hours. Maybe all night. The compute is there — the model is loaded, the database is running, the server is warm. But nothing happens until you type the next message.

That's test-time compute: work done when the user asks for it. Letta's research (arXiv:2504.13171) showed that shifting processing to idle periods — sleep-time compute — achieves 5× fewer tokens at test time and 15% more correct answers. But their implementation only processes conversation memory. Nobody had applied it to structured personal data.

We did.

The idea

Instead of the agent doing all its thinking when you ask a question, it does most of the thinking in the background — during idle periods when you're not using the system. When you finally ask "what's going on with Project Tempest?", the answer is already half-assembled.

The system maintains four background jobs that run every 30 minutes, but only when you're idle:

Priority	Job	What it does
1	Enrich connections	Finds hot records with few graph links, re-runs entity extraction
2	Generate summary	Compiles a weekly overview from the change log
3	Detect patterns	Discovers entities that co-occur but aren't linked
4	Update preferences	Learns behavioral patterns from your data

Each job consumes tokens from a configurable budget (default: 5,000 tokens per execution). When the budget runs out, lower-priority jobs get skipped. This means enriching connections (the most impactful job) always runs, while preference learning (the least time-sensitive) gets skipped first if resources are tight.

The trigger: idle detection

The engine only runs when you're not using the system. If you're actively writing notes or reading emails, the background jobs wait. This matters for two reasons:

Resource contention. The LLM (whether local via Ollama or remote via API) is a shared resource. Background jobs competing with user requests for model access would add latency to the interactions you actually care about.
Relevance. Sleep-time processing works on data that has settled. Running entity extraction on a note you're still editing wastes tokens — the note will change again in 30 seconds. Waiting until you're idle means processing stable data.

The idle detector is simple: if no user activity (API requests from the dashboard, agent messages, WebSocket heartbeats) has occurred in the last N minutes, the user is idle. The scheduler checks this condition before executing each run.

Job 1: Enrich connections

The highest-priority job. It finds records that are "hot" (recently accessed, heat score > 0.3) but poorly connected in the knowledge graph (fewer than 3 entity links). These are records you care about but that the system doesn't fully understand yet.

Here's what the query returned on a real run:

Optimal Fuse Burn Rate Calculations v3    heat: 0.5  links: 2
Recipe: Rodney's Smoked Eyebrows Marinade heat: 0.5  links: 2
Banned Substances List (and Why Rodney…)  heat: 0.5  links: 2

Three notes flagged. Each has a warm heat score (recently accessed) but only 2 entity links where the average for this user is 5+. The initial extraction caught the obvious entities, but a second pass might find connections to people, projects, or locations that were mentioned implicitly.

The engine re-enqueues them to the async extraction pipeline at priority: low — they won't compete with real-time user actions. When the extraction worker picks them up, it sends the full note content to the LLM for a more thorough entity pass than the initial CRUD hook provides.

Why prioritize this job? Because the knowledge graph is the foundation of search ranking, the digest engine, and the agent's contextual awareness. A poorly connected hot record means the system is blind to something you're actively working on. Enriching it improves everything downstream.

Cost on this run: 600 tokens (3 records × ~200 tokens each). Execution time: 16ms (just the enqueue — the actual extraction happens later).

Job 2: Generate summary

Aggregates a week of change log activity and the most active entities from the knowledge graph into a single pre-computed insight.

On this run, the change log query returned:

files:        201 inserts, 5 updates, 152 deletes
rss:          141 inserts
emails:       61 inserts, 32 updates, 7 deletes
notes:        2 updates
kanban_cards: 1 update

And the top graph entities by recent activity:

Madrid              (location)      28 mentions
micelclaw           (organization)  12 mentions
Meta Platforms, Inc.(organization)   7 mentions
Instagram           (location)       6 mentions
Victoria            (person)         1 mention

Two things jump out. First: 201 file inserts and 152 file deletes in one week — that's a bulk operation or a sync cycle, not manual activity. The summary captures this so the agent can mention it if asked "what happened this week?" without scanning 600+ change log rows at query time.

Second: "Instagram" classified as a location is an entity extraction error — the kind of noise the enrichment job (Job 1) and the merge candidates system are designed to catch over time.

The summary gets stored as a weekly_summary insight with a 7-day TTL. No LLM call needed — it's pure SQL aggregation.

Cost on this run: 50 tokens (fixed). Execution time: 14ms.

Job 3: Detect patterns

The most interesting job. It self-joins entity_links to find pairs of entities that co-occur in 3 or more records but have no direct link between them — latent connections nobody made explicit.

On this run, five patterns emerged:

Rodney ↔ Dolores     co-occur in 12 records, no direct link
Rodney ↔ Linda       co-occur in 11 records, no direct link
Rodney ↔ BoomClaw    co-occur in 10 records, no direct link
Warehouse B ↔ Rodney co-occur in 10 records, no direct link
Benny ↔ Rodney       co-occur in 10 records, no direct link

Every pattern radiates from "Rodney" — he's a hub entity that appears alongside four other entities across 10-12 records without any direct graph edge connecting them. The extraction pipeline created links from each note/email to "Rodney" and to "Dolores" independently, but never linked Rodney to Dolores directly. The co-occurrence pattern reveals the relationship that was hiding in plain sight.

Each pair becomes a connection_discovered insight with a 14-day TTL. The next time you search for "Rodney," the graph traversal finds Dolores, Linda, BoomClaw, Warehouse B, and Benny — even though no single record ever says "Rodney works with Dolores."

This is the job that produces the "how did it know that?" moments. The answer is always the same: it counted co-occurrences while you weren't looking.

The query itself — a self-join on entity_links filtered by NOT EXISTS — took 138ms. That's the heaviest operation in the pipeline, and it runs during idle time where nobody notices. At query time, the connections are already in the graph.

Cost on this run: 30 tokens (fixed — pure SQL, no LLM).

Job 4: Update preferences

The system learns behavioral patterns by analyzing your data over time. On this run, two patterns were detected:

Writing time distribution (last 30 days): All 50 notes created at hour 12 UTC. That's not a preference — that's a signal so strong it maxed out confidence immediately. The system now knows that if you're going to write a note, it's probably at noon.

Tag frequency:

safety: 11, personal: 9, r-and-d: 6, strategy: 5, humor: 5

Both get persisted via UPSERT with incremental confidence — each observation nudges the score up by 0.05, capped at 0.95. After multiple runs, the preferences look like this:

scheduling / preferred_writing_hour = "12"
  confidence: 0.95 (max), evidence: 20,650 observations

organization / preferred_tags = ["safety","personal","r-and-d","strategy","humor"]
  confidence: 0.95 (max), evidence: 417 observations

The agent uses these when it needs to make decisions. Scheduling a reminder? It knows noon is when you're active. Suggesting tags for a new note? It offers your most-used tags first. Creating a diary entry template? It matches your writing style.

If a preference is wrong, you delete it via the API. The system may re-learn it later if the pattern persists, but with reduced confidence — the deletion counts as negative feedback.

Cost on this run: 20 tokens (fixed — pure SQL, no LLM).

The real numbers: 700 tokens, 184 milliseconds

Here's the actual pipeline summary from the run above:

┌────────────────────┬────────┬────────┬───────────────────────────────┐
│ Job                │ Tokens │ ms     │ Output                        │
├────────────────────┼────────┼────────┼───────────────────────────────┤
│ enrich_connections │ 600    │ 16     │ 3 notes re-enqueued           │
│ generate_summary   │ 50     │ 14     │ 1 weekly_summary insight      │
│ detect_patterns    │ 30     │ 138    │ 5 connection_discovered       │
│ update_preferences │ 20     │ 16     │ 2 preferences updated         │
├────────────────────┼────────┼────────┼───────────────────────────────┤
│ Total              │ 700    │ 184    │                               │
└────────────────────┴────────┴────────┴───────────────────────────────┘

700 out of the 5,000 token budget — 14%. The full pipeline completed in under 200 milliseconds. Three of the four jobs are pure SQL with zero LLM calls. Only enrich_connections queues work for the model, and even that just enqueues — the actual extraction runs later at low priority.

Every execution gets logged to sleep_time_jobs for auditability. If a job fails, the error is recorded and the next job still runs — the pipeline is fault-tolerant by design.

The token budget: sleep-time vs test-time

Every sleep-time execution has a capped budget — 5,000 tokens by default, configurable per user. This prevents runaway costs from background processing. The jobs run in priority order and stop when the budget is exhausted.

The insight from Letta's research holds: spending tokens during idle time dramatically reduces what you need to spend during active conversations. When the agent already knows that Rodney is connected to Dolores across 12 records (because Job 3 discovered it overnight), answering "who works with Rodney?" costs a graph traversal query (~5ms) instead of a full cross-domain LLM analysis (~3,000 tokens).

We track sleep-time and test-time token usage separately in the token metrics dashboard, so you can see the trade-off directly: more sleep-time tokens → fewer test-time tokens → faster, cheaper responses.

The three-stage digest

The sleep-time engine powers an evolved version of the Digest Engine — the system that tells the agent "here's what changed since you last checked."

The original digest was simple: scan the change log, format a markdown file, write it to the agent's workspace. The agent reads it on the next heartbeat.

The v2 digest is a three-stage pipeline:

Stage 1 — Selection. Filter changes by relevance using configurable rules stored in a digest_rules table. VIP emails (from your boss, from specific contacts) trigger immediate notification via PostgreSQL LISTEN/NOTIFY. Routine changes (a synced contact updated its phone number) get buffered for the periodic digest.

Stage 2 — Correlation. Use the LLM to discover cross-domain connections between changes. "You received an email from Ana García. Ana is attending tomorrow's meeting. You have 2 unfinished notes about the project she's working on." This stage is why sleep-time matters — the correlation discovery happens in the background, not when the agent is trying to respond to you.

Stage 3 — Scoring. Rate each item by urgency, cross-domain relevance, and historical feedback (did the user act on similar insights before?). The output shifts from "what changed" to "what matters and why."

The scored digest gets written to DIGEST.md in the agent's workspace. The agent reads it and decides what to surface. Urgent items might trigger an immediate notification. Low-score items accumulate for a daily summary.

The world model

One output of the sleep-time engine is a materialized "world model" — a living document that summarizes your current state:

Active projects and their status
Key people and recent interactions
Upcoming deadlines and events
Behavioral patterns and preferences

The world model is updated incrementally. Each sleep-time run only modifies the sections affected by recent changes. The agent references it as persistent context — a pre-computed summary of "what's going on in your life right now" that doesn't need to be recomputed every conversation.

This is inspired by Daniel Miessler's PAI framework (MISSION.md, GOALS.md, PROJECTS.md pattern), adapted to structured data. Instead of the user maintaining these documents manually, the system generates them from real data.

Zero cost when idle

The most important design decision: when nothing has changed, the engine does nothing. Zero tokens. Zero queries. The scheduler checks for pending changes in the change log before executing any job. No changes → skip the entire run.

This means the system's cost is proportional to your activity, not to time. A weekend where you don't use the system costs nothing. A busy Monday with 50 emails and 10 notes triggers multiple enrichment passes. The cost follows the value.

Similarly, the digest delivery to the agent is conditional. No changes → no DIGEST.md written → no system event → the agent doesn't wake up → zero tokens consumed. This was a deliberate choice over a heartbeat model where the agent would check for updates periodically regardless.

What I'd do differently

I'd add entity type validation in the summary job. The real run showed "Instagram" classified as a location — a clear extraction error that propagated into the weekly summary. A simple validation step (is this entity type plausible for this name?) would catch obvious misclassifications before they pollute insights. The data exists to fix this; we just haven't built the filter yet.

I'd build the pattern detection job first, not the enrichment job. Enrichment (Job 1) improves the knowledge graph incrementally. Pattern detection (Job 3) produces visible, surprising insights that users actually react to. "Ana García is connected to Project Tempest" is a moment of delight. "We added 2 more entity links to your note about Tempest" is invisible maintenance. Leading with delight would have made the sleep-time engine feel valuable sooner.

I'd make the token budget adaptive. Right now it's a flat 5,000 tokens per run. A smarter approach: scale the budget with the amount of pending work. 3 new records → 1,000 tokens. 50 new records after a sync → 10,000 tokens. The budget should match the opportunity, not be a fixed ceiling.

I'd add a "sleep-time log" visible in the dashboard. Currently, the only way to see what the engine did is through the insights API and the sleep_time_jobs table. A visible log ("Last night I discovered 3 new connections, updated 2 preferences, and generated your weekly summary") would build trust and make the background processing feel tangible.

The takeaway

A personal AI system that only works when you talk to it is wasting 95% of its available compute. The data is sitting in PostgreSQL. The model is loaded in Ollama. The knowledge graph has gaps that a 30-token LLM call could fill. Why wait for the user to ask?

Sleep-time compute shifts the work from "the user asked a question and now we scramble" to "we already know the answer because we connected the dots overnight." Four jobs, a token budget, an idle detector, and a three-stage digest pipeline. The system gets smarter while you sleep.

The insight that makes it all work: spending tokens when nobody is waiting for a response is categorically cheaper — in latency, in user experience, and in total cost — than spending them when someone is staring at a loading spinner.

Next up: PII-aware routing — how we send sensitive data to local models and everything else to the cloud, without the user having to think about it.

Hybrid search with RRF: combining pgvector, tsvector, and a knowledge graph in one query"

Victor García — Tue, 24 Mar 2026 12:04:03 +0000

Here's a search query: "beach trip."

Full-text search finds nothing — no record contains the word "beach." But there's a note that says "Qué calor en Valencia, el agua estaba perfecta." Semantic search finds it because the embedding for "beach trip" is close to the embedding for a hot day at the beach in Valencia.

Now a different query: "Ana García."

Semantic search returns a dozen vaguely related records. Full-text search returns the 3 records that literally contain "Ana García." But neither shows you that Ana attended last week's meeting, is CC'd on 5 email threads, and appears in tomorrow's calendar — connections that only the knowledge graph knows about.

No single search method is enough. We needed all three, plus a way to combine them that doesn't require manual tuning.

The four signals

Our search pipeline produces four independent scores for every candidate result:

Signal	Source	What it catches	Tier
Semantic	pgvector cosine similarity	Meaning-based matches ("beach" → "calor en Valencia")	Pro
Full-text	tsvector + GIN + ts_rank	Exact keyword matches, fast and precise	Free
Graph	entity_links overlap	Relational connections ("Ana García" → meetings she attended)	Pro
Heat	record_heat table	Temporal relevance (recently accessed records)	Free (display), Pro (in ranking)

Free tier users get full-text search only — which is still fast and well-ranked thanks to tsvector with weighted columns (title gets weight A, content gets weight B, tags get weight C). Pro users get all four signals fused together.

The pipeline

The search happens in seven steps:

Query: "Ana García project update"
    │
    ├── 1. Vector search ──→ top-50 by cosine similarity
    ├── 2. Full-text search ──→ top-N by ts_rank (UNION ALL across domains)
    └── 3. Graph discovery ──→ N candidates via entity_links
                │
                ▼
         4. Deduplicate by (domain, record_id)
                │
                ▼
         5. Rank-normalize each signal to [0, 1]
                │
                ▼
         6. Detect degenerate signals
                │
                ▼
         7. Weighted fusion + multi-signal bonus
                │
                ▼
            Final ranked results with provenance

Let me walk through each step.

Step 1: Vector search

The query text is embedded on-the-fly using the same model that embeds records (qwen3-embedding:0.6b, 1024 dimensions). Then a cosine similarity query runs against the embeddings table:

SELECT domain, record_id, 1 - (embedding <=> $query_embedding) AS similarity
FROM embeddings
WHERE 1 - (embedding <=> $query_embedding) >= 0.3
ORDER BY embedding <=> $query_embedding
LIMIT 50;

The 0.3 minimum threshold filters garbage. The top 50 candidates move to the next step. If Ollama is down and we can't embed the query, this signal is simply skipped — the other signals still work.

Step 2: Full-text search

A UNION ALL query across all domain tables, using PostgreSQL's native full-text search:

SELECT 'note' AS domain, id AS record_id, ts_rank(search_vector, query) AS score
FROM notes
WHERE search_vector @@ plainto_tsquery('simple', $q) AND deleted_at IS NULL
UNION ALL
SELECT 'event', id, ts_rank(search_vector, query)
FROM events
WHERE search_vector @@ plainto_tsquery('simple', $q) AND deleted_at IS NULL
UNION ALL
-- ... contacts, emails, files, diary, bookmarks, kanban_cards
ORDER BY score DESC
LIMIT 50;

We use plainto_tsquery('simple', ...) instead of language-specific configurations. The simple configuration doesn't stem words, which matters for multilingual data — Spanish and English records coexist, and stemming rules for one language would butcher the other.

Each domain table has a search_vector tsvector column maintained by a trigger (or GENERATED ALWAYS AS ... STORED for newer tables). The vectors are weighted: title gets 'A', description/content gets 'B', tags get 'C'. A match in the title ranks higher than a match in the body.

Step 3: Graph discovery

This signal is different — it doesn't match text, it matches relationships.

The query is matched against graph_entities.normalized_name. If "Ana García" matches a Person entity, we find all records linked to that entity via entity_links:

-- Find entities mentioned in the query
SELECT id FROM graph_entities
WHERE normalized_name ILIKE '%ana garcia%' AND deleted_at IS NULL;

-- Find all records linked to those entities
SELECT source_type AS domain, source_id AS record_id
FROM entity_links
WHERE target_type = 'graph_entity' AND target_id = ANY($entity_ids);

The graph_score for each result is the overlap ratio: how many of the query's entities appear in the result's connections, divided by the total entities found in the query.

Step 4: Deduplication

The three signals produce candidate sets that overlap. A note containing "Ana García" might appear in vector search (semantically similar), full-text search (exact keyword match), and graph search (linked to the Ana García entity). We deduplicate by (domain, record_id) and track which signals produced each candidate.

Step 5: Rank-based normalization

Here's where it gets interesting. We do NOT normalize by raw scores. We normalize by rank position:

normalized_value = (N - position) / N

The top result in a signal gets 1.0. The bottom gets nearly 0. A candidate absent from a signal gets 0.

Why rank-based instead of min-max normalization? Because cosine similarity scores cluster. In a typical query, the top-50 vector search results might have similarities between 0.54 and 0.64 — a 10-point range. Min-max normalization would stretch this to 0.0–1.0, making the difference between rank 1 and rank 50 look huge when it's actually tiny.

Rank-based normalization treats position as the signal, not magnitude. First place is first place, whether it scored 0.99 or 0.55.

Step 6: Degenerate signal detection

Sometimes a signal doesn't discriminate. If all 50 vector search results have cosine similarities within 5% of each other, the signal is noise — everything "looks the same" to the embedding model.

When this happens, the weight assigned to the degenerate signal gets redistributed proportionally to the other signals. The search doesn't fail — it just relies more on the signals that are actually informative.

Step 7: Weighted fusion + multi-signal bonus

The final score combines all four signals:

score = (α × heat + β × semantic + γ × fulltext + δ × graph) / (α + β + γ + δ)

Default weights: all equal at 0.25 each. But the user can adjust them via the dashboard's ranking sliders — crank heat to prioritize recent activity, drop graph to ignore relationship signals, etc.

Then comes the multi-signal bonus:

Signals that found this result	Multiplier
1 signal only	×1.00
2 signals	×1.25
3 signals	×1.50

A result that appears in vector search, full-text search, AND graph search gets a 50% bonus. This rewards results that are independently confirmed by multiple methods — they're almost certainly relevant.

Heat doesn't count for the bonus calculation. It's a temporal signal, not a relevance signal — a record being hot doesn't mean it matches the query.

Two search modes

The system exposes two search endpoints that use this pipeline differently:

Standard search: `GET /search`

The default. Uses Reciprocal Rank Fusion with fixed weights:

rrf_score = Σ 1/(K + rank_i)    where K = 60
final = rrf_score × (1 + 0.1 × heat_score)

RRF is elegant: it ignores raw scores entirely, only caring about rank position. A result that's #1 in vector search and #3 in full-text search gets 1/(60+1) + 1/(60+3) = 0.0164 + 0.0159 = 0.0323. The K=60 constant smooths the curve — the difference between rank 1 and rank 10 is meaningful but not extreme.

Heat acts as a post-fusion tiebreaker with a maximum 10% boost. It never dominates relevance. A cold but highly relevant result always beats a hot but marginally relevant one.

Advanced search: `GET /search/advanced`

The full power mode. Four independent sliders, degenerate signal detection, multi-signal bonus. Used by the Search module in the dashboard where users can see and control exactly how results are ranked.

Every result includes full provenance:

{
  "provenance": {
    "heat_score": 0.52,
    "vector_score": 0.68,
    "fulltext_score": 0.52,
    "graph_score": 0.11
  }
}

Complete transparency. The user can see that a result ranked high because of semantic similarity (0.68) despite low graph connectivity (0.11), and adjust weights accordingly.

Free tier: surprisingly good without vectors

Free users don't get semantic search, graph search, or heat-weighted ranking. They get tsvector + GIN full-text search across all domains, with ts_rank() scoring and weighted columns.

This sounds like a big downgrade. In practice, it's surprisingly solid:

Searches for names, titles, and specific terms work perfectly — full-text search is exact.
Weighted tsvector means a match in the title ranks above a match in the body.
plainto_tsquery('simple', ...) handles both Spanish and English without configuration.
The UNION ALL across domains means one search bar finds notes, emails, events, contacts, and files.
GIN indexes make it O(log n) — fast even with tens of thousands of records.

The meaningful Pro differentiator is semantic search: finding "calor en Valencia" when you search for "beach trip." That's genuinely impossible with keyword matching. But for the 80% of searches where people type exactly what they're looking for, free tier search works fine.

Temporal expansion: the "wow" moment

Here's a feature that surprised us with how useful it turned out to be.

If the query mentions an entity that has upcoming events (within ±7 days), temporally close results get a boost:

temporal_boost = 1 + 0.15 × proximity

In practice: you search for "Ana García." The system finds that Ana has a meeting with you tomorrow. Notes, emails, and contacts related to Ana that were created or accessed in the last week get boosted. The search results naturally organize around "here's everything relevant to Ana before your meeting tomorrow."

We didn't plan this as a feature — it fell out of having the knowledge graph and the calendar in the same database. But it consistently produces the kind of results that make people say "how did it know I needed that?"

The tuning knobs

Five parameters control the pipeline behavior:

Parameter	Default	What it does
`MIN_SIMILARITY`	0.3	Cosine sim threshold. Higher = less vector noise
`RRF_FETCH_SIZE`	50	Candidates per signal. Lower = faster, fewer candidates
`RRF_K`	60	RRF smoothing constant. Higher = smoother rank differences
Multi-signal factor	0.25	Bonus per additional signal (1 + 0.25 × (count-1))
Degenerate threshold	5%	Signal suppression when score range is too narrow

We've left these at their defaults since implementation. The multi-signal bonus and degenerate detection handle most edge cases automatically. If we ever need to tune, the provenance metadata on every result tells us exactly which signal is helping or hurting.

Performance

All of this runs inside a single PostgreSQL instance, on the same machine that serves the API:

Operation	Latency
Vector search (50 candidates)	~15-30ms
Full-text search (UNION ALL, 7 tables)	~5-15ms
Graph discovery	~5-10ms
Fusion + scoring	~2-5ms
Total (standard search)	~30-60ms
Total (advanced search, 4 signals)	~50-80ms

No Elasticsearch. No Solr. No separate search service. pgvector and tsvector are PostgreSQL extensions that run in the same process as the rest of the database. One backup strategy, one connection pool, one operational concern.

For a personal system with a few thousand records per domain, this is more than fast enough. If we ever hit scale problems (unlikely for single-user), the first optimization would be reducing RRF_FETCH_SIZE from 50 to 20 — cutting candidate generation in half with minimal quality loss.

What I'd do differently

I'd implement full-text search from the very beginning, not as a free-tier afterthought. We built vector search first (Phase 3) and added tsvector later as a "fallback for free users." Turns out full-text search is essential even for Pro users — it catches exact matches that embeddings miss. "Show me the email from patricia@work.com" is a full-text query, not a semantic one.

I'd add provenance to the standard search too, not just advanced. We initially only exposed the score breakdown in /search/advanced. When we added it to the standard /search endpoint (in the provenance field), debugging search quality became ten times easier. Every bug report went from "the search is bad" to "this result has vector_score 0.8 but graph_score 0, why?"

I'd explore reranking with a cross-encoder. Our pipeline does retrieval + fusion but no reranking. A small cross-encoder model (like ms-marco-MiniLM) could re-score the top 20 results for higher precision. We deferred this because the current quality is good enough and adding another model to the Ollama queue would increase latency. But for a future post-MVP iteration, it's the obvious next step.

The takeaway

The trick to hybrid search isn't the individual signals — pgvector and tsvector are well-documented, and knowledge graph traversal is just recursive CTEs. The trick is the fusion: how you combine signals with different scales, different failure modes, and different strengths.

Reciprocal Rank Fusion solves the scale problem — ranks instead of raw scores. Degenerate signal detection solves the failure mode problem — a noisy signal gets suppressed instead of poisoning results. Multi-signal bonus solves the confidence problem — results confirmed by multiple methods are almost certainly good.

Four signals, one UNION ALL, one PostgreSQL instance, under 100ms. The search that finds "calor en Valencia" when you type "beach trip" — and also shows you that Ana García has a meeting with you tomorrow.

Next up: sleep-time compute for personal data — what your AI should be doing while you sleep, and why idle cycles are the most valuable compute you have.