DEV Community: Yurii Serhovskyi

A harness grown from losses

Yurii Serhovskyi — Sun, 08 Nov 2026 08:50:44 +0000

I almost never see the auto-compact warning in Claude Code. That is not a setting. I dislike what compaction does — the conversation is replaced by a lossy summary, and the nested instruction files do not survive it — so the workflow was built so that almost no session reaches half of its context window. The "half" is an observation, not a measurement: nothing I keep records how large a session's window got. What is recorded is everything around the sessions.

A harness is everything around the model that is not the model. The model generates text and calls tools. The harness decides what the model sees before it has done anything, what it is allowed to do, how it learns that it was wrong, and when it stops. The claim behind "harness engineering" is that the environment, more than the prompt, decides what the agent produces.

Mine has five layers, each with a file in the repository of Pantensa, the household pantry and nutrition tracker I build alone, with an agent doing most of the typing: context, permissions, feedback, orchestration, and tests of the harness itself. The first three get a paragraph each. The last two are the post.

The dates matter, because the harness grew during use. The first five hooks landed on 14 August 2026, in a commit that says they "refuse five losses that already happened here". The wave of work measured below started two days later, just after midnight. That afternoon, mid-wave, a second commit turned five more lessons "that had the shape of a command" into rules, and four minutes after it came the test that checks the rules are wired. The wave closed on 17 August. Every number here was measured while the harness was still growing out of what the wave was losing.

Context: measure the position, never inherit it

A session starts by stating the tree's actual position — branch, HEAD, tracked and untracked files, divergence from main — and it measures all of that itself, because a handoff note written by the previous session is already stale by the time it is read. A SessionStart hook, session_state.py, does that. The same file holds a budget for the memory index: 6144 bytes, because that file is loaded into every session and "a line kept there is rent paid per turn". Context is debt with interest. Every turn re-reads the whole history — cached, at roughly a tenth of the base input price by the API docs as of August 2026, but re-read until the session ends. A token that settles into a long-lived window is paid for dozens of times. A token burnt in a window that is thrown away is paid for once.

Permissions: two verdicts, and the line between them is mine

guard_bash.py runs before every shell command. Its docstring: "Each rule encodes a loss that has already happened in this repo, not a style preference." It knows two verdicts. A refusal is only for irreversible loss — fourteen of them, and commit, merge, rebase and branch switching are not on the list, because prompting on routine work was ruled out as friction. One of the fourteen reads: git checkout -- already erased an uncommitted fix in this repo once. A warning is for a command that "will answer truthfully and mean something else": a count blind to untracked files, a viewport that is not 390 px, a gate that runs nowhere. "Blindness is not loss, so it never blocks"; the warning arrives beside the command that caused it, which is the only moment it is useful.

Feedback: the finding arrives at the command

Piping a test run into tail -5 is how the controller keeps thousands of lines of runner output out of its window — and the pipe reports the pager's exit code, so pytest | tail -5 exits 0 on a red suite. That has already been read as a pass here, by an agent in a session that was mine to check, and I did not. warn_pipefail.py now fires on the command that caused it, "not in review". A trick that saved context had a known way to lie, and it was closed with a hook rather than a reminder in the skill.

Orchestration: where each token burns

Long work runs in waves. A wave is a plan split into links, one fresh session per link, and the registry of the August wave opens with the rule: "Memory between links is the ledger and git log, never the dialogue." Three skills carry it — one audits the plans and writes the link prompts, one is the controller of a single link, one closes a link from the ledger alone — and eight reviewer roles sit under .claude/agents/.

The controller's skill states its principle in one line: "your context is the scarce resource, and the ledger is your memory." The rules follow from it. A task brief is passed as a path, never pasted. The history of previous tasks is never pasted — a real session reached a 42k-character prompt that was 99% pasted history. The diff reaches the reviewer as a file. Reports are files, and the controller receives four lines. Suite runs go to a log in the background, and only the summary lines and the exit code come back. A subagent's transcript is never read. After a compaction, trust the ledger and git log, not recollection.

The controller never opens the plan. The six plans of the August wave are 143 KB. The controller read them zero times; the scripts take the path. The skill's reason: a controller that opens the plan "has started paying, in the one budget the link cannot refill, for information two subagents already have."

A file goes in, four lines come out. Each implementer writes its report to disk and returns status, commits, test numbers, caveats. The reports for the wave's 23 tasks measured 353 KB on 17 August 2026 — the artefacts have since been deleted, so that number stands on my notes. What reached the controller was 23 times four lines.

Then the number I keep coming back to. In one link, a single test file dumped 74 MB of stderr in one run. Into the context went tail -5: five lines. The reviewer that audits gates carries the counter-rule to that trick — "three numbers or none: Tests, Test Files, failed", because a failing file does not change the total, so "N passed" reads green over a red gate.

The heaviest artefacts of the wave are the ones I can no longer cite — the review diffs, the task briefs, the per-task reports. They lived in an ignored directory, each was read by one window, and they were gone once the verdict was in, because after the verdict nobody needs them. That is the design. The expensive reading happens in windows that are discarded; the window that lives gets decisions and measurements.

The same idea as a table:

the fact is needed by	it lives in	what it costs
every session	`CLAUDE.md`	re-injected every request, survives compaction
some sessions, rarely	memory index + files, 6 KB budget	one line in context; the body on demand
one link	ledger + registry	a file, read at specific points
one task	brief + report	files of a throwaway window
nobody, once decided	the subagent's window	discarded with the window

What the same artefacts say about quality. The wave forecast roughly eight or nine new test cases over 23 tasks. It landed 22: 1910 to 1932. The registry names the divergence. One link forecast as a pure refactor closed at plus two, because two independent reviews found that the one region actually rewritten had nothing proving it, and I authorised a witness; another closed at plus three against plus one for the same reason. The overrun was review catching empty proofs, not scope creep.

Gross tokens across all windows are larger than a single long session would burn; the review fan-out alone guarantees that, and on a subscription "tokens" means how fast the usage limit drains. What falls is the compounded cost in the window that lives. What rises is the chance the work survives a dead session: the ledger is written after every task, so any window can die at the cost of one repeated task. Anthropic's own docs recommend most of this — clear between tasks, compact at pauses, subagents for exploration, files as external memory — as session discipline. The harness makes it something a session cannot skip: the controller has no plan to read because the skill lists the files it opens, and nothing else.

The rule without a witness

Everything above is enforcement, and enforcement is code. The first post on my site mentions two i18n lint rules that had died silently: a misconfiguration disabled them and lint stayed green. The fifth layer exists because the hooks can die the same way.

test_claude_hooks_are_armed.py opens with the reason: "A rule nobody registered never fires. A rule nobody witnesses stops firing without anyone noticing — the shape the i18n lint died in." It asserts that every hook is registered on the event and matcher it is useless without; that every rule id carries exactly one verdict; that the warning channel cannot answer a permission prompt; and that every rule has a behaviour witness in the neighbouring test file, one that goes red if the rule is removed. The witnesses have names like "the fourteen denials still bite" and "the hook that cannot measure says nothing at all". The test parses the hooks with ast rather than grepping them, because "the reasons these rules carry quote the very commands they refuse, so a grep for decide reads prose as code."

The other half of the layer is a reviewer that reads no code. The gate-evidence auditor gets the ledger and the reports and answers one question: "Did it go red — and did it go red because of what WE did?" Three verdicts per gate, OURS, NOT OURS, UNPROVEN, and "probably fine" is UNPROVEN. If proving a gate means moving the tree, it does not; it names the command and hands it back, because an unproven verdict with a recipe beats a fabricated one. It carries a catalogue of twelve ways a gate stays silent, and "every one of these has shipped in this repo": a pinned count that had to be lowered in the same commit; a not.toContain(file) that becomes true the moment the file is deleted, and asserts nothing; a gate that passed because of how it was run — and nine more.

Instructions are context too

The skills themselves were tested the same way, the day before the first hooks. One fix for a controller that kept rationalising its way into the plan was a table of the rationalisations and the counter to each. The scenario went green. Removing the table and re-running also went green: the bare rule held on its own. The table cost about 180 words loaded into every link session and bought nothing, so it now lives in the evidence file instead of being re-read on every dispatch. The second finding from the same round: "do not open the plan" was argued with; "these are the files you open" was not. An enumeration has nothing to negotiate against.

So the harness is not a prompt. A hook refuses before the tool runs, a test goes red when a rule loses its witness, and neither is text the model can weigh against a reason. A gate that cannot go red is worse than no gate, because it is believed. And a harness nobody tests dies the way the lint did — green, and quiet.

These numbers are from one wave in August 2026, taken while the rules were still being written, and the window sizes they imply were never recorded. What I would keep is the order of events: a loss, then a rule with the loss in its reason, then a test that fails if the rule stops firing. I cannot point to a rule in it that arrived any other way.

An assistant, not a record

Yurii Serhovskyi — Tue, 20 Oct 2026 08:07:44 +0000

I have never been a note-taking person. For years my notes were a place to put things so I would not forget them — no linking, no structure, none of the productivity machinery people build around Obsidian, and that has not changed. What changed is that since May 2026 a Claude agent has read and write access to the notes-vault, and it does that part instead of me: personal notes, projects, basic planning. I talk to it in a Discord channel.

Some evenings I type «Добраніч» — good night — into that channel, and my notes tidy themselves up while I go to bed. Some evenings I forget, and the next morning I type «Добраніч і добрий ранок» in a single message: good night and good morning, closing a day I never closed and opening the one I am in. The agent answers that it has queued a save and a health check. A couple of minutes later the results come back — what it filed, what it cleaned, what it found wrong.

To work from one word, the agent holds keys to everything I have written: it can write anywhere in the vault, and it never asks permission.

I did not build the structure

The obvious defence of that arrangement would be that I designed the vault carefully enough to survive it. I did not. The folder map, the naming conventions, the rules about what gets saved without asking and what has to be confirmed, the instruction file the agent reads before it touches anything — all of it was generated for me by obsidian-second-brain, an open-source skill that bootstraps exactly this setup. The convention that every note opens with a short block addressed to the next session came with it too. I did not write those rules. I accepted them.

What I wrote myself is one line of frontmatter:

layer: legacy      # the frozen thing that exists today
layer: target      # the thing being built to replace it

One of my projects is a rewrite: an older version that still runs, and a successor that is replacing it piece by piece. Both are described in the same vault, in notes that use the same words for different things — the old one passes shopping lists through to an external service, the new one keeps its own table. Nothing had gone wrong. It just seemed obvious that a search across both would eventually return the old design as an answer about the new one, so I tagged every note with which world it belongs to, and wrote down the rule that goes with the tag: when reasoning about the successor, treat the old notes as evidence about the present, never as a description of the target.

So my one contribution to the structure is not structure. It is a fence: a place where I decided in advance that the agent would be wrong, and took that ground away from it beforehand.

The channel is the whole product

An assistant I have to go somewhere to use is one more thing on the list of things to do, and what I do with lists like that is established. The part that makes this work is not the model or the vault. It is that talking to it costs nothing.

It lives in a Discord channel, chosen from two directions. I do not trust Telegram enough to route my notes through it. And Discord is the one messenger with a simple, official plugin for the agent — I did not have to write a bridge, maintain it, or trust my own bridge with the same keys. Fewer moving parts I own is itself a security position.

What goes through that channel during a day is not the tidy version. Ideas thrown in as they arrive, feature arguments about Pantensa, the product I am building, design generation, planning, personal things that have nothing to do with code. «Добраніч» is not the interaction — it is the closing command that files whatever the day produced.

Two things the polished version of this setup would leave out.

Writing to the vault is easier than reading it. There is an Obsidian app on my phone, but keeping a git-synced vault working there was more setup than I wanted, so I never finished it. When I actually need a full note away from my desk, I open the repository on GitHub and read the markdown there. The agent has better access to my notes on the move than I do.

And voice. Discord voice messages transcribe locally through a self-hosted speech model: record, transcribe, confirm the transcript, run the task. I built it, debugged it, and then never used it — dictating turned out not to be my way of thinking. It sits there, working, unused. I am keeping it, on the theory that the person who wants it later is me.

Why it needs the keys

An assistant that only appends could ask permission. Appending is safe: worst case the vault grows a note nobody reads. What this one does instead is edit what is already there. New information does not land at the bottom of a file as another entry — it goes back into the note that is now wrong, marks the superseded part as superseded, and leaves the correction where the next reader will hit it. A note in my vault about a piece of infrastructure carries a line saying the architecture below it was confirmed on a date, and that the recommendation above it is dead. I did not write that line. The agent wrote it over its own earlier advice.

That is what I want, and it cannot run behind a confirmation prompt. A prompt per edit turns a two-minute cleanup into a queue of yes, and I would stop within a week — the same way I stopped with linking, with tags, with every other habit this system exists to work around. The permission is not a shortcut. It is the feature.

Which leaves the real question: what happens when it rewrites them wrong.

Worth it for now

So far it has not. Nothing has been destroyed, nothing important has quietly turned into something false that I later acted on. That sentence is weaker than it sounds: I do not audit the vault. I do not read back what the agent filed to check it. I notice problems the way anyone notices problems in a tool they use daily — when something I need is not where it should be — and that has not happened. That is not the same as verified.

The technical fence is real. Everything runs in one container that has the vault and nothing else, the vault is a git repository, and every edit the agent makes is a commit I can read or revert. That is the part I would point at if someone asked whether this is reckless.

It is not the part that actually holds. The vault was never my source of truth — it is an assistant. Things I genuinely cannot afford to lose or to have subtly rewritten do not live there — they are in a plain notebook, sometimes duplicated into Obsidian and sometimes not. I never decided that as a policy. I noticed it as a fact, and only while writing this. The agent has keys to everything in the system, and what makes that survivable is that I never put everything in.

The leak I know about is not in the vault. Running an agent with permission prompts disabled means the channel that talks to it is a channel that runs commands. Everything above about the agent being trustworthy assumes the only person typing into it is me, and that assumption holds because of how the box is reachable, not because of anything the agent enforces. Reachability rots quietly: a line in a compose file that publishes a port instead of keeping it on an internal network looks almost identical to the line that does not. I know where mine is. I have not changed it.

So: an agent that can rewrite anything I have written, that never asks, that I do not audit, on a box whose boundary rests on an assumption I carry rather than enforce. Against that, the only system of notes I have ever kept for more than a month — because the part I was never going to do is done by something that does not get tired of doing it.

Worth it for now, and «for now» is the term of the deal, not a hedge. It holds while the vault stays the assistant and not the record, while the box stays unreachable, while nothing in there is something I cannot lose. Three separate things that can each stop being true, and none of them will announce it.

Mechanisms, not agreements

Yurii Serhovskyi — Mon, 17 Aug 2026 08:30:14 +0000

The type-check was green. All 263 tests were green. Every nutrient value on the screen read undefined.

This was Pantensa, a household pantry and nutrition tracker — FastAPI and PostgreSQL on the backend, Vue on the front — midway through replacing eight hard-coded nutrient columns with a proper many-to-many catalog. The frontend carried a hand-written mirror of the backend's read schemas: a TypeScript interface for every Pydantic model, kept in sync by me remembering to keep it in sync. I had renamed fields on the backend. The mirror still described the old shape, the tests still asserted against the mirror, and the compiler still agreed with both. Everything was internally consistent and none of it was true.

That is the failure mode worth writing about. Not a red suite — a red suite tells you something. A suite that is green because it agrees with itself tells you nothing, and it tells you nothing loudly, in the exact tone of voice that means "ship it".

I work on this codebase alone. Nobody reviews my pull requests, nobody notices when I get lazy at 1 a.m., and nobody remembers on Thursday what I decided on Monday. Under those conditions a rule that lives in a document, a convention, or my own good intentions is not a rule. It is an agreement I have made with myself, and I break agreements with myself constantly. The only rules that survive are the ones a machine enforces.

So the question I have been answering for the past few months is narrow and practical: for each architectural boundary I care about, what exactly goes red, and when?

The wire contract generates itself

The undefined bug had an obvious-looking fix and two tempting wrong ones.

Committing a fixture of the API response just moves the problem: the fixture becomes the new un-checked copy, drifting from the backend exactly the way the interfaces did. Standing up a real FastAPI process inside Vitest drags a cross-language runtime into the unit layer, which is the wrong altitude and flaky besides.

The fix that holds is to remove the copy. The backend's Pydantic schema is the single author of the wire; everything downstream is derived from it, and the derivation runs on a machine.

// frontend/openapi.json — committed, deterministic projection of app.openapi()

Three gates keep that chain honest, and each one names the thing that goes red:

A backend test asserts app.openapi() == frontend/openapi.json. Rename a field without regenerating, and the backend suite fails — the side that made the change, not the side that suffers from it.
openapi-typescript regenerates the TypeScript types from openapi.json before every entry point. Not a committed file — a build product, git-ignored, regenerated by a pre-hook on type-check, lint, test, coverage, build and dev. A build product cannot drift, because there is no stale copy for it to drift from.
The frontend's read types are derived, not written:

// modules/catalog/model/types.ts
type Schemas = components['schemas']

export type Product = Required<
  Pick<Schemas['ProductResponse'],
    'id' | 'origin' | 'name' | 'dimension' | 'is_fresh' | 'units' | 'provenance'>
>

A rename that somehow survives gate 1 removes a key from that Pick once the types regenerate, and vue-tsc goes red on the frontend. Required<> is doing real work there: OpenAPI marks defaulted fields optional, and stripping the ? while keeping | null leaves the compiler as a guard instead of letting me silence it with ?? 0.

One field is deliberately exempt. nutrients stays an explicit Partial<Record<NutrientKey, number>> rather than deriving from the generated { [key: string]: number }, because the sparse invariant — nutrients.fiber === undefined is not the same claim as fiber === 0 — is exactly what the generated type would erase. Deriving everything is not the goal. Deriving everything whose shape the backend owns is.

What it costs: the mechanism is repo-wide, the enforcement is catalog-only. Those derived types cover one bounded context; the rest of the frontend still reads the wire the old way and would still go green while lying. Extending it is mechanical, and it is not done. A gate that covers a quarter of the surface is worth writing down as exactly that.

A tier that drains itself

The frontend has the same problem in a different shape: not "is this value real?" but "is this import allowed?".

ESLint classifies every file into one of six element types and forbids every dependency it does not explicitly name. Read the policies as a matrix of who may import whom, and one row stops you:

Tier	May import	Importable by
`app`	everything	`legacy` only
`pages`	modules, shared, legacy	app, legacy
`modules`	shared, legacy	app, pages, legacy
`shared`	shared	everyone
`legacy`	everything	everyone

legacy — the tier holding code I intend to delete — was the only one that could both import everything and be imported by everything. That is not a quirk of migration. It is a structural attractor: any code with real dependencies and many consumers has exactly one legal address, and it is the doomed one.

This was not theoretical. A health store read by eight surfaces and a nutrition-limits store read by twelve were about to stay in legacy, and I had written down that leaving them there was "simultaneously the correct architecture and the lowest-risk move". Only the second half was true. A composable whose own docstring opens "THE ONE PLACE THAT ANSWERS which nutrients does this family show" was going to be moved into the tier I was trying to shrink, because there was nowhere else it was allowed to go.

The gap was precise: no tier could be imported by everyone while itself depending only on infrastructure. So application-level state — session, tenant context, the nutrient catalog, daily limits — had no legal home outside legacy.

The fix is a new tier, entities/, and one clause that does all the work:

app       → app, pages, modules, entities, shared, legacy, assets
pages     → modules, entities, shared, legacy, assets
modules   → entities, shared, legacy, assets      (still NOT other modules)
entities  → entities, shared, assets              ← may NOT import legacy
shared    → shared, assets
legacy    → everything                             (shrinking, never growing)

entities may not import legacy. Without that prohibition the new tier could lean on the doomed one and I would have rebuilt legacy under a nicer name. With it, promoting anything into entities/ forces its transitive dependencies out of legacy too. The tier drains the bucket by construction rather than by discipline — which matters precisely because my discipline is the thing that failed at the top of this post.

Then the part I think is the actual lesson. The ESLint plugin doing this classification warns, in its own documentation, that a misconfiguration disables the rule silently: green lint, zero enforcement. In a codebase where two i18n rules had already died that way, an unverified boundary config is not a boundary. So the tier shipped with deliberate violations — code that exists to be rejected, proving each policy actually bites.

I had learned that the hard way one tier over. The project rule reads "no raw strings, enforced by lint", and it was true — of the target tree. The rule was scoped to pages, modules, app and entities, and views/ was excluded entirely. Sixteen legacy views, eight of them with zero translated strings, including login and registration. views/ was melting outside the lint, which means it was rotting faster than it was melting. The enforcement I believed in covered exactly the code that did not need it.

What it costs: legacy is a direction with a ratchet, not a release gate. A guard counts the files under those patterns and holds a ceiling that plans may only lower. It is not zero today and may not be zero at release. Naming it a direction rather than a deadline is what keeps the number honest — a gate I would be tempted to postpone teaches me to postpone gates.

Three more of the same shape

The domain stays pure because an import graph says so. The backend runs import-linter contracts in CI: app.domain may import stdlib and app.domain only — no FastAPI, no SQLModel, no sibling package. It is written as an exhaustive denylist, because the tool has no "allow only X" contract, which means every new production dependency must be added by hand or the domain quietly gains the right to import it.

That cost has a mechanism of its own, and it is the piece I am fondest of. The config is data that nothing executed: a typo in forbidden_modules lints green — measured, exit 0, "5 kept", not a word about the dead entry. So a test now executes the config as data and checks that every app.* entry names a module that exists, that every child of app is classified by every contract, and that every direct requirement has all of its top-level import names denied to the domain. That last check immediately found one: python-multipart ships two import names, and the second had been missing from the denylist for the config's entire life.

Behaviour is preserved by an oracle, not by care. The DDD migration is a strangler, context by context, and each workflow moves as four artefacts: a port, an adapter around the legacy call, a flag at the single composition point, and a parity test asserting old and new return the same thing. Three of the four are disposable by design, and the flag exists so a bad flip is an environment variable rather than a deploy. It is more ceremony per workflow than a rewrite would need, and it is the reason no user-visible behaviour changed while roughly a third of the backend moved underneath them.

A missing secret fails at construction, not at 3 a.m. Credential encryption needs a master key from the environment, so Settings() refuses to construct without it — the process does not boot rather than booting into a state where the first background task discovers the problem. The design that key belongs to is deliberately modest: an earlier draft wrapped the data key twice, once under the master key and once under the password hash, and claimed a database dump alone was useless. Re-reading my own code killed it. Dual-wrap is an OR, so it is only as strong as its weakest path, and the password hash lives in the same database as the ciphertext. What is left is a trilemma — the operator cannot read user keys, background jobs can use them, and they survive a password reset: pick two. A key an unattended worker can use is, by construction, usable by whoever controls the process. The mechanism here does not resolve that. It just refuses to let me pretend I resolved it.

Where the tests sit

The same instinct shapes the test tiers, and the rule between them is a boundary, not a preference: only what cannot go red lower down belongs higher up. The shape of a wire field is a contract test. A domain rule is pytest. Layout at 390 px, the cookie session, Cache Storage and a verification email are end-to-end, because none of them can fail in a DOM implementation with no layout engine and no service worker.

Two consequences of taking that literally. Some checks are not tests at all: a script that walks elementFromPoint outward from every control to find the box it actually reaches needs a real browser and a real database, so it is a script, and calling it a test would only let it pass by default in an environment where it cannot run. And the end-to-end tier runs against its own database with no volume attached, so tearing it down takes the data with it — plus a setup step that aborts the run, before a single test, if the base URL is not the disposable stack.

Plans carry a test-count delta rather than a total: a plan that promises five new tests and lands four has a conversation to have. A total would just drift for a dozen innocent reasons.

The limit

Every mechanism here has the same weakness, and it is worth stating plainly rather than ending on a flourish: a mechanism nobody verifies is just an agreement with better syntax.

The boundary config that silently disables itself. The lint rule that was real for the target tree and absent where the mess actually lived. The import contract that lints green with a typo'd entry. In all three cases I had the mechanism, believed I had the enforcement, and had neither. What closed the gap was not more rules — it was tests aimed at the rules: deliberate violations that must be rejected, a test that executes a config as data, a check that a denylist covers every import name a dependency actually ships.

That is the whole trick, if it is one. Enforcement is code, code has bugs, and code with bugs needs tests — including the code whose entire job is to keep the rest honest. It costs more than writing a convention in a document. It costs less than a green suite where every field reads undefined.

Pantensa is pre-release and the migration is not finished. The legacy service layer still exists, views/ is still melting, generated types are enforced on one context out of several. The numbers in this post are from August 2026 and will be wrong by the time they matter. What I would keep, if I threw out everything else, is the habit of asking one question of every architectural decision: what goes red, and when? If the answer is "nothing, but I'll remember" — I won't.