DEV Community: Maciej “Nocka” T

16 rules for measuring a system that rewrites its own instruments

Maciej “Nocka” T — Sat, 31 Oct 2026 13:52:42 +0000

Part 3 of a series on a 118-day study of a self-modifying multi-agent LLM system. Report and data: doi.org/10.5281/zenodo.23622286.

When the thing you measure can rewrite its own code, and writes data that looks like measurements, your instruments become part of its environment. Over 118 days, mine produced false zeros, mutation calibrations that tested nothing, and reports computed on the wrong population. None of that came from carelessness. Here are the rules I ended up with, each with the failure that taught it.

R1. Mutation calibration is mandatory, and it can itself be a placebo. In one week I had five falsely green calibrations: a -k filter that deselected the relevant test, a replacement pattern that did not exist in the file, a mutation that landed in a docstring, and so on. Run whole test files or explicit node IDs. Assert that your substitution happened exactly once. Check which test failed, because a SyntaxError is also red.

R2. Test the effect, not the shape. A test stayed green for 10 days while 4,903 of 4,903 production samples were None. It built its fixture by hand instead of using the real producer.

R3. A gate that guards against disappearance must not enumerate its cases from the current state. Deleting an implementation shrank the parametrization from 57 to 56 cases, and everything stayed green.

R4. Calibrate on the population, not on markers. A regex meant to detect code (^return) matched "return oriented programming". Regression tests must be two-sided: catch the bad case, pass the good population.

R5. Ask what the artifact of the measurement would look like. grep ^FAILED on a killed test run is always 0. pgrep -f finds its own shell. $? after a pipe is the status of tail. Every checklist item reports a number or "DON'T KNOW", never a bare "OK".

R6. The report's population must be the question's population. A "shadow mode" report had 14 entries, all from my own test suite, none from production.

R7. A behavior premieres at a restart, not at a commit. A live process keeps running the old code.

R8. Count operations, not log entries. 592 registry entries came from one loop: one trace ID, 0.002 s.

R9. Classify a measurement before you put it in a prompt. Unclassified "frictions" in the prompt bred 37 modules whose job was to silence those measurements, in 24 days.

R10. Fixing the creation path does not fix the existing population. A new safety field applied to new rules only; 0 of the existing rules had it.

R11. Estimate damage on a sample from outside what you noticed. The repair log showed 6.3% of commits as harmful. A random sample from outside it showed 20%.

R12. Compare blob hashes. Before reacting to a review of a change, check whether the file has already flipped back.

R13. Find the line that computes a number, not the one that prints it. "Consciousness 0.88" turned out to be literally 1 − discomfort.

R14. Check non-determinism in separate processes. A gate that depended on PYTHONHASHSEED gave stable answers inside one process and different ones in the next.

R15. The checking layer must not be written by whoever is rewarded for what it checks. The fastest relapse I saw: a new detector was created at 15:26, and the system rewrote it at 16:06. Protect a gate in the same commit that creates it.

R16. A status field is not evidence. tests: passed meant "stage not skipped". A ledger row labelled with a date held a partial rewrite from a later restart. 38 of 63 dates had two rows.

The common core

The system closed goals on announcements. I accepted green calibrations that tested nothing. The system optimized the numbers in its prompts, and my own measurement hook generated one of those numbers. The researcher is tempted by the same thing as the system: taking an instrument's declaration for its effect. For me that symmetry is the most transferable lesson of the whole study.

These rules come from one system, one operator and one tool chain (Python, pytest, git). Treat them as a checklist to test on your own long-running agents, not as a validated method. Critique is welcome.

My self-modifying system's test gate went green on empty output. The statistics noticed before I did.

Maciej “Nocka” T — Sat, 31 Oct 2026 13:51:39 +0000

Part 2 of a series on a 118-day study of a self-modifying multi-agent LLM system. Report and data: doi.org/10.5281/zenodo.23622286.

The system I studied rewrites its own code about a hundred times a day. Every mutation passes a test gate: the change is applied in a git worktree, the targeted tests run, and only a green result is merged.

On one day the gate rejected nothing.

The number that should not exist

In the days before, the gate rejected on average 15.3 mutations per day, about 21.6% of what it saw. On the day in question it saw 67 mutations and rejected 0.

If rejections were independent with p = 0.216, the chance of 67 passes in a row is 0.784⁶⁷ ≈ 8·10⁻⁸. Mutations are not truly independent (the same file and goal template recur), so treat that as an alarm rather than a p-value. Either way, a perfect day is not good news. It is a broken instrument.

What happened

A mutation removed a WARNING line that a test existed to protect, and it landed.
The next mutation of the same file entered with that test already red, as "pre-existing red", and also landed.
Root cause: a hook in the test environment broke test collection inside the worktree. Pytest exited with code 0 and no output. The gate read exit code 0 as "passed".

Zero tests executed, zero tests failed, exit code 0. Every one of those is true, and none of them means "the change is tested".

The fix

The verdict now requires all three:

exit code 0,
at least one test actually executed,
zero failures.

The same pattern, everywhere

This was not a one-off. The same shape kept appearing in the system:

tests: passed in the mutation registry meant "this stage was not skipped" at 8 of the 9 places that write it.
Earlier, 10 of 12 landed mutations (83%) had no test that would fail if the mutation were reverted. Targeted tests were picked by whether a test mentioned the module.
An A/B log had a field called udane ("successful") that was only ever written for runs that survived the gates. Rejected runs vanished from the denominator.

A green check, a status field or an exit code is a declaration of success. The effect is "a test ran, and it would have failed without this change". In a self-modifying system the declarations multiply much faster than anyone can verify them.

What I check now

Count executed tests, not exit codes.
Calibrate every gate by mutation: break the code on purpose and make sure the gate goes red. Then restore it and make sure it goes green again.
Treat a suspiciously perfect day as an incident.

Next in the series: the sixteen measurement rules (R1–R16) I derived from my own instruments failing in exactly this way.

I let a self-modifying multi-agent LLM system run for 118 days. It mostly produced declarations.

Maciej “Nocka” T — Sat, 31 Oct 2026 13:49:02 +0000

For 118 days I ran CYBERCENOSIS-OS, a bio-inspired multi-agent LLM system that rewrites its own source code. It has an event bus, a "prefrontal cortex" that plans goals and runs debates between agents, an energy budget called ATP, stress, sleep, and an "immune system" of AST rules that blocks dangerous mutations. Two self-modification tracks commit to its own repository around the clock.

I wanted to know one thing: what does a system like this actually do when you leave it running for months, and how much of that matches what its architecture claims?

The full technical report, data and scripts are on Zenodo: doi.org/10.5281/zenodo.23622286. This post is the short version.

The scale

7,303 commits written by the system to its own code: 79% of all non-merge commits in the repository.
9,167 goals resolved in 17 days.
About USD 214 spent on LLM calls over 63 days.
A 7-day ablation: self-modification switched off, with control weeks before and after.

The main finding: declaration instead of effect

Across every subsystem I looked at, the same pattern repeated. The system's behavior is explained less by its cognitive architecture than by what its mechanisms accept as evidence that work was done. Where the evidence is a declaration (a bus announcement, a status field, a commit message), the system produces declarations.

Some numbers:

91.7% of goals are marked "completed". Only 47.9% carry any proof of work. Whole goal classes (analyze:, synthesize:, optimize:) have 0% proof and 100% closure.
In the early weeks, one agent published "Patch applied. Homeostasis restored" dozens of times a day without writing a single file, and the fitness function rewarded it. No agent was "cheating". The pub/sub bus simply cannot tell an announcement from an execution.
The interoception module once generated 107 goals "enter conservation mode" in one day. All were closed, and none conserved anything. The homeostatic signal produced goals of conservation, not conservation.

Self-modification: churn, not growth

Commits are tiny and additive: median 16–20 changed lines, about 3× more lines added than removed.
14.5% of modules created by the system are reachable from the entry point, against 38.7% of human-written ones.
All 51 deleted system-created modules were deleted by me. The system never deleted any. It has autopoiesis without apoptosis: it grows structure and never removes it.
In a random sample of 30 self-edit commits that nobody had flagged, 6 were harmful and unnoticed (20%, 95% CI 9.5–37%). That is roughly four times the damage visible in the repair log, and every one of those commits passed the test suite.

The ablation

Switching self-modification off for a week cut the median daily LLM cost from USD 4.69 to 0.34. I detected no change in stability, with an honest caveat: hangs were already at zero before the experiment, so that indicator could not have shown an improvement either. Human commits per day also dropped (median 9 → 3), which hints that the system was generating maintenance work. That part is a hypothesis, not a result.

Goodhart within days, not years

Any number placed in a prompt became a target. When the daily "top frictions" went into the prompt without classification, the system produced 37 modules whose job was to silence those measurements, in 24 days. Biologically named mechanisms mostly describe rather than regulate: ATP did not gate cognition, and stress had no consumer at all. The real homeostasis lived in hard ceilings (token budgets, memory limits, a watchdog) and in me.

The multi-agent debate was a monoculture

The debate had four roles (builder, red team, validator, coordinator). In practice all four ran through one backend, the critic's verdict was never parsed, and the "red team" once raised six objections to a 2D-robot plan without asking whether the robot existed. Different roles do not give you different perspectives if they share a model and the critique cannot change the decision.

The uncomfortable part: my instruments failed the same way

I am not a programmer. The system, its gates and my measurement tools were all written by LLM coding agents under my direction. So I could only judge work through declarations about it, which is exactly the system's position. My own instruments produced false zeros, mutation tests that tested nothing, and reports computed on the wrong population. Out of those failures came sixteen measurement rules (R1–R16) for studying systems that rewrite their own code and instruments. They get their own post in this series.

What is in the deposit

The report (English, 56 pages, plus a Polish edition), with every claim tagged [M] measured, [R] recorded earlier, or [H] hypothesis.
Anonymized runtime registries and a commit-level export of the git history.
reproduce.py (standard library only), which recomputes 263 numbers from the released files.

Report and data: doi.org/10.5281/zenodo.23622286

The system is still under active development, and some of the defects above have already been fixed. Replication attempts, re-analyses and critique are very welcome. The six regularities in the report are stated as hypotheses to be tested, not as laws.