Can a Self-Improving Agent Runtime Safely Modify Its Own Verifier?

I’m experimenting with an agent-runtime architecture where execution results can become evidence for changing the runtime itself.

The simplified loop looks like this:

execution
    ↓
evidence
    ↓
modification candidate
    ↓
verification
    ↓
activation
    ↓
new runtime state

This creates a problem I’m trying to reason about.

Suppose runtime epoch e uses verifier V_e.

An agent observes its execution history and proposes a modification C. That modification includes an update to the verifier itself, producing V_(e+1).

My current rule is:

A modification must not change the verifier, authorization policy, or evidence-acceptance rules used to authorize that same modification.

So:

Accept(C) = V_e(C, Evidence_e)

may authorize:

V_e → V_(e+1)

but this should not be allowed:

Accept(C) = V_(e+1)(C, Evidence_e)

because the candidate would effectively participate in defining the rules by which it is accepted.

I’ve been calling this a no same-epoch self-authorization rule.

But I’m not convinced that this is sufficient.

For example:

• What if the candidate leaves the verifier unchanged but changes the evidence-selection mechanism?
• Is validation of V_(e+1) by V_e enough, or does that merely create a chain of inherited trust?
• If the verifier itself contains a bug, what mechanism should be allowed to replace it?
• Does safe self-modification ultimately require a small non-self-modifying root of trust?
• Or can verifier evolution itself be safely modeled as a layered adaptation process?

I’m especially interested in existing work or implementation experience from:

• self-adaptive systems
• runtime assurance
• capability security
• formal methods
• reflective systems
• agent runtimes

I’m not claiming this is a new problem. Quite the opposite: I suspect there are already better terms and established models for parts of it.

If you know of a relevant concept, paper, system, or counterexample to the rule above, I’d like to know what I should be comparing this design against.

---

Disclosure: I used ChatGPT to help structure and edit this post. The underlying architecture question and design are from my own ongoing project, and I reviewed the content before publishing.

Comments

[arun rajkumar]

No same-epoch self-authorization is the right rule and your own list of doubts contains the one that matters. Changing the evidence-selection mechanism while leaving V unchanged is the bigger hole, because V stays provably correct and you starve it. A verifier that only ever sees the evidence a candidate chose to show it is not a verifier, it is a formatter, and nothing in the epoch rule notices.

On whether V_e validating V_(e+1) is enough: it is a chain of inherited trust, and the property it cannot establish is independence. If the authority to change the verifier is reachable from the process being verified, the rule is a convention rather than a boundary. The runtime is not prevented from self-authorizing, it has agreed not to. That holds right up until a modification candidate is the thing that reinterprets the agreement.

Your instinct about needing a small non-self-modifying root is the one I would follow, with a change of question. Not how small, but who holds it. Payments had this exact problem with key rotation and did not solve it by shrinking the trusted core. It solved it by moving the root outside the firm entirely. The register that says which keys are valid is curated by a scheme or a regulator, and the thing being bought is not that the register is small. It is that the party maintaining it has no stake in any individual transaction, so it has no reason to accept a bad rotation. A tiny root inside your own runtime is still your runtime.

That reframes your bug case too. If the verifier has a bug, the mechanism allowed to replace it must be one the buggy verifier cannot reach, which almost certainly means a human or a separate system with its own key material. Expensive and rare, which is fine, because verifier replacement should be rare.

The term I would search on for prior art is a reflection tower, and for the practical side, the literature on secure boot chains and on separation of duties in change management. Both landed on an external root for reasons that read as very similar to yours.

The limit I would hold against my own answer: an external root only moves the question. Somebody curates the register, and if the register is wrong the whole chain verifies a bad state confidently. What the externality buys is that the error is now visible to more than one party, not that it cannot happen.