Make Dual-Plane Identity an Invariant Before Stale Completions Bind the Gate

Last Tuesday I watched an eval gate flip green on a batch I had already marked closed. A completion landed from the model plane two minutes after the compute lease died, and the coordinator still wrote the score. Have you ever trusted a late JSON blob just because the schema looked polite? That event order is the counterexample I keep on the whiteboard when people glue free compute and free model calls into one happy box.

I am not writing a deploy diary, and I am not selling a dashboard tour. I am reviewing whether dual-plane identity is a real protocol, or a comment that evaporates under preemption. If the worker is gone and the model answer is still in flight, who is allowed to bind? Stay with that question until we can inject the failure and watch the gate tell the truth.

Declared assumptions

This is an architecture review of a proposed coordinator, not a production postmortem with customer traffic. The workload is a sharded agent-eval batch, and every shard needs both a worker lease and a model completion. I assume leases can vanish without a graceful drain, and I assume completions arrive once, twice, or never. I also assume we would rather leave a hole in the histogram than bind a score to the wrong generation.

The invariant is small enough to tattoo on a sticky note. A score may commit only when compute_lease_id, lease_gen, model_request_id, and prompt_hash all match the live generation that sent the prompt. If any field disagrees, the coordinator must reject or replay. It must not silently compensate by “using whatever came back.” Can a borrowed lab still be useful under that rule? Yes, if you treat the two planes as separate failure domains instead of one warm fuzzy pipeline.

The violating event order

Here is the sequence that common shard-id keys fail to preserve. Read it as a failing test, not as folklore.

1. Worker W1 on lease L7 sends prompt P with model request R9.
2. The free server is reclaimed, lease L7 expires, and W2 starts on lease L8 reusing the same shard folder.
3. The model plane finally returns R9, and the coordinator binds it onto L8 because the shard id still matches.
4. The gate sees a plausible score for the wrong generation and turns green.

Does that look familiar? It should, because most retry helpers key only on shard id and treat the model as a function, not a plane. They never ask whether the compute generation that issued the prompt is still alive. That is how a late completion becomes a lying mean.

Coordinator -> ComputePlane: grant lease L7 gen=7
ComputePlane -> ModelPlane:  complete(R9, prompt_hash=H)
ComputePlane X              preempt, L7 dead
Coordinator -> ComputePlane: grant lease L8 gen=8 (same shard folder)
ModelPlane  -> Coordinator: late complete R9
Coordinator X               REJECT unless bind_key == (L7,7,R9,H)

Data flow and failure domains

The compute plane owns disk, subprocesses, and the sandbox clock. The model plane owns in-flight tokens, tail latency, and duplicate delivery. The coordinator owns bind decisions, and it must not collapse those two clocks into one monotonic counter. When a design doc says “run the eval on a free server with a free model,” it usually draws one box. I draw two boxes, because the failure classes are not cousins.

Failure domain A is preemption: the worker disappears, and local files may be half written. Failure domain B is late or duplicate completion: the model plane answers after the lease moved. Failure domain C is mixed identity: shard ids get reused, request ids collide, or prompt hashes are computed after a tool side effect. Which of those do you currently log as a first-class event? If the answer is none, the gate is theater, and the histogram is a mood.

Numbered review of the bind protocol

1. Split the run identity before any prompt leaves

I want a closed record before the worker is allowed to call the model. The record is not a vibe. It is a tuple the bind gate can hash without consulting leftover folders.

# proposal / unexecuted fixture — not a production client
from dataclasses import dataclass

@dataclass(frozen=True)
class DualPlaneId:
    batch_id: str
    shard_id: str
    lease_id: str
    lease_gen: int
    model_request_id: str
    prompt_hash: str

    def bind_key(self) -> tuple:
        return (
            self.lease_id,
            self.lease_gen,
            self.model_request_id,
            self.prompt_hash,
        )

Why include lease_gen instead of trusting lease_id alone? Free servers get recycled, and identifiers get reused faster than your mental model. Have you ever logged a recycled worker name that still passed a prefix check? I have in fixtures, and the passing prefix is how L8 inherits L7.

2. Make bind a pure function of the tuple

The coordinator should not merge a late completion into the current lease because the JSON looks healthy. It should answer REJECT, REPLAY, or COMMIT as an explicit enum. Silent merge is the green-gate lie.

from enum import Enum

class BindDecision(Enum):
    COMMIT = "commit"
    REJECT = "reject"
    REPLAY = "replay"


def bind(live: DualPlaneId, incoming: DualPlaneId, payload_ok: bool) -> BindDecision:
    if incoming.bind_key() != live.bind_key():
        return BindDecision.REJECT
    if not payload_ok:
        return BindDecision.REPLAY
    return BindDecision.COMMIT

Is REJECT too harsh for a borrowed-compute lab? Sometimes it is, and that is a tradeoff you should name. It is not an excuse to skip the function and hope the mean converges. Would you rather replay the shard under a new generation, or publish a score that cannot name its parents? I know which one I can defend in a review.

3. Put a watermark in front of the gate

Fan-in must close on committed shards, not on “we received N blobs.” A late R9 must not move the mean after the watermark. I would rather leave a documented hole than let L8 inherit L7’s score. The denominator is not “shards we launched.” The denominator is shards that reached a terminal bind decision.

4. Inject the two failure classes before you trust the lab

Readers should be able to run a discrete-event sketch, not admire a diagram. The sketch below is labeled unexecuted production code on purpose. I am not claiming wall-clock speedups, and I am not claiming a cluster shape. I am claiming that stale binds are countable.

# proposal / deterministic simulator — run locally, no network
import hashlib
import itertools
import random
from collections import Counter

PREEMPT = "preempt"
LATE = "late_complete"
DUP = "duplicate_complete"
OK = "ok"


def prompt_hash(text: str) -> str:
    return hashlib.sha256(text.encode()).hexdigest()[:16]


def simulate(n_shards=2000, seed=7):
    rng = random.Random(seed)
    stale_binds = 0
    decisions = Counter()
    for shard in range(n_shards):
        gen = 1
        live = DualPlaneId(
            batch_id="b1",
            shard_id=f"s{shard}",
            lease_id=f"L{shard}-g{gen}",
            lease_gen=gen,
            model_request_id=f"R{shard}-g{gen}",
            prompt_hash=prompt_hash(f"prompt-{shard}-g{gen}"),
        )
        events = [OK]
        if rng.random() < 0.25:
            events.append(PREEMPT)
        if rng.random() < 0.35:
            events.append(LATE)
        if rng.random() < 0.10:
            events.append(DUP)
        incoming = live
        payload_ok = True
        for ev in events:
            if ev == PREEMPT:
                gen += 1
                live = DualPlaneId(
                    batch_id="b1",
                    shard_id=f"s{shard}",
                    lease_id=f"L{shard}-g{gen}",
                    lease_gen=gen,
                    model_request_id=f"R{shard}-g{gen}",
                    prompt_hash=prompt_hash(f"prompt-{shard}-g{gen}"),
                )
            elif ev == LATE:
                # stale completion still carries gen-1 identity
                incoming = DualPlaneId(
                    batch_id="b1",
                    shard_id=f"s{shard}",
                    lease_id=f"L{shard}-g{gen-1}" if gen > 1 else live.lease_id,
                    lease_gen=max(gen - 1, 1),
                    model_request_id=f"R{shard}-g{max(gen-1, 1)}",
                    prompt_hash=prompt_hash(f"prompt-{shard}-g{max(gen-1, 1)}"),
                )
            elif ev == DUP:
                payload_ok = True
        d = bind(live, incoming, payload_ok)
        decisions[d.value] += 1
        if d is BindDecision.COMMIT and incoming.lease_gen != live.lease_gen:
            stale_binds += 1
    return decisions, stale_binds


if __name__ == "__main__":
    decisions, stale = simulate()
    print(dict(decisions), "stale_binds", stale)
    assert stale == 0, "stale completion bound to a live lease"

Run it as a property check, not as a benchmark ritual.

python dual_plane_bind_sim.py

The acceptance rule needs an explicit denominator. Denominator: shards that reached COMMIT, REJECT, or REPLAY. Numerator: commits whose DualPlaneId still names the generation that sent the prompt. Acceptance: zero stale binds under preempt plus late-complete injection for the simulated N. If you cannot say the denominator out loud, you do not have a gate. You have a vibe with error bars.

Tradeoff table

Choice	Latency	Correctness under preempt	Cost on borrowed planes	What you owe the next on-call
Bind on shard id only	Low	Breaks on folder reuse	Cheap until the gate lies	A green mean you cannot replay
Bind on request id only	Medium	Breaks when ids recycle	Wasted completions on REJECT	Orphan scores after reclaim
Dual-plane tuple + watermark	Higher tail	Holds in the simulator	More replays, fewer false commits	A hole you can name
Quorum of three model calls	Highest	Still needs the compute gen	Burns the free plane fast	A cost story, not an identity story

Notice the last row. Extra model votes do not repair a missing lease generation. Have you ever added another judge because the first one was “flaky,” when the real bug was a stale bind? That is how evaluation architecture turns into incense.

Where a free model plane and a free compute plane actually help

I wanted a lab that could exercise preemption and late completion without pretending I had a reserved fleet. That is a constraint, not a slogan, and it is why the fixture stays single process and deterministic.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. I treat MonkeyCode as a place where free model access and a free server option can host this dual-plane rehearsal. I am not freezing model names, quotas, hardware, or duration into this review, because those claims go stale and I will not invent them. If you replay the fixture against that lab, keep the bind function as strict as the simulator. Soften it later only after you have a counterexample that the enum cannot name.

If you want to watch your own coordinator swallow a late R9, take the fixture to a free server and a free model endpoint on MonkeyCode and log the bind enum. That is the only ask. The article still works if you run the simulator on a laptop and never leave localhost.

What I would change next

1. Replace shard-folder reuse with generation-scoped workspaces, so L8 cannot see L7 bytes.
2. Add compare-and-swap on the bind record, keyed by the full tuple, not by shard id.
3. Split retry budgets per plane, because model-plane retries should not extend a dead compute lease.
4. Property-test the permutation of {preempt, late, duplicate, payload_corrupt} instead of one happy path.
5. Emit the rejected tuple as a first-class event, so the histogram can show holes without pretending they were zeros.

Would I add a third plane for tools? Not yet. Tool side effects belong inside prompt_hash or they belong in a different protocol. Mixing them into bind without a hash is how domain C sneaks back in.

Limitations, and who should not use this

Do not use this design if you already have a closed consensus protocol across eval workers. Do not use it as a production SLO, and do not treat the simulator as a latency study. The sketch ignores disk corruption, clock jumps, and prompt mutation after the hash. It also ignores billing identity across vendors, which is a different invariant with a different denominator.

Who should not use this approach? Anyone hoping a free plane will mint a statistically closed leaderboard without a bind protocol. Anyone who needs exactly-once side effects in the tool plane. Anyone who will “just average the late scores” because the batch is already late. That last group is the reason I opened with a green gate that should have stayed red.

The counterexample I still want from you

Which event order still breaks the invariant: a duplicate completion with a fresh request id, a preempt that returns during replay, or a prompt_hash computed after a tool write? Should the system reject, replay, or compensate? If you cannot pick one without waving at shard id, the protocol is not closed yet, and the gate is still guessing.