Make Assumed Tool Success an Uncommitted Write Before Your Planner Advances

Last Tuesday I walked a constructed planner log that already felt finished, and then I asked where the executor commit lived. The model had emitted three tool results in one JSON object, each marked succeeded, and the planner had advanced. Nobody had dispatched the notify step, yet the ticket workflow already treated Slack as done and closed the incident. Have you ever trusted a success bit that the executor never wrote, then spent the afternoon explaining a closed ticket to an empty channel?

That event order is the entire architecture review, and it does not need a cluster to reproduce on a laptop. A planner that treats model prose as a commit is not being clever about latency at all. It is writing an uncommitted success bit into a durable log, and every downstream join will believe that lie. Why do we keep copying the model's mood into storage when the executor still has not spoken?

Declared assumptions

I am not narrating a named production outage, and I am not attaching latency numbers from a fleet I do not have. This is an architecture review of a planner and executor split that I can simulate locally with injected silence. The code below is a proposal and a fixture, not a benchmark and not an executed production patch. If you need dashboards and deploy runbooks, this is the wrong article, because those live in another lane.

I will defend five assumptions, and I will stop the review if any of them is false.

1. The planner is a model call that proposes steps and must never apply side effects itself.
2. The executor is a separate process, and it alone may touch files, mail, tickets, or charges.
3. A free model endpoint and a free server exist for that pairing, with no quota, hardware, duration, or permanence claim attached.
4. Delivery is at-least-once, so retries exist, and duplicate dispatches are a normal event rather than a surprise.
5. Irreversible tools require a commit record before the planner may advance, close a workflow, or tell a human that work is done.

Where do those assumptions break in your design? If the model process is also the executor, this review does not apply, and you are debugging a different failure domain.

The violating order

Here is the counterexample I keep reconstructing, because it is the common implementation wearing a confident JSON schema.

Planner:  propose(write_file, notify_slack, close_ticket)
Model:    {write_file: ok, notify_slack: ok, close_ticket: ok}  // fabricated success
Planner:  append StepSucceeded(notify_slack)
Planner:  append WorkflowClosed
Executor: (never received Dispatch(notify_slack))

The invariant I want is boring on purpose, which is usually a compliment in a protocol. StepSucceeded(k) may exist only if Commit(k) exists, and Dispatch(k) happens-before Commit(k). A WorkflowClosed record may exist only after every irreversible step in the plan has a planner-visible success that the join gate allowed. Does your agent preserve that chain, or does it photocopy the model's mood into the log and call the photocopy a commit?

Constraints, data flow, and failure domains

I treat this as a two-role protocol, not as a chatbot with plugins glued onto the same process. The planner proposes. The executor commits. The join gate refuses to let the planner invent the second role when the model gets impatient. A free server is also a non-sticky executor in this review, so planner memory is not a source of truth after a restart.

I use MonkeyCode here only as an open-source pairing of free model access and a free server option for those two roles. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I am not attaching model names, quotas, or hardware claims, because those are not part of the protocol under test. If you want a planner and executor to hang this join-gate fixture on, that pairing is the one I reach for when I am reviewing convergence rather than a vendor SLO.

Data flow I would actually implement, in order:

1. Planner emits Plan(steps) with stable step ids, and it is forbidden from emitting a success bit.
2. Dispatcher appends Dispatch(k) to an append-only log, then sends the payload toward the executor.
3. Executor runs the tool, then appends Commit(k) or Abort(k) using the same step id and a causal pointer.
4. Join gate allows Plan(k+1) or WorkflowClosed only after Commit(k), or after a compensation record that names the same step.

sequenceDiagram
    participant P as Planner
    participant G as JoinGate
    participant L as CommitLog
    participant E as Executor
    P->>G: Plan(steps) without success bits
    G->>L: Dispatch(k)
    L->>E: payload(k)
    alt executor commits
        E->>L: Commit(k)
        G->>P: allow Plan(k+1) or close
    else model fabricates success
        P->>G: StepSucceeded(k)
        G-->>P: reject, no Commit(k)
    else executor stays silent
        G->>L: Dispatch(k) retry, same step id
    end

Failure domains I care about, because they fail independently and look the same in a green ticket:

• The model fabricates a success object for a step that was never dispatched at all.
• The dispatch is lost, so the executor stays silent while the planner grows impatient and closes.
• The commit is written, then the ack is lost, so a naive retry duplicates an irreversible side effect.
• Two planner retries mint two step ids for one user intent, and the join never closes on either id.

Which domain are you actually debugging when the ticket is already green? If you start inside the prompt, you are already late, because the log accepted a speculative write.

Minimal simulator

The artifact is a state machine you can run without a cluster, and I labeled every path a proposal. I have not executed this against a production agent, and you should treat the numbers in the suite as a denominator for traces, not as a performance claim.

Save this as assumed_success_sim.py.

from __future__ import annotations

from dataclasses import dataclass, field
from enum import Enum, auto
from typing import Optional
import itertools
import random


class Kind(Enum):
    PLAN = auto()
    DISPATCH = auto()
    COMMIT = auto()
    ABORT = auto()
    SUCCEEDED = auto()
    CLOSED = auto()


@dataclass(frozen=True)
class Event:
    kind: Kind
    step: str
    seq: int


@dataclass
class Log:
    events: list[Event] = field(default_factory=list)
    _seq: itertools.count = field(default_factory=lambda: itertools.count(1))

    def append(self, kind: Kind, step: str) -> Event:
        ev = Event(kind, step, next(self._seq))
        self.events.append(ev)
        return ev

    def has(self, kind: Kind, step: str) -> bool:
        return any(e.kind is kind and e.step == step for e in self.events)


def happens_before(log: Log, a_kind: Kind, a_step: str, b_kind: Kind, b_step: str) -> bool:
    a = next((e for e in log.events if e.kind is a_kind and e.step == a_step), None)
    b = next((e for e in log.events if e.kind is b_kind and e.step == b_step), None)
    return a is not None and b is not None and a.seq < b.seq


class JoinGate:
    """Proposal: refuse planner-visible success without a causal commit."""

    def mark_succeeded(self, log: Log, step: str) -> Optional[str]:
        if not log.has(Kind.COMMIT, step):
            return f"reject: StepSucceeded({step}) without Commit"
        if not happens_before(log, Kind.DISPATCH, step, Kind.COMMIT, step):
            return f"reject: Commit({step}) does not follow Dispatch"
        log.append(Kind.SUCCEEDED, step)
        return None

    def close(self, log: Log, steps: list[str]) -> Optional[str]:
        for step in steps:
            if not log.has(Kind.SUCCEEDED, step):
                return f"reject: WorkflowClosed missing join on {step}"
        log.append(Kind.CLOSED, "*")
        return None


def naive_copy_model_success(log: Log, steps: list[str], fabricated: set[str]) -> None:
    """Violating implementation: treat model JSON as a commit."""
    for step in steps:
        log.append(Kind.PLAN, step)
        if step in fabricated:
            log.append(Kind.SUCCEEDED, step)
            continue
        log.append(Kind.DISPATCH, step)
        log.append(Kind.COMMIT, step)
        log.append(Kind.SUCCEEDED, step)
    log.append(Kind.CLOSED, "*")


def gated_run(log: Log, steps: list[str], silent: set[str], fabricated: set[str]) -> list[str]:
    gate = JoinGate()
    rejections: list[str] = []
    for step in steps:
        log.append(Kind.PLAN, step)
        if step in fabricated:
            reason = gate.mark_succeeded(log, step)
            if reason:
                rejections.append(reason)
            continue
        log.append(Kind.DISPATCH, step)
        if step in silent:
            rejections.append(f"replay: Dispatch({step}) with silent executor")
            continue
        log.append(Kind.COMMIT, step)
        reason = gate.mark_succeeded(log, step)
        if reason:
            rejections.append(reason)
    reason = gate.close(log, steps)
    if reason:
        rejections.append(reason)
    return rejections


def invariant_holds(log: Log) -> bool:
    for ev in log.events:
        if ev.kind is Kind.SUCCEEDED:
            if not log.has(Kind.COMMIT, ev.step):
                return False
            if not happens_before(log, Kind.DISPATCH, ev.step, Kind.COMMIT, ev.step):
                return False
        if ev.kind is Kind.CLOSED:
            planned = {e.step for e in log.events if e.kind is Kind.PLAN}
            if any(not log.has(Kind.SUCCEEDED, step) for step in planned):
                return False
    return True


def run_suite(n: int = 100, seed: int = 7) -> dict:
    rng = random.Random(seed)
    steps = ["write_file", "notify_slack", "close_ticket"]
    naive_violations = 0
    gated_violations = 0
    for _ in range(n):
        fabricated = {s for s in steps if rng.random() < 0.30}
        silent = {s for s in steps if s not in fabricated and rng.random() < 0.30}
        naive = Log()
        naive_copy_model_success(naive, steps, fabricated)
        if not invariant_holds(naive):
            naive_violations += 1
        gated = Log()
        gated_run(gated, steps, silent, fabricated)
        if not invariant_holds(gated):
            gated_violations += 1
    return {
        "traces": n,
        "naive_violations": naive_violations,
        "gated_violations": gated_violations,
        "acceptance": gated_violations == 0,
    }


if __name__ == "__main__":
    print(run_suite())

Numbered path to validate the conclusion

I want a reader to falsify me in a few minutes, not to trust the diagram. Run the fixture, then change one event order, then decide whether the gate should reject, replay, or compensate.

1. Run python assumed_success_sim.py and read the dictionary it prints for one hundred traces.
2. Confirm the denominator is traces, not steps, because a single illegal close is a failed workflow.
3. Inspect naive_violations; I expect a large count whenever fabricated success is in the mix.
4. Inspect gated_violations; acceptance is 0 / 100 traces, not a percentage of steps that looked fine.
5. Add a fourth step, or shuffle silent onto close_ticket, and rerun before you argue about latency.

The acceptance rule is explicit on purpose. Over one hundred shuffled traces of a three-step workflow, with a thirty percent chance of fabricated model success and a thirty percent chance of executor silence, zero traces may contain StepSucceeded without a causal Commit. Pass means 0 violations over 100 traces. Anything else is a protocol bug, not a prompt-tuning issue.

Failure analysis I would take to a design review

Silent executors are not a model-quality problem, and fabricated success is not a retry-budget problem. If you collapse those two into one timeout, you will replay an irreversible tool after it already committed, or you will close a ticket that never notified anyone. I would rather stall the planner in public than publish a green workflow with a hole in the log.

Duplicate dispatch after a lost ack is the third cousin of the same bug, and it needs the same step id rather than a fresh one. A new id looks clean in the planner, then the join waits forever, and a human marks the work done by hand. Have you watched a model invent a second request id because the first one felt stale in the prompt window?

Compensation is not a synonym for retry. Retry is for silence before commit. Compensation is for a commit you wish you could undo. If notify_slack is irreversible, a late Abort is a lie, and the honest move is a compensating send or a human gate. The join still must record that decision, or the next planner call will assume the hole was filled.

Tradeoff table

Choice	Latency shape	Correctness	Extra work	Characteristic failure
Copy model success into the log	One model round	Weak	Almost none	Closed tickets with missing side effects
Join gate after every irreversible step	Plus one executor round per step	Strong if the log is honest	More round trips	Planner stalls on silence
Fan-out steps, join once at the end	Overlapped executor work	Strong only if the join is real	Bursty dispatch	Partial commits with no compensation
Timeout, then compensate	Bounded wait	Depends on compensate actually landing	A second side-effecting path	Compensate times out, dual write

I would not pick the first row to save a round trip, because the round trip is the protocol. Would you skip the commit on a payment service because the model already sounded sure?

What I would change next

I would split tools into reversible and irreversible sets, and I would refuse a single code path for both. Irreversible tools get a two-phase dispatch: prepare with an idempotency key, then commit only after the executor acks the prepare. Reversible tools can be cheaper, but they still cannot accept a fabricated success bit from the planner.

I would also bind the idempotency key to the user intent, not to a step id the model minted during a retry. Model-minted ids are a random source, and random sources do not converge under replay. The join gate should see one intent key even when the prompt is paraphrased on the third attempt.

Property tests over shuffled event orders belong in the same fixture, not in a slide about evaluation culture. I want Dispatch after Commit to reject, StepSucceeded before Dispatch to reject, and WorkflowClosed with a missing join to reject. If those three cannot fail a build, the architecture is still a prompt.

Limitations, and who should not use this

This approach assumes you can put a log in front of the executor, even a file-backed log on one box. It does not assume multi-region consensus, and it does not claim an SLO for a free server that may disappear. The simulator is a convergence check, not a load test, and it will not tell you how a real model distributes fabricated success bits.

Do not use this if every tool is a local read with no side effects, because you are paying join latency for a protocol you do not need. Do not use this if you cannot make irreversible tools idempotent, because replay will double-fire and the gate cannot save you. Do not use this as a substitute for an incident review, because I did not operate a fleet in this article.

Teams that need hard cross-region guarantees should not pretend a laptop fixture is that guarantee. Single-process scripts that already apply tools inline should not grow a join gate just to look distributed. If your planner and executor cannot be separated even in a test, start there before you argue about models.

Which event order breaks the invariant in your log, and should the system reject, replay, or compensate? If a fabricated StepSucceeded arrives before Dispatch, I want a reject. If Dispatch exists and the executor stays silent, I want a replay with the same step id. If Commit already landed on an irreversible tool, I want compensation only when the abort is explicit and named. What does your workflow do with that third case when the executor process will not be there tomorrow?