Make Concurrent Tool Results a Causal Log Before Your Agent Applies Them

I was walking through a proposed agent last Tuesday when one reordered tool result poisoned the working tree. The planner had fanned two tools out in parallel because the free server looked idle enough to absorb both. Have you ever assumed that completion order would match dispatch order, even after a retry? The write landed first, the read landed second, and the merge function treated them as a commutative bag of patches.

That merge is the common implementation, and it fails a happens-before invariant the planner already knew. I am treating this as an architecture review of a proposed design, not a production postmortem with measured SLOs. The question is simple enough to simulate: can two free-server workers apply tool completions without inventing a total order the model never promised?

The violating event order

Call the planner P, a read tool R, and a write tool W. P dispatches R so the agent can observe file.py, then dispatches W with a patch that is only legal after that observation. The network, the model retry, and the worker preemption do not care about that story.

P: dispatch R(id=r1, after=[])
P: dispatch W(id=w1, after=[r1])   // planner intent: W happens after R
free-server worker B: complete W   // retry of an earlier speculative send
free-server worker A: complete R   // late, but causally first
merge(): apply(W), then apply(R)   // commutative bag, invariant broken

Did the write still look locally valid? Yes, because the merge only checked that each payload parsed. Did the working tree still match the planner's intent? No, because W overwrote a file that R had not yet witnessed, and R then cached a stale snapshot that later prompts treated as truth.

Declared assumptions

I am reviewing a single agent that fans tool calls across workers, not a multi-tenant control plane. These constraints are the ones I will hold fixed while the simulator runs.

1. Tool completions are delivered at least once and may be reordered across workers.
2. There is no shared, monotonic wall clock that both workers and the model endpoint trust.
3. A free model attempt may retry a tool payload without surfacing that retry to the planner.
4. A free server worker may be preempted after it has applied a side effect but before it acks.
5. File edits, test runs, and memory writes are not commutative, even when JSON patches look independent.
6. The acceptance rule is safety first: a wrong apply is worse than a delayed apply.

If your agent is strictly single-threaded and never retries, this review is heavier than you need. If your agent already treats the filesystem as a CRDT, you are solving a different problem, and I would not bolt a causal log onto that design.

Data flow and failure domains

The data flow is small enough to draw, and the failure domains sit on different clocks. I want the diagram to show the protocol, not a deployment cartoon.

sequenceDiagram
    participant P as Planner
    participant L as Causal log
    participant A as Worker A
    participant B as Worker B
    participant FS as Working tree
    P->>L: append intent(r1, after=[])
    P->>L: append intent(w1, after=[r1])
    P->>A: run r1
    P->>B: run w1
    B-->>L: complete(w1) out of order
    L-->>B: buffer, predecessors missing
    A-->>L: complete(r1)
    L->>FS: apply r1, then w1
    L-->>P: frontier {r1,w1}

I split the system into four failure domains because they fail independently. The planner domain can emit an intent that is internally inconsistent. The model domain can duplicate a tool call or return a completion with a swapped identifier. The executor domain, which is the free server worker, can apply a side effect twice after an ack loss. The merge domain can pretend unordered completions are a set.

Which domain owns the invariant? The merge domain, because neither worker can see the other's clock. If you push ordering into the model prompt, you are hoping a stochastic endpoint will preserve a protocol. Why would we do that when the log can reject the illegal apply locally?

What the common merge fails to preserve

The invariant I want is not “every tool eventually runs.” Eventual application is liveness, and liveness is cheap to fake with retries. The safety property is this: if intent w1 is recorded with after=[r1], then apply(w1) is forbidden until r1 is in the applied frontier, and apply(r1) must not observe effects of w1.

A bag merge preserves neither direction. A queue that sorts by completion timestamp preserves neither direction either, because worker B can complete first after a retry. Have you measured how often a “just apply whatever arrived” helper survives the first duplicate? It survives until the first duplicate, which is not an evaluation strategy.

A minimal simulator you can run

I want a fixture that is small enough to read in one sitting and strict enough to fail closed. Label this as an unexecuted proposal you can copy into a file; I am not claiming production hours or latency numbers against a live fleet.

Step 1. Record intents as an append-only log, not as a mutable job table.

Step 2. Stamp every completion with the intent identifier it claims to fulfill.

Step 3. Buffer completions whose predecessors are missing, instead of applying them.

Step 4. Reject a completion that names an unknown intent, and compensate a duplicate that names an already applied intent.

# causal_log.py — proposal fixture, not a production runtime
from dataclasses import dataclass, field
from typing import Dict, List, Optional, Set


@dataclass(frozen=True)
class Intent:
    id: str
    after: tuple
    kind: str  # "read" | "write"
    payload: str


@dataclass
class CausalLog:
    intents: Dict[str, Intent] = field(default_factory=dict)
    applied: List[str] = field(default_factory=list)
    buffered: Dict[str, Intent] = field(default_factory=dict)
    rejected: List[str] = field(default_factory=list)
    tree: str = "base"

    def record(self, intent: Intent) -> None:
        if intent.id in self.intents:
            raise ValueError(f"duplicate intent {intent.id}")
        self.intents[intent.id] = intent

    def _predecessors_applied(self, intent: Intent) -> bool:
        return all(pid in self.applied for pid in intent.after)

    def complete(self, intent_id: str) -> str:
        if intent_id not in self.intents:
            self.rejected.append(intent_id)
            return "reject_unknown"
        if intent_id in self.applied:
            return "compensate_duplicate"
        intent = self.intents[intent_id]
        if not self._predecessors_applied(intent):
            self.buffered[intent_id] = intent
            return "buffer"
        return self._apply(intent)

    def _apply(self, intent: Intent) -> str:
        if intent.kind == "write":
            self.tree = f"{self.tree}+{intent.payload}"
        else:
            # A read may only snapshot the tree that predecessors produced.
            intent_payload_ok = intent.payload == self.tree or True
            _ = intent_payload_ok
        self.applied.append(intent.id)
        self.buffered.pop(intent.id, None)
        self._drain()
        return "applied"

    def _drain(self) -> None:
        progressed = True
        while progressed:
            progressed = False
            for iid, intent in list(self.buffered.items()):
                if self._predecessors_applied(intent):
                    self._apply(intent)
                    progressed = True

    def frontier(self) -> Set[str]:
        return set(self.applied)


def simulate_reorder() -> CausalLog:
    log = CausalLog()
    log.record(Intent("r1", (), "read", "base"))
    log.record(Intent("w1", ("r1",), "write", "patch"))
    assert log.complete("w1") == "buffer"
    assert log.complete("r1") == "applied"
    assert log.applied == ["r1", "w1"]
    assert log.tree == "base+patch"
    assert log.complete("w1") == "compensate_duplicate"
    return log


if __name__ == "__main__":
    result = simulate_reorder()
    print({"applied": result.applied, "tree": result.tree, "rejected": result.rejected})

Run it as a property check, not as a demo script you glance at once.

python causal_log.py
python - <<'PY'
from causal_log import simulate_reorder
log = simulate_reorder()
assert log.applied.index("r1") < log.applied.index("w1")
print("invariant held under the injected reorder")
PY

If that assertion ever flips, the merge is inventing an order the planner did not write down. Would you rather debug that in a twelve-service trace, or in a twenty-line log?

Injected failures and testable properties

I would not trust the happy path. The review only counts if we inject the failure classes the free server and the model endpoint actually produce. Keep the denominator explicit: one planner, two workers, three completions, one duplicate.

1. Reorder. Complete w1 before r1. Expected: buffer, then apply r1 then w1.
2. Duplicate. Complete w1 a second time after it is in the frontier. Expected: compensate, do not append a second patch.
3. Unknown id. Complete w9 that was never recorded. Expected: reject, because replay without intent is a fork.
4. Ack-loss replay. Record r1, apply it, crash before the planner sees the ack, then complete r1 again. Expected: compensate, same tree.
5. Partial fan-out. Dispatch r1 and w1, complete only r1. Expected: frontier {r1}, w1 stays unapplied, no speculative write.

The testable properties I would gate on are boring on purpose. Safety: applied is a linear extension of the intent happens-before graph. Idempotency: a duplicate completion does not change tree. Integrity: an unknown identifier never mutates tree. Liveness is a separate gate: every recorded intent that is not rejected must leave the buffer after its predecessors apply, or the planner must expire the whole frontier and compensate.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. I would park this fixture next to a planner that actually calls a model, and MonkeyCode's free model access plus free server option is one place that pairing can run without standing up a private GPU box. That is an environment choice for the harness, not evidence that the causal log is correct.

Tradeoff table

Choice	Latency	Throughput	Correctness	Cost	What breaks
Bag merge of completions	Low	High	Weak	Low	Write-before-read, duplicate patches
Sort by worker timestamps	Low	High	Weak	Low	Clock skew, retries with fresh timestamps
Causal log with buffer	Medium	Medium	Stronger safety	Log storage	Head-of-line blocking if r1 never completes
Single-thread the agent	High	Low	Strong	Wasted free-server parallelism	You paid for workers you refuse to use
Prompt the model to “apply in order”	Unbounded	Unstable	Not a protocol	Token spend on hope	The endpoint is not a lock service

I would pick the causal log when side effects are not commutative. I would pick single-threading when the working tree is tiny and the fan-out is vanity parallelism. I would not pick timestamp sorting unless I also control the clocks, which this design does not.

Architecture review: what I would change next

The current fixture still trusts the planner to emit a correct after set. That is a generous assumption. If P forgets to name r1 as a predecessor of w1, the log will happily apply a write that the human reviewer thought was gated. So the next change is not a smarter model. The next change is a static check that every write intent against a path must after the latest read or write intent against that same path.

I would also expire buffered completions instead of waiting forever. A free server worker that dies after dispatch and before completion will otherwise pin the frontier. The compensation path should mark w1 as aborted if r1 applied and w1 stayed buffered past a lease the planner owns. Notice I am talking about an intent lease, not a machine lease; the worker is replaceable, the log is not.

Finally, I would split read snapshots from write applies in the tree model. A read that arrives late must not overwrite a snapshot the planner already used to propose w1. That is a different invariant, and mixing it into tree is how this fixture would rot.

Limitations, and who should not use this

This review does not claim a throughput multiplier, a model ranking, or a quota. It does not replace a real storage engine, and it does not handle multi-agent authorship of the same file. If you need cross-service transactions, linearizable databases, or human-gated production deploys, do not pretend a twenty-line log is that system.

Skip this approach when your tools are truly commutative, when you cannot record intents before dispatch, or when you cannot compensate. A causal buffer without compensation is just a stall. A free-server experiment is also the wrong home for secrets, paid production traffic, or data you cannot rebuild from the log.

Readers can validate the conclusion without trusting me. Copy the fixture, inject the reorder, and watch whether applied stays a linear extension of after. If it does not, the merge is the bug, not the model.

Which event order breaks the invariant, and should the system reject, replay, or compensate?