Two Free Nodes, One Agent: An Architecture Review You Can Reproduce

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The event order that made me stop trusting the happy path went like this: the agent called the free model endpoint, the acknowledgement was lost on the way back, the agent retried, and the second attempt consumed a fresh slice of the token allowance. In the same minute, the free server that held the agent's working state was recycled between the two calls, so the retry resumed from a checkpoint that no longer existed. Two nodes, two failure domains, one agent, and nothing in the middle coordinating them.

An architecture review should begin with exactly that sentence: a free model endpoint and a free server are not conveniences, they are two independently failing machines that happen to share one workload. This article is my review of that stack. MonkeyCode is an open-source project, and the offer I reviewed is its free model access — a 10M-token allowance at the time of writing — plus the free server option. You can decide for yourself whether the free tier is worth it; my job here is the inspection, not the sales pitch.

What I actually reviewed

The review subject is an AI agent that leans on two external resources: a free model endpoint and a free server acting as the executor. That is the smallest distributed system I know that still carries all the failure modes of a large one. Before drawing anything, I wrote down my assumptions, because a review without declared assumptions is just an opinion with a pen.

Constraint	Working assumption
Free model endpoint	Shared, quota-bounded capacity; no SLO; may throttle, time out, or lose an ack
Free server	Single recyclable instance; local disk may not survive recycling; cold start after idle
Network	Best-effort delivery; retries can duplicate a request
Token accounting	Allowance is consumed per attempt; a retry can double-count unless guarded
Your homework	Verify the current numbers before you plan capacity around any of this

Does that table match your environment? If your free server actually sits on persistent disk, or your endpoint is fronted by a proxy that deduplicates, some of the failures below shrink. The point of the table is to make those differences visible instead of implicit.

The data flow, drawn honestly

Most agent diagrams show one happy arrow from prompt to completion. Here is the flow I traced, including the edges that fail:

Agent driver          Free server (executor)       Free model endpoint
     |  checkpoint(state)      |                         |
     |------------------------>|                         |
     |  run(tool_call)         |                         |
     |------------------------>|                         |
     |                          |  prompt + context     |
     |                          |----------------------->|
     |                          |  completion / 429/t/o  |
     |                          |<-----------------------|
     |  result + new state      |                         |
     |<-------------------------|                         |

Each box is a failure domain, and each arrow is a retry boundary. The three boxes can fail in different orders, and the agent's correctness depends on which order actually happens. That is why the diagram comes before the solution: you cannot design a recovery until you can name the edge that failed.

The failure domains, one by one

Domain	Failure mode	What the agent sees	The tricky part
Free server	Recycled mid-run, disk wiped, cold start	Lost working state, slow first call	The client may not realize the executor identity changed
Free endpoint	Throttle, timeout, ack loss	Delayed or duplicated completion	Retrying is natural and dangerous at the same time
Network	Partial failure	Either or both of the above	Nobody knows which side actually failed
Token ledger	Allowance drift	Startling 429s near the budget boundary	Accounting must run on the driver, not inside the server

The pattern across all four rows is the same: the failure is not the crash, it is the ambiguous state after the crash. A recycled server is harmless if the driver holds the state; a lost ack is harmless if the driver can tell a retry from a double-execution.

The one change I would make next

After the review, I would add exactly one thing: a resume boundary between the driver and the two free nodes. It is a four-step protocol:

1. Checkpoint — the driver writes the agent state before every tool call.
2. Commit — the driver records the intent with a unique attempt_id.
3. Execute — the server runs the call; it no longer needs to survive alone.
4. Reconcile — after any failure, the driver compares what was attempted against what was acknowledged.

Here is the minimal fixture I use to test the boundary, a two-node failure simulator that fits in one file:

# review_hook.py — tiny two-node failure simulator for the review
import json
import pathlib
import random
import uuid

STATE_PATH = pathlib.Path("/tmp/agent_state.json")  # pretend this is the free server disk
ATTEMPT_IDS = set()

def checkpoint(state: dict) -> None:
    STATE_PATH.write_text(json.dumps(state))

def call_endpoint(prompt: str, attempt_id: str) -> dict:
    # pretend the free endpoint can throttle or lose an ack
    if random.random() < 0.1:
        raise TimeoutError("endpoint did not ack in time")
    if attempt_id in ATTEMPT_IDS:
        raise RuntimeError("same attempt executed twice — side effects may repeat")
    ATTEMPT_IDS.add(attempt_id)
    return {"answer": "ok", "attempt": attempt_id}

def run_tool_call(tool: str, state: dict) -> dict:
    attempt_id = str(uuid.uuid4())
    checkpoint(state)
    try:
        result = call_endpoint(tool, attempt_id)
    except TimeoutError:
        return {"status": "replay", "attempt": attempt_id}
    return {"status": "done", "result": result}

Then I run three checks, and every one of them must pass before I trust the stack:

1. Kill the executor process mid-call; the driver must resume from the checkpoint, not from memory.
2. Replay the same attempt_id; the ledger must refuse the second execution.
3. Force the endpoint timeout; the token accounting must record the attempt once, no matter how many retries follow.

If any check fails, the failure-domain review was not an exercise — it predicted the incident before the second user arrived.

Where the durable state should live

Option	Resume speed	Operational cost	What breaks
Free server disk	Fast, local	None	Everything, when the server is recycled
External object store	Slower, network hop	You manage credentials and buckets	Very little, but you now run infrastructure
Driver-side journal	Fastest, single writer	The client becomes the state holder	Client loss means state loss
No durable state	Instant, until something fails	None	The whole agent, permanently

My default for this stack is the driver-side journal with the attempt ledger, because it matches the weakest link: when both free nodes are unreliable, the only node left is the one running your code.

Who should not use this approach

If your workload needs a hard SLO, regulated data residency, or sub-second tool latency, do not build on the free tier, and do not take this review as permission to try. The checklist is what transfers to your real stack; the free resources are the laboratory. And if you just want a push-button deployment, this whole article is overkill for you.

The review, in one sentence

A free model endpoint and a free server are two independent failure domains, and the only thing that keeps your agent correct across both is a driver-side resume boundary that survives either of them dying.

Now the counterexample question I ask about every stack I review: take your own agent and list the event orders that would break it — server recycled mid-tool-call, endpoint retried after ack loss, or allowance drift at the budget boundary. Which one happens first for you, and should your agent reject, replay, or compensate? If you run this checklist against your own setup and something surprising falls out, I read every reply.