DEV Community

Robin
Robin

Posted on

Make the Free Server a Lease-Bound Executor Before Your Agent Double-Fires

Last week, a free-tier server rebooted at 02:47. The agent running on it had been accumulating context in memory, and when the process came back, the context was gone. Worse, a model call that had already triggered an external side-effect — opening a GitHub issue — ran again, creating a duplicate. The in-memory flag that was supposed to prevent that had melted with the process.

That event taught me a simple invariant: if your AI agent runs on any disposable server, every model call that can produce a side-effect must remain idempotent under process death. Not under a normal crash, not under a network partition — under a total loss of local state.

MonkeyCode's free model access and free server option give you exactly that kind of environment: a remote model endpoint and a compute node, both without SLAs. (Disclosure: This article was prepared as part of MonkeyCode's product outreach.) That combination is where the crash-only executor pattern turns a demo into something that behaves like a service.

The data flow

Here's the architecture I used, assuming the server can vanish at any instant:

  1. An orchestrator (on reliable infrastructure) places a task into a durable queue.
  2. The free server consumes the task and tries to acquire a lease in external storage (Redis, DynamoDB, even Postgres). The lease includes the task ID and an expiry.
  3. Before calling the model, the executor writes a side-effect record with status pending to the same external store.
  4. It calls the model endpoint. The call itself is safely retryable — it just produces text.
  5. If the model result requires an external side-effect (e.g., creating an issue, sending an email), the executor applies that effect after the model call, then flips the record to done.
  6. If the server dies at any point, the lease expires. The orchestrator sees the expired lease, re-enqueues the same task ID, and a fresh executor picks it up. It checks the side-effect record: done means skip, pending means finish the finalization, absent means start from step 3.

That last check is the entire trick. The side-effect record lives outside the disposable server, so a reboot cannot erase it.

Minimal code sketch

I'm simplifying the real client for clarity. The key is that the state lives in an external key-value store, not in the process.

def run_with_lease(task_id, storage, model_call, apply_effect):
    if not storage.acquire_lease(task_id, ttl=30):
        return   # someone else owns the task
    status = storage.get_effect_status(task_id)
    if status == 'done':
        return
    if status == 'absent':
        storage.set_effect_status(task_id, 'pending')
    result = model_call(task_id)
    apply_effect(task_id, result)
    storage.set_effect_status(task_id, 'done')
Enter fullscreen mode Exit fullscreen mode

That's the heart of the pattern. The lease prevents two executors from working the same task simultaneously. The side-effect record makes the operation idempotent across process death.

Failure analysis

Let's walk through three concrete failure modes and check whether the invariant holds.

Failure point What happens Invariant preserved?
Server dies before acquiring lease Task stays in queue; orchestrator picks it up later Yes — no side-effect attempted
Server dies after pending, before model call Lease expires; new executor sees pending, re-calls model, applies effect Yes — at-most-once (the effect is applied only once)
Server dies after applying effect, before marking done Lease expires; new executor sees pending (not done), re-calls model, applies effect again No — duplicate side-effect

The third row is the dangerous one. If you write the done marker after the effect, a crash in between creates a duplicate. The fix is to make the effect itself idempotent (e.g., unique constraint on the GitHub issue title) or to move the done marker into the same atomic transaction as the effect.

Tradeoffs

This pattern has real costs, and you should know them before adopting it.

Approach Pros Cons
In-memory flag Zero latency, no external calls Lost on reboot, breaks idempotency
External lease + side-effect log Survives crashes, enables safe retries Adds latency and infrastructure complexity
Idempotent side-effects only (no record) Simplest, no extra state Requires each effect to be naturally idempotent, which is rare

For a free server that can be reclaimed at any moment, the second row is the right baseline. The first row is why I hit the duplicate issue in the opening story.

How to validate the invariant

Don't trust the pattern — test it. Here's a minimal failure-injection harness that simulates a crash at every possible point:

class SimStorage:
    def __init__(self):
        self.leases = {}
        self.effects = {}
    def acquire_lease(self, task, ttl):
        import time; now = time.time()
        if task in self.leases and self.leases[task] > now:
            return False
        self.leases[task] = now + ttl
        return True
    def get_effect(self, task):
        return self.effects.get(task, 'absent')
    def set_effect(self, task, status):
        self.effects[task] = status

def run_executor(storage, task, crash_point):
    if not storage.acquire_lease(task, 30):
        return 'skipped'
    status = storage.get_effect(task)
    if status == 'done':
        return 'already-done'
    if crash_point == 'before-model':
        raise RuntimeError('crash before model')
    storage.set_effect(task, 'pending')
    result = f'result-{task}'
    if crash_point == 'before-effect':
        raise RuntimeError('crash before effect')
    storage.effects[task] = 'applied'
    if crash_point == 'after-effect':
        raise RuntimeError('crash after effect')
    storage.effects[task] = 'done'
    return 'completed'

for crash_point in [None, 'before-model', 'before-effect', 'after-effect']:
    s = SimStorage()
    try:
        run_executor(s, 't1', crash_point)
    except RuntimeError as e:
        print(f'crash at {crash_point}: {e}')
    s.leases['t1'] = 0
    result = run_executor(s, 't1', None)
    print(f'  after retry with crash at {crash_point}: status={result}, effects={s.effects}')
Enter fullscreen mode Exit fullscreen mode

If you run this, you'll see that after-effect produces two applied entries in the log. That's the duplicate you need to design around.

What I would change next

The honest answer: I would not run this pattern without making the side-effect itself idempotent. The lease gives you at-most-once for the executor, but exactly-once for the side-effect requires either an atomic finalizer or a unique constraint in the target system. And I would add a lease-renewal loop so long model calls don't drop their lease mid-request.

Here's the counterexample question for your design: if your server applies the side-effect and then crashes before marking done, and the lease expires, will your system reject the duplicate, replay the model call, or compensate with a new operation? Answer that question before you ship it.

Top comments (0)