Loops, Cron, and Idempotency: Agents That Operate While You Sleep

In October 2025, I set up a cron that fired the writer every 15 minutes. The lock was a file: existsSync('queue.lock') before writing, writeFileSync('queue.lock', pid) if free. Two ticks entered the window between the existsSync and the writeFileSync. Both saw the lock as free, both wrote, both ran. The queue ended up with two conflicting drafts and a corrupted state file pointing to both. I spent the morning cleaning it up.

existsSync followed by writeFileSync is a race condition with a friendly name. Filesystem locks need to be atomic at the syscall level, open(O_EXCL) or rename-after-write, not at the application level.

The real value of an agentic system comes when the agent operates without you present: processing a queue while you sleep, monitoring logs while you're in meetings. For this to work, three things need to be right: the loop, the scheduling, and the idempotency.

The three loop modes

Fixed cron: the agent runs at predefined intervals. Each tick is a complete execution from start to finish. It terminates. It waits for the next trigger. State between ticks lives on disk. Good for predictable periodic tasks: backup, sync, reporting.

Watch mode: the agent wakes up when something changes, via inotifywait, an MQTT subscriber, or a webhook receiver. Reacts to events. Good for event-driven tasks requiring low latency.

Pull loop: the agent runs in an infinite loop with a sleep at the end. Checks a condition. If it applies, executes. If not, sleeps. Good for queue workers, external API polling, supervision.

Cron is the simplest but has a latency window. Watch is the most responsive but requires event infrastructure. Pull is flexible but burns CPU and tokens while running in vain.

Cron via PM2

For loop scheduling, PM2 is the standard tool: simpler than system cron, with centralized logs and process supervision.

// ecosystem.config.cjs
module.exports = {
  apps: [{
    name: "content-ralph",
    script: "bash",
    args: ["-lc", "npx tsx ralph/tick.ts"],
    cron_restart: "*/15 * * * *",  // every 15 minutes
    autorestart: false,             // don't restart on crash
  }],
};

cron_restart is the key: PM2 kills the process on every cron trigger and relaunches it. Each tick is a fresh process with clean state. autorestart: false prevents a crash from becoming a crash-restart-crash loop.

Lock files: the second problem

You've configured cron every 15 minutes. A tick normally takes 12 minutes. One day it takes 18 because of a slow external API. The cron fires the next tick before the previous one finishes. Two instances running simultaneously. Race condition. Possibly double-charging the customer.

Solution: lock file with TTL. Each tick checks whether another instance is already running:

import { existsSync, statSync, writeFileSync, unlinkSync } from "node:fs";

const LOCK = "/tmp/content-ralph.lock";
const LOCK_TTL_MS = 30 * 60 * 1000;  // 30min

if (existsSync(LOCK)) {
  const age = Date.now() - statSync(LOCK).mtimeMs;
  if (age < LOCK_TTL_MS) {
    console.log("busy (age=" + age + "ms), skipping");
    process.exit(0);
  }
  console.log("stale lock (age=" + age + "ms), removing");
  unlinkSync(LOCK);
}

writeFileSync(LOCK, String(process.pid));

try {
  await runTick();
} finally {
  unlinkSync(LOCK);
}

The lock has a TTL: if a tick crashed without cleaning up, the next one detects the stale lock and removes it. The TTL should be longer than the maximum expected tick duration and shorter than the cron interval.

Idempotency: the golden rule

Lock prevents two simultaneous instances. Idempotency prevents damage if the agent reprocesses an item.

Scenario: you're processing a queue of emails to send. The tick crashed in the middle, some sent, some not. The lock released. The next tick starts. How does it avoid resending the ones already sent?

The wrong answer: "in-process memory with sent IDs." Memory disappears when the process dies.

The right answer: idempotent operations. Every operation must be safe to repeat. Two main patterns:

Status marker: each item has a status field (pending, processing, done). The tick atomically updates to "processing" when it picks up an item. When complete, updates to "done." If it crashed on "processing," the next tick decides: reprocess or mark as failed.

Hash-based dedup: compute a hash of the content and destination before sending. Check whether the hash was already processed. If yes, skip. If not, process and mark the hash.

async function sendEmail(item: QueueItem) {
  const hash = sha256(item.recipient + item.body + item.subject);
  if (await wasSent(hash)) {
    console.log("already sent: " + hash + ", skipping");
    return;
  }
  await mailgun.send(item);
  await markSent(hash);
}

Every operation that leaves the process (sends email, writes to database, calls external API) must be checked for idempotency before executing. Even if your logic seems impossible to repeat, it will repeat eventually.

Transactional workspaces

A pattern for multi-step operations: a workspace that becomes "atomic," either everything happened or nothing.

workspace/
+-- <task-id>/
    +-- status          -- marker (pending/processing/done/failed)
    +-- input.json      -- original task
    +-- step-1.out      -- step 1 output
    +-- step-2.out      -- step 2 output
    +-- final.out       -- only exists if everything succeeded

If it crashes between step 1 and step 2, the next tick sees partial files and status=processing. It decides: resume from step 2, or abort marking status=failed. No re-executing step 1.

Daily cap: protecting yourself from yourself

An infinite loop without a cap empties your token quota overnight. Two complementary caps:

const MAX_TICKS_PER_DAY = 100;
const MAX_TOKENS_PER_DAY = 1_000_000;

async function loopWithCap() {
  while (true) {
    const stats = await loadDailyStats();
    if (stats.ticks >= MAX_TICKS_PER_DAY) {
      await sleepUntilMidnight();
      continue;
    }
    if (stats.tokens >= MAX_TOKENS_PER_DAY) {
      await sleepUntilMidnight();
      continue;
    }
    await runOneTick();
    await sleep(60_000);
  }
}

Caps protect against a bug that makes the tick fire every 100ms instead of every 60 seconds. The daily cap triggers before the damage propagates.