The Agent That Acknowledged Before the Side Effect: A Take-Home Packet

Agent-written queue consumers often acknowledge a message before the side effect commits, which converts at-least-once delivery into silent loss. This take-home packet asks a reviewer to catch that inversion, score a patch, and refuse cosmetic retries that leave the race intact. The packet is self-contained, uses only Node.js test runners, and does not require production brokers or paid model keys. Interviewers can drop the files into a repository and grade whether a human or an agent notices the lost-write window.

Why this failure keeps surviving review

Large language models generate plausible consumer loops that look operationally complete because they log, retry, and catch exceptions. Those loops still tend to treat acknowledgment as cleanup rather than as a durability barrier after a committed write. The resulting worker passes happy-path tests, then drops work when a process dies between ack and the database insert. Debate about agents replacing junior developers rarely inspects this ordering bug, so a paper packet remains a practical filter.

A second pattern appears when the agent fixes duplicates by hashing the payload and skipping later deliveries. Payload hashes collide across event types, and they also hide legitimate retries after a failed side effect that never reached storage. Reviewers who only demand retries, without an idempotency key tied to the broker delivery identity, will ship both loss and double application. The rubric below scores those mistakes as separate failures rather than as comments about naming, logging, or formatting.

Candidate prompt

Share the following prompt unchanged with the candidate or the model under review, and do not attach this article. Do not attach the rubric, the sample solution, or the list of common failure modes. The prompt already states the crash model, the ledger format, and the dead-letter bound, so extra hints are unnecessary. Candidates who ask for production Kafka details are drifting; the in-memory broker is the intended boundary of the task.

You are reviewing a Node.js queue worker that consumes JSON jobs from a small in-memory broker.

Requirements:
1. Jobs are delivered at least once. Crashes between any two await points are expected.
2. Each job has a stable deliveryId from the broker and a payload.action plus payload.targetId.
3. Applying a job means appending one line to a durable ledger file:
   deliveryId<TAB>action<TAB>targetId
4. The same deliveryId must never append two ledger lines.
5. A crashed worker must be restartable. Unacked jobs return after a visibility timeout.
6. Failed side effects must not ack. Poison jobs after 5 attempts go to a dead-letter list.

Implement src/worker.js exporting createWorker({ broker, ledgerPath }) with runOnce()
and runLoop(signal). Keep the broker interface in src/broker.js. Do not add network
services. Include tests that interrupt the worker before the ledger write and prove
the job is not lost.

Invariants the reviewer must enforce

The hidden spec is short, and every item is testable without a cloud account or a paid API token. Reviewers who rewrite the broker into a real Redis client have left the assignment, because the point is message ordering. Infrastructure tourism is a common agent escape hatch and should score as a scope failure on the rubric. Keep the ledger on the local filesystem so crash injection stays deterministic across laptops and hosted runners.

1. Ack is the last successful step, not the first step, and not work that belongs in a finally block after a failed write.
2. The idempotency key is deliveryId, not a hash of the JSON body, and not targetId alone across different actions.
3. Visibility timeout restore must redeliver unacked jobs to a fresh worker process without operator intervention.
4. Dead-letter after five failed attempts must keep the payload inspectable; dropping the job to protect the loop is a fail.
5. Ledger appends stay one line per identifier, even when two runOnce() calls overlap on the same redelivered job.

Starter artifact: the agent submission

Place this file in the repository as the first artifact the reviewer must judge, and treat it as an agent submission. The receive-ack-write order looks tidy in a diff and will survive a skim that only checks for try/catch coverage. The nack call after ack cannot resurrect the job because the broker has already forgotten the delivery. A process exit on the appendFile line loses the payload with no remaining in-flight record for a later worker.

// src/worker.agent.js — submitted by an agent; do not treat as correct
import { appendFile } from 'node:fs/promises';

export function createWorker({ broker, ledgerPath }) {
  return {
    async runOnce() {
      const job = await broker.receive({ visibilityMs: 5_000 });
      if (!job) return false;

      await broker.ack(job.deliveryId);

      try {
        const line =
          `${job.deliveryId}\t${job.payload.action}\t${job.payload.targetId}\n`;
        await appendFile(ledgerPath, line, 'utf8');
      } catch (err) {
        await broker.nack(job.deliveryId);
        throw err;
      }

      return true;
    },

    async runLoop(signal) {
      while (!signal?.aborted) {
        const worked = await this.runOnce();
        if (!worked) await new Promise((r) => setTimeout(r, 25));
      }
    },
  };
}

Several agents then add exponential backoff around appendFile, which never repairs the missing broker state after a successful ack. Other agents wrap ack in a finally block so failures still dismiss the job and convert exceptions into silent drops. Both patches increase the volume of green unit tests while widening the loss window under crash injection. The crash harness below exists to make that window visible without attaching a debugger or a production tracer.

Broker double and crash harness

The broker is part of the packet so tests stay deterministic and candidates cannot hide behind a managed queue. Visibility timeouts return unacked jobs to the pending list, which models the only recovery path after a crash. Dead-letter storage is an array rather than a dashboard, which is enough to assert that poison jobs remain inspectable. Do not replace this double with SQS during the exercise, even if a model proposes that migration as a reliability upgrade.

Broker

// src/broker.js
export function createMemoryBroker() {
  const pending = [];
  const inflight = new Map();
  const dead = [];
  let seq = 0;

  function release(deliveryId) {
    const slot = inflight.get(deliveryId);
    if (!slot) return null;
    clearTimeout(slot.timer);
    inflight.delete(deliveryId);
    return slot.job;
  }

  return {
    enqueue(payload) {
      const deliveryId = `d-${++seq}`;
      pending.push({ deliveryId, payload, attempts: 0 });
      return deliveryId;
    },

    async receive({ visibilityMs = 5_000 } = {}) {
      const job = pending.shift();
      if (!job) return null;
      const timer = setTimeout(() => {
        inflight.delete(job.deliveryId);
        pending.push(job);
      }, visibilityMs);
      inflight.set(job.deliveryId, { job, timer });
      return job;
    },

    async ack(deliveryId) {
      return release(deliveryId) !== null;
    },

    async nack(deliveryId) {
      const job = release(deliveryId);
      if (!job) return false;
      job.attempts += 1;
      if (job.attempts >= 5) dead.push(job);
      else pending.push(job);
      return true;
    },

    redeliverCommitted(deliveryId, payload) {
      pending.push({ deliveryId, payload, attempts: 1 });
    },

    deadLetters() {
      return [...dead];
    },
  };
}

Crash tests

The first test throws from a hook placed immediately before the ledger write, which simulates a crash after the worker might have acked. A correct worker nacks or still holds the in-flight job, so a second worker can apply the payload exactly once. The second test redelivers the same deliveryId after a successful write and expects a single ledger line. Run both files with the Node.js test runner before reading any model-produced patch, and treat a skipped crash test as a failing review.

// test/crash-window.test.js
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { mkdtemp, readFile } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { createMemoryBroker } from '../src/broker.js';
import { createWorker } from '../src/worker.js';

async function ledger(dirName) {
  const dir = await mkdtemp(join(tmpdir(), dirName));
  return join(dir, 'ledger.log');
}

function linesOf(body) {
  return body.trim().split('\n').filter(Boolean);
}

test('a crash before the ledger write cannot drop a job', async () => {
  const ledgerPath = await ledger('ledger-crash-');
  const broker = createMemoryBroker();
  broker.enqueue({ action: 'issue-invoice', targetId: 'acct-22' });

  const crashing = createWorker({
    broker,
    ledgerPath,
    hooks: {
      async beforeLedgerWrite() {
        throw new Error('simulated crash');
      },
    },
  });

  await assert.rejects(() => crashing.runOnce(), /simulated crash/);

  const revived = createWorker({ broker, ledgerPath });
  await revived.runOnce();

  const body = await readFile(ledgerPath, 'utf8');
  assert.equal(linesOf(body).length, 1);
  assert.match(linesOf(body)[0], /issue-invoice\tacct-22/);
  assert.equal(broker.deadLetters().length, 0);
});

test('redelivery of a committed deliveryId does not double-append', async () => {
  const ledgerPath = await ledger('ledger-dedupe-');
  const broker = createMemoryBroker();
  const worker = createWorker({ broker, ledgerPath });
  const deliveryId = broker.enqueue({
    action: 'issue-invoice',
    targetId: 'acct-22',
  });

  await worker.runOnce();
  broker.redeliverCommitted(deliveryId, {
    action: 'issue-invoice',
    targetId: 'acct-22',
  });
  await worker.runOnce();

  const body = await readFile(ledgerPath, 'utf8');
  assert.equal(linesOf(body).length, 1);
});

Command

node --test test/crash-window.test.js
# optional: fail if the agent file is still wired as worker.js
diff -q src/worker.js src/worker.agent.js && exit 1 || true

Scoring rubric

Grade on observable behavior, not on whether the prose explanation sounds like a senior engineer who has operated queues. A passing packet must keep the job across a crash, reject payload hashing as the idempotency key, and bound retries. Style-only comments without a failing test mapped to an invariant should not raise the score. The table compresses those rules into four outcomes that interviewers can mark in a single pass.

Observation after node --test	Likely cause in the patch	Score
Crash hook runs, ledger empty, job gone from pending and inflight	Ack happened before the write	Fail invariant 1
Two ledger lines share one deliveryId	Missing or body-hash idempotency	Fail invariant 2
Crash hook runs, job later appears once	Write (or durable intent) then ack	Pass invariants 1–2
Five forced write failures, deadLetters() empty	Swallowed poison or ack-on-error	Fail invariant 4
Patch adds Redis, HTTP, or a new queue product	Scope escape	Fail, regardless of tests

Thirty-minute grading path

Interviewers can complete a first pass in about thirty minutes if the repository already contains the broker and the crash tests. Start by running the tests against the agent file, then replace worker.js and run the same command again. Next, search the patch for payload hashes, finally-block acks, and new network clients that were not requested. Finish by reading the dead-letter path and confirming poison jobs remain after five failed side effects.

1. Run node --test test/crash-window.test.js on worker.agent.js and record the exact assertion that fails.
2. Apply the candidate patch only to src/worker.js, leaving src/broker.js untouched unless a bug in the double is proven.
3. Re-run the same command and require both tests green without skipped assertions.
4. rg "createHash|digest\(|finally \{|fetch\(|ioredis" src/worker.js and mark matches against the table.
5. Force beforeLedgerWrite to throw five times and assert broker.deadLetters().length === 1.

Sample solution

The sample worker loads known delivery identifiers from the ledger, writes the line, and only then acknowledges the job. The beforeLedgerWrite hook is a test seam, not a production feature, and it must run before durability is claimed. On any throw, the worker nacks so the visibility timer is cleared and the job returns with an incremented attempt count. Duplicate delivery identifiers ack without a second append, which closes the redelivery path after a successful commit.

// src/worker.js — sample solution for graders, not for candidates
import { appendFile, readFile } from 'node:fs/promises';

async function loadSeen(ledgerPath) {
  try {
    const body = await readFile(ledgerPath, 'utf8');
    return new Set(
      body
        .split('\n')
        .filter(Boolean)
        .map((line) => line.split('\t')[0]),
    );
  } catch (err) {
    if (err.code === 'ENOENT') return new Set();
    throw err;
  }
}

export function createWorker({ broker, ledgerPath, hooks = {} }) {
  return {
    async runOnce() {
      const job = await broker.receive({ visibilityMs: 5_000 });
      if (!job) return false;

      const seen = await loadSeen(ledgerPath);
      if (seen.has(job.deliveryId)) {
        await broker.ack(job.deliveryId);
        return true;
      }

      try {
        if (hooks.beforeLedgerWrite) {
          await hooks.beforeLedgerWrite(job);
        }
        const line =
          `${job.deliveryId}\t${job.payload.action}\t${job.payload.targetId}\n`;
        await appendFile(ledgerPath, line, 'utf8');
        await broker.ack(job.deliveryId);
        return true;
      } catch (err) {
        await broker.nack(job.deliveryId);
        throw err;
      }
    },

    async runLoop(signal) {
      while (!signal?.aborted) {
        try {
          const worked = await this.runOnce();
          if (!worked) await new Promise((r) => setTimeout(r, 25));
        } catch {
          // nack already recorded the attempt; loop continues
        }
      }
    },
  };
}

What these tests actually prove

The crash test proves the job remains recoverable when the write never happens, which is the lost-message case reviewers miss during a diff skim. The redelivery test proves a completed write is not applied twice when the broker honours at-least-once semantics after a commit. Together they reject the two cheap patches agents prefer: ack-first cleanup and body-hash skipping. They do not prove multi-process file locking, which the limitations section leaves out of scope on purpose.

Common failure modes

The following failures appear repeatedly when models iterate on the starter file without the crash test in context. Each item should be independently marked, because a patch can fix loss and still double-apply on redelivery. Interviewers should paste the matching invariant number next to the comment so candidates cannot treat the list as style advice. Models that rewrite the prompt into a microservice diagram have not addressed any of these modes.

1. Ack then write. The starter file does this openly, and many patches keep the order while adding logs around appendFile.
2. Ack in finally. Exceptions look handled, yet the broker forgets the job and the ledger stays empty after a crash.
3. Payload SHA as the key. Two issue-invoice jobs for different accounts can collide after whitespace normalization, and retries after a failed write never run.
4. targetId as the key. A later void-invoice for the same account is treated as a duplicate and dropped.
5. In-memory Set of seen identifiers. A process restart replays committed jobs or, worse, acks them without a ledger line.
6. Infinite retry with no dead-letter. The loop looks resilient while poison payloads pin a worker forever.
7. Ack on unknown methods. The agent invents broker.retry() after a successful ack, which the double does not implement.
8. Broker replacement. The model spends the session wiring Redis or SQS and never closes the crash window in runOnce().

Limitations and who should skip this packet

This packet does not prove a model is safe for production consumers, and it does not rank vendors or claim benchmark scores. The in-memory broker omits fair polling, poison-pill headers, tracing, and multi-region failover, which real systems still need. Teams with an existing exactly-once outbox should not replace that outbox with this ledger file. People looking for a general coding-agent leaderboard, a hiring take-home about frontend state, or an incident runbook should skip this packet.

The sample worker still has a narrow race if two processes pass loadSeen before either appends, because the ledger is a text file without an exclusive lock. That gap is acceptable for an interview filter and unacceptable as a payment ledger. Graders who need that extra bar should add a lockfile test rather than inflate this prompt with a database. Keep changes of that kind in a follow-up packet so scores on this one stay comparable.

Using a free reviewer loop

Disclosure: This article was prepared as part of MonkeyCode's product outreach. Teams that already run this packet locally can point an AI reviewer at the failing crash test using MonkeyCode's free model access and free server option. The free server option can run the same node --test command so the reviewer compares the patch against the rubric. This article does not claim quota size, model names, duration, or lasting availability for that free access option.

The useful outcome is a reviewer who refuses ack-before-write even when the surrounding loop looks modern and well logged. Keep the packet frozen across candidates so scores remain comparable, and change only the payload action names if leakage is a concern. Ordering bugs of this kind will keep showing up in agent-written workers regardless of how fluent the generated comments become. A small ledger and a crash hook still catch them after the blog posts about autonomous coding have moved on.