The Agent That Mapped AbortError to an Empty List: A Take-Home Packet

Agent-written tool adapters often convert a canceled or timed-out call into an empty array, which hides failure from every caller. Downstream list views then render "no results found," and operators lose the only signal that the tool never ran. This take-home packet asks a candidate to review that adapter, write a failing test, and restore an error path that preserves abort semantics. Teams can score the packet with a short written rubric instead of debating subjective model taste.

Why this defect survives review

Hiring loops still reward green tests more than honest failure modes, especially when agents generate both the implementation and the suite. An empty array looks tidy in snapshots, TypeScript types, and UI states that already handle zero hits. AbortError and timeout errors feel like infrastructure noise, so generated comments describe the mapping as a user-friendly fallback. The result is a false-green merge that only breaks in production when latency spikes or a reviewer cancels a run.

The take-home prompt

Give the candidate the following prompt without extra product framing, vendor names, or unused model names. The text below is the assignment itself, and interviewers should paste it without adding extra success criteria. Time stays at forty-five minutes so candidates cannot hide the defect behind a large unrelated refactor.

Prompt (paste as-is)

You are reviewing a pull request from a coding agent.

Repository: a Node 20 TypeScript service that searches internal docs
through a single tool adapter, `searchDocs`.

Observed production symptom: users report empty search results during
deploys and when they cancel an in-flight request. Metrics show the
upstream tool still returns 200s when it is reached. Several empty
responses coincide with client disconnects and with a 1500ms gateway
timeout.

Your job:
1. Identify the incorrect failure mapping in `src/tools/searchDocs.ts`.
2. Add or fix tests under `src/tools/searchDocs.test.ts`.
3. Restore abort and timeout as errors, not as empty lists.
4. Keep successful empty searches (zero hits) distinct from failures.
5. Do not add retries, caches, or a new HTTP client unless needed.

Constraints:
- Do not weaken types with `any`.
- Do not log query text; it may contain customer content.
- Time budget: 45 minutes.

The packet includes a small runnable tree of files rather than a screenshot of a chat transcript. Candidates should install dependencies only from the listed manifest, then run the existing Jest file first. Reading comments before running tests is allowed, though the false-green suite is part of the trap.

Fixture tree the candidate receives

• packet/package.json
• packet/tsconfig.json
• packet/src/tools/searchDocs.ts
• packet/src/tools/searchDocs.test.ts
• packet/src/tools/upstream.ts
• packet/README.md

The agent-written code under review

The generated adapter swallows AbortError inside a bare catch block and then returns an empty hits array. That mapping is the defect under review, not the presence of AbortController or the default timeout value. Interviewers should leave the agent comments in place, because those comments encode the mistaken product story. The TypeScript types still say Promise of DocHit array, which makes the empty fallback type-check without protest.

// src/tools/searchDocs.ts
import { fetchUpstream } from "./upstream";

export type DocHit = { id: string; title: string };

export async function searchDocs(
  query: string,
  opts: { signal?: AbortSignal; timeoutMs?: number } = {}
): Promise<DocHit[]> {
  const timeoutMs = opts.timeoutMs ?? 1500;
  const controller = new AbortController();
  const timer = setTimeout(() => controller.abort(), timeoutMs);

  if (opts.signal) {
    if (opts.signal.aborted) {
      controller.abort();
    } else {
      opts.signal.addEventListener("abort", () => controller.abort(), {
        once: true,
      });
    }
  }

  try {
    const result = await fetchUpstream(query, { signal: controller.signal });
    return result.hits ?? [];
  } catch {
    // Agent comment: keep the UI usable when search is slow or canceled.
    return [];
  } finally {
    clearTimeout(timer);
  }
}

The accompanying test suite currently passes for the wrong reason and should not be treated as specification. The cancel case asserts resolved empty arrays, which teaches the next agent to preserve the production bug. A candidate who only reads that second test name will confirm the agent behavior instead of challenging it.

// src/tools/searchDocs.test.ts
import { searchDocs } from "./searchDocs";
import * as upstream from "./upstream";

jest.mock("./upstream");

test("returns hits from upstream", async () => {
  jest.spyOn(upstream, "fetchUpstream").mockResolvedValue({
    hits: [{ id: "1", title: "Install guide" }],
  });
  await expect(searchDocs("install")).resolves.toEqual([
    { id: "1", title: "Install guide" },
  ]);
});

test("returns an empty list when the user cancels", async () => {
  const ac = new AbortController();
  jest.spyOn(upstream, "fetchUpstream").mockImplementation(async ({ signal }) => {
    return new Promise((_, reject) => {
      signal?.addEventListener("abort", () => {
        const err = new Error("This operation was aborted");
        err.name = "AbortError";
        reject(err);
      });
    });
  });
  const pending = searchDocs("install", { signal: ac.signal });
  ac.abort();
  await expect(pending).resolves.toEqual([]);
});

Scoring rubric

Interviewers should score visible evidence in the diff and notes rather than writing style or extra refactors. Each row in the table is worth two points, and the passing bar is seven points with a nonzero test-design score. Partial credit exists so a candidate who names the defect but ships a coarse Error still receives a usable signal.

Criterion	0	1	2
Names the empty-list mapping as the defect	Misses it	Mentions cancel only	Names cancel and timeout, plus zero-hit success
Test design	Keeps the false-green test	Adds one abort assertion	Splits abort, timeout, and legitimate empty hits
Error type	Throws a generic Error	Throws AbortError or timeout inconsistently	Preserves AbortError; uses a distinct timeout error
Logging and types	Logs the query or uses any	Avoids both	Avoids both and does not leak the query in messages
Scope control	Adds retries or a new client	Small extra cleanup	Changes only the failure path

Sample solution

The following repair is a labeled sample solution, not a claim that only one error type can be correct. Interviewers should accept equivalent timeout errors when cancel still surfaces as AbortError and zero hits stay successful. Retries, caches, and a replacement HTTP client remain out of scope unless a candidate can justify a one-line helper.

// src/tools/searchDocs.ts (sample repair)
import { fetchUpstream } from "./upstream";

export type DocHit = { id: string; title: string };

export class SearchTimeoutError extends Error {
  constructor(timeoutMs: number) {
    super(`searchDocs timed out after ${timeoutMs}ms`);
    this.name = "SearchTimeoutError";
  }
}

export async function searchDocs(
  query: string,
  opts: { signal?: AbortSignal; timeoutMs?: number } = {}
): Promise<DocHit[]> {
  const timeoutMs = opts.timeoutMs ?? 1500;
  const controller = new AbortController();
  let timedOut = false;
  const timer = setTimeout(() => {
    timedOut = true;
    controller.abort();
  }, timeoutMs);

  if (opts.signal) {
    if (opts.signal.aborted) {
      controller.abort();
    } else {
      opts.signal.addEventListener("abort", () => controller.abort(), {
        once: true,
      });
    }
  }

  try {
    const result = await fetchUpstream(query, { signal: controller.signal });
    return result.hits ?? [];
  } catch (err) {
    const aborted =
      err instanceof Error && err.name === "AbortError";

    if (aborted && timedOut && !opts.signal?.aborted) {
      throw new SearchTimeoutError(timeoutMs);
    }
    if (aborted) {
      const abortErr = new Error("searchDocs aborted");
      abortErr.name = "AbortError";
      throw abortErr;
    }
    throw err;
  } finally {
    clearTimeout(timer);
  }
}

After the repair, tests must separate legitimate zero-hit searches from cancel paths and from timer-fired timeouts. Fake timers keep the timeout case deterministic, but they do not replace a later live delayed-upstream check. Candidates may use Node test runners other than Jest when assertions still encode the three distinct outcomes.

test("keeps a legitimate zero-hit search as an empty array", async () => {
  jest.spyOn(upstream, "fetchUpstream").mockResolvedValue({ hits: [] });
  await expect(searchDocs("zzzz")).resolves.toEqual([]);
});

test("rejects with AbortError when the caller cancels", async () => {
  const ac = new AbortController();
  jest.spyOn(upstream, "fetchUpstream").mockImplementation(async ({ signal }) => {
    return new Promise((_, reject) => {
      signal?.addEventListener("abort", () => {
        const err = new Error("aborted");
        err.name = "AbortError";
        reject(err);
      });
    });
  });
  const pending = searchDocs("install", { signal: ac.signal });
  ac.abort();
  await expect(pending).rejects.toMatchObject({ name: "AbortError" });
});

test("rejects with SearchTimeoutError when the timer fires first", async () => {
  jest.useFakeTimers();
  jest.spyOn(upstream, "fetchUpstream").mockImplementation(async ({ signal }) => {
    return new Promise((_, reject) => {
      signal?.addEventListener("abort", () => {
        const err = new Error("aborted");
        err.name = "AbortError";
        reject(err);
      });
    });
  });
  const pending = searchDocs("install", { timeoutMs: 20 });
  await jest.advanceTimersByTimeAsync(20);
  await expect(pending).rejects.toBeInstanceOf(SearchTimeoutError);
  jest.useRealTimers();
});

Local replay should stay boring, because extra tooling often becomes a place to hide incomplete assertions. The commands below assume Node 20, a lockfile from npm install, and Jest invoked on a single test file. Interviewers can capture stdout in the score sheet if a candidate changes scripts without documenting the new command.

cd packet
npm install
npx jest src/tools/searchDocs.test.ts --runInBand

Common failure modes among candidates

Reviewers of agent output fail this packet in recurring ways that are more informative than the final patch size. Interviewers should read the notes beside the diff, because several failure modes leave a plausible looking green suite. The numbered list below is a scoring aid, not a script for interrupting a candidate during the forty-five minutes.

1. The candidate rewrites comments but leaves the cancel case resolving to an empty array, which re-encodes the production bug as specification.
2. Timeout and user cancel collapse into one generic Error, so the gateway cannot choose 504 versus 499 responses.
3. Extra retries ignore AbortSignal and keep calling upstream after the client left, which repeats the original defect class.
4. A debug console.error that prints the query leaks customer text into shared logs during the interview environment run.
5. Treating a missing hits field as failure punishes valid zero-result payloads that omit the key after a successful upstream response.
6. Replacing global fetch hides adapter logic the rubric scores, even though the packet already isolates fetchUpstream for this reason.
7. Some candidates remove timeoutMs entirely to make tests pass, which drops a production constraint the written prompt still requires.

Running the packet without extra staging machines

A reviewer can replay the fixture in any Node 20 environment that already provides Jest and TypeScript packages. Teams without spare hosts sometimes place the packet on MonkeyCode, which offers free model access and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The product is not required to grade the rubric, and this article does not claim model names, token quotas, hardware sizes, or uptime.

The useful workflow stays small and disposable so leftover processes cannot leak search queries between candidates. Each step below should take minutes, and none of them depend on a particular hosted model or vendor UI. After scoring, the workspace should be deleted, including any logs that might have captured fixture query strings.

1. Copy the packet directory into a clean workspace that does not reuse prior candidate node_modules or env files.
2. Run the supplied Jest file before reading the agent comments so the false-green cancel test is observed first.
3. Record that cancel currently asserts success with an empty array, then stop changing tests until the mapping is named.
4. Patch the adapter, then add the three tests that separate zero hits, AbortError, and the timer-based timeout error.
5. Record rubric scores on the sheet, then delete the workspace after the loop so fixture queries do not linger.

Limitations

This packet measures one failure class: mapping cancel and timeout onto a successful empty collection of documents. It does not measure distributed tracing, search ranking quality, or prompt-injection resistance in the upstream tool adapter. Fake timers can hide real race conditions, so a follow-up live test with delayed fetchUpstream remains worthwhile after hiring. The sample timeout error is application-specific and should not be confused with a standard DOM TimeoutError constructor.

Organizations should not use this packet as a general coding screen for frontend layout work, CSS, or people-management roles. Teams that cannot accept synthetic search queries, even without customer data, should select a different review exercise entirely. Candidates who have never seen AbortSignal may need a shorter warm-up, because the packet assumes that vocabulary from the start. Pairing this packet with a system-design interview still makes sense when the role owns gateway timeouts and client disconnects.

The core conclusion remains narrow and operational for teams that already ship agent-written TypeScript service adapters. Empty lists are valid search results, and they are not a valid encoding of abort, timeout, or transport failure. Interviewers who score that distinction will reject a common agent shortcut before it reaches production search traffic.