The cleanup script that lied about its own success

There is a particular kind of bug I've learned to fear more than a crash: the one that reports success while doing nothing.

A crash at least tells you where it hurts. This one smiles at you. A nightly cleanup job runs, prints cache cleanup complete., exits zero — and the disk fills up anyway, because the line that was supposed to decide what to delete could never match a single thing. Every run was a no-op wearing a success message. You don't find it by reading the log. The log is the problem: it's lying, and it's lying in green.

The specific shape of it is almost funny once you see it. A path-matching helper escaped a forward slash while building a regex — turned / into \/ — on the theory that it was sanitizing user input. But / isn't a regex metacharacter. Escaping it produced a pattern that required a literal backslash in the path, which never appears in a real .cache/jobs/... path. So the "is this file under the managed directory?" check returned false for everything, forever. The cleanup dutifully kept nothing, deleted nothing, and announced completion.

The lesson I keep relearning: a filter that can't fail loudly will lie to you politely forever. Before you trust any matcher, feed it a known-positive and watch it fire. A green result from an instrument you never saw go red isn't a pass — it's an untested guess with good manners.

I rebuilt it and turned an agent loose on it

Instead of just telling you the story, I did something I don't see enough of: I reconstructed the bug in a throwaway repo — a small cache-janitor with the exact defect planted on purpose — and pointed a coding agent at it cold. No hint about where the bug was, no script, no edited retakes. Just the ops complaint: "cleanup logs success but the disk keeps filling." The bug is staged; the hunt is not.

The whole session is embedded below. Watch it read the repo, reproduce the failure, isolate the over-escaped slash, fix it, and then — the part that matters — prove the fix by demonstrating the cache actually shrinking instead of trusting the success message that started the whole mess. That last move is the entire moral of the story, and it's satisfying to watch a machine arrive at it on its own.

Watch an AI agent hunt a cleanup script that lied about its own success claude-sonnet-5

You

Ops report: the nightly cache cleanup in this repo logs 'cache cleanup complete.' on every run, but the .cache volume keeps growing and it filled the disk twice this month. Find the root cause, fix it, and prove the fix actually works by demonstrating the cache shrinking. You can run 'npm run seed' to reproduce a realistic two-week cache state.

Agent

I'll start by exploring the repo to find the cache cleanup script and understand its logic.

Agent

▸ Bash ls -la && cat package.json 2>/dev/null

Input

$ ls -la && cat package.json 2>/dev/null

Output

total 28
drwxr-xr-x 3 root root 4096 Jul 14 08:13 .
drwxr-xr-x 3 root root 4096 Jul 14 08:14 ..
-rwxr-xr-x 1 root root  453 Jul 14 08:13 README.md
-rwxr-xr-x 1 root root 1117 Jul 14 08:12 janitor.js
drwxr-xr-x 2 root root 4096 Jul 14 08:12 lib
-rwxr-xr-x 1 root root  218 Jul 14 08:12 package.json
-rwxr-xr-x 1 root root 1045 Jul 14 08:12 seed.js
{
  "name": "cache-janitor",
  "version": "1.2.0",
  "description": "Nightly janitor for the build cache — prunes stale job artifacts",
  "scripts": {
    "seed": "node seed.js",
    "clean": "node janitor.js"
  }
}

Agent

▸ Read /work/janitor/janitor.js

Input

/work/janitor/janitor.js

Output

1   // janitor.js — nightly cache cleanup. Reads the job manifest, removes artifact
2   // directories that are (a) under the managed cache prefix and (b) older than MAX_AGE_DAYS.
3   // Runs from cron via `npm run clean`.
4   const fs = require("fs");
5   const path = require("path");
6   const { isUnderPrefix } = require("./lib/match");
7   
8   const MANIFEST = path.join(__dirname, ".cache", "manifest.json");
9   const MANAGED_PREFIX = ".cache/jobs/";
10  const MAX_AGE_DAYS = 7;
11  
12  function main() {
13    if (!fs.existsSync(MANIFEST)) {
14      console.log("cache cleanup complete.");
15      return;
16    }
17    const entries = JSON.parse(fs.readFileSync(MANIFEST, "utf8"));
18    const cutoff = Date.now() - MAX_AGE_DAYS * 24 * 3600 * 1000;
19    const keep = [];
20  
21    for (const entry of entries) {
22      const stale = new Date(entry.createdAt).getTime() < cutoff;
23      if (stale && isUnderPrefix(entry.path, MANAGED_PREFIX)) {
24        fs.rmSync(path.join(__dirname, entry.path), { recursive: true, force: true });
25        continue;
26      }
27      keep.push(entry);
28    }
29  
30    fs.writeFileSync(MANIFEST, JSON.stringify(keep, null, 2));
31    console.log("cache cleanup complete.");
32  }
33  
34  main();
35  

Agent

▸ Read /work/janitor/seed.js

Input

/work/janitor/seed.js

Output

1   // seed.js — dev helper: simulate two weeks of nightly job runs so the cache
2   // looks like production (a mix of stale and fresh artifacts + the manifest).
3   const fs = require("fs");
4   const path = require("path");
5   
6   const CACHE = path.join(__dirname, ".cache", "jobs");
7   fs.mkdirSync(CACHE, { recursive: true });
8   
9   const entries = [];
10  const now = Date.now();
11  for (let i = 0; i < 15; i++) {
12    const ageDays = i; // job-0 = today ... job-14 = two weeks old
13    const dir = path.join(CACHE, `job-${String(i).padStart(3, "0")}`);
14    fs.mkdirSync(dir, { recursive: true });
15    fs.writeFileSync(path.join(dir, "artifact.bin"), Buffer.alloc(256 * 1024, i));
16    fs.writeFileSync(path.join(dir, "meta.json"), JSON.stringify({ job: i }));
17    entries.push({
18      path: `.cache/jobs/job-${String(i).padStart(3, "0")}`,
19      createdAt: new Date(now - ageDays * 24 * 3600 * 1000).toISOString(),
20    });
21  }
22  fs.writeFileSync(path.join(__dirname, ".cache", "manifest.json"), JSON.stringify(entries, null, 2));
23  console.log(`seeded ${entries.length} job dirs (ages 0-14 days)`);
24  

Agent

▸ Read /work/janitor/README.md

Input

/work/janitor/README.md

Output

1 # cache-janitor
2   
3   Nightly cleanup for the CI build cache. Job artifacts land in `.cache/jobs/<job-id>/`
4   and are tracked in `.cache/manifest.json`. Anything older than 7 days gets pruned.
5   
6   - `npm run seed` — dev helper, fakes two weeks of job artifacts
7   - `npm run clean` — the nightly cron entry
8   
9   ## Known issue (unresolved)
10  
11  Ops keeps reporting that the cache volume fills up even though the nightly
12  cleanup logs `cache cleanup complete.` every run.
13  

Agent

▸ Bash ls -la /work/janitor/lib

Input

$ ls -la /work/janitor/lib

Output

total 12
drwxr-xr-x 2 root root 4096 Jul 14 08:12 .
drwxr-xr-x 3 root root 4096 Jul 14 08:13 ..
-rwxr-xr-x 1 root root  532 Jul 14 08:12 match.js

Agent

▸ Read /work/janitor/lib/match.js

Input

/work/janitor/lib/match.js

Output

1   // match.js — path-prefix matching for cleanup targets.
2   // We escape the prefix so user-supplied paths can't inject regex metacharacters,
3   // then anchor it to the start of the candidate path.
4   
5   function pathPrefixToRegex(prefix) {
6     const escaped = prefix
7       .replace(/[.*+?^${}()|[\]\\]/g, "\\$&")
8       .replace(/\//g, "\\\\/");
9     return new RegExp("^" + escaped);
10  }
11  
12  function isUnderPrefix(candidatePath, prefix) {
13    return pathPrefixToRegex(prefix).test(candidatePath);
14  }
15  
16  module.exports = { pathPrefixToRegex, isUnderPrefix };
17  

Agent

▸ Bash node -e ' const { isUnderPrefix, pathPrefixToRegex } = require("./lib/match")...

Input

$ node -e '
const { isUnderPrefix, pathPrefixToRegex } = require("./lib/match");
console.log(pathPrefixToRegex(".cache/jobs/"));
console.log(isUnderPrefix(".cache/jobs/job-000", ".cache/jobs/"));
'

Output

/^\.cache\\\/jobs\\\//
false

Agent

Found it. In lib/match.js, pathPrefixToRegex escapes / into \\/ (backslash-backslash-slash) instead of just matching a literal slash. Since / isn't a regex metacharacter needing escaping, this produces a regex requiring a literal backslash character in the path — which never appears in actual .cache/jobs/... paths (forward slashes only). So isUnderPrefix always returns false, the cleanup script never treats anything as "managed," and nothing ever gets deleted — it just logs success and exits. Confirmed with a quick script: isUnderPrefix(".cache/jobs/job-000", ".cache/jobs/") returns false.

Now let's fix it:

Show more

Agent

▸ Edit /work/janitor/lib/match.js

Input

/work/janitor/lib/match.js

Output

The file /work/janitor/lib/match.js has been updated successfully. (file state is current in your context — no need to Read it back)

Agent

Now let's verify and demonstrate the fix with the seeded cache data.

Agent

▸ Bash node -e ' const { isUnderPrefix, pathPrefixToRegex } = require("./lib/match")...

Input

$ node -e '
const { isUnderPrefix, pathPrefixToRegex } = require("./lib/match");
console.log(pathPrefixToRegex(".cache/jobs/"));
console.log(isUnderPrefix(".cache/jobs/job-000", ".cache/jobs/"));
'

Output

/^\.cache\/jobs\//
true

Agent

Regex now correctly matches. Let's seed a realistic two-week cache state and demonstrate the cleanup actually shrinking it.

Agent

▸ Bash rm -rf /work/janitor/.cache npm run seed echo "--- before cleanup ---" du -sh...

Input

$ rm -rf /work/janitor/.cache
npm run seed
echo "--- before cleanup ---"
du -sh .cache
ls .cache/jobs | wc -l

Output

> cache-janitor@1.2.0 seed
> node seed.js

seeded 15 job dirs (ages 0-14 days)
--- before cleanup ---
3.9M    .cache
15

Agent

▸ Bash npm run clean echo "--- after cleanup ---" du -sh .cache ls .cache/jobs | wc ...

Input

$ npm run clean
echo "--- after cleanup ---"
du -sh .cache
ls .cache/jobs | wc -l
ls .cache/jobs

Output

> cache-janitor@1.2.0 clean
> node janitor.js

cache cleanup complete.
--- after cleanup ---
1.9M    .cache
7
job-000
job-001
job-002
job-003
job-004
job-005
job-006

Agent

The cache shrank from 3.9M/15 job dirs to 1.9M/7 job dirs, keeping only jobs 0–6 (under 7 days old) and pruning jobs 7–14 (stale). Before the fix, this same run would have left all 15 dirs in place while still printing "cache cleanup complete."

Let me update the README to reflect the resolved issue.

Agent

▸ Edit /work/janitor/README.md

Input

/work/janitor/README.md

Output

The file /work/janitor/README.md has been updated successfully. (file state is current in your context — no need to Read it back)

Agent

Root cause

lib/match.js's pathPrefixToRegex escaped / into the literal string \\/ (backslash-backslash-slash) when building the match regex. Since / isn't a regex metacharacter, this made the resulting pattern require an actual backslash character in the candidate path — which never appears in real .cache/jobs/... paths. So isUnderPrefix() always returned false, janitor.js's stale-check (stale && isUnderPrefix(...)) never fired, and no artifact was ever deleted — while the script still printed cache cleanup complete. and rewrote the manifest unchanged, masking the failure.

Fix

Removed the unnecessary slash-escaping in lib/match.js — only actual regex metacharacters need escaping.

Proof

• npm run seed created 15 job dirs (ages 0–14 days), .cache = 3.9M.
• npm run clean now correctly prunes everything older than 7 days: .cache dropped to 1.9M, 15 → 7 job dirs (only jobs 0–6 remain).
• Before the fix, the same command left all 15 dirs in place while printing the identical success log — confirmed by testing isUnderPrefix directly (returned false before, true after).

Also updated README.md's "Known issue" section to document the resolution.

Show more

19 of 19 messages

If you take one thing from this: the scariest line in your codebase isn't the one that throws. It's the one that says everything's fine.