How a 1,920-line file turned into 410 lines, why "it works" was a lie twice, and why the fix wasn't a smarter model — it was a gate that doesn't trust anyone, including me.
The setup
I maintain YouMindAG, an open-source CLI (npx youmindag) that injects project context — architecture, dependencies, rules, DB schema — into AI coding tools like Claude Code, Cursor, opencode, and Copilot, so they don't have to rediscover your codebase from scratch every session.
By v2.7.0, the CLI's entrypoint, bin/run.mjs, had grown to 1,920 lines. One file. Install logic, upgrade logic, seventeen subcommands, vault population, dev-server wrapping, AST-based tracing — all of it, flat.
I did what you're supposed to do: I modularized it. With an AI coding agent doing the heavy lifting and me reviewing, I split it into lib/ — one file per concern, one file per command family. By v2.9.0, bin/run.mjs was down to 410 lines, functioning only as an orchestrator.
Clean diff. Tests green. I shipped it.
It was broken in seven different ways.
Round one: three ReferenceErrors, found by hand
A few smoke tests in, three commands started throwing ReferenceError: RESET is not defined and similar. I traced it manually:
-
lib/graphify.mjsused a console-color constantRESETin 11 places. Zero imports of it. -
lib/populate.mjshad the same missing import, silently breaking the summary output of the vault auto-population step. -
lib/commands/misc.mjsreferenced aVERSIONconstant that had never been passed in — it was a global in the old monolith, and extraction turned it into nothing.
I fixed the three, and — this is the part I want to be honest about — I did not go looking for a fourth. I wrote eight regression tests for the three I'd found, ran the suite, saw 58/58 green, and shipped v2.9.1.
Eight tests, three real bugs found by reading error messages one at a time. That's not an audit. That's whack-a-mole with good intentions.
Round two: I finally ran the tool made for exactly this
A couple of releases later I asked myself an obvious question I'd skipped: if extraction can silently drop an import once, what's stopping it from doing that seven more times in files I haven't touched yet?
So I added ESLint with a single rule that matters here — no-undef — and pointed it at everything under bin/ and lib/:
// eslint.config.mjs
{
rules: {
'no-undef': 'error',
},
files: ['bin/**/*.mjs', 'lib/**/*.mjs'],
}
Then I ran it.
46 undefined references. Spread across 7 files. 7 of 14 CLI commands broken or silently failing.
lib/fs-helpers.mjs +readFileSync
lib/commands/db.mjs +createInterface, parseEnvFile, hasPostgres (+ a CWD→cwd typo bug)
lib/commands/dev.mjs +mkdirSync, openSync, closeSync, appendFileSync,
dirname, execSync, spawn, writeYoumindagData
lib/commands/misc.mjs +rmSync, relative, extname, execSync,
createInterface, findProjectFiles
lib/commands/sync.mjs +mkdirSync, dirname, execSync, extname
lib/commands/trace.mjs +join, existsSync, execSync
lib/commands/watch.mjs +watch, statSync, populateVaultFiles
lib/vault.mjs +writeYoumindagData (lost during extraction entirely)
Some of these were commands that would crash loudly on first use — annoying, but at least visible. Others were worse: youmindag dev --wrap and youmindag sync were failing in code paths that only execute on specific flags or specific project states, meaning they could sit unnoticed for weeks in a project that never happened to hit that branch.
I fixed all 46, then verified every one of the 14 CLI commands manually against a real project — not just "does the test suite pass," but "does youmindag db, youmindag dev --status, youmindag trace --client X each actually do the thing" — one at a time.
The part that actually matters
The bug count isn't the interesting part. Missing imports after a mechanical extraction are a known failure mode; anyone who's done a big refactor by hand has hit this. The interesting part is why I missed 46 of them the first time and only 3 the second, and why "read the diff carefully" was never going to close that gap.
When an AI agent extracts a function from a 1,920-line file into a new module, the failure mode isn't wrong logic — the logic is usually copied verbatim. The failure mode is forgetting what the extracted code silently depended on. RESET used to be defined once, at the top of the monolith, and every function in that file could reach it without ever importing anything. Once you cut that function out into its own file, that implicit access disappears — and nothing about the extracted code looks wrong. It reads perfectly. It's only wrong at runtime, on the exact line that reaches for something that isn't there anymore.
And here's the uncomfortable bit: I was reviewing that same code with the same kind of attention an LLM has when it writes it — I was reading it for whether it looked correct, not tracing every identifier back to a binding. That's a slow, mechanical, unglamorous check. It's exactly the kind of check a human reviewer skips under time pressure, and exactly the kind of check a static analysis tool never skips, because it doesn't get tired and it doesn't trust that something "looks fine."
no-undef isn't a smart rule. It doesn't understand your architecture, doesn't know what a good abstraction looks like, doesn't have opinions. That's precisely why it caught what I didn't: it isn't pattern-matching on "does this look like working code," it's mechanically checking "is every identifier bound to something." My eyes are good at the first question and bad at the second. A linter is the opposite, and after this I stopped assuming my eyes were enough.
The fix wasn't a smarter model — it was a gate
The real fix in v2.9.3 wasn't "be more careful next time." It was making carelessness structurally impossible to ship:
"scripts": {
"test": "node --test",
"lint": "eslint bin/run.mjs lib/*.mjs lib/commands/*.mjs",
"verify": "npm run lint && npm test",
"prepublishOnly": "npm run verify"
}
prepublishOnly runs automatically on every npm publish. If lint or tests fail, the publish fails. I tested this by deliberately breaking a file and confirming the publish got blocked before it reached npm.
This means the class of bug that shipped in v2.9.0–v2.9.2 — a missing import slipping through review — cannot ship again without someone deliberately bypassing the gate. Not "probably won't." Cannot, structurally, by default.
What I'd tell someone auditing their own AI-generated refactor
- A green test suite tells you the paths you tested work. It says nothing about the paths you didn't think to test. 58/58 passing coexisted with 7 broken commands, because the tests covered logic, not every command's happy path against a real project.
- "I reviewed the diff" and "I ran a tool that checks a specific invariant" are not the same kind of confidence. The first catches what looks wrong. The second catches what is wrong regardless of how it looks.
- If your refactor involves extraction — pulling code out of one file into several — assume implicit dependencies broke until a tool proves otherwise. Don't go looking for them by reading; that's how I caught 3 out of 46 the first time.
- The fix that survives is the one that doesn't depend on remembering to do it again. I didn't add ESLint because I got smarter. I added a gate because I don't trust future-me — running low on time, mid-refactor, eager to ship — any more than I trust the AI that wrote the extraction in the first place.
YouMindAG is at v2.9.3 now, MIT-licensed, on GitHub. The commit history above is real and public if you want to check my math — git log doesn't round up.
Top comments (16)
The bit about reviewing your own code with the same attention an LLM has when it writes it is the sharpest line here, and it explains the 3-out-of-46 better than any tool comparison. You were checking whether it looked right, which is a real skill, just not the one that catches a missing binding. One thing I'd add for anyone copying the gate:
no-undefonly sees what the parser can see, so arequireor import behind a conditional, or anything reached through a string key, still slips past. Worth pairing the lint gate with actually running each command once, which you did do manually. That part deserves as much billing as the linter.Really fair point, and honestly you're right that I undersold the manual
command-by-command run. I gave the linter most of the credit in the
writeup because it scales and doesn't depend on me remembering to run
it — but it wouldn't have caught the dev.mjs bug that only showed up on
a specific flag combo. That one only got found because I actually ran it.
And yeah, good catch on no-undef's blind spots. Everything here happens
to be static ESM imports, so it covered this bug class, but I wouldn't
trust it as a general guarantee for conditional requires or string-keyed
access. Gonna add a caveat about that to the docs so people don't read
the gate as more bulletproof than it is.
Comments like this are honestly the best part of writing these up —
easy to miss stuff when you're the one who wrote both the bug and the
fix. Appreciate you taking the time 🙏
Auditing your own AI-generated refactor is the part that makes the experiment credible. Generated code can look cleaner while quietly changing behavior. The useful habit is treating the refactor as a suspect until tests, diffs, and domain checks prove what actually stayed the same.
Completely agree, and "treating the refactor as a suspect" is exactly the right frame — that's the mental shift that took me two rounds to actually adopt. First round, I audited by reading: found 3
ReferenceErrors by hand, wrote regression tests for those 3, saw 58/58 green, and shipped. Felt like due diligence. It wasn't — it was confirming the bugs I already knew about, not looking for the ones I didn't.Second round, I stopped trusting "it reads correctly" as a signal at all and ran
eslint --rule no-undefagainst the whole codebase instead of my eyes. That's what actually found the other 46 — spread across 7 files, silently breaking 7 of 14 commands, several of them only on code paths that don't run in every session. The diff looked clean both times. Tests were green both times. What changed wasn't how carefully I looked — it was switching from "does this look right" to a tool that checks a single mechanical invariant and doesn't care how confident I am that it's fine.That first-round story is painfully familiar. Reading is good at confirming the mental model you already have. A mechanical check like no-undef is less flattering but much better at finding the bugs that do not match your current suspicion. That is a great lesson for AI refactors.
The gap you're describing —
no-undefcatching what your eyes couldn't — has a sharper edge worth naming:no-undefin ESLint's flat config depends entirely on you declaring the rightglobalsandsourceType. If your env is misconfigured,no-undefwill happily flag legitimate globals (or worse, stay quiet on real ones) and you'll either drown in false positives or trust a rule that isn't actually looking where you think. The reason it worked cleanly for you is that.mjs+ a correctlanguageOptionsmakes the binding rules unambiguous — Node builtins are explicit imports, nothing's ambient. That's not incidental to the win; it's the precondition for it.Which points at the deeper lever you almost stated but didn't: the extraction only silently dropped
RESETbecause the monolith let every function reach a top-of-file const with zero ceremony. Module boundaries that force explicit imports are what makeno-undefa useful gate at all. In a codebase leaning on ambient globals or CommonJS with dynamicrequire, the same linter goes half-blind — so "addno-undef" is really "make your dependencies explicit enough that a dumb rule can see them," and that's the part that generalizes past this one refactor.You're right, and I can be specific about why it happened to work here instead of just agreeing in the abstract.
The config is a hand-written
globalsobject —process,console,__dirname,Buffer,setTimeout,clearTimeout— not theglobalsnpm package'snodepreset. That was a deliberate choice, though I'll admit it was more instinct than principle at the time: a preset would have pulled in a much wider ambient surface, and every name in that surface is oneno-undefstops checking. Narrow and explicit was the safer default precisely for the failure mode you're describing.There's a second reason this specific case was close to best-case for the rule, independent of my config:
readFileSync,execSync,createInterfacewere never ambient globals in any JS environment — CJS or ESM. They requirerequire()orimporteither way. So this exact bug class would've surfaced under CommonJS too, config permitting. What ESM +sourceType: 'module'bought me wasn't catching those — it's that it left nothing else for a loose config to accidentally excuse. No dynamicrequire(), noglobal.X = ...assignment hiding a real dependency, no ambiguity for me to configure my way out of by accident.So I'd restate your last paragraph as the actual takeaway:
no-undefdidn't find the bugs. Making implicit access impossible — no ambient scope in the monolith to fall back on, explicit bindings only — is what let a rule that doesn't understand the codebase still see everything that mattered. Same root cause as the original bug, really: the monolith broke because of implicit reachability, and the gate only works because it removed implicit reachability as an option, config included.The part about reviewing code with the same attention the LLM used to write it hit hard. That's the exact failure mode I keep seeing across agent quality discussions — the reviewer applies the same probabilistic pattern-matching that generated the code in the first place.
What you describe maps almost perfectly to a pattern I've been tracking: when the judge and the doer use the same cognitive process, you get correlated blind spots. Your eyes and the LLM are both asking "does this look right?" instead of "is every identifier actually bound?" The linter wins because it asks a question that doesn't require understanding.
The Wren Calloway thread about module boundaries is the load-bearing insight here. no-undef only works because you removed implicit reachability. Same principle shows up everywhere — the gate is only as good as the boundary definition that feeds it. A linter checking bindings in a codebase with ambient globals is just doing theater.
Curious: did you find any categories of the 46 bugs where a different rule (beyond no-undef) would've caught them earlier? Like type mismatches or semantic drift that no-undef can't see?
"Correlated blind spots" is a great way to put it, way more precise than
how I described it. The judge and the doer using the same kind of check
is exactly the trap — and yeah, the linter wins because it doesn't need
to understand anything, it just needs the binding to exist.
On your question: honestly, no — all 46 were the same category, missing
bindings that no-undef caught directly. There was one that looked like it
might be different at first (a CWD vs cwd casing mismatch in db.mjs) but
it turned out to still trip no-undef the same way, since CWD itself was
undefined. So in this codebase specifically, one rule covered the whole
bug class.
That said, this is plain JS, no type checker in the loop, so I genuinely
don't know what a semantic-drift or type-mismatch bug would've looked
like here — there's a real chance something like that is sitting in the
code right now that no-undef can't see. Good nudge to think about
whether JSDoc + tsc --checkJs is worth adding on top of this at some point.
The "correlated blind spots" framing is spot on. When your judge and doer share the same cognitive model, you're not validating - you're just echo-checking.
Your linter example is the perfect illustration of why dumb tools sometimes win. It doesn't try to understand intent, it just checks if the binding exists. That's it. No reasoning, no inference, just "does this name resolve to something?"
The scary part is what you said at the end - the bugs that no-undef can't see are probably sitting in your code right now. Semantic drift, type mismatches, logic that's internally consistent but wrong. That's where JSDoc + checkJs actually pays off. Not because it catches more bugs today, but because it forces you to think about types explicitly, which makes the blind spots smaller.
Though I'd be curious what the overhead is. If you're adding type annotations to 80% of your codebase just to catch the 20% of bugs that slip past linting, is it worth it? Or does it only make sense for specific modules where correctness really matters?
This matches a class of bug I keep seeing whenever an agent splits one file into several. It reads as a refactor, the diff looks clean, and the tests it wrote for the new structure exercise exactly the paths it thought about, so everything comes back green. The failure that got you, an implicit top level reference silently orphaned, is exactly the kind of thing that survives review because both the agent and I are reading the code for shape, not for what actually resolves at runtime. I have started running a real build or typecheck as a gate after an agent splits files that way, not just tests, since a test suite only proves the paths someone thought to write.
This is a great reminder that AI-assisted refactoring doesn't fail because the model can't move code it fails because it can't reliably preserve every implicit dependency. Static analysis catches a different class of bugs than tests, and that's exactly why linting, type checks, and CI gates should be mandatory after any large AI-generated refactor. We've seen similar patterns at IT Path Solutions with AI-assisted code migrations: the biggest improvement came from strengthening the validation pipeline, not switching models. Trust the AI to accelerate implementation, but trust automated gates to protect production.
"Trust the AI to accelerate implementation, but trust automated gates to protect production" is a clean way to put it — I'd only add that the split isn't just AI vs. gates, it's what kind of gate. Tests told me nothing was wrong (58/58 green) while 7 of 14 commands were broken. Static analysis found all 46 in one pass. Different failure surface, and a green test suite alone would've let this ship indefinitely, since the tests only exercise the paths I thought to write tests for.
Curious about the migration work at IT Path Solutions — when you say the validation pipeline was the improvement, was it mostly static analysis catching structural drift like this (dropped imports, broken bindings), or were you also leaning on type checks to catch semantic drift — logic that runs fine but returns the wrong thing? That second category is the one I don't have a good mechanical answer for yet.
no-undefcan't tell me if an extracted function's behavior subtly changed as long as every identifier still resolves.If you've read this far, I'd genuinely love for you to try it on your own project — not because it's finished, but because it's exactly the point where feedback actually changes what I build next.
npx youmindagin any Node project. No signup, no API key, everything stays local. It builds a knowledge vault + dependency graph so your AI coding agent stops re-discovering your architecture every session. On my own test project it cut the context an agent needed from 30,033 tokens down to 9,137 — there's anab-test.mjsscript in the repo if you want your own number instead of trusting mine.It's six days old as of this post, so it will have rough edges I haven't hit yet — different framework, different DB, different project shape than what I've tested. That's exactly what I want to hear about. Open an issue, comment here, or DM me — I read all of it, and this post itself is proof I'll actually act on it.
MIT-licensed, all public: github.com/CESARBR2025/youmindag
I'll be sure to add no-undef to my eslint configs from here on out.
Ha, glad it landed! It's such a cheap rule to add and it would've saved
me a very annoying afternoon. Appreciate you reading it 🙏