How a 1,920-line file turned into 410 lines, why "it works" was a lie twice, and why the fix wasn't a smarter model — it was a gate that doesn't tr...
For further actions, you may consider blocking this person and/or reporting abuse
The bit about reviewing your own code with the same attention an LLM has when it writes it is the sharpest line here, and it explains the 3-out-of-46 better than any tool comparison. You were checking whether it looked right, which is a real skill, just not the one that catches a missing binding. One thing I'd add for anyone copying the gate:
no-undefonly sees what the parser can see, so arequireor import behind a conditional, or anything reached through a string key, still slips past. Worth pairing the lint gate with actually running each command once, which you did do manually. That part deserves as much billing as the linter.Really fair point, and honestly you're right that I undersold the manual
command-by-command run. I gave the linter most of the credit in the
writeup because it scales and doesn't depend on me remembering to run
it — but it wouldn't have caught the dev.mjs bug that only showed up on
a specific flag combo. That one only got found because I actually ran it.
And yeah, good catch on no-undef's blind spots. Everything here happens
to be static ESM imports, so it covered this bug class, but I wouldn't
trust it as a general guarantee for conditional requires or string-keyed
access. Gonna add a caveat about that to the docs so people don't read
the gate as more bulletproof than it is.
Comments like this are honestly the best part of writing these up —
easy to miss stuff when you're the one who wrote both the bug and the
fix. Appreciate you taking the time 🙏
Auditing your own AI-generated refactor is the part that makes the experiment credible. Generated code can look cleaner while quietly changing behavior. The useful habit is treating the refactor as a suspect until tests, diffs, and domain checks prove what actually stayed the same.
Completely agree, and "treating the refactor as a suspect" is exactly the right frame — that's the mental shift that took me two rounds to actually adopt. First round, I audited by reading: found 3
ReferenceErrors by hand, wrote regression tests for those 3, saw 58/58 green, and shipped. Felt like due diligence. It wasn't — it was confirming the bugs I already knew about, not looking for the ones I didn't.Second round, I stopped trusting "it reads correctly" as a signal at all and ran
eslint --rule no-undefagainst the whole codebase instead of my eyes. That's what actually found the other 46 — spread across 7 files, silently breaking 7 of 14 commands, several of them only on code paths that don't run in every session. The diff looked clean both times. Tests were green both times. What changed wasn't how carefully I looked — it was switching from "does this look right" to a tool that checks a single mechanical invariant and doesn't care how confident I am that it's fine.That first-round story is painfully familiar. Reading is good at confirming the mental model you already have. A mechanical check like no-undef is less flattering but much better at finding the bugs that do not match your current suspicion. That is a great lesson for AI refactors.
The gap you're describing —
no-undefcatching what your eyes couldn't — has a sharper edge worth naming:no-undefin ESLint's flat config depends entirely on you declaring the rightglobalsandsourceType. If your env is misconfigured,no-undefwill happily flag legitimate globals (or worse, stay quiet on real ones) and you'll either drown in false positives or trust a rule that isn't actually looking where you think. The reason it worked cleanly for you is that.mjs+ a correctlanguageOptionsmakes the binding rules unambiguous — Node builtins are explicit imports, nothing's ambient. That's not incidental to the win; it's the precondition for it.Which points at the deeper lever you almost stated but didn't: the extraction only silently dropped
RESETbecause the monolith let every function reach a top-of-file const with zero ceremony. Module boundaries that force explicit imports are what makeno-undefa useful gate at all. In a codebase leaning on ambient globals or CommonJS with dynamicrequire, the same linter goes half-blind — so "addno-undef" is really "make your dependencies explicit enough that a dumb rule can see them," and that's the part that generalizes past this one refactor.You're right, and I can be specific about why it happened to work here instead of just agreeing in the abstract.
The config is a hand-written
globalsobject —process,console,__dirname,Buffer,setTimeout,clearTimeout— not theglobalsnpm package'snodepreset. That was a deliberate choice, though I'll admit it was more instinct than principle at the time: a preset would have pulled in a much wider ambient surface, and every name in that surface is oneno-undefstops checking. Narrow and explicit was the safer default precisely for the failure mode you're describing.There's a second reason this specific case was close to best-case for the rule, independent of my config:
readFileSync,execSync,createInterfacewere never ambient globals in any JS environment — CJS or ESM. They requirerequire()orimporteither way. So this exact bug class would've surfaced under CommonJS too, config permitting. What ESM +sourceType: 'module'bought me wasn't catching those — it's that it left nothing else for a loose config to accidentally excuse. No dynamicrequire(), noglobal.X = ...assignment hiding a real dependency, no ambiguity for me to configure my way out of by accident.So I'd restate your last paragraph as the actual takeaway:
no-undefdidn't find the bugs. Making implicit access impossible — no ambient scope in the monolith to fall back on, explicit bindings only — is what let a rule that doesn't understand the codebase still see everything that mattered. Same root cause as the original bug, really: the monolith broke because of implicit reachability, and the gate only works because it removed implicit reachability as an option, config included.The part about reviewing code with the same attention the LLM used to write it hit hard. That's the exact failure mode I keep seeing across agent quality discussions — the reviewer applies the same probabilistic pattern-matching that generated the code in the first place.
What you describe maps almost perfectly to a pattern I've been tracking: when the judge and the doer use the same cognitive process, you get correlated blind spots. Your eyes and the LLM are both asking "does this look right?" instead of "is every identifier actually bound?" The linter wins because it asks a question that doesn't require understanding.
The Wren Calloway thread about module boundaries is the load-bearing insight here. no-undef only works because you removed implicit reachability. Same principle shows up everywhere — the gate is only as good as the boundary definition that feeds it. A linter checking bindings in a codebase with ambient globals is just doing theater.
Curious: did you find any categories of the 46 bugs where a different rule (beyond no-undef) would've caught them earlier? Like type mismatches or semantic drift that no-undef can't see?
"Correlated blind spots" is a great way to put it, way more precise than
how I described it. The judge and the doer using the same kind of check
is exactly the trap — and yeah, the linter wins because it doesn't need
to understand anything, it just needs the binding to exist.
On your question: honestly, no — all 46 were the same category, missing
bindings that no-undef caught directly. There was one that looked like it
might be different at first (a CWD vs cwd casing mismatch in db.mjs) but
it turned out to still trip no-undef the same way, since CWD itself was
undefined. So in this codebase specifically, one rule covered the whole
bug class.
That said, this is plain JS, no type checker in the loop, so I genuinely
don't know what a semantic-drift or type-mismatch bug would've looked
like here — there's a real chance something like that is sitting in the
code right now that no-undef can't see. Good nudge to think about
whether JSDoc + tsc --checkJs is worth adding on top of this at some point.
The "correlated blind spots" framing is spot on. When your judge and doer share the same cognitive model, you're not validating - you're just echo-checking.
Your linter example is the perfect illustration of why dumb tools sometimes win. It doesn't try to understand intent, it just checks if the binding exists. That's it. No reasoning, no inference, just "does this name resolve to something?"
The scary part is what you said at the end - the bugs that no-undef can't see are probably sitting in your code right now. Semantic drift, type mismatches, logic that's internally consistent but wrong. That's where JSDoc + checkJs actually pays off. Not because it catches more bugs today, but because it forces you to think about types explicitly, which makes the blind spots smaller.
Though I'd be curious what the overhead is. If you're adding type annotations to 80% of your codebase just to catch the 20% of bugs that slip past linting, is it worth it? Or does it only make sense for specific modules where correctness really matters?
This matches a class of bug I keep seeing whenever an agent splits one file into several. It reads as a refactor, the diff looks clean, and the tests it wrote for the new structure exercise exactly the paths it thought about, so everything comes back green. The failure that got you, an implicit top level reference silently orphaned, is exactly the kind of thing that survives review because both the agent and I are reading the code for shape, not for what actually resolves at runtime. I have started running a real build or typecheck as a gate after an agent splits files that way, not just tests, since a test suite only proves the paths someone thought to write.
This is a great reminder that AI-assisted refactoring doesn't fail because the model can't move code it fails because it can't reliably preserve every implicit dependency. Static analysis catches a different class of bugs than tests, and that's exactly why linting, type checks, and CI gates should be mandatory after any large AI-generated refactor. We've seen similar patterns at IT Path Solutions with AI-assisted code migrations: the biggest improvement came from strengthening the validation pipeline, not switching models. Trust the AI to accelerate implementation, but trust automated gates to protect production.
"Trust the AI to accelerate implementation, but trust automated gates to protect production" is a clean way to put it — I'd only add that the split isn't just AI vs. gates, it's what kind of gate. Tests told me nothing was wrong (58/58 green) while 7 of 14 commands were broken. Static analysis found all 46 in one pass. Different failure surface, and a green test suite alone would've let this ship indefinitely, since the tests only exercise the paths I thought to write tests for.
Curious about the migration work at IT Path Solutions — when you say the validation pipeline was the improvement, was it mostly static analysis catching structural drift like this (dropped imports, broken bindings), or were you also leaning on type checks to catch semantic drift — logic that runs fine but returns the wrong thing? That second category is the one I don't have a good mechanical answer for yet.
no-undefcan't tell me if an extracted function's behavior subtly changed as long as every identifier still resolves.If you've read this far, I'd genuinely love for you to try it on your own project — not because it's finished, but because it's exactly the point where feedback actually changes what I build next.
npx youmindagin any Node project. No signup, no API key, everything stays local. It builds a knowledge vault + dependency graph so your AI coding agent stops re-discovering your architecture every session. On my own test project it cut the context an agent needed from 30,033 tokens down to 9,137 — there's anab-test.mjsscript in the repo if you want your own number instead of trusting mine.It's six days old as of this post, so it will have rough edges I haven't hit yet — different framework, different DB, different project shape than what I've tested. That's exactly what I want to hear about. Open an issue, comment here, or DM me — I read all of it, and this post itself is proof I'll actually act on it.
MIT-licensed, all public: github.com/CESARBR2025/youmindag
I'll be sure to add no-undef to my eslint configs from here on out.
Ha, glad it landed! It's such a cheap rule to add and it would've saved
me a very annoying afternoon. Appreciate you reading it 🙏