Everyone keeps saying AI will replace developers.
Meanwhile I was sitting at 3:17 AM staring at a bug created by code that looked perfectly correct...
For further actions, you may consider blocking this person and/or reporting abuse
ran into this building my PM agent stack - AI gets you to 80% in minutes, that last 20% is brutal. I now treat AI-written code like an intern PR: approve it, but budget the review time.
"Approve it, but budget the review time" might be the most practical AI coding advice I've heard. The speed is incredible, but the review phase is where you find out whether you saved time or just borrowed it from your future self.
the borrowed-from-future-self framing is exactly right - and the debt lands in the places the AI cannot see: state mutations, side effects, anything that depends on context outside the file. the 80% is clean, the 20% is where it was quietly making assumptions about everything it did not ask about.
Couldn't agree more. The scary part is that the code often looks completely reasonable at first glance. Then you start tracing actual application state, external dependencies, and edge cases, and you realize it was making a bunch of silent assumptions the whole time. AI is great at filling in blanks, but software bugs usually live in the blanks that should never have been filled in without asking. That's why the final stretch isn't really debugging code, it's debugging context.
'blanks that should never be filled without asking' is the exact line. what makes it worse is the confidence level — the model writes the silent assumption the same way it writes the obvious thing, no hedging, no flag. the review pass that catches it has to be specifically hunting hidden state dependencies, not just reading for syntax. most review isn't running at that level of skepticism.
Exactly. The confidence is what makes it dangerous. A human developer will often leave clues when they're unsure: a comment, a TODO, a weird variable name, or they'll simply ask a question. The model doesn't really have that instinct. An assumption about a critical state transition gets written with the same confidence as a string formatting function.
That's why AI code review can't just be "does this code look correct?" It has to be "what assumptions is this code making that aren't stated anywhere?" The failure mode isn't usually bad syntax or broken logic. It's unstated dependencies, missing business rules, race conditions, and state that exists outside the current context window.
What's interesting is that AI is pushing reviews toward a different skill set. The valuable reviewer isn't the person spotting a missing semicolon anymore. It's the person asking, "What happens if this service returns stale data?", "Who else mutates this state?", or "Where did this assumption come from?" That's the level where most of the expensive bugs are hiding.
"Confident nonsense" is the perfect name for it. We see the exact same pattern in test automation — AI-generated test code that looks complete, has solid coverage numbers, but is testing something completely different from what it thinks it's testing.
The scariest part is that these tests pass. They pass cleanly. So you feel safe. Then the bug hits production, and when you trace back, you find that assert never actually touched the real edge case — it just ran the happy path and called it a day.
We started enforcing a rule: any AI-generated test has to include a one-line comment explaining what assumption it's validating, not what function it's testing. If even the model can't articulate the assumption, the test doesn't make it into the PR.
I like that rule because it shifts the focus from coverage to intent. A passing test only proves that reality matched the assumptions encoded in the test. If those assumptions are wrong, you get a green checkmark attached to a false sense of security. That's one of the nastier AI failure modes: it can generate tests that perfectly validate the implementation it just invented. The code and the test agree with each other, but neither agrees with the actual requirement. Requiring an explicit statement of the assumption forces the conversation up a level from "does this function return the expected value?" to "what property of the system are we claiming remains true?" In a way, that comment becomes more valuable than the test itself because it's the only place where intent is made visible instead of inferred.
And here's the part that makes it worse — the model generating the tests and the model writing the code are the same brain. They share the same blind spots. If the code misses an edge case, the test misses the exact same one, because both came from the same understanding of the problem. It's not "right code, wrong test." It's two things agreeing with each other on a shared misunderstanding.
Your point about the comment being more valuable than the test — I'd push it one step further. If the model that wrote the test can't clearly state what assumption it's validating, that test shouldn't exist. No test is better than a bad test, because at least you won't be fooled by the green checkmark.
I think that's the deeper failure mode. We often treat tests as an independent verification layer, but when the same model generates both the implementation and the test, they're not independent at all. They're two artifacts derived from the same mental model. If that model misunderstood a requirement, the code and test can reinforce each other perfectly while both being wrong. The result is a green build that measures consistency, not correctness. That's why I like your rule. Requiring the test to state the assumption it's validating forces it to expose the underlying model of the system. Once the assumption is visible, a human can challenge it. Without that step, you're effectively letting the model grade its own homework. And in practice, a bad test is often more dangerous than no test because it replaces uncertainty with misplaced confidence, which means the next reviewer is less likely to go looking for the bug in the first place.
Exactly. And the organizational side amplifies it — once the build is green, the incentive to keep looking vanishes. A failing test forces a conversation. A passing test closes it. The model didn't just generate the test. It generated the permission to stop thinking. That's the real danger — not the bug itself, but the false all-clear that follows it.
That's a really important distinction. Bugs are recoverable. False confidence is what lets them spread. A failing test creates friction, discussion, and investigation. A passing test creates closure. The danger isn't that the model wrote incorrect code or even an incorrect test. It's that it produced enough evidence to satisfy the process without actually validating the assumption. Once the dashboard is green, the PR is approved, and the deployment succeeds, the organization shifts from "prove this is correct" to "assume this is correct." The test becomes less of a verification tool and more of a social signal that the thinking has already been done. In that sense, the most expensive AI failure mode isn't hallucinated code. It's hallucinated certainty.
Yeah, AI, does code alright, but also produces CVEs and ISSUEs in code expanding and things. We knwo that AI CANNOT THINK ON ITS OWN, it ca nrequery itself, but CANNOT THINK like human beings, so we expect some errors from it, and we are more then happy to tell it to:
FIX THIS NOW!. And overall, AI is making us dull inCoding!That's a fair concern. AI is great at generating code, but it's not great at owning the consequences of that code. Security flaws, hidden bugs, and design tradeoffs still require human judgment. I think the challenge is making sure AI amplifies our skills rather than replacing the need to use them.
Yep, totally agreed on that point!
Man, that phrase 'debugging logic written by someone who technically does not exist' hits way too close to home.
You perfectly captured the great illusion of modern development. AI is like a hyper-caffeinated junior dev who has memorized every textbook but has never actually survived a production outage. It gives us that massive dopamine hit of writing 80% of the feature in 5 minutes, only to quietly trap us in a 6-hour psychological thriller over a single missing edge case.
Your point about missing context vs. senior consequences is spot on. 'Vibe coding' is fun until the vibe turns into an unintended 3 AM shift. Definitely treating my AI tools like an eager intern from now on. Great write-up!
That's exactly the tradeoff I've been noticing. The speed boost is real, but so is the cost of understanding what was generated. The best results I've had come from treating AI as a collaborator rather than an authority. The moment I start trusting it blindly, that's usually when the psychological thriller begins.
This is true. I just hope more people find this post and be more proactive when it comes to using AI. Not challenging it or using your own brain will make debugging harder in the future.
Exactly. AI is at its best when it's challenged, not blindly trusted. The faster it generates code, the more important it becomes to understand the assumptions behind that code. Otherwise we're just trading time spent writing bugs for time spent debugging them.
"Confident nonsense" – that's the perfect phrase for it
I felt this entire post in my bones. Especially the part about debugging AI‑generated code at 3 AM, wondering if you're being gaslit by a language model.
The thing that caught me off guard was exactly what you described: the code looks beautiful. Clean types, sensible variable names, good structure. Then somewhere deep in an async callback, there's an off‑by‑one error or a race condition that only appears in production under specific load. Good luck finding that.
Your point about tests being more important now is spot on. I've started treating AI‑generated code like I treat code from a new junior dev: review everything, test thoroughly, and never trust it blindly.
The shift from "AI as engineer" to "AI as fast junior with infinite confidence" is exactly right. It's a tool, not a teammate. And like any powerful tool, it can hurt you if you forget how it works under the hood.
Thanks for writing this – it's a good reality check for anyone who thinks the "80% in 2 minutes" means the last 20% will also be fast.
Cheers,
Jack
DEV.to/ggle.in
Appreciate that, Jack. The "fast junior with infinite confidence" analogy keeps feeling more accurate the longer I use these tools. What worries me now isn't the obvious bug anymore, it's the polished bug. The code is clean enough that your review brain relaxes, and that's exactly when hidden assumptions slip through. I've started noticing that the most expensive failures aren't syntax errors or broken logic, they're cases where the model made a reasonable assumption that nobody challenged because everything looked professional. The real skill seems to be shifting from reading code to interrogating assumptions. Not "does this work?" but "what would have to be true for this to work?" That's usually where the interesting bugs are hiding.
Appreciate that, Jack. The "fast junior with infinite confidence" analogy keeps feeling more accurate the longer I use these tools. What worries me now isn't the obvious bug anymore, it's the polished bug. The code is clean enough that your review brain relaxes, and that's exactly when hidden assumptions slip through. I've started noticing that the most expensive failures aren't syntax errors or broken logic, they're cases where the model made a reasonable assumption that nobody challenged because everything looked professional. The real skill seems to be shifting from reading code to interrogating assumptions. Not "does this work?" but "what would have to be true for this to work?" That's usually where the interesting bugs are hiding.
That line hit me: "what would have to be true for this to work?"
You just articulated the shift I couldn't name. Reading code asks "does this work?" — which the AI usually passes because it looks like it works. Interrogating assumptions asks "what hidden conditions is this code silently depending on?" That's where the real bugs live.
The polished bug is indeed scarier than the obvious one. Obvious bugs get caught in code review. Polished bugs get merged, deployed, and then surface at 2 AM when a specific edge case finally triggers that reasonable-but-wrong assumption.
I've started keeping a small "assumption log" during PR reviews — not just what the code does, but what the code believes about the world (timing, state, data shape, ordering). It's been surprisingly helpful.
Thanks again for this thread. One of the most valuable conversations I've had on here.
Cheers,
Jack
dev.to/ggle_in
Yeah, it's true actually - playing with a is interesting but when it comes to the end result, we hope it's going to be amazing but unfortunately it can't understand our mind thought's very, if we need to achieve that we need to iterate more time to attain that which is actually worse.
But still I use AI for all my tasks now a days like a addiction.
I think a lot of people are in the same boat. AI has this weird effect where it feels incredibly productive because you're always moving, always generating, always exploring ideas. The catch is that sometimes you spend three hours prompting, refining, and correcting something that would've taken one focused hour to build yourself.
What keeps me using it is that it's less of a tool now and more of a thinking partner. It's great for brainstorming, drafting, researching, and getting past blank-page syndrome. The problem starts when we expect it to understand the exact picture in our heads. It doesn't. It only sees what we tell it, and we're usually much worse at describing our thoughts than we think we are.
So the workflow becomes: AI generates, human steers. The more specific the vision, the more iterations it takes. That's not really AI being bad, it's the gap between what we imagine and what we've actually communicated.
And yeah, the addiction part is real. Once you get used to having an instant second brain available 24/7, going back to staring at a blank screen feels unnecessarily difficult.
The "confident nonsense" part got me. I've had AI write functions that looked so clean I didn't bother reading them — until they broke three days later and I had no idea what any of it did. Now I just ask it to explain every line before I save anything.
Honestly, that's become one of my favorite uses for AI. Not just generating code, but explaining it. If I can't understand why a function works after reading the explanation, I probably shouldn't be shipping it in the first place.
AI wrote me a button component last week. Four state managers fighting each other inside 80 lines of code. I don't even know what a state manager is. Spent three hours staring at it before just deleting everything and writing the world's ugliest button from scratch. It works though.
"The world's ugliest button" is exactly how half of software engineering breakthroughs happen. Sometimes deleting 80 lines of clever code and replacing it with 10 lines you actually understand is the senior-engineer move.
the "debugging logic written by someone who technically does not exist" part is the real experience. we hit this hardest on async flows — AI would generate code that looked correct, handled the happy path fine, and silently swallowed errors on retry logic. no exception, no obvious failure. just incorrect final state.
mental shift that helped: stop reading AI code top to bottom and start asking "what does this fail to handle." forces you to think about consequences instead of patterns.
started requiring AI to generate test cases before the implementation. catches a lot of the confident nonsense before it makes it into a PR.
what's the most expensive AI generated bug you've had to find?
I really like the shift from "what does this do?" to "what does this fail to handle?" That feels much closer to how experienced engineers review code in general.
The async/retry examples are especially painful because everything appears to work until some edge case quietly corrupts the final state. Those are the bugs that consume entire afternoons.
As for the most expensive one, it wasn't a crash—it was a piece of generated logic that looked perfectly reasonable but made an incorrect assumption about state updates. Nothing failed, no errors were thrown, the data was just subtly wrong. Those are always the worst because you spend hours proving everything else isn't broken first.
the "data just wrong, no error thrown" category ages badly — you only find it when something downstream breaks in a way that does not point back. we added assertions after key state mutations not to catch bugs, just to make wrong state visible before it travels 3 hops and becomes untraceable.
state assumption bugs in AI code trace to one root: model saw the happy path shape and wrote for that. do you write assertions defensively now, or still mostly at review time?
More defensively now. The bugs that worry me most are not crashes, they're silent state corruption. If something throws immediately, at least you have a starting point. If invalid state survives three service boundaries and shows up as a weird analytics discrepancy a week later, you've basically started a forensic investigation. Assertions after critical state transitions have become less about catching programmer mistakes and more about enforcing invariants while the causal chain is still visible. That's especially true with AI-generated code because it tends to optimize for the happy path shape of the problem. The implementation often looks reasonable, but hidden assumptions about state validity, ordering, or ownership slip through. I've found that the highest ROI assertions aren't around inputs and outputs, they're around the moments where state changes hands. That's where "this should never happen" becomes "this happened six hours ago and now nobody knows why."
The 6-hour-on-one-line pattern shows up almost everywhere AI-generated code lands in legacy systems. The expensive part isn't the line. It's that you can't read intent from variable names the agent picked from training data instead of from a deliberate design choice. Debugging time scales with how much intent you have to reverse-engineer.
That's a great observation. The bug itself is often the easy part. The hard part is reconstructing the reasoning behind the code when there was never any real reasoning to begin with. When intent isn't obvious, every debugging session turns into an archaeology project.
This lands hard.
The “zero to eighty percent” part is exactly the trap. AI can produce a lot of plausible code very quickly, but plausible code is not the same as a coherent repo.
The expensive part usually shows up later, when you have to ask:
Why does this file exist?
What contract was this function supposed to preserve?
Which layer owns this state?
Did this test prove behavior, or just prove the patch?
Did the AI fix the symptom while quietly moving the drift somewhere else?
That’s the part I’ve been working on with Scarab Diagnostic Suite: not replacing the coding agent, but putting a diagnostic layer around it so the repo can prove what is still true after the agent is done.
AI can absolutely speed up implementation. But without diagnostics, it can also speed up entropy.
I think that's the distinction a lot of people are missing. The bottleneck is no longer generating code, it's preserving understanding.
A repo can survive mediocre code. It struggles to survive lost intent.
The questions you listed are exactly the ones AI is weakest at because they're not local code questions, they're system questions. The model can tell you how a function works. It usually can't tell you why that function exists, what invariant it's protecting, or whether the change shifted complexity into a different part of the system.
That's why I like the idea of a diagnostic layer. The real value isn't checking whether the agent wrote valid code, it's checking whether the repository still obeys the contracts and assumptions that existed before the change. In a world where code generation is cheap, proof becomes expensive.
AI doesn't just accelerate implementation. It accelerates change. Diagnostics are what stop accelerated change from becoming accelerated decay.
This nails the real tradeoff. AI gets teams from 0→80% fast, but the debugging tax shows up when async/state boundaries get fuzzy. The "patterns vs consequences" point especially resonated — in my experience the best counterweight is stronger tests and explicit invariants before AI-generated code gets anywhere near production.
That's been my experience too. AI is excellent at recognizing patterns, but production issues are usually about consequences. A function can look perfectly valid in isolation while quietly violating assumptions somewhere else in the system. Strong tests help, but I think explicit invariants are even more valuable because they force the assumptions into the open. The moment those assumptions are written down, both humans and AI have a much harder time accidentally breaking them.
I felt this one. AI is amazing for getting momentum, but the second something weird breaks, you realise how important it is to actually understand every bit of what it gave you. Do you trust AI more for scaffolding than final code?
Definitely. I trust AI far more for scaffolding, boilerplate, and exploring ideas than for final implementation details. The closer code gets to business logic, edge cases, performance, or anything customer-facing, the more I want human judgment involved.
For me, AI is at its best when it's accelerating the journey—not deciding the destination.
You just summarized my nightmare in three sentences, the psychological thriller that I experienced while building a Django application recently. I had asked an AI to generate code for me, and one by one, it produced various snippets. For a moment, I was in awe, thinking that this was something I could never accomplish on my own. However, when I finally started the application, it was a complete mess. The database required the most work; it had missing fields and incorrect field types. Additionally, the middleware wasn't connecting with the database or the views. There were all sorts of other issues too. At one point, I considered deleting everything and starting over. However, I methodically corrected the problems, and it took me two weeks to get the application running.
That's the reason I treat AI like a genius child. It might possess more knowledge than you will accumulate throughout your entire life, but ultimately, it is still a child at the end of the day and is prone to making mistakes. If AI happens to mess up the codebase, I believe the responsibility lies with the person overseeing it, not the AI itself. Blindly trusting AI can lead to disastrous outcomes.
Two weeks fixing integration issues after a few minutes of generation is such a perfect example of the tradeoff. Generating code is becoming cheaper every day. Understanding how all those pieces interact is still where most of the engineering work lives.
I also like your "genius child" analogy. Incredibly capable, incredibly helpful, and still needs supervision before you hand it the keys to production.
The one-line fix after six hours is the real tax. I log webhook + PHP integration failures to CSV with replay notes so the next session does not restart the hunt. Happy to swap debugging habits — Eric, solo builder on elionmusic.com.
That's a smart habit. The actual fix is usually tiny; the expensive part is rediscovering the context that led to it. Capturing replay notes turns future debugging sessions from detective work into a lookup table.
The "confident nonsense" point really resonates.
One thing we've seen at IT Path Solutions is that AI-generated code rarely causes problems on the happy path. The trouble usually starts a few weeks later when someone has to debug an edge case, modify a workflow, or trace through assumptions that were never explicitly documented.
AI has definitely compressed the time it takes to build something. I'm not convinced it has compressed the time it takes to truly understand what was built.
That’s exactly the part people keep skipping. AI compresses building time. It does not compress understanding time.
The scary bugs usually aren’t on the happy path because AI is trained on patterns that look correct. The real damage shows up later when someone touches an edge case and realizes half the architectural decisions were basically undocumented assumptions wearing a nice TypeScript outfit.
At that point you’re no longer writing software. You’re doing digital archaeology with emotional damage.
😂😂😂
AI didn't remove debugging.
It just gave us bugs we didn't write ourselves.
That's probably the funniest and most accurate summary of AI-assisted development I've seen.
We used to spend hours writing bugs. Now we spend hours figuring out why the AI wrote them.
The productivity gain is real, but the work shifted from creation to verification. Instead of asking "How do I implement this?" we're asking "What assumptions did the model make while implementing this?" The code arrives faster than the understanding, and that gap is where most of the pain lives.
I needed this 3 weeks ago. The 'serial killer apartment' line is too real. Confident nonsense hits different when the AI is the one writing the eulogy for your weekend.
I'm convinced the most dangerous AI-generated code isn't the broken stuff. It's the code that's so clean, organized, and confident that you immediately trust it. That's how the psychological thriller gets its sequel.
感谢分享，6 小时这个数字很真实。AI 写代码不是省时间而是把时间从「打字」挪到「看 + 改 + 验证」，节奏完全不一样。