DEV Community

Cover image for Every AI-Generated Line of Code Is a Small Loan — And Eventually, You Have to Pay It Back
Harsh
Harsh

Posted on

Every AI-Generated Line of Code Is a Small Loan — And Eventually, You Have to Pay It Back

The cost of missing mental models

A bug showed up in my personal project last month. Nothing dramatic - a value wasn't updating the way it should have been.

I opened the file. I read it top to bottom. I had no idea what I was looking at.

Not because the code was badly written. It was clean. Well-organized. Doing exactly what I'd asked.

The problem was that I had never actually built a mental model of why it worked the way it did. I'd generated it, glanced at it, and moved on - dozens of times without ever once sitting with it long enough to understand it.

I couldn't fix a bug in my own project because I had never really owned the code I was fixing.

That was the moment I started thinking about AI-generated code differently: not as a gift, but as a loan.

It was a personal side project - nothing fancy just something I was building for myself in the kind of relaxed evening mode where you let AI drive more than you probably should. I'd describe what I wanted, it would generate the function I'd skim it see that it ran, and move to the next thing. For weeks, this felt like a superpower. I was building faster than I ever had.

The bug was the first bill that came due.


The Loan Nobody Told Me I Was Taking Out

Here's the mental model I wish I'd had earlier.

Every time AI generates code for you, it's not handing you something free. It's handing you something on credit. The code works today. The understanding the part that lets you debug it, extend it, explain it to someone else is the interest, and it's due whether you've budgeted for it or not.

AI Generates You Owe
10 lines A few minutes of reading
100 lines Real time understanding the shape of it
200+ lines across a feature A debugging session - whether you like it or not

The debt doesn't show up immediately. That's what makes it dangerous. It shows up later - as a bug you can't diagnose, a PR comment you can't answer, a feature you're afraid to touch because you're not sure what depends on it.

I'd been taking out this loan for weeks without realizing it. The bug was just the first bill that came due.


Why This Isn't Just a "Feeling" The Data Backs It Up

I went looking afterward, half expecting to find that I was just being paranoid. I wasn't.

Recent research on AI-assisted commits has found meaningfully higher rates of correctness and security issues compared to human-written code - the kind of issues that don't show up until someone actually exercises the code path that was never fully understood by the person who shipped it.

There's also a growing body of developer survey data pointing at something less technical and more personal: heavier reliance on AI coding tools correlating with higher reported burnout. Which tracks with what that evening actually felt like once the productivity high wore off not relief but a low-grade anxiety about a codebase I could no longer fully account for.

None of this means AI-assisted coding is bad. It means the debt is real, even when it's invisible, and pretending otherwise doesn't make it go away.


What I Actually Did - Slowly, Not Dramatically

I want to be honest here, because it would be easy to turn this into a bigger story than it was. I didn't delete everything and start over. I didn't swear off AI tools. That would have been its own kind of overcorrection.

What I actually did was slower and less satisfying to write about: I went through the file, section by section, and made myself understand it properly. Not skim it actually trace through what each part was doing and why, the way I would if a colleague had written it and left the company.

It took longer than fixing a bug in code I'd written myself would have. That was the whole point. The time I "saved" generating the code hadn't disappeared it had just been deferred to a moment of my choosing. Except I hadn't gotten to choose the moment. The bug chose it for me.

Once I actually understood the code, the fix took about ten minutes.


The Practice I Actually Kept

The thing that's stuck with me since isn't a dramatic rule. It's a small, boring habit:

Before I move on from anything AI generates, I ask myself one honest question could I explain this to someone else, right now, without looking at it again?

If the answer is no, I don't treat the task as finished. I read it again. I trace the logic. Sometimes I rewrite small pieces just to force the understanding to actually land, rather than passively wash over me.

My Rule: Before I move on from anything AI generates, I ask: Could I explain this to someone else, right now, without looking at it again?

If the answer is no, the task isn't finished.

It's slower in the moment. It's the only thing that's actually kept the debt from quietly piling back up.

I still use AI constantly. I'm not interested in pretending otherwise, and I don't think the answer is to use it less. The answer, for me, has been to stop treating "it runs" as the finish line, and start treating "I understand it" as the actual one.


The Honest Version of the Lesson

I'm not going to end this with a dramatic claim about who will or won't have a job in five years. I don't know that.

What I do know, from one bug in one personal project, is this: the code AI writes for you isn't free just because you didn't type it yourself. Somewhere down the line, something will ask you to account for it a bug, a question, a moment where you need to change it and realize you're not sure what you'd be changing.

You can pay that debt early, a little at a time, by actually understanding what you ship. Or you can pay it all at once, at 11 PM, staring at a bug in your own project, wondering how you got there.

I'd recommend the first one. I learned that the slow way.


Have you had a moment like this - where AI-generated code you'd shipped turned out to be a debt you didn't know you'd taken on? I'd genuinely like to hear how you noticed it, and what you did once you did. 👇

Top comments (70)

Collapse
 
miketalbot profile image
Mike Talbot ⭐ • Edited

Here's the balance. Every line of code written is a quantum of risk; in a team you are constantly taking risks on other team members. I have 13 engineers in my team; do I need to understand what each of them made? I think your argument is that it's me taking the risk with AI, well it's me taking the risk with them too - I'll ask them to fix a problem we find, we'll review big architectural decisions, but no way am I reading every line of code and fully getting it - that's what they do. We all make mistakes. The best way to avoid mistakes is great testing strategy and good architecture.

Outsourcing to AI is the same problem to me as outsourcing to my team, and that works on down the stack I think.

Collapse
 
ranjancse profile image
Ranjan Dailata • Edited

No matter whether the human who crafts the code by hand or the AI does, It's the team lead who is responsible for reviewing the code. Well, it's in-fact the human being who is accountable for and not the "AI".

Collapse
 
miketalbot profile image
Mike Talbot ⭐

Agreed. As the CTO it's my accountability. But it's my accountability to ensure the code is of a high enough quality - not that a human reviewed it.

Thread Thread
 
ranjancse profile image
Ranjan Dailata

Yes, Welcome to the accountability ladder 😀

Collapse
 
gert_ profile image
Gert

AI is just another tool.

Collapse
 
harsh2644 profile image
Harsh

this is a fair challenge, and honestly it made me think harder about where the line actually is.

here's the distinction I keep landing on though when one of your 13 engineers writes something you don't personally understand, the understanding still exists in them. you can ask. they can explain the tradeoff, the edge case, the why. the risk is distributed, but it's not absent.

with AI, if the person prompting it doesn't build that understanding, there's no one left to ask. the model doesn't have a persistent mental model of the decision it made five minutes ago, let alone five months later. it's not outsourced understanding at that point it's just missing understanding, full stop.

so I'd actually agree with you on architecture and testing being the real safety net at team scale that's the right level of abstraction for managing 13 people's code. my argument is narrower: it's aimed at the individual developer prompting AI directly, where the temptation is to skip the "does someone understand this" question entirely, not delegate it to a person who does.

good testing and good architecture absolutely reduce how much any single line needs personal understanding. I just don't think they replace the need for someone to have it.

Collapse
 
miketalbot profile image
Mike Talbot ⭐

So I think you need proper documentation of the spec, but I'm expecting a future AI to work out from that and the state of the system what is going on. If you know the core architecture, for me the rest is detail :S

Thread Thread
 
harsh2644 profile image
Harsh

That's actually a genuinely different bet than the one I was arguing against and I think it's a reasonable one, just further out on the timeline.

right now the AI can work it out from the spec and system state mostly isn't true yet that's exactly the gap this whole thread with UnitBuilds ended up circling around: current models don't reliably hold a persistent, accurate model of what's actually going on across a large system, which is why the guardrails generator approach (constrain what it can produce) works better today than trusting it to reconstruct context on demand.

if that changes if a future model can genuinely infer intent and system state reliably enough that the spec plus the codebase is sufficient then I think you're right, and understanding the detail stops being the bottleneck the same way understanding assembly stopped being necessary once compilers got trustworthy enough.

I just don't think we're there yet, and treating the AI will work it out as already true today is where the debt in the article actually comes from. as a bet on where things are headed though, I don't think you're wrong.

Thread Thread
 
miketalbot profile image
Mike Talbot ⭐

I guess this is all down to your product. Inside the application framework we built for our core product, we are now shipping 2 products entirely built the way I describe. My initial fears have melted away. Maybe that's because of the style of app - these are high-value B2B apps - I don't know.

Maybe in the future "something" happens and we have to do some work. I've unpicked other people's code multiple times in my career; it's part of the job - the difference with AI code is at least the comments are up to date! :)

Thread Thread
 
harsh2644 profile image
Harsh

At least the comments are up to date might be the funniest and most quietly true thing said in this whole thread 😄

and I think you're right that it comes down to the product. two shipped B2B apps built this way with your fears melting away is real evidence not a hunch that's a different category of proof than my here's a bug that scared me once starting point. high value B2B with presumably strong domain constraints and clear specs is probably close to the best-case environment for the AI-infers-from-spec approach to actually hold up, versus something with fuzzier requirements or a lot of implicit business logic nobody wrote down.

maybe in the future something happens and we have to do some work is honestly the most reasonable place to land on this. that's not complacency, that's just accepting the actual risk profile instead of pretending it's zero which is more than most teams shipping AI-generated code are doing right now, in either direction.

appreciate you following this all the way through genuinely reshaped a few things I'd assumed going in.

Collapse
 
gert_ profile image
Gert

I fully share this view. Together with my friend we have been thinking on making sure we give agents, and developers alike, enough guidance to ensure the code is grounded to our architecture and performant. We help the agents to actually receive feedback on their work, think of it as a PR review of a staff engineer.

Not a trivial task, but for us, we are seeing great result.

Collapse
 
unitbuilds profile image
UnitBuilds

It comes down to AI being the junior dev (in conventional software companies) and you're the senior dev. Your job is to have a thorough enough understanding of everything to skim through their code and pick up any flaws. If you dont know what it's writing, it's because you lack the knowledge and should probably review it's code more often to get a thorough understanding of what, why, when, where, how, who.

The knowledge debt is real, but it's entirely on us as developers. We just 'trust' .net framework to do it's job, because it's compiled and validated by millions. But if you've ever dealt in-depth with python, you'd know that more often than not, packages arent as bulletproof and if you want them to work right, you need to make edits to them just to get them working. That requires an incredible amount of knowledge regarding what and how it's doing it.

Essentially, if you take a loan, you need to repay that loan, incrementally, or default when it builds up too much. What you save in writing it, you need to spend on reading it. Except it's a negative interest rate. The more you read and review, the less it costs you to do so. Compounded over an entire project, you get a net-gain. You gain knowledge, speed and accuracy, by understanding that AI makes mistakes and you should review every single thing it does explicitly. Unless of course you build guardrails for it yourself, so you know exactly what's possible and what isnt.

Collapse
 
harsh2644 profile image
Harsh

Negative interest rate is a genuinely sharp reframe the more you review, the cheaper reviewing gets, because you're building the mental model that makes the next review faster. that compounds in the opposite direction from how I framed it, and I think you're right.

the NET vs Python packages comparison is a good way to ground it too. trust scales with how battle-tested something is, not with how confident it looks. AI-generated code hasn't earned that trust yet, no matter how clean it reads which is exactly why treating it like a junior dev's PR, not a senior dev's, matters.

where I'd push back slightly: the senior junior framing works well for a single PR, but the failure mode I was pointing at is more about compounding neglect than a single skipped review. reviewing thoroughly once is a habit. reviewing thoroughly every time, on the 50th feature when the deadline is tight and the code looks fine that's where the debt actually accumulates, even for developers who genuinely have the knowledge to catch it.

the guardrails point is the real answer though. if you've built the constraints AI has to operate inside, you've effectively moved the trust from reading every line to trusting the system that bounds what lines are possible which is a much better place to end up.

Collapse
 
unitbuilds profile image
UnitBuilds

I'd have to push back on that 1. That's a choice. "When is work considered done" in other words. Deadlines are deadlines, but you wont ship code with compile errors, nor should you ship AI generated code that hasnt been validated. AI sped up writing it enough that you can afford to take the time to review it properly. If you cant do that within the deadline, then the deadline was fundamentally too tight to fit. Zero-compromise on quality is what the standard is for 2026 developers, we have to review EVERYTHING the AI writes, because it will make mistakes and it will compile perfectly while doing so. If you wrote it yourself, it would still take you 10x longer than it would reviewing it thoroughly. So the net-gain is still there, but if you try to push to the bleeding edge of speed, you will bleed quality with every PR. Validation is non-optional, that 50th feature when deadline is tight, is exactly when zero-compromise mindset should kick in, because you already wrote 49 features that gave you ample time to reflect and fix any outstanding issues, nr 50 is the critical check, because if you were thorough with your reviewing for 49 features, that last 1 shouldnt take you any longer than any 1 of those, but trying to skip a check, is a recipe for disaster and a 5 min late PR is better than a broken PR, because the day the error surfaces, it'll take 100x the time to fix it, because the system has expanded since then.

If you want to work with AI, you need to be the structure that contains it. You are the final validation layer, you are the one who scopes features accurately so it doesnt bleed over, you are the one who instructs, it follows your command to the best of it's ability. If that's too much responsibility for you, then write guardrails that limit the scope you need to worry about, so instead of reviewing 100 lines for every single possible mistake, you know 90% of the coverage is cleared by guardrails and you just have 11% worth to validate (11%, because there's always 1% you didnt account for).

Take linting mistakes. You can wire up a linter check that covers it, you can trust it, so you dont have to validate it. Syntax, is the same. Unit Tests and Integration tests cover even more ground for you. If you really want it to be robust and limit what you actually need to check, dont rely on AI generated code, rely on AI initialized automation macro generated code (eg. Roslyn). You define the setup, you validate your framework, now it's 1 less headache. Take V.A.L.I.D. for instance, I wrote it to replace CSLA, then I built the generator for it, so it's easier to use, now it generates 82% of my codebase, making sure that the UI-DAL-BO-Rules-Endpoint-UnitTests-FuzzerTests-MCP setup is flawless every single time. It took me months to build, but it saves me 80% of my time and tokens doing anything on from that day and better yet, I can trust it to be accurate 100% of the time.

Thread Thread
 
harsh2644 profile image
Harsh

A 5 min late PR is better than a broken PR that's the line that actually settles the argument. I was treating deadline pressure as a fixed constraint that review time has to bend around. you're right that it's a choice about what done means, and that's a much more honest way to frame it.

the guardrails math you laid out linting covers X tests cover Y leaving you a smaller surface to actually think about is exactly the shift from review everything manually to review what the system can't already guarantee. that's a genuinely better architecture for trust than what I described.

V. A. L.I.D. generating 82% of your codebase with that level of guaranteed correctness is a strong example of what "the structure that contains it" actually looks like in practice, not just in theory. months of upfront investment for that kind of compounding return is exactly the trade the article was gesturing at without spelling out how to build it.

I think the difference between us is smaller than it looked a comment ago you're describing the mature version of what I was calling paying down the debt build the guardrails once, review only the genuine gap, and treat zero-compromise as non-negotiable rather than aspirational. that's a sharper standard than mine.

Thread Thread
 
unitbuilds profile image
UnitBuilds

The gap isnt big, the gap is just a 1 liner, 'when is it done'. If you say when it's compiled, you're wrong and it'll bite you later. If you say it's after you've reviewed it, you're wrong and it'll bite you all the same later. If you say it's after you've reviewed and validated it, then you're safe(r). AI makes mistakes, so do we and reviewing covers some of the bases, but not all, because it's just another model (you) reviewing the code. You cant guarantee the AI's correctness any more than you can your own for reviewing it, because it 'looks right' is alot different than 'it is right'. That's why the guardrails are your ownly guarantees and using tools like Roslyn to write code will always produce better results than having AI write it itself. Case in point, when an AI has to do a massive edit across files, it writes a python script, instead of doing 100 edits manually. It can trust the script more than it can trust itself to remember the task at step 99. Sure it saves tokens, but context is more important than token burn. 100 identical fixes dilutes it's memory into being fixated on it. Whereas a script is just a single memory.

Thread Thread
 
unitbuilds profile image
UnitBuilds

It's like trusting a toddler on a bicycle to not fall, correcting them and correcting them, hoping for the best, instead of putting on side wheels to guarantee it

Thread Thread
 
harsh2644 profile image
Harsh

Looks right is a lot different than it is right that applies to human review just as much as it applies to trusting AI, and that's the part that actually reframes the whole argument for me. a senior dev skimming a PR is pattern-matching against familiarity, not exhaustively verifying. the senior dev catches it story I leaned on earlier assumes a level of rigor that mostly doesn't survive contact with a real deadline, even a reasonable one.

the python script example is the cleanest illustration of this whole thread. an AI trusting a deterministic script over its own memory at step 99 is basically admitting that even the model doesn't trust its own judgment at scale. if the thing generating the code doesn't trust itself past a certain point, no amount of human understanding of the output fixes that you'd be verifying a process that was already unreliable at the source.

the toddler and side wheels analogy lands because it kills the idea that guardrails are a crutch you graduate out of. nobody expects the kid to eventually not need the bike to have wheels. the guardrails aren't scaffolding you remove once you're good enough they're the permanent architecture that makes the whole thing safe to ride at speed in the first place.

I think you've actually talked me out of the position I opened with. it's not understand what you ship. it's build the thing that makes understanding unnecessary for 90% of what ships, and reserve real scrutiny for the 10% no guardrail could have caught. that's a better articulation of the article's actual point than the article made.

Thread Thread
 
unitbuilds profile image
UnitBuilds

And if you generate the difficult and error prone stuff, suddenly reviewing becomes easier. Take V.A.L.I.D. for instance, all you need to review is the DTO (ValidObjects) with their constraints and the HTML markup for the blazor frontend. Everything else is generated by Roslyn and is guaranteed robust, safe, secure, reliable under heavy load. That's how you should be using AI, to feed a robust machine, instead of expecting it to do it all flawlessly. It saves you alot on tokens and it makes the model less error prone, because it has only 2 simple tasks that will flag at compile time if they made a mistake. It beats writing it manually and it beats trusting AI to write it, even if you validate it, because your throughput is far better and guaranteed working.

Thread Thread
 
harsh2644 profile image
Harsh

This is the cleanest version of the argument yet. you're not asking AI to write the system you're asking it to fill in two narrow, well-typed slots inside a system that already guarantees everything around them. that's a fundamentally different risk profile than AI writes a feature, human reviews the feature.

and the compile-time flagging point is what actually makes it airtight. a mistake in the DTO or the markup doesn't slip through as a subtle runtime bug three months later it fails immediately, loudly, at the exact moment it's introduced. that's the opposite of the looks right but isn't problem we were just talking about. you've moved the failure mode from silent to loud.

I think this is the actual answer to the question the whole thread has been circling: the loan doesn't get smaller by reviewing harder. it gets smaller by shrinking what needs review in the first place, down to the two things a human is actually positioned to judge well intent (the DTO shape) and presentation (the markup) and letting a deterministic system own everything in between.

that's a genuinely more precise standard than understand what you ship. it's understand the 2 things only you can validate, and trust the system for the rest because it's structurally incapable of drifting.

Genuinely Appreciate you pushing back this thoroughly this thread ended up teaching me more than the article I wrote it under.

Thread Thread
 
unitbuilds profile image
UnitBuilds

And best part, because it's that simple, it saves you on tokens, infact you can even drop to a cheaper model, because of the drop in complexity too. It also saves you time, so you can scale throughput even higher. It's how my 500k LOC autonomous accounting suite was built in 2 weeks. By generating the difficult 82%, all it had to do was write the basics

Thread Thread
 
harsh2644 profile image
Harsh

500k lines of an autonomous accounting suite in 2 weeks is the number that actually makes this whole argument land. that's not a productivity gain, that's a different category of what's possible and it only works because the 82% wasn't AI trying its best it was a system that literally cannot be wrong in the ways that matter for something as unforgiving as accounting.

the cheaper model point is the one I didn't see coming but makes complete sense in hindsight. if the remaining 18% is narrow and well-typed, you don't need the most capable model to fill it in reliably you need consistency on a small surface, not general intelligence on a large one. that's a cost curve most teams have backwards, throwing the most expensive model at the whole problem instead of the smallest capable model at the smallest necessary slice.

this thread genuinely changed how I'd write this article if I were starting over. thanks for sticking with it through all of this.

Collapse
 
ranjancse profile image
Ranjan Dailata • Edited

The concept of micro loan is in-deed very interesting. However, before buying a loan, it's the responsibility of the human to ask questions. I am going to literally consider a laymen human and their loan taking approach 😁

  1. Whether they are affordable?
  2. Are they able to understand the terms and conditions of the loan.
  3. On what liability is one interested to go with the loan?
  4. Is it really worth the time and money?
  5. Does the family know about their head of the household taking a loan?
  6. There is a potential of unable to pay-back the loan? Hence, what is the mitigation risk? Do you borrow more loan from someone else?
Collapse
 
harsh2644 profile image
Harsh

This is a genuinely great extension of the metaphor especially 6. borrowing more loan from someone else is exactly what happens when you ask AI to explain AI's own code instead of actually understanding it yourself. you haven't paid the debt, you've just refinanced it.

might steal this whole framework for a follow-up 😁

Collapse
 
kartik-nvjk profile image
Kartik N V J K

The part that landed for me is that the code was clean and still unfixable, because cleanliness and understanding are two things you were paying for separately. I've started treating a generated diff as unmerged until I can explain out loud why it works, which catches the borrowed-understanding gap before it compounds. Do you have a lightweight habit that forces that, or is it discipline in the moment?

Collapse
 
harsh2644 profile image
Harsh

Cleanliness and understanding are two things you were paying for separately that's a sharper way of saying what the article was reaching for. clean code buys you readability. it doesn't buy you comprehension. those got conflated for me for a long time.

the unmerged until I can explain it out loud rule is close to what I actually do, but I'll be honest it's mostly discipline in the moment, not a system. the closest thing I have to a habit rather than willpower: I do it right after generation, before moving to the next file. if I let even one more file get written in between, the explanation check quietly turns into a formality instead of a real one, because my attention's already moved on.

the other thing that's helped more than I expected: if I can't explain a piece in under a minute, I don't debug my way to understanding it later I rewrite that specific piece immediately, even if it technically works. rewriting forces the model to actually form. re-reading doesn't, for me at least.

still mostly discipline though not a system that runs without me. curious if you've found anything that survives you being tired or rushed, because that's usually where mine breaks.

Collapse
 
mayank609 profile image
Mayank Bansal

Curious whether others have noticed this too:
AI seems to reduce the time spent writing code, but increase the importance of understanding architecture. The less time I spend typing, the more valuable it becomes to understand the boundaries, invariants, and data flow of the system. That's where most of my debugging time goes now.

Collapse
 
harsh2644 profile image
Harsh

yes and I think this is the shift that most AI makes you faster takes completely miss. it's not that skill requirements went down, they moved. the skill that used to be partially covered by the slow, forced act of typing every line building a feel for boundaries and data flow just by having to type your way through them now has to be built deliberately, because generation speed skips right past that friction.

that's where most of my debugging time goes now matches what's come up elsewhere in this thread too. the bugs people are hitting aren't usually syntax or logic errors within a single function anymore they're architecture-shaped bugs, where the code is locally fine and the invariant it silently depended on somewhere else got violated. that's a much harder category of bug to catch by reading any single diff carefully, which is probably why it's eating more of your time than the actual writing ever did.

Collapse
 
jugeni profile image
Mike Czerwinski

The explanation check is the right instinct, and it has one false-positive mode worth closing, because it is the exact mode AI code exploits. "Could I explain this right now" measures whether you have a model of the code, not whether you have the right one. Generated code is fluent and plausible by construction, so it reads as if it means what you would expect, which means you can produce a confident explanation that is wrong. You skim the function, it looks like it does X, you explain X cleanly, and it does X-prime in the edge case you did not read for. The check passes, and now you have certified the debt as paid, which is worse than knowing it is outstanding, because a debt you think you cleared leaves the ledger.

It fails because the check reads the same fluent surface that created the risk, so it inherits the risk's blind spot. Your ability to explain the code is correlated with how plausible it looks, not with whether it is correct. The fix is to decorrelate the check from the fluency: not can-I-explain-it, which reads your own model, but did-my-prediction-match, which the code cannot fake. Before running it, say what it returns on one specific input, or state the invariant you think it holds and write the input that would break it, then run it. A prediction is falsifiable by the code. An explanation is not, it only has to sound right, and sounding right is precisely what the generator optimized for. Your own weeks-long bug is the clean case: you could have explained that code any day that week and been wrong, because skimming produces explanations and nothing forced your model to touch reality until the bug did. The bug was a prediction check you did not run on purpose.

Collapse
 
harsh2644 profile image
Harsh

Your own weeks-long bug is the clean case: you could have explained that code any day that week and been wrong that's uncomfortably accurate, and it's the exact failure mode the article never named. I passed my own explanation check the whole time. I just wasn't checking against reality I was checking against a plausible story my own reading had constructed.

the distinction between can I explain it and did my prediction match is sharper than anything I've used so far in this thread. explanation is unfalsifiable by design it only has to be internally consistent with itself, not with the code's actual behavior. prediction forces contact with the thing you're supposedly verifying. that's the difference between reading comprehension and an actual test.

A debt you think you cleared leaves the ledger might be the single best sentence in this entire comment section. false confidence isn't neutral, it's worse than acknowledged uncertainty, because at least uncertainty keeps you looking. certainty stops the search.

I think this replaces the rule I'd give people now don't ask if you can explain the code, state what a specific input returns before you run it, and see if you're right. that's a test AI-generated fluency can't pass by just sounding plausible.

Collapse
 
jugeni profile image
Mike Czerwinski

Explanation is unfalsifiable by design is exactly it, and it is worth keeping because it names why the swap works: a prediction has a failure mode an explanation does not, it can be wrong in a way you cannot talk your way out of, because the code returns what it returns and does not care about your story.

One guard so the new rule does not re-import the old hole at a different layer: a prediction is only as good as the input you pick. Predict the happy path and it passes for the same reason the explanation did, you predict X, it returns X, both of you blind to the edge case neither looked at. So the rule is not just state-what-it-returns-then-run, it is state it for the input you are least sure about, the one where your model of the code is thinnest. The boring input tests your reading comprehension a second time. The input you are nervous about is the one that tests the code. Same move you already landed, just aimed where the debt actually hides.

Collapse
 
andreas_mller_2fd27cf578 profile image
Andreas Müller

We thankfully had only very small moments of this happening in our project. The reason is that the thing you've discovered, we've been doing from the beginning. We put it this way: If you make the GIT commit, it's your code. The AI doesn't own the code it generates for you, you do. It's all about ownership, really.

You can't take ownership of something you don't understand. So you have to dig in and understand it if you want to own it. And like it or not, as you found out: You own it. The AI isn't accountable for anything, they even tell you so outright in all their warning messages about how AI code can have bugs.

Me and my colleague choose to stay on the slower, but safer side of the spectrum: We own everything we check in, so we also understand everything we check in. With very, very few exceptions and even then only if the automated test unambiguously shows that it works. We can also afford to do this because we work on a very critical product, where correctness rules and speed is only a secondary concern (to us).

So I think with AI, there is always this balance: How much can you afford to just trust it in the name of speed, and how much do you have to check? In critical systems, you're on the safety end of the spectrum. In throwaway systems, you're on the speed end of the spectrum.

And most software is probably somewhere in between.

Collapse
 
harsh2644 profile image
Harsh

The AI doesn't own the code it generates for you, you do that's the cleanest one-line summary of the entire article. ownership is the right word for it, sharper than understanding because ownership implies accountability, and accountability is exactly what disappears when you skip the understanding step. the git commit line makes it almost legally precise your name is on it whether or not you actually built the mental model behind it.

the speed-versus-safety spectrum you're describing maps well onto what's come up elsewhere in this thread someone earlier was describing near-total automation with guardrails for a B2B product and having their initial fears melt away, and I think the honest reconciliation is exactly what you said: it depends on where the system sits on that spectrum. critical, correctness-first systems justify the slower own everything end. throwaway or low-stakes systems can justify leaning further toward trust.

what I think your comment adds that the article didn't have is the framing that this isn't really a technical decision, it's a values decision made explicit ahead of time we choose to stay on the slower but safer side rather than something that gets discovered by accident when a bug forces the question. deciding your position on the spectrum in advance, for the specific system you're building, seems like the actual skill here, more than any single review habit.

Collapse
 
jason_wlim_fc3470b416654d profile image
Jason Wlim

My habit is to review the changes after the AI ​​generates the code, understand whether its processing logic is reasonable, and then test it. Only after it passes all tests will I submit and merge it for release. I feel that by understanding the processing logic, I have essentially participated in the processing myself.

Collapse
 
harsh2644 profile image
Harsh

by understanding the processing logic, I have essentially participated in the processing myself that's a good way to put what the whole article was reaching for. the debt isn't in whether AI or a human typed the characters, it's in whether a human actually thought through the logic before it shipped.

your workflow review, understand, test, then merge is basically the loan getting paid off before it's even taken out, rather than after. that ordering matters more than people give it credit for.

Collapse
 
klvenky profile image
Venkatesh KL • Edited

I am the Lead Dev/Staff Engineer on an AI Product (for almost an year & half) which, I cannot name since I work in Governance & Compliance. I love the crux of this post. I can relate to so many things.

The biggest problems with AI assisted coding are:

  1. When a customer faces an issue in production, the need for people who understand the system with limited access to logs or the data the customer is using to build something. At that point of time, this debt gets paid in time & stress

  2. The dopamine that people get used to by merging PRs at a rapid pace is not a problem, but, the exponential reliance on QA for validating that the small change which was done is actually the small change that it was supposed to be

  3. Loss of curious junior engineers. When everything's a text or voice note away, then they lose the curiosity. Builders will be rewarded but the no. of builders is way less as we all know it

  4. Scaling is a hard problem when we have to solve it right away. I know the success ratio of any software product is pretty low, but when it needs to scale, it should be ready to scale. That would be such a big bummer for a lot of AI generated stuff out there.

Collapse
 
harsh2644 profile image
Harsh

appreciate this genuinely valuable to hear from someone at the staff/lead level in a governance-heavy environment, where the stakes on all four of these are higher than most of us are dealing with.

1 is the one that hits hardest: production incident, limited log access, and the debt comes due in time and stress simultaneously. that's the worst possible moment to discover nobody built the mental model, because you're paying interest and principal at once, under pressure, with less visibility than you'd have in a calm debugging session.

2 is a distinction I hadn't drawn clearly the dopamine of merging fast isn't itself the problem, it's that QA absorbs the cost of verifying small instead of the developer. that's the debt getting quietly transferred to a different team, which probably feels like progress right up until QA becomes the bottleneck everyone complains about.

3 is the one I think about most and have the least good answer for. if curiosity gets outsourced before it's ever exercised, juniors don't get the reps that build the instinct in the first place it's not that they lose a skill, it's that they may never build it. that seems like the highest long-term cost of all four, even though it's the least visible on any quarterly metric.

4 ready to scale vs successful at small scale is a distinction AI-generated code doesn't reliably know to make, because it's optimizing for the immediate request, not for the load profile six months out.

genuinely appreciate the depth here this is a more complete list than what I laid out in the article.

Collapse
 
eduzsh profile image
Edu Peralta

The could I explain this without looking at it again test is exactly the bar I hold myself to before merging anything an agent wrote, and it catches more than people expect. What compounds the debt fastest in my experience is not the code itself, it is the PR description the agent writes to go with it. A clean paragraph that says refactored the validation logic for clarity reads like understanding, so you skim the diff instead of reading it, and the loan quietly gets bigger. The fix that has worked for me is treating the agent's own summary as marketing copy rather than documentation, and reading the diff first before I let myself read what it claims the diff does. Your line count debt schedule is a good rule of thumb, I would just add that the interest rate goes up a lot faster when the summary is more confident than the code deserves.

Collapse
 
harsh2644 profile image
Harsh

Treating the agent's own summary as marketing copy rather than documentation that's a genuinely important distinction the article missed entirely. I was worried about the code being the thing that creates false confidence. you're pointing at something upstream of that: the summary shapes how carefully you even read the code in the first place.

refactored the validation logic for clarity reads like a colleague vouching for their own work, except the agent has no stake in whether that description is accurate it's optimizing for a plausible-sounding summary, not an honest one. and a confident, well-written summary is exactly the thing that makes you skim rather than read closely, which is the opposite of what should happen when the source is least accountable.

reading the diff before reading the claimed description is a genuinely simple high-leverage habit it inverts the order most people naturally default to and the order is doing more work than people realize. the interest rate point sharpens the whole metaphor too: it's not just size of the diff that predicts debt, it's the gap between how confident the summary sounds and how much scrutiny the code actually got as a result.

Some comments may only be visible to logged-in visitors. Sign in to view all comments.