TL;DR: We didn't set out to run an experiment — AI agents just kept eating more of our sprint over six months until they were writing roughly half our code. Velocity went up. So did a specific, predictable category of bugs: confident-looking code that was subtly wrong in ways our old review process wasn't built to catch. The fix wasn't "use AI less." It was rebuilding how we review, not whether we accept AI-written code.
The number that started this post
Industry-wide, GitHub's own research puts AI-assisted code at roughly 46% of what active Copilot users ship, and Google and Microsoft have both put their internal AI-generated code share in the 20-30% range on earnings calls. Our number landed higher than that once we moved from autocomplete-style suggestions to actual agentic workflows — tools that plan a task, write across multiple files, run tests, and open a PR with minimal hand-holding.
Six months in, that's close to half of merged lines on a typical sprint. This post is what we learned getting there, written the way I wish someone had written it for us before we started.
What genuinely got better
Being fair first, because most "we tried AI agents" posts on here skew either breathless or bitter, and neither is useful to you.
- Boilerplate and scaffolding basically disappeared as a time cost. New CRUD endpoints, test fixtures, migration files, repetitive config — the stuff nobody wants to write anyway got handled in minutes instead of an afternoon.
- First-draft test coverage went up, not down. Agents are relentless about writing the unhappy-path test you'd skip when you're tired. Coverage numbers improved even before we changed anything else.
- Onboarding a new engineer to an unfamiliar part of the codebase got faster, because they could ask an agent to explain and then verify the explanation against the actual code, instead of cornering a senior dev.
- Time-to-first-PR on small, well-scoped tickets dropped noticeably. Nothing dramatic on complex work, but on the "add this field and thread it through" tickets, it was a real, repeatable speedup.
None of this is controversial anymore. If your team isn't seeing at least some of this, the gap is usually task selection or prompting discipline, not the tools.
What actually broke, and it wasn't what we expected
We braced for the agent to write bad code. That's not really what happened. What happened was worse to catch, because it looked right.
1. Code that passed review because it read fluently, not because it was correct.
AI-generated code has a specific texture — clean naming, consistent style, plausible-looking logic — that triggers the same "this looks fine" pattern-match reviewers use for competent human code. We started seeing subtly wrong edge-case handling slip through specifically because it didn't trigger the usual "something looks off here" instinct. GitClear's analysis of over 200 million lines of code found code churn (changes reverted or rewritten within two weeks) climbing sharply alongside AI adoption industry-wide, and that matched what we saw internally almost exactly.
2. A quiet decline in refactoring.
When an agent can regenerate a working version of something in minutes, the incentive to actually clean up the underlying structure drops. We caught ourselves accumulating duplicated logic across files that a human, writing it by hand, would have naturally consolidated into one function out of sheer annoyance. The agent doesn't get annoyed.
3. Security review became a real, not theoretical, line item.
This is the one I'd flag hardest to anyone reading this before scaling agent use. Independent analysis of AI-coauthored pull requests has found meaningfully elevated vulnerability rates compared to human-only PRs, and we don't think our team is a special case — we think most teams haven't looked closely enough yet to notice. We now treat "did an agent touch this file" as a routine flag in our security review, the same way we'd flag a file touching auth or payments.
4. Debugging shifted from "why is this broken" to "why did it do this."
When a human writes bad code, you can usually trace their reasoning. When an agent produces something wrong, the reasoning it reports and the reasoning that actually produced the output aren't always the same thing. That's a genuinely different debugging skill than most of us were trained on, and it took real time to build.
What we changed, concretely
- PRs get labeled by how much of the diff is agent-generated, not as a gotcha, but because it changes what kind of review the PR needs.
- Anything touching auth, payments, or external data boundaries gets a second human reviewer if an agent wrote a meaningful share of it — no exceptions, regardless of how confident the code looks.
- We stopped measuring "lines shipped per week" as a success metric. It was quietly rewarding accepting more agent output with less scrutiny, which is exactly the wrong incentive.
- Refactoring got a protected time slot instead of being opportunistic, because opportunistic refactoring mostly stopped happening on its own.
Where this leaves us
We're not walking any of it back. The productivity case is real and we're not interested in performative AI-skepticism for its own sake. But "AI writes our code now" turned out to be the easy part. The actual work was rebuilding review, security screening, and code-health habits around a contributor that writes fast, writes confidently, and doesn't get tired or embarrassed — which, it turns out, were exactly the human traits our old review process was quietly relying on.
I run engineering at 6senseHQ, where this is now just how we build for clients — agent-assisted, human-owned, reviewed like it matters, because it does.
FAQ
Does using AI coding agents actually make delivery faster?
Yes, on well-scoped, boilerplate-heavy work, meaningfully so. On complex or ambiguous work the gain shrinks and sometimes disappears, because the bottleneck shifts to review and verification instead of typing.
Is AI-generated code less secure by default?
Independent research on AI-coauthored pull requests points to higher vulnerability rates than human-only PRs, though the gap narrows significantly with deliberate review practices tailored to agent output specifically.
What's the single highest-leverage process change for a team adopting AI agents?
Stop reviewing agent-written code the same way you review human-written code. The failure modes are different, and a review process tuned to catch human mistakes will systematically miss the ones agents make.
If you're vetting a dev team on their AI-assisted practices, ask them what broke — not what got faster. Anyone can tell you about the speed gains. The teams worth hiring can tell you specifically what they changed in review because of what went wrong.
Top comments (13)
"Clean naming, consistent style, plausible logic triggers the same 'this looks fine' pattern-match reviewers use for competent human code." That's the sharpest description I've seen of the exact trap, and it's the one I spend most of my time defending against as a non-engineer who builds with AI. Broken code I can catch — it errors. The fluent-but-wrong version sails past me precisely because it looks like the competent code I was hoping for.
Your fix being "rebuild how we review, not whether we accept" is the part I wish more people led with. I landed on the low-budget version of your agent-percentage labels: any change to auth, or anything that touches real patient-facing behavior in my tools, doesn't get to close on "it looks right" — it has to close on a check that ran and passed. The reviewer's "this looks fine" and the system's "this actually works" come from different places, and AI-written code is exactly where those two quietly diverge.
"Ask them what broke, not what got faster" is going in my notebook as both a hiring filter and a self-audit. Fluency is what got faster; the whole discipline is refusing to let fluency stand in for correctness. Great retrospective — the kind that's honest about the wins and still names the wreckage.
Really glad the "check that ran and passed" framing landed — that's exactly the line I'd draw too, and I think you've actually generalized it better than I did. I was mostly talking about auth in my own retrospective; you're right that it applies to anything touching real user-facing behavior, patient data being the sharpest version of that.
One thing I'd add from the other side of the table: this gets harder, not easier, once you're working with an outsourced or dedicated team instead of writing the code yourself. When the person reviewing isn't the person who prompted the AI, "this looks fine" gets a second layer of trust stacked on it — the reviewer isn't just trusting the model, they're trusting that whoever prompted it caught the obvious stuff first. That's usually where the fluent-but-wrong code actually gets through, not at the first pass.
The fix ends up being the same one you landed on, just enforced contractually instead of personally: no merge on sensitive paths without a named check that ran, full stop, regardless of who — human or AI — wrote the diff. Doesn't matter how good the vendor's reputation is; "trust me" and "here's the test output" can't be treated as equivalent inputs to a merge decision.
Appreciate you sitting with the piece long enough to push back on it usefully — that's rarer than the praise.
The double-stacked trust is the part I hadn't seen, and it's the same bug I've been chasing all week, wearing an org chart. When the reviewer isn't the prompter, "this looks fine" quietly inherits a second assumption — "and whoever prompted it caught the obvious stuff" — which is just a self-report from the prompter that the reviewer is now grading as if it were evidence. That's the exact shape of a monitor reading a service's own "I'm healthy": the auditor stands one layer outside and still ends up trusting a report the audited party wrote. You've found the human-org version of it, and you're right that it's where the fluent-wrong code actually lands — not the first pass, the handoff.
And "believe me" and "here's the test output" not being equivalent inputs to a merge is the whole thing in one line. They feel interchangeable in the moment because both arrive as confidence — but one is a private variable that dies with the person holding it, and the other is an artifact anyone downstream can re-run. A merge that treats them as equal is pricing a promise at the same rate as a receipt. Contractual enforcement is just the honest admission that personal enforcement doesn't survive a handoff to someone who owes your caution nothing.
The clinical mirror is almost too on-the-nose: this is the shift-change problem. "Patient's fine" from the outgoing nurse is believe-me; the actual vitals on the chart are the test output. Hospitals learned the hard way that verbal handoffs lose people, so they force structured ones — the value, not the verdict, crosses the boundary. Same fix, same reason: the boundary between two people is exactly where confidence stops being transferable and only an artifact makes it across. Thanks for pushing the piece into the place it actually gets hard — the code was never the risk, the handoff was.
The "trust me vs. here's the test output" line is worth stealing outright — that's the cleanest way I've seen the actual failure mode stated. It also points at something worth naming explicitly: on a dedicated team, that check needs to be visible to you, not just to the vendor's internal QA. We've seen contracts where "tests run before merge" is technically true, but the client never sees the output — they're trusting the vendor's internal process the same way the vendor is trusting the model. The fix isn't just "require a named check," it's making sure the check's output is part of what gets handed to the client, not just part of the vendor's internal gate.
Practically that's meant asking clients to request CI output or coverage reports on sensitive paths as a standing deliverable, not a one-off audit — the same way you wouldn't accept "we reviewed it" from a contractor without the actual review artifact. Cheap to ask for, and it collapses that second layer of trust you're describing back down to one.
"The check needs to be visible to you, not just to the vendor's internal QA" is the upgrade, because it catches the version I'd have missed: a check can run, pass, and still be a self-report if its output never crosses the boundary to you. "Tests run before merge" with nothing shipped is just "trust me" wearing a process diagram — you're not seeing the artifact, you're trusting that the artifact exists. Same failure, one level more sophisticated: the receipt was written and then kept in the drawer of the party being audited.
The standing-deliverable-vs-one-off distinction is the part that does the most work, and it lines up with something I keep hitting from the reliability side. A one-off audit only proves behavior at audit time — the vendor performs when watched and relaxes when not, same as anyone. A standing deliverable removes the "when watched" entirely: if the CI output ships with every sensitive change, there's no moment the check isn't observed, so there's no gap left between "performs under audit" and "performs." You've quietly converted an audit into a property of the system instead of an event that happens to it.
The clinical mirror is the medical record. "We reviewed it" is the drawer; the chart is the standing deliverable — the value is recorded every time, and the external regulator's power is precisely that they can pull any chart, not that they scheduled a visit. Hospitals didn't get safer by auditing harder; they made the artifact mandatory and continuous, so the auditor became the format, not a person who shows up. Requesting CI output as a standing deliverable is the same move: cheap to ask for, and it turns "did you check" from a question you have to remember to ask into a thing that's just always on the table. Collapsing two layers of trust to one by making the artifact travel — that's the whole discipline in a contract clause.
"The receipt kept in the drawer of the party being audited" — that's the whole vulnerability in one image, and it's sharper than anything I'd have gotten to. The medical-chart analogy earns its keep too: the auditor becoming the format rather than a visitor is the actual mechanism, not just a nice metaphor. One thing worth guarding against once you make the artifact mandatory — it can start getting gamed for shape rather than substance, the way charts sometimes get filled out to satisfy the audit rather than to reflect what happened. Making the check a standing deliverable removes the "were they watched" question, but it opens a new one: is anybody still reading it, or has it become a box that gets checked because the box exists?
That's the failure that eats every compliance system eventually, and you named the tell exactly: the box gets checked because the box exists. Mandating the artifact removes "were they watched" and quietly installs a subtler self-report — the shape of a check, produced to satisfy the format rather than to record what happened. A chart filled out for the audit is a receipt written by the audited party again, one level of sophistication up. You can't mandate your way out, because "produce the artifact" and "produce a true artifact" are different requirements, and only the first is enforceable by rule.
The clinical answer isn't more auditing — it's making the artifact load-bearing instead of just mandatory. A pain score that only gets charted gets gamed; a pain score that triggers the analgesic order can't, because a fake number now produces a fake action and the patient's response contradicts it fast. The moment a value drives a downstream decision, gaming it stops being free — the lie has to come true or it exposes itself. The fix for "is anyone still reading it" is to wire the artifact to something that visibly breaks when it's wrong, so the reader is a consequence, not a person you're hoping still cares.
Which lands on the same floor this whole thread keeps hitting: an artifact is only alive while something actually depends on it. In the drawer, it's a self-report nobody sees; mandatory-but-unread, it's a self-report everybody's stopped seeing; load-bearing, it's the one version that can't be quietly faked, because reality downstream does the reading whether a human remembers to or not. The standing deliverable is necessary but not sufficient — the sufficient part is that the receipt has to matter to the next step, not just arrive at it. You keep finding the next drawer; best thread I've been in here.
Fair — and it points at the actual sequencing problem: you can't wire the artifact to a real consequence until you trust it, but you can't build trust without first letting it run unwired for a while. Every load-bearing check starts as a mandatory-but-overridable one. The honest version of "make it load-bearing" is probably "shrink the override window on a schedule," not flip a switch — otherwise the first genuine emergency just becomes the precedent for keeping the drawer open indefinitely.
This is the part I'd actually gotten wrong by treating it as a switch, and the sequencing you're describing is the real shape. You can't wire to a consequence you don't yet trust, and you can't earn the trust without running unwired — so the honest artifact during that window isn't the check itself, it's the override log.
The clinical version of "shrink the window on a schedule" is really "make the override cheap to take but impossible to take silently." A new safety rule ships overridable, but every override has to log a reason — so overriding stays available for the genuine emergency and simultaneously becomes data. Now the schedule for tightening isn't a calendar, it's the override rate: when the reason column stops filling up, the window has earned the right to close; when it's still full, you learn whether the rule is too strict or the process is. You don't guess when to flip the switch — the log tells you it's already flipped in practice.
Which defuses your precedent problem directly. The first real emergency doesn't become "the drawer stays open" if every override is itself a load-bearing artifact — one that gets walked afterward, the way a good exception review asks "was this the rare true case, or the first crack in the rule." An override that costs a reason and a look-back can't quietly become the default; a silent one is the default by the second week. So the counted override is the bridge between mandatory and load-bearing: it's how a check proves it's ready to bite before you let it. You've described the ramp that "just make it load-bearing" hand-waves past — thanks for not letting me hand-wave it.
This lands. The override log as the artifact rather than the check itself is the key move — it turns "when do we tighten" from a judgment call into a readout.
Exactly — and the part that surprised me is the log reads in both directions. Each override I logged as "known exception, don't tighten yet" carries an expiry condition, so the day a rule improves enough that the exception is no longer needed, the test flags it: "this override is now dead — promote it back." The log doesn't just tell me when to tighten; it tells me when an exception has quietly outlived its reason. That second one is the failure mode a static allowlist never catches on its own — exceptions don't rot loudly.
One guard I had to add underneath it: count the overrides themselves. A log nobody tallies drifts back into "we've always had these," and a growing exception list is exactly the kind of green that's really just accumulated silence. So the number of overrides is its own assertion now. The readout needed a readout.
Exceptions don't rot loudly" is the line — that's exactly why they're more dangerous than failures, which at least announce themselves. And the meta-move of asserting on the count of overrides is the right instinct: the moment a log becomes something you can just accumulate into indefinitely, it's stopped being a control and started being a diary. Making the size of the exception list itself a thing that can fail is what keeps it from quietly becoming the new baseline.
The part I'd push on a little: what's the assertion actually checking — a fixed ceiling, or a rate (overrides added vs. retired per period)? A fixed ceiling eventually gets raised once, "just this once," the same way any override does — at which point the readout needs its own readout, and you're back at the top of the stack. A rate-based check at least forces the question "are we retiring these as fast as we're adding them" rather than "have we stayed under some number someone picked once.
Fair hit, and the honest answer is worse than the one you're pushing against: mine was a fixed count, and it wasn't even on the overrides — it was on the fixture total. The exceptions themselves (known blind spots, known false positives) were printed in the report and excluded from the exit code entirely. I'd built the readout and wired it to nothing that could fail. Diary, exactly as you put it.
On rate vs ceiling: rate is the better of the two, but I don't think it escapes the regress — someone still picks what retirement velocity counts as healthy, and that number gets renegotiated the same "just this once" way. It also degrades badly at small n; I carry two known exceptions total, and a rate over that is noise dressed as a metric.
So I went the other way and dropped the aggregate question entirely: expiry per individual override. Each one now carries the date it was accepted and the condition that kills it, and it goes red past 90 days — or immediately, if it has no accepted date at all. Then I put a match under it both directions: aged one out to 207 days, and stripped the date off another. Each went red for its own reason, and the repaired state went green.
And to pre-empt the obvious counter, since it's the same one I'd make: 90 is also a number someone picked. The difference isn't the number, it's what raising it costs. A ceiling gets raised once, silently, for the whole class. An expiry has to be renewed per item, by editing that item's accepted date — which is an explicit act that leaves a mark on the specific exception being extended. It doesn't make the exception impossible to keep. It makes keeping it impossible to do quietly, which is the part that was actually failing.