DEV Community

Cover image for How Many Lines of Code Do You Ship to Prod a Week? This Week I Shipped One.
vadim albarov
vadim albarov

Posted on

How Many Lines of Code Do You Ship to Prod a Week? This Week I Shipped One.

#aws #claude #ai #devops

Yes, one. But I am as exhausted as if I had shipped a full feature.

'BackupRetentionPeriod': 0,
Enter fullscreen mode Exit fullscreen mode

That is the whole fix. It took a week, more Claude Code sessions than I want to count, a /code-review pass before every pull request, one restart from scratch, and a diff that grew to a couple hundred lines before it shrank back to this. The sentence that made the one-liner possible was fetched in the very first turn. It lost - to a file on my laptop I had not updated in months, and to the model's memory, which was just as old.

The setup

Every weekday morning a Step Functions workflow restores an analytics replica from the main database's snapshot, runs some SQL, flips a DNS record, and deletes yesterday's copy. The replica lives one day.

Except a restored RDS instance inherits its backup retention from the snapshot. So every disposable replica came up with the source's week-long automated backups and started snapshotting itself within minutes. Wasted money, and another copy of PHI nobody asked for.

I know RDS. Backups are one setting. On a normal instance I would have flipped it without thinking. But this was a restore from a snapshot, and a snapshot carries the source's settings with it - so I had a real doubt: does that one setting still apply at restore time, or does the snapshot win and you have to modify afterwards? Guess: probably one parameter. Doubt: maybe not in this path.

And the workflow was agent-written. It worked, I had reviewed it, but I had not typed it and did not know every corner of it. Finding the right dict while also settling my doubt felt like a job for the model. So I asked for the change: reliable, no race conditions, check the edge cases, and - I actually wrote this - trivial and bulletproof.

I know vague prompts do not work well. But "trivial and bulletproof" did not feel vague to me. In my head it had a strict meaning: the smallest change that works, plus proof that nothing around it breaks. The model read it differently - as a license to extend the scope. At least that is what I think now; the model did not share its thoughts.

"Verified"

The planning turn was impressive. It read the module, queried both production accounts, confirmed the inherited retention, found the first automated backup landing about a minute after each restore. It fetched a stack of AWS docs. Then it wrote:

And you cannot fix it at restore time. I checked the RDS API model shipped with the local CLI:
RestoreDBInstanceFromDBSnapshot -> BackupRetentionPeriod: False

That did not fight my belief. It confirmed my doubt, with a checkmark. The thing I was unsure about had been settled by someone who looked.

The plan that followed was several times the change I had hoped for: modify the instance after restore, absorb the reboot that causes, add IAM, add a retry, widen the timeouts. I asked why. It answered every line - confidently, consistently, with a perfect track record from the values it had just pulled out of production. I had a guess, a doubt, and no working knowledge of the code. It had reasons. I was persuaded, which is not the same as convinced. I said "implement."

Here is what I missed. In the same response, to show that retention is inherited, it had quoted the RDS User Guide:

"When you don't explicitly set this value, the restored database inherits the backup retention period from the source snapshot or instance."

When you don't explicitly set this value. The doc was saying the value can be set. The model quoted it for the second half and moved on. Asked about it later, it said it had noticed the conflict and "trusted the stale artifact over the doc."

The loop

Every session that week ran the same cycle. /code-review finds real defects. The model fixes all of them, correctly. At least one fix lands outside the original ask - a cleanup path that was already weak, a timeout that "should" go up while we are here. The next review has new code to examine. Repeat.

The first review said a mutation did not belong inside a polling Lambda; the fix moved it into its own Step Functions state, with a wait, a three-way Choice, and IAM on a different role. The second review found several bugs in that new surface, and a pre-existing orphan-cleanup problem while it was in there. The third review found bugs in the cleanup rewrite. Each pass was right. Each fix was good. The ask had touched one Lambda; the diff now covered two Lambdas, the state machine, IAM, and the variables file, and had rewritten instance cleanup and database connection handling along the way. None of it was the task.

Midweek I threw it all away and started from scratch in a fresh session, hoping a clean context would find a better direction. It ran the same check against the same CLI and produced the same design with fresh confidence. Starting over resets the conversation. It does not reset the tools.

/code-review is good at its job, and that is the problem. It reviews the diff in front of it. It has no way to say the diff should not exist. So every pass certified the direction by omission and handed back a list of things to fix inside it.

The lottery

After the third pass I stopped asking for fixes and asked which findings were real, which were theoretical, and which were irrelevant - and said "none" was a fine answer. That got the most honest reply of the week: a handful real, a few theoretical, one right for the wrong reason, one pure scope creep that had spawned half the last round. "I'd stop here." It even noted the design still lost a race with RDS's first automated snapshot. I asked: commit or rewrite? Commit. I was about to.

Then I ran /code-review one last time, as I always did. This one went deep for reasons I still do not know: twenty parallel agents, the maximum, each reading the repo on its own. They used up my almost‑fresh five‑hour session token limit in just 20 minutes. The run died when the limit hit. Three agents came back. One of them said, high severity: the restore API already accepts BackupRetentionPeriod.

One of twenty. One of the three that finished. A minute later on the limit and I would have shipped a couple hundred lines and no budget left to look again.

The model re-fetched the API reference, found the announcement, confirmed my CLI predated it, and said plainly that its verification had been wrong. Everything came out except the one line.

One line.

Where "verified" came from

The CLI on my laptop was many months old. AWS added the parameter after that build, and the CLI never updates its own API model - it is a JSON file frozen into the installer. Every time the agent checked, in every session, it got the same honest wrong answer.

Half of that is mine. An agent trusts the output of commands on my machine the way I trust my eyes. Stale tools mean stale observations, and the agent cannot tell. Keeping the CLI current used to be housekeeping. With an agent doing the checking, it is ground truth - and when it rots, the agent does not fail. It succeeds at verifying the wrong world.

The other half: the parameter also postdates the model's training. So the model's memory said no, the local file said no, and I had walked in doubting. Three against one fresh sentence from a live doc. Nothing in the loop weighs a source by its date, so the newest one counted least.

What I took from it

  1. A "verified" that lands on your own doubt is the one to check. A claim that contradicts you gets scrutiny for free. A claim that confirms your uncertainty gets none - it feels like closure. It was the one question I had come with, and I let the model close it by assertion.

  2. "Why?" buys a justification, not a check. Every answer was about how to do the thing safely. None was about whether it was needed, because I never asked that. The premise question is: what fact, if wrong, makes this whole plan unnecessary - and where did we verify it?

  3. "Verified" has a date. So does the model. Ask when the tool was current. Then remember the tools on your machine are yours to keep current; an agent inherits every stale binary you leave it and turns it into confident output.

  4. Diff review certifies direction by omission. Use /code-review for what it is - a defect pass. Ask the premise question separately, in plain words, before the second review, not after the fourth.

  5. A fresh session is not a second opinion. Same model, same tools, same stale CLI, same answer. Independence needs a changed input: a different model, a different source, or a human who knows the system. I was that human, and I had already deferred.

  6. Parallel agents under a hard budget are a lottery. Twenty agents share one window. When it hits, the unfinished ones are gone - not returned, not cached. I bought twenty tickets and got to read three.

The uncomfortable version

I did not lose a week because the model was wrong. Models are wrong sometimes; that is priced in. I lost it because the wrong answer matched my own doubt, and a wrong answer that agrees with you does not feel wrong. The model was not lying. It was thorough about the wrong question, standing on a file I had let go stale, and I let its thoroughness stand in for the check I should have made. What finally broke the loop was one lucky agent and one human question - and I would rather not rely on the luck again.

The one line is committed. One last thing I did check this time, against the botocore changelog rather than a file on my laptop: the Lambda runtime's bundled SDK is newer than the release that added the parameter, so the call will not be rejected client-side. The fix that took a week to find took a few minutes to verify. That ratio is the whole article.

Sources

Top comments (0)