DEV Community

Geminate Solutions
Geminate Solutions

Posted on

AI Software Development: Merge Gates Every CTO Should Require

If your team writes much of its code with AI assistants, your rules for merging that code need to get stricter, not looser. In AI software development, the safe default is simple: no AI-generated change reaches your main branch unless a named person owns it, tests show what it does, and someone other than the assistant that suggested a new dependency has checked it.

You probably know this situation already. Velocity looks great. Pull requests land faster than ever, from your own engineers or from an outsourced team. But when something breaks, nobody can explain why the code was written that way, and "the AI wrote it" is starting to show up in incident notes.

Why does AI software development need different merge rules?

Code from an assistant is not worse by default. It fails in different places, and your current review habits were built to catch human mistakes.

An engineer who writes a function by hand usually understands it. An assistant produces code that looks finished, follows the house style and compiles, so reviewers skim it. Its typical defects are quiet ones: an error swallowed in a catch block, a permission check on the list endpoint but missing from the detail endpoint, a retry loop with no limit, or a package name that sounds right but is not the one you meant.

Volume makes it worse. When one engineer can open five pull requests a day, a reviewer facing a 900-line diff approves on trust. Tests generated in the same session as the code often check what the code does, not what it should do. They pass, and they show nothing.

What does it cost to get this wrong?

The cost rarely appears in the sprint where the code merged. It appears months later.

You end up with a codebase nobody fully understands, so every change takes longer than it should. You get security gaps a basic audit would have caught, found first by a customer or a penetration tester. You get dependencies nobody chose, some unmaintained, some under licences your enterprise buyers will ask about during due diligence.

If your team is outsourced, you can end up owning code that only the vendor's prompts can explain. You will feel that handover risk the day you switch partners or hire your first in-house lead.

The speed was real, and so is the bill for skipping the gates.

Who can skip most of this?

If you are a solo technical founder building a prototype to test demand, with no customer data and no payments, heavy gates will slow you down for little gain. Keep a short list: no secrets in the repo, a committed lockfile, and reading every diff before you merge it. Add the full set of gates the day real users or real data arrive.

Which pull request rules should you require?

Start with the pull request itself, because that is where ownership gets decided.

  • Small diffs. Set a soft limit of around 400 changed lines, not counting generated files and lockfiles. Anything larger gets split or gets a second reviewer. Past that size, reviewers stop reading properly.
  • AI use disclosed in the description. This is context, not blame. A checkbox in the PR template ("large parts generated by an assistant") tells the reviewer where to slow down.
  • The author explains the change in their own words. If they cannot write two sentences on why they chose this approach and what could break, the PR is not ready.
  • Required reviews through branch protection. On GitHub or GitLab, require at least one approval from someone other than the author and block direct pushes to main. Use CODEOWNERS so changes to auth, billing and data access always reach the person who owns that area.

Decision rule: if a change touches authentication, payments, permissions or personal data, require two human reviewers no matter how small it is.

What test coverage should you expect on AI-written code?

A coverage percentage is easy to game, especially when the assistant also writes the tests. Ask for proof of behaviour instead.

  • New logic ships with tests that would fail without it. Reviewers should ask: if I deleted the key line, would a test go red?
  • Coverage must not drop on the changed files. Tools like Codecov or SonarQube can enforce diff coverage in CI. That is fairer than a global target on an old codebase.
  • Edge cases are named. Empty input, a timeout, a duplicate request, a user from another tenant. Ask the author to list which of these the tests cover.
  • Critical paths get integration tests, not only mocked unit tests. An assistant will happily mock away the exact failure you care about.

If your team has no tests today, do not demand a coverage number overnight. Require tests on every new change and every bug fix, and the number will rise over time.

How should you check dependencies an assistant adds?

Assistants suggest packages from memory. Sometimes the package is outdated, sometimes it is abandoned, and sometimes it does not exist at all. A package that does not exist is the riskiest case, because anyone can register that name later and fill it with malicious code.

Put these checks in CI so they do not depend on how closely a reviewer is paying attention:

  1. Lockfiles are committed and CI installs from them (npm ci, pip install with hashes, or the equivalent for your stack).
  2. Vulnerability scanning runs on every PR with Dependabot, Snyk, npm audit or pip-audit. Decide which severity level blocks a merge.
  3. Every new direct dependency gets a one-line justification in the PR: why this package, who maintains it, and when it was last released.
  4. Licence checks flag copyleft or unknown licences before that code lands in a product you sell.
  5. Secret scanning with Gitleaks or GitHub push protection, because assistants paste example keys into config files more often than you would expect.

Who owns AI-generated code after it merges?

The person who merged it, every time.

Write that into your engineering handbook and your vendor contract. "The assistant wrote it" is never an acceptable root cause in a postmortem. If you work with an outside team, the contract should also state that you own 100% of the code and IP, that the repository lives in your organisation's account, and that the vendor will walk your engineers through any module when asked.

One more ownership rule: nobody merges code they could not debug at 2 a.m. If an engineer cannot explain a generated module, it gets simplified or gets a walkthrough before it merges.

What order should you roll these out in?

Do not switch everything on in one week, or your team will find ways around it.

  1. Turn on branch protection, required reviews and secret scanning. This takes an afternoon.
  2. Add dependency and vulnerability scanning to CI. Set it to warn for two weeks, then to block.
  3. Add the PR template with AI disclosure, and adopt the ownership rule.
  4. Enforce diff coverage on changed files, then add integration tests for your three most important user journeys.

If your team is outsourced, ask the vendor to show you these gates running in their pipeline before the next release. A slide describing them is not enough.

If you already suspect your codebase has drifted, run an audit before you add gates. This guide to an AI code audit covers what to look for and in what order.

If you want a second opinion first, Geminate Solutions will review your repository and merge process for free, under NDA, and reply within 24 hours.

Top comments (0)