Five AI Agents After the Code: What GitLab Duo Did in a Real Pipeline

Quick answer: I put five GitLab Duo agents (plan, review, fix, secure, release) around a small Python service and let them work after the code was written. They planned a feature in about two minutes, sent one fix back for review three times, diagnosed a pipeline race, and cut container scan findings from 166 to 1. They never merged anything: I did, and I made one mistake doing it.

Writing the code is no longer the slow part of building a small service. The slow part is what comes after: planning the change, reviewing it, fixing the pipeline when it breaks, triaging security findings, and releasing without breaking production. I entered the GitLab Transcend hackathon ("Life After Code") to find out how much of that work AI agents can carry, and where they have to stop.

This article is the honest account. It uses one real project, real merge requests and real verdicts, including two mistakes. Every number comes from the project's own pipeline and merge request history, which is public.

The project

The service is ro-cited-answers, an MIT-licensed demo that answers Romanian questions about public procedures with one rule: no answer without a source. Four gates refuse anything the documents cannot support. I built the first version before the hackathon. The entry is about what I added around it afterwards: a GitLab CI/CD pipeline and a set of GitLab Duo agents, all in the public repository:

https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/43427415/ro-cited-answers

The live service, version-stamped on its health endpoint:

https://ro-cited-210002791432.europe-north1.run.app/health

The nine stages, and who covers what

The contest defines nine stages after the code: plan, create, verify, package, secure, release, configure, monitor and govern. The pipeline covers them like this:

• Verify: offline tests of the grounding gates, a reproducible evaluation set, and a boot check.
• Package: the container image, pushed to the GitLab registry and to Google Artifact Registry.
• Secure: SAST, secret detection and container scanning.
• Release: a Cloud Run candidate revision that receives no traffic, a check against it, then promotion, a second check on production, and automatic rollback if that fails.
• Configure: production settings live in one file, deploy/service-config.json, and a drift job compares Cloud Run against it.
• Monitor: a schedule re-checks production every six hours, including a nonsense-path control, and each request sends one span to GitLab Observability that records the outcome and the gate, never the question text.
• Govern: AGENTS.md holds the rules every agent must follow.
• Plan and create: handled by the agents below.

There is no service account key anywhere. The organization policy blocks key creation, so GitLab signs a token for each job and Google Cloud trusts it through Workload Identity Federation.

The five agents

Each agent is a custom GitLab Duo flow with its own service account. The definitions live in the repository (.gitlab/duo/flows/) and a sync script applies them, so the repo is the source of truth. All of them are bound by AGENTS.md: they may open merge requests and write notes, and they may not merge or deploy. A sixth flow, for configuration drift, came later; the five below are the core.

1. Plan: rca-plan

It starts by itself when an issue is created. On issue #1, which asked for tracing, it posted a full plan about two minutes later. It had read AGENTS.md, noted that no gate rule changed, and limited the trace attributes to an allowlist that never includes the question text. That plan became the actual feature, merged later as !6.

One flaw: when I assigned issue #3 to the security agent, the plan agent ran on it too, because "issue created" is also an event. It was a double run, noted as a follow-up.

2. Review: rca-review

It starts when it is assigned as reviewer. Its single question is whether a change can make the system answer something it used to refuse.

The best example is MR !5. The review agent went through three rounds on it: two high findings, then three medium findings, then a ready verdict. I accepted two low findings on the merge request. Both earlier rounds were right. It also found a real bug in a later merge request, !7: it noticed the code assumed the first traffic entry was the serving revision, which is wrong. That was fixed before merge.

One limit I learned: Duo redacts strings shaped like Romanian personal ID numbers in the agent's context, so on !6 the reviewer saw "[REDACTED]" where my test used a sample value. It also could not read .gitlab/duo/ at first, because that directory is in Duo's context exclusion. The per-path review rules had to move into AGENTS.md.

3. Fix: rca-fix

It starts when a pipeline fails. I planted a realistic bug (a NameError) on a branch as !2. The boot check failed in pipeline 2954432632. The fix agent opened !3 with a one-character fix, ran the offline gate tests (6 of 6) and a local smoke check (4 of 4), and the merge request pipeline went green.

Later it diagnosed a failure I had not planted. Two merges landed 8 seconds apart and shared one Cloud Run tag, candidate. One pipeline validated the other commit's revision and failed 6 of 7 checks. Nothing wrong reached production. The fix agent opened issue #4 describing the race, offered three options, and wrote that it "needs a human decision". It did not change code. The fix I chose (!5) tags each candidate with its commit and promotes only if that commit is still the head of main.

4. Secure: rca-secure

I assigned it an issue to triage the scanner output. It produced a table of 173 findings: 166 container, 7 SAST, 0 secrets. It dismissed none. Instead it opened !4, switching the base image from python:3.12-slim to Alpine with a non-root user. After the merge, the main pipeline's container scan showed 166 findings go down to 1, one Medium in zlib. The merge request pipeline could not show this, because container scanning does not run on merge request pipelines, so the before and after numbers were only measurable after merge. A human merged it, because production safety (candidate check plus automatic rollback) was the real control.

5. Release: rca-release

It starts when a pipeline on main succeeds or fails. It writes a release summary as an issue. On its first run, it tried to check production itself and was blocked by the Duo sandbox proxy. It reported "not verified" instead of guessing. We then allowed exactly the two Cloud Run hostnames in .gitlab/duo/agent-config.yml, and later runs checked the live service themselves.

That "not verified" line is my favorite result of the project, because the whole service is built on the idea of refusing to claim what you cannot back.

What went wrong

The race. The shared candidate tag above was my design error, and the pipeline's own version check caught it. The agents helped diagnose it. They did not prevent it.

Merging before reading the verdict. I wrote a command that read the review verdict and merged in one step. For !13, a deduplication fix for the fix agent, the verdict was NOT READY and the high finding was correct: a push of the same commit succeeds twice. The suggested remedy also failed my own test. !14 replaced it with a unique claim commit and a push that only succeeds when the branch is absent, and it was merged after a READY verdict. The rule I kept: read the verdict in one step, merge in a separate step.

Duplicate work. The fix agent opened duplicate merge requests (!8 and !9) for one YAML failure, and the plan agent double-ran on #3. Both became follow-ups.

One thing I have not verified live. !14 relies on the git tool inside the agent sandbox accepting a lease flag on push. I tested the logic locally. I will know if the sandbox accepts it on the next real failure.

What a human still decides

Every merge. The agents have no merge permission by design, and in this project I said "merge" explicitly each time, after reading the verdict.

Also: which options to take when an agent writes "needs a human decision", whether a finding is acceptable, anything that touches the contest's rules, and the video and the submission themselves.

What I would not claim

The pipeline reaches the Hands-off path for the cases above, but I only tested those cases. The agents ran a handful of times, not thousands. The service itself is a demonstrator on a small document set, and its model calls go to a hosted open-weight model, not to anything I run locally. Duo credit cost per run is unknown to me.

How to reproduce

1. Fork the public repository on GitLab. It is MIT licensed, and the README section "The post-code pipeline" describes the stages.
2. Set the CI variables the pipeline expects (the model API key as masked and protected, and the production URL), and point the Google Cloud Workload Identity Federation condition at your namespace.
3. Apply the flows with deploy/duo_sync.py using a GitLab token. Unchanged flows are skipped; a changed flow is released as a new frozen version.
4. Open an issue and watch the plan agent respond. Open a merge request and assign the review agent. Break a test on purpose and watch the fix agent.

The rule to take with you: let agents advise and open merge requests, keep the merge as a separate human step, and make production prove itself before it takes traffic.

Which step of your own pipeline would you never hand to an agent, and what is the evidence behind that line?

---

Originally published on devaland.com: Five AI Agents After the Code: What GitLab Duo Did in a Real Pipeline