Karlis Gutans

Posted on Oct 20 AI-assisted

I automated my refactoring method with GitLab Duo agents. The agent writes the fix, a CI gate decides the merge.

#gitlab #ai #python #refactoring

AI writes code fast. The code runs and passes the tests. It is still hard to change.

I refactor a lot of AI-generated code. The same problems come back every time:

Functions grow long and deeply nested.
Files collect responsibilities that belong somewhere else.
The next AI session needs more and more context to make a safe edit.

For the GitLab Transcend Hackathon (theme: Life After Code) I tested how far my own refactoring process can be automated with GitLab pipelines and the GitLab Duo Agent Platform. The result is Liepa refactoring.

My manual method

My method has three steps:

Take the most characteristic keywords of each file.
Check whether those keywords belong to the area of the app where the file lives.
If they do not, the file mixes responsibilities. Refactor it with an AI assistant, one file at a time.

It works. It is also slow, and it depends on me remembering to do it.

What I built

Liepa refactoring has three parts:

liepa scan: a Python scanner. It runs standard checks (function length, nesting, cyclomatic complexity, duplicated function bodies) and gives every file an AI-readiness score from 0 to 100.
A refactor flow: a GitLab Duo agent flow. It scans the project, picks the lowest-scoring file and acts on that one file.
A control dial: a config file that sets how much control is handed to the agent.

The scanner uses the Python standard library only: ast, keyword, collections. No runtime dependencies.

Turning the keyword method into code

The AI-readiness score has seven checks. Six are familiar: context size, function length, nesting, complexity, docstrings and import fan-out. The seventh, keyword_areas, is my manual method.

How it works:

Collect the names in a file: functions, classes, arguments, attributes. Split snake_case and CamelCase into words. Drop Python keywords, builtins and generic words like data or value.
Keep the 7 most frequent words.
The area of a file is its directory.
For every word, find its home area: the directory where the word is relatively most frequent across the whole project.
The penalty is 10 × (keywords whose home is elsewhere) / 7.

This is a real result from the tool's own CLI file:

src/liepa/cli.py  91
`main` has complexity 12 (limit 10); top keywords span 2 areas:
path (src/liepa), config (src/liepa), score (src/liepa), parser (src/liepa),
add (src/liepa/checks), argument (src/liepa), files (src/liepa);
consider splitting the file

One keyword out of seven lives in another area. The penalty is small: 1.4 points. That is correct. It is a hint to look, not a verdict.

The control dial

.refactor-agent.yml lives in the repository:

mode: suggest # suggest | propose | auto

paths:
  include: ["src", "examples/messy/liepa_imin"]

thresholds:
  min_score: 60
  max_function_lines: 50
  max_nesting: 4
  max_complexity: 10

auto:
  require_score_improvement: true
  require_passing_tests: true
  max_changed_lines: 200

The three modes:

suggest: the agent posts one refactoring suggestion on the merge request. No files change.
propose: the agent refactors the file, runs pytest and ruff, re-scans, and opens a merge request for a human to review.
auto: the agent does the same as in propose. Then a CI job decides whether the merge request merges.

The agent never decides the merge

This is the most important design decision. In auto mode the LLM only opens the merge request. A small deterministic job, refactor_auto_merge, checks five conditions:

Only refactorable Python files changed. No tests, CI files, config or flows.
Every changed file scores strictly higher than before.
The change is under 200 added plus deleted lines.
The pipeline is green. GitLab merges only when it succeeds.
The merge request head is the exact commit that was scored.

The gate posts its decision on the merge request. This is the real note from the demo run:

Condition	Result	Details
Only refactorable files changed	pass	1 changed files, all scanned Python files
AI-readiness score improves	pass	`reports.py`: 40 → 51
Change size	pass	124 changed lines, limit 200
Pipeline green	pass	enforced by GitLab
Head SHA unchanged	pass	checked `0dc870fc4e71`

The gate and its limits are read from the target branch, not from the merge request. A refactor branch cannot loosen the rules that judge it.

This was not my first version. Duo Code Review found that the first auto mode trusted the merge request's own config and code. A refactor could have edited the limits that judged it. The fix: run the gate from a git worktree of the target branch.

The demo

I needed a codebase bad enough to give the agent real work. So a Duo Developer agent built liepa-imin: a toy insurance back office with parties, policies, claims, renewals and reports. Its reports.py scored 29: a 92-line function, complexity 20, 6 levels of nesting and 17 imports. Behaviour tests and golden files guard the output.

All three modes ran on the same file:

Mode	What happened	Score
suggest	one comment with three refactoring steps	28 → 28
propose	refactor MR, reviewed and merged by me	28 → 40
auto	refactor MR, merged by the gate with no human review	40 → 51

The auto merge request merged about one minute after it was opened.

What did not work

Triggers need a human. A GitLab flow trigger fires only on a person's action. A CI job, a bot or another flow cannot start it. GitLab staff confirmed this in the hackathon Discord. Every demo run was started by hand from the Duo CLI.
The first auto pipeline was rejected. The gate job needed jobs that existed only in branch pipelines. I moved lint, tests and scan into merge request pipelines.
The sandbox has no PyPI access. The GitLab-hosted sandbox cannot install packages, so propose runs locally for now.
The score can be gamed. Docstrings and split functions raise the score without always improving the design. The size limit and the one-file rule reduce this. They do not remove it.

Why this works

The agent writes, the code decides. An LLM is good at producing a refactor. A small deterministic gate is better at deciding whether it may merge.
Small steps. One file per run keeps every change reviewable, even in auto mode.
Evidence over claims. Every agent session, job and merge request is linked in the README. It is clear what ran and what is still untested.

What is next

Publish the flow and add a trigger, so assigning a reviewer starts it.
Run the dry run and a refused case on a real merge request.
Give the bot token its own least-privilege identity.
Keep running the flow until reports.py reaches 60.

How do you refactor AI-generated code? How much control would you hand to an agent that does it for you?

DEV Community