A 1,200-line merge request hit my queue at 4:50 p.m.
Eleven files changed. Three were generated lockfiles. Five were config updates. Two were actual source files. One of those source files removed a retry block.
I had no time to read every line. So I built a GitLab CI job that splits the diff into risk buckets before a human looks at it.
The goal is not to replace review. It is to make the first five minutes of review smarter.
Why whole-diff summaries fail
Most AI review tools send the whole branch to a model.
That creates three problems:
- Long diffs get truncated, so the risky line is often the one the model never sees.
- Generated files and lockfiles drown the real changes.
- You might accidentally send tokens, keys, or customer data sitting in the diff.
I wanted the opposite: a small prompt, only for large MRs, with redaction built in.
Where MonkeyCode fits
For the online path I use MonkeyCode's free model access. That keeps the experiment at zero cost while I tune the prompt.
The free server option matters too. I do not need to provision a runner, keep a GPU alive, or babysit a local endpoint. The CI job can make a short hosted call and stop.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
The rest of the workflow is plain GitLab CI and Python.
The triage contract
The job only runs on merge requests. It never posts comments or merges anything.
It writes one Markdown artifact called diff-triage.md.
The file has three sections:
- Skip check - if the diff is under 200 added or deleted lines, no model call happens.
- Offline classification - path-based heuristics produce a rough bucket instantly.
- Online classification - selected source hunks go to the free endpoint for semantic risk labels.
Every file gets exactly one label:
-
mechanical- generated, lockfile, or formatting-only. -
test-only- test changes with no source change. -
infrastructure- CI, Docker, config. -
behavior-change- source code that may affect runtime. -
needs-human-eye- the model is unsure or the file matches a danger pattern.
If the online call fails or times out, the script falls back to offline mode. That way the job still completes.
The CI job
Here is the .gitlab-ci.yml:
stages:
- triage
diff-triage:
stage: triage
image: python:3.12-slim
rules:
- if: $CI_PIPELINE_SOURCE == 'merge_request_event'
variables:
TARGET: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME
before_script:
- git fetch origin $TARGET --depth=1
- pip install --quiet requests
script:
- python diff_triage.py --target origin/$TARGET --mode ${TRIAGE_MODE:-offline}
artifacts:
paths:
- diff-triage.md
when: always
Set TRIAGE_MODE=online in the project CI/CD variables once you are ready for the model path.
The Python script
The script is intentionally small. The offline classifier is deterministic and runnable. The online transport is a stub you replace with MonkeyCode's documented request shape, because that shape can change.
import os
import re
import subprocess
import sys
from pathlib import Path
DANGER_PATTERNS = [
(r'(?i)(password|api[_-]?key|secret|token|bearer)', 'sensitive'),
(r'(?i)(timeout|retry|rate[_-]?limit|circuit)', 'resilience'),
(r'(?i)(auth|session|permission|role)', 'access-control'),
]
def run_git(target):
diff = subprocess.run(
['git', 'diff', '--name-status', target + '...HEAD'],
check=True,
capture_output=True,
text=True,
).stdout
files = []
for line in diff.splitlines():
if not line.strip():
continue
parts = line.split('\t')
status = parts[0]
path = parts[1] if len(parts) > 1 else parts[0]
files.append((status, path))
return files
def total_changed(target):
stat = subprocess.run(
['git', 'diff', '--numstat', target + '...HEAD'],
check=True,
capture_output=True,
text=True,
).stdout
added = deleted = 0
for line in stat.splitlines():
if not line.strip():
continue
try:
a, d, _ = line.split('\t', 2)
added += int(a) if a.isdigit() else 0
deleted += int(d) if d.isdigit() else 0
except ValueError:
continue
return added, deleted
def offline_label(status, path):
p = path.lower()
lockfiles = ['package-lock.json', 'yarn.lock', 'pnpm-lock.yaml', 'poetry.lock']
if any(seg in p for seg in lockfiles):
return ('mechanical', 'lockfile')
if p.endswith(('.min.js', '.min.css', '.map')):
return ('mechanical', 'generated')
if any(seg in p for seg in ['.github/', 'dockerfile', '.gitlab-ci.yml', 'renovate']):
return ('infrastructure', 'build or CI config')
if p.startswith('test') or 'test' in p or p.endswith('.spec.js') or p.endswith('.test.py'):
return ('test-only', 'test change')
if p.endswith(('.yml', '.yaml', '.toml', '.json', '.ini', '.env')):
return ('infrastructure', 'config file')
return ('behavior-change', 'source file')
def redact(text):
return re.sub(
r'(?i)(password|api[_-]?key|secret|token|bearer)\s*[:=]\s*\S+',
r'\1=<redacted>',
text,
)
def online_label(path, text):
# Replace this stub with MonkeyCode's free endpoint request.
# Return JSON with keys: kind, risk, reason.
return {
'kind': offline_label('M', path)[0],
'risk': 'medium',
'reason': 'online transport not configured; offline fallback used'
}
def main():
target = sys.argv[sys.argv.index('--target') + 1]
mode = sys.argv[sys.argv.index('--mode') + 1]
files = run_git(target)
added, deleted = total_changed(target)
output = ['# Diff triage', '']
output.append(f'- files: {len(files)}')
output.append(f'- added: {added}')
output.append(f'- deleted: {deleted}')
output.append('')
if added + deleted < 200:
output.append('Small diff. Skipping model triage. Review normally.')
Path('diff-triage.md').write_text('\n'.join(output))
return
for status, path in files:
kind, reason = offline_label(status, path)
if any(re.search(pattern, path, re.IGNORECASE) for pattern, _ in DANGER_PATTERNS):
kind = 'needs-human-eye'
reason = 'danger pattern in path'
risk = 'low'
if kind == 'behavior-change':
risk = 'high'
if mode == 'online':
text = subprocess.run(
['git', 'diff', 'HEAD', '--', path],
check=True,
capture_output=True,
text=True,
).stdout
redacted = redact(text)
result = online_label(path, redacted[:4000])
kind = result.get('kind', kind)
risk = result.get('risk', risk)
reason = result.get('reason', reason)
output.append(f'## {path}')
output.append(f'- status: {status}')
output.append(f'- kind: {kind}')
output.append(f'- risk: {risk}')
output.append(f'- reason: {reason}')
output.append('')
Path('diff-triage.md').write_text('\n'.join(output))
if __name__ == '__main__':
main()
What breaks
The largest weakness is the online classifier. Free model endpoints can be rate-limited or unavailable. The job falls back, but the fallback is only as smart as path patterns.
A second weakness is truncation. I cap each file diff at 4,000 characters before sending it. The cap is worse than the classic 1,200-line MR if one file has a huge function. The cap is intentional for cost and privacy, not ideal for accuracy.
A third weakness is that a label is not a decision. risk: low can still hide a subtle bug. The artifact helps prioritize; it does not approve.
Who should not use this
Skip this workflow if:
- Your compliance rules forbid sending source snippets to a third-party model.
- Your repo contains regulated code, healthcare data, payment data, or customer secrets.
- You expect the model to replace human review.
- You need exact API shapes and do not want to maintain a small adapter layer.
For teams that just need to separate mechanical from behavior-change, even the offline path can save a real chunk of review time.
Try offline mode first. If you already have a MonkeyCode free account, switch one merge request to online and compare the labels against your own review notes. The diff artifact becomes a review map, not an oracle.
Top comments (0)