I'm building Criterion — a GitLab AI agent that reviews merge requests and surfaces forgotten design decisions before they cause production incidents.
This is my submission for the GitLab AI Hackathon. Deadline: March 25. The clock is very much ticking.
What Criterion Does
Every engineering team has this problem: a developer refactors some auth code, not knowing it was carefully designed 8 months ago to handle an enterprise SSO edge case. It merges. It breaks in production. A P0 incident follows.
Criterion sits inside your GitLab workflow and catches this before it happens. When you open a merge request, Criterion:
- Traces git blame on changed lines back to the original commits, MRs, and issues
- Asks Claude to extract not just WHAT was decided, but WHY — the business reason, the security constraint, the incident that caused it
- Posts a structured comment directly on your MR before you merge
- Uses confidence-based actions: block at 90%+, warn at 70%+, inform at 50%+
The goal is simple: prevent the $50K production incident that happens when developers break historical constraints they didn't know existed.
What I've Built So Far
Here's what's actually working:
Infrastructure
- FastAPI backend with async webhook receiver
- Celery + Redis for background job processing
- PostgreSQL with SQLAlchemy 2.x async ORM
- Alembic migrations
- Docker Compose local development
- Streamlit dashboard (skeleton)
- Health check endpoint that actually tests dependencies
GitLab Integration
- Webhook receiver with HMAC token verification
- Idempotency — duplicate webhooks are safely ignored
- MR upsert, blame fetching, commit tracing, issue fetching
Analysis Engine
- Git blame → commit → MR → issue tracing pipeline
- Claude integration with structured prompts and untrusted content isolation
- WHY/WHAT/CONTEXT/TRADEOFFS extraction
- Acceptance criteria parsing from issue descriptions
- Confidence scoring → action mapping
- Results persisted to PostgreSQL
The Honest Part
I'm a junior developer. I had mentorship throughout this process — structured, rigorous, the kind that doesn't hand you answers but asks uncomfortable questions until you find them yourself.
Even with that, it was hard.
Not "hard" in a dramatic way. Hard in the quiet way where you stare at a SQLAlchemy error for an hour, or realize your Celery task is silently swallowing exceptions because you're reusing a rolled-back session, or discover your loop is only returning results from the last iteration and you've been losing data the whole time.
The bugs that don't crash loudly are the ones that teach you the most.
Where I Am Today
Today I'm not writing new features. I made a deliberate call to stop and do three things:
- Test what I have — actually verify the pipeline works end to end
- Fix what's broken — find the quiet failures before they compound
- Document what works — so I can demo it clearly
There's still a lot to build: MR comment posting, the frontend dashboard, deployment, the demo video. And 19 days to do it.
Excited? Yes. Exhausted? Also yes. Stopping? No.
Tech Stack
Python FastAPI Celery Redis PostgreSQL SQLAlchemy 2.x Claude (Anthropic) python-gitlab Streamlit Docker Alembic
If you're building something for the GitLab AI Hackathon or just following along, I'd love to connect. Drop a comment — especially if you've built something similar and hit walls I haven't found yet.
Next update: comment posting, frontend, and whether I'm still sane.
Top comments (0)