DEV Community

Cover image for I built an AI code review GitHub App that tracks your team's mistake patterns, here's how
Ahmad Mustafa
Ahmad Mustafa

Posted on

I built an AI code review GitHub App that tracks your team's mistake patterns, here's how

Most AI code review tools tell you what's wrong in a PR. CodePulse tells you what your team keeps getting wrong, week after week.

I built CodePulse as a GitHub App that does two things regular code review bots don't: it stores every finding per developer over time, and sends a personalized weekly digest summarizing each person's recurring issue categories. The idea is simple — if you're writing the same type of bug every week, you should know about it.

How it works

Install the GitHub App on your repos. Every time a PR is opened, CodePulse fetches the diff, runs a two-pass AI analysis using Groq's Llama 3.3 70B with structured tool-calling, and posts inline review comments pinned to exact lines with severity labels — Critical, High, Medium, Low.
Every finding gets stored against the developer who wrote it, the repo, and the PR. This is what makes CodePulse different from a one-shot reviewer — it builds a history.
Every Sunday, developers who opt in get a personalized email digest aggregating their issue categories from the past week. Not a wall of comments — a summary of patterns.

The Stack

Backend runs on Node.js, Express, TypeScript with Prisma ORM and Neon PostgreSQL on Azure. Frontend is React with TanStack Router on Vercel. Weekly digests go out via Resend, triggered by a GitHub Actions cron job.
The trickiest part was the two-pass analysis — a file triage pass first to skip lockfiles, minified assets, and generated files, then a chunked deep review pass that returns typed JSON per issue via structured tool-calling. Getting Groq to return consistent structured output at speed was the core engineering challenge.

What I learned
Webhook reliability is everything in a GitHub App. HMAC-SHA256 verification, idempotent processing, and logging every delivery from GitHub's Advanced tab saved me hours of debugging. If you're building on GitHub webhooks, treat the Recent Deliveries panel as your best friend.
Multi-tenant isolation also required more upfront schema design than I expected, scoping every query by GitHub App installation ID from day one is non-negotiable.

Try it

Live at getcodepulse.vercel.app — install the GitHub App on any repo and open a PR. Inline comments appear within 1–3 minutes.
Source on GitHub: https://github.com/ahmadmustafa02/CodePulse

I'm Ahmad Mustafa, a Full Stack Developer based in Islamabad, Pakistan. Building AI-powered products and publishing research in deep learning for IoT security and medical imaging.

LinkedIn: https://www.linkedin.com/in/ahmadmustafa01/
Website: https://ahmadmustafa.me/
Github:

Top comments (0)