I gave myself 48 hours to validate an AI code review tool. Here's how I built it.

Matthew Phelan — Mon, 09 Mar 2026 12:36:46 +0000

I wanted to test a simple idea: what if every pull request you opened got an instant AI code review?

Not a replacement for human reviewers — just a first pass that catches the obvious stuff before your teammates even look at it.

So I gave myself 48 hours to build it, ship it, and see if anyone actually finds it useful.

What it does

CodeReview.ai is a GitHub App. You install it on a repo, and every time a PR is opened, it:

That's it. No dashboard, no config files, no pricing tiers. Install and go.

The whole thing is two API routes: one for the GitHub webhook, one for waitlist signups.

I set four metrics to decide if this is worth continuing:

If I hit 3 of 4, I keep building. If not, I scrap it and move on.

Diffs are truncated to 8,000 characters. Large PRs won't get full coverage.
GPT-3.5-turbo is fast and cheap but not as sharp as GPT-4. Review depth has a ceiling.
50 reviews per day per installation. I'm a solo dev and my OpenAI bill needs to stay sane.

If you want to kick the tires:

If you install it and it's annoying rather than helpful, I genuinely want to hear that. The whole point of the sprint is to find out.