A collaboration between Four Eyed Gens and beTheNOOB.
🔗 Repo: github.com/DhruvP2205/raptor
🦅 Building Raptor: A Hackathon Judging Platform That Checks Its Own Homework
Most hackathon platforms trust the submission form and move on. We built Raptor to question that. If a platform decides who wins money and recognition, every step of that decision should be something you can actually check, not just something you have to trust.
Here is what we actually built, and how it works under the hood.
🗺️ The Full Journey, Start to Finish
Before getting into each piece separately, here is the whole pipeline in one picture. Every stage below is a real, checked state on the server, never just something implied by which screen someone happens to be looking at. The dashed amber path shows what happens when verification flags something instead of approving it automatically, and once an organizer approves it, it simply rejoins the normal pipeline at the same step it would have reached anyway. The two boxes at the end, voting and certificates, both run after results are published, not one after the other in a strict order.
Each of these stages gets its own section below, so this is just the map. Keep it in mind while reading the rest, since the order here is the actual order the platform enforces, not just the order this post happens to explain things in.
🔍 Catching Problems Before a Judge Ever Sees Them
Every submitted project gets checked against its own GitHub commit history before a judge ever opens it.
| Check result | What happens next |
|---|---|
| ✅ Commits fall inside the event window | Auto approved, moves straight to judging |
| ⚠️ Some commits outside the window | Flagged for organizer review |
| 🚫 Repo is private or not on GitHub | Flagged for organizer review |
| ❌ Organizer disqualifies it | Must attach a written reason, permanently |
Nothing gets removed from a competition without a real explanation sitting right next to it, forever.
🧮 How a Score Actually Gets Built
A judge does not just type one number. Here is the actual chain:
| Step | What it is | Example |
|---|---|---|
| 1️⃣ Criteria scores | Each rubric criteria, weighted | Functionality (60%) × 90 + Quality (40%) × 80 |
| 2️⃣ Judge raw total | Weighted sum, plus bonus added on top | 86 + 5 bonus = 91 |
| 3️⃣ Average raw total | Average across every judge who actually finished | Judges who never submitted are excluded, not counted as zero |
| 4️⃣ Final score | Scaled to the event's display scale | e.g. out of 5 ⭐ |
🎯 The key design choice: bonus points get added after the weighted average, in full units, not blended proportionally in. That keeps the math honest. A small bonus can nudge a project up. It cannot let a weak project beat a strong one just because the bonus was generous.
⚖️ Normalization: Why It Exists, and What the Data Actually Showed
Different judges score differently. Some are naturally generous, some are naturally strict, and that has nothing to do with the quality of the work in front of them.
So every judge has a personal scoring profile, built from their history across every event they have judged:
Judge's raw score → compare to their own average and spread → normalized score
A tough judge's 4/5 stops meaning something different from a generous judge's 4/5.
📊 What we actually tested: we ran a real simulation on real fixture data (30 judges, 40 projects) to see how well this correction recovers a true ranking.
| Judges seeing many projects each | Judges seeing only a few each |
|---|---|
| Normalization clearly helps | Effect shrinks, sample size matters more |
That is the kind of finding you only get by testing your own formula against real numbers instead of trusting it because it looks correct on paper.
🏆 One Leaderboard, Every Event
Wins do not live inside a single event. Raptor keeps a platform wide leaderboard tracking one person across everything they have ever competed in:
- 🥇 1st place finishes
- 🥈 2nd place finishes
- 🥉 3rd place finishes
- 🌟 Special award wins
- 🗳️ Audience choice wins
All of it feeds one running total, so someone who quietly placed well across five small events gets the same visibility as someone who won one big one.
🛠️ The Infrastructure Underneath
We wanted this to run anywhere with zero cloud dependency. Five containers, two networks:
host-exposed ports
│
┌───────────────┴───────────────┐
│ │
web (Next.js) api (NestJS)
│ │
└────────── app network ──────────┤
│
worker (BullMQ) ── on both networks
│
data network
│
┌────────────┴────────────┐
postgres redis
| Container | Role | Why it exists |
|---|---|---|
🌐 web
|
Next.js frontend | The only thing a browser talks to |
⚙️ api
|
NestJS backend | Every rule, every permission check, lives here |
🔄 worker
|
Background jobs (BullMQ) | Handles slow work: GitHub checks, leaderboard recomputes |
🗄️ postgres
|
Database | The single source of truth |
⚡ redis
|
Cache and rate limiting | Login limits, gallery limits, queue storage |
🔒 The network isolation is the important part. postgres and redis sit on a data-only network with zero exposed ports to the outside world, not even to your own machine. web cannot reach the database directly at all, even if it wanted to. Every single thing a browser does has to go through api, which is the only place permission checks happen. If someone compromised the frontend, they would hit a dead end trying to reach the database.
docker compose up and that is the whole stack. No cloud account, no external API calls, no signup anywhere.
🧑💻 Five Accounts, Five Jobs
Before going role by role in detail, here is the full picture in one image. Every row below is a completely separate account type, and none of them can act outside what their own row shows, a judge cannot publish results and an organizer cannot score a project.
A visitor never needs an account just to look around. A user is the one actually building something. A judge only ever sees what they are assigned. An organizer runs their own event end to end. An admin can step into any event platform wide, but every single time they do, it gets written to a permanent log.
For a judge
Invited → Accept / Decline → Assigned queue of projects → Scoring screen
Once accepted, a judge gets a queue of assigned projects. The scoring screen brings the rubric, bonus tracks, and written feedback together in one place, and a review can be revised as many times as needed right up until judging closes.
For an organizer
One dashboard, one event at a time:
- 📨 Invite judges
- 🔍 Review flagged submissions
- 🧑⚖️ Assign projects to judges
- 📈 Watch judging progress live
- ⚖️ Run normalization
- 📋 Draft and publish results
- 🎓 Issue certificates
For an admin
Same toolkit, platform wide instead of event wide:
- Step into any event to help
- Export data across every event at once
- See the full audit trail for everything that happened
🕵️ Every time an admin uses that wider access, it gets logged automatically. The power exists for real operational need, but it is never invisible.
Built with a self-hosted first mindset, because the tools deciding who wins should be something you can actually run and check yourself, not something you have to trust blindly. 🦅
🔗 Check out the full code: github.com/DhruvP2205/raptor
A collaboration between Four Eyed Gens and beTheNOOB.
Top comments (5)
The organizer-review branch is an important distinction from treating commit dates as a final verdict. How do you pin the evidence used for that decision? I'd keep the submitted SHA, the checked commit range, the event-window timezone, and the review outcome together, so a later force-push or branch update cannot silently change what a judge is evaluating. A fixture where the branch moves after verification would be useful alongside the scoring simulation.
You're right, this is a real gap. Clean submissions don't get a commit SHA stored, and there's no re-check if the branch moves after review. Already fixed locally, just not pushed yet given the time crunch, thanks for catching it. 🥳
Thanks for checking the actual path. A clean submission is the important regression case here: approve commit A, move the branch to B, then confirm that judging either still resolves A or explicitly invalidates the approval. That would test the stored SHA and the behavior around it, rather than only asserting that the field is populated.
Already Fixed and tested locally, judges now resolve the pinned commit so a later branch move can't change what gets evaluated. GitLab and Bitbucket use an organizer-confirmed binding instead. Not pushed yet, submission window's closed and this is now judging period, goes up once that wraps. Thanks for follow up. 🤩
🔗 Slide deck: dhruvp2205.github.io/Raptor-Demo/
💻 Source code: github.com/DhruvP2205/raptor
📝 LinkedIn post: linkedin.com/posts/softwaredevelop...
📰 LinkedIn article: linkedin.com/pulse/building-raptor...
🤝 Built with beTheNOOB: linkedin.com/in/dhruv-prajapati-cy...