Sanjay Kumar Sah

Posted on Oct 1 Originally published at github.com

Building Juryza: Hackathon Judging You Can Defend

#hackathonraptors #opensource #hackathon #webdev

If you have ever participated in or organized a hackathon, you already know the open secret: most hackathon judging is an accidental lottery.

A team spends 48 sleepless hours building a brilliant distributed system, only to be evaluated by Judge Alice, who strictly scores every project between a 2.0 and a 3.0. Meanwhile, another team building a simple mock UI happens to draw Judge Bob, who hands out 5.0s like candy on Halloween.

When the raw scores are averaged in an Excel spreadsheet, the second team takes home the grand prize, while the first team walks away wondering what they did wrong.

Worse, most hackathon platforms require bloated cloud subscriptions, break the moment event Wi-Fi stutters, and hide their scoring algorithms behind a proprietary black box.

We built Juryza to solve this once and for all: The self-hosted hackathon platform with judging you can mathematically defend.

Quick Roles & Features Tour

Here is a quick look at the live platform across all four key personas: organizer dashboard, blind judge scoring, the Bradley–Terry pairwise duel arena, and the Notion-style participant project editor:

juryza.vercel.app

⚡ Quick Links & Live Demonstration

🐙 GitHub Repository: https://github.com/sanjaysah101/Juryza
🎥 YouTube Demo: https://youtu.be/gfsorn4l_WI
💻 Live Local Stack (100% Offline Capable):

git clone https://github.com/sanjaysah101/Juryza.git
cd Juryza
docker compose up

Open http://localhost:8080 — no cloud account, no credit card, and zero external network calls required at runtime. The boot sequence spins up PostgreSQL 18 and Next.js 16, runs tracked Drizzle migrations, and seeds:

The DOGFOOD Fixture Event: 41 projects, 30 judges, 126 scores, duplicate submissions, and edge-case calibration data.
A Live Sandbox Event: Submissions open so you can register, create a team, and write projects immediately.
Hackathon Raptors Real Showcase (Opt-in): 31 real-world hackathons with authentic placements (SEED_SHOWCASE=1 docker compose up).

Demo Accounts Ready to Test

Role	Email	Password	What You Can Test
Organizer	`organizer@juryza.test`	`organizer-password-123`	Rubric weighting, judge assignment planner, conflict-of-interest detection, integrity dashboard
Judge A	`judge.a@juryza.test`	`judge-a-password-123`	Blind rubric scoring console & keyboard-navigated pairwise duel arena
Judge B	`judge.b@juryza.test`	`judge-b-password-123`	Proves absolute blind isolation (Judge B gets 403 on Judge A’s scores)
Participant	`participant@juryza.test`	`participant-password-123`	Notion-style block editor, slash menu, autosave, team invites
Admin	`admin@juryza.test`	`admin-password-123`	User roles, audit logs, system-wide controls

🏗️ The Architecture: Single Container, Zero Cloud Lock-In

We designed Juryza around a single foundational constraint: A hackathon platform must survive in a basement with zero internet connectivity.

┌─────────────────────────────────────────────────────────────────┐
│ Browser Client (Next.js 16 + React 19 + React Compiler)         │
│  - Notion-style Tiptap 3 Block Editor                           │
│  - Pairwise Duel Arena & Command Palette (⌘K)                   │
└───────────────────────────────┬─────────────────────────────────┘
                                │ Session Cookie / Bearer Token
                                ▼
┌─────────────────────────────────────────────────────────────────┐
│ Next.js 16 App Router (apps/juryza)                             │
│  - Better Auth (Multi-tenant sessions, rate-limited, RBAC)      │
│  - REST API & OpenAPI 3.1 Engine (/api/openapi.json)            │
│  - Trust Boundary: handle() turns exceptions into JSON          │
└───────────────────────────────┬─────────────────────────────────┘
                                │
        ┌───────────────────────┴────────────────────────┐
        ▼                                                ▼
┌──────────────────────────────┐        ┌──────────────────────────────┐
│ Pure Domain Engines          │        │ Data Layer                   │
│  - z-score normalizer        │        │  - Drizzle ORM               │
│  - Bradley–Terry MM solver   │        │  - PostgreSQL 18             │
│  - Quadratic voting budget   │        │  - Advisory Locks & Txns     │
│  - TF-IDF similarity matcher │        │  - Ed25519 Signed Certs      │
└──────────────────────────────┘        └──────────────────────────────┘

The Tech Stack

Framework: Next.js 16 (App Router, React 19, React Compiler)
Database & ORM: PostgreSQL 18 with Drizzle ORM in an isolated monorepo workspace (packages/database)
Authentication: Better Auth with local email/password sessions, admin controls, and per-event permission hierarchies
Rich Text Engine: Tiptap 3 with custom slash commands, floating toolbar, markdown shortcuts, task items, and autosave
Design & UI: Tailwind CSS v4 + shadcn/ui on Base UI primitives
Monorepo & Tooling: Bun + Turborepo + Biome

🧮 The Mathematics: Judging You Can Defend

Most hackathons fail because arithmetic averages confuse judge calibration with project quality. Juryza introduces three distinct, mathematically rigorous evaluation layers:

1. Cross-Judge z-Score Normalization

Judges systematically differ in two ways:

Level Bias ( $\mu$ ): Some judges are harsh (averaging 2.5); others are generous (averaging 4.2).
Spread Bias ( $\sigma$ ): Some judges use the entire scale (1–5); others cluster tightly between 3 and 4.

Juryza standardizes every score against each judge’s individual distribution:

z = \frac{\text{raw} - \mu_j}{\sigma_j}

\text{normalized} = \text{clamp}\left(\text{globalMean} + z \times 0.9, 1, 5\right)

Flat Judge Handling: If a judge gives every single project the exact same score, their variance is zero ( $\sigma_j \le 10^{-9}$ ). They provide zero ranking information. Juryza automatically sets their $z = 0$ , mapping them to the global mean so they neither artificially inflate nor penalize the teams they saw.
The Epsilon Division Bug: During fixture verification, we uncovered a critical floating-point trap: when a judge assigns identical scores, floating-point arithmetic produces a standard deviation around 1e-16 rather than 0.0. A naive if (sd > 0) check caused division of noise by noise, unfairly tanking projects in the leaderboard! We hardened this with strict epsilon tolerances and regression test suites.

2. Bradley–Terry Pairwise Duel Arena

When judges only review 2 or 3 projects, variance estimates ( $\sigma$ ) can be noisy. For high-stakes rounds, Juryza features a Pairwise Duel Arena.

Instead of awarding arbitrary numbers, judges are shown two projects side-by-side and pick the stronger one.

Under the Bradley–Terry model, the probability that project $i$ beats project $j$ is:

P(i \text{ beats } j) = \frac{s_i}{s_i + s_j}

We fit project latent strengths $s_i$ using Hunter’s (2004) Minorization-Maximization (MM) algorithm:

s_i^{(t+1)} = \frac{W_i}{\sum_{j \neq i} \frac{n_{ij}}{s_i^{(t)} + s_j^{(t)}}}

Active Pair Selection: Random pairings waste judge attention (matching a clear #1 project against a broken prototype yields zero information). Juryza dynamically selects pairs that:
1. Have the fewest prior head-to-head comparisons by the current judge.
2. Have the closest current estimated latent strengths ( $s_i \approx s_j$ ), maximizing information gain per click!

3. Quadratic Community Voting with Credit Budgets

Allowing public voting often leads to ballot-stuffing and social media brigade spam.

Juryza implements Quadratic Voting:

Every participant gets a fixed credit budget (e.g., 16 credits).
Allocating $v$ votes to a single project costs $v^2$ credits (1 vote = 1 credit, 2 votes = 4 credits, 3 votes = 9 credits, 4 votes = 16 credits).
Voters can support multiple projects or strongly back their favorite, but monopolizing votes becomes quadratically prohibitive.
Enforced inside PostgreSQL transactions with advisory locks to prevent concurrent race conditions.

✍️ The Participant Experience: Notion-Style Project Write-ups

Hackathon forms are usually ugly <textarea> inputs that ruin carefully formatted presentations.

In Juryza, participants write project pages using a modern Notion-style block editor built on Tiptap 3:

Slash commands (/h1, /code, /image, /todo, /callout)
Floating bubble toolbar for inline formatting
Real-time background debounced autosaving
Notion-like sidebar properties (tracks, demo URLs, GitHub repositories, license tags)
Hard deadline enforcement: At the closing second, the backend seals all mutations with atomic PostgreSQL checks. No late edits, no sneaky post-deadline commits.

🔍 Plagiarism, Rival Comparisons & Signed Certificates

Juryza goes beyond basic submission portals:

Rival Comparison Matrix: Pick any project, and Juryza uses TF-IDF cosine distance and leaderboard neighbors to find its closest competitors, comparing them side-by-side across every rubric criterion.
Duplicate Detection: Flags suspicious duplicate submissions or repo recycling across events.
Cryptographically Verifiable Certificates: Winners and judges receive verifiable certificates signed with Ed25519 keys, verifiable at /certificates/<serial>.
OpenAPI 3.1 & Developer API: Every single action in the UI is backed by clean, fully typed REST endpoints documented at /docs/api with personal API bearer tokens.

💡 What We Learned Building Juryza

Building for 100% offline forces clean architecture. Without relying on cloud services (AWS S3, Clerk, Supabase, Vercel Blob), you realize how much faster and more reliable local software can be. A full PostgreSQL instance with Drizzle and Next.js spins up in seconds and runs anywhere.
Fairness is an engineering discipline. Hackathon organizers don't need another subjective spreadsheet; they need statistical rigor with an audit trail that explains why team A placed over team B.
The power of unified REST APIs. Because the Juryza frontend consumes the exact same REST API that external CLI checkers and webhooks use, testing the platform was extraordinarily clean—88 comprehensive unit and live integration tests run with zero flaky mocks.

🚀 Get Involved!

Juryza is open-source under the MIT License. We want hackathons worldwide—from university clubs to massive global events—to run on software that is transparent, fair, and self-reliant.

⭐ Star the repo on GitHub: https://github.com/sanjaysah101/Juryza
📺 Watch & share the demo video: https://youtu.be/gfsorn4l_WI

Let us know in the comments: How does your local hackathon handle judging, and what algorithm would you like to see next?

DEV Community