Most hackathon portals answer one question — who won? — with a row in a database. The row renders, everyone moves on, and nobody asks the harder question that no row can answer on its own: how would you know if it were wrong?
Dogfood Portal is a self-hostable platform for running a hackathon's submissions and judging: teams submit, judges score, and it publishes a final leaderboard. I gave myself two rules for the 72 hours. It has to come up from a single docker compose up with the network unplugged — no Redis, no message broker, no background worker, no CDN, two services in the compose file and one of them is Postgres. And the results it publishes have to be verifiable by anyone, offline, without trusting my server. The second rule is the whole post.
Here is the thing that rule forces you to admit:
A hackathon portal's "final results" are a
SELECTover a table. Anyone who can reach the database canUPDATEa score after the fact, and the results page will render the new number exactly as confidently as the old one. Nothing on the page knows it changed.
That is not a Django problem and it is not a Postgres problem. A database stores current state; it was never a record of what happened. Treating "it's in the database" as "it's the result" is a quiet category error — reaching for a tool whose contract is close to the one you want, reading storage as if it were proof, and shipping the gap without noticing it is there. So the results here carry their own proof, and there is one command that re-checks all of it with the database container stopped.
What I built
Dogfood Portal is a single Django 5.2 + PostgreSQL 16 monolith — one web service, one db service, nothing else. It boots offline, runs the migrations, seeds a demo event (an organizer, judges, teams, and scored submissions) on first start, and serves a public project gallery, a judging flow, and a normalized leaderboard. The interesting half is underneath: the final results are exported as a signed, reproducible bundle, and there is an offline verifier that re-derives and re-checks the whole thing from that bundle alone.
$ docker compose exec -T web python src/manage.py release_bundle /state/release
$ docker compose exec -T -w /app/src web python -m normalize.release /state/release
[PASS] ranking.csv matches the signed run
[PASS] signed run reproduces from the pinned ballots
[PASS] normalization.published event is on the audit chain
[PASS] audit checkpoint signature verifies
[PASS] every artifact traces to one operator public key
RESULT: VERIFIED
That second command needs no database and no network. It is the point of the project, so the rest of this post is what "verifiable" had to actually mean, and what generalizes past one Django app.
What a database won't do for you
The naive design is one table: results, with a rank and a score per project, written when the organizer clicks publish. It renders fine. It is also unfalsifiable — there is nothing to check it against, because the row is simultaneously the claim and the only evidence for the claim.
So publishing does not write a row you have to trust. It writes a chain you can follow:
- the normalized ranking is computed and frozen as a signed run — the pinned input ballots, the model settings, the resulting scores, and an Ed25519 signature over all of it;
- that publication is recorded as a
normalization.publishedevent on an append-only audit hash-chain, where each event carries the hash of the one before it; - the chain head is sealed with a signed checkpoint;
- the exported
ranking.csvis bound back to the signed run; - and every signature in the bundle traces to one operator public key.
The verifier walks that chain backwards. It re-runs the estimator from the pinned ballots and checks the result it gets matches the signed one; it re-checks every signature; it confirms the published event actually sits on the chain and that the checkpoint over the chain verifies; it confirms one key signed all of it. Any broken link fails the run and names itself — signed run does NOT reproduce, not a generic false. With the db container stopped, from the bundle alone, on a machine that has never seen my server.
Everything that verifier needs travels in one exported bundle: the ranking CSV, the signed run with its pinned inputs and result, the audit checkpoint and the slice of chain leading up to it, and the single public key the whole thing is signed under. The export self-verifies before it will finish writing, so a bundle that does not check out is never produced in the first place. For the everyday case — "prove to me this leaderboard is the one you actually published" — there is a smaller dogfood.results-certificate.v1 you can hand to a single team, and it verifies through the same path. The bundle is the audit; the certificate is the receipt; both are checked by code, not taken on faith.
The transferable version is small: sign the result; don't ask people to trust the server. If the only evidence for a number is the system that produced it, you have not published a result, you have published an assertion.
The measurement: a leaderboard implies a precision the ballots don't have
Raw judge scores are not comparable. One judge's 7 is another's 9, and if two projects were seen by different judges, their averages are measuring the judges as much as the work. So the leaderboard is not a mean — it is a ridge / partial-pooling estimate that models each ballot as quality + judge_bias + noise, fits it with a gauge that pins each connected group of judges-and-projects to a common zero, and picks the regularization strength by cross-validation rather than by taste.
That gauge is the subtle part, and it is where most naive score-normalization quietly goes wrong. Judge biases and project qualities are only defined relative to each other, so the model has a free parameter you have to pin down or the numbers wander: add a constant to every judge's bias and subtract it from every quality, and nothing observable changes. The fix is to require each connected group of judges-and-projects to sum to zero. "Connected" is meant literally — two projects are in the same group if a chain of shared judges links them — and it carries an honest limit worth stating out loud: two projects that share no judge, even transitively, live in different groups with no common zero, so the model refuses to rank them against each other rather than inventing a bridge. On this event everything was one connected component, so the limit never bit; the code enforces it regardless, and a self-check confirms the gauge sums return to zero within floating-point noise — about 1e-17 — before a run is ever signed.
On the seed event — 126 ballots, 41 projects, 30 judges across 8 tracks, all one connected component — this is not cosmetic: cross-validation lands on λ = 10, and correcting for judge bias is enough to break a tie that the raw averages leave at the top.
Then it measures its own confidence, and this is the number I did not get to choose:
Across all 40 adjacent pairs on the leaderboard, not one is statistically resolved. For the closest, the model's probability that the higher-ranked project is actually better than the one just below it is about 0.518 — a coin flip. Correcting for judge bias tightened within-submission spread by about 1.08×, not the dramatic factor a demo would prefer.
The honest reading is that these judges were fairly consistent and the field was close, so a strict ranking would be inventing separation the ballots do not contain. The tool therefore publishes the pairwise probabilities and intervals next to the ranking, so "3rd vs 4th" reads as the coin flip it is. Publish the uncertainty, not just the order — a ranking that hides it is the same flattering lie as a benchmark that reports its fastest run.
The bug that only appears after a round-trip through Postgres
The signed run is supposed to be reproducible: sign it once, and anyone re-running the estimator on the same ballots should land on the same bytes and the same signature. It worked in memory and failed the moment the run was reloaded to verify it.
The cause was -0.0. The gauge that pins each group to a common zero legitimately produces negative-zero components; in memory that is what got signed. But the run is stored as jsonb, and the reload path normalizes -0.0 to 0.0. So the verifier hashed a projection that differed from the signed one in a byte that is numerically invisible — -0.0 == 0.0 is true — and the signature check failed for a reason no arithmetic could see.
The fix was not to special-case the comparison. It was to normalize negative zero inside the projection that gets hashed, before signing, so the run signs exactly what the database will hand back. Hash what you'll read back, not what you happen to be holding — a signature over an in-memory object is a signature over a serialization decision you forgot you were making.
Append-only, not delete
A score is never edited in place. Each revision is an append-only row keyed UNIQUE(ballot, version), with a database CHECK keeping every score in 1–5, and the current ballot is just a pointer at the latest version. A withdrawal is a soft state flip, not a DELETE, and the admin path refuses to cascade-delete anything a scored ballot depends on. History survives the people who would rather it didn't.
Underneath the ballots is the log that makes them evidence rather than just rows: an append-only audit chain on a schema I froze on purpose, so its meaning cannot drift out from under old events. There is a single head row and a stream of events, and each event stores its own payload, the hash of that payload, the hash of the event before it, and a row hash over the whole record — the ordinary hash-chain shape, where altering any past event breaks every hash that follows it. Writing one event takes a select_for_update lock on the head so two concurrent writers cannot fork the chain, links the new event to the current head, and advances the head — all inside a single database transaction.
The load-bearing claim there is that the data write and its audit event are one atomic thing — that you can't get the score without the chain entry, or vice-versa. The way I believe that is a test that can fail: it monkeypatches the audit chain head's save() to raise mid-transaction, then writes a ballot. Afterwards, the ballot, the judge assignment, and the audit event are all absent, and the chain head is still at sequence 0. A test that cannot fail is decoration; this one holds a real invariant, because it forces the exact failure the invariant is supposed to survive.
Permission is a grant on an event, not a role
Judging platforms tend to leak along their authorization seams: a judge from one event nudges a score in another, an organizer glimpses a draft they should not. The tempting shortcut is a global role — is_judge, is_organizer — checked in a view. It is the same category error as the results table, convenient and wrong at exactly the edges that matter. So permission here is a grant scoped to a specific event, and it is enforced in the service layer where the write actually happens, not only in a view decorator that some second code path could quietly bypass. To keep myself honest about that, a mutation test deletes the check and expects a test to go red — if removing an authorization guard changes nothing, the guard was never load-bearing in the first place. Invitations are cut from the same cloth: each is signed and single-use, and "single-use" is enforced by taking a row lock and marking the invite spent inside the accept transaction, so two clicks on the same link cannot both succeed. The honest boundary is that this enforcement lives in the application's service layer, not yet in the database as row-level security — a deferral I wrote down on purpose rather than a thing I hope nobody notices.
Living without a broker
The "no moving parts" rule means the usual answers were off the table, so a few things are hand-built that a bigger stack would outsource. Rate limiting and caching run on Django's database-backed cache — a table created at boot, not a Redis. Static files are served in-process by WhiteNoise, vendored into the image, because there is no CDN to reach on a machine with the cable pulled. And the entrypoint does in sequence what an orchestrator usually spreads across services: wait for the database, generate the secret key race-safely with O_EXCL on first boot only, migrate, create the cache table, seed the demo event if the database is empty, ensure the admin exists, then hand off to gunicorn. It is less clever than a broker, and it has the property I wanted: one command, offline, and nothing to stand up first.
The contract is replayed, not assumed
A grader checks the portal by hitting a fixed set of routes and asserting on the responses. That contract is easy to break by accident — a refactor renames a path, a template grows a nav link, and the grade drops for a reason unrelated to the feature you were building. So the routes the grader depends on are frozen, feature routes are appended after them and never linked from the base navigation, and a stdlib-only tools/replay.py re-runs the same seven checks the grader runs, from the same contract file the grader reads. It is stricter in exactly one way — it does not follow redirects, so a route that only reaches 200 after an append-slash bounce is a visible failure rather than a silent pass — and it ends on a green 7/7 checks green. The compatibility claim is a thing the build re-measures, not a thing I remember being true.
Proof that it runs on a machine that isn't mine
"Works on my machine" is the oldest lie in software, and a portal that only boots on the laptop that built it has proven nothing about whether you can run it. Two things guard against that. The first is that single docker compose up: a cold clone, an empty volume, the network off, and the container brings itself all the way up — waits for the database, generates its secret, migrates, seeds the demo event, creates the admin — with no manual step I could have forgotten to document, because there is nowhere to hide one. The second is continuous integration that does exactly this on hardware that has never seen my working tree: it clones fresh and runs the suite against a real Postgres on every push, so "it boots and it passes" is a fact re-established by a stranger rather than a thing I merely remember working. The image build is gated as well — Django's own system checks at --fail-level WARNING, plus a makemigrations --check that fails the build when a model changed without its migration, which is precisely the mistake that turns a green local run into a broken clone. And the tests come in two deliberate layers: the pure-Python core — the hash-chain, the signing format, the estimator's math — has tests that need no database and run anywhere, while the full suite runs against Postgres in CI. The proof that something reproduces is someone else reproducing it, so the whole build is arranged to make that the cheap and default path rather than a favor I have to ask.
A signature is evidence about a result, not about my character
It is worth being precise about what the cryptography does and does not buy, because the failure mode of a security feature is someone trusting it past its contract. A valid signature proves two things: the bytes have not changed since the key-holder signed them, and anyone can re-derive the ranking from the pinned ballots and get the same answer. It does not prove the judging was fair, that the ballots were honest, or that I did not sign a bad result on purpose. A determined operator with raw database access can still change data — the chain does not make that impossible, it makes it detectable: the re-derivation or a hash link fails, and the verifier tells you which one. Detectable is a weaker and far more honest promise than tamper-proof, and it is the one the design can actually keep.
The same restraint applies to the review diagnostics — leave-one-out residuals, judge influence, coverage gaps. They ship behind a banner that says, in the product, that they are descriptive and not fraud detection, because a residual is a question worth a human's attention, not a verdict. Describe what the proof covers, and put the limit where the reader is, not in your own head.
I made the documentation fail the build
This is the practice that makes any number in this post safe to write. A README figure goes stale in the commit after the one that earned it, and prose does not fail a build, so a handful of tests do nothing but read a document and compare it to the code — the threat-model table has to match the controls that actually exist, and a doc that names a route or a setting has to name one the code really defines. For each claim, ask what file would contradict it, then read both in one test.
The honest-claims rule is the same tool pointed at myself: every capability the docs describe has to map to a feature that actually shipped in src/. The week of submission that rule cost me two lines — a demo script called the replay checker a "byte-for-byte" comparison when it runs status-and-content checks and follows no redirects, and a config comment claimed a test asserted something no test asserted. Both were true-ish and both were wrong, so both got reworded before the freeze. A capability you can describe but not point at is not a feature, it is a plan.
What it does not do
Multi-event support is partial, by a decision recorded in the threat model rather than an oversight. The diagnostics are descriptive and not fraud detection. Row-level security is deferred, so raw-database tampering is detectable, not prevented. The normalizer will tell you two adjacent ranks are a coin flip and will not pretend otherwise to make a cleaner slide. And it is one person's 72 hours — the seam is real, it is written down, and the verifier is exactly the thing that keeps the seam from being load-bearing.
Try it
git clone https://github.com/pal-123456789/dogfood-portal
cd dogfood-portal
docker compose up --build --wait
Open the gallery at /projects and the signed leaderboard at /normalize/results, publish the results, then export a bundle and — with the db container stopped — run python -m normalize.release over it and watch it re-derive the ranking and check every signature offline.
Prefer to watch first? Here's the walkthrough:
- Repository: pal-123456789/dogfood-portal
- Docs (architecture, data model, judging, ablation, threat model): pal-123456789.github.io/dogfood-portal
- Runbook / 60-second demo: run.md
- Interactive walkthrough site: dogfood-site-eta.vercel.app
- Video walkthrough: youtu.be/-NThDgcTt-8
- On X: the thread
If you take one thing from this, let it be the move rather than the app: don't ask anyone to trust that your results are right — hand them the thing that lets them check, offline, without you in the room. Judging is a place people are right to be a little suspicious, and the honest answer to suspicion is not a louder claim, it is a smaller thing they can verify for themselves. The database will always tell them whatever it currently says. Whether that matches what actually happened is a different question — and the whole point of the 72 hours was to make that question answerable by someone who has no reason to take my word for anything.
Top comments (0)