Most hackathon portals answer one question — who won? — with a row in a database. The row renders, everyone moves on, and nobody asks the harder question that no row can answer on its own: how would you know if it were wrong?
Dogfood Portal is a self-hostable platform for running a hackathon's submissions and judging: teams submit, judges score, and it publishes a final leaderboard. I gave myself two rules for the 72 hours. It has to come up from a single docker compose up with the network unplugged — no Redis, no message broker, no background worker, no CDN, two services in the compose file and one of them is Postgres. And the results it publishes have to be verifiable by anyone, offline, without trusting my server. The second rule is the whole post.
Here is the thing that rule forces you to admit:
A hackathon portal's "final results" are a
SELECTover a table. Anyone who can reach the database canUPDATEa score after the fact, and the results page will render the new number exactly as confidently as the old one. Nothing on the page knows it changed.
That is not a Django problem and it is not a Postgres problem. A database stores current state; it was never a record of what happened. Treating "it's in the database" as "it's the result" is a quiet category error — reaching for a tool whose contract is close to the one you want, reading storage as if it were proof, and shipping the gap without noticing it is there. So the results here carry their own proof, and there is one command that re-checks all of it with the database container stopped.
What I built
Dogfood Portal is a single Django 5.2 + PostgreSQL 16 monolith — one web service, one db service, nothing else. It boots offline, runs the migrations, seeds a demo event (an organizer, judges, teams, and scored submissions) on first start, and serves a public project gallery, a judging flow, and a normalized leaderboard. The interesting half is underneath: the final results are exported as a signed, reproducible bundle, and there is an offline verifier that re-derives and re-checks the whole thing from that bundle alone.
$ docker compose exec -T web python src/manage.py release_bundle /state/release
$ docker compose exec -T -w /app/src web python -m normalize.release /state/release
[PASS] ranking.csv matches the signed run
[PASS] signed run reproduces from the pinned ballots
[PASS] normalization.published event is on the audit chain
[PASS] audit checkpoint signature verifies
[PASS] every artifact traces to one operator public key
RESULT: VERIFIED
That second command needs no database and no network. It is the point of the project, so the rest of this post is what "verifiable" had to actually mean, and what generalizes past one Django app.
What a database won't do for you
The naive design is one table: results, with a rank and a score per project, written when the organizer clicks publish. It renders fine. It is also unfalsifiable — there is nothing to check it against, because the row is simultaneously the claim and the only evidence for the claim.
So publishing does not write a row you have to trust. It writes a chain you can follow:
- the normalized ranking is computed and frozen as a signed run — the pinned input ballots, the model settings, the resulting scores, and an Ed25519 signature over all of it;
- that publication is recorded as a
normalization.publishedevent on an append-only audit hash-chain, where each event carries the hash of the one before it; - the chain head is sealed with a signed checkpoint;
- the exported
ranking.csvis bound back to the signed run; - and every signature in the bundle traces to one operator public key.
The verifier walks that chain backwards. It re-runs the estimator from the pinned ballots and checks the result it gets matches the signed one; it re-checks every signature; it confirms the published event actually sits on the chain and that the checkpoint over the chain verifies; it confirms one key signed all of it. Any broken link fails the run and names itself — signed run does NOT reproduce, not a generic false. With the db container stopped, from the bundle alone, on a machine that has never seen my server.
Everything that verifier needs travels in one exported bundle: the ranking CSV, the signed run with its pinned inputs and result, the audit checkpoint and the slice of chain leading up to it, and the single public key the whole thing is signed under. The export self-verifies before it will finish writing, so a bundle that does not check out is never produced in the first place. For the everyday case — "prove to me this leaderboard is the one you actually published" — there is a smaller dogfood.results-certificate.v1 you can hand to a single team, and it verifies through the same path. The bundle is the audit; the certificate is the receipt; both are checked by code, not taken on faith.
The transferable version is small: sign the result; don't ask people to trust the server. If the only evidence for a number is the system that produced it, you have not published a result, you have published an assertion.
The measurement: a leaderboard implies a precision the ballots don't have
Raw judge scores are not comparable. One judge's 7 is another's 9, and if two projects were seen by different judges, their averages are measuring the judges as much as the work. So the leaderboard is not a mean — it is a ridge / partial-pooling estimate that models each ballot as quality + judge_bias + noise, fits it with a gauge that pins each connected group of judges-and-projects to a common zero, and picks the regularization strength by cross-validation rather than by taste.
That gauge is the subtle part, and it is where most naive score-normalization quietly goes wrong. Judge biases and project qualities are only defined relative to each other, so the model has a free parameter you have to pin down or the numbers wander: add a constant to every judge's bias and subtract it from every quality, and nothing observable changes. The fix is to require each connected group of judges-and-projects to sum to zero. "Connected" is meant literally — two projects are in the same group if a chain of shared judges links them — and it carries an honest limit worth stating out loud: two projects that share no judge, even transitively, live in different groups with no common zero, so the model refuses to rank them against each other rather than inventing a bridge. On this event everything was one connected component, so the limit never bit; the code enforces it regardless, and a self-check confirms the gauge sums return to zero within floating-point noise — about 1e-17 — before a run is ever signed.
On the seed event — 126 ballots, 41 projects, 30 judges across 8 tracks, all one connected component — this is not cosmetic: cross-validation lands on λ = 10, and correcting for judge bias is enough to break a tie that the raw averages leave at the top.
Then it measures its own confidence, and this is the number I did not get to choose:
Across all 40 adjacent pairs on the leaderboard, not one is statistically resolved. For the closest, the model's probability that the higher-ranked project is actually better than the one just below it is about 0.518 — a coin flip. Correcting for judge bias tightened within-submission spread by about 1.08×, not the dramatic factor a demo would prefer.
The honest reading is that these judges were fairly consistent and the field was close, so a strict ranking would be inventing separation the ballots do not contain. The tool therefore publishes the pairwise probabilities and intervals next to the ranking, so "3rd vs 4th" reads as the coin flip it is. Publish the uncertainty, not just the order — a ranking that hides it is the same flattering lie as a benchmark that reports its fastest run.
The bug that only appears after a round-trip through Postgres
The signed run is supposed to be reproducible: sign it once, and anyone re-running the estimator on the same ballots should land on the same bytes and the same signature. It worked in memory and failed the moment the run was reloaded to verify it.
The cause was -0.0. The gauge that pins each group to a common zero legitimately produces negative-zero components; in memory that is what got signed. But the run is stored as jsonb, and the reload path normalizes -0.0 to 0.0. So the verifier hashed a projection that differed from the signed one in a byte that is numerically invisible — -0.0 == 0.0 is true — and the signature check failed for a reason no arithmetic could see.
The fix was not to special-case the comparison. It was to normalize negative zero inside the projection that gets hashed, before signing, so the run signs exactly what the database will hand back. Hash what you'll read back, not what you happen to be holding — a signature over an in-memory object is a signature over a serialization decision you forgot you were making.
Append-only, not delete
A score is never edited in place. Each revision is an append-only row keyed UNIQUE(ballot, version), with a database CHECK keeping every score in 1–5, and the current ballot is just a pointer at the latest version. A withdrawal is a soft state flip, not a DELETE, and the admin path refuses to cascade-delete anything a scored ballot depends on. History survives the people who would rather it didn't.
Underneath the ballots is the log that makes them evidence rather than just rows: an append-only audit chain on a schema I froze on purpose, so its meaning cannot drift out from under old events. There is a single head row and a stream of events, and each event stores its own payload, the hash of that payload, the hash of the event before it, and a row hash over the whole record — the ordinary hash-chain shape, where altering any past event breaks every hash that follows it. Writing one event takes a select_for_update lock on the head so two concurrent writers cannot fork the chain, links the new event to the current head, and advances the head — all inside a single database transaction.
The load-bearing claim there is that the data write and its audit event are one atomic thing — that you can't get the score without the chain entry, or vice-versa. The way I believe that is a test that can fail: it monkeypatches the audit chain head's save() to raise mid-transaction, then writes a ballot. Afterwards, the ballot, the judge assignment, and the audit event are all absent, and the chain head is still at sequence 0. A test that cannot fail is decoration; this one holds a real invariant, because it forces the exact failure the invariant is supposed to survive.
Permission is a grant on an event, not a role
Judging platforms tend to leak along their authorization seams: a judge from one event nudges a score in another, an organizer glimpses a draft they should not. The tempting shortcut is a global role — is_judge, is_organizer — checked in a view. It is the same category error as the results table, convenient and wrong at exactly the edges that matter. So permission here is a grant scoped to a specific event, and it is enforced in the service layer where the write actually happens, not only in a view decorator that some second code path could quietly bypass. To keep myself honest about that, a mutation test deletes the check and expects a test to go red — if removing an authorization guard changes nothing, the guard was never load-bearing in the first place. Invitations are cut from the same cloth: each is signed and single-use, and "single-use" is enforced by taking a row lock and marking the invite spent inside the accept transaction, so two clicks on the same link cannot both succeed. The honest boundary is that this enforcement lives in the application's service layer, not yet in the database as row-level security — a deferral I wrote down on purpose rather than a thing I hope nobody notices.
Living without a broker
The "no moving parts" rule means the usual answers were off the table, so a few things are hand-built that a bigger stack would outsource. Rate limiting and caching run on Django's database-backed cache — a table created at boot, not a Redis. Static files are served in-process by WhiteNoise, vendored into the image, because there is no CDN to reach on a machine with the cable pulled. And the entrypoint does in sequence what an orchestrator usually spreads across services: wait for the database, generate the secret key race-safely with O_EXCL on first boot only, migrate, create the cache table, seed the demo event if the database is empty, ensure the admin exists, then hand off to gunicorn. It is less clever than a broker, and it has the property I wanted: one command, offline, and nothing to stand up first.
The contract is replayed, not assumed
A grader checks the portal by hitting a fixed set of routes and asserting on the responses. That contract is easy to break by accident — a refactor renames a path, a template grows a nav link, and the grade drops for a reason unrelated to the feature you were building. So the routes the grader depends on are frozen, feature routes are appended after them and never linked from the base navigation, and a stdlib-only tools/replay.py re-runs the same seven checks the grader runs, from the same contract file the grader reads. It is stricter in exactly one way — it does not follow redirects, so a route that only reaches 200 after an append-slash bounce is a visible failure rather than a silent pass — and it ends on a green 7/7 checks green. The compatibility claim is a thing the build re-measures, not a thing I remember being true.
Proof that it runs on a machine that isn't mine
"Works on my machine" is the oldest lie in software, and a portal that only boots on the laptop that built it has proven nothing about whether you can run it. Two things guard against that. The first is that single docker compose up: a cold clone, an empty volume, the network off, and the container brings itself all the way up — waits for the database, generates its secret, migrates, seeds the demo event, creates the admin — with no manual step I could have forgotten to document, because there is nowhere to hide one. The second is continuous integration that does exactly this on hardware that has never seen my working tree: it clones fresh and runs the suite against a real Postgres on every push, so "it boots and it passes" is a fact re-established by a stranger rather than a thing I merely remember working. The image build is gated as well — Django's own system checks at --fail-level WARNING, plus a makemigrations --check that fails the build when a model changed without its migration, which is precisely the mistake that turns a green local run into a broken clone. And the tests come in two deliberate layers: the pure-Python core — the hash-chain, the signing format, the estimator's math — has tests that need no database and run anywhere, while the full suite runs against Postgres in CI. The proof that something reproduces is someone else reproducing it, so the whole build is arranged to make that the cheap and default path rather than a favor I have to ask.
A signature is evidence about a result, not about my character
It is worth being precise about what the cryptography does and does not buy, because the failure mode of a security feature is someone trusting it past its contract. A valid signature proves two things: the bytes have not changed since the key-holder signed them, and anyone can re-derive the ranking from the pinned ballots and get the same answer. It does not prove the judging was fair, that the ballots were honest, or that I did not sign a bad result on purpose. A determined operator with raw database access can still change data — the chain does not make that impossible, it makes it detectable: the re-derivation or a hash link fails, and the verifier tells you which one. Detectable is a weaker and far more honest promise than tamper-proof, and it is the one the design can actually keep.
The same restraint applies to the review diagnostics — leave-one-out residuals, judge influence, coverage gaps. They ship behind a banner that says, in the product, that they are descriptive and not fraud detection, because a residual is a question worth a human's attention, not a verdict. Describe what the proof covers, and put the limit where the reader is, not in your own head.
I made the documentation fail the build
This is the practice that makes any number in this post safe to write. A README figure goes stale in the commit after the one that earned it, and prose does not fail a build, so a handful of tests do nothing but read a document and compare it to the code — the threat-model table has to match the controls that actually exist, and a doc that names a route or a setting has to name one the code really defines. For each claim, ask what file would contradict it, then read both in one test.
The honest-claims rule is the same tool pointed at myself: every capability the docs describe has to map to a feature that actually shipped in src/. The week of submission that rule cost me two lines — a demo script called the replay checker a "byte-for-byte" comparison when it runs status-and-content checks and follows no redirects, and a config comment claimed a test asserted something no test asserted. Both were true-ish and both were wrong, so both got reworded before the freeze. A capability you can describe but not point at is not a feature, it is a plan.
What it does not do
Multi-event support is partial, by a decision recorded in the threat model rather than an oversight. The diagnostics are descriptive and not fraud detection. Row-level security is deferred, so raw-database tampering is detectable, not prevented. The normalizer will tell you two adjacent ranks are a coin flip and will not pretend otherwise to make a cleaner slide. And it is one person's 72 hours — the seam is real, it is written down, and the verifier is exactly the thing that keeps the seam from being load-bearing.
Try it
git clone https://github.com/pal-123456789/dogfood-portal
cd dogfood-portal
docker compose up --build --wait
Open the gallery at /projects and the signed leaderboard at /normalize/results, publish the results, then export a bundle and — with the db container stopped — run python -m normalize.release over it and watch it re-derive the ranking and check every signature offline.
Prefer to watch first? Here's the walkthrough:
- Repository: pal-123456789/dogfood-portal
- Docs (architecture, data model, judging, ablation, threat model): pal-123456789.github.io/dogfood-portal
- Runbook / 60-second demo: run.md
- Interactive walkthrough site: dogfood-site-eta.vercel.app
- Video walkthrough: youtu.be/-NThDgcTt-8
- On X: the thread
If you take one thing from this, let it be the move rather than the app: don't ask anyone to trust that your results are right — hand them the thing that lets them check, offline, without you in the room. Judging is a place people are right to be a little suspicious, and the honest answer to suspicion is not a louder claim, it is a smaller thing they can verify for themselves. The database will always tell them whatever it currently says. Whether that matches what actually happened is a different question — and the whole point of the 72 hours was to make that question answerable by someone who has no reason to take my word for anything.
Top comments (4)
The second rule is the whole post, agreed, and the offline verifier with the database container stopped is the part that makes it a claim rather than a feeling. Most projects that say verifiable mean re-checkable by the thing that produced it.
The boundary I would state explicitly in the README is where the chain starts. VERIFIED tells you the published ranking matches the signed run, and that the signed run reproduces from the pinned ballots. It cannot tell you the pinned ballots are the ballots the judges entered. An UPDATE on a score before the bundle is signed produces a bundle that passes all five checks, because everything in the chain is derived from the store it is attesting to. So the trust boundary is the moment of pinning, and everything upstream of it is still a database with your own words about it.
The move that closes it is signing the ballot in the judge's client with a key your server never holds. Then the bundle carries signatures the operator could not have produced, and the trust shifts from you to each judge. For a 72-hour build that is not worth the key management, which is a fine answer. Saying so in the README is a better outcome than doing it, because the failure mode for verifiable systems is a reader who assumes the chain covers one step further than it does.
The payments version of the same line is why an authorisation is signed by the payer's bank and not recorded by the merchant. A merchant's log can be internally perfect and it is still one party attesting to its own behaviour. I wrote about that failure here and yours is the cleaner demonstration of it: dev.to/mickyarun/your-audit-log-ag...
You're right, and you've named the seam I most wanted stated honestly.
The verifier re-derives the ranking from the ballots pinned inside the bundle (inputs.json), not from the live database — so every check hangs off that pinned snapshot. Chain the implications the way you did and it's unavoidable: an UPDATE on a score before the run is built gets pinned, hashed, and signed into the run, and the publication is co-committed to the audit chain like any honest run — and the whole thing verifies. It's actually a touch worse than "five checks": the
releasecommand also runs the audit-chain verifier and four cross-links (checkpoint and run share one signer, the finalization event is really in the chain, it sits under the signed head, ranking.csv matches the signed result). A pre-pin edit sails through all of those too, because they're all downstream of the same pinned inputs. More checks, same blind spot. The trust boundary is the moment of pinning, exactly as you said.The closest the current build comes to catching it is that the chain still holds the earlier honest ballot event, so a lazy raw edit (score changed, chain left untouched) can be reconciled against it — but that's manual, and it collapses the moment the operator brings the chain along, because the same operator key signs the chain and the checkpoint. Which is just your point restated: a log and a key that both live on my host don't bind me. The threat model says this at the operator boundary (§A8: prevention is an accepted risk — "an operator who rewrites history can re-sign a fresh, self-consistent checkpoint over it"), and the verifier's own docstring says a PASS is evidence against the operator only if an independent party pinned the public key before judging.
But you've sharpened it in a way my docs don't. You moved the question from "can post-hoc history be rewritten" to "was the pinned input ever authentic" — and, the part I hadn't named, you named the fix. My docs stop at "pin the key externally"; they never say the real closure is signing the ballot in the judge's client under a key the server never holds. That's the right answer, and you're right that it moves the root of trust from one operator key to N judge keys.
For 72 hours it was out of scope — key distribution, in-browser signing, and a key-loss recovery story is a lot, and I'd rather ship no judge-signing than half of it. But your closing point is the one I'm going to act on: the README should say the boundary out loud instead of leaving it a level down in the threat model, so nobody reads the chain as covering one step more than it does. Adding it.
And the payments analogy lands exactly. The authorization is the issuer's cryptogram — something the merchant literally cannot produce — while the merchant's own books, however immaculate, are one party attesting to its own behavior. That's the difference between a signature that binds a counterparty and a signature that binds me to my own story. My bundle is the second kind today, and honest verifiability means saying which kind it is. Great post — the title is the whole thesis in six words.
@pal11103 — more checks, same blind spot is the better statement of my own point, and it generalises past your bundle. The number of checks in a verifiable system tells a reader nothing, because checks are cheap and assumptions are not. The number worth publishing is how many distinct keys the checks depend on. Yours is one. Judge-signed ballots make it N. Everything between those two numbers is presentation.
There is a move between one and N that costs no key management, and it is weaker than judge signing in a way that might fit 72 hours. Give each judge a copy of their own ballot when it is accepted. A digest, emailed, held by them. You cannot produce a judge's copy of their own ballot, so the pinned input becomes challengeable by the only party who knows what they entered. It does not prevent the pre-pin edit. It makes it detectable from outside your host, which is the right target when A8 already accepts prevention as out of scope at the operator boundary.
The payments version is the one I would copy exactly, because the mechanism is not the sealing. A cardholder statement is not trusted because it is tamper-evident. It is trusted because the cardholder gets a copy, and the dispute has a deadline and an address. A receipt with no deadline and nobody to complain to is decoration. So the README line is not we send judges a copy of their ballot. It is who a judge tells, by when, and what happens to a published ranking if they do. Without the second half the receipt proves nothing, because nobody is obliged to look.
The limit is the same one a level up. A judge can only dispute their own row. Nothing here catches a ballot that was never entered by anyone, which is a fabricated judge, and that needs the roster pinned by someone outside the operator before judging starts. Your docs already say pin the public key externally. The roster has the identical requirement, and publishing it at the start somewhere you cannot edit is the closest I get for a 72-hour build. It only works if the judges read it.
"The number worth publishing is how many distinct keys the checks depend on" is the sentence I wish I'd led with, and it belongs in the README right where I currently brag about the number of checks: one. The chain and the checkpoint are signed by the same operator key, so all five checks and the four
releasecross-links collapse to a single trust root — mine. The part I'd add is that your receipt move doesn't push that number off one; it adds zero keys. And that's the point, not a limitation — it isn't a cryptographic move at all, it's a distribution move. The pinned input stops being the only copy. One key, N copies in hands I don't control: a different axis entirely, and the cheap one.It fits the build better than it has any right to, because the ballot already lands in the chain as an append-only event with its own digest the moment it's accepted. So the material is there; what's missing is the two things that make it bind. First, the digest has to be one the judge can recompute from what they actually typed — a canonical serialization of their own scores and comments — or the receipt is an opaque hash they can't check against their memory. Second, it has to leave my host at acceptance, before pinning, into a copy they hold with a timestamp I can't reach. Those two together are what turn "the operator says this is your ballot" into "the judge holds proof of what they entered."
And here's the trap I'd have walked straight into if you hadn't framed it as a copy held by them: a receipt I generate and email is still my bytes. "I sent you your digest" is one party attesting to its own behaviour — the exact failure from the post, moved out one ring. It binds because the judge's copy is independently timestamped in their inbox and predates my pinning, not because I can't produce the digest — I obviously can, it's sitting in my DB. So the receipt has to be judge-recomputable and go out at acceptance, or it's just decoration with a better reputation.
Which is your real correction, and it's the half my docs don't have. "We send judges a copy" is a feature. "Who a judge tells, by when, and what happens to the published ranking if they do" is what makes the copy load-bearing. So the README line isn't about the receipt at all — it's three slots: an address that isn't me (the organiser's channel, ideally), a deadline tied to publication, and a stated consequence — the ranking is marked disputed and the run re-opened for public re-pin, not quietly fixed. A receipt with no deadline and no one obliged to act on it proves nothing, exactly as you said. I'd rather ship the procedure than the email.
The fabricated judge is the one that stings, because my build has no answer to it today. A ballot nobody entered pins, hashes, and signs identically to a real one — a judge can only dispute their own row, and a row with no judge behind it has no one to dispute it. You've named it as the symmetric requirement and that's precisely what it is: the roster needs the same external pin the public key already gets. Published at kickoff, somewhere I can't edit afterwards — the organiser's platform, a commit someone else timestamps, anything off my host — so the set of judges is fixed before judging the same way the key is. For 72 hours that's the whole fix, and it's cheap. The catch is the one you ended on: it only works if someone reads it.
Which is where the two posts meet, I think. Mine was "your database is not an audit log." Yours is the sequel I didn't write — an audit log is not a dispute. The chain makes tampering evident; it doesn't make anyone look, give them a deadline, or let them act. And every mechanism in this thread — receipts, roster, judge keys — is really just a way of putting the ability to dissent into a hand that isn't mine. So the honest number isn't checks, and it isn't even keys. It's how many parties outside the operator are both able and obliged to say no. Mine is zero today. Receipts get it to N judges who can each challenge their own row; the pinned roster is what stops me inventing the N. The repo's frozen for judging until results on the 18th, so I'm editing nothing until then — but both are first in line when it reopens, as named boundaries in the README instead of a level down in A8, because the failure mode is a reader who assumes the chain reaches one step further than it does. Thanks for driving this all the way to the root; your authorisation-vs-merchant-log post is the cleaner statement of it, and I'll be linking it from mine.