Holdfast: a webhook safety net where Gemma explains what broke and code decides what happens

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

My day job is message queues in containers. The first rule there is boring and absolute: a message isn't acknowledged until it's safely stored. Lose power a millisecond later and it's still there.

Webhooks don't work like that. Stripe or GitHub sends your app a message, and if the app is down, slow, or chokes on a field that was renamed overnight, the message is usually just gone. The person who finds out is the shop owner, two days later, asking why three payments never showed up.

Holdfast is that queue rule, turned into a product for the people who get that phone call: freelance developers and small agencies who look after integrations for a handful of clients. It sits between the provider and the app and:

1. Saves every webhook to disk before saying "got it".
2. Checks the signature, and keeps forged or replayed messages aside.
3. Delivers patiently: retries when the app is down, holds when a retry won't help.
4. Explains what broke in two voices: a plain sentence the client can read, and a precise fix for the developer. Gemma writes both; code checks every word.
5. Sends held messages again, never twice.

Who pays: the freelancer who maintains "the Stripe thing" for five small clients. One lost payment webhook is a lost order and an awkward conversation. Holdfast turns that into a note they can forward:

Kabir's ₹1050 payment for Ginger chai, 500 g couldn't reach Chai & Co. because the sender changed how it sends customer information. Holdfast is keeping the message safe, and it can be sent again once the app is updated to expect the new format.

That's a real answer from the deployed site, written by Gemma 3 4B running on Render.

Demo

Live: https://holdfast-o9ba.onrender.com

It has two modes, switchable from the header:

• Demo (/demo, no sign-up): Chai & Co., a pretend tea shop with made-up customers. Seven buttons break it on purpose: the shop goes down, a field is renamed, a payment arrives twice, a signature is forged, an old payment is replayed. The relay, retries, checks and explanations are the real engine.
• Live (/live): sign in with GitHub, pick one of your repositories, and Holdfast adds itself as the webhook. GitHub's test ping arrives within seconds. If you don't have an app, deliver to Holdfast's built-in test app and switch it to "Server down" or "Turns messages away" to watch a real webhook get held, explained and resent.

Landing Page:

Demo:

Event Gemma:

One honest note: Gemma runs on two CPU cores, so an explanation takes up to a minute and a half (48 s and 83 s in my last two live runs). The page shows the built-in rules' answer instantly and swaps in Gemma's when it's ready.

Code

Tanay3484 / holdfast

A safety net for webhooks: stores every event, holds what your app fails on, explains why with an open-weight model, and resends safely.

Holdfast

A safety net for webhooks. Holdfast sits between a provider (Stripe, GitHub) and your app. It stores every event before acknowledging it, holds the ones your app fails on, explains why in plain words with an open-weight model, and resends them safely once the problem is fixed.

Built for the Hacktoberfest 2026 Open-Source AI Challenge, Week 2: Side Hustle.

How it works

provider ──▶ /in/{route} ──▶ check signature ──▶ save to disk ──▶ "got it"
                                                      │
                     deliver to your app ◀────────────┘
                       ├─ accepted      → done
                       ├─ worth a retry → wait a little longer each time
                       └─ won't work    → hold it safely
                                             │
                     code collects the facts ┘  (signature, every try, what changed since the last success)
                     Gemma names the cause from a fixed list and explains it; code checks every word
                     resend: never twice, never anything that failed its signature check

• Code decides, the model explains.…

View on GitHub

Go, about 5,900 lines plus 1,550 lines of tests (59 tests). Every feature has a spec in specs/ with numbered requirements.

How I Built It

Open-weight model: Gemma 3 4B, served by Ollama in a private service next to the app. Stack: Go standard library, SQLite (pure-Go driver), server-rendered HTML, a little vanilla JS. Hosting: a single Render Blueprint (more below).

Code decides, the model explains

The rule I held to: the model never decides what happens to a message. Retries, holds, duplicate detection and resends are plain, tested Go. Gemma gets the facts code collected (signature result, every delivery attempt, the app's error, what changed in the payload) and does two jobs:

1. pick a cause from a fixed list of eight, under a JSON schema, and
2. write the explanation.

Then code checks what it wrote. The client summary is rejected and replaced with the built-in rules' wording if it contains jargon ("HTTP", "payload", "endpoint", an error number) or promises something nobody did. Titles always come from code.

The bug that taught me the most

First version, renamed-field scenario. Gemma got the cause right, but the client summary said:

A payment notification failed to reach the Chai & Co. system. The notification is being kept safe and reviewed.

Nobody was reviewing anything. Worse, the developer note said the app "now requires customer_id", which is backwards: Stripe changed, the app didn't.

The fix wasn't a better prompt alone. It was moving more work into code:

• Code finds the rename. A shape-diff walks both JSON bodies and spots that data.object.customer_id disappeared while data.object.customer.id appeared. That's one rename, with a direction. Gemma is told "the sender now sends X where it used to send Y; the app did not change" instead of a raw list of paths.
• The summary has a required shape: what happened to this payment (who, how much), why in everyday words, what happens next.
• A Promise check rejects "being reviewed", "will resend", "our team is…".
• The developer fix must name the field that moved, or the rules' fix is used.

After the fix, on the same scenario:

For your developer: The sender now uses data.object.customer.id instead of data.object.customer_id. Change the app to expect data.object.customer.id.

Model size, measured

Model	Where	Result on the renamed-field case
Gemma 3 1B	laptop	Wrote "the app will resend the held events" (a promise) and told the sender to change. The checks now catch both and fall back to the rules' answer.
Gemma 3 4B	laptop GPU	Named the customer and amount, explained the change plainly, and pointed at the exact field. About 16 s.
Gemma 3 4B	Render, 2 CPUs	Same quality. 92 s for one explanation.
Gemma 3 4B	Render, after the fixes below	Two live runs, both explained by Gemma: 48 s and 83 s.

On a laptop CPU with two threads, one explanation was 740 prompt tokens read in 15 s and 115 tokens written in 14 s. That's why the page shows a rules answer first.

How I built it: an unlock game

I built this with Claude Code, spec first. The four core pieces started as stubs I had to write by hand, each a "level" with known-answer checks. go run ./cmd/unlock prints a progress board, and while a level was locked, every part of the website it powered showed a "WIP: write this file to reveal how this will look" tile. Writing a file literally revealed part of the site.

Being honest about who wrote what (it's recorded in tasks.md):

• Level 1, signatures: I wrote GitHub's HMAC check. Stripe's was started by me and finished by Claude.
• Level 2, retry policy: written by me (backoff, Retry-After, the hold rules), polished by Claude.
• Levels 3 and 4, shape diff and resend plan: written by Claude at my request. Level 3 was a worked example to teach me recursion, which I'd stalled on, and level 4 was the last night before the deadline.

Two safety details I'm glad I didn't skip

• Server-side request forgery. Live mode lets you type the URL Holdfast forwards to. Without care, anyone could point it at holdfast-ollama:11434 or a cloud metadata address and read the answers on the event page. Destinations are checked when saved and every connection is checked again at dial time, so a hostname that later resolves to a private IP is still refused.
• Duplicates. Providers deliver at least once. The resend plan sends each event once, in arrival order, never one that was already delivered, and never one whose signature failed.

Why Does Open Innovation Matter?

• The data is the business's customers. Payment webhooks carry names, amounts and addresses. With an open-weight model on the same private network, none of it goes to an AI company, and the freelancer can tell their client that truthfully.
• No per-message bill. A small shop's integration might send hundreds of events a day. One small model on one server explains every failure for a flat cost.
• I could look inside the failure. When Gemma promised a review that wasn't happening, I could see exactly what it was given, change what code tells it, and constrain its output with a JSON schema. The fix was engineering, not hoping a vendor's model changes.
• Swappable. The model is one setting (HF_MODEL). I measured 1B against 4B and chose on evidence.

My Agent Session

Built pair-programming with Claude Code: spec first (brief, requirements, design, tasks, each approved by me), then implementation task by task, with subagents for coaching me on my levels, writing specs, and drafting this post. The coach was told never to write my code, only give hints and review.

Building Holdfast: a webhook safety net with Gemma

You

Status is approved. Set up the rest today: the frontend website, and leave certain things as "WIP: work on this file to reveal how the website will look", something like a game for me, where every file I code, I unlock a bit of the website. Obviously the coding bits are still to be done by me.

Agent

Plan: the four core pieces (signature checks, retry policy, JSON shape diff, resend plan) become documented stubs that return wip.ErrLocked. Each is a 'level' with known-answer checks in internal/unlock. The relay falls back to simpler behaviour while a level is locked, and every part of the site a locked level powers renders a WIP tile naming the file to write. go run ./cmd/unlock prints the board.

▸ Write internal/verify/verify.go (stub + contract + GitHub's documented known answer)

Input

internal/verify/verify.go (stub + contract + GitHub's documented known answer)

Output

File created

▸ Bash go run ./cmd/unlock

Input

$ go run ./cmd/unlock

Output

HOLDFAST · UNLOCK BOARD  0 of 4 levels
[··········]  Level 1  Signature check  LOCKED
  next → open internal/verify/verify.go (H1)

Then the engine: SQLite store that commits before the relay answers 200, a delivery worker with a one-try fallback, rule-based diagnosis plus Gemma via Ollama under a JSON schema, a client note, and the website with a live demo shop (Chai & Co.) that visitors can break on purpose.

▸ Bash fire every demo scenario at the local server, read the dashboard

Input

$ fire every demo scenario at the local server, read the dashboard

Output

3 payments held safely … What happened: the app's server was down.

You

So shouldn't we first have some kind of struct present that will be needed when unmarshalling?

Agent

No struct: Of has to accept any JSON. Unmarshal into any, then a type switch over map[string]any, []any, string, float64, bool and nil tells you what each node is. The rest is a small recursive walk that records the kind at each path.

You

Just write it for me so I can understand what you're trying to tell me. I will do the next one on my own.

Agent

▸ Edit internal/shape/shape.go: walk(s, path, v) records the kind, then recurses int...

Input

internal/shape/shape.go: walk(s, path, v) records the kind, then recurses into object keys (path.key) and array items (path[]); record() marks Mixed when array items disagree; Diff compares two shapes and sorts by path

Output

Level 3 UNLOCKED (7/7 checks)

It is the folder-listing idea: write down what this thing is, and if it contains other things, call walk again for each with a longer path. tasks.md records that I wrote this level at your request.

You

Ok just check and lmk if Function 1 is ok?

Agent

▸ Bash go run ./cmd/unlock -v

Input

$ go run ./cmd/unlock -v

Output

Level 1  Signature check  4/10 checks
✓ GitHub: accepts GitHub's own documented example
✓ GitHub: rejects a body changed after signing
✓ GitHub: reports a missing header
✓ GitHub: rejects a header without sha256=

GitHub is correct, and decoding the hex and comparing raw bytes with hmac.Equal is better than comparing hex strings. The remaining six checks are Stripe's: same seal, but over "t.body", any v1 may match, then a freshness check.

You

Gemma only says: "A payment notification failed to reach the Chai & Co. system. The notification is being kept safe and reviewed." Is this temporary? Because come on it should say more than that.

Agent

▸ Bash read the held event's page

Input

$ read the held event's page

Output

For your developer: The app is now requiring a 'customer_id' field, which was previously absent…  (backwards: the sender changed, not the app)

Not temporary: the prompt only asked for 'two short sentences, no jargon', nothing checked for promises ('reviewed'), and Gemma was handed raw paths and guessed the direction. Fix by moving work into code: detect renames (customer_id → customer.id) and tell the model the sender changed; require what happened / why / what next; a Promise check; and the developer fix must name the moved field or the rules' fix is used.

▸ Bash renamed-field demo again on gemma3:4b

Input

$ renamed-field demo again on gemma3:4b

Output

Nisha's ₹1050 payment … couldn't reach Chai & Co. because the sender changed how they send customer information. Holdfast is keeping the message safe and it can be sent again once the system is updated.
For your developer: The sender now uses data.object.customer.id instead of data.object.customer_id.

▸ Bash same demo on gemma3:1b

Input

$ same demo on gemma3:1b

Output

"The app will resend the held events." / "Ensure the sender updates the customer ID field."  → guards now replace both

You

I want to go 4b since the good answers is what we want. How will we host it on Render though? I want to mimic my Week 1 setup.

Agent

Same shape as Week 1: a public web service plus a private Ollama service with the model baked into its image, now on 2c-4g because gemma3:4b needs about 3.5 GB loaded. Measured on a CPU with 2 threads: 740 prompt tokens in 15 s, 115 output tokens in 14 s, so the page shows the rules' answer at once and swaps in Gemma's when it arrives.

▸ Edit relay: save the rules' diagnosis first (pending), replace with the model's; r...

Input

relay: save the rules' diagnosis first (pending), replace with the model's; resume pending ones after a restart

Output

tests pass

You

Can you do Level 4 for me? And then overhaul the website since it's no longer a game. Can we have two modes, Demo and Live? Live will be for users to connect GitHub. I don't have a Stripe account, so we will be honest about it in our submission.

Agent

▸ Edit internal/replay/replay.go: DedupeKey (GitHub delivery id, Stripe event id, el...

Input

internal/replay/replay.go: DedupeKey (GitHub delivery id, Stripe event id, else sha256 of body); Plan sorts a copy by arrival, sends each key once, never a forged or already-delivered event

Output

Every level unlocked. Holdfast is ready to ship.

Live mode: GitHub OAuth (admin:repo_hook only), Holdfast adds the webhook to the chosen repo, delivery to the user's app or a built-in test app that can be made to fail. Routes become dynamic; user-typed destinations go through a guard that refuses private addresses at save time and again at dial time, so nobody can point Holdfast at its own Ollama service. Tokens and webhook secrets are sealed with AES-GCM. Stripe connects by hand, and the site says so.

▸ Bash go test ./internal/netguard/

Input

$ go test ./internal/netguard/

Output

ok  (refuses 127.0.0.1, 10.x, 169.254.169.254, holdfast-ollama:11434; dial-time check refuses loopback)

You

The GitHub login looks a bit weird. Also there's too much text on the demo and live page. We want it attractive to customers so they will pay.

Agent

The page in the screenshot was rendering with almost no CSS: the stylesheet was linked as app.css?v=dev, so the browser kept an old cached copy. Assets are now linked with a hash of their contents. Then the copy: Live becomes a headline, one line, a dark 'Continue with GitHub' button, three chips and three step cards; Demo becomes 'Break the shop. Lose nothing.' with short button labels.

You

For the how it works section, can we add an animation rather than textboxes? Makes it easier for people to understand.

Agent

A 15-second SVG/CSS loop: a payment travels Stripe → Holdfast → the shop, is saved, checked, finds the shop down, is held, explained, then sent again while a duplicate is crossed out; five captions light up in step. Headless screenshots showed it frozen until I added --disable-threaded-animation: transform and opacity animations run on the compositor, which virtual time doesn't drive.

You

Deployed. GitHub seems to be working fine.

Agent

▸ Bash render logs (holdfast)

Input

$ render logs (holdfast)

Output

WARN model unavailable; used rules err="ollama: Post \"http://holdfast-ollama:10000/api/chat\": dial tcp …:10000: connect: connection refused"

▸ Bash render logs (holdfast-ollama)

Input

$ render logs (holdfast-ollama)

Output

ollama ready with gemma3:4b
Detected service running on port 11434

Ollama is healthy on 11434, but the Blueprint's fromService hostport used Render's default port 10000. The rules answered every held event, so the site stayed usable. Fix: PORT=11434 on the private service.

▸ Bash after the redeploy: send a renamed-field payment to the live site

Input

$ after the redeploy: send a renamed-field payment to the live site

Output

after 92s: Kabir's ₹1050 payment for Ginger chai, 500 g couldn't reach Chai & Co. because the sender changed how it sends customer information…
Explained by Gemma, checked by code.

20 of 20 messages

My First User

My first user is a real app with a real repository: Tasklet, a small todo app I built (source) that turns GitHub issues into todos. It's strict on purpose, like a real handler, and it has a maintenance switch that makes it answer 503. I signed in to Holdfast's Live mode with GitHub, connected Tanay3484/tasklet, and pointed it at Tasklet's webhook address. My coding agent ran the rest from the command line while I was out, and I checked the results when I got back.

Step	What happened
Connect the repo	Holdfast added the webhook. GitHub's test ping was acknowledged in 0.27 s.
Open issue #1	GitHub → Holdfast acknowledged in 0.82 s. The todo appeared in Tasklet within 4 s.
Tasklet in maintenance, open issue #2	Holdfast tried 5 times in 34 s (waiting 2, 4, 8, 16 s), got 503 each time, and held it. Gemma explained it in 40 s.
Close issue #1	The todo was ticked off within 4 s.
Maintenance off, then "Send all held again"	I pressed it in Live mode. Issue #2's todo appeared in Tasklet, once. Nothing was lost while the app was down.

One detail I'll own: Tasklet runs on Render's free plan, which doesn't keep a disk, so its todos reset whenever it falls asleep. That's why issue #1's todo had disappeared by the time I resent #2. Holdfast's copy of every message was still there.

Did anyone pay? No. My first user is me and my own app, so I won't claim a sale. What the run convinced me of is the thing I'd charge for: I switched an app off, opened an issue, and nothing was lost, and I got a sentence explaining why it was held that I could forward to a client. What I'd build next: an email or Slack alert the moment something is held, one-click Stripe sign-in, and a share link for the client note.

Something real from deploy day: on the first deploy, every Gemma call failed with connection refused on holdfast-ollama:10000. Ollama was healthy on port 11434, but Render's service-to-service address used its default port. The rules took over automatically, so the site stayed fully usable, which is exactly the fallback I designed for. One line in render.yaml (PORT=11434) fixed it.

Then two more, the same afternoon:

• Out of memory. Render killed the Gemma service. Inside the container, llama.cpp sees the whole host (60+ GB, 16 CPUs), not my 4 GB plan, so it kept a prompt cache that could grow to 8 GB and a second, repacked copy of the weights. Three LLAMA_ARG_* settings (no prompt cache, no repack, 2 threads) brought it back under 4 GB.
• A runaway answer. Some diagnoses ran on for hundreds of tokens and never closed their JSON. I reproduced it on my laptop's CPU with Render's exact settings: in 3 of 5 runs Gemma tried to end a string with a curly quote (”}), which JSON treats as ordinary text, so it never stopped and started repeating itself. A second cause: Go wrote the JSON format's fields alphabetically (cause, fix, summary), so Gemma poured the whole story into the developer's fix and left the owner one line. The fix: length limits on each field, code that cuts a field at the stray ”}, a check that the summary ends like a sentence, and the fields listed as cause, summary, fix. Before: 2 of 5 runs usable. After: 4 of 4, in 16–40 s.

Each time, the visitor still got a correct, plain-language answer from the rules. Gemma makes the explanation better; it was never the thing keeping a payment safe.

What isn't built: one-click Stripe sign-in. Stripe Connect needs a Stripe platform account, which I don't have, so Stripe connects by hand. You paste one address into Stripe's webhook settings and its signing secret into Holdfast. The site says so plainly.

Prize Categories

• Best Use of Render. Render is the AI runtime: one Blueprint runs the public Go service (with a persistent disk for the event store) and a private Ollama service with Gemma baked into the image, talking over Render's private network. The model is never exposed to the internet.
• Best Use of Gemma. Gemma 3 4B writes every explanation and client note. Its output is constrained to a schema and checked by code, and I measured it against 1B to choose the size.