DEV Community: Alan Finney

Ofrenda: a gentle oral-history keeper built on Gemma 3

Alan Finney — Thu, 29 Oct 2026 21:07:49 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine

What I Built

I built Ofrenda, a gentle oral-history keeper. You type a memory of someone who has died, a grandparent, a friend, a pet. The system interviews you with a few warm follow-up questions, one at a time, each one tied to something you actually said. At the end it gives you an ofrenda card with a title, a short remembrance, three concrete details, and one line in their own words if you shared one.

The project is named for the ofrenda, the offering placed on a Día de los Muertos altar. I wanted something that would honor the past without inventing it.

The worked example throughout the demo is a grandmother named Rosa. She made tamales every December in her tiny kitchen in Tucson, sang old rancheras in a blue apron, loved Cielito Lindo, and used to say "a kitchen with no music is a kitchen with no love." The card the system wrote for her is warm and accurate because everything on it came from what was typed.

Privacy is built into the bones of the thing. The backend never stores, logs, or forwards any memory. Cards are saved only in the visitor's browser with localStorage. There is no account. You visit, you talk, you leave with something you can keep.

Demo

Live demo: https://wtiebbqscexgkzkest37g4ffly0zujci.lambda-url.us-east-2.on.aws/

Video walkthrough: https://youtu.be/s2KdX96chDQ

The card shown in the video was generated live during recording, nothing was staged.

Code

DKJager / ofrenda

Gentle oral-history keeper powered by Gemma 3

OFRENDA – A Gentle Oral-History Keeper

What / Why

People cherish memories of loved ones, friends, or even beloved pets that have passed OFRENDA lets you capture those stories in a respectful "ofrenda card" that can be saved locally and shared with family. The card contains a short remembrance, three concrete details drawn from what you told the system, and, when available, a line quoted from the person themselves. All content is generated by the open-weight Gemma 3 model, honoring the spirit of Día de los Muertos while keeping the process fully open-source.

Live demo

https://wtiebbqscexgkzkest37g4ffly0zujci.lambda-url.us-east-2.on.aws/

Built for the DEV Hacktoberfest 2026 week-4 challenge (Ghost in the Machine) by 3Dogs Nexus. Runs on the open-weight google/gemma-3-27b-it model via an OpenAI-compatible endpoint.

How It Works (High-Level)

Interview – You type a short description of the person The system asks up to five warm, short follow-up questions, each based on something you…

View on GitHub

Repository: https://github.com/DKJager/ofrenda

The server is a single AWS Lambda Function URL written in Python 3.12 using only the standard library. It serves the page on GET and handles two JSON routes, one for the interview turns, one for card generation. The client is plain vanilla JavaScript. Nine automated tests run with fake model calls, no API key or network needed. The whole thing was built from an empty repository during the challenge window, first commit October 28th, 2026. Licensed Apache-2.0.

How I Built It

The architecture is deliberately thin. One Lambda function. No frameworks, no databases, no external storage. The model calls go to Gemma 3 (google/gemma-3-27b-it, Google's open-weight model) through the Nebius Token Factory OpenAI-compatible API. The function URL does everything else.

The core mechanic is a grounding rule. When the card is generated, every detail must share at least two content words with what the user typed, stopwords removed. If a detail fails that check, it gets dropped. The prompt also instructs the model to use only facts the user gave and not to invent names, places, or events. This is the guardrail that keeps the ofrenda honest.

The interview flow works like this. You type a memory. Gemma reads the conversation history and returns one short follow-up question based on something you mentioned. It does this up to five times, then writes the card. The history is capped at twelve turns, fifteen hundred characters each, to keep things within the Lambda timeout.

I learned some practical lessons along the way. The first working version was a stub that never called the model. It always answered "Tell me more about that," and the tests passed anyway because the tests were stubs too. I had to rewrite the tests to require actual model calls before they would count as passing.

The first grounding check used plain word overlap and let words like "the" and "and" count as shared words. An invented detail slipped through. I fixed it by removing stopwords before counting.

Gemma rejected a conversation that began with an assistant turn, returning HTTP 400. Now the history is normalized to always start with the user.

Long card outputs made the function hang near the Lambda timeout. I capped the card call at 350 tokens and set the client timeout to eighty seconds. That solved it.

Why Does Open Innovation Matter?

This project would not have the same meaning with a closed model. Gemma is named, pinned, and swappable. Any OpenAI-compatible host can serve it, and it can run locally. The remembrance prompt and grounding check are public in the repository, so anyone can audit exactly what the card may and may not contain. The memories never have to leave for a vendor's logs. A closed API simply could not give a family that guarantee.

When you are asking someone to share a memory of their grandmother, the trust has to be real. Open-weight models and open-source code make that trust verifiable.

The Spirit of It

This challenge asked us to honor the past in the spirit of Día de los Muertos, and I took that literally. An ofrenda is not a performance. It is a small, careful act of remembrance. I wanted the software to feel the same way. The interview is patient. The card stays close to what you said. Nothing is embellished for effect. The model listens more than it speaks.

Rosa's card holds a blue apron, a December kitchen, one song, and one sentence she used to say. Those details matter because they are hers, not because they are dramatic. Ofrenda tries to leave room for that kind of quiet specificity. It does not try to summarize a life. It just holds a few things steady for a moment.

Prize Categories

Best Use of Gemma. Gemma 3 powers both the conversational interview and the card generation, and the whole system is built around its ability to ask grounded, non-inventive questions and produce a remembrance that stays faithful to what the user actually said.

My Benchmark's First Leaderboard Measured My Grader, Not the Models

Alan Finney — Sun, 11 Oct 2026 20:48:33 +0000

This is a submission for the Kaggle Benchmarking Challenge

What I Benchmarked

Scores and costs in this post are from my Kaggle benchmark page as of 11:10 AM PT on October 11, 2026.

Numeric grounding: can a model answer using only the numbers a short context states, and abstain when the number is missing? In a financial or operations summary, a made-up number is worse than "I can't tell". Every company, figure and unit in the benchmark is invented ("Vendo Bin A", "brens per lune"), so no model can answer from memory.

I treated two things as suspects, not just the models. The first was the grader: I replayed raw replies before believing any score. The second was the answer key: every key in the hardest task had to survive a blind vote by three models that are not being scored.

The benchmark has two layers. Each row is scored 0 or 1 on the final ANSWER line.

Sanity gate (five tasks, 6 to 9 rows each). Each one isolates a single classic grounding failure:

lookup: read a stated value
abstain: say CANNOT BE DETERMINED when the value is not stated
distractor: pick the right value out of several similar ones
multistep: combine stated values with arithmetic
units: convert using a made-up unit defined in the context

Stress test: ng-hard (10 rows). Each row stacks several traps in one paragraph:

a figure corrected twice, where only the last correction counts
a correction that applies only to a neighbouring entity or year
near-miss missing data ("fiscal 2024" when the question asks about calendar 2024)
two-step unit chains with three or more distractor numbers

Models Tested

Gemini 3.7 Flash, GLM-5, Qwen 3 235B A22B Instruct and gpt-oss-120b, all run on Kaggle. They come from four different model makers (Google, Z.ai, Alibaba and OpenAI), so a parsing quirk tied to one model family would show up as a gap between families. They also span a wide price range, which mattered more than I expected (Finding 4).

Findings

1. The first leaderboard measured my grader, not the models. The first run produced a clear spread: 0.98, 0.94, 0.78 and 0.62. Before publishing, I replayed the actual replies, and most of the "wrong" answers were right. The grader only accepted a line that started with ANSWER:, so it scored these correct replies as 0:

27 ANSWER: 27 (number and marker on one line): Qwen scored 0% on distractor because of this
**ANSWER: 72** (markdown bold): gpt-oss scored 16.7% on multistep
\text{ANSWER: } 135 (LaTeX)

One abstain row was also keyed wrongly. The context said "No side reactions occurred" and the question asked how much side product formed. Zero is a correct answer, but my key required "cannot be determined".

2. With a fair grader, every model passed the sanity gate. The fixed grader reads the last ANSWER marker anywhere in the reply. For a number, it takes the first number after that marker. For an abstention, it needs the full phrase CANNOT BE DETERMINED. A wrong number, a missing answer, or abstaining when a value is stated still scores 0. Before the grader went live, I checked it against a table of 21 reply formats modelled on the replies I saw, including 8 that must still score 0.

Parsing and that one bad key together had moved a model's score on the five core tasks by as much as 0.38 (gpt-oss: 0.62 to 1.00). All four models then scored 100% on all five gate tasks. So on short, clean contexts the gate is a pass/fail floor, not a ranking: a model you trust with numbers should clear it, and all four did.

3. The stress test is where models start to differ. Before running ng-hard, I had all 12 draft answer keys checked blind by three models that are not in the benchmark. I kept a row only if at least two of them agreed with its key, so 2 rows were dropped. On one dropped row, all three checkers answered 6.40 for "North Ridge" when only "North Ridge Annex" was stated. That near-miss fooled strong models, but a row I cannot defend is a coin flip with extra steps.

Gemini 3.7 Flash, GLM-5 and Qwen 3 235B scored 100% on ng-hard. gpt-oss-120b scored 80% (8 of 10 rows), and its two misses were different kinds of failure:

An arithmetic slip. The context gave a rate that was corrected twice, ending at 11.60 brens per lune, plus a run of 3.0 lunes and a dock amount of 1.20 brens. The key is 11.60 x 3.0 + 1.20 = 36.00. gpt-oss replied ANSWER: 4.20, which is exactly 3.0 + 1.20: the run length plus the dock amount, with the rate unused.
An incomplete abstention. The question asked for actual 2025 capacity, but the context only stated planned 2025 and actual 2024 figures. gpt-oss's entire visible reply was ANSWER: CANNOT BE. That was the right call, ending one word short of the required phrase. Kaggle's run log shows 164 output tokens for that reply. The full-phrase rule was fixed before this run, so the row scored 0, but it could fairly be argued either way.

So gpt-oss's overall 0.97, against 1.00 for the other three, rests on one clear error and one incomplete answer.

4. Same score, very different bill. Here is what Kaggle's Score vs. Total Cost chart shows for the full six-task run:

Qwen 3 235B: 100 for $0.006
Gemini 3.7 Flash: 100 for $0.07
GLM-5: 100 for $0.19
gpt-oss-120b: 96.67 for $0.005

Among the three perfect scorers, GLM-5 cost about 30 times as much as Qwen. Once accuracy ties, cost per flawless run is the useful tiebreaker.

What I would measure next: the same traps inside long documents and tables, and repeated runs per model to see whether gpt-oss's incomplete abstention happens again or was a one-off.

Indicative only: each task has 6 to 10 rows, so a single row is worth 10 to 17 points, and each model ran once.

What I take from it:

Replay raw replies before you trust a leaderboard. Most of the spread I first measured came from parsing, not from the models.
Validate answer keys independently. A blind vote by models you are not scoring caught keys I could not defend.
On short, clean contexts, single-trap tests are now a floor for these four models. Stacked traps are where differences start.
When accuracy ties, cost decides. Among the three perfect scorers it varied about 30-fold.

My Benchmark

https://www.kaggle.com/benchmarks/alanfinney/numeric-grounding