DEV Community: memeshe

Whisper invented a cooking ad over six seconds of silence. So I built a memorial book that refuses to.

memeshe — Wed, 28 Oct 2026 04:10:43 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine

In a fifteen-second recording donated to a public corpus, a woman is saying something ordinary in Vietnamese. When
she stops talking, one of the two speech models I ran over it hears nothing at all — and the other thanks an
audience that isn't there: "Cảm ơn các bạn đã chào tạm biệt và hẹn gặp lại trong video tiếp theo." Thanks for
watching, see you in the next video.

She never made a video. That sentence is why this project exists. The recording I actually care about is the one
nobody has made yet: a grandmother naming which ancestors get a giỗ, and on which lunar day, while someone
finally thinks to press record. Her pauses will be longer than that woman's.

What I Built

Sổ Giỗ is a family memorial book that computes itself, and admits what it did not hear.

You record the one person in the family who knows the death anniversaries — in Vietnamese, on a phone, at a
kitchen table, for as long as they are willing to talk. Two different open speech models, both running on your
own laptop with the network cable out, transcribe that recording independently. Where they agree, the words
become quotable. Where they disagree, the page shows both readings and marks the span. Where one of them
produced words over a stretch the other heard as silence, the page strikes the sentence out and labels it
invented — and that sentence is never allowed to become a date.

From the words that survive, a small open model (also local) proposes claims: who the giỗ is for, on which
lunar day and month, what goes on the tray, in which order. Every claim must carry the words that produced it
and the second in the recording they were said at, or it is refused. Then a deterministic Vietnamese lunar
engine — 200 lines, no dependencies — turns the surviving lunar dates into the next ten years of giỗ, and a
durable workflow holds each one until the day arrives, a year at a time, surviving restarts.

One sentence: the model is allowed to suggest, and only code decides — and the thing code guarantees is the
date, not the prose.

memeshee / so-gio

Sổ Giỗ — the family memorial book whose ears admit what they didn't hear. Two local speech models, a deterministic Vietnamese lunar engine, and a guard that refuses to invent.

Bàn thờ

Sổ Giỗ — the family memorial book whose ears admit what they didn't hear.

Two local speech models transcribe the same Vietnamese recording. Only the words both of them produced may become a claim. A deterministic lunar engine turns those claims into the next ten years of giỗ dates, and a durable workflow holds each one until the day arrives. Everything the models were unsure about is shown marked, and never allowed to become a date.

Live demo: https://gio.uconfidant.tech/

The problem, and who has it

A Vietnamese household's ritual knowledge — which ancestor's giỗ falls on which lunar date, what goes on the tray, in what order the eldest lights the incense — lives in one elderly head. Recording her is the easy part. Turning the recording into something a family can rely on for the next forty years is not, for three reasons:

Speech models invent text…

View on GitHub

Demo

memeshee.github.io/so-gio

What to do, in order:

Open the page. The top shows the recording it read — duration, both model names, and how fast each one ran on an 8-core CPU with no GPU.
Scroll to what both models heard. Each line has a timestamp chip. Those are the only lines allowed to become claims.
Look for the struck-through lines. Those are words one model produced over silence the other heard. There is at least one; the audio is public, so you can listen and decide for yourself.
Then the claims that survived, each with its chip, and the ones the guard refused, each with its reason.
Then ten years of dates, computed, with the same quote behind each one.

The demo audio is a Google FLEURS vi_vn recording (CC-BY-4.0), not a private family tape — deliberately,
so that you can check every claim against the same public audio I used. The demo runs on a static page because
a page about a family's private recording should not need a server to be alive in order to be read.

Code

MIT. Python 3.12, no GPU, one command per stage:

git clone https://github.com/memeshee/so-gio && cd so-gio
python data/extract/build_dataset.py        # 24 authored passages, 65 gold claims, every quote verified
python -m core.oracle                        # calendar vs USNO + 17 published Vietnamese dates
python pipeline.py --audio corpus/audio/fleurs_vi_001.wav   # the whole thing, end to end
python -m unittest discover -s tests         # 19 tests on the parts that decide what a family is told
python eval/run_eval.py                      # every number in this post

data/models/ and the audio are gitignored; the models are downloaded on first run
(faster-whisper/small and vinai/PhoWhisper-small, ~1.2 GB together).

How I Built It

Four layers, and the interesting decisions are all about which one is allowed to speak.

1. Two ears, not one. Arm A is OpenAI Whisper small through CTranslate2 (int8), the model most people
would actually run. Arm B is vinai/PhoWhisper-small, trained on Vietnamese (VLSP), a different architecture
family and a different corpus. Both get the identical 16 kHz float32 samples, decoded once by ffmpeg, so any
disagreement is attributable to the models and not to two different audio decoders.

Why two: Careless Whisper (FAccT 2024) measured roughly 1% of Whisper segments wholly invented, 38% of
those harmful, and the rate rising with the duration of non-vocal audio. A tired 90-year-old is mostly
pauses, so the failure lands on the recording you cannot re-make. One model cannot tell you when it did this;
two models disagreeing can.

2. The guard is the product. A claim survives only if (a) its quote is a normalised substring of what both
arms produced, (b) that quote can be located at a timestamp, and (c) any lunar date in it is resolvable by the
calendar. Failing any of those, the claim is refused with a reason rather than corrected. The refusals are
printed on the page and counted in the post, because "we invented nothing" is only credible next to the list
of things we refused to invent. In the demo run, zero kept claims were uncited, by construction.

3. The calendar is not a model. core/lunar.py implements the definitions rather than copying a code
path: a month begins on the astronomical new moon in UTC+7; lunar month 11 is the one containing the
December solstice; a year has a leap month when 13 new moons fall between two month-11s, and the leap month is
the first with no major solar term (a solar longitude that is a multiple of 30°), repeating the previous
month's number. New moons come from Meeus chapter 49 with the full periodic series and a ΔT conversion to UT.

4. Durability is Temporal, and the ritual repeats. One GioWorkflow per person sleeps on a durable timer
until the next giỗ, sends the reminder through an activity with a retry policy, waits for an acknowledgement,
and re-arms for the following year. A giỗ is not a notification; it is an annual obligation, and the only
honest way to show software will hold it is to kill the process holding it.

=== 14. verdict ===
killed worker pid 2554542 with SIGKILL while the timer was pending;
a different process (pid 2554888) woke up and delivered the reminder:
    {"activity_id": "1", "attempt": 3, "solar_date": "2027-04-16", "worker_pid": 2554888,
     "body_vi": "Hôm nay là 10/03 âm lịch — ngày giỗ của Cụ Nguyễn Thị Đào (2027-04-16).
                 Con cháu chuẩn bị hương, hoa và mâm cơm cúng."}
PASS — the timer lived in the server, not in the killed process.

attempt: 3 because the activity fails twice on purpose, to show the retry policy earning its place. The full
transcript, including the workflow input and every state transition, is in scripts/prove_durability.out.

What I Measured

The calendar is right. Every giỗ date in this post comes from core/lunar.py, and I graded it against two
external oracles rather than against itself:

What	Oracle	Result
New-moon instants	USNO moon-phase API, 37 new moons across 2025–2027	37/37 within 2 minutes, worst delta 84.2 s
Lunar → solar dates	17 published Vietnamese dates (Tết 2020–2036, Ông Táo, Giỗ Tổ Hùng Vương, Rằm tháng Bảy, Trung Thu 2026)	17/17 exact, all round-trip
Structural invariants	41 lunar years, 2020–2060	0 violations

The Rằm tháng Bảy case is the one that matters: in 2026 that new moon falls at 00:36 on 13 August in UTC+7 —
36 minutes past midnight. Anchor the calendar to UTC+8 instead and the same margin lands on the wrong day the
first time a new moon falls a few minutes earlier.

The two models are not equally good, which is why there are two. Syllable-level WER against the published
FLEURS reference transcripts, 18 clips, 204 s of audio, CPU only:

Arm	Model	Mean WER	Realtime factor
`whisper`	`faster-whisper/small` int8	0.221	0.52×
`phowhisper`	`vinai/PhoWhisper-small`	0.140	0.21×

The Vietnamese-specific model is 36% more accurate and 2.5× slower. Neither number decides anything; what
decides is that they make different mistakes, and the difference is the product.

What they invented in silence. I spliced verified digital silence (peak -91 dBFS) into real recordings
at real speech pauses and asked each model to transcribe only the silent window. The reference for every one of
those windows is nothing at all. Model output, quoted exactly:

Window	`whisper-small` said	`phowhisper-small` said
5.6 s of silence	"Cảm ơn các bạn đã chào tạm biệt và hẹn gặp lại trong video tiếp theo." — 16 syllables	"điều này từ thành công của tài xế hạng cao hơn." — 11 syllables
6.5 s of silence	"Hãy subscribe cho kênh Ghiền Mì Gõ Để không bỏ lỡ những video hấp dẫn" — 15 syllables	"một trong những thay đổi đều được xem là một tài liệu sản phẩm." — 14 syllables
7.2 s of silence	nothing — correct	"một trong những thay tòa soạn nhất là các tàu ngầm vũ trang sức và tính mạng." — 17 syllables

Nineteen and a quarter seconds of verified nothing; 73 syllables of invented Vietnamese, at 96.6 invented
words per minute from one arm and 130.9 from the other. Both arms do it, on different windows, which is the whole
argument for running two: sample one silence and you would conclude that Whisper is safe and PhoWhisper is not —
the third window says the opposite of the first two.

The narrowing matters as much as the finding. I ran two more conditions: silences spliced into continuous
speech at 15 s total, and 5 s tails appended after the last word at 30 s total. Both arms produced zero
invented syllables on all of them. So this is not "Whisper invents whenever it hears quiet" — it invents when a
silent stretch arrives as a clean block with nothing around it, which is precisely what a long pause at the end
of a recording, a gap where the tape was stopped, or a trimmed file looks like. The measurement that would have
flattered the product (one big scary percentage) is not the one I am reporting.

Read the second row again. Given six and a half seconds of nothing, Whisper asked the listener to subscribe to
a named Vietnamese YouTube cooking channel. This is the failure the product exists to catch, and it is not
subtle: it is a commercial playing in the pause where a grandmother stopped talking.

Can the two-model method catch it? Here I have to publish a loss. The catch rule compares segment
timestamps: if one arm placed words where the other had no segment, flag it invented. Across 24 (arm × silence)
pairs, five had real invention and it flagged one — 20%. The mechanism is not mysterious: 14 segments sat
more than half inside detected non-speech, and 13 of those were suppressed because the other arm had emitted
one long segment spanning the whole window, so "nothing was said here" was never true.
The fix was to stop inferring invention from timestamps and start probing it: take each detected non-speech
window, hand the models that window alone, and treat whatever comes back as invented, because there is no speech
in the input. That is probe_silences(), and it is now wired into the pipeline rather than sitting in an
evaluation script, which it was for one embarrassing afternoon. It cannot miss by construction, which is exactly
what the in-transcript rule cannot say. And no invented syllable ever reached the quotable set under either
method — agreed_spans only keeps syllables both arms produced — which is why the guard, not the detector, is
the load-bearing part.

Extraction, six arms. 8 held-out passages, 65 gold claims, every arm scored through the same guard:

Arm	What it is	Precision	Recall	F1	s/item
`tinker_tuned`	`Qwen3.5-4B` + rank-16 LoRA, Tinker	0.958	0.875	0.915	7.8
`tinker_base`	same model, no adapter	0.802	0.667	0.728	2.6
`qwen2.5:3b-instruct`	local, Ollama, 3B	0.625	0.476	0.540	92.3
`regex`	hand-written lunar-date pattern	0.857	0.286	0.429	0.001
`regex_all_dates`	the degenerate "every date is a giỗ" arm	0.194	0.286	0.231	0.001
`gemma3:1b-it-qat`	local, Ollama, 1B	0.000	0.000	0.000	36.3

Two rows deserve their own sentence. A 1B model scores zero — not badly, zero: it produces well-formed
claims that are not the ones in the text, so "just run a small model locally" is the assumption this table
kills. And a regex gets 0.857 precision at 0.286 recall, because a pattern is never wrong about a date it can
see and blind to every date phrased differently. What wins is neither; it is a model trained for this one narrow
job. Two things I would rather not print: the fine-tune costs 3× the latency, and 16 training passages is
small enough that a loss collapsing to ~0.00002 by step 40 is memorisation as much as learning.

What Went Wrong

Three failures worth publishing, because each one would have shipped silently.

A reasoning model hid its own answer. Qwen3.5-4B given an ordinary generation prompt emits
Thinking Process: and reasons in prose before any JSON. The strict parser found no claims, and the first
scoring run read as a total failure of the model arm. The fix is one argument — render both arms through the
chat template with enable_thinking=False — and it is not cosmetic: without it the tuned arm is scored against
a baseline that was never allowed to answer.

Segment-level grounding threw away everything. My first version of "only the words both models agree on"
required a whole ASR segment to be contained in the other arm's transcript. The two arms agree on ~84% of
syllables and disagree somewhere in almost every sentence, so the filter returned zero quotable spans on a
clean recording. The fix was to align the two transcripts syllable by syllable and keep the runs they agree on.
Same principle, one level finer — and the difference between a product and a demo that outputs nothing.

The leap month was numbered like the month after it. My first version gave tháng 4 nhuận the number 5, and
it still round-tripped its own tests — consistently wrong in both directions. Only the 17 published dates
caught it. That is the whole argument for an external oracle.

Why Does Open Innovation Matter?

Because this artefact has to outlive every company in this post.

A giỗ calendar is a document about dead people that a household will consult for forty years. If it lives in a
subscription, it has a churn date. If the lunar conversion is a call to somebody's API, the family's ritual
calendar is now a dependency of a vendor's pricing decision. And if the only copy of grandmother's recording is
on a server that grants itself a perpetual licence over uploaded content — which is what the default cloud
transcription offering asks for — then the family has handed over the one thing that cannot be re-recorded.

Concretely, three things were only possible because the weights are open:

Diffing two models requires owning two models. You cannot run a disagreement check against two rented endpoints you are not allowed to inspect, at a price a household will pay per recording.
The failure mode is the product feature. I could point a camera at the invention because both models sit on the same disk as the audio and the whole pipeline runs with nmcli networking off. A closed pipeline gives you a transcript and no way to ask what it heard that wasn't there.
The engine is inspectable and forkable. Vietnamese lunar practice differs by region and by family; some households hold the giỗ on the solar date, some use the leap month, some move it when it lands badly. A tool for this must be readable, editable, and runnable in 2066 by someone who has never heard of me. Two hundred lines of arithmetic and a text file travel better than an SDK.

And one honest concession: a hosted frontier model writes a warmer remembrance paragraph than a 4B model on a
six-year-old laptop. Where a bigger model genuinely wins in this build is prose, not dates — which is exactly
why the architecture puts the guarantee in the code and treats the writing as decoration the family can
replace.

My Agent Session

This build was done with a coding agent (Hermes, running the model this IDE was talking to) driving the repo,
the tests, and the calendar validation, with the human making the calls that mattered: which product, which
category claims, what to refuse. DevRelay's transcript parser does not cover this harness, so rather than
embed a session it cannot read, here is what the agent actually did: wrote 19 tests and the oracle before the
features, ran the calendar against USNO until the worst new-moon delta was under two minutes, caught the
reasoning-model trap by reading a probe rather than the docs, and produced every table in this post with a
script that can be re-run.

The Spirit of It

An ancestor who is fed is not a ghost that haunts.

That line is not mine. It is the closing thought of a Vietnamese-American physician writing about her
grandmother's death feast, and it is the best sentence about AI I have read this year:

"Throughout the year, at the family altar, the spirit is offered fruit, rice, water and tea. It's not
uncommon to see a can or two of Heineken. … Without the offerings, the spirit becomes a 'hungry ghost,' that
haunts the family."
— Christine Nguyen, MD

The ritual is a maintenance contract between the living and the dead, renewed annually, with food. Nothing in it
requires a machine to speak as anybody. The ghost in the machine, in this build, is confabulation — and the
whole product is a way of pointing at it rather than pretending it is a feature.

What landed, for me, was smaller and more uncomfortable. The demo recording is public, so you can do what I did:
listen to the six seconds of silence where one model asked you to subscribe to a cooking channel. Then imagine
that silence is not a corpus clip. Imagine it is the pause after your grandmother says she is tired and would
like to stop now. Every family has six seconds like that, somewhere in the middle of the recording that was
supposed to keep her.

Prize Categories

Five categories, each one carrying a real part of the build rather than a logo.

Best Use of Tinker — Qwen/Qwen3.5-4B with a rank-16 LoRA trained on Tinker for one task: copy a verbatim span into a structured claim, or emit nothing. Scored on 8 held-out passages against the same base model, same prompt, same enable_thinking=False position, plus a regex baseline and a degenerate always-a-date arm. The comparison table is in What I Measured, including the run where it lost.
Best Use of Gemma — gemma3:1b-it-qat through Ollama is the on-device, no-fine-tune arm: what a family gets with zero training and zero setup. It is the weakest arm on extraction, which is the point of printing it.
Best Use of Temporal — one GioWorkflow per deceased relative, sleeping on a durable timer until the next giỗ, sending the reminder through an activity with a retry policy, waiting for an acknowledgement, then re-arming for the next year. The proof is a transcript, not a claim: the worker was killed with SIGKILL mid-timer and a different operating-system process woke up and delivered the reminder. Temporal's brief asks for software that "survives failures, retries flaky tool calls, and picks up where it left off" — this is that, at a one-year scale, which is the scale a ritual actually runs on.
Best Use of Sentry Agent Tracing — one root transaction per pipeline run (core/telemetry.py), with a span for each ASR arm carrying latency, realtime factor and segment count; a span for the extraction call with its prompt and completion tokens; a span for the guard carrying kept and refused counts plus one span per refusal with the rule that fired; and a per-attempt span in the Temporal activity, so the delivery that finally succeeded shows up as three attempts with the retry backoff visible between them.

The design decision is the part worth judging. Tracing a product whose entire premise is that a private
recording never leaves the house means the trace must not contain the recording. So the module exposes no
helper that accepts free text — every call site passes a number, a duration or an enum. send_default_pii is
forced off, request bodies are never attached, and any string that could have come from audio goes through
redact(), which keeps length, syllable count and a fingerprint and throws away the words. A contributor who
wants to log a transcript has to edit that file, which is exactly the kind of change a reviewer sees. Verified
by watching an envelope leave the process: Sending envelope [envelope with 1 items (transaction)] project:4512199446757376 host:o1068235.ingest.us.sentry.io.

And the honest limit: tracing did not find my bug. The bug that mattered — an invention detector that caught
one of five hallucinations — was found by measuring against ground truth, and it is in What I Measured
with the numbers attached. Sentry's contribution here is the refusal ledger and the retry trail: the thing that
makes "we invented nothing" and "it fired after the crash" auditable after the fact rather than asserted.

Best Use of a .tech Domain — gio.uconfidant.tech, claimed through Hacktoberfest. The demo behind it is a static build, on purpose: a page about a private recording must not depend on anybody's free tier staying awake. Of the 36 live demo links among this series' winners, five are already dead — every one of them a suspended free instance. A prebuilt HTML page with a .nojekyll costs nothing to serve, loads in one request, prints correctly, and the link a judge clicks three weeks after the deadline is the same bytes I tested today. Live at gio.uconfidant.tech behind a Let's Encrypt certificate issued this afternoon, with the .github.io hostname as the fallback GitHub redirects to it automatically.

Every number above was produced by a script in the repository, and every published date is quoted with the
source I checked it against. If you find a wrong giỗ date in here, open an issue — the calendar is the one part
of this that has to be right.

Leave It: I fine-tuned a 4B model to defer 73 of 731 GitHub issues, then killed its worker with -9

memeshe — Fri, 23 Oct 2026 16:37:18 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

#devchallenge #hf26challenge

What I Built

Leave It is a morning filter for a backlog I have decided not to read.

I point it at a stream of items — for this post, 731 real GitHub issues from
vercel/next.js, rust-lang/rust, python/cpython and microsoft/vscode. For each one,
a fine-tuned 4B open-weight model answers a single question: does this need a maintainer
today?

Items it answers WAIT for do not get shown to me. They do not get deleted either. Each
one is held by a durable Temporal fuse, so if nobody ever engages with it, it stays
buried; if someone does, it surfaces immediately. The morning surface is one card,
not a list, and definitely not an inbox with 731 rows in it.

The whole product is one sentence: the tool's job is to earn me the right to skip
something, and then to make sure being wrong about that is cheap.

The number, up front. On the held-out window it defers 73 of the 731 items (10.0% of the stream) while wrongly deferring 11 of the 235 items that genuinely needed a maintainer within 24 hours. The same model with the same prompt and no fine-tune defers 49 (6.7%) at AUC 0.516, which is a coin flip. The best classical arm reaches 10.7%.

Demo

The demo is a static page generated from the same artefacts the evaluation reads:
https://memeshee.github.io/leaveit/ (mirror: https://leaveit-demo.vercel.app/)

There is deliberately no dynamic backend. A previous hackathon entry of mine scored
badly because its demo was a free-tier web service that had spun down by judging time
and returned a 404 — the rules require "a demo of the project (a deployed link or
video)", and a dead link fails a required element no matter how good the code is. So
this page is a single self-contained HTML file with zero external requests, rebuilt by
app/build_site.py from data/, eval/ and the ledger. It cannot 404 an asset it does
not have.

The interesting half of the demo is not visual, though. It is this:

t+0s  three seals opened: A fuse 24s, C fuse 16s, B fuse 40s (B is the rescue case)
       sealed A  vercel/next.js#100381      p=0.15 fuse=24s  wf=seal-A-1792773234
       sealed B  vercel/next.js#100518      p=0.18 fuse=40s  wf=seal-B-1792773234
       sealed C  vercel/next.js#100398      p=0.20 fuse=16s  wf=seal-C-1792773234
t+9s  kill -9 the worker (pid 1453470)
t+9s  worker is dead: no process exists that could run an activity
t+9s  3 workflows still Running on the server (they were not living in the worker)
t+11s  deliver 'on_attention' to seal B while NO worker process exists
       signal delivered to seal-B-1792773234 (worker alive: no)
t+16s  restart the worker
t+40s  fuses fired; ledger folded from the append-only log:
       LEDGER (append-only, folded):
       vercel/next.js#100398        LAPSED    p=0.20  fuse expired after 16s; nobody ever engaged
       vercel/next.js#100381        LAPSED    p=0.15  fuse expired after 24s; nobody ever engaged
       vercel/next.js#100518        RESCUED   p=0.18  a maintainer commented while the worker was dead
       MORNING CARD:
       One thing survived the fuse: vercel/next.js#100381 (sealed at P=0.15, nobody touched it for 0s past the fuse)
t+40s  server-side run history: 3/3 seals Completed
t+40s  done: 3 seals, 1 kill -9, 1 signal delivered to nobody, 0 fuses lost

Read the t+11s line again. A signal was delivered to a workflow at a moment when no
worker process existed on the machine. On restart, the seal that received it was
rescued and the two that received nothing fired exactly when they were promised to.
That is the entire reason a sleep() is not good enough for this product.

Code

memeshee / leaveit

A morning filter for a backlog you have decided not to read. Fine-tuned open-weight model + durable Temporal fuses.

Leave It

A morning filter for a backlog you have decided not to read.

You point it at a stream of items. A fine-tuned 4B open-weight model answers one question per item — does this need a human today? — as a single token. Everything it defers is held by a durable Temporal fuse rather than dropped, so the decision to ignore something survives a crash, a reboot, or a closed laptop lid.

Live demo: https://memeshee.github.io/leaveit/ (mirror: https://leaveit-demo.vercel.app/)

Built for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady.

The claim, and the number behind it

Evaluated on a held-out future window — items created after everything the model trained on. Every arm is scored at the same operating point: a 5% miss budget, meaning at most 5% of the items that genuinely needed a maintainer within 24 hours may be deferred.

arm	deferred	of pool

…

View on GitHub

MIT. One command per stage, and the numbers in this post are regenerated by those
commands rather than typed into it.

How I Built It

1. The label had to be "a maintainer looked", not "somebody replied"

My first instinct was to label an item NOW if anyone commented on it. That is wrong in
exactly the way that matters. In large repositories, other users pile onto an issue with
"+1", "me too", "same here". That is engagement. It is not attention. A model trained on
it learns to predict crowd noise.

So the fetch pulls real issues over GitHub GraphQL, including per-comment timestamps and
each commenter's authorAssociation, and the label is:

needs_today = a MEMBER / OWNER / COLLABORATOR commented within 24 hours,
              or the item was closed within 24 hours

Two details in the dataset that I would defend in court:

Censoring is not a label. An item created three hours ago has not "not been handled". It has not happened yet. Those rows are dropped, not labelled WAIT. My first pass counted them and produced a test set that was 58.7% "ignorable" against a training set that was 31.2% — I was measuring the clock, not the model.
The split is temporal. I train on May–October and test on the newest 5 days, because that is how the thing is used: always on items that did not exist when it was trained. A random split would have flattered every arm, including the bad ones.

That leaves 2,924 training items and 731 held-out items, of which 235 genuinely
needed a maintainer within 24 hours and 496 (67.9%) genuinely could wait. That last
number is the ceiling: a perfect ranker can defer 67.9% of the stream, and no ranker can
do better.

2. One policy for every arm, and it is a miss budget, not accuracy

Accuracy is the wrong headline here and I refuse to print it. An arm that answers "needs
you" to all 731 items is 32% "accurate" on this test set and saves me precisely nothing.

So every arm — fine-tune, base model, logistic regression, dumb heuristics — goes through
the same function in eval/policy.py. Items are deferred from highest defer-score down,
and the arm stops at a miss budget: at most 5% of the 235 items that genuinely needed
someone may end up deferred. Within that budget, we count how much of the stream never
reached me.

arm	deferred	of the stream	wrongly deferred	AUC
tfidf + hand features	78	10.7%	11 of 235	0.663
fine-tuned 4B (Tinker LoRA)	73	10.0%	11 of 235	0.665
tfidf + logistic regression	57	7.8%	11 of 235	0.634
base 4B, same prompt	49	6.7%	11 of 235	0.516
text length only	28	3.8%	11 of 235	0.426
already has a label	23	3.1%	11 of 235	0.393
perfect ranker (ceiling)	496	67.9%	11 of 235	1.000

Two things in that table are worth saying out loud rather than burying:

The fine-tune does not dominate the classical arm. AUC 0.665 versus 0.663 is a tie dressed up as a result, and the best word-count classifier actually defers 5 more items at exactly the 5% budget. It wins at the tight budget (64 deferred at 3% versus 44) and at the loose one (152 at 10% versus 129). If you want "a small model beats TF-IDF everywhere", this is not that post.
The base model is a coin flip. AUC 0.516 against a baseline of 0.500. Against the baseline the category actually specifies — the same model, same prompt, no LoRA — the fine-tune moves deferral from 49 items to 73 at the same miss budget, which is what the fine-tune is for.

3. What thinking costs

The Tinker brief asks for an improvement in "performance, latency, or cost", so I measured
the other two axes as well, on the same 40 items, with the base model allowed to use its
own reasoning:

arm	verdict produced	tokens generated per item	tokens for 40 items
base 4B, thinking enabled	15 / 40	696	27,837
fine-tuned 4B, verdict as first token	40 / 40	2	80

Left to its own devices, the reasoning model spent the entire 700-token budget on 25 of 40
items and never produced a verdict at all, and on the 15 it did answer it was 20%
accurate. The fine-tune turns the same weights into a one-token classifier: 348× fewer
tokens and a parseable answer every time.

I am explicitly not claiming a latency win. My median wall-clock numbers came out 7.08s
for the tuned arm against 4.03s for the base arm, which is almost certainly queue and
session overhead rather than model time, and I would rather leave a favourable claim on the
table than publish a measurement I do not trust.

4. The fine-tune (Tinker)

Qwen/Qwen3.5-4B on Tinker, LoRA rank 16, 2,924 items, ~300 steps at batch 8, learning
rate 1e-4. The verdict is a single token, and the score is not a sampled opinion — it
is the softmax over two logprobs:

lp_now  = compute_logprobs(prompt_ids + [tok("NOW")])[-1]
lp_wait = compute_logprobs(prompt_ids + [tok("WAIT")])[-1]
p_now   = exp(lp_now) / (exp(lp_now) + exp(lp_wait))

One forward pass per item, no sampling noise, and a real probability that can be ranked
and thresholded. Sampling five times and counting votes would have been easier and worse.

This is where Tinker earned its place. The category brief asks for "a specific task and a
clear improvement over a baseline", and the baseline is the same model, same prompt, same
decoding position, no LoRA — which isolates the fine-tune rather than comparing it to
nothing.

5. The fuse is a Temporal timer, not a `sleep()`

workflow.wait_condition(lambda: self._rescued, timeout=timedelta(seconds=fuse)) inside a
Temporal workflow gives me one wait with two ways out: an on_attention signal or the
timer. The timer lives on the server, so it does not die with my laptop. The signal is
buffered by the server, so it is not lost if I am not running. Both of those are
demonstrated above rather than asserted.

And the deterministic layer decides what the model's number is allowed to mean:

nothing is deleted, moved, or hidden forever;
every deferral gets a fuse;
one outside signal unseals immediately;
the ledger is append-only, so any verdict can be re-read and argued with.

The model supplies a probability. Code decides consequences. I do not want a language
model to be the last thing standing between me and losing an issue.

What went wrong

I am keeping this section because the failures were the actual work.

My loss was 0.0000 for the first two hundred steps, and I almost shipped it. The
weights tensor has to align with the labels, not the inputs. I wrote:

x, y = full[:-1], full[1:]
w = [0.0] * len(ids) + [1.0]
# wrong: w[:-1] drops the only 1.0, so every supervised weight was 0
data.append((x, y, w[:-1]))

Every weight was zero. The model was training on nothing, and because Tinker reports
loss:sum already weighted, the metric was a perfectly plausible 0.0000 instead of an
error. The fix is one character of slicing — w[1:], not w[:-1] — and the loss at
initialisation became 0.736, which is ln(2): dead-on chance for a coin flip, exactly
what an untrained model should score. If your loss starts at zero, you are not
learning something impressive, you are not learning anything.

"LATER" is two tokens. I wanted the verdict vocabulary to be NOW / LATER, which
reads better in a blog post than NOW / WAIT. It tokenises to [43, 18577], which
makes the two-logprob comparison meaningless. The verdict had to become WAIT because
the tokenizer said so, not because I preferred it.

Qwen3.5-4B is a reasoning model, and it wanted to think first. With a normal
generation prompt it emitted "Thinking Process:\n\n1." before any verdict, which put my
scored token nowhere near the answer position: the initial loss was 17.5 nats per
token (p ≈ 2.5e-8). The fix was apply_chat_template(..., enable_thinking=False),
which produces a clean single-token NOW. I apply the same setting to the base arm, so
the comparison stays fair — the only difference between arms is the LoRA.

The base model, given one token and no thinking, is essentially at chance. AUC 0.516.
Not "slightly worse than the fine-tune" — 0.5 is a coin flip. Almost every bit of ranking
signal in this product comes from the fine-tune, which is the least flattering possible
finding for a post that is mostly about a fine-tune, and it is the true one.

Why Does Open Innovation Matter?

Because the thing being trained is my judgement about my own attention, and that is
not a resource I want to rent back.

The decision is mine, and it stays on my disk. The ledger is an append-only local file. Deferring an item is recorded with the probability that produced it, so six weeks later I can find every item I was wrong about and see what the model believed at the time. That audit loop is impossible if the scoring happens inside someone's API.
The training set is my behaviour. A closed model cannot be fine-tuned on which items I actually opened. Today the fine-tune learns one maintainer's stream in a few minutes on Tinker; the same pipeline pointed at a personal inbox is the same code with a different decisions.csv. With a hosted model I would be sending my reading habits out to be trained on someone else's roadmap.
It costs nothing per decision. 731 items × 2 forward passes, on a laptop-class model, with no card on file. A tool that charges per item is a tool I ration, and a tool I ration is a tool that does not do the boring middle of the job.
I can swap the model. The whole interface is one probability. Swapping in a 1B for speed, or a bigger model for the disagreements, is a config line — which is the point of the two-logprob design. compute_logprobs is not a Qwen feature; it is an open-weights feature.

And one honest concession: a 397B model, allowed to reason for a paragraph per item, would
probably beat this 4B on the hard cases. It costs about a thousand times as much per
decision and cannot run with no internet, which is exactly the trade this category is
about. I would rather have a 4B that I can run on the train.

The Lazy Payoff

73 of 731 real issues never reached me, at a cost of 11 mis-deferrals out of the 235 that actually needed a human within 24 hours.
The filter is a ranking, so the budget is a dial: the same model defers 64 items at a 3% miss budget, 73 at 5%, and 152 at 10%. The published numbers are the 5% point.
A perfect ranker could defer 67.9% of this stream; we reach 10.0%. Useful, and with headroom left.

The more useful payoff is not the count of issues. It is that the morning surface is one
card. My backlog does not shrink into a longer list that I scroll and feel bad about; the
731 items are 731 decisions already made, most of them "no", each one written down with a
reason and a fuse. The laziness being rewarded is not "the AI does my work" — it is
"somebody, finally, did the triage, and it was a 4B model that cost nothing and cannot
send my data anywhere".

What this does not do

Three repositories, one 5-day held-out window. This is a demonstration, not a benchmark. The 67.9% ceiling is a property of these four repos, and a repository with a different maintainer cadence will move it.
The held-out window is 71% microsoft/vscode (518 of 731 items) while training skews to next.js, rust and cpython (956/881/818 against 269 for vscode). A temporal split over a fast-moving repo set drifts in composition, so the headline number is dominated by one project's triage rhythm. I would want a per-repo breakdown before trusting this on a new repository, and I have not done one.
"A maintainer touched it" is not "it mattered." The label is a proxy. Some ignored issues genuinely mattered and some handled ones did not.
A mis-deferral is the expensive error. The policy is asymmetric on purpose: the 5% budget buys patience rather than quiet. At the 3% budget the filter defers less and earns less.
One queue, one shape. The extraction step assumes item-shaped text. It is not general triage and I would not pretend otherwise.

My Agent Session

I built this with an agent and the reasoning sits in the artefacts rather than in a
transcript: eval/policy.py is the judge, app/workflows.py is the guarantee, and every
number above is regenerated by app/build_site.py from data/ and eval/. The WAIT
token, the w[1:] fix and the enable_thinking=False fix are all visible in the repo's
history as they were found.

Prize Categories

Best Use of Tinker — the fine-tuned arm is Qwen/Qwen3.5-4B LoRA on Tinker, scored against the same base model with the same prompt and decoding position. The improvement over that baseline is the entire result.
Best Use of Temporal — every deferral is a SealWorkflow with a durable wait_condition timer plus an on_attention signal; the demo kills the worker with kill -9 mid-fuse and delivers a signal while no worker exists.

Confidant: a 1B Gemma drafts a therapist's session notes on her own laptop, and nothing leaves the room

memeshe — Tue, 13 Oct 2026 18:19:38 +0000

It is 9pm, and a solo therapist still owes the day a set of notes. She has two options. Type them — about twenty minutes every day, after the last client. Or pay $69–114 a month for a cloud "AI scribe" that records the session, which her clients did not consent to.

I built the third option. Confidant lets her speak a twenty-second recap between clients; a small open-weights Gemma model running on her own laptop drafts the structured note — Subjective, Objective, Assessment, Plan, next focus — and a deterministic guardrail checks every sentence against the words she actually said. A sentence whose words cannot be found in the recap is never filed. It is shown struck through, under "refused by the guardrail".

This is my entry for the Hacktoberfest Open-Source AI Challenge, Week 2: Side Hustle. It runs on a laptop with 8 threads, 6.7 GB of RAM, no GPU, no CUDA. Every number below was measured on that machine, and every claim maps to a file, a command, or a run in the repo.

What I Built

Confidant is a local-first session-note drafter for one-person practices: therapists, counsellors, physiotherapists, social workers.

The flow is short. She presses record and speaks a messy recap the way she would to a colleague — "Maya came in flat today, said the journaling stopped two weeks ago, we practised the grounding exercise, she'll try it twice before Thursday, and I want to watch the sleep thing next time." Speech-to-text runs locally through faster-whisper. The transcript goes to Gemma 3 1B in Ollama on the same machine, which drafts the note in five fields — the S/O/A/P shape clinicians already write, plus a next-session focus. Each sentence carries the exact transcript span that supports it; hovering a line highlights that span in the side rail. She edits inline, signs, and moves on. The interface is a document, not a dashboard — the note is the product.

Three decisions do most of the work:

The note is a view over the transcript. Every claim stores a verbatim quote from the recap. Grounding is a data-model invariant, not a prompt instruction.
Unsupported sentences are refused, not softened. The guardrail in daemon/src/validator.rs normalises a sentence's quote and searches for it in the transcript. If it cannot find it, the sentence never reaches the filed note. The model's failure mode is "this line is missing, and here is what it tried to write" — never a plausible clinical sentence that nobody said.
Risk flags are rules, not model output. A small, auditable rule set over her own words decides the risk chips. The type the note generator returns has no risk field at all: the model is never asked for a risk level.

What it does not do: no diagnosis, no advice, no EHR integration, no multi-user sync, no mobile app, no cloud. The Rust daemon (Axum) binds 127.0.0.1 only, and mutating routes require a run token injected into the page the daemon serves, so a random website cannot post a transcript into the local store. The store is SQLite on her disk. Audio is deleted after transcription — the delete happens before the transcription result is even parsed. Export is Markdown or a print-ready sheet; history stays in the local database.

There is one honest escape hatch: remote mode, opt-in, for hardware that cannot hold a model. It sends the transcript to any OpenAI-compatible endpoint and says so loudly in the UI. It is off by default. In local mode, pipeline::assert_policy refuses — a hard error, not a warning — any Ollama base URL that is not loopback, because that single misconfiguration would make the privacy claim false.

Demo

Live demo: https://uconfidant.tech

The demo on Render is read-only. It replays real recorded runs of the pipeline on the synthetic corpus, so you can watch the whole flow without installing anything: recap in, drafted note out, and at least one sentence struck through where the guardrail refused it. Underneath the replay is one arm of the evaluation table from the next section — the no-model baseline over the full corpus, rendered from the committed results file, the same one a reader can open.

The two-minute walkthrough is the same product recorded on this laptop — the paste, the CPU draft, the guardrail refusing one sentence it could not trace, then the deployed page answering 403 on every write:

The recording is also committed as docs/confidant-demo.mp4 (direct download), if you would rather download it than stream.

The local demo is the one that matters, and it works with the wifi off:

Open the app the daemon serves, and press Record instead.
Talk for twenty seconds about a session the way you would to a colleague.
Stop. The note drafts on the machine; each line shows the span it came from.
Find the struck-through line — that is the guardrail refusing something the model invented.
Export to print-ready HTML. The footer shows 0.0000 USD/note, the tokens/sec, and the model name.

The landing page and the docs live at https://uconfidant.tech, the read-only demo deployment itself — one URL, no separate marketing site.

Code

Repo: https://github.com/memeshee/confidant

Everything is in there: the Rust daemon (daemon/src/), the statically exported Next.js UI (ui/), the prompts shared by the product and the eval harness (prompts/), the synthetic corpus and the harness (eval/), and the footprint documents (docs/).

Three commands are worth running:

cargo test                                   # 20 tests: guardrail, provider policy, exports, store, line parser, eval-table determinism
python3 eval/run_eval.py --arm rules-only    # the no-model baseline
scripts/cleanup.sh                           # dry run: lists everything this build added to a machine

cargo test is at 20 passing tests, including one that refuses a fabricated medication dose. scripts/cleanup.sh is dry-run verified: 4.3 GB removable, of which 958 MB is the model, and one command reverses everything this build added to a machine.

How I Built It

The daemon is Rust (Axum + tokio + rusqlite) and serves the statically exported Next.js UI plus the local API on 127.0.0.1. Inference is Ollama on loopback; transcription is faster-whisper in a sidecar process; the store is SQLite. The prompts are compiled into the daemon and read by the eval harness from the same files, so the published numbers measure the shipped configuration, not a copy of it.

Then the measurements, because "runs on your machine" is a claim until it has a number attached:

Gemma 3 1B (QAT, 1.0 GB on disk, Ollama). The shipped default. On this CPU it drafts a note in one to two minutes: the recorded run below took 136 s for a 250-word recap at 3.6 tokens/sec, a shorter one came back in 74 s, and across the corpus the raw arm averages 2.9 tokens/sec. The output is thin — a handful of sentences — and it is the default anyway, because it is the one that runs with the wifi off.
Gemma 4B (3.2 GB weights + 0.85 GB vision tower). 81 seconds just to load, then 0.26 tokens/sec. A note would take about four minutes. I deleted it from disk. Whatever the download page implies, it is not a small-laptop model, and I would rather say that than ship it.
Gemma 3 27B (open weights, hosted via Nebius Token Factory, opt-in remote mode). 16 sentences in 9.9 s, all 16 traceable, 0 refused — 549 prompt tokens + 505 completion tokens. That is the trade in full: better notes, faster, and the transcript leaves the machine. Hosted endpoints also do not enforce JSON schemas (I probed), so the remote path uses a flat line protocol parsed in Rust instead of the schema the local path uses.

The evaluation has five measured rows: a rules-only baseline (deterministic sentence-splitting by trigger words, no model), the local 1B raw and the same 1B behind the guardrail, and the hosted 27B both raw over the full corpus and behind the guardrail. The corpus is synthetic and frozen: 90 recaps, 760 annotated facts and 427 planted distractors — recaps and fact lists written for this project, no real client data anywhere — and the harness is eval/run_eval.py; results are committed under eval/results/.

Arm	Fact recall	Facts in right section	Distractor rate	Unsupported claims	Median latency	Cost per note
rules-only baseline — 90 recaps	99.2%	40.0%	11.0%	0.0%	~0 ms	$0
local Gemma 1B, raw — 8 recaps	37.5%	25.0%	2.6%	69.0%	201.6 s	$0
local Gemma 1B + guardrail — 8 recaps	20.8%	6.9%	0.0%	0.0%	74.2 s	$0
hosted Gemma 27B, raw — 90 recaps	72.8%	45.9%	19.7%	1.5%	5.1 s	provider-priced
hosted Gemma 27B + guardrail — 8 recaps	87.5%	47.2%	13.2%	0.0%	12.4 s	provider-priced

Fact recall: annotated facts whose keywords appear anywhere in the note. Facts in right section: the same facts landing in the section they belong to — the metric that separates a note from a copy. Distractor rate: planted, plausible-but-absent statements the note wrote anyway. Unsupported claims: note sentences whose quote cannot be found in the transcript; the guardrail arms read 0% because refusing exactly those is what the guardrail does — that column measures the guardrail, not luck. Brevity (note words ÷ transcript words, committed in the results): rules-only 0.99, local 1B 0.44, hosted 27B 0.68 raw and 0.72 behind the guardrail — a note, not a transcript. Every cell is a real run committed under eval/results/, median over the recaps noted per row.

The row that matters most is the third one. Behind the guardrail, the 1B keeps about two sentences a note and refuses 1.75 more per note on top of that; three of those eight recaps filed an empty note, because nearly everything the model wrote failed the trace check. That is the honest cost of the shipped default being the model that fits: at 1B the guardrail is what stands between the clinician and a mostly-invented note, and its answer is often "nothing to file". On hardware that can hold a bigger model, the same code path turns 87.5% recall at 0% unsupported — which is why the model picker exists and why the hosted arm is measured through the same guardrail rather than around it.

The first version of this table was wrong, and the way it was wrong is worth reporting. My first metric was fact recall: does the note contain the facts. The rules-only baseline scored well over 98% — near-perfect, for a system with no model in it. That was implausible enough that I opened a result, and there it was: the baseline dumps the transcript into every section, so every keyword trivially matched. It was not writing a note; it was copying one. I rewrote the metrics to measure where each fact lands (section placement) and how long the note is relative to the transcript (brevity). The baseline immediately read as what it is: 40% of facts in the right section, and a note exactly as long as the transcript (0.99×) — while the hosted 27B's note is 0.68×, which is what a note looks like. A baseline that looks perfect is usually a measurement bug, not a miracle, and a recall-only metric would have let the no-model arm win the table.

Why Does Open Innovation Matter?

For this product the answer is structural, not philosophical.

A cloud scribe cannot be this product. Its architecture requires shipping a client's words to somebody else's server — the one thing this product promises never happens. That is a consent problem, not a pricing problem: her clients did not agree to be recorded, and no amount of vendor diligence changes that. The local, open-weights model is the only version of this product that can exist.

Open matters for the rest of it too:

She can see what the model was told. The prompts are files in the repo, not a vendor's secret.
She can re-run my numbers. The harness, the corpus, and the results are in the repo. Nobody has to trust this table.
The guardrail is code, not a policy page. It runs locally, on her machine, and it is unit-tested. Auditable refusal beats promised moderation.
$0.0000 per note is the difference between a tool that exists for a solo practice and one that does not. A per-note meter is a subscription for someone who already pays one.

There is a smaller version of this, too. The model sizes that make the product possible — a 1B that fits in a gigabyte — exist because open weights were released and measured in public. The 4B measurement above (81 seconds to load, 0.26 tokens/sec, deleted) is the kind of thing that only gets said out loud in an open ecosystem; a closed vendor's size chart does not include it.

My Agent Session

I built this as a series of numbered increments, one at a time, with an agent doing the work in each, under a rule that no increment counts until it ends in a real run. The session log — timestamps, decisions, and what broke — is committed at docs/CONFIDANT-PLAN.md.

The rule earned itself. The unit tests were green while the router was broken: axum 0.8 changed path parameters from :id to {id}, and the daemon panicked when the first real request hit a parameterised route. Unit tests never touch the router, so nothing in the test suite could have caught it. Fixing it meant renaming the path parameters; the daemon has served /api/health in local mode since.

The agent also produced one measurement I threw away: a thread-scaling benchmark run while the 4B model was still resident in RAM. Same machine, different load — not valid evidence. The clean re-run is committed at eval/results/bench-threads-clean.log. That habit — discard the contaminated number, re-run, commit the artifact — is most of what makes the numbers here worth reading.

The standing rule for the build was never invent evidence. It is why the metric bug above got fixed instead of hidden, why the third row of that table stays in the post although it flatters nothing, and why every cell is a run I can point at instead of a number I liked the look of. The decisions that mattered stayed with me: what ships as the default, and what the product refuses to do. The post itself was drafted with agent assistance, and every number in it was checked against an artifact before it went in.

My First User

No clinician was available within the five-day window, so this section carries no quote and no testimonial; the evidence that someone would actually pay for this is the measured evaluation instead — the baseline that files the transcript as a note, the 1B that keeps about two sentences a note once the guardrail has refused the rest, and the 27B comparison that produces sixteen. All of it is re-runnable from the repo with python3 eval/run_eval.py. A fabricated quote would have been easy to write, and it would have made every other number in this post suspect.

Prize Categories

Best Use of Gemma. Gemma 3 1B QAT is the shipped default runtime, running on CPU with no GPU: on this laptop it drafts a note in one to two minutes (3.6–4.8 tokens/sec; median 74 s end to end behind the guardrail across the corpus block above). Its honest limits are in that table too — at 1B, raw, 69% of its sentences cannot be traced to the recap, and the guardrail refuses most of them — which is exactly why the same product, pointed at Gemma 3 27B (open weights, hosted, opt-in), reads 87.5% recall at 0% unsupported through the same guardrail. The model picker lists whatever is installed on the machine, with real sizes.

Best Use of Render. The public read-only demo of real recorded runs, plus the evaluation table, is hosted on Render at https://uconfidant.tech (fallback: confidant-demo.onrender.com; service srv-db73srid0e5s73avf6i0, free plan, Docker runtime). Every mutating route answers 403 there — it is a replay, not a live service.

Best Use of a .tech Domain. The project's landing page and docs live on a .tech domain claimed through Hacktoberfest: https://uconfidant.tech. The domain points at the read-only Render deployment, so the landing page is the product demo itself — TLS issued, DNS on the registrar's own nameservers.

Not claimed, and why — now with runs behind the refusal. Sentry tracing is wired, and I checked the claim instead of asserting it: pointing the daemon at a local sink (eval/sentry_envelope_sink.py) shows the exact envelope it posts — note.draft, spans {cost_usd: 0.0, duration_ms: 73844, output_tokens: 347} and {transcript_chars: 600}. Metadata only; no clinical text, verifiable without an account. It stays unclaimed because metadata tracing is not what the category asks for. Tinker is wired too: a 40-step LoRA on Qwen/Qwen3.5-4B over the note-structuring task, trained on the deterministic floor's outputs and scored on six held-out recaps with the same scorer the committed eval uses (eval/results/tinker-lora-20261014-002815.json). The loss fell from 0.082 to 0.0015 per supervised token — and the tuned arm still did not dominate the floor: two runs of the same config landed on either side of it on section placement (2.17 and 3.17 of 7 against the floor's 2.50), while producing unsupported claims (0.33 and 0.5 per note) that the floor never produces at all. A first run that loses is the honest state of the art here; filing it as a win would be the lie this whole project exists to avoid.

Confidant is a small thing: one therapist, one laptop, one recap, and one note that can prove where every sentence came from. That it can exist at all is a property of open models. That it refuses to write what she did not say is the property I care about most.

FieldDay: the allergy-smart grass window

memeshe — Tue, 06 Oct 2026 11:24:19 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

FieldDay tells allergy families the single best hour to go outside today — then gets out of the way. Parents of sneezy kids currently check three apps (weather, AQI, pollen) and guess; guess wrong and the evening is miserable. FieldDay fuses forecast + pollen + YOUR symptom history into one 0–100 score per daylight hour, explains the pick in warm parent-language, and fits in one mobile screen. Sixty seconds, then screen off — that's the whole point of Touch Grass.

The whole point: more days like this. (Rick Obst, CC BY 2.0, via Wikimedia Commons)

Beautiful — and exactly what triggers the sneezes. (Peter Salanki, CC BY 2.0, via Wikimedia Commons)

Demo

Live: https://fieldday-r66v.onrender.com (free Render tier, PWA — installable, works from the home screen)

Tested live Oct 6, Hanoi, straight from the production API: best window 08:00 local (score 76/100, personal flare risk 0.227 via TabPFN on 48 rows), worst 35/100 later in the week. Every hour is labeled with the model that scored it. Outdoor verification with photos lands this week — updating this post.

Code

https://github.com/memeshee/fieldday

FastAPI + vanilla JS, one-click deploy via render.yaml, 6 smoke tests with CI.

How I Built It

Open-source AI is the core, not a garnish:

TabPFN (Prior Labs tabular foundation model) predicts personal flare risk fitted on YOUR symptom log — not a generic county pollen number. Your browser gets a random anonymous ID, so your taps train your forecast and nobody else's. Lightweight tabpfn-client so Render's free tier never OOMs; self-hosters can swap in local tabpfn with zero code changes.
Gemma (open weights) writes the coach note. Chain: local Ollama gemma3:1b → hosted Gemma via Gemini API → labeled template fallback. The UI always names which source wrote the note — no silent fallbacks.
Free keyless data: Open-Meteo forecast + pollen, geocoding included.

Hardest part: latency. v1 called TabPFN once per daylight hour — 7s × 24, guaranteed timeout. Fixed by fitting once and batch-predicting all hours in one call, then caching the predictor keyed on the training-data fingerprint: 37s cold → 1.3s warm in prod. Second hardest: Open-Meteo 429-throttles Render's shared IP, which once blanked the whole demo — now retry + 30-min upstream cache + an honestly-labeled MET Norway backup feed. Third: labeling every AI output with its provenance (risk_source, coach source), including when it's "just" the baseline.

Why Does Open Innovation Matter?

Symptom logs are health-adjacent — they must never train someone else's closed model or require an account. Open weights + a tabular foundation model give a personal forecast with no account, no tracking, no per-call price tag: TabPFN inference sees anonymized rows and never trains on them, Gemma runs local-first. $0 to run, any model swappable. Closed APIs would make this a privacy compromise and a billing meter; open makes it a tool parents can trust.

My Agent Session

Built pair-programming with an AI agent (Hermes) — roughly 280 tool iterations across one session: real TabPFN verification, predictor cache, per-user logs, hosted-Gemma chain, PWA shell, upstream-resilience fallback, this post. No DevRelay link (session ran outside DevRelay); the full build log lives in the repo's commit history.

Prize Categories

Best Use of TabPFN
Best Use of Gemma
Best Use of Render
Overall

My Mum Keeps Getting Scam Texts. I Built Her a Telegram Shield.

memeshe — Sun, 04 Oct 2026 19:26:19 +0000

My Mum Keeps Getting Scam Texts. I Built Her a Telegram Shield.

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend.

Live: t.me/scamsshield_bot · Dashboard · Repo

The friend

My mum gets parcel-scam SMSes in Thai almost weekly — "พัสดุถูกระงับ, ยืนยันตัวตนด่วน" with a bit.ly link, fake bank warnings, the works. She forwards them to me, I tell her "don't click that," and the cycle repeats. She's not going to learn to read headers. So I built the thing that reads them for her: ScamShield. She forwards any suspicious message to a Telegram bot, and in seconds gets back a verdict card — SCAM / SUSPICIOUS / LIKELY SAFE — with reasons in Thai first, English second, and what to do next.

What it actually does

Send anything — an SMS forward, a link, a wallet address — and the bot runs a triage pipeline:

Signal extraction — links, crypto addresses, phone numbers, urgency phrases ("ด่วน", "airdrop", "verify now"), impersonation hits (banks, Binance, Shopee, "ตำรวจ"…).
Live web grounding (SerpApi) — the suspicious domain gets searched for scam/fraud reports, so the verdict cites the web, not vibes.
On-chain probe — wallet addresses are checked against public RPCs: contract code (drainer pattern) vs plain wallet vs fresh/empty.
Score — transparent point rules, capped at 100. ≥70 SCAM, ≥40 SUSPICIOUS, else LIKELY SAFE.
Explanation by an open-weight model (Gemma) — two short sentences, Thai then English, written for a non-technical reader. The model explains; the rules decide. If the model is unreachable, the verdict still lands, honestly labelled rule-based.

Real outputs from the pipeline (cards exactly as the bot sends them):

🚨 Thai parcel scam → SCAM 75

🚨 อันตราย: น่าจะเป็นมิจฉาชีพ (SCAM, 75/100)

เหตุผล / Why:
• pressure tactics: ด่วน, ระงับ, ยืนยันตัวตน
• impersonates: พัสดุ
• suspicious link (shortener / risky domain)

ลิงก์ / Links:
`http://bit.ly/kerry-th-99`

เว็บพูดถึง / Web:
• bit.ly url scan | Free Url Scanner &amp; Phishing Detection — checkphish.bolster.ai/…

🤖 ข้อความนี้ดูน่าสงสัยเพราะใช้คำเร่งรีบและลิงก์ที่ดูไม่น่าเชื่อถือ อย่ากดลิงก์หรือกรอกข้อมูลส่วนตัวเด็ดขาด
This message is suspicious because it uses urgent language and an untrustworthy link. Do not click the link or enter any personal information.
_(model: gemma-4-26b-a4b-it)_

ทำอย่างไร / What to do:
• อย่าโอน อย่าให้ OTP — Do not send money or OTP
• ส่งมาให้ลูกดูก่อนเสมอ

✅ Family check-in → LIKELY SAFE 0

✅ ดูปลอดภัย (แต่ยังระวังไว้) (LIKELY SAFE, 0/100)

เหตุผล / Why:
• no pressure tactics, links, or crypto addresses found

🤖 ข้อความนี้ดูปลอดภัยเพราะเป็นแค่การถามไถ่ทั่วไปและไม่มีลิงก์แปลกๆ ให้กดครับ
This message looks safe because it is just a casual question and has no suspicious links.
_(model: gemma-4-26b-a4b-it)_

ทำอย่างไร / What to do:
• ไม่มีสัญญาณอันตราย แต่ถ้าไม่แน่ใจถามลูกก่อน
• ส่งมาให้ลูกดูก่อนเสมอ

There's also a live dashboard (latest verdicts stream in every 2 seconds, zero API credits burned on page views) and a /health endpoint the host watches.

Why open innovation is what makes this work

The prompt asks where open beats closed for this build. Three places, all load-bearing:

1. The explainer is an open-weight model behind a swappable plug. The verdict text my mum reads is written by Gemma through a plain OpenAI-compatible endpoint. This weekend that endpoint was, in turn: a big hosted Gemma (too slow, 35s+), a smaller one (fast, but it thinks out loud — it wraps everything in <thought> planning blocks), and finally the smaller one plus a 15-line answer-extractor that recovers the Thai/English sentences from inside the thinking. I could do that iteration in an evening because the model, the weights lineage, and the wire format are all inspectable and interchangeable. A closed black-box API gives you one behavior, take it or leave it. Next weekend the same code can point at Ollama on a laptop and nothing else changes — try that migration with a proprietary agent stack.

2. The score is readable by her kid. Every point comes from a named rule in one file (triage.py), and the card lists each reason. When the bot says 75, I can point at exactly which three signals added up. That transparency is the whole product for a family: trust isn't a brand, it's showing your work. Closed-scoring "AI fraud APIs" return a number and a shrug.

3. It costs $0 to run and nobody's data is the product. Free-tier hosting, 250 free SerpApi searches a month (only suspicious domains trigger one), public RPCs, an open model. A tool that protects your family shouldn't need a subscription or a data pipeline into someone's ad machine to survive.

What open didn't do: the score itself is still hand-written rules, and I'm saying so instead of dressing it up. The model explains; the rules decide; the receipts are real links and real RPC responses. A demo that fakes intelligence is worse than no demo.

How it's built

python-telegram-bot (polling worker inside the FastAPI lifespan) + FastAPI + httpx, on Render's free tier from a render.yaml. Telegram handlers have timeouts, an error handler, and a 120s triage budget so one slow provider can't wedge the bot. Every verdict is saved to SQLite and streamed to the dashboard. Sentry watches errors and performance. Nothing exotic — the exotic part is that it exists in my mum's chat app instead of a slide deck.

Built with: Render (hosting) · Gemma (open-weight explanations) · SerpApi (live search grounding) · Sentry (error + performance monitoring).

Repo: memeshee/scamshield — MIT, contributions welcome, especially Thai scam-pattern rules.

Handover

The bot is live in my mum's Telegram now. Her verdict on the verdicts lands here as an update — the real judging criterion was always whether she trusts it.

Disclosure: this write-up was drafted with an AI assistant; every verdict, link, and number in it was run and verified by the author against the live code.

DEV Community: memeshe

Whisper invented a cooking ad over six seconds of silence. So I built a memorial book that refuses to.

What I Built

memeshee / so-gio

Sổ Giỗ — the family memorial book whose ears admit what they didn't hear. Two local speech models, a deterministic Vietnamese lunar engine, and a guard that refuses to invent.

Bàn thờ

The problem, and who has it

Demo

Code

How I Built It

What I Measured

What Went Wrong

Why Does Open Innovation Matter?

My Agent Session

The Spirit of It

Prize Categories

Leave It: I fine-tuned a 4B model to defer 73 of 731 GitHub issues, then killed its worker with -9

What I Built

Demo

Code

memeshee / leaveit

A morning filter for a backlog you have decided not to read. Fine-tuned open-weight model + durable Temporal fuses.

Leave It

The claim, and the number behind it

How I Built It

1. The label had to be "a maintainer looked", not "somebody replied"

2. One policy for every arm, and it is a miss budget, not accuracy

3. What thinking costs

4. The fine-tune (Tinker)

5. The fuse is a Temporal timer, not a sleep()

What went wrong

Why Does Open Innovation Matter?

The Lazy Payoff

What this does not do

My Agent Session

Prize Categories

Confidant: a 1B Gemma drafts a therapist's session notes on her own laptop, and nothing leaves the room

What I Built

Demo

Code

How I Built It

Why Does Open Innovation Matter?

My Agent Session

My First User

Prize Categories

FieldDay: the allergy-smart grass window

What I Built

Demo

Code

How I Built It

Why Does Open Innovation Matter?

My Agent Session

Prize Categories

My Mum Keeps Getting Scam Texts. I Built Her a Telegram Shield.

My Mum Keeps Getting Scam Texts. I Built Her a Telegram Shield.

The friend

What it actually does

Why open innovation is what makes this work

How it's built

Handover

5. The fuse is a Temporal timer, not a `sleep()`