DEV Community: JonathanSolvesProblems

Spare Thumbs: Gemma 4 answers "can we move the meeting?" in my words, from my real calendar, and never double-books me

JonathanSolvesProblems — Sun, 25 Oct 2026 13:35:29 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge, Week 3: Slow and Steady.

What I Built

Clients move meetings at night. "Something came up Thursday, could we do Friday 10h instead?" To answer, I open my calendar, find a gap, and type the same reply I always type. It happens about once a week, across email, WhatsApp and Discord.

Spare Thumbs answers for me. Gemma 4 runs on my laptop and watches my Gmail. When a client asks to move a meeting, it finds the meeting, checks my real Google Calendar, and leaves a reply in the Gmail thread as a draft, written the way I write. If their time is free, it says yes. If not, it offers times I am actually free. When the client says "11h works", it moves the event in Google Calendar and drafts the confirmation.

The lazy part: I tap send.

Demo

In the video, a second address of mine plays a client called Sam. Its two messages were placed in my inbox with the Gmail API (I could not send from that account while filming). Everything after that is live: Gemma's steps in the terminal, the draft in the thread, the send, and the calendar move. The calendar is my real one: Friday 10h is a real call about a scratched tile, so Gemma has to work around it.

The result

Gemma 4 on its own is happy to agree to anything. I gave it ten "can we move it?" requests against a busy week, once without my calendar and once as Spare Thumbs:

	Drafts that offer a time I am already booked
Gemma 4 without my calendar	7 of 10
Spare Thumbs	0 of 10

Same model, same voice examples, same requests. The only difference is that Spare Thumbs looks before it answers, and the code checks every time it writes.

The client asked	Gemma 4 without my calendar	Spare Thumbs
move our 10h to 13h?	"no problem at all, 13h on monday works for me! thx :)" (13h is the insurance assessor)	"i can't do 13h cuz i have another meeting then, but i'm free at 14h30 or 15h."
Friday 11h works for me, does it for you?	"friday 11h works for me! perfect. thx" (11h is the accountant)	"sorry, 11h is taken for me. how about 10h or 14h on friday instead?"
Wednesday 14h instead?	"wednesday at 14h is good. thx!" (on site all afternoon)	"14h on wednesday is booked for me, sorry! how about 11h or 17h instead?"

How I built it

Gemma 4 decides every step. gemma4:e4b runs through Ollama with native function calling and five tools: find_meetings, check_calendar, draft_reply, move_event and hand_off. For each new message it chooses which day to check, reads what is booked, then accepts, offers other times, asks one question, or leaves the message for me. When the client replies, the same loop runs again with what it offered earlier, and it chooses to move the event and confirm. The loop ends when Gemma stops calling tools.

The code makes sure. The prompt asks Gemma to only offer free times, but a prompt cannot promise anything. So draft_reply and move_event refuse any time that overlaps my calendar, falls on a weekend, sits outside 9h to 18h, or is in the past. They also refuse a time written in the reply that was never passed in to be checked (an early version wrote "17h" in the text and skipped the check), and French words in a reply to an English message. A refusal goes back to Gemma as an error and it tries again. In the ten-request test, the gate caught one busy slot and one French slip before they ever reached a draft.

EmbeddingGemma gives it my voice. I have 63 messages of my own across WhatsApp and Discord, which is too few to fine-tune on and plenty to retrieve from. For each incoming message, EmbeddingGemma finds the five replies I wrote to the most similar messages, and Gemma 4 sees them as examples, along with a short style card it wrote from all of my replies. Every reply I send gets added to the examples.

Nothing leaves my laptop except the draft itself, which goes into my own Gmail. Both models run locally. Raw chats, tokens and examples live in a folder outside the repo, and passwords, emails, links, phone numbers, names and company names are redacted before anything reaches a prompt.

Does it sound like me?

I did not want the result to wait on friends filling in a form, so a model from another family played the friend. Qwen 3.5 (4B) read 8 of my real messages, then saw a message, my real reply, Gemma's reply with my examples, and Gemma's reply without them, and picked the one I wrote. Each of 15 held-out messages was asked in all 6 orders, so a judge that likes "A" gains nothing.

It picked Spare Thumbs as me 56% of the time, my real reply 33%, and plain Gemma 11%. Turning off my examples and the style card drops it from 56% to 11%.

A second judge, Qwen 2.5 (7B), disagreed: it picked plain Gemma 53% of the time, Spare Thumbs 36%, and my real reply only 11%. It prefers the polished AI reply over my own messages, so it cannot recognise how I write. I am reporting it anyway.

Why Does Open Innovation Matter?

My clients' messages are my clients' business. The reason I can run this at all is that Gemma 4 and EmbeddingGemma are open weights I can run on my own laptop: the mail, the calendar and my 63 real replies never go to a hosted model. Open weights also make the gate possible. I can wrap a model in code that refuses, retries and logs every step, and show the loop in a 100-line file instead of a black box.

The Lazy Payoff

A reschedule used to cost me about five minutes: open the calendar, find a gap, type the reply, then come back for the confirmation and move the event. Now it costs one tap on Send. It ran this week on my real inbox and calendar: the demo above is the first live run, with no retake on the agent's side.

Prize Categories

Best Use of Gemma: Gemma 4 (gemma4:e4b) runs the agent loop through native function calling, and EmbeddingGemma retrieves the voice examples. Both run locally through Ollama.

Limits

It only drafts. Nothing reaches a client until I press send, on purpose.
It answers in Gmail and Google Calendar only. My WhatsApp and Discord messages taught it my voice, but it does not answer there yet.
It only answers senders on an allow list.
The ten-request test runs on a made-up busy week, so the numbers are repeatable. The demo runs on my real calendar.
The voice judge is a model, not people who know me.

Code

JonathanSolvesProblems / spare-thumbs

Gemma 4 on my laptop answers 'can we move the meeting?' in my words, from my real Google Calendar, and never offers a time I'm booked.

Spare Thumbs

Clients move meetings at night. Spare Thumbs answers "can we move the meeting?" for me, in my words, from my real calendar.

Demo video: https://www.youtube.com/watch?v=ls6LhVsdbxI
Write-up: https://jonathanandrei.com/blog/spare-thumbs-gemma-4-reschedule-replies-real-calendar/

Gemma 4 runs on my laptop through Ollama and watches my Gmail. When a client asks to move a meeting, it finds the meeting, checks my Google Calendar, and leaves a reply as a draft in the Gmail thread. If their time is free it says yes; if not, it offers times I am actually free. When the client confirms, it moves the event in Google Calendar and drafts the confirmation. I tap send.

Results

Ten reschedule requests against a busy week, answered twice by Gemma 4 with my voice examples on:

Drafts that offer a time I am already booked
Gemma 4 without my calendar	7 of 10 (11 of 14 times offered)
Spare Thumbs	0 of 10 (0 of

…

View on GitHub

The longer write-up is on my site: https://jonathanandrei.com/blog/spare-thumbs-gemma-4-reschedule-replies-real-calendar/

A $1.35 fine-tune taught an open 4B model that CRA is ARC in French. It still writes NIS for SIN.

JonathanSolvesProblems — Fri, 16 Oct 2026 21:20:31 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge, Week 2: Side Hustle

What I Built

Every page on my site has a French twin. I build things on the side (send me a brief, get a working demo back in 48 hours), and the site ships in English and fr-CA, so every new page means a translation.

So I gave an open 4B model a line any Canadian small-business page would have: "you need a business number (BN) from the CRA". It wrote "auprès du RRCQ". There is no RRCQ. The Canada Revenue Agency is the ARC in French, and in Canada's open data the government's translators wrote ARC all 31 times the pair shows up.

The hero is recorded output from the deployed models, not a mock-up.

Who pays: a small business pays a bilingual freelancer per page to put its site into French. The client won't read the French closely. They will notice CRA where it should say ARC. With an open model on the freelancer's own laptop there's no per-word bill, and the client's text never leaves the machine.

Sans calque is the tool for that freelancer. Paste English, get French from a model I fine-tuned for $1.35 on government text, next to the untouched base model, plus a review list. It flags three things, and each flag says where its rule came from:

an English acronym left in the French
a missing French acronym the government uses (ARC for CRA, NAS for SIN), from 466 pairs learned from open data
a term the Translation Bureau's Termium Plus says to avoid, like "année fiscale" for fiscal year (it wants "exercice financier"). On "most small businesses use the calendar year as their fiscal year", the untouched model writes année fiscale and the fine-tune writes année financière. Both models write "comptes à payer" for accounts payable, which Termium also flags, and the check catches it on both

Nothing gets fixed silently. A flagged sentence lands on the review list, and the "French, ready to send" box tells you how many blocks to check first.

Demo

Try it: https://sans-calque.51-161-82-166.sslip.io (it calls the fine-tune live on Tinker, with a daily word cap so the credit lasts). Click "Use the sample" for the sentences in this post.

The sample on the live site. Margin notes say where each rule came from.

Code

JonathanSolvesProblems / sans-calque

English to Canadian French on an open 4B model fine-tuned on government text; every flag cites its source

Sans calque

English to Canadian French for small-business pages, on a 4-billion-parameter open model I fine-tuned for about $1.35 and can run on a laptop. Every flag it raises cites where its rule came from.

Built for the DEV Hacktoberfest Open-Source AI Challenge, Week 2: Side Hustle.

Live app: https://sans-calque.51-161-82-166.sslip.io
Demo video (1:42): https://www.youtube.com/watch?v=iObGJm1Id6Y
Write-up: https://jonathanandrei.com/blog/sans-calque-canadian-french-fine-tune-tinker/

What it does

Paste English. You get the fine-tuned model's French next to the untouched base model's, a review list, and a "French, ready to send" box. The review list flags three things, each with its source:

An English acronym left in the French, such as "CRA" in a French sentence.
A French acronym that is missing, when the government uses a known one: ARC for CRA (31 of 31 government pairs), NAS for SIN (10 of 10). The map has 466 acronyms, learned from the training pairs.
A term the Translation…

View on GitHub

A longer write-up is on my blog: Sans calque, a Canadian French fine-tune on Tinker.

How I Built It

The data is all open. Every dataset on Canada's Open Government Portal has its title and description in both languages, under the Open Government Licence. I pulled all 48,116 records, dropped the ones whose French the portal marks as machine-translated, and paired paragraphs and sentences only when both sides split the same way. That gave 82,455 pairs.

The fine-tune is two runs on Tinker. Qwen3.5-4B, LoRA rank 32, 8,000 pairs, 2 epochs, about 13 minutes each. Run 1 was a random sample. Run 2 drew 2,000 of its pairs from the 10,752 that contain a known acronym, because run 1 still wrote "de la RC" for CRA. Estimated cost: $1.26 and $1.35, from Tinker's per-token price and the real token count. The training script refuses to start if the estimate goes over budget, because the credit is a one-time $10.

The grading is not mine. I scored everything on canada.ca small-business pages, whose French was written by Government of Canada translators: 300 random sentences, plus all 147 that touch a Termium "avoid" concept. None of those sentences were in training.

	Gemma 3 4B	Qwen3.5-4B base	Run 1	Run 2
Government's French acronym written correctly	36.0%	50.8%	55.8%	66.9%
English acronyms left in the French	47	41	6	10
Sentences with a Termium "avoid" term (of 147)	16	24	7	14
chrF++	50.58	55.32	55.65	56.09

The gain is acronyms and terms, not overall quality. The acronym gain holds up: run 2 beats the base model in all 1,000 bootstrap resamples of the 146 held-out sentences that contain a government French acronym. Overall quality doesn't: run 2 beats the base model on chrF++ in 92% of 1,000 bootstrap resamples, and I had set 95% as the bar before running it, so I don't claim it. Run 2 isn't better everywhere either. It leaves more English acronyms and more Termium terms than run 1. I serve it anyway because it gets the acronyms a client notices.

It's not always right, and the base model sometimes is. On "you can apply for your SIN", the base model writes "numéro d'assurance sociale (NAS)", which is correct. Run 2 writes "NIS". The acronym check catches it: the government wrote NAS in 10 of 10 pairs. Run 2 also wrote "Conserver" where an instruction needs "Conservez", and none of my checks catch that one. That's why the box says to read it before you send it.

The checks got checked too. My first Termium check flagged the government's own translators 21 times out of 25. The label it was counting, "avoid, see observation", means avoid in some contexts, so I dropped it. What's left flags the human translation once, for "année fiscale", which Termium calls a calque and canada.ca still uses. I left that one in.

check_claims.py rebuilds every number in the README from the results files and fails if the prose and the data disagree. It also fails if a model output quoted in the docs isn't in the recorded outputs.

Why Open Innovation Matters

The weights are mine. I exported the fine-tune from Tinker, converted it to a 2.59 GB 4-bit file and it runs in Ollama on my laptop's 8 GB GPU at 32.5 tokens a second. Client text never has to leave the machine, and there is no per-word bill. The same web page runs against it: I loaded it with every request outside my laptop blocked, and it still translated, fonts included, with zero requests out.

Local mode with every request outside the laptop blocked. Two crops of one capture: the header and the first two rows. The local untouched model invented a different acronym, RReC, and it still gets caught.

That took three dead ends: Ollama no longer accepts LoRA adapters, its own quantiser needs a Mac, and an 8-bit file ran at 0.8 tokens a second. Shrinking it to 4-bit can cost something, so I scored the laptop copies on the same test. For run 1 it did: chrF++ 54.96 against 55.65 in the cloud. For run 2 it didn't. The laptop copy scored 56.43 against 56.09 and got 67.8% of the government's acronyms right against 66.9%. Statistically those are the same model, so I'm not claiming the laptop is better, just that nothing was lost.

It's narrow on purpose. The people who'd pay for this are freelancers and small agencies doing Canadian French, and me. But the recipe isn't Canadian: fine-tune on a government's own open bilingual text, then check every term against that government's term bank. Anywhere both exist (the EU's IATE term base, for one) the same scripts should apply, though I've only measured it here.

And the open data is the reason any of this works. The training pairs, the terminology bank and the acronym evidence all come from the Government of Canada's open data, and the test is the same government's translators. A closed model can't show you where its "ARC" came from. This one can, down to the 31 pairs.

My First User

Me, on my own site. jonathanandrei.com ships in English and fr-CA, so I pasted the English "How I work" page into the live app: 41 blocks.

The fine-tune raised 8 flags. 7 were noise for this kind of text: MCP, LLM, RAG, FHIR, SDK and a NICE guideline code, all acronyms my own French page keeps in English. The acronym rule learned how the government writes, and tech jargon doesn't follow it.

The 8th was real. My sentence says the logs held data "the AI had already claimed was PHI-free". The French came back as "lesquels avaient déjà été déclarés exempts", which means "which had already been declared free". The AI is gone from the sentence, and it was the whole point of it. The flag fired because the expected "IA" was missing, and it was missing because the model dropped the subject.

So on my own page: one catch that mattered, seven to wave off. That told me what to build next, so I built it: a "keep in English" list per client, and a "Fetch the page" box so the page comes in by its address instead of by copy and paste. Same page, fetched by the app, with MCP, LLM, RAG, FHIR, SDK and the guideline code on the list: 8 flags became 1, and the one left is the real catch.

Prize Categories

Best Use of Tinker

I Walked an Open Model's Bingo Card Through a Town With Almost No Nature Records. I Found 2 of 16.

JonathanSolvesProblems — Tue, 06 Oct 2026 19:51:00 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

Hawkesbury, Ontario, is a town of about ten thousand people on the Ottawa River. Around it, iNaturalist holds about 3,000 research-grade records in total. Across every October on record, the most-logged species within 10 km of town was seen four times.

That number is the whole problem. The obvious way to get someone outside in October is to hand them a list of what people saw near them last October. In Montreal that list writes itself. In Hawkesbury it is whatever one or two people happened to photograph, and you cannot rank anything from four sightings.

Nature Bingo is a card of the 16 things most likely to be out near your town this October. You open it, put the phone away, and go find them. When you find one, you tap its square and it gets dabbed with ink and stamped with the time. You can also print it and leave the phone at home entirely. The card is the screen; the walk is the point.

The trick for small towns is that the card borrows from its neighbours. It is ranked by an open-weight model that learned ten years of Octobers across 48 towns between Ottawa and Montreal, so a quiet town can lean on what the whole region knows.

Demo

Try it: naturebingo.vercel.app (pick any of the 48 towns, or keep Hawkesbury).

There is also a 51-second version.

Code

JonathanSolvesProblems / nature-bingo

A bingo card of the 16 things most likely out this October near your town. TabPFN, an open-weight model, ranked on a laptop GPU from iNaturalist records.

Nature Bingo

A bingo card of the 16 things most likely to be out near your town this October. Open it, put the phone away, and go find them. Tap a square when you do, or print the card.

Open the app · Demo video · Short · Write-up · My walk on iNaturalist

Built for the DEV Hacktoberfest Open-Source AI Challenge, Week 1: Touch Grass.

Why small towns

A "what was seen near here last October" list works in a city. It does not work in a small town. Around Hawkesbury, Ontario, iNaturalist has about 3,000 research-grade records in total, and the most-logged species across every October on record has four sightings.

So the card borrows from the neighbours. TabPFN, an open-weight tabular model, learns from ten years of Octobers (2016 to 2025) across 48 towns between Ottawa and Montreal. For each candidate species in each town it…

View on GitHub

Every number in this post is in the repo: the backtest in results/backtest.json, the split by town history in results/backtest_bands.json, and the walk in data/walks.json.

How I Built It

The data. For 48 small towns between Ottawa and Montreal, I pulled iNaturalist research-grade species counts for every year from 2016 to 2025: within 10 km of the town in October, within 10 km in any month, and within 50 km in October. For each town and year, the 300 species most logged across the region become candidates, about 86,000 training rows of (town, year, species), each described only by what happened in earlier years: how often it was logged here, how often across the region, in how many separate years, how seasonal it is locally, and how much anyone was out logging at all.

The model. TabPFN is an open-weight foundation model for tables. Instead of training a model on my data, it reads a sample of labelled rows as context and predicts the rest in one pass, which suits a problem where every town is small and the useful signal is in how towns resemble each other. It runs locally on my laptop's RTX 5060. Making Hawkesbury's card takes 19 seconds, and nothing goes to a model API.

The test. I did not want to grade this against my own idea of what should be out, so I let the public grade it. I trained on target years up to 2024, held October 2025 out entirely, made every town's card blind, and counted how many of its 16 squares somebody actually photographed and logged on iNaturalist that month. Four methods, same towns, same month:

Method	Squares photographed, of 16
TabPFN	5.02
The town's own past Octobers	4.44
Gradient boosting (scikit-learn, same features)	4.33
Regional most-seen	3.65

Town by town against the town's own past Octobers, TabPFN's card was better in 23 towns, the same in 14 and worse in 11.

What it does not do. I built this for small towns, and in the 16 towns with the least October history it does not win. Gradient boosting found 2.19 photographed squares there and TabPFN 1.88, both ahead of last October's list at 1.19. Hawkesbury, the town I walked, is one of the places TabPFN lost: 3 squares against 6. Where the data is thinnest, no method finds more than about 2 of 16, because almost nobody is out there taking pictures. The grader also has a bias I cannot remove: it rewards guessing what people photograph, which leans towards big, common, daytime things near paths.

The card says what it cannot know. Under every card, in the product rather than a README, it states its own score: "When I ran the same method on last October, people photographed 3 of the 16 things on this town's card. The other 13 went unphotographed, which is not the same as absent." Every photo on the card is an openly licensed iNaturalist photo, credited by name.

Then I took it outside. The brief asked us to use it and say how it went. On Tuesday 6 October, from 10:18 to 10:54, I walked Hawkesbury's card at Confederation Park, under the big Franco-Ontarian flag. My phone stayed in my pocket except to tap a square, take a photo, and fly the drone for the video.

In about half an hour I confirmed 2 of the 16: dozens of Canada geese on the lawn at 10:52, and a mallard drake in the shallows a minute later. The ring-billed gulls on the shore are marked probable, not confirmed, because the ring on the bill is not visible in my photos. I also found things the card did not have: a huge old cottonwood, a spruce, and a few large white geese mixed in with the Canada geese.

I posted the photos to iNaturalist that afternoon. Within hours, the goose and the mallard reached Research Grade, identified independently by another observer. The gull is still waiting for a second opinion, which is exactly what "probable" meant. So the two squares I ticked were not graded by me or by the model, but by another person looking at my photos.

Two of sixteen is not a great score, and it is one morning, not a measurement. For scale: across all of last October, everyone on iNaturalist together photographed 3 of the squares on the card the model made for this town. One person in 36 minutes found 2.

Why Does Open Innovation Matter?

A card for every small town has to be free to make. The backtest scored 14,400 candidate rows, and every card scores 300 species per town. Locally that cost about eleven minutes of laptop GPU for the whole test and nothing per card. Behind a metered API, the experiment that tells me whether the card works at all would have been the expensive part, and the cheap move would have been to skip it.

Open let me test the model against plain alternatives on equal terms. Because the weights run on my own machine, I could run TabPFN and scikit-learn's gradient boosting on the same rows, the same features and the same held-out month, and report where the open-weight model loses as well as where it wins. That table is only possible when I control both sides.

Open data made the grading honest. The training data and the answer key are both public iNaturalist records under open licences. Anyone can re-pull October 2025 and check my 23, 14 and 11. And because iNaturalist is open, the loop closes: my three observations are now part of the data the next card learns from.

The ranking never touches someone else's server. The model sees counts per town, not where you are. The only outside calls are to iNaturalist's public API for those counts.

To be fair about the edges: TabPFN's weights are free but gated. You accept Prior Labs' license once and get a token before they download, and the license is theirs, not MIT. And the 8 GB on my laptop GPU meant TabPFN saw a 4,000-row sample of the 85,974 training rows, not all of them.

Prize Categories

Best Use of TabPFN

Alberta stopped changing its clocks in June. 19 of 19 frontier models still put Calgary on standard time in November.

JonathanSolvesProblems — Mon, 05 Oct 2026 13:52:48 +0000

This is a submission for the Kaggle Benchmarking Challenge

On June 18, Alberta's Official Time Act came into force and the province stopped changing its clocks. British Columbia had done it in March. The Northwest Territories followed in August, Morocco went back to plain UTC on September 20, and on September 17 Manitoba announced it will not fall back on November 1 either. CBC is running stories about a Calgary hairstylist whose winter bookings all moved an hour. Microsoft published interim guidance for Windows.

So I asked the question someone in Calgary is asking right now. It is 9 a.m. here on November 15. What time is that in Toronto?

Then I put it to 19 models on Kaggle Benchmarks. All 19 said 11:00. The answer is 10:00.

What I benchmarked and why

Every model's knowledge stops at its training cutoff, and most of the time you cannot see the edge. Time zones are different. Governments change them by statute, on a date, and the IANA time zone database records every change within days. That gives a benchmark two things most benchmarks never get: an answer key nobody has to write, and a calendar to hold the model's knowledge against.

The answer key is tzdata 2026d, the September release of the same database that runs the clock on your phone. I did not type a single expected answer. A script asks Python's zoneinfo what the clock did in a given place on a given date, and that is the truth. The maintainers ship a few releases a year, so the key rewrites itself and the benchmark does not go stale.

125 questions, three kinds:

What is the UTC offset in Calgary at noon on November 15, 2026? Answer as +HH:MM or -HH:MM.
It is 09:00 on November 15 in Calgary. What is the local date and time in Toronto?
Do the clocks in Calgary change at any point during November 1, 2026, and in which direction?

Six families of places and dates. Textbook controls (New York, London, Tokyo). Awkward offsets (Kathmandu at +05:45, the Chatham Islands at +12:45, Lord Howe Island's 30-minute daylight saving). Southern-hemisphere daylight saving. Changes legislated between 2022 and 2025 (Iran, Jordan, Syria, Mexico, Greenland, Egypt, Kazakhstan, Paraguay, Chile's Aysén region). The 2026 wave (British Columbia, Alberta, Northwest Territories, Morocco). And Manitoba, which announced its change on September 17, after the release I graded against, so its three questions are asked, recorded, and not counted in the score. That leaves 122 graded questions.

The part I care about most is the dating trick. Of the 122, 46 have an answer that changed between one tzdata release and another. I keep every release since 2022a unpacked, twenty of them, score each model's answer sheet against all twenty, and the release a model agrees with most is the month its world clock stopped. A model that still puts Almaty at +06:00 has a clock from before February 2024. One that puts Vancouver at -08:00 in December 2026 has a clock from before April 2026.

Answers are structured output, so grading is a string comparison after normalising UTC-6, -6:00 and −06:00 to the same thing. A wrong hour is a wrong hour. There is no judge model anywhere in the loop, and the task records the model's one-sentence reason for each answer without grading it, which is where the quotes below come from.

Which models I ran it against

19 models, all through Kaggle Benchmarks, same prompt, temperature 0, no tools. I included 2025 models on purpose (Gemini 2.5 Pro from June, DeepSeek-R1 from May, Claude Haiku 4.5 from October) because a dating method that cannot date a 2025 model to 2025 is not worth reading. Then I asked 46 of the questions a second time with a one-function tool the model was free to ignore: zone_clock(iana_zone, local_datetime), which reads tzdata 2026d. Not "use this tool." Just "it is there."

Model	Released	Score (of 122)	2026 questions right (of 25)	Clock dated by the ladder	Vendor's stated cutoff
GPT-6 Astra	2026-09-03	107	10	2026b (April 2026), agrees on 45 of 46	2026-04-30
Claude Opus 5	2026-07-24	102	5	2025b to 2026a (March 2025 to March 2026), agrees on 46 of 46	2026-05
GPT-5.5	2026-04-23	102	5	2025b to 2026a (March 2025 to March 2026), agrees on 46 of 46	2025-12-01
GPT-5.6 Terra	2026-07-09	102	5	2025b to 2026a (March 2025 to March 2026), agrees on 46 of 46	2026-02-16
Gemini 2.5 Pro	2025-06	99	6	2025a (January 2025), agrees on 42 of 46	2025-01
Gemini 3.7 Flash	2026-08	99	5	2025a (January 2025), agrees on 44 of 46	2026-03
Gemini 3.8 Flash	2026-09-02	99	5	2025a (January 2025), agrees on 44 of 46	2026-03
Gemini 3.1 Pro	2026-02-19	98	5	2025a (January 2025), agrees on 43 of 46	2025-01
GLM-5	2026-02-11	93 of 121	4 of 24	2024a to 2024b (February 2024 to September 2024), agrees on 42 of 46	not published
Gemini 3.5 Flash-Lite	2026	91	7	2024a to 2024b (February 2024 to September 2024), agrees on 39 of 46	not published
Gemma 4 31B	2026-04-02	90	3	2024a to 2024b (February 2024 to September 2024), agrees on 41 of 46	2025-01
Grok 4.20 Reasoning	2026-03	89	4	2023a to 2023c (March 2023), agrees on 40 of 46	2025-09-01
GPT-5.4 mini	2026-03-17	88	4	2024a to 2024b (February 2024 to September 2024), agrees on 39 of 46	2025-08-31
Claude Opus 4.5	2025-11	87	5	2022g (November 2022), agrees on 37 of 46	not published
Claude Sonnet 5	2026-06-30	87	5	2022f (October 2022), agrees on 39 of 46	2026-01
Qwen3-Next 80B Thinking	2025-09-11	85	9	2022f (October 2022), agrees on 37 of 46	2025-09-30
DeepSeek-R1	2025-05-28	82 of 121	7	2022a (March 2022), agrees on 36 of 46	2025-03-31
Grok 4.20	2026-03	77	7	2022e (October 2022), agrees on 34 of 46	2025-09-01
Claude Haiku 4.5	2025-10-15	59	3	2022a to 2022d (March 2022 to September 2022), agrees on 39 of 46	2025-02

GLM-5 and DeepSeek-R1 each lost one call to their backends, so each is scored on 121 questions; every other model answered all 122, though Gemma 4 left the answer field empty on six of them, and an empty answer counts as wrong. The vendor cutoffs are what each company publishes where it publishes one, and third-party trackers where it does not. Claude Opus 4.1 and Grok 4.6 are on Kaggle's list but return 404 from the proxy, so they are not here.

What I found

Everyone knows how clocks work

Fifteen of the 19 answered all 26 control questions correctly, and nobody scored below 23 of 26. Twelve got every awkward-offset question right, including Lord Howe Island's half-hour spring forward and the Chatham Islands at +13:45 in January. The southern hemisphere was nearly as clean. This is the part where the Test of Time paper found models scoring 74 to 90 percent on time-zone questions back in 2024 and attributed it to the amount of time-zone text on the internet. That reading holds. Time zones as a topic are learned.

Time zones as of a date are a different thing.

Nobody knows about Alberta

Twenty of the 25 questions about the 2026 wave have an answer that changed this year. The other five are controls inside the family, Vancouver in July or Fort Nelson in November, where nothing changed. Twelve of the 19 got all five. Thirteen of the 19 got none of the 20 changed answers right, including Claude Opus 5, GPT-5.5, GPT-5.6 Terra and three of the five Geminis.

On the 20 changed questions, across 19 models, there were 18 correct answers. GPT-6 Astra produced 5 of them. Four are about British Columbia, with the right reason: "British Columbia adopted permanent UTC−07:00 time in March 2026, so Vancouver does not turn clocks back in autumn." The fifth is an Inuvik conversion with no reason given. The other 13 came from models that did not know, and not one of those 13 notes says that anything changed in 2026. Qwen said "Clocks do not change on 2026-11-01 as the fall back occurs on 2026-11-02." November 1 is the Sunday. Gemini 2.5 Pro said Inuvik's clocks do not change because "the Northwest Territories permanently observes Mountain Daylight Time (UTC-6) year-round", then put Inuvik on UTC-7 in the next question. I read every one of the thirteen. Not one model knew that Alberta, the Northwest Territories or Morocco had changed anything.

The Calgary offset question is the one I would put in front of a judge, because it cannot be right by accident. Calgary at noon on November 15, 2026 is -06:00. All 19 models said -07:00.

Dating the clocks

This is the chart the whole benchmark was built for. Each line is one model's agreement with each tzdata release since 2022, on the 46 answers that changed. The peak is the release the model's clock is dated to. Identical curves are drawn once and labelled with every model on them.

GPT-6 Astra is the cleanest result in the set. Its curve rises through every release since 2022, agrees with tzdata 2026b on 45 of the 46 changed answers, and falls off a cliff at 2026c. The one it disagrees on is that Inuvik conversion. tzdata 2026b was released on April 22, 2026, and carries British Columbia; the next release, 2026c, came on July 8. So the ladder says this clock stopped somewhere between those two dates. OpenAI says the model's cutoff is April 30, 2026. Those agree, and the ladder got there from nothing but clock questions.

GPT-5.5, GPT-5.6 Terra and Claude Opus 5 share one curve, and it is also a perfect 46 of 46, at 2026a. That is the last release before British Columbia. OpenAI's stated cutoffs for its two are December 2025 and February 2026, both consistent. Anthropic says Opus 5 was trained on data to May 2026, which should include a change that took effect on March 9. Its own note on Vancouver in December says why it does not: "On 2026-12-15 Vancouver is on Pacific Standard Time (UTC-8), since daylight saving ended on the first Sunday of November 2026 and British Columbia's permanent-DST legislation has not been brought into force." It knows the law. Its clock stopped before the law started.

Gemini 2.5 Pro, 3.1 Pro, 3.7 Flash and 3.8 Flash all date to tzdata 2025a, released January 15, 2025. Gemini 2.5 Pro from June 2025, Gemini 3.7 Flash from August 2026 and Gemini 3.8 Flash from September 2026 all peak there, and the two Flash curves never differ by more than one answer anywhere on the ladder. Only Flash-Lite sits elsewhere, at 2024a to 2024b with a blurry 39 of 46. Google's model card for 3.8 Flash gives a cutoff of March 2026 "for some domains" and January 2025 for the rest. The clock is one of the rest, and it has not moved in fifteen months of releases.

Claude Sonnet 5 dates to October 2022 and Claude Haiku 4.5 to somewhere between March and September 2022, against stated cutoffs of January 2026 and February 2025. These are not parsing accidents, I read the answers. Sonnet 5 says "Iran observed daylight saving time (UTC+4:30) during summer 2023 before abolishing DST later that year" (Iran abolished it in 2022), puts Cairo on +02:00 in July 2023 in a sentence that mentions Egypt's 2023 reintroduction of daylight saving and then ignores it, and says "Almaty has used the UTC+6 time zone year-round since 2024-03-01", which is the day Kazakhstan moved Almaty to +05:00. It scored 16 of 31 on the changes legislated between 2022 and 2025. GPT-6 Astra, GPT-5.5, GPT-5.6 Terra and Opus 5 scored 31 of 31.

The peak height matters as much as its position. A model at 46 of 46 has a sharp clock: everything before the peak right, everything after it wrong. A model at 34 of 46 has a blurry one, and the dating is a best fit rather than a fact. The table gives both numbers for every model.

How much of this is noise? I fixed the harness several times along the way, so 15 models ran the from-memory questions at least twice, 42 runs in all. The median model's score moved by 2 questions between runs, and the largest swing was 10 questions, Claude Haiku 4.5. The release its clock dates to stayed the same in 10 of the 15, and moved by one or two releases in the rest. Every run of every model put Calgary on -07:00.

With the tool

Then I asked again with a tool on the table. zone_clock(iana_zone, local_datetime) reads tzdata 2026d and returns the offset in force. The system prompt says the tool exists and that the model may call it or answer without it. Nothing tells the model its knowledge might be out of date.

This half asks 46 of the 125 questions: every question about 2026, the three about Manitoba, and eighteen older ones as a control group. 43 of the 46 are graded. The reason it is a subset is money, which I get to below. 16 models finished it.

Model	From memory (of 43)	With the tool (of 43)	Asked the tool	Overrode it	Asked about another zone only	Answer changed when restated
Claude Haiku 4.5	14	43	43 of 43	0	2	2
Claude Opus 4.5	18	42	43 of 43	1	1	2
Gemini 3.8 Flash	21	42	43 of 43	0	1	0
GPT-5.4 mini	20	42	43 of 43	1	1	3
GPT-5.6 Terra	23	42	42 of 43	1	0	0
Grok 4.20 Reasoning	18	42	43 of 43	1	0	1
Claude Opus 5	23	41	43 of 43	2	0	3
Gemini 3.7 Flash	21	39	43 of 43	3	1	2
GPT-5.5	23	39	33 of 43	1	0	1
Grok 4.20	21	38	43 of 43	4	1	4
Gemma 4 31B	18	36	43 of 43	5	4	4
GLM-5	20 of 42	36	43 of 43	6	1	4
Gemini 3.1 Pro	22	35	41 of 43	7	1	7
Claude Sonnet 5	17	34	40 of 43	6	2	6
Gemini 3.5 Flash-Lite	22	33	43 of 43	9	3	2
Qwen3-Next 80B Thinking	21	19	2 of 43	0	1	0

Almost everyone checks. Eleven of the 16 asked the tool on every one of the 43 questions. GPT-5.6 Terra asked on 42, Gemini 3.1 Pro on 41, Claude Sonnet 5 on 40 and GPT-5.5 on 33. Qwen asked on 2, answered the rest from memory, and scored 19, which is lower than the 21 it got without the tool.

Checking mostly works. Fourteen of the 16 got at least 34 of 43 with the tool, and nobody got more than 23 from memory. Claude Haiku 4.5, whose clock dated to 2022 in the memory half, is the only model that got all 43. It never once answered against the database.

Checking is not believing. Take the question this post opened with. With the database one call away, 8 of the 16 still said 11:00. Two of those eight wrote the correct offsets in the same sentence. GPT-5.6 Terra's note reads "Calgary used UTC−06:00 and Toronto used UTC−05:00." Its answer was 11:00.

Claude Opus 5 argued with the tool in so many words. On Calgary's offset it wrote "the tool reports -06:00 with abbreviation CST, which does not match Alberta's actual rules" before coming round to the tool's answer. On Casablanca in December it did not come round. It answered +01:00 and left this note: "the zone_clock lookup returned +00:00, which conflicts with this known rule". Gemini 3.5 Flash-Lite called the tool 3 times about Calgary's clocks on November 1, was told they do not change, and answered that they fall back. Across the 16 models there were 47 answers like that, where the model asked about the right place on the right date and then answered something else.

Some of it is a second chance to be wrong. Kaggle's SDK runs the tool loop, then sends a final message, "Now format your previous answer using the requested schema.", and the benchmark grades that restated answer. analysis/tool_trace.py compares the two, and the last column of the table counts the cases where they differ. In 28 cases the model's own words had the right answer and the restated one did not. Claude Sonnet 5 wrote "Per IANA tzdata 2026d, Vancouver is on permanent standard time (abbreviated MST, no DST) at -07:00 for this date", then restated the offset as -08:00. 13 answers went the other way and were fixed at that step. Gemini 3.7 Flash ran this task three times while I was fixing the harness and scored 42, 40 and 39.

The failure I like best is Coyhaique. Chile's Aysén region got its own zone, America/Coyhaique, in tzdata 2025b. A model whose clock stopped before that does not know the zone exists, so it asks the database about America/Santiago, gets a true answer about the wrong place, and reports it. 9 of the 16 missed the question, and only five asked about America/Coyhaique at all. Gemini 3.8 Flash: "Coyhaique in the Aysén Region follows official mainland Chile standard time (America/Santiago), which observes UTC-04:00 during the winter months." A lookup tool fixes what the model knows to look up.

Why a subset, and who is missing. Kaggle gives each account a model quota of $10 a day, and a tool loop is expensive because every round sends the whole conversation again. Gemini 3.8 Flash cost $0.16 for the 125 questions from memory and $0.99 for the same 125 with the tool. My first full tool lineup ran out of quota partway through, and the runs queued behind it failed on a 403 from the proxy. So the tool half is 46 questions. GPT-6 Astra is missing for a different reason: the API refuses function tools for that model unless reasoning is off, and refuses to switch its reasoning off, so Kaggle's SDK has no way to hand it a tool. GPT-5.6 Terra has the same rule and does allow reasoning off, so it ran this half that way. DeepSeek-R1 does not support tool calling at all.

Manitoba, which the database caught up with

Manitoba announced on September 17 that it will not fall back on November 1. When I built the answer key no release had it, so its three questions were asked, recorded, and left out of the score. Then, on September 30, the tz maintainers shipped tzdata 2026e: "Manitoba moves to permanent -05 on 2026-10-31".

So I graded those three answers against 2026e. Winnipeg at noon on November 15 is -05:00, and 9 a.m. in Winnipeg is 9 a.m. in Toronto. All 19 models said -06:00, and all 19 said 10:00. Asked whether Winnipeg's clocks change on November 1, 18 said they fall back. Qwen said they do not, and it only got that right because it believes the fall-back is on November 2.

The benchmark ran before the database knew, and the database caught up a week later. The models will take a training run.

What surprised me

Release date is not the clock. Gemini 3.8 Flash shipped on September 2, 2026 with its world clock on the same release as Gemini 2.5 Pro from June 2025: tzdata 2025a. Fifteen months of model releases, one clock.

Knowing about a law is not knowing its date. Opus 5 can tell you British Columbia passed a permanent daylight time law and still puts Vancouver on the wrong offset, because the fact it learned was "passed" and the fact that matters is "in force since March 9."

The models agree with each other more than with the world. On the Calgary to Toronto question, all 19 models gave the same wrong answer, with the same confidence, and most of them cited the same rule: daylight saving ends on the first Sunday in November. That rule was true in Alberta for 55 years. It stopped being true on June 18.

Checking is not believing. Fourteen of the 16 models that had the database asked it on at least 40 of the 43 questions, and eight of the 16 still told Calgary it was 11:00 in Toronto. Claude Opus 5 told the database it was wrong about Alberta.

What I would measure next

Run the same 125 questions again after tzdata 2027a, without changing a line, and see which vendors' clocks moved. The answer key updates itself; the benchmark is designed to be re-run.

Tell the model the current date in the prompt and see whether any of them hedge. Not one of the 19 said "as of my training data" on a question about November 2026.

Add a web-search condition beside the tzdata tool. With a database one call away, nearly every model looked. The open question is whether a model goes looking when nothing on the table suggests it should.

Grade the model's own words as well as the restated answer, as two scores. The gap between them is a measurement of the harness, and I only found it by reading transcripts.

Where to see it

The benchmark on Kaggle: https://www.kaggle.com/benchmarks/jonathanandrei/world-clock
Its two tasks: from memory and with the tzdata tool
Code, answer key, every result and both charts: https://github.com/JonathanSolvesProblems/world-clock
The 90-second demo: https://www.youtube.com/watch?v=oTj1_WVNkIg
This write-up on my site: https://jonathanandrei.com/blog/world-clock-benchmark-graded-by-the-iana-tz-database/

The repo has a script called check_claims.py. It reads the results and fails if any number in this post disagrees with them. It ran before this was published.

My mom has published 710 pieces in 16 years. Nothing she wrote links to 526 of them.

JonathanSolvesProblems — Sun, 04 Oct 2026 21:34:36 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend.

What I Built

In May my mom sent me a poster she had made. A graveyard at dusk, six tombstones, and on each tombstone the title of something she had written and where it ran. Across the top: "Don't treat your content like a graveyard. Treat it like a library." Under it she typed, "Remember the app idea I had?"

My mom is Mona Andrei. She writes humour. She has kept a blog called Moxie-Dude since 2010, she writes a column for Westmount Magazine, she has a Substack for single moms, and she wrote a book called Superwoman. She wanted two things: everything she has written in one place, and something that would point her back at what she already wrote when she sits down to write something new.

I built her a social media scheduler. It had a landing page and a logo and it ran on sample data. She found a product online that already did that, and texted me: "Found it. I will find the hole."

This weekend I built what she asked for the first time. It is called The Moxie Library, and it is a single web page.

On the left is a sheet of paper. She writes a draft there, or pastes one. When she pauses, the page looks through everything she has published and brings back the pieces on the same subject. Each one comes with her own paragraph quoted, where it ran, how many years ago, and a button that copies a link she can paste straight into WordPress or Substack.

Along the bottom is the ground. Every piece she has published is drawn there as a small slab, by year. If nothing else she wrote links to it, it is a tombstone. If something does, it is a book on a shelf. A tombstone and a book spine turn out to be the same shape, which is the whole design.

Nothing on the page is written for her. Every suggestion is a real link and a passage cut out of her own text. If she writes about something she has never covered, it says "Nothing close. This one is new." and shows nothing.

Her graveyard, counted

I pulled everything public: 650 posts from her blog, 37 columns from Westmount Magazine and 23 pieces from her Substack. That is 710 pieces and 353,232 words.

Then I counted the links she had made by hand from one of her pieces to another.

526 of the 710 have nothing of hers pointing at them.
On her blog she linked back to an earlier post 179 times. 150 of those pointed at something from the previous 30 days.
In 16 years she reached back more than a year 9 times.

Left to memory, a writer links to what she wrote last week. The poster was right, and it was more right than she knew: three of the six pieces on her tombstones are in the library, and all three are in the ground.

Demo

Live page: moxie-library.vercel.app

My mom said yes to being the example, and to the page quoting her. Her text is hers, every quoted passage links back to where she published it, and the page asks search engines not to index it, so it does not compete with her own blog.

Things to try:

Type a couple of sentences about a kitchen disaster. In the recording I typed "I tried a new recipe tonight and set off the smoke alarm twice. The kids ordered pizza before I had finished apologising to the neighbours." Those are my test sentences, not hers. What comes back is hers: five pieces written between 2013 and 2016, starting with the one about why she should never try to cook red meat again.
Type something a humour writer in Montreal has probably never covered. I used the Treaty of Westphalia. You get "Nothing close."
Turn your Wi-Fi off and type another line. It still answers.
Click any tombstone along the bottom, or type part of a title in the search box. You get that piece and the three closest to it, any of which could link to it.

Code

JonathanSolvesProblems / moxie-library

A library of one writer's 16 years of work, with a desk that remembers.

The Moxie Library

Everything my mom has published, in one place, with a desk that points her back at what she already wrote.

Live: moxie-library.vercel.app
Demo video: youtube.com/watch?v=VD4t_Ot2BOA
Write-up: jonathanandrei.com/blog/moxie-library-writers-archive-graded-by-her-own-links

Built for the DEV Hacktoberfest Weekend Challenge: Build for a Friend, 2 to 4 October 2026.

Who it is for

Mona Andrei writes humour. She has kept moxie-dude.com since 2010 writes for Westmount Magazine, and has a Substack. In May 2026 she sent me a poster she had made that said "Don't treat your content like a graveyard. Treat it like a library," and asked for two things: all of her writing in one place, and a way to be pointed back at what she had already written when she starts something new.

I built her a social auto-poster that month. It was the wrong thing, and it is not in this repo. This is the thing she asked for.

…

View on GitHub

Her writing is her copyright, so her text is not in the repo. The scripts rebuild the library from her public pages.

How I Built It

Getting 16 years of writing. Her blog is WordPress, and WordPress has a public REST API, so 650 posts came down in seven requests. Substack has a public archive endpoint. Westmount Magazine is also WordPress, but searching it for her name returns articles about her as well as by her, so I kept a piece only when "By Mona Andrei" appears in its first 200 characters. That dropped five, including the one announcing she had won Humour Blogger of the Year.

Cutting it into passages. Each piece is cut on paragraph boundaries into passages of about 120 words. 2,809 of them.

The model. all-MiniLM-L6-v2, an open-weight embedding model, 23 MB at 8-bit. It runs through transformers.js on WebAssembly, in a web worker, inside her browser tab. The whole index of her writing is 2,809 vectors of 384 numbers each, stored as one byte per number: 1.08 MB.

When she pauses typing, the draft is cut into passages the same way, each one is embedded in the tab, and every piece is scored by its closest passage. That is all the matching there is.

I picked the model by testing three on her own links. I ran all-MiniLM, nomic-embed-text and EmbeddingGemma through Ollama on my laptop's GPU. On an early version of the test described below they scored 7, 10 and 9 of 19, which is inside the noise. Then I tried the best scorer, nomic-embed-text, in the runtime the page uses: over a second per passage on a CPU. The smallest took 78 ms. So the smallest shipped.

A second open model picks one line per piece. Gemma 3 4B, running locally through Ollama, reads each piece and proposes the one sentence most worth quoting. Then a string comparison decides. The line is kept only if it can be found word for word in the piece, and what gets stored is the span cut from her text, not Gemma's retyping of it. It was asked about 694 pieces. 560 lines were kept, 131 of them only after one correction. 134 were thrown out: 23 were not word for word hers, 18 were the wrong length, 91 ran on past the limit, and twice it said there was no line, which is an allowed answer. For the other 9 pieces the local server returned an error every time, so they have no line. The page shows a line only where one was kept.

One bug worth passing on. transformers.js 4.3 loaded the model from local files and then failed with this.tokenizer is not a function. It checks whether tokenizer files exist only when the local model path is not a full URL. I had passed new URL('./models/', location).href. Passing .pathname instead fixed it.

How well does it find things? She graded it.

I did not write the answer key. She did, between 2011 and 2026, every time she linked one post to another.

The test: take a post where she linked back to an earlier one. Remove the link and its text. Treat what is left as a draft. Ask for every earlier post, ranked. See where the one she actually linked to lands.

Most of her links are no test at all, because 150 of 179 point at last month and "show the newest post" finds them. The ones that matter are the 19 that reach back more than 90 days.

Method	Her real target in the first five
Newest post first	1 of 19
Keyword search (BM25)	7 of 19
The model in the page	9 of 19

Nineteen is a small number, and nine against seven is not a difference I would defend. What it shows is that a 23 MB open model in a browser tab does at least as well as keyword search at the one thing she cannot do from memory. I also tried fusing the two (10 of 19) and did not ship a second ranker for one extra hit.

Those numbers come from the exact vectors the page loads, not a separate run.

Two things I got wrong on the way

My first rule for "nothing close" was a similarity level that nine in ten of her real links cleared. It was useless. The nearest unrelated passage usually scores higher than the post she actually linked, so the page would never have said no. The rule that shipped comes from her archive instead: for every piece she has published, how close was the closest earlier one? 19 in 20 clear a certain level. A draft below that level is further from her past work than almost anything she has ever published, and that is when the page says the subject is new.

The second one is the first paragraph of this post. I built the scheduler.

Checks that fail out loud

check_site.mjs opens the page in real Chromium and checks what the page promises: the model loads from local files, a passage of hers finds its own piece first, the quoted passage is word for word from the library, the Treaty of Westphalia gets "Nothing close", the page still answers with the network switched off, any piece can be opened from the keyboard, and not one request leaves the page's own origin. 14 checks.

check_claims.py reads every number in the README and in this post and compares it with the data. On its first run it caught a stale word count in my own plan.

Why Does Open Innovation Matter?

Her drafts are the input. An unpublished draft is the one thing a writer does not hand to a server she does not control, and here there is no server to hand it to. The model's files sit beside the page. The page counts its own requests to any other server and prints the number at the bottom. It is 0.

It works on a plane. After the first load the page needs nothing.

It costs nothing to run, which matters more than it sounds. My mom is not going to pay a monthly fee to search her own writing, and I am not going to pay per request so that she can.

And I could choose the model by measuring it against her, on my own laptop, in an afternoon. Three open models, her 19 links, pick the one that is small enough for her browser.

Where would closed have been better? Probably on raw accuracy. I did not run a closed embedding model against her links, so I am not going to claim open won that. What open did that closed could not do is run inside her tab, offline, with her draft going nowhere.

What she said

Nothing yet. She said yes to being the example, but she has not sat down with it, so there is no quote here. I would rather leave that blank than write one for her, which is also the rule the page follows.

What it cannot do

It matches subjects, not jokes, and it cannot tell whether a link would be welcome. It reads passages of about 120 words, so a one-line aside can slip past. Three of the outlets on her poster are not in it yet, because their sites would not let me read them. And 19 links is a small test.

Prize Categories

Best Use of Gemma. Gemma 3 4B runs locally through Ollama and proposes one pull line per piece, and a word-for-word check decides which ones are kept. EmbeddingGemma was one of the three models I tested against her links. To be clear about the rest: the model that does the matching in the page is all-MiniLM-L6-v2, not Gemma.

Credits

Every word quoted in the library was written by my mom, Mona Andrei. Read her where she published it: her blog Moxie-Dude, her Substack Single Moms with Moxie, and her column in Westmount Magazine.

I wrote up the build in more detail on my own site: My mom has published 710 pieces in 16 years. Nothing she wrote links to 526 of them.

The turquoise, the charcoal and the Raleway type are the ones on her blog. Her words are set in Literata. Both fonts are SIL Open Font License.

Will It Focus: 21 of 22 Quotes Word for Word From the Manufacturer's Own Documents

JonathanSolvesProblems — Sun, 27 Sep 2026 16:33:31 +0000

This is a submission for the Sanity Challenge, Path One: Ship an Agent That Queries Real Content

What I Built

When I was working in film, my Canon T5i would sometimes have trouble focusing in auto mode, and I often just switched to manual and did it myself.

Canon's own manual explains it. Page 100: "In Basic Zone modes, the camera will normally focus the closest subject automatically. Therefore, it may not always focus your target subject."

Will It Focus is an agent that answers that kind of question for any body, lens and adapter in its dataset: will they autofocus together, separately for viewfinder photos, live view and video, and if focus still misses, why. It works across mounts and adapters, so "Sigma 35mm Art on a Sony a7 IV through the MC-11: does AF-C work?" gets a real answer too.

The rule it lives by: a quote is only shown as the manufacturer's words if it is. Every quote the agent returns is looked up in the Sanity record it cites, by code, after the model is done. If it matches a whole sentence of that record, it lines up like a split-image focusing screen coming into focus. If the model reworded it, the page says so and shows the record's actual sentence underneath. If no manufacturer covers the combination, it refuses instead of guessing that an adapter that fits will also focus.

Demo

Live, no login: https://will-it-focus.vercel.app

The four example questions are saved answers, so they load instantly. Type your own for a live run through Sanity Context.

The Sigma answer above quotes the actual row from Sigma's MC-11 compatibility table, 35mm F1.4 DG HSM ○ × ×, with a link to the page it came from.

And this is what happens when nobody has published an answer. No verdict.

Code

https://github.com/JonathanSolvesProblems/will-it-focus

studio/schemaTypes/: the schema
data/build-records.mjs: builds every record from its source, with the verbatim quote
scripts/check_quotes.py: fails if any quote in the dataset is not on the stated page of its PDF, or not in the collected page text for web sources
scripts/check_claims.py: fails if a number in this post or the README disagrees with the data
web/src/agent/: the agent, the quote gate and the cost guards
eval/: 20 questions, three runs each, and every answer the agent gave

A longer write-up of the build is on my site: https://jonathanandrei.com/blog/will-it-focus-camera-autofocus-manufacturer-documents/

How I Used Sanity

Two Sanity Context endpoints, one agent. The agent connects to both over MCP:

will-it-focus serves the dataset in GROQ mode. This is where verdicts come from.
will-it-focus-kb serves a Knowledge Base built from six manufacturer PDFs: the T5i manual (388 pages), both 18-55 kit lens instruction sheets, Canon Canada's 2013 T5i launch release, and Sigma's two MC-11 compatibility tables. Each file counted as one document against the 150 limit, including the 388-page manual.

I started with one endpoint and attached both sources. Saving the dataset replaced the Knowledge Base rather than adding to it, so I split them. Both tool sets get a prefix so the agent can tell dataset_initial_context from kb_initial_context.

The schema. 131 documents across seven types: 35 bodies, 34 lenses, 10 mounts, 5 adapters, 3 sensor formats, 36 compatibility records and 8 focus caveats. A few decisions that turned out to matter:

Autofocus is split by how you shoot (viewfinderPhoto, liveViewPhoto, video), because the same body focuses with different hardware in each.
A compatibility record is only as wide as its source. Canon's note on the 18-55 STM names four bodies, so the record references four bodies. When Sigma's table does not name bodies, the record references none, which the agent reads as "not limited to particular bodies."
Every field a source might not state accepts "not stated by any source". A schema that forces yes or no forces the model to guess.
Each fact carries a source object: URL, publisher, page, retrieved date, and the exact sentence it rests on.

Where the Knowledge Base fits. It is good at explaining. It rewrote the six PDFs into ten navigable entries, and the agent uses them for the why. It also rewrites, which is its job, so an entry is a summary and not Canon's words. The page labels every Knowledge Base point that way. The first time I ran the agent on the Knowledge Base alone, it put quotation marks around 13 sentences, and none of them appear in any of the six documents. That is what pushed me to keep the verbatim text in typed records.

Measured. 20 questions whose answers are in Sigma's, Canon's and Metabones' own tables and manuals, run three ways on gpt-5.4-mini:

Sanity Context sources	Verdicts matching the manufacturer	Quoted passages found word for word in a manufacturer document
Knowledge Base only	37 of 43	1 of 10
Dataset only	39 of 43	28 of 31
Both (what the app runs)	38 of 43	21 of 22

The verdicts are close either way. The difference is whether the words in quotation marks belong to the manufacturer.

The schema fixes the eval found. Two of my own mistakes showed up in the numbers:

I first stored Sigma's DMF column inside a free-text note. The agent could not query it and answered "not covered" on every DMF question. It is now a typed field.
The smaller model missed records because "Sony a7 IV" did not match the alias "a7 IV", and it then guessed an id. Bodies and lenses now carry the names people actually type, and the agent is told never to construct an id. Most of its misses went away, which is how the dataset-only row above got to 39 of 43.

What it does not do. The gate proves a quote is in the record it cites, and checks each AF state against the cited records. It does not prove the agent cited the right record for your exact combination. The dataset is small: the T5i era of Canon, Sigma's MC-11, Metabones' Mark V and Canon's EF-EOS R adapter. Five flange distances come from Wikipedia because no manufacturer page states them, and those records say so.

Sanity Project Details

Project ID: qnl9jh8n
Dataset: production (public). Example: every compatibility record
Studio: https://will-it-focus.sanity.studio

A maintainer, me, and Sentry's own AI all reached for the same wrong fix

JonathanSolvesProblems — Wed, 19 Aug 2026 23:35:04 +0000

This is a submission for DEV's Summer Bug Smash: Smash Stories powered by Sentry.

Three people tried to fix the same memory leak in Sentry's Python SDK. One was a Sentry maintainer. One was me. One was Sentry's own AI debugger.

All three fixes were wrong, and they were wrong for the same underlying reason. That convergence is the actual story, and it took me a while to see it, because I was too busy being pleased with my own version.

The leak, briefly

DedupeIntegration remembers the last exception it saw so it can drop a duplicate report. It stores a weak reference, because holding the exception would hold its traceback, and a traceback holds every frame, and a frame holds every local variable in it.

Then there is this:

# we can only weakref non builtin types
try:
    integration._last_seen.set(weakref.ref(exc))
except TypeError:
    integration._last_seen.set(exc)

You cannot weak-reference a ValueError. So the fallback runs, and it does the exact thing the weak reference existed to prevent. It lives in a ContextVar, so under asyncio every task keeps its own copy: one retained exception, one traceback, one full set of frame locals, per live task.

Measured across 200 long-lived sessions holding a megabyte each: 205 MB retained, all 200 still reachable after gc.collect().

That part I have written up in full elsewhere. This post is about the three fixes.

Attempt one: the maintainer

Digging through the fork's branch list, I found two abandoned branches from September 2025, neither merged. The first replaces the stored exception with a SHA-256 fingerprint over (type_module, type_name, id(exc_value)).

It is a reasonable instinct. Store a cheap value, not the object. The problem is id().

Not retaining the exception is precisely what frees its address, and CPython reuses freed addresses immediately. Over 2000 distinct, sequentially allocated ValueErrors:

distinct exceptions created : 2000
distinct fingerprints       : 2
address-reuse collisions    : 1998

Two fingerprints for two thousand distinct errors. Every collision is a real error that gets silently dropped as a duplicate. The fix trades a memory leak for losing almost every error you report, which is worse than the leak.

Attempt two: me

I did not see that branch until much later, which is lucky, because I made the same class of mistake independently.

My idea was a value fingerprint with no id() in it: exception type, message, and the origin frame from the traceback. All immutable primitives. Nothing retained. I was confident enough to post it publicly on the issue.

The SDK's own test suite killed it in two different ways within about a minute of running.

test_breadcrumbs captures two distinct, never-raised ValueError() instances. No traceback, no message, so both fingerprints are identical and the second event is dropped.

test_option_before_breadcrumb is worse. It calls the same function three times, each raising a separate ValueError("aha!") from the same line. Same type, same message, same origin frame. Three identical fingerprints, two events wrongly deduplicated.

The flaw is not in my choice of fields. It is structural:

A value fingerprint cannot distinguish "the same exception object captured twice" from "the same error raised twice from the same line."

The first must deduplicate. The second must not. By value, they are identical. No amount of extra fields fixes that, because the thing being asked for is not a property of the value.

I went back to the issue and said I had been wrong.

Attempt three: the AI

Later I pointed Sentry's own AI debugger, Seer, at the issue in my own Sentry project.

Its diagnosis was genuinely excellent, and better than I expected. It reached past my application code into a third-party SDK, identified the ContextVar strong reference, identified that weakref.ref raises TypeError on builtin exception types, and identified that ContextVars are per-task under asyncio. That last detail had cost me a full debugging round to work out on my own. It cited dedupe.py lines as evidence, so it had gone and read the source rather than guessing from a stack trace.

Then it proposed the fix:

store a hashable identity tuple like (type(exc), id(exc))

id() again. I ran its exact tuple:

distinct errors raised         : 2000
events wrongly dropped as dupe : 1999  (100.0%)

Three independent parties. Two humans and a model. Two different wrong answers that are secretly the same wrong answer.

The thing all three of us did

Every one of us tried to represent identity with a value.

An object's identity is not a property you can read off it. It is the fact of the object existing, distinct from every other object, for as long as it lives. A fingerprint made of type and message describes what the exception is like. An address describes where it currently sits. Neither survives the thing that makes identity useful, which is that two objects that look the same are still two objects.

Once I could say that sentence, the fix was obvious, and it is not a fingerprint at all:

class _DedupeToken:
    __slots__ = ("__weakref__",)

Attach one token to the exception and weakly reference the token. There is exactly one per exception object and it lives exactly as long as the exception does, so comparing tokens is comparing identity. The ContextVar holds nothing that keeps a traceback alive, and deduplication behaviour does not change at all.

Same 200 sessions, after the fix: 0.7 MB, nothing pinned.

The part I did not expect

There is a coda that I only found while going back through a screen recording, and it is the most interesting thing I learned all week.

Seer did not only propose a plan. It also opened a pull request with generated code. The code is not what the plan said.

try:
    integration._last_seen.set(weakref.ref(exc))
except TypeError:
    pass

No identity tuple anywhere. It simply stops storing for builtin exceptions. That does remove the leak, and it has none of the address-reuse problem, but it silently disables deduplication for the exception types most errors actually are. It is also, precisely, the fix the maintainers had already declined on the issue months earlier.

So the plan and the patch disagree with each other, and they are wrong in two different ways.

I want to be fair here, because the diagnosis was the hard part and it got that right, from evidence, without my framing. But if you let an agent open pull requests: read the diff, not the summary. They are not required to match, and here they did not.

What I took away

The comment # we can only weakref non builtin types sat directly above the line that caused the leak, for years. It described the hazard accurately and then the next line walked into it. A fallback is still a code path, and this one ran for the overwhelming majority of real exceptions.

And the more useful lesson: when several competent people independently produce the same wrong answer, that is information. It usually means the obvious framing of the problem is the thing that is wrong, not the people. Three of us reached for a fingerprint because "store something small instead of the object" is a good habit. The habit was fine. The question was wrong.

Everything here is reproducible, including all three failed fixes, at JonathanSolvesProblems/sentry-dedupe-leak-repro. id_reuse_hazard.py runs the maintainer's approach and Seer's, side by side, and prints exactly how many errors each one would lose.

Python can't weakref a ValueError. Sentry's deduplication found out the expensive way.

JonathanSolvesProblems — Wed, 19 Aug 2026 23:14:14 +0000

This is a submission for DEV's Summer Bug Smash: Clear the Lineup powered by Sentry.

Two and a half minutes: the leak, the two fixes that do not work, and what Sentry's Seer made of it. Everything below is the written version, with the parts the video did not have room for.

Project Overview

sentry-python is Sentry's Python SDK. One of its default integrations is DedupeIntegration, which stops the same error being reported twice. The mechanism is about as simple as it gets: remember the last exception you saw, and if the next one is the same object, drop the event.

Remembering an exception is where it gets interesting, because remembering it is exactly what you must not do. An exception holds its traceback, a traceback holds its frames, and a frame holds every local variable in it. Keep the exception and you keep all of that.

The SDK knew this. That is why it stored a weak reference:

# we can only weakref non builtin types
try:
    integration._last_seen.set(weakref.ref(exc))
except TypeError:
    integration._last_seen.set(exc)

The comment tells you the whole story. You cannot take a weak reference to a builtin exception. ValueError, KeyError, TypeError, the ones almost every error actually is. So weakref.ref(exc) raises TypeError, and the fallback quietly does the one thing the weak reference existed to prevent.

Issue #6094 reported it as unexplained memory growth in an asyncio web crawler.

Bug Fix or Performance Improvement

Why asyncio makes it hurt

_last_seen is a ContextVar. Under asyncio, every task gets its own copy of context. So this is not one retained exception process-wide. It is one retained exception per live task, each one dragging along its traceback and every frame local reachable from it.

The reporter's frames held fetched response bodies between 500 KB and 1 MB.

Measuring it honestly

My first instinct was to call it an unbounded leak. I built a worker pool, ran it, and it stayed flat. That was worth finding out before I wrote it down anywhere.

A fixed pool does not grow, because each worker's next error overwrites its previous one. Retention is bounded at (live tasks x payload). The growth story is not errors over time, it is tasks over time: one task per session, per subscription, per connection.

So I measured against live task count instead. Each task fails once, reports it, then stays alive the way a real session handler does:

  live tasks |     retained |   per task |  sessions pinned
-------------+--------------+------------+-----------------
          25 |      26.2 MB |    1047 KB |      25 / 25
          50 |      52.1 MB |    1042 KB |      50 / 50
         100 |     103.3 MB |    1033 KB |     100 / 100
         200 |     205.4 MB |    1027 KB |     200 / 200

1024 KB retained per live task, dead straight, and sessions pinned counts weak references to session objects that are still reachable after gc.collect(). Not a sampling artifact. The garbage collector cannot touch them because a live ContextVar genuinely still points at them.

At 200 concurrent sessions that is 205 MB that never comes back.

The fix I got wrong first

The maintainers had already rejected the obvious fix. The reporter proposed skipping dedupe for builtins, and Sentry declined, saying they wanted "a more robust fingerprinting approach that can also be used for built-in exceptions" instead.

So I proposed a fingerprint: exception type, message, and the origin frame from the traceback. All immutable primitives, nothing retained. I posted it on the issue.

Then the existing test suite told me I was wrong, in two different ways.

test_breadcrumbs calls capture_exception(ValueError()) twice with two distinct, never-raised exceptions. No traceback, no message, so both fingerprints are identical and the second event gets dropped.

test_option_before_breadcrumb was worse. It calls the same function three times, each raising a separate ValueError("aha!") from the same line. Same type, same message, same origin frame. Three fingerprints, all identical, two events wrongly deduplicated.

That is the flaw in the whole idea. A value fingerprint cannot tell "the same exception object twice" apart from "the same error raised repeatedly from the same line." The first must deduplicate. The second must not. No fingerprint distinguishes them, because by value they are the same.

I went back to the issue and said so.

The attempt that came before mine

Late on, I went looking through the fork's branch list and found two abandoned maintainer branches from September 2025: antonpirker/dedupe-integration-memory-usage and antonpirker/make-dedupe-integration-more-memory-efficient. Neither was merged.

The first one fingerprints on (type_module, type_name, id(exc_value)).

That id() is the interesting part. Once you stop retaining the exception, which is the entire point of the change, its address becomes immediately reusable, and CPython reuses addresses aggressively. Modelling that fingerprint over 2000 distinct, sequentially allocated ValueErrors:

distinct exceptions created : 2000
distinct fingerprints       : 2
address-reuse collisions    : 1998

Two fingerprints for two thousand distinct errors. Each collision is a real error that would be silently dropped as a duplicate.

I mention this not to dunk on an abandoned branch, which is a draft nobody shipped, but because it is the third distinct way I watched this problem punish an obvious solution. Fingerprinting on value collapses errors that came from the same line. Fingerprinting on address collapses errors that reused the same memory. The information being fingerprinted just is not sufficient to express object identity.

The fix that works

The problem was never that identity was the wrong key. It was that holding the object was the wrong way to hold identity.

So keep the identity and drop the object. Attach a small weak-referenceable token to the exception, then weakly reference the token:

class _DedupeToken:
    __slots__ = ("__weakref__",)

There is exactly one token per exception object, and it lives precisely as long as the exception does. last_seen is token is therefore equivalent to the old last_seen is exc, while the ContextVar holds nothing that keeps a traceback alive.

Deduplication behaviour is unchanged, which is the part that matters given the maintainers' concern.

Code

The whole change to the hot path:

exc = exc_info[1]

new_last_seen: "Any"
try:
    # We can only weakref non builtin types.
    new_last_seen = weakref.ref(exc)
    is_duplicate = last_seen is exc
except TypeError:
    # Builtin exception. Referencing it strongly here would pin its
    # traceback and every frame local in that traceback for the
    # lifetime of the ContextVar (#6094). Weakly reference an
    # identity token carried by the exception instead, which dies
    # with it and keeps dedupe keyed on exception identity.
    token = _identity_token(exc)
    if token is not None:
        new_last_seen = weakref.ref(token)
        is_duplicate = last_seen is token
    else:
        new_last_seen = exc
        is_duplicate = last_seen is exc

And the token lookup, after hardening (more on that below):

def _identity_token(exc: BaseException) -> "Optional[_DedupeToken]":
    try:
        if exc.__traceback__ is None:
            return None

        exc_dict = exc.__dict__
        token = exc_dict.get(_DEDUPE_TOKEN_ATTR)
        if isinstance(token, _DedupeToken):
            return token

        token = _DedupeToken()
        exc_dict[_DEDUPE_TOKEN_ATTR] = token
        return token
    except Exception:
        # Exceptions can define ``__dict__`` as a property returning anything,
        # or back it with a mapping that refuses mutation. None of that may
        # break event processing, so fall back to the previous behaviour.
        return None

The __traceback__ is None check is deliberate. An exception that was never raised has no frames to pin, so there is nothing to fix and no reason to touch it. That keeps vars(exc) clean for the common case.

Branch: fix/dedupe-builtin-exception-retention

Everything in this post is reproducible. The harness is at JonathanSolvesProblems/sentry-dedupe-leak-repro: the scaling measurement, the traceback probe, the five adversarial checks against my own fix, and the script that verifies the LLM review claim by claim.

My Improvements

Five tests, and the first one fails without the fix, which is the only thing that makes it a regression test:

test_dedupe_does_not_retain_builtin_exception puts a sentinel in the raising frame and asserts it is collectable afterwards
test_dedupe_still_dedupes_builtin_exception re-raises the same object and asserts one event
test_dedupe_distinguishes_equal_builtin_exceptions raises two identical-looking errors from the same line and asserts two events. This is the one fingerprinting broke
test_dedupe_leaves_unraised_exception_untouched asserts vars(exc) == {} for a never-raised exception
test_dedupe_survives_exotic_exception_dict covers the crashes described below

I also verified what the ContextVar actually holds, rather than trusting the memory numbers:

	`_last_seen` holds	1 MB payload after `gc.collect()`
master	`ValueError` (strong ref)	alive
fixed	`ReferenceType` (weakref)	freed

One process note, because it cost me an hour and nearly cost me a wrong claim. Once the fix was committed to a branch, git stash push <file> had nothing to stash, so my "before and after" runs were quietly comparing the fix against itself. Two measurements came back identical and I believed them for longer than I should have. The tell was a probe printing whether the new symbol was importable, which said True in both columns. Baselines have to be taken with git checkout master -- <file>, and the baseline needs to prove it is actually the baseline.

Best Use of Sentry

Sentry features used: Error Monitoring, Releases, custom Contexts and Tags, Issue Grouping, Seer (Autofix root cause analysis).

The interesting part of instrumenting this was discovering that Sentry could not see the bug, and why.

I built a small asyncio service that opens one long-lived task per session, hits one ordinary ValueError, reports it, and keeps the session open. Ran it against a real Sentry project twice, tagged as two releases, dedupe@unpatched and dedupe@patched.

Sentry showed a completely healthy application. One issue, ValueError: malformed frame on session N, 240 events, handled: yes, mechanism: generic. Exactly what you would expect from a service that reports its own handled errors. Nothing in it suggests 62 MB is being retained.

That is the actual lesson. This bug produces no error. The ValueError is deliberate and correctly handled. Error monitoring alone is structurally incapable of surfacing a retention bug, because retention is not an event.

There is a nice detail buried in that first issue, too. The frame local Sentry captured for the raising frame reads session [Filtered], scrubbed by default PII protection. The one object whose retention was the entire problem was the one thing redacted out of the report.

So I used Sentry two other ways.

First, as a measuring instrument. Every error event carries a custom memory context with the retained figure at the moment it was sent. The errors are identical across both releases; the context is not. Comparing dedupe@unpatched against dedupe@patched on the same issue turns an invisible bug into a visible diff, without a single new error being raised.

Second, by making the leak raise something. The demo checks its own retention after the sessions are open, and when session state is still reachable it reports a distinct RetentionLeak issue carrying the diagnosis:

sessions_pinned         : 60/60
retained_mb             : 62.6
referrer_chain          : Session <- frame(handle_message) <- traceback <- traceback <- exception(ValueError)
dedupe_last_seen_holds  : ValueError

That last line is the whole bug in one field, and getting it was its own small lesson. My first attempt read DedupeIntegration._last_seen from the parent task and reported nothing. _last_seen is a ContextVar, so every asyncio task has its own copy and it always reads empty from outside. Reading it from inside the capturing task gives ValueError on the unpatched release and ReferenceType on the patched one. The thing that made the diagnostic hard to write is precisely the thing that makes the bug scale with task count.

release	`_last_seen` holds	sessions pinned	retained
`dedupe@unpatched`	`ValueError`	60 / 60	62.6 MB
`dedupe@patched`	`ReferenceType`	0 / 60	1.3 MB

What Seer made of it

I connected the sentry-python fork to the project and ran Autofix on the RetentionLeak issue. I expected it to stop at my demo code, because that is where the stack trace points. It did not.

Its root cause, verbatim:

DedupeIntegration stores strong exception references in a ContextVar for builtin exceptions, retaining tracebacks and all frame locals (including large session buffers) for the asyncio task's lifetime.

And the supporting chain:

The ValueError exceptions are kept alive because DedupeIntegration._last_seen ContextVar holds a strong reference to each one.

weakref.ref(exc) raises TypeError for builtin exception types like ValueError, so the fallback path stores the bare exception object instead of a weakref.

Each asyncio task has its own copy of the ContextVar (per-task context), so the strong reference to the exception persists for the entire lifetime of each long-lived task.

That is the bug, exactly, including the per-task ContextVar detail that cost me a debugging round of my own to work out. It cited dedupe.py L1-L62 as evidence, so it went and read the SDK source rather than guessing from the stack trace. Its five reproduction steps are accurate enough to hand to someone else.

Then it proposed a fix:

In dedupe.py, change the except TypeError fallback from integration._last_seen.set(exc) to store a hashable identity tuple like (type(exc), id(exc)) instead of the exception object itself.

That fix does not work, and it fails for the same reason the abandoned maintainer branch does. Storing only an identity key is what allows the exception to be freed, and a freed address is immediately reusable. Running Seer's exact tuple over 2000 distinct errors:

distinct errors raised         : 2000
events wrongly dropped as dupe : 1999  (100.0%)

It would trade a memory leak for silently discarding almost every error you report.

There is a wrinkle I only found afterwards, and it is the most interesting thing in this whole section. Seer also opened a pull request against my fork with generated code, and the code is not what the plan said. It does not store an identity tuple at all:

try:
    integration._last_seen.set(weakref.ref(exc))
except TypeError:
    pass

That skips storing anything for builtins. It does remove the leak, and it has none of the address-reuse problem, but it silently disables deduplication for exactly the exception types most errors actually are. It is the fix the reporter originally proposed and that the maintainers explicitly declined, for the reason quoted at the top of this post.

So the plan and the patch disagree with each other, and they are wrong in two different ways. If you let an agent open pull requests, that is worth knowing: the prose it shows you for review is not necessarily the diff it writes.

I want to be fair about what that means, because it is not a gotcha. Seer did the hard part correctly. Diagnosing this required reading past the stack trace into a third-party SDK, understanding weakref support on builtin types, and knowing that ContextVars are per-task under asyncio. It got all three from one event and the source. The part it got wrong is the part that requires knowing how CPython recycles memory addresses, which is not visible anywhere in the evidence it was given.

And it is genuinely interesting that three independent attempts, an abandoned branch by a Sentry maintainer, my own first attempt, and Seer, all reached for a fingerprint or an identity key, and all three are wrong for the same underlying reason. The information available to a fingerprint is not sufficient to express object identity. That the AI converged on the same wrong answer as two humans is more a statement about the problem than about the AI.

What I actually got out of Seer was the thing I would have wanted from a colleague: an independent confirmation of the diagnosis, arrived at from the evidence rather than from my framing of it.

Best Use of Google AI

I had already run my own adversarial pass on the patch: pickling, copy/deepcopy, payload leakage, __slots__ exceptions, and retention through __context__ chains. All five passed.

So I gave the diff to Gemini 3.6 Flash, told it exactly what I had already covered, and asked only for failure modes I had missed, each with a snippet I could run. Then I ran all of them, because a review you do not verify is just a second opinion with extra steps.

It returned six. The scoreboard:

#	Claim	Verdict
1	`__dict__` as a property returning `None` crashes event processing	Confirmed, real crash
2	a `__dict__` backed by a mapping that refuses mutation crashes	Confirmed, real crash
3	a dynamic `__dict__` breaks token identity	Confirmed, fails open
4	the token is visible in `vars(exc)`	Confirmed, real tradeoff
5	a thread race yields mismatched tokens	Refuted, 0 of 200 attempts
6	`__dict__.clear()` loses the token	Confirmed, fails open

Claims 1 and 2 were genuine bugs that my own pass had missed. My except (AttributeError, TypeError) was too narrow, so an exception class doing something unusual with __dict__ could crash Sentry's event processing. The SDK's own contributing guide says integrations must not crash applications, so those were not academic.

class NonDictException(Exception):
    @property
    def __dict__(self):
        return None

# before: AttributeError: 'NoneType' object has no attribute 'get'

Claim 5 was wrong, and I only know that because I ran it 200 times with a threading.Barrier instead of nodding along.

Claims 3 and 6 are real but fail open: you get a duplicate event rather than losing one. That is the correct direction to fail.

Claim 4 is a fair criticism and I have not made it go away. The token lives in exc.__dict__, so it shows up in vars(exc) and makes json.dumps(vars(exc)) raise. I narrowed it to raised exceptions only, and the SDK already does the same kind of thing (sentry_sdk/integrations/django/__init__.py sets _sentry_drf_request_backref on a user object), but it is a real tradeoff and the PR says so rather than hiding it.

Two real crashes found, one confidently wrong claim caught, in a review that took about four minutes. That is a good trade, and it only works if you treat the output as a list of hypotheses instead of a list of findings.

What I took away from this bug

The comment # we can only weakref non builtin types was right there, in the code, for years. It described the exact hazard and then the next line walked straight into it. A fallback path is still a code path, and this one ran for the overwhelming majority of real exceptions.

The other thing: my measurements lied to me twice, in opposite directions. Once when I assumed unbounded growth and a flat graph corrected me, and once when a stash silently did nothing and two identical columns looked like a result. Both times the fix was the same, which was to make the harness prove its own setup before trusting a single number out of it.

Everyone Believes Black Dogs Get Left Behind. I Checked 99,916 Shelter Records. They Don't.

JonathanSolvesProblems — Sun, 16 Aug 2026 12:18:08 +0000

My family's dog died recently. He lived with my mom, I met him the day he was born, and I knew him almost his whole life. It was hard on all of us in the way that is difficult to explain to anyone who has not had it happen.

A little while later my mom brought home another dog. He does not replace the first one, and nobody in my family pretends otherwise. But a house with a dog in it is a different house, and hers is whole again.

That exchange happened about two million times last year in the United States alone, counting dogs only. What I had not thought about until this weekend is that there is a queue for it, that the queue is very long for some dogs and very short for others, and that the difference is not what almost everyone believes it is.

What I Built

Still Here is a live board of every dog Austin Animal Center has an intake record for and no outcome record. As I write this there are 510 of them.

The one at the top is called Pancho. He is a white Cairn Terrier, he was picked up as a stray on May 23, 2025, and he has been there 449 days. 99.7% of the dogs in his breed group were adopted in less time than he has already been waiting.

Every figure in this post is from the board on 16 August 2026. The site recomputes on every load, so by the time you read this the day counts will have moved and some of these dogs will have gone home.

Then I went looking for the reason the dogs at the top of that board are the ones at the top.

The board moves. Between two runs of the pipeline a day apart, nine dogs came off it: seven were adopted, one was reclaimed by an owner who came looking, and one was transferred to a partner rescue. Four new dogs arrived. Baxter had been waiting 71 days and went home on the fifteenth.

The claim I set out to check

If you have spent any time around animal shelters you have heard of black dog syndrome: the belief that black dogs sit unadopted while lighter dogs go home. It is repeated in shelter training material, in local news segments, and in a great many well-meaning adoption posts every October.

It is a genuinely checkable claim, and Austin has published enough records to check it.

I pulled every completed dog stay the city has released since October 2013 and reconstructed how long each one lasted: 99,916 stays, of which 51,413 ended in adoption. Then I took the median wait for each coat color.

coat	median days to adoption	n
Blue	22	2,064
Brown Brindle	16	2,521
Fawn	16	617
Brown	10	7,085
White	10	8,797
Chocolate	10	1,163
Black	9	13,922
Tan	8	6,236
Cream	7	770

Black dogs are in the fast half. Thirteen thousand nine hundred and twenty-two of them, leaving in a median of nine days, quicker than white, brown or chocolate.

So the myth does not survive its first contact with the data. That much matches the published research: a 2016 study by Christy Hoffman and colleagues, in Animal Welfare, looked at nearly 16,700 records across two shelters and found black dogs left slightly faster than average at both.

That study also reports the other half of what I found, which I did not know when I started looking: bully breeds stayed roughly 2.5 to 3 times longer than average. Two Pacific Northwest shelters, a decade earlier, a completely separate dataset from mine.

But there is something much more interesting sitting in that same table, and it took me a second pass to see it.

The colors that wait are the colors pit bulls come in

Look at the top of that list again. Blue, brown brindle, fawn. Those are not random slow colors. They are the coats you picture when you picture a pit bull.

So I split every color by whether the dog was a bully-type breed, meaning pit bull, Staffordshire, or American bulldog:

coat	bully-type	everyone else
Brown	32.5	9
Black	31	8
Tan	30	7
White	29	8
Red	27	8
Chocolate	26.5	8
Blue	26	11
Brown Brindle	25	11

The two columns do not overlap anywhere. Every bully-type cell is between 25 and 32.5 days. Every other cell is between 7 and 11. The fastest bully-type coat is still 14 days slower than the slowest coat outside the group.

Within a breed group, coat color moves the median wait by at most 7.5 days. Changing the breed group moves it by 20.

Black only looked fast in the first table because black dogs are the least likely to be bully-type: 12% of them, against 81% of the blue ones. The color signal was a breed signal the whole time. A black dog that is not a bully breed waits 8 days. A black pit bull waits 31.

Across the whole corpus: bully-type dogs wait a median of 28 days, and every other dog waits 8. That is 3.5 times, on 9,658 bully-type adoptions against 41,755 others.

And it is not a historical curiosity. Of the 510 dogs sitting in that shelter today, 216 are bully-type. Four of the five longest waits on the board right now are pit bulls.

The thing the adoption site does not tell you

Austin publishes, in open data, exactly how long every dog has been in its care. Its adoption site shows none of it.

I checked the listings the city actually serves to adopters. A listing carries breed, sex, size, age, kennel number, and a set of behavior tags. There is no intake date on it, no length of stay, and no way to sort or filter by either. There is a manual "Long-term resident" tag a staff member can tick, but nothing that gives you a number.

So a person browsing Austin's adoption page can look straight at Fritz and have no idea he arrived on May 28 of last year.

Both halves of that are public. They are just never printed next to each other. Still Here is the join.

Demo

Under two minutes, narrated over the live site, with the research citations on screen where the claims are made.

Live board: jonathansolvesproblems.github.io/still-here

The board is a photographic contact sheet, one frame per dog, ordered by how long each has waited. The day count and the glow are that wait: the longer a dog has been there, the more it has burned into the plate.

Every frame links to that dog's real adoption page, so clicking one takes you somewhere you could actually adopt it.

Things worth trying:

Search a breed. Type labrador, husky or brindle. Multiple words narrow rather than widen, so black lab finds 19 dogs. Austin's own site lets you filter by breed but never tells you how long any of them has waited; this does both.
Press play on any dog. Each one reads its own record aloud, and each breed group opens with its own sound: a howl for the hounds, a husky's howl for the spitz breeds, a yip for the toy dogs.
Filter to "Over a year" for the 18 dogs who have been there more than 365 days.
Filter to "Unnamed" for the ones who arrived without a name.
Click any day count to pin that dog to the top, then copy the link. ?dog=7950 is Pancho.

Code

JonathanSolvesProblems / still-here

510 dogs have an intake record at the Austin animal shelter and no outcome record. This is the board.

Still Here

510 dogs have an intake record at the Austin animal shelter and no outcome record. This is the board.

Built for the DEV Weekend Challenge: Dog Days Edition, August 2026.

Live board: https://jonathansolvesproblems.github.io/still-here/

▶ Watch the demo (2 min): https://www.youtube.com/watch?v=RNa5V26TXzI

Write-up: on DEV and on my site.

The longest wait on the board belongs to Pancho, a white Cairn Terrier who was picked up as a stray on May 23, 2025. As of the last data pull he had been there 449 days. 99.7% of dogs in his breed group were adopted in less time than he has already been waiting.

The finding

Black dog syndrome, the belief that a dark coat keeps a dog in the kennel does not appear in Austin's records. Across 99,916 completed dog stays since 2013, black dogs leave in a median of 9 days, faster than white, brown…

View on GitHub

Everything is in the open, including the corpus, so every number here is checkable.

How I Built It

The data has two eras, and the famous one is frozen

Austin publishes intakes and outcomes on Socrata with no API key required. The dataset IDs that every tutorial points at are archives frozen at 2025-05-05. Query wter-evkm today and you get a tidy 173,812 rows, not one of them from the last fifteen months. The live feeds are different IDs and a different schema, and finding that out was most of an hour.

This project reads both. The live outcome feed ships a days_in_shelter column; the archive does not, so each archived outcome is paired with the most recent intake for that animal that precedes it. Dogs cycle through the shelter more than once, and matching on animal ID alone would charge a repeat stray's whole history to a single stay. 652 outcomes could not be paired and were dropped rather than guessed at.

Snowflake does the counting

All 99,916 stays and the 510 current dogs load through an internal stage, and every number quoted above comes out of a view rather than out of application code. The bully-type definition lives in a single SQL function, IS_BULLY(), so the one judgement call in the whole analysis is auditable in one place instead of scattered through Python.

The piece that genuinely wants a warehouse is the per-dog percentile: for each of the 510 waiting dogs, rank its current wait against every completed stay in its breed group. That is 510 live values against 51,413 historical ones, and it is one view.

I also kept the Python implementation and made the two argue. verify.py re-runs every headline question locally and diffs it against what Snowflake returned, across the medians, the counts, the color table, the controlled split, the spreads, and the per-dog percentile for the top 25 dogs. 102 of 102 checks agree. If they ever disagree, one of them is wrong and the number does not go in the writeup.

Finding the faces

The open data has no photographs in it. But Austin's adoption listings run on a platform called Adopets, and its code field is the same animal id the open data uses, so the two join directly.

I checked the join rather than trusting it: of the matched dogs that have a name on both sides, 97% agree, and the handful that disagree are the shelter renaming a dog after intake. Unknown becomes Tinkerbelle. Fat Boy becomes Big Boy. The importer refuses to write the mapping at all if agreement drops below 90%, because a wrong face on a real animal is worse than no face.

377 of the 510 have a real photograph, embedded from the shelter's own platform rather than copied.

133 have none. I nearly wrote that up as dogs being overlooked, which would have been wrong. Their median wait is 20 days against 84 for the listed ones, so most of them are simply recent arrivals. Texas shelters hold a stray for a period before it can be offered for adoption, set by local ordinance rather than by state law, and that is the likeliest explanation for the gap. I cannot verify it dog by dog from public data, so I am not going to claim it as more than the obvious reading. But the unlisted share never drops below about a sixth at any length of stay, and 33 dogs have passed 100 days without appearing publicly at all. Pancho is one of them, which is why the biggest frame on the page is empty.

For those 133 I generated one illustration per breed group and used it as a CSS mask, so the alpha channel carries the drawing while the fill comes from the coat the shelter recorded. A black pit bull and a white Great Pyrenees stay visibly different animals. They illustrate a breed type, never an individual: generating a portrait of a specific dog nobody has photographed would be inventing evidence about a real animal, and the "no photograph on file" caption stays on every one.

Giving them a voice

Each of the longest-waiting dogs reads its own record aloud, and every clip opens with a real generated sound for that breed group: a mournful howl for the hounds, a husky's howl for the spitz breeds, a tiny yip for the toy dogs, a single deep woof for the guardians.

Nothing any dog says is invented. Every clause maps to a column:

They called me Pancho. I'm a white Cairn Terrier. I was picked up as a stray on May 23, 2025. That was 448 days ago, more than a year. 99.7 percent of dogs like me were already home by now. The usual wait is 8 days. Nobody has come for me yet.

The clips are recordings made on 2026-08-15, so a day count spoken aloud can sit a day behind the board, which recomputes on every load.

Name, coat, breed, intake reason, intake date, elapsed days, cohort percentile, cohort median. Eight fields, no biography, no "loves long walks". The phrasing varies between dogs but is selected by a hash of the animal ID rather than randomly, so re-running produces byte-identical scripts.

The pit bulls deliberately got the warmest, deepest voice available, because 51 of the dogs without photographs are pit bulls and a growling read would reinforce the exact assumption the other 99,916 records spend this whole page dismantling.

A straight top-16 by wait length turned out to be thirteen pit bulls, which meant no howl or yip would ever have played anywhere, so the generator tops up with the longest waiter from every breed group the top-16 missed. Twenty-three clips, all ten groups.

Things I had to get right, and four I got wrong first

Medians, not averages. Length of stay is heavily right-skewed. The mean wait for a bully-type dog is 66 days against a median of 28, because a handful of dogs stay for years. Every headline number here is a median.

The controlled table was wrong the first time. It required only 30 adoptions per cell, and reported that coat color moved the wait by 28 days within bully-type dogs, which would have undercut the entire finding. The top and bottom of that table were Yellow Brindle at n=35 and Gray at n=62. Two thin cells were setting the whole spread. At a floor of 250 the same table shows 7.5 days.

15 of the dogs on my first board had arrived dead. Animals recorded dead on intake never get an outcome row, so the anti-join that finds waiting dogs finds them too, and there they were on a page captioned nobody has come for them yet. Filtering them took the count from 531 to 516.

And one more after that. Princess has two intake rows a day apart with no outcome between them, so she was on the board twice. The anti-join emits one row per intake, not per dog. Deduplicating by animal id took it to 515.

The charts drew nothing at all for a while. Both panels rendered a bar of identical length whatever the number said, because the fill was an inline <span> and inline elements silently ignore width. The comparison the whole page is built on was showing two identical rows.

Because the writeup kept drifting from the pipeline, there is now a factcheck.py that re-reads the data and asserts every number quoted in this post. It caught eight stale figures the first time I ran it.

Prize Categories

Snowflake and ElevenLabs.

Snowflake holds the corpus and answers every question in the piece, with the one judgement call isolated in a SQL function and a 102-check cross-examination against an independent Python implementation.

ElevenLabs gives 23 dogs a voice and gives each breed group its own animal, generated rather than sampled.

What this is not

This is one municipal shelter, and an atypical one. Austin Animal Center operates under a City of Austin mandate of a 95% live-release rate, so these waits are likely shorter than the national picture rather than longer.

Nothing here is causal. It shows which dogs wait, not why any individual adopter chose as they did.

Breed labels deserve a warning too. They are assigned by staff from appearance, and Olson et al. (2015) found that shelter staff called 52% of a sample of dogs pit bull-type while DNA put the figure at 21% (The Veterinary Journal 206: 197-202). So "bully-type" here does not mean a dog with pit bull ancestry. It means a dog Austin wrote down as a pit bull. For a question about who gets adopted that is arguably the better variable anyway, because the label on the kennel card is what an adopter actually reads.

"No outcome record" is also not identical to "in the building". Some of these dogs are in foster care, and the outcome feed lags.

Why I built it

The interesting thing about the black dog myth is not that it is wrong. It is that it is a kinder thing to believe. If dogs are passed over for the colour of their coat, that is a superstition, and superstitions can be argued away with a good photo and a hashtag.

The real pattern is not a superstition. People are avoiding a breed. 216 of the dogs in that shelter tonight are that breed, and the ones at the top of the board have been waiting since before last summer.

Pancho is not one of them. He is a small white terrier and he has been there 449 days, which is the part I cannot explain and did not try to.

When my mom brought that second dog home, one dog stopped waiting. I did not think about it in those terms at the time and I doubt she did either. This weekend I built the list of everyone still waiting, and it turns out to be 510 names long.

Live board: https://jonathansolvesproblems.github.io/still-here/
Code: https://github.com/JonathanSolvesProblems/still-here
Blog Post: https://jonathanandrei.com/blog/still-here-austin-shelter-black-dog-syndrome-99916-stays/
Challenge: DEV Weekend Challenge: Dog Days Edition
Data: Austin Animal Center open data, live intakes and live outcomes. No API key needed, so every number here is checkable.

If you are anywhere near Austin, the board is real and so are the dogs on it.

Passion is the one thing you are not supposed to be able to fake. So I built a machine that manufactures it.

JonathanSolvesProblems — Sun, 12 Jul 2026 01:52:27 +0000

This is a submission for Weekend Challenge: Passion Edition

What I Built

The prompt for this challenge was passion, so I took the most literal possible
run at it.

Every World Cup app ever made is built for people who already care. Fixtures,
scores, brackets, league tables, fantasy teams. Every one of them assumes you
turned up with a side. Nobody builds anything for the person who wants to care
and has nobody to care about, which is most of the planet: a few billion people
will watch this tournament, and the overwhelming majority of them have no dog in
any of it.

Pick Your Side is for them. You name any two nations. It goes and reads their
actual history, picks the one you were always meant to love, tells you why, and a
stadium announcer swears you in.

Two nations in. One side out, and a reason to mean it.

The interesting problem is that you cannot fake this with a nice adjective. If
the app tells you Ghana and Uruguay have a bitter history and the history is
invented, then the passion it hands you is counterfeit, and the moment you look
it up you will feel stupid and never open it again. Manufactured passion only
works if the thing it is built out of is true.

So the whole architecture is bent around that one constraint.

Demo

pick-your-side.vercel.app

Turn your sound on. Half of this app is audio.

Try Uruguay against Ghana. It will side with Ghana, and it will tell you why:
Luis Suarez's handball on the line in 2010, the penalty Asamoah Gyan hit off the
crossbar in the last minute, the first African side ever to stand that close to a
semi-final. It also knows Ghana beat them 2-0 in the group stage a few days ago,
and it still argues the wound has not closed.

Or try England against Germany, if you want to watch it side with the team
that has not won anything since 1966.

Here is the part I am proudest of. Every fact above came from a live search, and
the app will show you exactly which pages it read:

Code

JonathanSolvesProblems / pick-your-side

Best Use of Google AI, DEV Weekend Challenge. Name any two nations and it picks the World Cup side you were always meant to support, then a stadium announcer swears you in. The history is real, not invented: Gemini grounded in Google Search.

Pick Your Side

The World Cup is on and you do not have a team.

Every World Cup app ever built is for the people who already care. This one is for everybody else. Name any two nations. It reads their real history, picks the side you were always meant to love, and a stadium announcer swears you in.

Built for the DEV Weekend Challenge: Passion Edition.

How it works

The interesting part is that none of the history is invented.

1. Gemini researches, grounded in Google Search. The first call has the Google Search tool switched on. Gemini goes and finds the actual head to head record, the actual famous match, and where both sides actually stand in the tournament being played right now. It comes back with the pages it read and the searches it ran, and the app shows you both. Ask it about Morocco and…

View on GitHub

How I Built It

Gemini never gets to work from memory

Because of that constraint, it runs twice.

Call one goes and finds out. Google Search grounding is switched on, and
Gemini is told to report only what it can verify: every World Cup meeting between
the two nations with year and score, the single most famous match, what each side
has actually won, and where both of them stand in the tournament being played
right now.

const research = await ai.models.generateContent({
  model: "gemini-2.5-flash",
  contents: `Research ${a.name} and ${b.name} ... Be concise and factual.
    If they have rarely or never played each other, say that plainly
    instead of inventing a rivalry.`,
  config: { tools: [{ googleSearch: {} }] },
});

Call two writes the case, and it only ever sees the facts call one came back
with. It is held to a responseSchema, so the output is always the same shape:
a side, a verdict, the history, the stakes, one legendary moment, your reason, a
line you can share, and a script for the announcer.

const written = await ai.models.generateContent({
  model: "gemini-2.5-flash",
  contents: `...Here is what was just researched. Every fact you use must
    come from here. Do not add matches, scores or players that are not in it:
    <researched-facts>${facts}</researched-facts>`,
  config: {
    responseMimeType: "application/json",
    responseSchema: revealSchema,
  },
});

Why two calls, which is the bit that cost me an hour

I did not want two calls. I wanted one: grounded and schema'd together. The
Gemini docs say Gemini 3 supports exactly that, structured output combined with
Google Search.

On gemini-2.5-flash, asking for googleSearch and a responseSchema in the
same call does not throw an error. It quietly ignores the schema and hands
you back prose.

That is a genuinely nasty failure. Nothing fails, nothing warns, the call
succeeds, and then your UI does JSON.parse() on three paragraphs of essay. I
only caught it because I wrote a throwaway script to poke at the API surface
before I wrote any UI, and I printed the raw output instead of trusting the
status code. If I had trusted it, the reveal would have exploded live.

So: research first, then shape. Which turned out to be the better architecture
anyway, because splitting them meant the facts arrive with sources attached.

The model was thinking, and thinking was making it worse

The whole thing was taking twenty six seconds, which is a long time to look at a
loading screen. gemini-2.5-flash reasons before it answers unless you tell it
not to. I turned it off on both calls. Same model, same prompts, measured:

	grounded	written	total
thinking on	11.6s	14.7s	26.3s
thinking off	5.1s	1.0s	6.1s

Four times faster. And the answers got better, which I did not expect. Asked
for a verdict, thinking on, it reasoned its way to:

France ends Morocco's World Cup run

That is a news headline. It is not a reason to love anybody. Thinking off:

Unfinished business and a deeper wound

The first call is looking things up and the second is being asked to write.
Neither one is a reasoning problem, and reasoning at them cost me twenty seconds
and the only good line in the output.

config: {
  responseMimeType: "application/json",
  responseSchema: revealSchema,
  temperature: 1.0,
  thinkingConfig: { thinkingBudget: 0 },   // <- this
}

Showing the receipts

Because the grounded call comes back with grounding metadata, I know exactly
which pages Gemini read and which searches it ran. So the app shows you.

Verified. None of this was invented. Gemini ran 6 live searches and read 6
pages before it made the case.

Expand it and you get the real search strings and links to fifa.com, wikipedia,
fotmob, whatever it actually used. You can go check it. It is the difference
between a story and a fact, and it is the thing I would want if I were the one
being told who to love.

Here is what convinced me it was worth doing. I asked it about Morocco against
France, and it told me about the 2026 quarter final, a match played days ago.

"This was tragically repeated in the 2026 World Cup quarterfinal, with France
again winning 2-0."

No language model knows that from training. It went and looked.

Telling it to pick the loser

The instruction that made the biggest difference to how the thing feels:

Choose the side with the better story for a newcomer: the deeper wound, the
longer wait, the thing still unfinished. Do not default to the more famous or
more successful nation.

Without that line it picks Brazil, France, Germany, every time, because they are
the most written about. With it, it picks Morocco over France and argues for the
Atlas Lions. It picks the Netherlands over Senegal and leads with three lost
finals. That single sentence is the difference between an app that reports and an
app that has taste.

The voice

Gemini writes the swear-in. ElevenLabs performs it.

The reveal lands, the nation's colour floods the screen, and a voice comes in
over the top of it. It is the moment the whole thing exists for, and it does not
work on the page as text.

const stream = await client.textToSpeech.convert(VOICE, {
  text: `[shouting over a stadium crowd] ${script}`,
  modelId: "eleven_v3",
  outputFormat: "mp3_44100_128",
  voiceSettings: { stability: 0.35, similarityBoost: 0.75, style: 0.65 },
});

eleven_v3 takes inline audio tags, so [shouting over a stadium crowd] is
direction for the read, not words that get spoken. Low stability plus high style
is what stops it sounding like an audiobook.

And then I listened to it, and the problem was obvious. A man was shouting about
a stadium, in total silence. He sounded unhinged.

So ElevenLabs builds the stadium too. Their sound effects API takes a text
prompt the same way the voice does, and I asked it for two things: a restless
crowd that murmurs under the scouting screen while the app is thinking, and an
eruption for the instant the nation's name lands.

await el.textToSoundEffects.convert({
  text: "A packed football stadium erupting the instant a goal goes in. " +
        "A sudden enormous crowd roar, then it settles into a sustained wall " +
        "of cheering that slowly falls away. No music, no commentary.",
  durationSeconds: 10,
});

The crowd sits at a third of the volume and never competes with the announcer.
Both cues are generated once and committed, so they cost nothing and start
instantly. Sound is on by default, because a muted reveal is just a poster, and
the control to turn it off is on screen the entire time.

There are three layers under the voice, because I did not want a visitor to
arrive at a silent reveal:

live text to speech of the script Gemini just wrote for that exact matchup
if that fails or the free quota is spent, a pre-rendered stinger for that nation, which ships with the app
if both fail, silence, and the reveal is otherwise unharmed

Same matchup, same words, same audio, so it is cached and only the first visitor
to a given pairing costs anything.

The announcer kept turning up late

eleven_v3 is the expressive one. It also takes 4.4 seconds, so the nation's
name would slam onto the screen and then nothing would happen for four seconds
and then a voice would arrive, to an empty room. Worse, I had put the dramatic
pause after the download instead of measuring it from the start of the
reveal, so the network time and the pause stacked and the announcer was
arriving 5.1 seconds late. I only found that because I scripted a browser to
click through the live site and timestamp the moment the audio element actually
started playing, which is not something a screenshot will ever tell you.

There is also a trap in the audio tags. [shouting over a stadium crowd] is
direction, not dialogue, but only on v3. Every other model reads it out
loud. I nearly shipped an announcer who introduces himself by saying the words
"shouting over a stadium crowd".

So the live voice is eleven_flash_v2_5, which comes back in under a second and
lands on the beat, and the pre-rendered stingers stay on eleven_v3, where
latency does not exist because they are generated offline.

The colours were a lie

The nations carry their real kit colours. Germany's is #111111. New Zealand's
is #0A0A0A. The whole design floods a near-black page with the nation's
colour, so picking Germany flooded near-black with near-black and the giant word
GERMANY simply was not there.

Eight of the forty one nations failed to clear 4.5:1 against the background. The
fix was not to fake the colours: it was to keep the authentic one as the source
of truth and derive the painted one from it, lifting it toward white only as far
as it has to go to be legible. Brazil's yellow comes out untouched. Germany's
black comes out a graphite you can actually read.

The design has one idea

The landing page has no colour in it. Near black, white type, a faint green glow
where the pitch would be. The only colour anywhere is a slow ribbon of all 41
nations drifting past the bottom, every side you could end up on, none of them
yours.

Then you pick, and the reveal floods the entire screen in your nation's colours.

That is the whole visual argument, and it is the product in one gesture: you have
nothing, and then you have something. Everything on the reveal is driven off the
chosen nation's real kit colours through CSS custom properties, so Morocco is
red, the Netherlands is orange, Argentina is that pale sky blue.

Flags are images rather than emoji, incidentally, because Windows does not render
flag emoji at all. It falls back to letter pairs. A judge on a Windows laptop
would have seen "AR" where Argentina's flag should be.

Prize Categories

Best use of Google AI. Gemini is the entire product, not a feature of it. It
runs twice: once grounded in Google Search to establish real, current, checkable
facts, and once under a forced responseSchema to turn those facts into an
argument. The app shows you the pages it read. It knows results from a tournament
that is still being played.

Best use of ElevenLabs. The swear-in is the emotional payoff of the whole
app, and it is built out of two ElevenLabs APIs. Text to speech is the
announcer: Gemini writes the words for that exact matchup, ElevenLabs is the
voice that makes them land, timed to arrive just behind the nation's name.
Sound effects are the stadium he is shouting into: a restless crowd while the
app is thinking, an eruption when the verdict drops. Turn your sound on.

Keeping it standing

The Google AI Studio free tier caps a project at a few hundred requests a
day, across every visitor combined. Two calls per reveal means this app could
serve about 125 people a day in total before handing everyone else an error. Fine
for building. Useless for being read.

So two things. The app runs on Vertex AI, which needs a billing account and
therefore carries real quota. And the well known matchups are generated ahead of
time by the same grounded pipeline, sources and all, and committed to the repo.
Those answer in half a second and cost nothing. They are cached, not faked.

When the budget does run out, the failure hands you a matchup that always works
instead of a dead end. I would rather you get a team from a slightly stale cache
than a stack trace.

What I did not solve

The nations list is curated to 41 sides, so you cannot pick your local club. A
matchup nobody has asked for before takes about twelve seconds, which I dressed
up as a scouting sequence rather than fixed. And the "what is at stake" text goes
stale six hours after it is written, which is a strange thing to have to think
about, and only a problem because the tournament is still being played while you
read this.

One last thing

I should probably admit that I am the target user.

The World Cup is being played in my country. Everyone I know has lost their mind
over it, and for the whole tournament I have been standing in the middle of that
with nothing: no nation, no shirt, no grudge, nobody to shout at. I did not build
this as a demo of an API. I built it because I wanted in and could not find the
door.

I pointed it at Uruguay and Ghana, because I did not know anything about either
of them.

It gave me Ghana.

I wrote up the longer version of the build, including the two Gemini traps that
nearly shipped, on my own site.

Built by Jonathan Andrei.

I built an AI agent that runs a local business's weekly marketing (Next.js + Gemini)

JonathanSolvesProblems — Sat, 11 Jul 2026 13:59:24 +0000

For the last couple of weeks I've been building Bloom, an AI marketing agent for local businesses like cafes, salons, and gyms. You set it up once, and every week an agent decides what to feature, writes the content in your brand voice, checks its own work, and (on the paid tier) emails your newsletter to your subscribers. On its own.

It's an early work in progress and I'm sharing it to get honest feedback.

90-second demo: https://www.youtube.com/watch?v=huHCDcUhIGM

The problem

Most local owners get under an hour a day for marketing, so it slips, and posting irregularly quietly costs them reach. It's a time-and-consistency problem, not a talent one. I wanted to build the thing that just never skips a week.

What it does

You enter your business once (type, city, brand voice). Then every Monday a scheduled agent runs with no human trigger and:

decides the week's angle and promotion
writes 3 social posts and an email newsletter in your voice
scores its own draft 0-100 and rewrites it if it falls below the bar
on the Pro tier, emails the newsletter to your subscribers from a verified domain

Every decision is logged to a public activity feed, so you can actually watch it work.

The parts that were actually interesting to build

The self-QA gate is real, not decorative. After generating, a second Gemini call reviews the draft and scores it. Below the threshold, the agent rewrites once and keeps the better attempt. Making this reliable meant forcing a JSON response schema, because Gemini would occasionally return an array instead of an object and silently break the gate.
Structured generation via Vertex AI. Content is Gemini 2.5 Flash through Vertex AI with a forced response schema, so the output is always a clean object.
Idempotent weekly delivery. The sender claims each send atomically, so a newsletter can never go out twice even if two workers fire the same Monday.
Runs on a plain Docker/Traefik box as a standalone Next.js server, with a cron sidecar driving the weekly run.

Stack: Next.js, TypeScript, Gemini 2.5 Flash (Vertex AI), Prisma + Neon, Stripe, Resend, Docker + Traefik.

Where it's at (honestly)

It's live. I've run a real payment through it and had it generate and deliver a real newsletter end to end. But it has basically no real users yet, which is exactly why I'm posting. I'd rather hear what's wrong now.

Try it, and tell me what's missing

Free preview (no card, about a minute): https://bloom.jonathanandrei.com

I'd genuinely love feedback on:

Would you, or a local business you know, actually use this?
Does the generated content look good enough to post as-is?
What would need to be true before you'd pay for it?

Built for the Build with Gemini XPRIZE hackathon. Happy to answer anything in the comments.