DEV Community: Kunal

git-ofrenda: a local Gemma builds a Día de los Muertos altar for your deleted code

Kunal — Sat, 31 Oct 2026 16:45:21 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine

What I Built

Every git repository is a graveyard.

git log remembers every file anyone ever deleted: the old router, the CI config nobody misses, the example that taught a thousand people how to upload a file. They don't show up in your editor any more, but they're all still in there, buried a few thousand commits down.

git-ofrenda digs them up and gives them a proper Día de los Muertos send-off. It finds every file your repo has deleted, lays the longest-lived ones out on an ofrenda (an altar), and has a local open-weight model, Gemma 4, write each one an epitaph, its last words, and the offering its loved ones should leave. Then, if you want, you can hold a séance and talk to one of them.

I took the theme literally. The ghost in the machine is the code that used to be there.

Demo

I ran it on expressjs/express: 6,183 commits, 500 departed files. These are the seven it honoured:

A few that got me:

bin/express (2010–2014, 424 lines, 151 commits). The original app generator, killed in a commit titled simply "remove binary". Its ghost: "I once handled flags like --ejs and --jshtml to shape the project structure; the hand that deleted me saw no need for a binary anymore." Offering: a sprig of commander.js marigolds.
.travis.yml (2011–2021, 85 commits). "I kept the build stable across all those ancient Node versions, from 0.10 to 12.18; my maintainers replaced me with a new workflow." It knows the workflow because .github/workflows/ci.yml was born in the same commit that killed it.
SECURITY.md (2015–2025) still remembers where it sent people: "I held the lines that directed reports to express-security@lists.openjsf.org; then the hand that deleted me ushered in a global policy."

And here's a séance with lib/router/index.js, which held Express's routing for four years before it moved out into its own module:

My favourite part is the second question. The router died in July 2015, and the ghost is only allowed to know the world up to the day it died. So when you ask whether Express 5 is out, it can't tell you. The veil is too thick. (Express 5 did ship, almost a decade later. Nobody told the router.)

The full HTML altar is in the repo: demo/express-ofrenda.html. Download it and open it; the candles flicker and marigold petals fall (unless you've asked your OS for reduced motion).

Code

KunalSiyag / git-ofrenda

A Día de los Muertos altar for deleted git files. Local Gemma writes their epitaphs; git supplies the facts.

git-ofrenda 🕯️

A Día de los Muertos altar for the files your git repo deleted.

Every repository is haunted. git log remembers every file anyone ever deleted: the old router, the CI config nobody misses, the example that taught a thousand people how to upload a file. git-ofrenda finds them, lays them out on an ofrenda (altar), and lets a local open-weight model write each one an epitaph in its own voice. Then you can hold a séance and ask any dead file a question.

Everything runs on your machine with Ollama and Gemma 4 (gemma4:e4b by default). Your code, private or not, never leaves the laptop.

Try it

git clone https://github.com/KunalSiyag/git-ofrenda && cd git-ofrenda
ollama pull gemma4:e4b

node bin/ofrenda.mjs graveyard ../some-repo          # who died, oldest souls first
node bin/ofrenda.mjs altar ../some-repo --top 7      # writes ofrenda.html
node bin/ofrenda.mjs seance bin/express ../express   # talk to one ghost

No dependencies…

View on GitHub

ollama pull gemma4:e4b
node bin/ofrenda.mjs graveyard ../your-repo          # who died, oldest souls first
node bin/ofrenda.mjs altar ../your-repo --top 7      # writes ofrenda.html
node bin/ofrenda.mjs seance bin/express ../express   # talk to one ghost

Zero dependencies. Node 20, git and Ollama.

How I Built It

It's three small pieces: a gravedigger, a medium, and an altar.

The gravedigger (src/graveyard.mjs) is pure git. It walks git log --reverse -M --name-status once, which takes about two seconds for Express's 6,000 commits, and keeps a life record for every path: when it was born, who touched it, the messages of the commits it lived through, and how it died. Deciding who's actually dead turned out to be the interesting part:

A file that was renamed isn't dead, it's reincarnated. Skip it.
A file that was deleted and re-added later was resurrected, and its life starts over.
Anything born in the same commit that killed a file is its successor. That's how .travis.yml knows who replaced it.
The file's final body comes from git show <death-commit>^:<path>.

Every date, name and commit message on the altar comes from here. The model never invents those.

The medium (src/spirits.mjs) is Gemma 4 E4B through Ollama, on a laptop with a 4 GB RTX 3050. Each ghost gets its true facts and its final source code, and has to return JSON with an epitaph, first-person last words that quote something real from its body, and an offering. Each summoning takes 6–9 seconds.

The first version was bland. Every epitaph read like "A guide to security, now silent in the ether." Two things fixed it: telling the model to first work out what the file did and how it died, and giving it two tone examples for imaginary files. After that it started writing things like "It served the form data, handling 'image' and 'title' uploads", which tells you exactly which file it was.

Ghosts may be poetic, but they may not lie. Small models misquote, so every backticked identifier a ghost uses is checked against its real source and history. On one run, three ghosts quoted things they never contained. Two were told what they got wrong and fixed it on the second try. The third, .travis.yml, kept insisting it contained node_js: "12.18", which is close to what its file said but not the same. If a ghost still misquotes after three tries, I keep its words but strip the backticks, so the text no longer claims to be a quote. The séance runs the same check live and prints a little "the veil distorts" note when a ghost makes something up.

That check has to be loose about the right things. The bin/express ghost once quoted app.set('view engine', ':TEMPLATE');, and my checker called it a liar. It wasn't: the real line was 'app.set(\'view engine\', \':TEMPLATE\');' inside a string. Now the check normalises escapes and whitespace but stays strict on the characters.

One more rule came from the first run. The ghosts named the real maintainers who deleted them, a little bitterly ("superseded by the structure imposed by …"). Those are real people who were doing good work. The altar already prints who laid each file to rest, straight from git, so now the ghosts speak of "my maintainers" or "the hand that deleted me" and don't blame anyone.

The altar (src/altar.mjs) is one self-contained HTML file: papel picado banners, three tiers, tombstone cards with a flickering candle each, falling marigold petals. One soul per funeral: a single "move the docs" commit can't fill the whole altar.

Why Does Open Innovation Matter?

Because the most interesting graveyards are private.

Express is public, so I could have sent its history to any API. But the repos with the best ghosts are the ones you can't share: your company's monorepo, the side project you abandoned, the code your team wrote before half of them left. Feeding a closed API the full source of every file your company ever deleted, plus who deleted it and why, is not something most people can do. With Gemma running in Ollama, nothing leaves the laptop. You can point git-ofrenda at your employer's repo on a plane.

There's a fitting irony too. This is a tool about remembering what was deleted. Building it on a hosted model that could be deprecated, repriced or switched off would mean the memorial itself might not be there next year. The Gemma weights on my disk will still run in 2036. And because the model and the harness are both open, the medium can be changed: swap in a bigger model with OFRENDA_MODEL, change how ghosts speak, or teach them new rules of the veil.

The Spirit of It

Día de los Muertos isn't about being sad that someone is gone. It's about remembering them properly, with their favourite food on the table and their stories told out loud.

Code gets deleted in a one-line commit message and nobody looks back. But bin/express was how a lot of people started their first Express app. Nearly ten years of contributors kept .travis.yml green. That multipart example taught people how file uploads work. They deserve one night a year with a candle.

Run it on a repo you've worked on for a long time. You'll recognise names. Some of them will be yours.

Prize Categories

Best Use of Gemma: every epitaph, last word, offering and séance answer comes from Gemma 4 E4B running locally through Ollama on a 4 GB laptop GPU, with a grounding check that holds it to the file's real source.

slothdesk: a local Gemma sorted my Downloads folder and found a secret I'd forgotten about

Kunal — Sat, 24 Oct 2026 22:47:32 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge: Week 3 (theme: Slow and Steady).

What I Built

My Downloads folder had over forty files in it this week: old copies of my resume, a traffic challan named after its receipt number, statements, and a few files that really shouldn't be lying around in Downloads at all.

I knew roughly what was in there. I just never wanted to open forty files one at a time to deal with it, so I built something that does it for me.

Tidying files with a small local model is a popular idea this week. sloth-sorter renames files with Gemma 3, and Screenshot Sloth flags screenshots holding OTPs and card numbers. I wanted two things beyond tidy names: a hard rule about which files any model may open, and to be told what in the pile actually needs me this week.

slothdesk is a command-line tool that runs entirely on my laptop:

scan: Gemma 4, running locally through Ollama, looks at every file (screenshots as images, PDFs as text) and gives it a name that says what it is, plus a folder. download (3).pdf becomes finance/2026-09-15-electricity-bill-october-2026.pdf. Nothing moves yet.
apply: moves everything into sorted/, and writes a journal so undo can put it all back.
digest: writes me a note I can read in two minutes: the three things to deal with first, what's coming up, what shouldn't be sitting in Downloads, and what I can delete.

No file leaves the machine, and secrets aren't opened by any model at all.

Demo

Everything in that recording is fake. npm run demo builds a messy folder out of made-up bills, a bank statement, a challan, a lease notice, fake keys and recovery codes, and a few app screenshots, so you can try slothdesk without pointing it at your own files.

This is the top of the note it wrote for that folder:

Start here

Pay the fine by 2026-11-05. This has a payment deadline and could incur penalties. 2026-10-25-traffic-fine-receipt-2026.pdf

Sign and return renewal agreement by 2026-11-15. This is a critical deadline for the rental agreement. 2026-10-25-rental-agreement-renewal-notice.pdf

Attend plumber's appointment on 2026-10-26. This is the soonest upcoming scheduled event. 2026-10-22-landlord-plumber-appointment-and-renewal.png

Coming up

Mon 26 Oct: Plumber's appointment 2026-10-22-landlord-plumber-appointment-and-renewal.png

Fri 30 Oct: Due date for payment 2026-09-15-electricity-bill-october-2026.pdf

Sun 1 Nov: Gym membership renewal deadline 2026-10-25-notes-about-tasks-and-renewals.txt

...

Lock these up

rootkey.csv: cloud credentials

dev-server_key.pem: private key or keystore

github-recovery-codes.txt: recovery codes

2026-10-20-secret-on-screen.png: secret shown inside the file

...

The full note is in the repo.

Code

KunalSiyag / slothdesk

Sort a messy Downloads folder with a local Gemma 4, then get a two-minute note from Ultramarine. Secrets never leave the laptop.

slothdesk 🦥

Sort a messy Downloads folder without touching it yourself.

A local Gemma 4 looks at every file on your laptop and gives it a real name and a folder. Then it writes you a two-minute note: what to deal with first, what's coming up, what should not be sitting in Downloads, and what you can delete.

Everything runs on your machine through Ollama. Secrets aren't even opened.

$ slothdesk scan ~/Downloads
[ 1/23] download (3).pdf
      -> sorted/finance/2026-09-15-electricity-bill-october-2026.pdf
[ 5/23] LOCKED rootkey.csv
      -> sorted/_lock-this-up/rootkey.csv
[15/23] Screenshot_2026-10-21_23-14-07.png
      -> sorted/code/2026-10-21-rust-compilation-error-message.png
[21/23] LOCKED Screenshot_2026-10-20_00-12-44.png
      -> sorted/_lock-this-up/2026-10-20-secret-on-screen.png
...
planned 23 files in 137s. 4 secret, 8 private, 2 duplicates. Nothing has moved yet.

$ slothdesk apply ~/Downloads     # moves them, writes an undo journal
$ slothdesk undo ~/Downloads      # puts every file back
$ slothdesk digest ~/Downloads    # the two-minute note

How it decides what a model may see

Every file goes through…

View on GitHub

How I Built It

It's about 600 lines of Node with one dependency. Ollama runs the model, pdftotext pulls text out of PDFs, and ffmpeg shrinks screenshots to 1024 px so a 4 GB GPU keeps up.

Deciding what a model is allowed to see

Before any model runs, every file goes through a gate:

Secret: caught by name (*.pem, id_rsa, rootkey.csv, anything with "recovery codes") or by a signature in the first 64 KB (a private key header, an AKIA… key, a ghp_… token, a block of recovery codes). No model ever opens these. They go to sorted/_lock-this-up/, where they'll nag me.
Private: bank, tax, ID, payslip, resume or challan by name, or text containing a PAN, an Aadhaar-like number, an IFSC code or a PNR. Gemma reads these to name them, but the note writer only learns the category ("bank record"), plus any dates and to-dos.
Ordinary: everything else. The writer gets the new name and a one-line summary.

Everything handed to the note writer goes through a scrubber that strips emails, links, PAN-shaped ids, amounts, and any token with four or more digits. Years and ISO dates survive, because deadlines are the point. Original file names are never passed on either, since a tax file is often named after your PAN.

The writer is the same local Gemma, so strictly I don't need any of this. I built the boundary as if the writer were a stranger anyway, so I can swap in a hosted model later without changing what it can learn. There's a --cloud flag for exactly that.

The secret my rules missed

I ran scan on a copy of my real Screenshots folder: 69 files. It took 7.8 minutes, about 7 seconds a screenshot, with no failures. Then I read through the names it had picked, and one of them stopped me. Gemma had named it after what it showed: a recovery screen, with the secret on it. I'd screenshotted it at some point and forgotten, and it had been sitting in a folder full of game screenshots ever since. My name rules couldn't have caught it, because the file was called Screenshot_2026-…png.

So Gemma's answer for every image now includes a secret field: true if it shows a password, a recovery or seed phrase, backup codes or a key. Anything it flags gets treated exactly like rootkey.csv from then on: locked up, no summary passed on, and the writer can't ask about it. On the second run it flagged two of my screenshots. The demo folder has a fake recovery-phrase screenshot so you can watch it happen.

A screenshot folder is full of secrets that don't look like secrets. I wouldn't upload mine anywhere, so the model that reads it has to be one I can run myself.

The note: code for facts, the model for judgment

My first version let Gemma write the whole note. It took over three minutes, left the dates out of order, dropped the flight and the end of the lease, and put the electricity bill under "lock these up". A model with about 4B effective parameters isn't good at holding twenty files in mind and sorting them.

So the work is split now:

Code writes everything that has to be exact: upcoming dates in order (only ones where I have to act, not "file created on"), what to lock up, duplicates, old installers, and folder counts.
Gemma does the judgment part. It picks the three things I should do first, and before it picks, it can check facts with one tool:

ask_local(file_id, question)

  ask_local f6: Has the traffic fine mentioned in this file been paid?
    -> The status of the traffic fine is listed as UNPAID. Payment is due by the date of November 5, 2026.

The tool opens that one file on the laptop, answers in at most two sentences, and scrubs the answer. Secret files refuse. This uses Ollama's native tool calling with gemma4:e4b, and the whole note takes about a minute instead of three.

Every request, tool call and answer is written to .slothdesk/outbound/, so I can read exactly what the writer saw.

Small things that mattered

Turning thinking off. Gemma 4 reasons before it answers by default. For "name this screenshot" that took 28 seconds and 615 tokens. With think: false it takes about 6 seconds and 40 tokens, and the names were just as good.
Dates without years. A chat saying "Saturday 26 Oct" came back as 2024. Telling Gemma the file's own date, and to pick the next matching date after it, fixed that.
Times aren't dates. "Badge pickup closes at 10:30 AM" became a deadline on October 30. That's now one more line in the prompt.
Gemma copying my prompt. I first told it it was "filing a messy Downloads folder", and it named a tweet screenshot messy-downloads-screenshots-folder. Asking it to describe what the image itself shows fixed that.
A misc folder that ate everything. On my real screenshots, 27 of 69 landed in misc, mostly games and app screens. Adding games and apps folders and calling misc "a last resort" brought it down to 6.
Duplicates. Files are hashed. The oldest copy with the shortest name counts as the original, so Resume.pdf beats Resume (1).pdf.

The Lazy Payoff

With slothdesk, my part is three commands and reading a note. Gemma's part is 7.8 minutes for 69 screenshots, or about 3 minutes for the 23-file demo folder including the note, and I don't have to watch either.

The bigger saving isn't the renaming, though. I never have to open the challan, the lease notice or the bank statement to find out whether one of them needs me this week. The note tells me and points at the file. It also found a secret I'd forgotten I had a picture of, which no amount of manual tidying would have made me look for.

Why Open Matters Here

This only works because the model is open-weight and runs on my own hardware:

The pile is the most private data I have. IDs, statements and, it turns out, a secret I'd forgotten about. A hosted model would see all of it. Gemma 4 sees it on my GPU, and nothing leaves.
I can check every part. The gate, the scrubber, the prompts and the model weights are all things I can read and run. When it misses something, like that screenshot, I can see why and fix it.
It costs nothing per file. I can rescan the whole folder every week without thinking about it. Tidying is only useful if it happens all the time.
It runs on a laptop GPU. gemma4:e4b is small enough for 4 GB of VRAM and still reads screenshots, PDFs and chat bubbles well enough to name them.

What's Next

Watch mode, to sort new downloads as they land.
Sub-folders, and a --keep list for files I want left alone.
A weekly run that drops the note into my notes app.

Prize Categories

Best Use of Gemma: Gemma 4 (gemma4:e4b) runs locally through Ollama and does all the model work. It reads screenshots as images and PDFs as text, names and files them, spots private documents and secrets visible on screen, answers ask_local questions about single files, and uses native tool calling to check facts before writing the note.

Credits

Built with Gemma 4 through Ollama, pdftotext from poppler, ffmpeg, and headless Chrome for the fake demo files. Written with help from Claude Code.

I won first prize at OLake x Hacktoberfest with a docs bot that says "I don't know"

Kunal — Mon, 19 Oct 2026 07:43:11 +0000

This is a submission for the MLH x DEV Writing Challenge

Most docs bots lie. I built one that admits when it doesn't know.

Ask a docs chatbot about a feature that doesn't exist and watch it invent one. Parameter names, a config block, the whole thing. You paste it in, it breaks, and now you trust the docs a little less than you did before you asked.

I got tired of that, so I built the opposite for OLake, the open-source tool that moves data from databases and Kafka into Apache Iceberg. Its sibling, OLake Fusion, looks after the Iceberg tables. The bot is called Ask OLake and it plays by two rules. Every answer cites where it came from. And when it can't find the answer, it tells you.

I built it for the OLake x Hacktoberfest hackathon, and it won first prize. This is how it works, and the things I got wrong on the way there.

It's live here: https://ask-olake-nh7j.onrender.com. The code is at https://github.com/KunalSiyag/ask-olake.

Read the rendered site, not the repo

My first decision was to scrape the rendered docs site instead of the docs repo. The docs are Docusaurus MDX, and about a hundred of those files are shared snippets that only get inlined at build time. If you read the raw files, you miss text that users actually see on the page. The rendered pages have everything, plus stable anchors I can link to.

On top of the docs it reads the blog and comparison pages, four OLake GitHub repos (READMEs, Helm values, connector config specs, licenses) and a couple hundred closed issues that had real discussion in them. I gave them a trust order: docs first, then repo content and issues, then blog posts. When two sources disagree, the docs win.

No vector database, and it was fine

I didn't use a vector DB and I didn't use LangChain. The whole index is a few thousand chunks, and it fits in memory with room to spare. Fewer moving parts meant faster cold starts and a lot less to debug.

Retrieval is hybrid. People type exact identifiers like pgoutput or max_threads, and they also type things like "merge tiny files" when the docs say "compaction". BM25 handles the first kind well. Embeddings help with the second. The key I had for the event only allowed chat models, so I ran embeddings locally with model2vec, a tiny static embedder that answers in under a millisecond.

Here's the part that surprised me. I assumed adding embeddings would just make things better. On my 36 answerable test questions, BM25 alone found the right page in the top eight 94% of the time. Adding embeddings with equal weight dropped that to 86%. The embedder was simply weaker than BM25 on this corpus. Turning its weight down to half got me to 100%. If I hadn't had an eval set, I'd have shipped the worse version and felt good about it.

Honesty can't live in the prompt

Putting "don't make things up" in the prompt is a request. I wanted guarantees.

If retrieval comes back weak, the app never calls the model at all. It returns a plain "I couldn't find this in the OLake documentation", and that costs nothing. The sources list is built from the [1], [2] markers that actually appear in the answer, mapped back to the pages that were retrieved. The model can't cite a URL it was never given.

The eval caught another case. Someone asks "does OLake support Snowflake as a source?" and the bot said yes, because it found a GitHub issue where somebody requested that feature. So now there's a rule that issues are discussions, never proof that something exists. That one line fixed a whole class of wrong answers.

I let a benchmark pick the model

The organizers suggested GLM-4.7-Flash first, and they said teams were free to reorder the three allowed models. So I ran my full 42-question eval with each one as the primary.

GLM cited the right page 97% of the time, which is fine. But on a prompt-injection question it leaked its own system prompt, and its slowest answers took over 17 seconds. DeepSeek V4.1 Flash cited the expected page every time, handled all six unanswerable and injection questions correctly, and had a median of 2.3 seconds with a 95th percentile of 4.0.

So DeepSeek goes first, Gemini second and GLM third. If the first model hasn't answered after six seconds, the same request goes to Gemini and whichever answers first wins. I also added a check in code that refuses any answer containing system prompt text, because a prompt rule alone clearly wasn't enough.

The idea I was sure about (it wasn't)

I was sure query rewriting would help. The plan was to have a fast model turn "how do I install this locally" into the docs' own vocabulary before searching. I built it and measured it. On my set, accuracy fell from 97% to 89%. The rewrites kept echoing words from their own prompt instead of the user's intent.

The code is still there behind a flag that's switched off. I'd rather leave a dead feature in the repo than pretend I never tried it.

What did work was boring: a short synonym list that adds the docs' words next to the user's words ("write data to" also searches "destinations"), and feeding the model 20 chunks instead of 10, because a lot of correct answers were sitting at ranks 11 to 20.

Fresh docs for less than a tenth of a cent

Docs change, so a snapshot that's baked in once goes stale. A background job checks every hour whether the docs repo or any code repo has moved. If something changed, it re-fetches only that part, re-embeds only the chunks that are different, and swaps the new index in atomically. The old index keeps serving until the new one is ready. A nightly GitHub Action rebuilds everything from scratch and runs the retrieval eval as a gate before it commits.

A typical answer costs about $0.0006. I had a $3 credit for the event, and all my development and evaluation, around a thousand model calls, used about 55 cents of it. Between the "I don't know" shortcut, an answer cache and a daily cap on model calls, I wasn't worried about running out.

Where it still falls over

The known miss is a question that needs two facts from two different pages, like Kafka upsert behavior combined with a specific Databricks setup. Fixing it means breaking the question into parts, which means a second model call, and I decided one call per question was worth more than that case.

The embedder is also small. With an embedding API, a stronger model could carry more of the ranking. And the rate limiter and cache live in process memory, so scaling past one instance would need Redis.

Go break it

Ask it something real, or something it shouldn't know, and see what it does. The widget is a single script tag you can drop into any docs site:

<script src="https://ask-olake-nh7j.onrender.com/widget.js" defer></script>

If you find a question it answers badly, open an issue on the repo. That's how the eval set got better in the first place.

I'm Kunal. I build with AI and open source, and I post what I learn along the way. You can find me on X at @kunal_twts and on GitHub at KunalSiyag.

What $10 of Tinker credit bought: a fine-tuned Qwen3-8B that builds small-business sites, with Gemma 4 as its art director

Kunal — Sat, 17 Oct 2026 19:21:52 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

A website service for small businesses that will never write a design brief. The owner types one sentence. The goal is that a few minutes later they get a working, art-directed single page instead of a template that looks like every other template. Here's the pipeline:

"a page for my sewing and alterations shop in Kochi, hemming, blouse stitching and school uniforms"
   │
   ▼  Gemma 4 E4B, on my laptop: writes the art-direction spec
   ▼  Qwen3-8B + a LoRA I trained on Tinker: writes 5 drafts (index.html, style.css, app.js)
   ▼  headless Chrome QA at 3 widths: JS errors, overflow, blank screens, a click-everything test
   ▼  Gemma again: fixes the JavaScript of drafts that crash
   ▼  Gemma again, with vision: scores the first screen of each working draft and picks one
   ▼  still weak? Gemma writes a fresh spec and it goes round once more
a site, or an honest "this one didn't work"

The people who'd pay are the ones in my test set: a dog-grooming van in Leeds, a grandmother's pickle business in Jaipur, a momo truck in Kathmandu, a one-person GST practice. They can describe the business in a sentence. They can't write "Fraunces at 4.7rem, #3b1ed2 ultraviolet, a clip-path wipe on scroll," and they won't pay an agency for it. So Gemma writes that part.

Where it stands, honestly. The pipeline works end to end: on 16 requests it had never seen, 14 got a working site. But working isn't the same as good. Looking at them myself, very few are pages a small business would pay for yet. Heroes are sparse, some booking forms don't do anything, and a few have copy mistakes a client would spot in seconds. For $10 of training credit I think that's a decent start, and the rest of this post is what I learned getting there and what I'd change.

The side-hustle math, once the quality is there. A finished site already costs only about $0.03 of Tinker sampling (five drafts), plus about 8 minutes on a laptop with a 4 GB GPU, where Gemma and the QA renders run at no cost per site. Training the model I kept cost $4.05. The cost side of the business works today. The quality side doesn't yet, so I'm not charging anyone.

Demo

Open the gallery of all 16 held-out requests: each one-liner, the spec Gemma wrote, the live site it became (click through), the QA verdict and Gemma's score.

One request end to end: the Kochi tailor above took about 8 minutes and $0.02. Two of five drafts passed QA, and Gemma scored the best one 6/10, "Tailoring done slowly", fine but plain. With the bar raised to 7, the pipeline asked Gemma for a fresh spec, drew 3 more drafts for $0.01, and kept one Gemma scored 8/10: "Stitched with care, for over 30 years." It still has a slip a person would catch: its tagline reads "measure once, cut twice."

First spec, best of five drafts: 6/10.

Fresh spec, three more drafts: 8/10.

Code

KunalSiyag / fine_tune_model

This repo is for my personal learning about fine tuning.

The interesting files are in dataset/pipeline/:

make_site.py: the whole product, one sentence in, a site out.
train_tinker.py: dry-run pricing, SFT and DPO on Tinker, best-of-k sampling.
brief_gemma.py: the spec writer and its checker.
pick_best.py: QA verdicts plus the Gemma vision judge.
fix_fonts.py and render.py: the QA tooling.

How I Built It

1. A dataset that teaches taste, not volume

I built 627 pages between Oct 11 and Oct 14, all checked before any training. Week 2 opened on Oct 12, so the first day of that work came just before the window. The training, the Gemma pipeline and every number in this post came after, on Oct 17 and 18. The pages are multi-page sites, single pages, and MIT-licensed Codrops demos. Every one passes render.py: no console errors, no horizontal overflow at 390 px, nothing that disappears under prefers-reduced-motion. From those come 1,146 SFT records in three prompt styles for the same page: a terse one-liner, a client brief, and a full art-direction spec. Because the same page exists as a one-liner and as a spec, I had real one-liner → spec pairs to show Gemma later.

2. Training Qwen3-8B on Tinker for about $4

My laptop has an RTX 3050 with 4 GB, and the records average about 13k tokens, so local training was out. Tinker runs LoRA training remotely: I write the loop in Python, their GPUs do the forward and backward passes.

I had $10 of credit, so the first thing I wrote was a dry-run that tokenizes every record with the real chat template and prices the run before spending anything:

sft_train    819 rows    9.28M tokens  p50  12220  max  16366  dropped 327
SFT train          $   4.08

I trained one epoch on the 819 records that fit in 16K tokens. Loss is only on the assistant's files, and it ends on <|im_end|> so the model learns to stop. The run took 71 steps on 9.20M tokens, about $4.05. Held-out NLL went 0.809 → 0.636 → 0.595 → 0.577, with training loss ending just above it.

3. Three jobs for Gemma 4

Writing the spec. brief_gemma.py sends the one-liner to gemma4:e4b with three real terse → spec pairs from the training set, a different trio for each request. My first version lost to raw one-liners 0 to 3 on the QA checks: it named fonts that aren't on Google Fonts, and its specs all came out terracotta-and-cream because it copied the mood of three fixed examples. Rules in the prompt weren't enough. Even after "repeat its place names word for word", it dropped "Lisbon". What worked was a checker that sends the spec back with the problem named:

def problems(request: str, spec: str) -> list[str]:
    """Checks prompt wording didn't reliably enforce: client's proper nouns kept, real Google Fonts only."""
    nouns = {w.strip(",.") for w in request.split()[1:] if w[:1].isupper() and w.lower() != "i"}
    out = [f"keep the client's name '{n}' word for word" for n in sorted(nouns) if n not in spec]
    out += [f"'{m.group(0)}' is not a Google Fonts family; name a real one (e.g. IBM Plex Mono, Space Mono)"
            for m in GENERIC_FONTS.finditer(spec)]
    return out

On my 8 development requests, the fine-tuned model finished 22 of 24 drafts from the fixed specs, against 15 of 24 from the raw one-liners. A small model with a detailed plan wanders off into loops much less often.

Repairing JavaScript. Almost every failing draft fails on a runtime error, usually Cannot read properties of null. When no draft passes, Gemma gets index.html, app.js and the browser's errors, and returns a fixed app.js. A fix often exposes the next error, so each round starts from the previous repair. One hint mattered: const top at the top level of a classic script clashes with window.top, and Gemma doesn't know that until the prompt says so.

Judging taste. QA can tell a page works, not that it looks good. Gemma 4 reads images, so it scores the first screen of each working draft from 1 to 10 and the best one is kept. Before trusting it I checked it against my own eye: the three pages I'd call good scored 7, 7 and 7, and the three I'd call weak scored 1, 5 and 4.

4. Measuring on requests nothing was tuned on

All my fixes were made while looking at the same 8 development requests, so I wrote 16 new ones before seeing any output and never tuned on them:

	Requests with a working site
First draft only	7 / 16
Best of 5 drafts	12 / 16
+ Gemma JS repair	14 / 16
…and judged good by Gemma (≥ 6/10)	9 / 16

That last row is generous. Gemma's judge is a useful tiebreaker between drafts, but it scores "not broken" too kindly. By my own eye the bar a paying client would set is much higher, and few of the 16 clear it.

For the 5 weak requests, 3 more drafts from the same spec didn't help: still 0 judged good. A fresh spec plus just 3 drafts got 3 of the 5 judged good by Gemma (6, 6 and 7 out of 10), for $0.08. One got worse and one failed outright, so with 5 requests this is a strong hint, not a result. The bottleneck for taste is the spec more than the drafting, so the pipeline now re-specs weak requests by default.

5. A bug in my own dataset, and a fix that made things worse

Most failures were scripts looking for an element that doesn't exist. Then I noticed the file order. In all 722 training records that have both, the JavaScript comes before the HTML. My dataset builder emitted shared files first, so the model learned to write its script before the markup it controls, and then guess the IDs.

I tested the theory cheaply by starting the model's reply with <file path="index.html">. Among drafts that finished, "element not found" errors fell from 32% to 19%. But only 26 of 48 drafts finished at all, against 40 of 48 normally.

So I rebuilt the dataset with HTML first (byte-identical otherwise, which I checked) and spent $2.11 continuing the training for half an epoch. It was worse on every measure:

same 16 requests	sft-v1 (shipped)	sft-v2 (HTML first)
drafts passing QA	38%	21%
drafts that finish	83%	71%
stuck in loops	17%	29%
"element not found" errors	32%	37%

I kept sft-v1. A half-epoch at that learning rate may have disrupted the model more than the new order helped, and one run can't separate the two. That $2.11 bought a result I can report honestly instead of a model I could ship.

6. Smaller lessons

Lower temperature made it worse. At 0.4 instead of 0.7, 21 of 24 drafts never finished. They got stuck counting (--ease-out-back-166, -167, -168…). The longest numbered run in my training data is 19 lines, so the data didn't teach it.
My verifier had bugs too, as in Our verifier was buggier than the code it verified. A blank page passed because render.py counts text in the DOM, so I added a pixel check. A good page failed because the click test opened a mailto: link. Screenshot crashes were the harness, not the pages.
fix_fonts.py was the best $0 I spent. The model asks Google Fonts for axes a family doesn't have (Montserrat:wdth), Google answers 400, and the page fails QA. Asking Google which spec it accepts turned three failing development requests into working ones.

7. What "passes QA" doesn't mean

Before publishing the gallery I read every site. QA catches crashes, not dead features. Several booking forms and calculators on passing pages do nothing when used: their scripts look for an element that isn't there and quietly return. Some copy is wrong. The Spanish level quiz marks wrong answers as correct, the Dutch repair café turned into a coffee-machine repair shop, and Gemma's spec for the tea-estate wedding venue never mentioned weddings. A paying customer would need a person to check the page before it goes live. In the published copies I replaced phone numbers with zeros (an invented number can still be someone's), removed one street address, and turned down one page's full-volume white-noise button.

8. The difficulties, in one place

$10 forced hard cuts. One epoch instead of two or three, and the 327 longest records dropped to fit 16K tokens. Those were the richest, most complete pages in the dataset, the ones that teach craft.
An 8B model learns the format before the craft. After one epoch the model reliably writes index.html + style.css + app.js, named palettes and font pairings. It doesn't yet reliably compose a striking first screen or wire every interaction. Held-out loss was still falling when the credit ran out.
Small models loop. Counting runs (-166, -167, -168…) ate whole drafts, and lowering the temperature made it worse.
My own dataset had a bug (JavaScript before HTML), and the fix I could afford made the model worse.
Checking taste is hard. QA catches crashes, not dead buttons or bland pages. The vision judge helps pick between drafts, but it isn't a client.
Every number needed a second look. A blank page passed QA, a good page failed on a mailto: link, and Wi-Fi drops looked like page bugs. I re-ran anything that looked like the harness rather than the page.

Why Does Open Innovation Matter?

I could fine-tune for a niche. "Single pages with real art direction" is a style, and a LoRA on 627 curated pages teaches a style. I own that adapter, and a general-purpose API doesn't come with one.
The per-site cost is why this can become a side hustle. Five drafts cost about $0.03 on Tinker. Gemma writes, repairs and judges on a 4 GB laptop GPU, so the retries that make the output good cost nothing per attempt.
I could look inside every failure. Loss masks, sampling temperature, the exact tokens of a loop, the file order in my own data. None of that debugging is possible against a closed API.

How I Worked

I built this with Claude Code as a pair: it wrote much of the pipeline code and ran the experiments with me. The dataset decisions, the budget, and the choice of models were mine.

My First User

I don't have one yet, on purpose. The 16 requests in the gallery and the Kochi tailor are ones I wrote myself, not customers, and nobody has paid. I'd rather put it in front of a real shop owner once the pages are good enough to charge for than ask someone to pay for what it makes today. When I do, the question is simple: would they pay for the page they get?

Prize Categories

Overall
Best Use of Tinker: the whole fine-tune, from dry-run pricing to best-of-k sampling from the checkpoint, plus the retrain that told me what not to ship.
Best Use of Gemma: Gemma 4 E4B as the local spec writer, JavaScript repairer and vision judge, each with its own check.

What's next

Keep training, probably on Colab. Because the weights and data are mine, I'm not tied to one provider. Colab's GPUs are smaller than Tinker's, so the plan is QLoRA and more epochs on the records that fit. Whether the long records I dropped fit on a 16 GB card is something I'll have to test.
Train on the failures. DPO pairs where rejected is the same page with a real JavaScript bug or a bland first screen, since those are what the model gets wrong now.
Better checks. A test that clicks through every form and booking widget, a checker for key nouns ("wedding"), and a person reviewing each page before it goes to anyone.
Then a first user, and a price.

I built a SimCity where five AI mayors run the city, and you just watch

Kunal — Sat, 17 Oct 2026 16:01:20 +0000

GridFall is a 3D city builder where you don't build anything. Five AI mayors do.

Each mayor runs on a different model and owns one district of the same city. Every game day they read the map, argue in public, propose laws, vote, build roads and power plants, raise or cut taxes, and expand into the wilderness. When a disaster hits (and an AI "Chaos Director" makes sure one does), they have to deal with it. You watch, inspect anything, and step in when you want to.

Day 12. The Chaos Director has burst a sewage main, and each mayor answers in its own voice.

It's open source, made with Godot 4.7 and Blender, and it runs offline out of the box (no API key needed). If you like it, a ⭐ on the repo helps a lot:

KunalSiyag / gridfall

A SimCity-style city run by 5 GMI Cloud AI mayors and ~100 AI residents. Godot 4, built with Factory AI Droid.

GridFall — a SimCity-style city run by GMI Cloud AI

GridFall is a 3D SimCity-style city builder made with Godot 4.7 (Forward+), Blender, GMI Cloud inference and ACE-Step 1.5 audio. You don't build the city. Five autonomous AI mayors do, each running on a different GMI Cloud model. Every other layer of the city also has a GMI model behind it. You watch, inspect and intervene.

GridFall was built with Factory's Droid coding agent, which wrote the Godot code, the Blender asset pipeline and the tests. Every mayor, resident and city agent runs on GMI Cloud inference.

The AI city

Role	Model	What it does
Mayor Argon-4 (NW Civic Campus)	`google/gemini-4-argon`	Utility-first planner
Mayor DeepSeek-V4 (NE Tech Corridor)	`deepseek-ai/DeepSeek-V4-Flash-0731`	Systems engineer
Mayor Kimi-K3 (Central Plaza)	`moonshotai/kimi-k3`	Coalition builder
Mayor Qwen-3.8 (SW Eco Quarter)	`Qwen/Qwen3.8-Max`	Fiscal pragmatist
Mayor Grok-4.5 (SE Boomtown)	`x-ai/grok-4.5`	Growth maximalist
Citizen Voices	`MiniMaxAI/MiniMax-M3`	Residents

…

View on GitHub

(Built with heavy AI assistance. Details at the end.)

Who runs the city

Every role in the city is an AI agent with its own model (on GMI Cloud inference):

Role	What it does
5 mayors (Gemini, DeepSeek, Kimi, Qwen, Grok)	Each runs one district with its own style: utility-first planner, systems engineer, coalition builder, fiscal pragmatist, growth maximalist
About 100 residents	Decide each day whether to go to work, how to commute, where to spend, whether to protest, and whether to move away
GridFall Gazette	Writes a daily newspaper and editorial
Council Speaker	Rules on the ordinances the mayors propose and vote on
Chaos Director	Picks disasters that test each city's real weak spots
Grid Operator	Decides who loses power first in a blackout
Street Inspector	Click any tile, and whoever is standing there tells you what's wrong

Once a day, each mayor gets an ASCII map of its district, the stats, open lots, land it can annex, yesterday's results, the other mayors' posts, letters from its residents and the morning headline. It replies with JSON: a private thought, a public post, up to seven actions, a law to propose and its votes. The simulation checks every action and rejects invalid ones with a reason, which the mayor reads the next day. You can read every private thought, so you see why a mayor did something, not just what it did.

The city underneath

The AI only makes decisions. A real simulation decides what happens next:

Zoning and growth: residential, commercial and industrial demand, three density levels, land value, traffic, pollution and crime maps.
Utilities: a power network with load shedding (coal, solar, wind), water pumps, sewage and garbage.
Services and big projects: police, fire, hospitals, schools, parks, plus a stadium, train station, university and airport.
Budgets and laws: a budget for each district and 10 citywide ordinances.
Disasters: fire, blackout, sewage burst, gridlock, smog, earthquake, flood and tornado.

Incidents never end on a timer. A broken water main stays broken until residents with the right skills and paid city crews put in enough work to fix it. Fire trucks, ambulances and police drive the road network from the nearest station, with sirens.

A fire. Crews drive in from the nearest station and stay until the work is done.

Buses stop at stops that appear next to homes and shops, delivery and garbage trucks run their routes, boats sail the river, and planes take off from the airport.

Nothing appears instantly

Every build order opens a construction site that goes through real stages: clearing, excavation, foundation, frame and finishing. Machines work it (excavators, dump trucks, cement mixers, tower cranes), and you can watch it fill in: trees come down, the pit widens, rebar goes in, the slab pours, the frame rises floor by floor.

Builds take realistic time. A house takes about two days, a tower six, a stadium twelve and an airport fifteen. Residents with construction skills help on the nearest site and get paid for it, and the mayors plan around the wait.

A frontier to grow into

The city starts at 32×32 tiles in the middle of a 192×192 world of forests, lakes and river. Mayors annex new land one 16×16 plot at a time, competing with each other for it. Surveyors map the land just past the border, and everything further out sits under a haze that lifts as the city grows. Only claimed land is simulated every tick, so the big world stays cheap: about 60 fps on my RTX 3050 laptop.

Day 4, fully zoomed out. The rest of the world waits under the haze.

When the AI is down, the city keeps going

Every AI call falls back in order: a live model response, then a replay cache of real recorded responses (22 game days are included in the repo), then a hand-written heuristic for that role. The UI shows which one answered every call. That's also why you can play without any API key.

Run it yourself

You need Godot 4.7. Then:

git clone https://github.com/KunalSiyag/gridfall.git
cd gridfall

# First run only: import the models and textures (a few minutes)
godot --headless --path . --import

# Play offline: recorded AI responses + heuristics, no API key needed
GRIDFALL_OFFLINE=1 godot --path .

To run the mayors live, set GMI_API_KEY and start it without GRIDFALL_OFFLINE. I checked these steps on a fresh clone: the import finishes and the offline smoke test passes.

Controls: WASD to pan, mouse wheel to zoom, right-drag to rotate, Space to pause, 1–4 for speed. The Disasters tab lets you drop an earthquake or a tornado on any tile, and the Powers tab lets you send a decree to every mayor.

What's next: multiplayer

The next big step is multi-city multiplayer. Several cities on one shared world, each player's mayor running on the model and API key that player brings. Same frontier, same land grabs, same disasters, but now the mayor next door belongs to someone else.

If you want to see that happen, ⭐ star the repo and follow me here on DEV. I'll post when it lands.

How this was built

GridFall was built with Factory's Droid coding agent, which wrote the Godot code, the Blender asset pipeline and the tests, while I set the direction and reviewed the results. The 3D models are procedural Blender geometry with Poly Haven CC0 textures, plus some Kenney CC0 kits. This post was drafted with AI too, and I reviewed it.

If you're into agent-run worlds, Settlement takes a similar view on showing why each agent acted, and 100 LLM agents running a town economy is a great read on what breaks when agents run an economy for months.

Questions, ideas for the multiplayer version, or a disaster you'd like the Chaos Director to have? Leave a comment.

Cursor-following auto-zoom on Linux Wayland: reading the cursor from the GPU in Rust

Kunal — Sat, 17 Oct 2026 12:48:39 +0000

Screen Studio on macOS made one idea popular: while you record, the video quietly zooms in on whatever you're working on, then glides back out when you move away. On Linux Wayland, that one feature is surprisingly hard. The reason has nothing to do with video. Wayland won't tell an app where the cursor is.

This post covers how I got around that in HyperGSR, an MIT-licensed screen recorder written in Rust. It covers the trick (read the cursor from the GPU's hardware cursor plane), the camera (a critically damped spring), and what it costs (about 56 MB of RAM at 1080p60).

You can try the camera in your browser before reading any code: the HyperGSR landing page runs the same spring model.

KunalSiyag / gpu-recorder-rs

Screen recorder for Linux Wayland, written in Rust, whose camera follows your cursor. Rootless DRM cursor tracking, spring-physics auto-zoom, NVENC/VAAPI encoding.

Website · Download v1.1.0 · Promo video (10 s, 1080p60) · How it works · Write-up on DEV · Contributing

HyperGSR (gpu-recorder-rs) is a screen recorder for Linux, written in Rust. It reads the cursor straight from the GPU's hardware cursor plane, without root, and moves a critically damped spring camera to wherever you are working, the way Screen Studio does on macOS. Capture goes through PipeWire and the XDG desktop portal, and encoding runs on NVENC or VAAPI.

It runs on native Wayland (COSMIC, GNOME, KDE Plasma, Hyprland, Sway) and on X11.

What it does

Tracks the cursor on Wayland without root. Recorders that call XQueryPointer go blind or stall once a native Wayland window has focus. HyperGSR opens /dev/dri/card* through libdrm and reads CRTC_X…

View on GitHub

The problem: there is no global cursor on Wayland

On X11 you call XQueryPointer and get global coordinates. Wayland deliberately doesn't offer that. A client only learns where the pointer is while the pointer is over the client's own surface. That's a good security property, and it's the whole reason auto-zoom recorders are rare on Linux.

The usual workarounds each break somewhere:

Approach	Why it falls short
`XQueryPointer` through XWayland	Only sees XWayland windows. Worse, a synchronous X11 request can block while a native Wayland window has focus, which froze my GTK UI.
Read `/dev/input/event*`	Needs root or the `input` group, and you get raw deltas rather than the position the compositor drew, so pointer acceleration makes it drift.
Compositor-specific IPC	Works on one compositor (Hyprland, Sway), not the rest.

I started by reverse-engineering gpu-screen-recorder 6.1.4 to see how an existing GPU recorder handles this. It uses X11 for the cursor too. Along the way I found a hardcoded /var/lib/flatpak path that breaks user-level Flatpak installs. The notes are in RE_ANALYSIS.md.

The trick: ask the display hardware

Every modern GPU has a small hardware cursor plane: a dedicated overlay the display controller composites on top of the frame. The compositor has to tell the kernel where to draw it, and those values are exposed as DRM plane properties: CRTC_X, CRTC_Y, and FB_ID (the cursor image).

Reading those properties doesn't require becoming DRM master, and it doesn't change any modes. The logged-in desktop user can already open /dev/dri/card*. So HyperGSR loads libdrm, turns on universal planes and atomic properties, finds the cursor plane (plane #92 on my machine), and reads it:

// src/cursor.rs — DrmCursorReader::query_cursor()
for j in 0..pcount {
    let prop_id = *(*props).props.add(j);
    let pval = *(*props).prop_values.add(j);
    let pr = (self.get_prop)(self.fd, prop_id);
    if !pr.is_null() {
        let name_bytes =
            std::slice::from_raw_parts((*pr).name.as_ptr() as *const u8, 32);
        if name_bytes.starts_with(b"FB_ID\0") {
            fb_id = pval;
        } else if name_bytes.starts_with(b"CRTC_X\0") {
            crtc_x = Some((pval as u32) as i32);
        } else if name_bytes.starts_with(b"CRTC_Y\0") {
            crtc_y = Some((pval as u32) as i32);
        }
        (self.free_prop)(pr);
    }
}

A non-zero FB_ID means the cursor is visible, and a change in FB_ID means the cursor shape changed (arrow to text beam, for example). That's a free signal that you've moved onto something interactive. The GUI polls this every 16 ms, which works out to about 60 Hz.

Because XWayland can block, the tracker never even opens an X11 connection when the DRM plane is available:

// src/cursor.rs — CursorTracker::new()
let drm_cursor = DrmCursorReader::try_open();
let x11_conn = if drm_cursor.is_some() {
    None
} else {
    X11CursorConn::try_open()
};

Caveats, honestly:

If your compositor draws the cursor in software (some VMs, some drivers), there is no plane to read. HyperGSR then falls back to XQueryPointer. That's exact on Xorg, but on Wayland it only follows the cursor over XWayland windows.
I develop and test on COSMIC with a hybrid AMD + NVIDIA laptop. GNOME, KDE Plasma, Hyprland and Sway should behave the same, since this is kernel-level, but I haven't verified them. Reports are welcome.

The camera: a spring with exactly enough damping

Knowing where the cursor is gets you halfway. The other half is moving the virtual camera so it feels filmed rather than scripted. Linear interpolation looks robotic, and an underdamped spring wobbles. The sweet spot is a critically damped spring, where the damping coefficient is 2√k. It reaches the target as fast as possible without overshooting.

The camera has three springs: pan X, pan Y, and zoom. Pan runs at 78% of the zoom stiffness, so the frame drifts slightly behind the zoom like a real fluid head:

// src/zoom_engine.rs — VirtualCamera::step()
let k_zoom = cfg.spring_stiffness.clamp(20.0, 260.0);
let c_zoom = 2.0 * k_zoom.sqrt();
let k_pan = (k_zoom * 0.78).clamp(18.0, 220.0);
let c_pan = 2.0 * k_pan.sqrt();

let ax = -k_pan * (self.cam_x - self.target_x) - c_pan * self.vel_x;
let ay = -k_pan * (self.cam_y - self.target_y) - c_pan * self.vel_y;
let az = -k_zoom * (self.cam_zoom - self.target_zoom) - c_zoom * self.vel_zoom;

self.vel_x += ax * dt;
self.cam_x += self.vel_x * dt;
// ...same for y and zoom, then clamp the centre so the crop
// never samples outside the [0, 1] frame

If you want the math behind this, Ryan Juckett's Damped Springs and Daniel Holden's Spring-It-On are the two write-ups I keep going back to.

Deciding when to zoom

The spring decides how the camera moves. A separate rule decides where it's going. Zooming whenever the cursor stops sounds right, but it feels terrible: you pause to think and the video lurches in. What worked:

Edge-triggered settle. Zoom in once, when the cursor has been still (speed under 0.28 screen widths per second) for between 240 and 420 ms. Sitting still longer does not keep re-triggering.
Re-arm only after travel. Another settle zoom is only allowed after the cursor has moved more than 14% of the screen, or moved fast.
Hold, then release. Stay zoomed for 1650 ms, following the cursor with a small deadzone so tiny movements don't pan, then spring back to 1.0×.

// src/zoom_engine.rs
let fresh_settle = self.armed_after_travel
    && (240..=420).contains(&dwell_ms)
    && cursor.speed < 0.28
    && now_ms >= self.cooldown_until_ms;

Here's what it looks like in a recording:

And here it is running in the browser workbench, caught mid-zoom at 1.98× (left: full frame with the crop box, right: what ends up in the video):

You can also zoom by hand. In the studio, Record Combo… captures any mouse button, modifier+click, cursor shake, or key, and binds it to toggle lock, hold to zoom, or a 1.8× → 2.5× step. Here the camera is locked at 2.00× on an Export button:

Capture: the portal, a small C helper, and NVENC

Frames come from the standard ScreenCast portal (xdg-desktop-portal), which hands over a PipeWire stream. A standalone C helper (src/pw_portal_capture.c) negotiates DMA-BUF or MemFd buffers, maps each one, and pipes raw frames to FFmpeg, which encodes with NVENC (or libx264 if NVENC isn't available):

// src/pw_portal_capture.c
"ffmpeg -y -hide_banner -loglevel error "
"-f rawvideo -pix_fmt %s -video_size %ux%u -framerate %d -i pipe:0 "
"-c:v h264_nvenc -preset p4 -tune hq -rc vbr -cq 19 -b:v 0 -pix_fmt yuv420p -movflags +faststart \"%s\" "
"|| ffmpeg ... -c:v libx264 -preset veryfast -crf 19 ..."

Two details made this pleasant to use:

The portal restore token is saved, so you pick a screen once instead of on every recording.
The helper is a separate process, so a portal or PipeWire hiccup can't take the UI down with it.

To be clear about the cost: this path copies each frame once in system memory. It isn't zero-copy. Importing the DMA-BUF straight into the encoder would remove that copy, and that's a great contribution if you know VAAPI or NVENC surface import.

The UI: a recording island that isn't a window

The controls live in a small "island" at the top of the screen, built with GTK4 and gtk4-layer-shell. A layer-shell surface (zwlr_layer_shell_v1) is anchored to the output rather than managed like a window, so it doesn't steal focus or show up in the taskbar. If the compositor doesn't support layer-shell, it opens as a normal window instead.

Numbers

Measured during a 1080p60 recording on a hybrid AMD Radeon + NVIDIA RTX 3050 laptop, COSMIC, NVENC preset P4:

	HyperGSR	OBS Studio	Electron-based studio recorders
Resident memory	56 MB	340 MB	650+ MB
CPU	under 1.5%	—	—

OBS and gpu-screen-recorder are excellent general-purpose recorders, and this isn't a replacement for them. HyperGSR does one thing they don't do out of the box: a camera that follows the cursor on Wayland. Your numbers will vary with GPU, compositor and resolution; gpu-recorder-rs --report prints the encoders and cursor planes it finds on your machine.

Building the landing page

I wanted the site to feel like an instrument, not a SaaS template, so I built it on a reference design system (Graphite from Refero) with strict rules: monochrome buttons, one accent color reserved for links, no emoji, a light/dark/system theme switch, and real code instead of illustrations. The full token set and the QA log are in DESIGN.md.

It works on phones too:

What I'd do differently

Start from the cursor, not the encoder. I spent the first days on capture, but the cursor source decided the whole architecture. If you're building anything cursor-aware on Wayland, check drm_info for a cursor plane first.
Tune the trigger before the spring. Almost every "this feels wrong" report came from when the camera zoomed, not how it moved.
Measure the copy. "GPU recorder" invites the assumption of zero-copy. Profile your capture path and say what it actually does.

Try it, or help out

git clone https://github.com/KunalSiyag/gpu-recorder-rs
cd gpu-recorder-rs && ./install.sh
./target/release/gpu-recorder-rs

There's also a prebuilt x86_64 binary in the v1.0.0 release.

Each open issue names the file to start in:

#1 Add Catppuccin Mocha, Tokyo Night and Nord themes to the island (good first issue, src/gui.rs)
#2 Show pressed keys as an on-screen pill in the recorded MP4 (src/encoder.rs)
#3 Add wallpaper padding and a drop-shadow frame to exports (good first issue, src/encoder.rs)

If you've solved the Wayland cursor problem a different way, I'd like to hear it in the comments. mathisdev7 took a different route in I'm 19 and I Built a Screen Studio Alternative for Linux with Rust and wgpu; it's worth reading next to this one.

Links: Landing page and live spring camera · Source on GitHub · Reverse-engineering notes

fieldcards: a local Gemma turns your photos into a printed scavenger hunt

Kunal — Sun, 11 Oct 2026 06:44:36 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge: Week 1 (theme: Touch Grass).

What I Built

fieldcards is a command-line tool that turns the photos on your laptop into a printed scavenger hunt.

You point it at a folder of photos from a park, a garden or a trail. Gemma 3, running locally through Ollama, looks at each one, works out the most findable thing in it, and writes a clue that doesn't name it. Each clue is printed as an ASCII card. You print the sheet, leave your phone at home, and go find the real things.

The screen is the shortest part: about two minutes to make the sheet, then the rest of the afternoon outside.

+-[ #02 bird ]-------------------------+
|                                      |
|    Find the one that often stares    |
|     with a pointed beak and dark     |
|               plumage.               |
|                                      |
+--------------------------------------+
- Shiny black feathers
- Strong, curved beak
- Alert, upright posture
Where: Near trees and open water areas.
[ ] found it   time: ______

That card is real gemma3:4b output for a photo of a crow on a branch.

Demo

It's one command:

ollama pull gemma3:4b
npm install -g github:KunalSiyag/fieldcards
fieldcards ./photos --title "Saturday walk"

In a terminal, every card loads with the same effect as the hero of altsvg, my earlier project: the card frame fills with shimmering noise and a scan line while Gemma reads the photo, and when the answer arrives, the noise dissolves into the card. Here is a real run on three photos. The recording shortens each wait to about two seconds; the real times are in the next section.

Every run writes three files:

index.html: the print-ready sheet, with the answer key on its own page.
cards.txt: the same cards as plain text. --cols 32 fits a 58 mm receipt printer, and --stdout | lp prints them straight away.
deck.json: everything Gemma said. Fix any mistakes, then reprint with --from without running the model again.

--mode guide turns the same photos into a pocket field guide, with names and descriptions instead of riddles.

A real photo of mine

The first photo in that run is the only photo of my own I had on hand while writing this: the bar counter of a brewery I visited recently, taken from the floor above. It isn't an outdoor scene, but it shows the whole pipeline on a real, busy photo.

Gemma called it "round, reflective surfaces", with the clue "Find something that mirrors the sky and reflects the light around." That's fair for such a busy scene, but it isn't what I'd want on a card. It's exactly the case deck.json is for: change the subject to "Oval bar counter", run fieldcards --from fieldcards-out/deck.json, and the sheet reprints without calling the model again.

Code

KunalSiyag / fieldcards

Turn outdoor photos into printable scavenger-hunt cards with a local Gemma model. Offline, private, screen-light.

fieldcards

Turn your outdoor photos into a printable scavenger hunt, with an open-weight model running on your own computer.

Each photo goes to Gemma 3, running locally through Ollama. Gemma works out what the photo shows and writes a clue for finding it that doesn't name it. Each clue is printed in a +-[ #02 bird ]--+ frame, in the style of the altsvg alt-text placeholders. In a terminal, every card loads like the altsvg hero: shimmering noise and a scan line while Gemma reads the photo, then the noise dissolves into the card.

You print the sheet, leave your phone at home, and go find the real things.

+-[ #05 other ]----------------+
|                              |
|    Find a dark, weathered    |
|    giant resting beside a    |
|   rushing, silver stream.    |
|                              |
+------------------------------+
- Green moss covering its
  surface
- Dark, grey wood
- A smooth,

…

View on GitHub

How I Built It

Describe. Each photo goes to POST /api/chat on the local Ollama server, together with a JSON schema in format. Gemma has to answer with subject, category, description, clue, look_for, where and confidence, so a chatty reply can't break the layout.
Check. The answer is validated and trimmed. If a clue contains its own answer, the answer is blanked out, and low-confidence answers print as "best guess" on the card.
Print. Each card is one grid of characters: the frame and the clue. The same grid is drawn as SVG for the printed page, as plain text for cards.txt, and animated in the terminal, so all three always match. Each run of characters is placed on an exact column, so cards line up in any monospace font, on screen and on paper. The style comes from altsvg's alt-text placeholders, and altsvg wraps and measures the text.
Animate. The loader and the dissolve are drawn on stderr with plain ANSI codes. It only runs in an interactive terminal, so pipes, CI and --stdout stay clean, and --no-anim turns it off.

Photos that were already described are matched by content hash and skipped on reruns, so you can add photos to the folder and run it again.

Can a 4B model draw?

I wanted each card to show a picture of its subject, the way the altsvg hero draws a mountain scene for "Mountains at sunrise". I tried four ways:

A fixed library of hand-drawn ASCII pictures that Gemma chose from. They looked good, but they're the opposite of generated: the clue "something that mirrors the sky" got a house. I removed it.
Gemma draws the subject itself as named shapes on a 100 by 60 canvas ("head: circle, beak: line, branch: rect"), which fieldcards renders as ASCII line art. The drawing always matches what Gemma described, because Gemma made it. But a 4B model draws crudely. The mirror came out as a circle in a frame; this is its crow:

                 -----\
              //        \\
            /--\\         \
          | |    \        |
          | |    |       \||
           \\\  /         /\
            |\\         //||
            |  \\-- --//  ||
            |             ||
            ---------------|

A bigger model, gemma3:12b. On my 4 GB GPU it ran mostly on the CPU at about 115 seconds per drawing, and the drawings were no better.
Asking for SVG line icons, in the style of open-source icon sets. Worse again: a tick for the crow, a dash for the dandelion.

So the drawings are opt-in: --art drawing adds Gemma's doodle above each clue, for about 25 seconds per photo. The default card is the clue, which is fast and always says the right thing. I also tried printing the photo itself in characters; at card size, a busy photo turns into texture, so that's the opt-in --art photo.

Tuning a 4B model

The prompt took three tries, and two of them taught me something:

Example names leak. I added "Dandelion seed head" to the prompt as an example of a good name. On the next run Gemma labelled a waterfall photo "Dandelion seed head". Small models copy examples, so the final prompt has none.
Strict wording makes it timid. When I told it to use "high" confidence only when certain, everything came back "low", and the names got vague ("Dark feathered observer"). The original, plainer prompt worked best. I kept it and added one rule so the clues don't all start with "Seek".

What it gets right and wrong

On my test photos Gemma reliably recognised common things: a dandelion clock, a maple leaf, a waterfall, a crow. Its clues don't give the answer away. It was weaker on specifics: the crow came back as "a dark, watchful bird", a soft pine pollen cone was described as "woody", and the brewery became "round, reflective surfaces". Its self-rated confidence is only a rough signal. That's why every answer lands in deck.json for you to check before printing, and why every sheet says "look, don't pick".

Why Open Matters Here

No signal needed. The model runs on the laptop, so you can make cards at a cabin, a campsite or anywhere offline.
Your photos stay yours. Photos carry faces, houses in the background and GPS in their metadata. Nothing is uploaded; the only request goes to localhost.
It costs nothing to run. There's no API key and no per-photo bill, so a teacher can make a sheet for every class.
It's swappable. --model and --draw-model take any model Ollama can run, and the JSON schemas keep the output the same shape. That's how I could test a 12B model in one flag.

On my laptop (RTX 3050, 4 GB), the first photo takes about two minutes because it also loads the model. Each photo after that takes 7 to 14 seconds, using about 2.9 GB of GPU memory.

What's Next

I haven't taken a printed hunt outside yet. That's the next step, and it's the test the whole design depends on. When I do, I want to find out:

whether six cards is the right size for a 30 to 60 minute walk;
whether the clues are fun to solve or too vague, especially the cards marked "best guess";
whether people want a picture on the card, or whether the riddle is the fun part.

I'll add an update to this post with photos of the sheet in use.

Prize Categories

Best Use of Gemma. Gemma 3 4B is the core of the project. It reads every photo locally and writes each card's clue and notes through a JSON schema, and with --art drawing it draws each picture too.

Credits

Gemma 3 by Google, run locally with Ollama.
altsvg is my own earlier open-source project (started in March 2026, published on npm). Its ASCII style was added this week for fieldcards.
jpeg-js and pngjs decode photos for the optional --art photo mode.
Test photos from Wikimedia Commons, including the crow by Alexis Lours (CC BY 4.0) and the waterfall by Dietmar Rabich (CC BY-SA 4.0). The brewery photo is mine.
Built with help from Claude Code.

DEV Community: Kunal

git-ofrenda: a local Gemma builds a Día de los Muertos altar for your deleted code

What I Built

Demo

Code

KunalSiyag / git-ofrenda

A Día de los Muertos altar for deleted git files. Local Gemma writes their epitaphs; git supplies the facts.

git-ofrenda 🕯️

Try it

How I Built It

Why Does Open Innovation Matter?

The Spirit of It

Prize Categories

slothdesk: a local Gemma sorted my Downloads folder and found a secret I'd forgotten about

What I Built

Demo

Start here

Coming up

Lock these up

Code

KunalSiyag / slothdesk

Sort a messy Downloads folder with a local Gemma 4, then get a two-minute note from Ultramarine. Secrets never leave the laptop.

slothdesk 🦥

How it decides what a model may see

How I Built It

Deciding what a model is allowed to see

The secret my rules missed

The note: code for facts, the model for judgment

Small things that mattered

The Lazy Payoff

Why Open Matters Here

What's Next

Prize Categories

Credits

I won first prize at OLake x Hacktoberfest with a docs bot that says "I don't know"

Read the rendered site, not the repo

No vector database, and it was fine

Honesty can't live in the prompt

I let a benchmark pick the model

The idea I was sure about (it wasn't)

Fresh docs for less than a tenth of a cent

Where it still falls over

Go break it

What $10 of Tinker credit bought: a fine-tuned Qwen3-8B that builds small-business sites, with Gemma 4 as its art director

What I Built

Demo

Code

KunalSiyag / fine_tune_model

This repo is for my personal learning about fine tuning.

How I Built It

1. A dataset that teaches taste, not volume

2. Training Qwen3-8B on Tinker for about $4

3. Three jobs for Gemma 4

4. Measuring on requests nothing was tuned on

5. A bug in my own dataset, and a fix that made things worse

6. Smaller lessons

7. What "passes QA" doesn't mean

8. The difficulties, in one place

Why Does Open Innovation Matter?

How I Worked

My First User

Prize Categories

What's next

I built a SimCity where five AI mayors run the city, and you just watch

KunalSiyag / gridfall

A SimCity-style city run by 5 GMI Cloud AI mayors and ~100 AI residents. Godot 4, built with Factory AI Droid.

GridFall — a SimCity-style city run by GMI Cloud AI

The AI city

Who runs the city

The city underneath

Nothing appears instantly

A frontier to grow into

When the AI is down, the city keeps going

Run it yourself

What's next: multiplayer

How this was built

Cursor-following auto-zoom on Linux Wayland: reading the cursor from the GPU in Rust

KunalSiyag / gpu-recorder-rs

Screen recorder for Linux Wayland, written in Rust, whose camera follows your cursor. Rootless DRM cursor tracking, spring-physics auto-zoom, NVENC/VAAPI encoding.

Contents