DEV Community: Gavin Rose

OnBlink: Communicate with minimal movement

Gavin Rose — Mon, 09 Nov 2026 18:57:17 +0000

This is a submission for the Presage SmartSpectra SDK Challenge

What I Built

OnBlink lets someone speak a phrase or open an app using only their eyes and a regular webcam.

A blink signal opens a menu. The highlight steps through phrases like "Yes", "I need help" or "I'm thirsty", and a second signal picks one. The Mac says it out loud, or opens an app. It's built for people who can't easily speak or type, and for the people around them.

Highlights:

Two ways to select. Double blink (SmartSpectra's built-in blink detection) or long blink (close your eyes for about a second), so people can use whichever signal is easiest for them.
Your own phrases and apps. Add, edit, reorder and test menu items. Each one speaks a sentence or opens an app.
Heart rate check-ins (optional). If someone's heart rate stays well above their own baseline level from earlier in the session, OnBlink offers a short check-in: "I am fine", "I need a break" or "I need help".
Private option prompts (optional). Options can be read softly, or sent to headphones or a single earbud, while the chosen phrase is spoken out loud for everyone.
Guided setup. A 10-step tutorial, a framing guide that waits until your face is in position, and a practice run.

Demo

Check out OnBlink

Code

Repository: https://github.com/gavin-rose/onblink

Setup takes three commands (npm install, npm start, then paste a free Presage API key in the tutorial). The README covers requirements, usage, settings, data handling and how the detector was tuned. OnBlink runs on macOS.

How It Responds to Human Signals

OnBlink uses five live SmartSpectra signals:

Signal	How OnBlink uses it
Blinking	Double blink input: blink, wait for "Blink again", blink once more
Face landmarks	Long blink input, measured from how open each eye is
Pulse rate	Live heart rate display and optional check-ins, using only readings the SDK marks stable
Talking	Ignores eye closures while the person is speaking
Validation status	Drives the framing guide, pauses input when the person moves out of view, and pauses check-ins during motion

Integrating the SDK. OnBlink is an Electron app built on the SmartSpectra Node SDK. The renderer requests faceMetrics and cardioMetrics, decodes each metrics message, and feeds blinks, landmarks, talking and pulse into small, separate modules: the menu, the long blink detector and the check-in logic. The main process handles settings, speech and opening apps, and the page can only ask it to run a saved action by its id.

Long blink. SmartSpectra's blink signal is a short detection pulse, which is perfect for quick double blinks. For hold-based input, I measured eye openness directly from the face landmarks:

Each eye is compared to its own recent level, so it works for people whose eyes naturally open different amounts.
A closure that lasts 0.7 to 2.5 seconds counts as a long blink. It's ignored if the head moves or the person is talking.
Some people find a deliberate one-second close easier than a quick double blink, and in my own testing setup it was the more dependable option. It's also very hard to trigger by accident, which matters when a false trigger means the computer speaks for you.

I tuned the thresholds with labeled recordings: I pressed a key after each blink, long blink and wink, then replayed the recordings through the detector. It caught all 5 deliberate long blinks and didn't fire on the 15 other labeled actions. The analysis scripts are in the repo's research folder.

Check-ins. During the first minute of steady readings, OnBlink learns the person's usual heart rate. If stable readings stay well above that level (15 bpm or 20% for 45 seconds on the default setting), it offers a check-in through the same menu. It only reacts to rises, waits until the person is in position and not using the menu, and appears at most once every 10 minutes. It's an offer to talk, never a health judgment.

Prize Categories

Best Thoughtful Real-World Use

The need. People with limited speech or movement often have the hardest time asking for the simplest things: a drink, help, or a break. Dedicated eye-tracking devices exist, but they're expensive and need special hardware. OnBlink only needs a laptop webcam.

Designed for the people using it:

A choice of signal and pace. Double blink with relaxed or quick spacing, or long blink, plus an adjustable menu speed.
Hard to trigger by accident. Single blinks are ignored, input pauses when the person is out of view, and the menu closes on its own if nothing is picked.
Quick phrases first. Yes and No come first, then urgent needs, so common answers take the fewest steps.
Speaking up without the effort of starting. The check-in gives someone an easy way to speak up when their body suggests something might be wrong.
Considerate of the room. Option prompts can stay quiet or go to an earbud, so only the chosen phrase is spoken out loud.
Doesn't stop on its own. The screen stays awake during a session, and auto stop is off by default, so nobody has to restart it.
Easy for a caregiver to set up. A tutorial with a practice run, plus a settings screen with a Test button for every phrase.

Consent and Data Handling

Consent first. Before the first session, OnBlink shows a plain-language consent screen, also part of the tutorial, explaining what the camera is used for. Nothing starts until the person accepts.
Camera. On only after pressing Start, and off when pressing Stop or at an optional time limit.
SDK telemetry. Turned off (enableTelemetry: false). Presage's telemetry and privacy document says the SDK transmits no telemetry when it's off.
Storage. OnBlink saves only the menu items, settings and the API key (encrypted with the macOS Keychain), locally on the user's computer. Deleting the app's settings folder removes everything.
Never stored or sent. Video, eye or face data, and heart rate. The heart rate level used for check-ins is kept in memory only and discarded when the session stops.
Sharing and AI services. OnBlink shares nothing and uses no AI services. Phrases are spoken locally by macOS, and only when the person picks them.
Optional features. Check-ins are off until the person turns them on.
Not a medical device. OnBlink is a communication aid. It does not diagnose, treat or monitor any condition, and it should not be relied on in an emergency. Check-ins are only an offer to talk.

Agent Session

I built OnBlink with Claude Code: exploring what the SDK could measure, recording and analyzing my own blink data, and building and refining the app. The SDK was pretty easy to understand and use but I definitely leaned hard into Claude for the visuals as that isn't normally my wheelhouse.

Try playing my game!

Gavin Rose — Wed, 28 Oct 2026 19:28:09 +0000

Gavin Rose

Oct 10

Plinth! My first arcade-style game.

#showdev #android #flutter #gamedev

4 min read

Haunt Engine

Gavin Rose — Wed, 28 Oct 2026 14:42:34 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine

What I Built

Haunt Engine is a game where you get to write a prompt that fuels the haunted house you play.

You type a place: "a cozy bakery", "the last video rental store on earth", "the classroom where my grandmother taught". A model running on your own laptop designs a building from it, room by room. It names the rooms and furnishes them. It casts the people who never left, gives them something to say, hides the keys and decides what hunts you. The game builds that design into a top-down level and drops you in the front door with a lantern.

You can talk to anyone you find. The model plays them in character, so a ghost might tell you where the key is, if you ask the right way. Get caught too many times and you die. The model then writes your epitaph from what actually happened: who took you, in which room, and what you were holding. Your headstone stays in that haunt, on the tile where you fell. Next time you go in, you walk past your own grave.

I took the theme literally. The ghost in the machine is the thing that designed the building you're trapped in, and it's also the voice of everyone inside it.

Demo

Haunt Engine runs entirely on your machine, so there's no hosted link; the repo below gets you playing in three commands. The GIF above is the hand-written demo haunt, St. Agnes Elementary, After Hours, on the easiest difficulty, where marigold petals lead you to your next objective.

Code

gavin-rose / haunt-engine

Haunt Engine

Describe a place. Walk through what's waiting in it.

You type a premise ("a cozy bakery", "the classroom where my grandmother taught"). An open-weight model (Gemma, running on your own machine) turns it into a haunted building: rooms furniture, characters with things to say, a chain of objectives, something that hunts you The game builds that into a top-down horror level you play in the browser. When you die, the model carves your epitaph, and your grave stays in that haunt for your next visit.

Everything runs locally. There are no accounts, no API keys and no server besides your own.

What you need

Node.js 20.9 or newer
Ollama 0.30.5 or newer
About 10 GB of free memory and 8 GB of disk for the 12B model. Developed on an Apple Silicon MacBook Pro with 24 GB of memory.
A browser with WebGL (any current Chrome, Firefox, Safari…

View on GitHub

npm install
npm run setup:model   # pulls Gemma 4 12B and gives it a 16k context
npm run dev

MIT licensed. The art keeps its own CC-BY-SA license, and every artist is credited in the game.

How I Built It

The model is Gemma 4 12B (the quantization-aware gemma4:12b-it-qat build) which I ran Ollama locally.

The rule I built everything around: the model designs, code builds. The model never places a tile or touches game state. It writes a blueprint, a JSON document of rooms, doors, items, characters, objectives and scripted events. Ollama's structured output holds it to a JSON schema generated from Zod. Then deterministic code turns that blueprint plus a random seed into a level and enforces every rule.

That split is what made a 12B model good enough. A few things made it work:

Closed vocabularies for anything that gets drawn. My first blueprints asked for furniture I had no art for. Now every floor, wall, piece of furniture and item of clothing is an enum over the art I actually have. The model chooses from 61 props, and it dresses characters from LPC sprite layers: body, hair, coat, hat, wounds. The browser composites and recolors those layers, so the model can invent a character nobody drew and still get a sprite.
Repair instead of retry. Small models get references wrong. They connect a door to a room that doesn't exist, or lock a key behind its own door. Rather than asking again (which costs minutes on a laptop), repair.ts fixes the blueprint deterministically. It reconnects orphaned rooms and moves keys until a fixpoint search proves every room is reachable. It also tops up haunts that are too easy. Early on, Gemma wrote a haunt with two objectives and no way out, and it was over almost as soon as it began. Now every haunt has at least three objectives, ends in an escape from the room farthest from the start, and has something hunting you.
A generator that respects the art. Rooms are laid out by walking the connection graph and drawn in 3/4 view, with real wall faces. Corner autotiling handles the ceiling trim. Furniture has placement rules: desks go in rows and mirrors hang on walls. Lighting is per-tile line of sight, blurred into soft darkness under a cold moonlight grade.
Never pay for the same haunt twice. At about 11 tokens a second, a summon takes five to eight minutes. Every haunt is saved the moment it exists. Typing a prompt again loads it instantly, and a summon keeps generating even if you close the tab.

Two Ollama details did make it difficult to work with. Gemma 4 thinks before answering by default, which made every request far slower; reasoning_effort: "none" turns that off. And Ollama's default 4k context truncates a whole building, so npm run setup:model creates a haunt-gemma alias with 16k.

The rest of the stack is open source, too: Next.js 16, Phaser 4, Zod, and Liberated Pixel Cup art. The sound is synthesized live with the Web Audio API: wind, footsteps that change with the floor, and a heartbeat that quickens as something gets closer. That means there are no recordings to license.

Why Does Open Innovation Matter?

Because a game like this is a terrible fit for an API bill.

Every haunt is roughly 5,500 characters of JSON, and every conversation and epitaph is another call. With open weights, a player who spends an evening summoning twenty haunted houses costs nothing, and nobody needs an account or a key. Clone the repo and the haunted house lives on your machine, along with your graves.

Open weights also let me make the model fit the job rather than the other way round. I could pin the exact build, give it a bigger context window and switch off its thinking. It's held to a schema, and when it slips, the code repairs the blueprint and moves on. None of that depends on a provider keeping a model alive or changing its behavior under me. The haunts you summon this October will be the same buildings next October.

And the art could only come from an open commons. Every room, prop and person in the game was drawn by the Liberated Pixel Cup community and shared so that projects like this could exist. The model can dress a character in anything from that commons because the commons is open.

The Spirit of It

Día de los Muertos falls right after Halloween, and the game borrows from it. On the easiest difficulty, a trail of marigold petals leads you through the dark to your next objective. Marigolds are what families scatter to guide the dead home, and here they guide the living.

Bastion, the Local Helper

Gavin Rose — Thu, 22 Oct 2026 15:15:03 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

What I Built

Bastion is a local agent that does chores based on the prompt you give it.

You describe a chore in plain English:

Every Friday, go through my Downloads folder and move all .jar files into a folder called minecraft.

Every morning at 9:30, check the news on artificial intelligence and summarize anything new.

A local model writes a small Python script for it, tests it as a dry run in a sandbox, and shows you exactly what it will be allowed to do. It slowly fixes itself if any issues are present, then, you only have to approve it once. From then on the chore runs on schedule as plain code. No model is involved, unless you asked for the results to be summarized. Every run after that is fast, cheap and predictable, and the only thing you do is read the result.

It's built for people who don't write code. You never have to edit a script. You read a plain list of permissions ("can reach news.google.com", "can move files in ~/Downloads and to the Trash"), look at a sample result, and click approve.

How I Built It

The open pieces

Gemma 4 12B (Google's QAT build) writes and repairs scripts, run locally with Ollama.
Gemma 3 4B writes the daily summaries.
Bastion talks to Ollama over stdio: the prompt goes in on stdin, a JSON schema goes in with ollama run --format, and structured JSON comes back on stdout. The grammar keeps the model's output valid, and length limits in the schema stop runaway loops.
Elixir / Phoenix LiveView runs the background service and the dashboard.

How a chore gets made

"every Friday, move .jar files from Downloads into Downloads/minecraft"
        │
        ▼
 look around (read-only, sandboxed): list folders, fetch the feed or API, check tools exist
        │
        ▼
 Gemma 4 12B plans (JSON): permissions, schedule, which library example to start from
        │
        ▼
 Gemma 4 12B writes the script (plain Python on top of a small tested helper kit)
        │
        ▼
 dry-run test in the sandbox ──fails──► look again, read the error, revise (up to 2 times)
        │ passes
        ▼
 you approve, in plain English
        │
        ▼
 one supervised process per chore runs it on schedule

OTP does the boring, important parts. Every approved chore gets its own GenServer under a DynamicSupervisor. Runs happen in unlinked tasks, so a crashing script never takes its scheduler down. Failures retry with backoff (seconds for a broken run, 10 to 30 minutes when a site is rate limiting). A janitor process marks interrupted runs as stopped, and every run can be stopped from the dashboard, which kills the actual sandboxed process.

The sandbox is enforced by the kernel, not by asking nicely. Every script runs under macOS sandbox-exec:

Network: blocked, except for a per-run proxy that only forwards to the domains you approved. It logs anything it blocks.
Writes: blocked, except for the chore's own memory folder and the folders you approved. Test runs are dry runs where even those are read-only.
Programs: nothing runs except Python and the commands you approved. No shell.
Secrets: SSH keys, cloud credentials, the Keychain, browser data, Mail, shell history and .env files can't be read at all.
Deleting: scripts never delete. Anything removed goes to the Trash, and any run that moved files has an Undo button.

It respects scraping and other bot rules. Scraped pages respect robots.txt, feeds and public APIs are preferred, rate limits are waited out instead of hammered, and sites that send Content-Signal: ai-input=no (like Lobsters) never have their content passed to a model. You still get the plain list, just not an AI summary of it.

What I learned about small local models (this was the most interesting part)

I tested three 12B models on the same chores, drafted from scratch with no example to copy:

Model	What happened
Gemma 4 12B, community "abliterated" build	Garbled text: `retmah=10`, `efetch.fcji`, invented hostnames
Gemma 3 12B (official)	Clean text, but sent the same failing request five times in a row
Gemma 4 12B (official)	Read the API's error messages and corrected its own requests

None of them can reliably learn an unfamiliar web API on its own, so Bastion leans on things code can check:

probes verify every URL before it's used;
Bastion, not the model, decides whether a request actually worked;
when an API rejects a parameter, Bastion reads that API's docs page and hands the model the real names;
the model adapts small, tested library examples instead of starting from nothing.

Where the models do shine is on your own machine. Cleanup chores worded the way a non-programmer would ask (duplicate files, screenshot filing, old installers, big untouched files) passed on the first try, 4 out of 4.

Why Does Open Innovation Matter?

For this project it's not a nice-to-have, it's the whole design:

Your files stay yours. Bastion's models see folder listings, git repos, the names of things in your Downloads. With open weights running locally, none of that leaves the laptop. The only network traffic is the requests a chore makes to the sites you approved.
Doing nothing costs nothing. A chore that runs every day for a year would be hundreds of paid API calls. Here the model runs once, and daily runs are just Python.
I could swap and tune models freely. The three-way comparison above took an afternoon: pull a model and change one line of config. When the community build kept garbling text, I switched to Google's official one. When a model's default 8K context was too small for my prompts, a three-line Ollama Modelfile gave it 16K. A closed API wouldn't let me do any of that.
Structured output at the grammar level. Ollama enforces the JSON schema while the model generates, and it enforces length limits too. That's what stopped a small model's repetition loops from wrecking a run.
Local chores work offline. Tidying folders, finding duplicates, checking git repos: none of it needs the internet at all.

The Lazy Payoff

I've been running it on my own machine this week:

Daily news chores (AI news, and research) turn a feed of 30 to 100 headlines into a handful of relevant bullets, with Markdown and PDF export.
A weekly Downloads tidy for Minecraft .jar files, set up in one sentence.
Test runs on my real folders showed what the cleanup chores would handle: 423 screenshots and screen recordings to file into monthly folders, 134 old installers and archives to send to the Trash (one of them a 91.8 GB zip from 97 days ago), and duplicate sets across about 35,000 files. Each of those is a chore I'd been putting off for months, and now it's one sentence and one click.

Magen: Local Penetration Testing

Gavin Rose — Mon, 19 Oct 2026 00:44:06 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

Magen is a local-only security testing copilot. It captures traffic while you have the proxy running, flags leaked secrets and personal data, and uses a local open-weight model to reason about what it found and to write and run discovery probes. The one thing that defines it: every part runs on your machine. Nothing sensitive leaves the laptop.

The problem it solves is specific. When you security test a real application you are handling live session tokens, .env files, and other people's personal data. You cannot paste any of that into a hosted LLM without creating a new liability. A model running locally removes that constraint, so the tool can reason over the real data instead of a redacted shadow of it.

Who would pay for it: freelance security testers, small teams, and bug bounty hunters who want an AI assistant but cannot ship client data to a third party.

What it does today:

A scope-locked intercepting proxy (built on mitmproxy) that only records hosts you explicitly opt in.
Deterministic secret and PII detection with stable redaction, so a report can reference a secret by a token like «jwt:7c5b0229» instead of the value.
Local anomaly detection across captured flows, using robust statistics and no model at all, to surface the one request that behaves unlike its siblings.
Local-model triage, plus an agentic "explore" mode that writes standard-library Python probes, runs them in a guarded sandbox, and scores findings from the evidence they produce.
A loopback-only dashboard to drive all of it.

How I Built It

The open pieces are the project, not an accessory to it:

Inference: Gemma (gemma3:12b-it-qat, with gemma3:4b as a fast fallback) running locally through Ollama.
Capture: mitmproxy as the interception engine.
Everything else is the Python standard library, on purpose, so the tool works offline with almost no dependencies.

The design choice I am happiest with is how the agent handles secrets. When the model writes a probe to test an endpoint, Magen injects the real captured token into the probe's process through an environment variable at run time. The model never sees the token and never writes it into code, yet the probe still authenticates for real. The model reasons, the machine holds the secret, and nothing leaves the device.

Why Does Open Innovation Matter?

This is the whole reason the project exists. Three things a closed API could not give me:

I can hand the model real credentials and real responses, because they never leave the machine. That is the core capability, and for a security tool it is not a tuning preference, it is a hard requirement. Cloud reasoning over live secrets is a non-starter, not a trade-off.
It costs nothing per run and does not depend on anyone's pricing or uptime.
I can swap models freely and, in principle, fine-tune one for this exact task.

Running an open model locally was not a nice-to-have here. It was the only honest way to build something that touches credentials. While a hosted open-weight model could have done work here, I felt it went against the principals of what I was trying to accomplish, and that meant making external requests.

What Worked, What Didn't, and What I Would Change

This project ended up being more of a learning experience than a finished product, and I think that is worth noting.

The deterministic parts worked well: capture, secret and PII detection, redaction, and the statistics-based anomaly detection. Those are reliable, genuinely useful on their own, and they need no model.

The local-model layer is where I fell short. The models that run comfortably on my hardware were not quite strong enough for the agentic security reasoning I was asking for. The small model is fast but shallow. The larger one reasons better but takes close to a minute per step and was still inconsistent; it would reach for libraries that were not available, pick weak endpoints to probe, or misjudge whether a response was actually a finding. That is not the model being bad. It is a mismatch between a general-purpose local model with no task-specific tuning and the fairly precise job I wanted it to do.

Two things would have closed most of that gap, and I had time for neither: fine-tuning a model for this narrow task, and simply iterating longer on the prompts and the evidence format the model has to produce.

The bigger lesson was about scope. I started too broad. Under time pressure I kept adding features: a service manager, a one-click demo, a managed test browser, multiple probe goals. Each one widened the surface area, and each one introduced bugs I then had to chase instead of improving the core. A tighter starting scope, something like "capture traffic, scan it for secrets locally, and produce a redacted report," would have been a smaller and far more solid product, with the local model doing one job it could actually do well. Improper scoping at the very start was probably the single biggest thing working against me.

So I do not read this as a failure. The privacy thesis holds up, the deterministic tooling is good, and I now know exactly where a local-first AI tool earns its keep and where it still needs either a stronger model or real tuning time. That is a useful thing to carry into the next build.

Prize Categories

Best Use of Gemma: Magen runs Gemma locally through Ollama as its reasoning and discovery engine.

Plinth! My first arcade-style game.

Gavin Rose — Sat, 10 Oct 2026 23:58:37 +0000

I just shipped Plinth, a small one-tap arcade game for Android. You stand on a column, a new column rises next to you, and you tap to stop it, ideally at your height. If you stop it too high, the extra gets sliced off, so every column after that is shorter. Stop it too low and you fall.

👉 Get Plinth on Google Play · plinth-mobile.web.app

It's built with Flutter and Flame. Here's the story!

Faking 3D to promote performance

Plinth looks 3D, but it's plain 2D canvas drawing. Every column is a box in isometric projection: three filled polygons (top, left side, right side) computed from grid coordinates.

The only real trick is draw order. Boxes further "back" have to be painted first, so each frame I sort everything by gx + gy and paint back to front (the painter's algorithm). The player gets drawn last, except when falling into a gap, where it slots into the sort so the columns in front correctly cover it.

Game feel is a pile of tiny things

The first playable version worked but felt dead. What fixed it was a slew of small touches, none of which were too hard to implement...

Manipulating the character on the character: a little crouch before a hop, a stretch in the air, a squash on landing.
A rising chime for perfect stops that climbs a scale with your streak, so a streak of perfects sounds like a melody.
Screen shake and debris when a column gets sliced.
A "NEW BEST!" moment mid-run whenever you beat your previous best score, complete with confetti.

All of the sound effects were generated with a short Python + numpy script (sine waves, envelopes, some filtered noise).

Syncing coins across devices without a backend

I wanted coins and unlocks to follow players between phones, without running a server. Google Play Games' Saved Games stores a blob per player for free, so I only needed a merge strategy for when a local copy and the cloud copy disagree.

A plain "take the max coins" merge is wrong: spend coins on one phone, sync an older copy, and you get the coins back. Instead, coins are stored as per-device running totals of earned and spent:

Each device only ever increases its own counters, so merging is just "take the max per device". Owned items merge as a union.

The rejection that taught me to test release builds

My first production submission was rejected: "Plinth keeps stopping." It had never crashed for me once. Here is where I used Claude Code to find a quick fix.

The cause: release builds run R8 (code shrinking), debug builds don't. The ads SDK pulls in an old version of WorkManager, which uses Room, which finds its generated database class by name at startup. R8 renamed it, so the app crashed before drawing a single frame:

Failed to create an instance of androidx.work.impl.WorkDatabase

The fix was a keep rule plus a newer WorkManager:

# android/app/proguard-rules.pro
-keep class * extends androidx.room.RoomDatabase { <init>(); }

// android/app/build.gradle.kts
dependencies {
    implementation("androidx.work:work-runtime:2.11.2")
}

The real lesson: before every upload, install a release build fresh on an emulator and actually play it. flutter run in debug mode tells you nothing about R8.

While testing that, I found a second problem: on a phone with no Google account signed in (like many review devices), my startup code triggered the interactive Play Games sign-in, which opened Google's full account login screen. Now the app only silently checks sign-in status at launch, and the interactive flow lives behind a "Sign in" button on the leaderboard.

Two small gotchas worth knowing

AdMob's app-ads.txt check vs Firebase Hosting. firebase init offers a single-page-app rewrite that sends every unknown URL to index.html with a 200 status. That means /app-ads.txt "exists" but returns HTML, and AdMob reports a mismatch. Use real files with cleanUrls and no catch-all rewrite.
The consent form is not optional in the EU and UK. Google's UMP SDK ships inside google_mobile_ads. Request consent before initializing ads, and add a "Privacy options" entry so players can change their choice later.

Monetization: rewarded only

Every ad in Plinth is opt-in: continue a run once, double your coins, or earn coins in the shop. No banners and no interstitials. It earns less per player than forced ads would, but I'd rather people keep playing than bounce off an ad between every run.

Try it

Plinth is free on Android: Google Play. If you give it a go, I'd love to hear your best score, and any feedback on how it feels. And if you're building something similar in Flutter and Flame, ask away in the comments.

Rockhounds should try this!

Gavin Rose — Sat, 10 Oct 2026 23:05:47 +0000

Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass Submission 🌿

Gavin Rose

Oct 10

Pocket rockhound buddy that works anywhere

#devchallenge #hf26challenge

8 min read

Pocket rockhound buddy that works anywhere

Gavin Rose — Sat, 10 Oct 2026 14:27:51 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

I love to explore nature, and while I'm familiar with the minerals, fossils, and stones around my area, I wanted something that could help people that may not know as much, and more importantly, something that could be used completely offline so that it would still work even if you are far into a trail.

The idea is that a photo alone isn't normally enough to identify a rock, and even using flagship close models like gemini can give mixed results. There are several tests you can perform at the site or at home such as scratching it, weighing it in their hand, or putting vinegar on it to see if there is a reaction. This app takes a photo for a first guess, then walks you through those same physical tests one at a time:

Can your fingernail scratch it? A coin? A nail?
Rub it on an unglazed tile. What color is the powder?
Does a magnet stick to it?
Put a drop of vinegar on it. Does it fizz?
Is it a shell with two halves? Is each half symmetrical?

Each answer updates the odds, and the app always asks whichever test will narrow things down the most next. Most identifications take 5 to 8 tests. Between questions the phone is in your pocket and the rock is in your hand, which is the part I like best: the screen is the shortest part of the experience.

The model knows 94 things: 53 minerals, 22 rocks and 19 fossils. It also knows where you're standing. It looks up the bedrock under you from open geologic map data, so in limestone country it leans toward calcite and crinoids, on granite toward feldspar and mica. It even knows the age of the rock. A trilobite can't turn up in rock that's 100 million years too young.

When you're done, you get a specimen label (type, hardness, streak, where to look for more) and can save the find with its GPS location. Finds export as GeoJSON, so you can map everything you've found if desired.

It's for anyone who picks up rocks on a hike, at a creek or on a beach and wonders what they're holding. That covers kids, rockhounds, and people like me with pockets full of crinoids.

Demo

Try it: [https://blue-math-567b.gavin-rose.workers.dev/]. Open it in Safari or Chrome on your phone and add it to your home screen. The first launch downloads the model (about 63 MB, Wi-Fi is recommended). After that, it works wherever you go! While a connection is available, it will use a larger hosted open weight model to give better results more quickly, and will fall back to the lightweight model when necessary or if you toggle the other one off.

I took it outside

I went out on a few walks near home, where the bedrock is shale and sandstone, and here's how it went.

Crinoids, three different ways. Loose stem discs in my palm: crinoid stems, 92%. A stem lying along a rock face: 87%. A slab of fossil hash: 88%. Each one took about five tests.

It got things wrong too. A small white chip came back as calcite, and it wasn't. One find only reached 19% for "conglomerate", which is really a shrug dressed up as an answer. And when I photographed nothing to test the app, it confidently named a rock anyway.

It crashed. On my iPhone the app downloaded the model, then Safari killed the page. In the field I had to time my taps to get in before it went down.

The questions felt off. The fossil-shape question ("stacked discs? coiled shell?") kept coming up before anything suggested a fossil, so for most rocks I was tapping "None of these".

Every one of those turned into a fix, covered below. Taking it outside was the most useful testing I did.

How I Built It

It is an installable web app (PWA) written in TypeScript with Vite. There's no app store and no account. The AI runs directly in the browser.

The photo model: SigLIP 2, on the phone

The photo step uses SigLIP 2, Google's open-weight image and text model, running fully on-device through Transformers.js and ONNX Runtime Web. On phones it runs as a 4-bit model (63 MB) on the CPU. On desktops with WebGPU it uses the fp16 version on the GPU.

I didn't start with SigLIP. I started with CLIP, then benchmarked five open backbones on the same images and the same held-out split, training the same small classifier on top of each:

Backbone	Top-1	Top-3
CLIP ViT-B/32 (where I started)	43.5%	61.9%
CLIP ViT-B/16	48.6%	66.3%
DINOv2 small	47.7%	62.4%
DINOv3 small	51.2%	67.0%
SigLIP 2 base	56.0%	72.9%

SigLIP 2 won clearly, at the same 63 MB on the phone. Swapping it in took about an hour. That's only possible because every one of these models is open and runs in the same runtime.

Training it on rocks

Off the shelf, SigLIP 2 manages 29% top-1 across all 94 classes. I trained a small classification layer on top of it, using about 5,000 openly licensed images:

Wikimedia Commons: Creative Commons photos of minerals and rocks, with attribution saved for every file.
GBIF: CC0 and CC-BY photos of fossil specimens from natural history museums. These were the key to fossils, including plenty of the loose crinoid discs I actually find.
My own field photos: always trusted and never filtered.

Web data is messy. A search for "granite" returns mountains, statues and kitchen counters. So the training script uses the model itself as a filter: it drops any image that looks more like a landscape, building or diagram than a rock specimen. Museum photos had the opposite problem: a tiny fossil next to a big ruler and a color card. For those, the script tries a grid of crops and keeps the one that looks most like a fossil and least like a ruler.

The trained model gets 62.3% top-1 and 78.6% top-3 on held-out images, up from 29% and 45% for the untrained model. Top-3 matters most, because the photo is only a starting point and the physical tests do the rest.

The tests: odds that update with every answer

Each of the 94 entries has its properties written down: hardness range, streak color, luster, how it breaks, whether it's magnetic or fizzes, and for fossils, shape and age. The app keeps a probability for every candidate and updates it with each answer. Answers are treated as noisy, because people misjudge luster and vinegar is a weak acid.

To pick the next question, it calculates which one is expected to shrink the uncertainty most (information gain). A few rules keep it sensible. It always opens with a broad "What did you find?", and fossil-only questions wait until the evidence actually leans toward a fossil.

Geology from where you're standing

With your GPS position (GPS works without signal), Streak looks up the bedrock in Macrostrat, an open geologic map database. It turns rock types into odds: a granite map unit boosts quartz, feldspar and mica, and "crinoidal limestone" in the description boosts crinoids directly.

It never rules anything out. Creeks, glaciers and landscaping gravel carry rocks for miles, so local geology only boosts candidates, and there's a "Found it somewhere else?" switch. Fossils also get an age check: outside their time range they're down-weighted, unless you're standing on river gravel, which can hold fossils of any age. Before a trip you can save the geology for about 20 km around you, so it works offline too.

Optional online mode: Llama 4 Scout

With signal, Streak can also send the photo to Llama 4 Scout, Meta's open-weight vision model, running on Cloudflare Workers AI. The request goes through the app's own Cloudflare Worker, so the phone only ever talks to the app's domain. It sends the photo and the name of the bedrock, never your coordinates. The larger model's answer is blended with the on-device one, and if anything fails, it quietly falls back to offline. A toggle on the home screen turns it off.

Fixing what the field test found

The crash: iOS was killing the page because loading the 176 MB fp16 model needed several copies in memory at once. Phones now get the 4-bit model, and there's a crash guard: if the page dies while loading the model, the next launch automatically drops to the lightest setting instead of crashing in a loop.
"That's a rock" about nothing: before identifying, the app now checks whether the photo shows a rock, mineral or fossil at all, by comparing it against descriptions of both specimens and non-specimens (an empty hand, the sky, a room). If not, it says so. Any result under 35% confidence now shows as Unknown with its closest matches, instead of a confident wrong name.
Fossil questions too early: fixed by the question ordering described above.
Rare minerals coming up too often: a rarity prior. Halite dissolves in rain, so it almost never lies on the ground outside deserts, and the app shouldn't jump to it.

Why Does Open Innovation Matter?

For this app, open models weren't just a nicer choice. Several features only work because of them.

It works with no signal. Good rocks are in road cuts, creek beds and canyons, where you usually have no bars. A closed vision API is useless there. Because SigLIP 2 is open, its weights sit on the phone and run in the browser. The geology lookup is open data too, cached ahead of time.

I could host the weights myself, and that saved the project. On the first real test, the model wouldn't download. My network was blocking huggingface.co entirely. With a closed API that would have been the end of it. Because the weights are open, I bundled them into the app and served them from my own Cloudflare site, split into 20 MB pieces the browser stitches back together. Now the app works on any network that can load it.

I could swap models and measure. I benchmarked five open backbones on my own data and switched to the winner for an 11-point gain, without asking anyone's permission or changing any API contracts.

I could train it on my own rocks. Museum and web photos don't look like a phone photo of a dusty pebble in your palm. With open weights I train my own layer on top of the model and keep adding my own field photos. Every crinoid I photograph makes the next ID better.

Your location stays on your phone. Offline mode sends nothing anywhere. Online mode sends a photo and a bedrock name to a model I chose and deployed, not your coordinates to a company whose retention policy I'd have to trust. Where you find good rocks is something collectors like to keep to themselves.

It costs nothing to run. On-device inference is free. Hosting is on Cloudflare's free tier, and online mode fits in Workers AI's free daily allowance, roughly 120 to 160 photo IDs a day.

Everything underneath is open. Open models (SigLIP 2, Llama 4 Scout), open runtimes (Transformers.js, ONNX Runtime), open data (Macrostrat, GBIF, Wikimedia Commons). It's also honest about its limits: it shows you the odds, the closest alternatives, and when it doesn't know.

My Agent Session

I built the app with the help of Claude, where I asked the agent to do the user interface and wiring so I could focus on the logic and the AI under the hood.

Stay on topic with trance

Gavin Rose — Sat, 03 Oct 2026 18:04:59 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

trance is a small Mac app that keeps you focused on what matters most to you in the moment.

To start, just type what you're doing ("answer emails", "study for my upcoming assignment", "do anything except doomscroll"), pick a track, and get on with your day. Every 45 seconds, a Gemma model running locally looks at what you've actually been doing. From there, it decides whether to stay quiet, nudge you back, or tell you it's time for five minutes away from the screen.

I built this for my friend that has mentioned previously having a hard time staying on track. He was looking for something that could help him focus, provide insights, and gently keep him on track.

Most focus apps block the websites or applications you don't want to visit: you list the bad sites and they slam the door. In my opinion, that doesn't fit how people actually drift away from their focused tasks. YouTube is a distraction until it's the lecture you need. Slack is a distraction until you're clearing your inbox. This app doesn't block anything. Instead, it tries to understand what you're doing, compared with what you said you wanted to do.

What it does:

Notices drift. Each window you land on gets labeled against your intent: on-task or off-task, plus what kind of thing it is (work, reference, feed, video, game, etc.). The live view shows the label a few seconds after you switch.
Nudges. When you continue drifting, a small card slides into view, just like a notification. This nudge is intended to bring you back on track without disruption, but also helps the model judge better throughout the session if you tell it that whatever you're doing is actually on-task.
Recommends breaks. Each track has a working period and a recommended break period. If you remain on task, this app will let you continue working. If you start drifting, tab switching, or seem otherwise unfocused, it will suggest taking a break and show a nice calming visual during the break. Breaks are 5 minutes, but you can come back early or add time if you think you need it.
Tracks. Deep Work, Study, Create, Admin Sprint and Cruise each set what on-task looks like, how much switching is normal and how often breaks come. You can also make your own track if you need something a little more customized, along with setting the intent for the session.
Recaps. Whenever you end a session, Gemma writes a short summary of how the session went, any potential time sinks, what you did well, and what you could do better next time.

Screenshots

How I Built It

The open pieces: Gemma 3 4B (instruction-tuned), running locally in Ollama, inside a Tauri app written in Rust.

Gemma reads, then the Rust code decides. Every 45 seconds the model returns a structured opinion of how your focus is going. I used Ollama's format field, which takes a JSON schema, so that the answer always parses properly. Then plain code applies the rules: drift has to show up in two reads in a row before trance says anything, nudges have cooldowns, and flow is never interrupted.

What a 4B model got wrong, and what fixed it

This is the part that required the most time and attention. My first version of the prompt gave Gemma all of the raw activity logs and asked for a verdict. On a 12B Gemma model, this was a pretty good approach. The problem is that a 12B model running constantly while you're doing other work eats a lot of resources, and some people may not have the computer to handle that. When I switched to a 4B model and did a test run, the model decided that me spending 90% of my time on distractions instead of the task at hand was on-task. To combat this, the model now gets the window information and labels it, the Rust code calculates switch rates, scroll rates, typing, on-task time, etc., and the model then gets this structured information rather than being expected to do all of the calculations itself and getting less than ideal results.

Another problem was stale labels. Each window gets labeled the first time you open it, and that label sticks for the session. A browser card game got marked as work and stayed that way, making me look far more on-task than I was. Now the labeler also gets a screenshot of the window, and if a later check-in sees the screen and disagrees with the label, the label is corrected on the spot.

Why Does Open Innovation Matter?

Because of what trance has to look at. To do its job, trance sees every window title you have open, which sites you're on, and how much you're typing and scrolling. With the glance feature on, it sees a capture of your screen about every 45 seconds. With an open model in Ollama, the most private data trance handles never leaves the workstation or laptop. Screenshots are read and deleted, and session logs are plain JSONL files in the user's own directories. No third parties peeking at your usage, no training being done on your data, etc.

Cost and customization. If you're doing a full work day session, you're probably going to leave this running in the background, even while away from the computer. Running locally, each check-in costs nothing but a bit of battery; on a metered API, a full work day in the background adds up fast. Using Ollama also gives you access to a whole plethora of open models, so you can swap them on command to see what works best for your situation.

Where a closed model would have been easier: To be honest, a big hosted model would have needed fewer guardrails and probably would have produced better results more quickly. Most of my weekend went into making a 4B model reliable, but the solutions to those pesky edge cases turned into features that otherwise might not have been considered. The rules that decide whether trance interrupts you are just code that anyone can read: no black box, and no server calls.

My Agent Session

I've always had a weak spot with frontend development and making things look professional, clean, and polished. Because I used Claude Code to create the user interface, I was able to spend my time where I shine best, building the functionality. Claude also helped me track down a few bugs with rendering and permission issues.

Prize Categories

Best Use of Gemma: Gemma 3 4B runs locally through Ollama. It reads user intent, labels windows (with vision), judges focus every 45 seconds, and writes the session recap, all with no cloud calls!

Atlas

Gavin Rose — Sun, 19 Apr 2026 19:25:20 +0000

Explore Earth, One Swipe at a Time

Inspiration

Some of us spend hours every day scrolling. The format is compelling, and the habit is already formed. The question we asked ourselves was what if that scroll actually taught you something?

Atlas takes the interaction pattern everyone already knows and fills it with things worth knowing about the planet we live on... things like geology, ecology, ocean science, extreme weather, wildlife, and geography. No headlines, no opinions. Just Earth facts, beautifully presented.

What It Does

Atlas is a vertical swipe feed of Earth fact cards, built and deployed as a Flutter web app. Beyond the content, Atlas has a full progression system aimed to keep people engaged as they learn about the Earth.

XP and levels: you earn XP for every new card you read, every quiz you reveal, and every card you save. The XP system has eight explorer levels from Seedling to World Wonder, and each level unlocks a new ambient visual theme (Ocean, Forest, Dusk, Aurora, Glacier, Desert, Volcanic).
Achievements: ten unlockable achievements tied to meaningful milestones; exploring five nature cards, saving five cards, revealing five quizzes, exploring all categories, and more.
Daily streak: the app tracks consecutive days of use and surfaces your streak in the HUD once it hits day two.
Dynamic content: the feed mixes 60+ curated cards with live data pulled from four public APIs, refreshed every 24 hours: NASA APOD (astronomy images), USGS Significant Earthquakes (real-time seismic events), Wikipedia article summaries (25 curated Earth topics, four chosen at random per session), and Open Trivia DB (science and geography questions turned into Real or Fake cards).

How We Built It

The entire app is written in Flutter, hosted on Firebase Hosting.

Stack/Packages:

Flutter/Dart
Provider for state management
SharedPreferences for local persistence
http for API calls
flutter_animate for card entry animations and toast transitions
share_plus for native share sheet
Firebase Hosting

Challenges We Ran Into

Swipe-one-card-at-a-time. Flutter's PageView will happily let you queue multiple animateToPage calls, which causes the view to snap to the wrong card when the user swipes quickly. The fix was disabling the PageView's built-in gesture handling entirely and replacing it with a manual drag detector and an animation lock.

Accomplishments We're Proud Of

A scroll experience that feels genuinely close to TikTok or Reels, built entirely in Flutter for web, with no JavaScript.
A progression system that changes the visual environment of the app as you level up, so the reward is aesthetic and immediate, not just a number going up.
Live data from four different public APIs, unified into the same card format as the curated content, with graceful offline fallback.

What We Learned

Short form content, especially as text, forces content to be better. Writing 60+ cards where every headline had to land in under 15 words was harder than any of the actual programming or engineering! XD

We observed that while typically Flutter can be lacking for web development, when developing for designs that are mobile-first, Flutter was still able to shine although it felt like it took a little longer than doing something react-based may have taken.

What's Next

User-generated cards: a simple submission form where anyone can contribute a fact, queued for moderation before it enters the live feed.
More API sources: NOAA for weather extremes, iNaturalist for real-time species observations, NASA Earth imagery.
Personalisation — mood-based filtering (Relax / Curious / Mind-Blown tags are already on every card, just not yet surfaced in the UI).
Native mobile builds: the codebase is already cross-platform; the only web-specific concession is SharedPreferences over localStorage.

Built With

flutter dart firebase nasa-api usgs-api wikipedia-api open-trivia-db provider shared-preferences flutter-animate glassmorphism

Resources

The repo is publicly available here
The build itself can be found here

Scarab | Weekend Data

Gavin Rose — Sat, 18 Apr 2026 12:48:38 +0000

Scarab can give you a summary on what data is missing from your data with only a few clicks
Download Now

The new way to explore data

Gavin Rose — Thu, 16 Apr 2026 17:12:37 +0000

Scarab was built to allow people with no programming or statistical language experience analyze their data. It produces code snippets for those that want to bring insights into their own program, and can help gain valuable insights from your data as you progress. Free while in the testing phase!

debugr.gumroad.com