Tuft: a nature quest that turns your screen black, judged by an open model on your own device

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

I wanted an app whose best feature is that you stop using it.

Tuft gives you one small quest, like find moss or lichen, find sky between branches, or find a shadow with a shape. You pick 5, 10 or 20 minutes and tap Go outside. The screen goes pure black (off pixels on an OLED phone), the quest is read aloud, and the app disappears into your pocket.

When the timer ends the phone buzzes and talks to you. You take one photo, no retakes. A vision model decides whether it completes the quest, says what it saw, and drops the result into a small field log on your device, thumbnail included.

So the screen only matters for the first ten seconds and the last ten. Everything in between is you, outside.

I made it for anyone who needs a reason to look up for ten minutes, and for a school club or hiking group that wants a tool it can host on its own.

Demo

Live app: https://tuft.onrender.com. On a phone, use Add to Home Screen.

There's also a benchmark page at https://tuft.onrender.com/bench/ for testing the judge on your own photos.

Code

Divyansh0208 / Tuft

Tuft

A quest that ends outside. Tuft hands you a small nature quest, turns your screen black, and tells you to put the phone in your pocket. When time is up you take one photo, and an open-weight vision model on your own device decides if you did it.

Built for the Hacktoberfest 2026 Open-Source AI Challenge, Week 1: Touch Grass.

• App: / (single page, no build step, no backend)
• Benchmark: /bench/ (measure the judge on your own photos)

How it works

flowchart LR
  quest["Quest<br/>built-in pool, or written by a local model"] --> pocket["Black pocket screen<br/>wake lock, timer, voice cue"]
  pocket --> shot["One photo, no retakes"]
  shot --> judge{"Judge"}
  judge -->|"Ollama reachable"| vlm["Vision model via Ollama<br/>e.g. gemma3:4b"]
  judge -->|"otherwise"| clip["CLIP in the browser<br/>transformers.js, offline after first load"]
  vlm --> verdict["Spoken verdict"]
  clip --> verdict
  verdict --> log["Field log<br/>stored on this device only"]

Loading

• Pocket screen: pure black (off pixels on…

View on GitHub

No backend, no build step, no framework. It's one index.html, a service worker, a manifest and the benchmark page. MIT licensed.

How I Built It

The judge has two modes. If Ollama is running and has a vision model, Tuft uses it. It picks up Gemma, LLaVA and similar models on its own, and the same model can write new quests. If there's no Ollama, it falls back to CLIP running in the browser through transformers.js.

CLIP compares the photo with the quest description and with six "not this" labels: a room indoors, a computer screen, a face, a plain wall, a road, a blurry photo. The quest passes if its description ranks first or scores above 25%. A second pass picks what the photo shows from eleven nature words, so the spoken verdict can say what it saw. If Ollama dies halfway through, Tuft switches to CLIP instead of showing an error.

The black screen. It's a black overlay plus the Screen Wake Lock API, because phones pause timers and audio when the screen locks. Phones also slow timers down in the background, so when the page comes back, Tuft checks the real clock instead of trusting the timeout.

Offline. A service worker caches the page, the CLIP library and its files the first time they load, and transformers.js caches the model weights. The idea is that after one online run, the judge works with no signal.

A benchmark page. At /bench/ you add photos, tag each one with its quest and whether it truly completes it, and run both judges. You get accuracy with 95% intervals. The page picks a threshold on one half of your photos and tests it on the other half. It also has a "can't tell" band, an exact paired test, and a gallery of every miss. I checked the paired test against counts from another entry in this challenge (19 vs 6 should give p = 0.015) and it matched.

A bug worth telling. An early decorative 3D layer put perspective on the page body. That quietly changes what position: fixed means, so the black screen would have covered the page instead of the whole phone. A black screen that's only partly black defeats the point, so it's fixed in the shipped version.

What I haven't tested

I haven't run a field study, so there are no accuracy numbers in this post. The benchmark page is there so anyone can measure it, including you. I also haven't checked the offline path in airplane mode on a real phone. CLIP matches photos to descriptions, it doesn't know species, and a photo that looks like the quest can fool it. The built-in pool has only ten quests.

Why Does Open Innovation Matter?

• The judge should work where the signal doesn't. An app that wants you outside is going to end up somewhere with no bars. A closed vision API is a network call. Tuft's judge is open weights in your browser.
• Your surroundings stay yours. Photos are resized and judged on the device. The only network traffic is the first download of assets, plus your own Ollama address if you set one.
• Nothing to pay for. No keys, no per-photo cost, no rate limits, no accounts. Any static host works, so a club can run its own copy.
• Swapping models is one text field. The CLIP weights are open too, so a hosted model getting retired can't break the app.
• The trade-off. Closed frontier models are better at recognising species. I picked "good enough, private, and everywhere" over "best, sometimes", and I'm not claiming open wins on accuracy.

Prize Categories

Best Use of Render: Tuft is a static site on Render. The HTTPS it provides is what makes the wake lock, the service worker and Home Screen install work.