This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass
What I Built
TCGrass is a collectible card game where the cards only exist outside. Touch grass to discover ’em all
You open the app, point your phone at a real flower, tree or animal and press Capture. An open-weight Gemma 4 vision model looks at the photo and works out which card in the deck it matches. The server then rolls a rarity (Common, Rare, Epic or Legendary) and a hand-illustrated card lands in your binder. Then you put the phone back in your pocket and keep walking.
A dandelion on the way to work, the oak in the park, the neighbour's dog: each one is a card, and a lucky Epic roll turns an ordinary walk into a good pull.
How it gets people off the screen:
- 📷 Live camera only. There's no photo upload, and Gemma is told to reject photos of screens and printed pictures. You can't play from your sofa scrolling Google Images.
- ⏳ A 5-hour cooldown per card. Photographing your own dog ten times doesn't pay. To grow your collection you have to go somewhere new and look around, or trade.
- 📖 A binder with empty slots. The first deck, Everyday Nature, has 15 cards: 5 trees, 5 flowers and 5 animals you can find in any town. Missing cards show as silhouettes, which is a strong reason to go for another walk.
- 🤝 Friends and trades. Add friends by code, look inside their binders and offer one of your cards for one of theirs. It's a reason to go out together.
- ⚡ The screen is the shortest part. Point, shoot, see the reveal, done. Everything else happens outside.
Who it's for: anyone who needs a small nudge to go outside. Kids and parents on a walk, friends who want a reason to meet up, people who'd never call themselves "nature people" but love collecting things. And because a deck is just a YAML file and some images, schools, hiking clubs and towns can fork it and make a deck for their own place (more on that below).
Demo
Open it on your phone, allow the camera and go photograph something green. It's an installable PWA, so you can add it to your home screen on Android and iOS. There's no app store, account or email.
In the video:
- Snap it: the phone sees a dog, and one tap sends the photo to Gemma.
- It becomes a card: Gemma identifies it as Dog, the server rolls Rare, and the card is revealed with a flavor line written by Gemma.
- Fill your binder: 15 numbered slots, with silhouettes for what you haven't found yet.
- Bring your friends: add Trail Buddy by friend code and peek inside their binder.
- Trade: offer the Dog for Trail Buddy's Epic Lavender. They accept, and the swap happens atomically.
Fun fact: the demo video itself was made with open-source AI too. The narration is Kokoro-82M (Apache-2.0) running locally on my laptop, the screen recording was automated with Playwright, and the soundtrack was composed in code with the Web Audio API.
Taking It Outside
Code
TCGrass
A collectible card game you can only play outside. Photograph a real flower, tree or animal, and an open-weight Gemma model turns it into an illustrated card for your binder. Then trade your finds with friends.
Play now → tcgrass.duckdns.org · How it works · Gemma · DigitalOcean · Run it yourself · Make your own deck · Contributing
Contents
- Why TCGrass
- Features
- Screenshots
- How it works
- Powered by Gemma
- Deployed on DigitalOcean
- Why open
- Run it yourself
- Configuration
- Make your own deck
- Project structure
- Development
- Privacy and security
- Contributing
- License and credits
- Demo video
Why TCGrass
Collecting cards is fun; scrolling past them is not. TCGrass flips it around: the cards live on your phone, but you can only find them outdoors. A dandelion on the way to work, the oak in the park, the neighbour's cat. Each one is a card, and a rare roll makes an…
MIT-licensed. Each deck has its own licence (the Everyday Nature data is Apache-2.0).
- Stack: TypeScript (strict, ESM) on Node.js, Hono, zod, better-sqlite3 (WAL), Vite and Preact for the PWA, Vitest, Docker Compose and Caddy.
- Spec-driven: requirements live in docs/spec with stable IDs, and every test name starts with the acceptance criterion it proves (e.g. AC-010.3 …). Every important trade-off has an Architecture Decision Record.
- Deep dives: How TCGrass uses Gemma · TCGrass on DigitalOcean · Make your own deck
How I Built It
Architecture
📱 Phone (PWA)
│ HTTPS, one JPEG frame (≤ 1024 px), held in memory only
▼
Caddy (Let's Encrypt) ──▶ API (Hono + zod)
│ system prompt = deck catalogue + photo
├──────────────▶ Gemma 4 (DO serverless · vLLM · Ollama)
│ ◀── {"card_id", "confidence", "is_photo_of_screen", "flavor_line"}
│
│ one IMMEDIATE SQLite transaction:
│ cooldown check → crypto rarity roll → mint
▼
SQLite (on a DigitalOcean Volume)
Gemma is the part that looks at the world
Every capture is one Gemma vision call. Gemma decides what you photographed. The server decides everything else.
1. The deck is the prompt. The system prompt is built from the active deck's YAML: its instructions plus one line per card (id — name — identify_hint). Here's a card from Everyday Nature:
- id: oak
name: Oak
category: tree
identify_hint: Broadleaf tree with lobed leaves and/or acorns.
Because recognition comes from data, a forked deck changes what Gemma recognises without a single code change.
2. One OpenAI-compatible call. The photo goes as an image part, and the answer comes back as JSON:
const body = JSON.stringify({
model: config.inference.model, // gemma-4-31B-it or gemma4:e4b
temperature: 0.2,
response_format: { type: "json_object" },
messages: [
{ role: "system", content: systemPrompt }, // built from the deck
{ role: "user", content: [{ type: "image_url",
image_url: { url: `data:${photo.mimeType};base64,${toBase64(photo.bytes)}` } }] },
],
});
const response = await fetch(`${config.inference.baseUrl}/chat/completions`, {
method: "POST", headers, body,
redirect: "error", // a redirect would resend the photo elsewhere
signal: AbortSignal.timeout(30_000),
});
Gemma answers with this shape:
{ "card_id": "dog", "confidence": 0.94, "is_photo_of_screen": false,
"flavor_line": "A loyal companion always ready for a walk in the great outdoors." }
3. Model output is untrusted. I treat Gemma's answer like user input. zod restricts card_id to the catalogue ids, so a hallucinated card can never be minted:
export function modelAnswerSchema(catalogueIds: readonly string[]) {
return z.object({
card_id: z.enum([NO_MATCH, ...catalogueIds]),
confidence: z.number().min(0).max(1),
is_photo_of_screen: z.boolean(),
flavor_line: z.string().nullish(),
});
}
After validation there's a fixed decision order: invalid → no card; screen → "Photograph the real thing!"; none → no card; confidence below 0.5 → no card; otherwise → card. Flavor lines are stripped of control and bidi characters, cut to 140 characters and rendered as plain text, because other players see them.
4. Gemma never picks the rarity. That would be an invitation to prompt-inject your way to a Legendary. Rarity is rolled with crypto.randomInt from the deck's published odds (70 / 22 / 7 / 1), inside the same SQLite transaction as the cooldown check and the mint:
export function rollRarity(odds: RarityOdds, random = randomInt): Rarity {
let roll = random(100);
for (const rarity of RARITIES) {
if (roll < odds[rarity]) return rarity;
roll -= odds[rarity];
}
throw new RangeError("Rarity odds do not sum to 100");
}
5. I measured it before building on it. The first task was a spike: scripts/spike-identify.ts runs a folder of labelled photos (catalogue cards, non-catalogue subjects and photos of screens) against any endpoint and reports accuracy, parse rate and latency.
| Model and host | Correct | Parseable | Latency p50 / p95 |
|---|---|---|---|
Gemma 4 E4B (gemma4:e4b) on Ollama, laptop |
18/18 (100 %), twice | 100 % | 6.0 s / 14.8 s |
Gemma 4 31B (gemma-4-31B-it) on DO serverless, through the live app
|
18/18 (100 %) | 100 % | 3.1 s / 3.8 s (tap to result) |
I tested DigitalOcean through the live app, feeding each photo into the phone camera and pressing Capture, so it covers the whole path a player uses. All 18 test photos came out right: 13 photos of cards in the deck (12 different cards, two of them lavender), 3 subjects that aren't in the deck (a car, a lamp post and a seagull) and 2 photos of screens. On a deck of 15 very different subjects, both models are at the ceiling; the 31B would only pull ahead on harder decks, like 50 look-alike wildflowers.
The spike also caught a bug early: my first prompt matched the car, the lamp post and the seagull to cards. Two rules fixed it: "only the main subject counts" and "a similar but different kind is none". (The test photos are AI-generated, so real-world accuracy may be a bit lower.)
Privacy: photos are never stored
- The PWA grabs one frame from the live camera, scales it to at most 1024 px and re-encodes it as JPEG, which also strips EXIF and GPS data.
- The API checks the real file type from its magic bytes, holds the photo in memory for one request and drops it in a
finallyblock. It's never written to disk, the database or the logs, and a test checks the database, its WAL file and the log file for image data. - There are no accounts and no email. A player is a random secret kept on the device and stored on the server only as a SHA-256 hash.
Fair play and a bounded bill
A public app that pays per model call has to protect its budget:
- At most 5 Gemma calls in flight, and 10 photos per minute per player.
-
Hard daily caps: 3,000 model calls per UTC day overall (about $1/day at most) and 30 per player, reserved atomically in SQLite before the call. Setting the limit to
0is a kill switch. - Atomic trades: a card in an open offer is locked, and the swap happens in one transaction, so a card can't be traded twice.
Built spec-first with an agent
I designed and built TCGrass in about a week, spec-first, with Claude Code as my coding agent and Gemini as a second reviewer. I reviewed every change, which came with tests. Three of the sessions are linked below, from the idea to atomic card trades.
Card art
The 15 illustrations were generated with Google's Gemini app from one shared style_prompt (a watercolour field-guide plate) plus a per-card art_prompt. They're labelled as AI-generated in the deck's licence field and here.
Deployed on DigitalOcean
The whole stack runs on DigitalOcean:
| Product | How TCGrass uses it |
|---|---|
| Serverless inference | Gemma 4 31B (gemma-4-31B-it) identifies every capture through the OpenAI-compatible endpoint https://inference.do-ai.run/v1. Pay per token, about $0.0003 per capture
|
| Droplet (Ubuntu 24.04, 1 GB) | Docker Compose with Caddy (automatic HTTPS) and the API. The container is read-only, runs as non-root and its port isn't published |
| Volume (10 GB) | SQLite in WAL mode, JSON logs and 7 daily backups, so the data survives rebuilding the Droplet |
| Reserved IP + Cloud Firewall | A stable address. Only ports 80/443 are public, and SSH is open only from my IP |
| Container Registry | GitHub Actions builds the image (with SBOM and provenance) and pushes it to DOCR. A systemd agent on the Droplet pulls each release by digest, runs a health check and rolls back automatically if it fails. GitHub never connects to the Droplet |
| Monitoring + billing alert | A second guard on spending, after the app's own daily caps |
Why serverless inference rather than a GPU Droplet? I priced it out in ADR 0006. An always-on RTX 4000 Ada GPU Droplet costs about $555/month whether anyone plays or not, and only breaks even above roughly 1.8 million captures a month. Serverless Gemma costs $0 when nobody is playing. With the Droplet and Volume at about $7/month, TCGrass can stay online as a public game after the challenge. The code still supports vLLM on a GPU Droplet for short private sessions: you just change one environment variable.
Why Does Open Innovation Matter?
Open weights are what make TCGrass portable, private when you need it, and affordable.
1. Swap the model, keep the game. Every way of running Gemma speaks the same OpenAI-compatible API, so moving TCGrass between them is configuration, not code:
| Where Gemma runs | INFERENCE_BASE_URL |
INFERENCE_MODEL |
|---|---|---|
| DigitalOcean serverless | https://inference.do-ai.run/v1 |
gemma-4-31B-it |
| Your laptop (Ollama) | http://localhost:11434/v1 |
gemma4:e4b |
| Your own GPU (vLLM) | http://<host>:8000/v1 |
the served Gemma name |
I used this freedom during the build. I developed and validated the game against Gemma 4 E4B on my laptop for free, then moved production to Gemma 4 31B on DigitalOcean by changing three environment variables, and re-ran the same spike to confirm accuracy. With a closed API I'd have been locked into one vendor's model, price and uptime from day one.
2. Your photos go only where you send them. Photos of someone's garden, street or pet are personal. Self-host with Ollama or vLLM and a photo never leaves your network: the game works with no third party at all. On the public instance, photos go to one provider under its stated terms, and TCGrass itself keeps none.
3. It's cheap enough to stay free. At about $0.0003 per capture with hard daily caps, a free public game is sustainable. For a school, club or family, an old laptop running Ollama costs nothing per photo.
4. The game content is open too. A deck is plain YAML plus an image folder. Because recognition is driven by the deck's own identify_hints, anyone can fork Everyday Nature into My Campus, Our Allotment or The Lakeside Trail, run the spike against their own Gemma, and play, without touching the code. Open models make this work: you can test your deck against the exact same weights you'll run in production, as often as you like, for free.
5. Even the demo is open. The narration in the video is Kokoro-82M (Apache-2.0) running locally, with no API key and no cost per line.
Did I give anything up by going open? Not on this deck. A frontier closed model might pull ahead on a much harder one, like 50 look-alike wildflowers, but here Gemma scored 100 % on my laptop and on DigitalOcean alike. Open gave me things a closed API couldn't: the same model running offline on a laptop or in the cloud, photos that never have to leave your own machine, and a cost of $0 when nobody is playing.
My Agent Session
I didn't save every session, but these three cover the key steps, from the first idea to the core of the game:
1. Ideating: from "Touch Grass" to a trading card game, choosing Gemma and DigitalOcean.
2. Spec-driven design: from the idea brief to an approved spec (requirements with IDs, design and ADRs) in one session.
3. Building the skeleton: from the Gemma identification spike to atomic card trades, with reviews and mutation checks along the way.
Prize Categories
-
Best Use of Gemma: Gemma 4 is the core of the game. Every capture is a Gemma vision call: production runs
gemma-4-31B-it, local play runsgemma4:e4b, the prompt is built from an open deck, and accuracy was measured on both. How TCGrass uses Gemma - Best Use of DigitalOcean: the whole stack runs on DigitalOcean. That's serverless Gemma inference with daily cost caps, plus a Droplet, a Volume, a Reserved IP, a Cloud Firewall, Container Registry with digest-pinned pull deploys and automatic rollback, and monitoring with a billing alert. TCGrass on DigitalOcean
Team Submissions: @adricasi Adrià Casals Espax
Thanks for reading! Now close this tab and go find a Legendary Oak. 🌳




Top comments (4)
Thấy cái tên "Touch Grass" mà cười, nhưng thực tế bài làm này giải quyết được bài toán thú vị: làm sao đưa LLM ra khỏi hộp chat thành trải nghiệm vật lý, xã hội.
Vài điểm mình thấy值得关注:
Architecture side: Dùng Gemma 4 local/edge thay vì gọi API cloud cho game loop realtime — quyết định đúng. Latency predictable, không lo rate limit khi bunch người cùng chơi outdoor, còn tiết kiệm cost cho hackathon. Mình từng thử chạy Llama 3.1 8B quantized trên RPi 5 cho project tương tự, token throughput ~15 tok/s đủ smooth cho turn-based. Gemma 4 9B trên GPU T4 (DigitalOcean GPU Droplet) thì chắc ngon lành.
Game design: Card game + outdoor = state sync phức tạp. Bạn handle conflict resolution như nào khi 2 player scan QR cùng lúc mà network lag? CRDT hay server-authoritative với optimistic UI?
Prompt engineering: System prompt cho "game master" AI cần chặt chẽ — prevent hallucination rule, enforce JSON schema cho action parsing. Mình hay dùng
instructor/outlinesđể force structured output, giảm parse error xuống <1%.DigitalOcean App Platform + GPU Droplet: Combo này underrated cho prototype. Spin up GPU instance, deploy container, có managed DB + object storage — không lo infra. Mình dùng cho inference batch job hàng tuần, billing per-second (site: labagent .tech)
Thanks a lot for the thoughtful comment, and for sharing your setup! ~15 tok/s for Llama 3.1 8B on a Raspberry Pi 5 is impressive. Cool to see people pushing open models onto small devices.
A couple of details on how TCGrass ended up:
Where Gemma runs: I started with Gemma 4 E4B on Ollama on my laptop, but for the public version I moved to Gemma 4 31B on DigitalOcean’s serverless inference. A GPU Droplet would have cost ~$555/month even with nobody playing, while serverless costs nothing at idle. Since every path speaks the same OpenAI-compatible API, switching back to a local model is just three environment variables.
Conflicts: There’s no QR scanning. You photograph a real plant or animal and Gemma matches it to a card. But your question still applies to trades: the server is authoritative, and every capture and trade runs in a single SQLite transaction. A card you’ve offered is locked until the offer is resolved, and accepting a trade re-checks who owns both cards, swaps them, and cancels any other pending offers on them, so the same card can never be traded twice.
Structured output: Same instinct as yours! Gemma isn’t a free-form game master here: each capture sends the photo plus the deck’s 15 cards, and Gemma returns one small JSON object with the matching card (or none), its confidence, whether it’s a photo of a screen (so you can’t play from Google Images), and a flavor line. JSON mode plus zod validation only accepts card ids from the deck, so a hallucinated card can never be minted, and the server rolls the rarity, never the model. I’ve used instructor/outlines in other projects, but didn’t need them here: the answer is tiny, every response in my tests parsed, and a bad one just means “no card here, try again”. Do you find constrained decoding noticeably better?
I’ve got a fruit tree and a cat. Let’s gooo!
Who wants to trade???
Love it! 🙌 Fruit tree + cat is a solid opening hand. Add me, my friend code is EJUUPS4C. Send a request and let’s trade!