Doppel: find two different things that look similar, outside

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

Doppel is a walking game about lookalikes. You take a photo of anything outside then walk about 50 meters, then photograph another one which could be its twin: a different kind of thing that looks the same (a pinecone and an artichoke, or moss and broccoli). EmbeddingGemma 2 runs on your phone which judges how alike the two photos look and shows its reasons.

It's for anyone on a walk who wants a reason to look at things more closely around them. You can do three pairs a day for 10 to 15 minutes then it tells you to stop. You can't lose and you have fun looking around during your walk and your photos never leave the phone.

Demo

• Live: https://doppel-ws59.onrender.com (Chrome on Android or desktop; needs WebGPU; downloads a 388 MB model once, then works offline)
• No time to go outside? Try a demo round: one full round with bundled CC0 photos and the real model, no camera needed, nothing saved. On a desktop it's the first button you see.
• Video:
  

Code

vinmay / doppel

Doppel started as a fork of Sense Hunt and Warmr (same author, same challenge window).

Doppel

Find two different things that look the same. Outside.

Photograph anything outdoors. Walk about 50 metres. Then find its twin: a different kind of thing that looks the same, like a pinecone and an artichoke, moss and broccoli, or bark and elephant skin. EmbeddingGemma 2 runs on your phone and judges how alike the two photos look, and it shows its reasons:

• Doppel sees: what it thinks each photo is ("pinecone ↔ artichoke"), so you know it understood you.
• A twin score from 0 to 100, in four bands: Uncanny, Cousins, Distant, Strangers.
• What they share: "Both: ridged · layered · wrinkled", the visual qualities both photos have.

Three pairs a day, about 10 to 15 minutes, then "Done for today". You can't lose. Photos never leave the phone, and after the first load…

View on GitHub

How I Built It

How I got here

Doppel was my third try at this challenge.

1. Sense Hunt. A daily list of prompts ("a tree with peeling bark"); you photograph one and the model checks it. It worked but it was getting too hard as the prompt generator needed to have a controlled list of words and something achievable for the player.
2. Warmr. Hot/cold for the real world: find a hidden daily secret using photos. It depended too much on where you lived. If your street didn’t have the secret, you couldn’t win.
3. Doppel. I was then looking for a judgement that could be done similarly by a player as well as the model. We are very good at "these two look alike" games and so is image embedding. Doppel is that judgment.

How it works

EmbeddingGemma 2 came out the day after the challenge opened. It can embed both photos and text into the same space and it's small enough to run in the browser on WebGPU and Doppel leans on it for everything.

The score is just how close your two photos are scaled to 0-100 which is calibrated on 23 of my own outdoor photos.

The rest comes from comparing photos to text. At build time I embedded a list of about 200 everyday things ("a tire", "moss") with a few things you shouldn't photograph (screens, rooms, people), and 45 visual words like "round" or "ridged". The closest thing is what "Doppel sees". A visual word shows up under "What they share" when it fits both photos noticeably better than usual. The same lists suggest what to look for on your walk ("Try: seed pod · corn on the cob · seashell") and catch you photographing the same kind of thing twice.

A pair of very different things that still look alike gets a small bonus, so a pinecone and an artichoke beat a cloud and fog. The "Need an idea?" hints were written by Gemma 4 E4B, running locally through Ollama.

Nothing leaves the device. Your photos and scores live in IndexedDB, and a service worker keeps the app and the model available offline. The colors come from Sanzo Wada's A Dictionary of Color Combinations (1933).

On my walk

I played a full day in my neighborhood with three pairs and these are the results Doppel showed me -

• Tire ↔ manhole cover: 49, Distant. Both round, ridged, gray. Fair.
• Power lines ↔ street lamp: 37, Distant. Doppel's verdict: "If you squint. Hard."
• Manhole cover ↔ sidewalk: 83, Uncanny. The one that surprised me: up close, the cover and the speckled concrete really do read as the same texture.

One observation I had was on my Pixel 8 Pro each photo takes 15-20 seconds to read which is much slower than the 1.7 s on a laptop. It turned out fine because that's time spent looking around instead of at the screen.

Does it work?

These numbers come from the real model running in a browser.

I tested on 44 CC0 photos with the real model in a headless browser on a laptop iGPU. (docs/eval.md):

	Result
Twins vs strangers, mean score	58.6 vs 19.9
Twins scoring Cousins or better	6 of 8
Strangers scoring Distant or worse	8 of 8
Same-kind pairs refused	3 of 4
Screens, rooms, documents, blurry photos refused	4 of 4
Photo embed time (laptop iGPU)	median 1.7 s

The first run passed five of the six original criteria and only 5 of 8 twins reached Cousins (moss and broccoli sat at exactly 50).

Why Does Open Innovation Matter?

• It works on a walk with no signal. The model is open-weight and small enough (388 MB) to run in the browser on WebGPU, so after one download Doppel works in airplane mode, in a park, on a trail.
• Your photos never leave your phone. A game that asks you to photograph your street all day shouldn't upload your street. Here there's nothing to upload to: no server, no account.
• It costs nothing to run. No API bill per photo means no ads, no subscription, no limits.
• The model is swappable. The game only needs "embed a photo" and "embed a text". The calibration script re-derives every number from photos, so a better open model is a re-run, not a rewrite.

Prize Categories

• Best Use of Gemma: EmbeddingGemma 2 (released Oct 6) is the core of the game: every photo is embedded on-device in the browser (LiteRT-LM, WebGPU), and the score, the labels, "What they share" and the twin suggestions all come from it. Gemma 4 E4B, run locally through Ollama, wrote the idea hints.
• Best Use of Render: Doppel is deployed on Render as a static site (render.yaml in the repo). Render serves the app and the LiteRT-LM WebGPU runtime that each phone downloads to run EmbeddingGemma 2 locally.