This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady
What I Built
SnapRecall is a screenshot search tool. It watches your screenshots folder, lets an AI model read every image in the background, and builds a private search index. Later, you type whatever you remember about the screenshot ("rahul payment 2500", "docker permission error", "wo flight ticket wala") and the matching screenshots show up.
How I got the idea
I take a lot of screenshots. A payment confirmation, a WhatsApp chat I might need as proof, an error message I want to Google later, a ticket, a recipe, a random detail from a group chat. I never organize them, because the whole point of a screenshot is that it takes one second.
The cost shows up later. My screenshots folder has thousands of files, all named like this:
screenshot_2026-10-24-14-55-16.png
screenshot_2026-10-24-14-55-41.png
screenshot_2026-10-24-15-02-07.png
When I need one specific image, those names tell me nothing. I end up scrolling through hundreds of nearly identical thumbnails, squinting at each one, and sometimes giving up. It is not a big problem, but it happens often, and it is exactly the kind of tedious work that a computer should do instead of me.
When I read the "slow and steady" theme, this was the first thing I thought of. It is a task that should happen quietly in the background, with no effort from me, while I do something else.
Two ideas, one choice
I had two possible solutions:
-
Rename on save. The moment a screenshot is taken, an AI reads it, summarizes it, and renames the file with the important words. A WhatsApp chat about rent becomes something like
whatsapp_rahul_rent_payment.png. - Search by memory. Leave the files alone. When I cannot find a screenshot, I type the words I remember from inside it, and the app shows me the matches.
Renaming looked simple, but when I thought it through, it had problems. A filename is one guess about what I will remember later, and the AI's guess is often not the thing I remember. A screenshot of a chat could be filed under the contact's name, the topic, or the amount, and I might search by any of them. Renaming files can also break references in other apps or sync tools, and after a few thousand files, long auto-generated names start to look the same again.
Search works with whatever I happen to remember, and it does not touch my files. So I chose search as the core, and I treat smart renaming as a possible add-on for later (with a preview and an undo, not silent changes).
Who it is for
Anyone with a messy screenshots folder: students who screenshot notes, developers who save error messages, and people who screenshot payments, chats and tickets as proof. I also built it with Hindi and Hinglish in mind, because that is how I actually think and type. A lot of the text in my screenshots is in Hindi or mixed, and I did not want a tool that only works for clean English.
Features
- Background indexing: new screenshots are picked up automatically soon after they are saved
- A slow, throttled backfill of your old screenshots, so a big backlog never needs your attention
- A vision model reads each screenshot and produces a short summary, tags, an app type (chat, payment, code, error, ticket...) and the text it can see
- Hybrid search: keyword matching for exact names, amounts and error codes, plus semantic search for vague memories
- English, Hindi and Hinglish queries
- A simple React interface with a thumbnail grid, a larger preview, and live indexing progress with pause and resume
Demo
[demo]
Code
AnmolCanCodes
/
snaprecall
SnapRecall:Find any screenshot by typing what you remember about it..Built for the Hacktoberfest DEV Challenge, Week 3 (theme: "Slow and Steady")
SnapRecall
Find any screenshot by typing what you remember about it.
SnapRecall watches your screenshots folder, lets an AI model read every image in the background, and builds a searchable index. Instead of scrolling through thousands of files named screenshot_2026-10-24-14-55-16.png, you type "rahul payment 2500" or "docker permission error" and the matching screenshots appear.
Find any screenshot by typing what you remember about it.
AI-powered screenshot indexing · Hybrid search · English, Hindi & Hinglish
Built for the Hacktoberfest DEV Challenge, Week 3 (theme: "Slow and Steady").
Table of contents
How I Built It
The stack
- Backend: Python 3.11 and FastAPI
- Frontend: React 18, TypeScript and Vite, with plain CSS and no UI framework
-
Storage: SQLite for everything, with FTS5 for keyword search and
sqlite-vecfor vector search -
Folder watching:
watchdog - Images: Pillow for downscaling and thumbnails
- Models: a vision model that reads each screenshot and an embedding model for semantic search, either hosted (Gemini API) or local (Ollama)
- Optional OCR: Tesseract, in English and Hindi
SQLite was a deliberate choice. For a personal tool, one file with no database server to install is the right amount of infrastructure, and it holds the metadata, the keyword index and the vector index in the same place.
The pipeline
+---------------------------------------------------------------------------------------+
| INGESTION LAYER |
| |
| Screenshots Folder (SCREENSHOT_DIR) |
| │ |
| ├─► [Startup Backfill Scanner] ── (scandir walk, skips temp/ignored files) ──┐ |
| │ ▼ |
| └─► [Live Watcher (Watchdog)] ── (write stability check, move/delete sync) ─►│ |
+--------------------------------------------------------------------------------------│+
│
▼
+---------------------------------------------------------------------------------------+
| QUEUE & BACKGROUND WORKER |
| |
| SQLite `images` Table (status: pending ──► processing ──► done / failed) |
| │ |
| │ • Claims newest first (ORDER BY mtime DESC, id DESC) |
| │ • Throttles between images (WORKER_THROTTLE_SECONDS) |
| │ • Daily budget tracking & auto-resume at midnight |
| │ • Exponential backoff on 429 rate limits & retry-after parsing |
| ▼ |
+---------------------------------------------------------------------------------------+
│
▼
+---------------------------------------------------------------------------------------+
| INDEXING PIPELINE |
| |
| 1. Checksum & Deduplication: |
| Computes SHA-256. If file hash already indexed, copies vector & metadata instantly |
| (skips LLM & embedding calls completely — zero cost & instant!). |
| |
| 2. Image Preprocessing: |
| Reads dimensions, downscales longest side to MAX_IMAGE_SIDE (default 1024px), |
| generates base64 payload. |
| |
| 3. Parallel Extraction: |
| ┌────────────────────────────────────┐ ┌────────────────────────────────────┐ |
| │ Vision Analysis (Gemini or Ollama) │ │ Optional OCR (Tesseract eng+hin) │ |
| │ Structured JSON output: │ │ Extracted verbatim text strings │ |
| │ - summary (concise overview) │ │ (enabled if OCR_ENGINE=tesseract) │ |
| │ - tags (3-8 categorical tags) │ │ │ |
| │ - app_type (chat, payment, code...)│ │ │ |
| │ - visible_text (Hindi / Hinglish / │ │ │ |
| │ English verbatim text) │ │ │ |
| └─────────────────┬──────────────────┘ └─────────────────┬──────────────────┘ |
| └─────────────────┬───────────────────────┘ |
| ▼ |
| 4. Document Text Assembly & Embeddings: |
| Combines summary + type + tags + visible text + OCR into unified document text. |
| Generates dense embedding vector (EMBEDDING_DIM = 768). |
| |
| 5. Storage & Triggers: |
| - Updates `images` table with metadata & status='done'. |
| - SQLite trigger updates `images_fts` (FTS5 BM25 index with unicode61). |
| - Inserts vector into `vec_images` (sqlite-vec KNN virtual table). |
| - Generates lightweight WebP thumbnail in `data/thumbnails/{id}.webp`. |
+---------------------------------------------------------------------------------------+
▲
│
+---------------------------------------------------------------------------------------+
| HYBRID SEARCH & RETRIEVAL |
| |
| User Query (Search Box) ────────────────────────────────────────────────────────┐ |
| │ │ |
| ▼ ▼ |
| [Keyword Search Leg] [Semantic Leg]|
| - Sanitizes query (safe tokens & Devanagari) - Embeds query with task |
| - BM25 matching via SQLite FTS5 - KNN vector search via |
| - Extracts highlighted snippets sqlite-vec |
| │ │ |
| └───────────────────────────────┬────────────────────────────────────────┘ |
| ▼ |
| [Reciprocal Rank Fusion (RRF_K=60)] |
| Fuses ranks: score = Σ (1 / (60 + rank)) |
| Graceful fallback: degrades to keyword-only if provider offline |
| │ |
| ▼ |
| FastAPI REST API (/api/search) |
| │ |
| ▼ |
| React 18 Frontend (Search, Previews, Progress Bar) |
+---------------------------------------------------------------------------------------+
Finding screenshots: the watcher and the queue
On startup, a scanner walks the screenshots folder and queues anything new or changed. It skips hidden files, temporary downloads and unsupported formats, and it cleans up records for files that were deleted. After that, a live watcher takes over. When a new screenshot appears, it waits until the file size stops changing, so it never reads an image the operating system is still writing.
Every image becomes a row in SQLite with a status (pending, processing, done or failed). That table is the job queue. The worker claims the newest images first, so the screenshot you took five minutes ago becomes searchable before the ones from last year.
What the model does with each screenshot
For every image, the worker:
- Computes a SHA-256 hash. If an identical image was already indexed, it copies the existing metadata and vector, and no model call happens at all.
- Downscales the image so its longest side is at most 1024 px by default, which cuts bandwidth and cost.
- Asks the vision model for structured output: a one or two sentence summary, 3 to 8 tags, an app type (one of 13, such as chat, payment, code, error, ticket or email), and the visible text, with names, amounts, error codes and Hindi or Hinglish preserved as written.
- Optionally runs Tesseract OCR in parallel. It is off by default, because the vision model already reads the text.
- Merges all of that into one document, turns it into a 768-dimension embedding, and stores everything, plus a small thumbnail.
The original file is never renamed, moved or modified.
Why the search is hybrid
There are two very different ways people look for a screenshot, and one search method cannot cover both.
When the memory is exact (a name, an amount like "2500", an error code), keyword search wins, because semantic search tends to blur exact tokens together. When the memory is vague ("the bus ticket one", "the chat where we decided the date"), keyword search fails unless the words match, while semantic search finds it by meaning.
SnapRecall runs both and merges the two ranked lists with Reciprocal Rank Fusion, where each result scores the sum of 1 / (60 + rank) across the lists it appears in. A result that ranks well in either list rises to the top, and there are no raw scores on different scales to compare. The keyword side sanitizes the query into safe tokens and handles Devanagari correctly. If the embedding service is unavailable, search falls back to keywords only instead of failing, and the response says it is degraded.
Slow and steady, on purpose
For this challenge, the theme decided how the indexer works. Reading thousands of images with a model is heavy, and a tool that tries to do it all at once either freezes your computer or burns through API limits and dies halfway. So the worker is patient by design:
- It processes one image at a time, with a configurable pause in between
- On an API rate limit (HTTP 429) it reads the
Retry-Afterheader and backs off, and that does not count as a failed attempt - It supports a daily request budget. When the budget or the provider's quota runs out, it pauses itself and resumes automatically at local midnight
- The queue lives in SQLite, so after a restart it continues with the images that are still pending
- Newest screenshots go first
You keep working, and the index quietly catches up. The backend also has a pytest suite covering the pieces that are easiest to get subtly wrong, like query sanitizing, rank fusion and the worker's behavior.
One codebase, two ways to run
The model layer sits behind a single setting, LLM_PROVIDER. The database, search, worker and interface are identical in both modes.
Hosted (gemini) |
Local (ollama) |
|
|---|---|---|
| Vision model | Gemini API (repo default: gemini-3-flash-preview) |
Gemma, for example gemma3:4b
|
| Embeddings | gemini-embedding-001 |
EmbeddingGemma |
| Needs internet | Yes | No, after the models are downloaded |
| Where screenshots go | Sent to Google's API | Never leave your machine |
| Hardware | Any laptop | A GPU or a fast CPU with 8 GB+ RAM |
| Cost | Free tier with limits, then pay per request | Electricity only |
I started with a fully local plan. Then I looked at my laptop, which is old and weak, and a vision model would crawl on it. Hosted mode keeps the backend light: the app only watches a folder, writes to SQLite and serves a few endpoints, while the heavy model work happens remotely. That is the mode my demo uses.
I want to be precise about what is open. In hosted mode, the models are Google's hosted models, and they are not open-weight. The open-weight path is local mode, with Gemma and EmbeddingGemma.
Why Does Open Innovation Matter?
SnapRecall is designed around the open-weight philosophy:
Deployment freedom: The architecture works identically whether connected to hosted APIs or run entirely locally via open-weight models like Gemma.
Long-term preservation: Unlike proprietary cloud services that can deprecate models or shut down APIs, open-weight models remain functional offline indefinitely
Predictable economics: Local processing has zero marginal cost per image, enabling you to index tens of thousands of screenshots without subscription fees.
True privacy: Screenshots frequently contain OTPs, bank statements, personal chats, and proprietary code. Open-weight models empower you to keep sensitive personal data completely confidential.
Does running your own model mean it works without an internet connection?
Yes. In local mode, you pull the models once through Ollama, and the whole pipeline runs without a network connection. A closed API cannot offer that, because the model lives on someone else's server. My demo runs in hosted mode, so I am not presenting offline results here, but the switch is one setting in the same codebase.
Does it keep personal messages and files on your own machine?
This is the strongest reason open weights matter for this project. Screenshots hold OTPs, bank messages, private chats and ID photos, which is some of the most sensitive data on a device. In local mode, nothing leaves the machine: not the images, not the extracted text, not the vectors, and not the search queries. In hosted mode, downscaled screenshots and search queries are sent to the provider over HTTPS, and the README says so plainly. A closed vision API has only that second path. An open model gives the user a choice.
Does it let you set it up once and never pay for it again?
In local mode, yes. After setup, indexing tens of thousands of screenshots has no per-image cost, and re-indexing is free. That matters for a tool whose whole job is to process a huge number of images, because per-image pricing grows with every backlog and every model change. On the hosted route I used the free tier, and its limits shaped the design directly, which is why the worker is throttled, respects rate limits and has a daily budget.
Where did the open approach work better than a closed one?
I will be honest about both sides. The hosted closed model was the easiest way to get running on a weak laptop. That is a real advantage for convenience. The open route won on three things that matter more for this kind of tool:
- Privacy. A private, fully local mode exists only because open weights exist.
- Longevity. Hosted models get deprecated and APIs get shut down. A model you run yourself keeps working offline indefinitely.
- Portability. Because the project is not tied to one vendor, moving between providers is a config change and not a rewrite.
For a tool that reads people's screenshots, the person who owns them should decide where they go. Open weights are what make that decision possible.
Before SnapRecall, finding an old screenshot meant scrolling through thumbnails for minutes. Now it means typing a few words.
The Lazy Payoff
It actually saves you a ton of time. Instead of scrolling forever to find a specific screenshot, you just type whatever you remember about it. I dropped the project in the friend's group chat and the hype is real ,everyone wants this to exist, so I'm dropping the whole thing in a few days as soon as possible :)
Prize Categories
Best Use of Gemma: SnapRecall's local mode is built around Gemma. It uses Gemma (for example gemma3:4b) to read screenshots and EmbeddingGemma for semantic search, all running through Ollama on the user's own machine, and it is selected with a single setting.
Best Use of GitHub Copilot: I used GitHub Copilot to plan the architecture and to speed up parts of the code. The decisions about what to build, how the search works and how the indexer behaves were mine, and the project runs end to end.
Thank you for reading. If you try SnapRecall on your own screenshots folder, I would like to hear how it goes. Issues and ideas are welcome on the repo.
Top comments (2)
5000 screenshots and no find, type-what-you-remember is the right fix.
yo, thanks broski :)