DEV Community: ShalyX

Screenshot Graveyard: I built my friend a memory for the screenshots they never revisit

ShalyX — Mon, 05 Oct 2026 04:16:01 +0000

Hacktoberfest Weekend Challenge: Build for a Friend Submission 🤝

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend.

What I Built

My friend saves everything as a screenshot.

A restaurant they want to try. A product they might buy. A tweet worth rereading. A design idea. A technical explanation. A recommendation somebody sent them.

At the moment they save it, the screenshot feels useful.

A week later, it is just image number 6,842 in the camera roll.

That was the problem I wanted to solve this weekend.

I built Screenshot Graveyard, a tiny memory layer for screenshots.

You drop in screenshots. It reads what is actually visible, turns each image into a compact memory of what it is and why you probably saved it, and stores that memory alongside the original screenshot in the browser.

Later you can ask something vague like:

what was that private access thing I saved?

You do not need the filename. You do not need the exact title. You do not even need to remember the proper noun.

Screenshot Graveyard searches the recovered memories, finds the likely match, and brings the original screenshot back up.

That is the whole product.

I deliberately kept it narrow because the useful moment is not “AI organized my photos.”

It is:

I remembered the idea, but not enough of it to find the screenshot — and the app found it anyway.

The friend I built it for is real, but I am keeping them anonymous. Their screenshot habit is the product brief.

Demo

Live app:

https://screenshot-graveyard.vercel.app

Try this flow:

Drop in a few screenshots.
Let Screenshot Graveyard recover a memory for each one.
Search with a fuzzy description instead of the exact words in the screenshot.
Open the result and confirm it surfaced the original image.

The archive itself lives in the browser using IndexedDB.

There is no account and no server-side screenshot library.

Code

https://github.com/ShalyX/screenshot-graveyard

The repository includes the Flask backend, browser UI, RapidOCR pipeline, Gemma integration, structured-output recovery logic, IndexedDB archive, and smoke tests.

How I Built It

The final architecture looks simple:

Screenshot
   ↓
RapidOCR
   ↓
literal text
   ↓
Gemma 3 4B
   ↓
title / summary / why saved / entities / search terms
   ↓
IndexedDB
   ↓
fuzzy natural-language retrieval
   ↓
original screenshot

It did not start that way.

Version one: let Gemma do everything

My first version ran Gemma 3 4B locally through Ollama.

I sent the screenshot to Gemma and asked it to both read the image and understand it.

Architecturally, that was beautifully simple.

In practice, it exposed two problems immediately.

The first was speed. Vision inference on my machine took long enough that a small batch felt broken even when it was technically still processing.

The second problem was worse because it attacked trust.

A screenshot contained the name:

VEYRA

Gemma read it as:

VETRA

Then the retrieval layer repeated the same wrong spelling back to me.

The app had successfully remembered the screenshot — incorrectly.

For this product, that is not a cosmetic bug.

If Screenshot Graveyard quietly changes the names inside the things you saved, it is manufacturing memories instead of recovering them.

The architecture changed because of one typo

That VEYRA → VETRA failure made the separation of responsibilities obvious.

An LLM is useful for questions like:

What is this screenshot about?
Why might somebody have saved it?
Which future query should retrieve it?
Does this fuzzy question refer to this memory?

It should not be the only authority for:

exact product names
usernames
acronyms
project names
visible quoted text

So I split the pipeline.

RapidOCR owns literal text. Gemma owns meaning.

For text-heavy screenshots, RapidOCR first extracts the on-screen text. That transcript is then sent to Gemma with an explicit rule: distinctive names and proper nouns are evidence, not material to “improve.”

I also added a small grounding pass. If Gemma produces a suspicious near-match to a distinctive OCR token, the stored title is corrected back toward the literal evidence.

If OCR finds very little useful text, the system falls back to Gemma's multimodal image understanding.

That gave me a much better division of labor:

OCR: what does the screenshot literally say?
Gemma: what does it mean to this person?

Gemma creates a memory, not just a caption

Each screenshot becomes a small structured record:

{
  "title": "...",
  "category": "ideas",
  "summary": "...",
  "whySaved": "...",
  "entities": ["..."],
  "searchTerms": ["..."]
}

The field I care about most is whySaved.

Generic image classification can tell me there is a restaurant menu in a screenshot.

Screenshot Graveyard is trying to reconstruct something closer to intent:

You probably saved this because you wanted to compare this ramen place later.

That makes the archive searchable by the future thought, not only by the pixels.

Retrieval is deliberately small

I did not add a vector database.

When you search, the browser sends Gemma a compact catalog of recovered metadata — not every original image again.

Gemma ranks plausible matches and returns screenshot IDs with short reasons.

The browser then resolves those IDs back to the original screenshot already stored in IndexedDB.

That means the retrieval loop is:

vague memory → semantic match → original evidence

not:

vague memory → generated answer that replaces the screenshot

The screenshot remains the source of truth.

I had to make structured output fail gracefully too

Dense screenshots exposed another model failure mode: Gemma occasionally returned useful content wrapped in slightly malformed JSON.

The first implementation treated that as a total failure.

That was silly.

The server now handles common structured-output mistakes such as code fences, trailing commas, and truncated closing brackets. If a response is still unusable, it retries once with a stricter instruction.

Batch processing is also incremental. If screenshot 4 of 5 fails, the first three recovered memories remain visible instead of the whole interface looking stuck.

For a weekend app, those boring failure states mattered more than adding another feature.

From Ollama to hosted Gemma

The original build was fully local: Gemma ran through Ollama.

That made the privacy story strong, but it created a judging problem. A live demo should not depend on my laptop staying online, and local vision inference was too slow on my hardware anyway.

So for the submitted version, I moved the same Gemma family to Hugging Face Inference Providers, currently serving google/gemma-3-4b-it, and deployed the app on Vercel.

The browser archive is still local.

The hosted backend does not maintain a screenshot database. It receives screenshot content or OCR-derived text for inference and returns the recovered memory.

I want to be explicit about that distinction: the submitted version is not fully on-device anymore.

Why Does Open Innovation Matter?

The most useful thing about building this around Gemma was not a slogan about open models.

It was portability.

I started with Gemma 3 4B on my own machine through Ollama.

When local inference turned out to be the wrong deployment choice for judging, I did not have to redesign the product around a completely different intelligence layer.

I could keep the same model family and move inference to a hosted provider.

That mattered because the product's behavior had already been shaped around Gemma:

structured screenshot memories
semantic “why saved” reconstruction
fuzzy retrieval
multimodal fallback
strict proper-noun handling

Open-weight AI let the model be part of the architecture instead of a dependency on one closed endpoint.

It also made the local version real, not hypothetical. I was able to run the same core intelligence on my own hardware while iterating on the product.

And the biggest lesson from the build was that “use AI for everything” was the wrong approach anyway.

Open tools gave me the freedom to compose the system:

RapidOCR for literal evidence. Gemma for interpretation. Browser storage for memory.

Each part does the job it is actually good at.

What I Would Improve Next

I am intentionally not turning the weekend build into a giant photo-management platform.

The core loop I care about is still:

save screenshot → recover meaning → forget the exact details → find it again

The next improvements would stay inside that loop:

stronger OCR grounding for unusual names
faster batch ingestion
duplicate screenshot detection
an optional fully local mode for people who prefer on-device inference
better confidence signals when the model is uncertain

No accounts. No social layer. No giant dashboard.

The graveyard only needs to remember.

Prize Categories

Best Use of Gemma

I am entering Best Use of Gemma.

Gemma is not an optional assistant attached to Screenshot Graveyard. It performs the semantic work that makes the product useful: turning screenshots into personal memories, reconstructing likely intent, handling multimodal fallback, and matching fuzzy future queries back to saved evidence.

The build also exercised one of the reasons an open-weight model matters in practice: I moved the same core model from local Ollama inference to hosted Hugging Face inference without changing the product into something else.

Screenshot Graveyard

Your camera roll remembers everything.

You remember none of it.

VoiceDebt: the voice-note inbox for people with good intentions

ShalyX — Mon, 05 Oct 2026 02:43:20 +0000

Hacktoberfest Weekend Challenge: Build for a Friend

devchallenge #weekendchallenge #hf26challenge

What I Built

My friend sends voice notes like podcasts.

Eight minutes. Twelve minutes. Long enough that I can listen with every intention of replying properly — and still finish with three questions unanswered, one link unsent, and a detail I absolutely meant to acknowledge.

That became VoiceDebt.

VoiceDebt turns the voice notes you keep meaning to reply to into the things you actually owe the sender.

Not just a summary.

It separates a voice note into four useful layers:

THE ACTUAL STORY — what happened
YOU OWE THEM — the questions, promises, decisions, calls, or things you actually need to send back
DO NOT FORGET — dates, personal details, sender commitments, and context worth remembering
SUGGESTED REPLY — a concise reply that covers the important bits

The key word is owe.

A friend can tell me they quit their job, ask if I am free Saturday, remind me to send an Airbnb link, mention an interview on Tuesday, and then spend another five minutes on unrelated lore.

A normal summary compresses that into one paragraph.

VoiceDebt tries to preserve the social obligations hiding inside it.

The Bug That Defined the Product

The synthetic demo worked.

The first real voice note was more useful.

I uploaded a 23-second message from a sales rep explaining that an address could not be edited after payment and that they would process a refund the next morning.

VoiceDebt understood the story, but the first version produced this under YOU OWE THEM:

Confirm when the refund will be processed tomorrow morning.

That was backwards.

The sales rep owed me the refund.

That one mistake forced me to define the product more carefully:

Reply debt is always from the listener's perspective.

After the fix, the exact same note correctly showed NO DEBT, while the promised refund timing moved into DO NOT FORGET.

A second real note exposed another boundary. Someone explained transport options, prices, drop-off points, and warnings. VoiceDebt initially turned "you'll need to find your way from the gate" into a task.

But that is not reply debt either.

It is logistics.

So the final rule became:

VoiceDebt tracks social and communication obligations to the sender — not every task implied by a conversation.

That distinction is the heart of the product.

Demo

Live app: https://voicedebt.onrender.com

Source: https://github.com/ShalyX/voicedebt

The core flow is deliberately small:

Drop in a voice note.
Whisper transcribes it.
Gemma separates the story from the listener's actual reply debt.
VoiceDebt surfaces the things worth remembering.
It drafts a reply.
Only notes with real listener obligations enter the Reply Debt inbox.
Send the reply and clear the debt.

The public app also includes a synthetic Amaka example so the interaction can be explored without exposing anyone else's private conversation.

How I Built It

VoiceDebt is intentionally simple.

The frontend is plain HTML, CSS, and JavaScript. The backend is a small Node server using built-in APIs. There is no framework-heavy application layer and no server-side conversation database.

The AI path is:

voice note
   ↓
Whisper large-v3
   ↓
transcript
   ↓
Gemma 3
   ↓
structured reply debt
   ├── actual story
   ├── listener obligations
   ├── do-not-forget context
   └── suggested reply

Whisper for transcription

Audio is sent to openai/whisper-large-v3 through Hugging Face Inference Providers.

The transcript remains visible in the UI because speech recognition is not perfect. Real testing already surfaced examples such as "Airbnb" becoming "urban" and "Lekki" becoming "Leckie."

Instead of pretending those errors do not happen, VoiceDebt keeps the source transcript inspectable and tells the reasoning model not to turn garbled or uncertain phrases into confident facts.

Gemma for reply-debt reasoning

The transcript is then sent to google/gemma-3-12b-it.

Gemma returns a strict structure containing:

summary
debt[]
remember[]
reply
vibe

The interesting part was not getting an LLM to write a reply.

The interesting part was teaching it what counts as debt.

The final reasoning rules are conservative:

an explicit question can be debt
"send me X" can be debt
"call me when you can" can be debt
a promise the listener previously made can be debt
a sender promising a refund is not listener debt
advice is not debt
a price is not debt
a transport recommendation is not debt
something the listener may need to do for themselves is not automatically debt
when the obligation is ambiguous, omit it instead of manufacturing one

And zero debt is a completely valid result.

A local Reply Debt inbox

The analyzed inbox lives in the browser with localStorage.

VoiceDebt itself does not persist uploaded audio or transcripts in a server-side history.

Only notes with actual listener obligations count toward the Reply Debt inbox. Informational notes can still be analyzed and remembered without making the user feel artificially behind.

Render

Render hosts the public Node application and the server-side inference flow.

The repository includes a render.yaml Blueprint with:

the Node runtime
the health endpoint
Whisper and Gemma model configuration
a secret Hugging Face token supplied through Render rather than committed to source

That made it easy to keep the repo reproducible while still deploying a real working inference path.

Why Open Innovation Matters

Voice notes are unusually personal data.

They contain family updates, work problems, money, addresses, relationships, jokes, plans, names, and all the context people do not usually put into polished text.

Using open models matters here for more than putting an "open" badge on the stack.

Whisper and Gemma keep the intelligence-heavy parts of VoiceDebt replaceable and inspectable.

I can change the speech model.

I can swap the reasoning model.

I can change the exact semantics of "reply debt" in my own prompt and application logic.

And the product can evolve toward more local inference without needing to be redesigned around one proprietary assistant.

The current demo uses Hugging Face-hosted inference, so I do not claim that audio stays entirely on-device.

VoiceDebt keeps no server-side conversation history, while the analyzed inbox is stored locally in the browser.

That distinction matters to me.

"Open" should describe the actual architecture, not become vague privacy marketing.

What I Learned

The best part of this build was that the hardest problem was not technical plumbing.

Whisper worked.

Gemma worked.

Render worked.

The harder problem was deciding what the product actually means by debt.

Real voice notes forced that definition to get sharper:

someone else's promise is not your task
useful advice is not automatically an obligation
context and action are different
a model should be allowed to say "nothing you owe them right now"

That is what made VoiceDebt feel less like an audio summarizer and more like a product.

Current Limitations

Whisper can still mishear proper nouns, accents, or casual speech.

The inbox is browser-local, so it does not sync across devices.

And VoiceDebt is intentionally not trying to ingest WhatsApp, Telegram, iMessage, contacts, or entire messaging histories in this weekend version.

The goal was to get one small behavior right:

I listened to your voice note. What do I actually need to reply to?

Prize Categories

Best Use of Gemma

Gemma is the reasoning core of VoiceDebt. It turns an unstructured transcript into listener-specific reply debt, memorable context, and a reply draft while enforcing the directionality and ambiguity rules that emerged from real-world testing.

Best Use of Render

Render hosts the complete public VoiceDebt application and its server-side inference path, with environment-managed secrets and the repository's Blueprint configuration.

DEV Community: ShalyX

Screenshot Graveyard: I built my friend a memory for the screenshots they never revisit

What I Built

Demo

Code

How I Built It

Version one: let Gemma do everything

The architecture changed because of one typo

Gemma creates a memory, not just a caption

Retrieval is deliberately small

I had to make structured output fail gracefully too

From Ollama to hosted Gemma

Why Does Open Innovation Matter?

What I Would Improve Next

Prize Categories

Best Use of Gemma

VoiceDebt: the voice-note inbox for people with good intentions

devchallenge #weekendchallenge #hf26challenge

What I Built

The Bug That Defined the Product

Demo

How I Built It

Whisper for transcription

Gemma for reply-debt reasoning

A local Reply Debt inbox

Render

Why Open Innovation Matters

What I Learned

Current Limitations

Prize Categories

Best Use of Gemma

Best Use of Render

Links