This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady 🦥
What I Built
I built Later, a small open-source tool that reads the screenshots you took "for later" while you sleep, and hands you a morning digest of what you meant to remember.
My screenshots folder did not need more screenshots. It needed someone to read it.
It had 575 files, the oldest from January 2024. Every one was taken with the same quiet promise: I'll deal with this later. Later never came, because the files are all called Screenshot_20261020-213412.jpg and nobody opens them.
So now an open-weight model on my own laptop opens them, one by one, overnight. In the morning:
Do these. Deadlines, exams, events and tickets sorted by date. "Friday 11:59 PM" in a Tuesday group chat becomes Fri 23 Oct, 23:59, in 2 days, with a calendar button and a reminder the day before.
Saved for someday. Articles, recipes, products, places, error messages, loose to-dos.
Shred pile. Expired OTPs, memes, duplicates. Nothing is deleted. You decide.
Search by meaning. "That Wi-Fi password." "Train to Pune."
Tidy, with undo. Screenshot_20261020-213412_Chat.jpg becomes Deadlines/2026-10-23 DBMS assignment 3 deadline.jpg. One command puts everything back.
Demo
Later runs on your own laptop and opens as a web page on 127.0.0.1. Nothing is hosted, because the whole point is that your screenshots never leave your machine.
The repo includes later demo-pack, which writes synthetic screenshots (fake OTPs, a fake chat about a deadline, a recipe, a train ticket) so you can try everything without exposing anything real. I did not put my own screenshots in the demo, because they contain OTPs and private chats.
git clone https://github.com/Rehan1604/hackw4.git
cd hackw4
pip install -r requirements.txt
ollama pull gemma3:4b
ollama pull embeddinggemma
python -m later doctor
python -m later demo-pack demo_screenshots --add
python -m later sleep --now --limit 5
python -m later
The last command opens the morning digest in your browser: the night strip, the Do these list, search, and the tidy preview.
Code
Later
A sloth that reads the screenshots you took "for later" while you sleep.
Screenshots are the laziest note-taking system ever invented: two buttons and the problem feels handled. Then the file sinks into a folder of a thousand others called Screenshot_20261020-213412.jpg, and later never comes.
Later is the lazy second half. Point it at your screenshots folder and go to bed. Overnight, on your own laptop, an open-weight model (Gemma 3) reads every screenshot, one at a time, and works out what you meant to remember. In the morning you get:
-
Do these: deadlines, exams, events and tickets, sorted by date, each one click away from your calendar (
.ics, with a reminder the day before a deadline). - Saved for someday: the articles you meant to read, the earbuds you meant to buy, the recipe, the café, the plumber's number, the error message.
- Shred pile: expired OTPs…
How I Built It
Everything runs on the laptop, with no cloud account and no API key:
Gemma 3 4B (vision) through Ollama reads each screenshot and returns structured JSON.
EmbeddingGemma turns each result into a vector for search by meaning.
Tesseract OCR reads small text exactly.
SQLite (FTS5 for keywords) stores everything. FastAPI serves the digest on 127.0.0.1, and the page is plain HTML, CSS and JavaScript with no build step.
My laptop is an Intel i5 with integrated graphics and 16 GB of RAM. On it, Gemma 3 4B averages about 33 seconds per screenshot. In a chat app that would be unusable. For a job that runs while I sleep it does not matter, so I designed around it. The sloth has rules.
Work while I'm not. later sleep --until 07:00 waits until the keyboard has been quiet for a few minutes, stays off a draining battery, and lowers Ollama's CPU priority. later schedule sets it up once as a nightly Windows task that wakes the laptop.
Never lose your place. Every screenshot is keyed by a hash of its contents, not its name. Kill it, reboot, rename the files: it resumes where it stopped.
Don't read anything twice. A 256-bit perceptual hash spots near-duplicates. If the OCR text matches too, the copy reuses the earlier answer with no model call.
Trust, but verify. Ollama's structured outputs constrain Gemma to a JSON schema, so kind is always one of fifteen known values and every field is present:
python
body = {
"model": "gemma3:4b",
"messages": [
{"role": "system", "content": SYSTEM_FULL},
{"role": "user", "content": prompt, "images": tiles}, # tiles + OCR transcript
],
"format": SCHEMA, # kind is an enum, every field required
"stream": False,
"options": {"temperature": 0, "num_ctx": 4096},
}
A valid shape is not a correct answer, so plain Python checks the content afterwards. Relative dates resolve against when the screenshot was taken, so "Friday" in a Tuesday chat means that Friday. A date years away is dropped. A meme does not get a to-do.
Two locks on the secrets. A screenshot counts as private if the model says so or a deterministic check fires: OTP wording next to a code, a card number that passes the Luhn checksum, ID numbers, balances, password: lines, lab-report terms. Digits are then masked in the database, thumbnails are blurred on the server, and the full image is sent to the browser only after you click Reveal.
The night strip. The digest opens with proof of the night's work: a band that shades from night to dawn, one bar per screenshot, as tall as the seconds it took. While the sloth works you can watch the bars appear. It is a progress bar that is honest about being slow. The page loads no external fonts, scripts or images.
Here is the shape of what the model returns for a chat screenshot (an illustrative example):
json
{
"kind": "deadline",
"title": "DBMS assignment 3 deadline",
"summary": "Group chat says the DBMS assignment is extended to Friday 11:59 PM.",
"action": "Submit it on the portal",
"date": "2026-10-23",
"time": "23:59",
"sensitive": false,
"keep": true,
"tags": ["dbms", "assignment", "deadline"]
}
What broke along the way
Four things broke, and each one taught me something.
I optimised the wrong thing
My first runs took 60 to 110 seconds per screenshot. I assumed the image was the cheap part and the text was the cost, and I sliced every tall screenshot into two tiles to be sharper. Then I wrote a benchmark instead of guessing. On my CPU, one image takes Gemma about 30 seconds to read, while 450 tokens of text take 1.6 seconds. Extra tiles doubled the cost for almost nothing once OCR was in the loop.
So Tesseract now reads every screenshot first. If it finds plenty of text, Gemma gets the transcript and never sees the pixels: about 20 seconds. A map, a meme or a photo still gets the image: about 55. Thread count barely mattered either (12 threads took 28 s, 4 took 37 s).
The model invented dates
On my first 41 screenshots, Gemma attached a date to 27. That sounded great until I looked: most were ordinary notes and AI chats with no date anywhere on the screen.
I added one rule. A date has to be backed by the screenshot's own text: a month and day, a numeric date, or a word like "Friday" it can resolve. Re-checking the stored results removed 23 of the 27. The 4 that survived were real. A model that sounds confident about a deadline is more dangerous than one that says nothing.
My "reset" did not reset
I had mixed my synthetic demo screenshots into my real database, so I deleted the data folder to start clean. The folder did not exist. Python from the Microsoft Store virtualises %LOCALAPPDATA%, so my files were somewhere else entirely. The fix was boring: set LATER_HOME to a normal folder and keep a separate home for demos. My real data and my demo data can no longer touch.
The terminal closed in the middle of the night
I woke up to a closed terminal. Because the queue is keyed by file hash and resumes by design, I re-ran one command and it carried on. One screenshot had failed once and read fine on the retry. That is the whole reason the sloth saves its place after every single screenshot.
What I would do next
Honest limits, so you know what you are getting:
A 4B model makes mistakes. The validation catches invented dates and missed secrets, not every wrong summary. The privacy checks are a safety net, not a guarantee, so look before you share a folder.
It is tuned for English-heavy screenshots from my own phone and laptop. I have not tested other scripts.
The scheduling and keep-awake pieces are Windows-only. The rest should run anywhere Ollama does, but I have only tested Windows.
The tests use a fake Ollama server. They check the plumbing (resume, duplicates, privacy, digest order, calendar files, tidy and undo, the web API). They cannot judge the model. Only real screenshots can, which is what the numbers below are for.
The idea I most want next: let the digest learn which things I actually open and which I always dismiss. If you build it, contributions are very welcome.
Why Does Open Innovation Matter?
Because screenshots are the most private files you own. They hold OTPs, bank balances, lab reports and your friends' messages. The people in those group chats never agreed to be uploaded anywhere. With an open-weight model, "your data stays on your laptop" is enforced in code: Later refuses to talk to a model server that is not on the same machine unless you explicitly switch that check off.
Because slowness is free when the weights are yours. A hosted vision API bills per image and is built to answer in a second, which is speed I do not need. On my own laptop, a slower night costs nothing. Open weights let me spend time, which I have plenty of overnight, instead of money.
Because I can change the brain. One environment variable swaps in gemma3:12b on a machine with more memory. I can cap threads, change how often the image is used, and use EmbeddingGemma's task prefixes for search. None of that is a feature request to anyone.
Because it works when nothing else does. No internet, no key to leak, no deprecation email. The model only changes when I choose to pull a new one.
Where closed would have won: a frontier hosted model would read messy screenshots more accurately, and in a second rather than 33. I narrowed that gap with OCR and validation and accepted the rest. The privacy gap is the one I could not have closed any other way.
The Lazy Payoff
Setup took a few minutes, once. After that, I pointed Later at my real folder and went to bed. These numbers come straight from python -m later stats:
575 screenshots, oldest from January 2024. 574 were read by the model and 1 was recognised as a duplicate with no model call. 527 were read in a single run of 4 h 51 min while I slept.
5 h 13 min of CPU work, 32.8 seconds per screenshot, about 10.6 tokens per second. 473,770 tokens read, 75,701 written. Total cost: some electricity.
What was in there: 211 code and error screenshots, 118 "other", 108 chats, 58 loose to-dos, 46 things to read, 15 contacts. Only 17 had a real date and 1 was a true deadline. My folder is not full of deadlines. It is full of things I wanted to find again, which is why search matters more than the calendar button.
45 screenshots were flagged private, blurred and digit-masked.
By hand, at around 20 seconds each (my estimate), that is about 3 h 11 min of opening, reading and deciding. The honest number is higher, because I never did it. The oldest screenshot has been sitting there for almost two years.
If you have a screenshots folder you are afraid to open, give it to a sloth.
Prize Categories
Best Use of Gemma. Gemma 3 4B reads every screenshot (vision plus schema-constrained JSON), and EmbeddingGemma powers search by meaning. Both run locally through Ollama on a laptop CPU.
__ This project comes from a human. AI helped with its design and code.
Top comments (0)