Nothing To Do: an agent that opens the mail you're avoiding

There is a letter on my kitchen counter I have not opened in nineteen days. I know roughly what it is. I am not ready.

That's not laziness. It's avoidance, which is a different and more expensive thing. Every tool I tried asked me to solve it by looking at the letter. So I built one that looks for me, reads it aloud in a slow voice, and says the thing I actually need to hear: nothing to do.

The most common output of this agent is silence. That's not a limitation. It's the product.

What it does

You photograph the envelope and drop it in a folder. Overnight, a local agent OCRs it, decides whether it needs action, and, most of the time, files it silently.

In the morning, a single markdown digest lists only what matters: a two-line verdict, a deadline, and the one question to ask if it's ambiguous. Anything that needs a human gets a short audio readout in a deliberately slow voice. Anything that doesn't, you never hear about again.

Here it is running on my actual Downloads folder:

Couldn't read — needs you  ChatGPT Image Sep 29, 2026, 08_27_49 PM
Nothing to do              Divyansh's_Goyal_Resume.pdf     A resume. Nothing needs doing.
Couldn't read — needs you  Futuristic Blue Dragon Glass Panel.png
Weekly batch               HTML_CSS_Revision.pdf           Follow the revision sheet to complete the Web Development ev
Couldn't read — needs you  Retro Hacktoberfest Coding Challenge.png
Couldn't read — needs you  banner.png
Couldn't read — needs you  favicon.png
Soon                       final-notice.txt                A final notice for $47. It is due November 30.
Couldn't read — needs you  hacktoberfest-2026-completionist-certifi
Couldn't read — needs you  hacktoberfest-2026-completionist-plus-pl
Couldn't read — needs you  thumbnail.webp
Couldn't read — needs you  vinks.jpg
Couldn't read — needs you  wallp.webp
Couldn't read — needs you  wllpr-desktop-4k.png

14 processed, 0 skipped, 0 errors
DRY RUN — nothing written, no audio generated

Eleven of the fourteen files were wallpapers, screenshots, and icons. All eleven came back UNSURE: the OCR gate refused to guess. One was a resume, which the model correctly classified as informational and filed silently. One was an actual final notice. That's the tool working exactly as designed: silence for the noise, a signal for the one thing that matters.

The core insight

You cannot build an anxiety-reduction tool on infrastructure that adds anxiety.

Privacy isn't a feature here. It's a precondition for the product existing. I built the cloud version first. It worked. I deleted it. The prompt said "do not include personal information" and I still could not make myself send a letter with my diagnosis code to a server I don't control, and I'm the person who wrote the tool.

The local version isn't a cheaper version of the cloud one. It's the only version I would actually use.

Where open was load-bearing, not decorative

The stack:

• Model: llama3.2:3b via Ollama
• OCR: pytesseract
• Voice: piper-tts (or espeak-ng)
• State: sqlite3
• Scheduling: systemd

Nothing leaves the machine. There is no code path that opens a socket to anything except 127.0.0.1:11434.

That claim survives because I had to prove it to myself first. I tried a 7B escalation tier. On a 7.7 GB laptop, Ollama returned a 500 after 14.6 seconds of paging. I know the number because I could read the server log:

[GIN] 2026/10/21 - 17:15:26 | 500 | 14.621843626s | 127.0.0.1 | POST "/api/chat"

The triage model alone works. The cascade OOMs. So the default config pins both tiers to llama3.2:3b, and swapping in a larger model is one environment variable.

A closed API would have either worked silently at a cost I couldn't see, or throttled me without telling me why. I could only make this architectural decision because every layer was inspectable.

The design that isn't obvious

The model never decides whether to interrupt you. classify() returns a structured verdict, and route() is pure Python, applied in this exact order:

1. Confidence under 0.60 → UNSURE
2. Not action_required → SILENT (the most common outcome)
3. Amount matches a prior doc → DUPLICATE
4. No deadline → WEEKLY (batched to Sundays)
5. Under 7 days and bill/legal/tax → URGENT
6. Otherwise → SOON

Silence is the most common output and the whole point. DUPLICATE catches the "this is the same $47 invoice from last month" case. The model is a triage clerk, not an oracle. If it ever decides to wake you, that's a bug.

The scheduling is load-bearing too: systemd with ConditionACPower=true runs the timer at 2:30am only when the laptop is plugged in. Like a sloth that only moves when it must.

The honest failures

Two real ones from my runs:

1. A CSS revision sheet was flagged WEEKLY. The model saw an imperative sentence ("follow the revision sheet") and treated it as an obligation. It isn't. It's a reference doc. That's the false-positive mode: any imperative becomes an action.
2. The 7B tier doesn't run on my laptop at all. I shipped on 3B. A better-tuned prompt and a bigger model would help, but I chose to ship with what I could measure.

I would rather name these than pretend the tool is finished. Precision on action_required matters more than recall here. A false positive means the agent wakes you about a coupon, you stop trusting it, and the product dies. A false negative means one letter slips into UNSURE, you see it, and you handle it.

What I'd build next

• A learned snooze threshold. Resurface timing should adapt to how quickly you actually resolve things, not 2 ** snoozes days.
• A second-opinion path. When triage says SILENT but the file is over N pages, run the escalation tier anyway. Cheap safety net.
• Local voice replies. Say "snooze" out loud to the WAV and the item pushes back a week. Piper and Whisper, both local, still offline.

The repo

vinksgoyal / nothingtodo

nothingtodo is a local-first Python 3.11+ project scaffold for turning unstructured input into actionable tasks using OCR, text-to-speech, and locally hosted language models; business logic will be added in a subsequent step.

Setup

make install
ollama pull llama3.2:3b
ollama pull qwen2.5:7b
sudo apt install tesseract-ocr poppler-utils

View on GitHub

Runs on a laptop with no internet. Costs nothing. Swappable model. Silent by default. The whole point is what it doesn't say.