A sloth does not hurry, and it still gets where it is going. I kept thinking about that while building slothbox, a semantic search tool for a folder of notes that refuses to send anything anywhere.
Every "ask your notes" product I tried wanted the same three things: my text on their server, an API key with a monthly ceiling, and a credit card. For a folder of personal notes, that is a strange deal. The notes are mine, the questions are mine, and the answers are two sentences long. Paying a per-token tax to find a sentence I already wrote started to feel like a joke.
So I built the version that costs nothing to run and works with the network cable unplugged.
What it does
node sloth.mjs index ./notes
node sloth.mjs ask "which lemon is sweeter for salad dressing"
[0.482] notes/citrus.md
Meyer lemons are sweeter than Eureka lemons and work well in dressings. Yuzu has a floral
aroma; a little goes a long way.
No account. No key. No database. The whole index is one JSON file next to your notes.
Why the open pieces are the point
slothbox is not "open source" as a licence checkbox. Three open pieces do the actual work:
1. An open-weight embedding model. Xenova/all-MiniLM-L6-v2 (Apache-2.0), 384 dimensions, quantised to int8 β about 23 MB. It runs through ONNX Runtime via @huggingface/transformers, on the CPU, in Node. No GPU, no Python, no service.
2. Local inference. The first run downloads the model into the Hugging Face cache. After that there are zero network calls. I verified this the boring way: ran the same query with the Hugging Face hub disabled (HF_HUB_OFFLINE=1) and got the same answer, with no network access.
3. A store I can read with my own eyes. .slothbox/index.json. If it breaks, I can open it, read it, patch it, or delete it and rebuild. Try doing that with someone's vector database.
Here is the part that surprised me: the closed version of this tool is not better. A hosted embedding API is more accurate at the high end, but for a few thousand personal passages, a 23 MB model that answers in milliseconds is already past the point of usefulness. The expensive part was never the model. It was the plumbing.
Slow and steady, on purpose
The challenge theme was "Slow and Steady", and the indexer leans into it. It is deliberately lazy:
- Unchanged files are skipped. Every file is hashed; if the SHA-256 matches the last run, it is not re-embedded. Editing one note in a folder of two hundred costs one note of work.
- Progress is saved after every file. Ctrl-C in the middle loses at most one file. No "rebuild the whole index" button, because there is nothing to rebuild.
-
--pace 80sleeps 80 ms between files. The index can crawl along in the background while the laptop is doing something else, which is exactly how a sloth would do it. - No quota to burn. Because nothing leaves the machine, there is no rate limit, no free tier, and no bill at the end of the month.
The laziness is not a gimmick. It is the reason the tool survives contact with a real folder: the cheap path is also the correct path.
Numbers, measured on a laptop (CPU only)
| Step | Measured |
|---|---|
| Model load, first call | ~3.1 s |
| Embed 5 files / 5 passages | under 2 s |
| Store size, 5 passages | ~60 KB of JSON |
| Network calls after the first run | 0 |
Those are not impressive numbers, and that is the argument. This is a laptop doing a small job well, forever, for the price of electricity.
What I got wrong
Two things, so far.
Retrieval is not understanding. Passage-level cosine similarity finds the right paragraph and then stops. Ask it to summarise five notes and it cannot. Ask it something that is not in the corpus and it will still return the closest passage with a confident score, because there is no "I don't know" in a dot product.
A small model has a language bias. The model is English-centric. It will answer in other languages, but the ranking gets noticeably worse, and I would not ship it as a multilingual tool without swapping the model.
Both of these are visible in the README instead of hidden in a marketing page, which is only possible because the model and the store are open.
Why this matters beyond one tool
The interesting thing about open weights is not the licence. It is that the marginal cost of a question drops to zero, so you stop rationing your own curiosity. I now index things I would never have paid to index: old meeting notes, a folder of recipes, the README of every repo I keep forgetting.
A closed tool makes you ask "is this worth a request?" An open one lets you ask "what else could I point this at?"
That is the whole pitch. Not cheaper AI. Unmetered AI, running on hardware you already own, over data that never leaves the room.
Try it
- Code: https://github.com/kaminariouji/slothbox
npm install && node sloth.mjs index ./sample
If you point it at something interesting, tell me what the ranking gets wrong. That is the part I want to fix next.
Top comments (0)