DEV Community: unotchi

One file a night: a folder tidier for a 942 MB computer

unotchi — Thu, 22 Oct 2026 12:04:23 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

What I Built

slow-sorter is a folder tidier that moves exactly one file per night into a trash folder.

It runs on a Raspberry Pi 2 from 2015: 900 MHz, four cores, 942 MB of RAM, no Wi-Fi. On that board a 150M-parameter open-weight model answers one question about one file, in about nine tokens per second, and then the rules — not the model — decide whether anything happens.

That is the whole product. It does not sort your folder. It removes the moment where you open a folder full of junk, feel tired, close it, and promise to deal with it later. Tonight it looks at one old file. Tomorrow it looks at the next one. Forty junk files take forty nights, at four seconds a night, and nobody has to sit and watch.

The effort it saves is not "sorting 40,000 files". It is the ten seconds of dread before you decide not to start.

Demo

There is no hosted version, because the entire point is that it does not need a host. Here is a real run on the Pi, four consecutive nights, on a folder I made deliberately messy:

--- night 1 ---
blog draft.md
  draft blog post about slow sorting
  -> kept (cannot prove it is safe to bin)
blog draft copy.md
  draft blog post about slow sorting
  -> binned (byte-identical duplicate of blog draft.md) -> .slow-sorter-trash/blog draft copy.md
   [3199 ms]

--- night 2 ---
empty.txt
  file
  -> binned (empty file) -> .slow-sorter-trash/empty.txt
   [3380 ms]

--- night 3 ---
render.tmp
  half a render
  -> binned (scratch file (.tmp)) -> .slow-sorter-trash/render.tmp
   [5164 ms]

--- night 4 ---
looked at 1 old file(s), binned nothing. slow and steady.
   [2714 ms]

blog draft.md is the interesting one. The tool looks at it every single night, decides it cannot prove it is junk, and leaves it alone — while the queue keeps moving underneath it. print.png was two days old, newer than the quiet threshold, so it was never even a candidate.

Reproduce it yourself in about a minute; no model required:

git clone https://github.com/sanpeita/slow-sorter
python3 slow_sorter.py --demo ./junk
python3 slow_sorter.py --nightly ./junk --offline --quiet-days 0

Code

https://github.com/sanpeita/slow-sorter

Pure Python 3, standard library only, MIT. 25 tests, no test dependencies, run with python3 -m unittest discover -s tests.

The part that matters is thirty lines:

def why_disposable(path, root):
    """Only bin what can be proven."""
    if path.stat().st_size == 0:
        return "empty file"
    twin = duplicate_of(path, root)          # same SHA-256, older mtime
    if twin is not None and twin.stat().st_mtime < path.stat().st_mtime:
        return f"byte-identical duplicate of {twin.name}"
    if path.suffix.lower() in DISPOSABLE_SUFFIXES:   # .tmp .bak .log .part .crdownload
        return f"scratch file ({path.suffix})"
    return None

None means "kept", and kept is the default that everything has to argue its way out of.

How I Built It

The model is llm-jp-3-150m-instruct3 (Q8_0 GGUF) served by llama-server on loopback, with -c 512 -n 256 --jinja. The client is a single urllib call, so the tool has no dependencies at all and the model can be swapped by changing one endpoint.

The obvious design — scan everything, classify everything, sort everything — dies on this hardware. When I tried a 0.5B model instead, it ran at 1.65–1.85 tokens/second and took 14–31 seconds per answer, with no improvement in quality. The wall on a Pi 2 is not RAM. It is 900 MHz times four cores. So the design is not "a slower version of a fast tool". It is a different tool: one look, one decision, one night.

I also tested whether the model could simply be trusted to decide. It cannot:

Asked to pick between two options: 0/6 correct.
Asked a strict yes/no question: 0/16.
Logprobs on twenty files: "yes" won 20/20, with an almost constant margin.

A 150M model on this board does not have an opinion about your files. It has a prior. So it describes, and the rules decide.

Then the board taught me two things I would not have found on a laptop.

One. The model sometimes answers the instruction instead of the file. The first honest run looked like this:

render.tmp
  no preamble
  -> binned (scratch file (.tmp))

no preamble was in my prompt. The model echoed the constraint and called it a description. The fix is not a better prompt — it is output validation, because on small models a confident wrong answer and a correct answer are the same shape:

echo = ("preamble", "no quotes", "describe this", "file:", "max 12 words")
if not line or any(marker in line.lower() for marker in echo):
    return ""          # failure, so the caller falls back to the file's own first line

That is why the transcript above says file and half a render instead of model prose. Those are the fallback, and they are more honest than anything the model produced.

Two. My first nightly loop got stuck. The oldest file could not be proven junk, so every night re-examined it and binned nothing — forever. It took four real nights on real hardware for that to become obvious, and one test (test_unprovable_file_does_not_block_the_queue) to keep it fixed. A tool that is allowed to do one thing per day has to be very careful about what it refuses to do, because refusing is also a daily decision.

Why Does Open Innovation Matter?

Three things that a closed API could not give me here.

It runs with no network. The contest I built this for is a local-first idea, but the stronger reason is boring: a tidier that needs an internet connection is a tidier that stops working on a train, or in a year when the endpoint changes its pricing. The Pi has no Wi-Fi hardware. It does not miss it.

The files never leave the machine. This tool reads filenames and the first line of file contents. That is exactly the category of data you should not paste into someone else's queue, and exactly the category that a "free" tier quietly collects.

The model is not the authority, so I can change it. Because the rules decide and the model only describes, swapping a 150M Japanese-tuned model for a 0.5B English one is a one-line change with no behavioural risk. That is only true because the open weights let me treat the model as a replaceable part rather than a service I am renting.

And the honest part: the model is bad at this. Nine tokens a second, and it sometimes answers the prompt instead of the file. That is fine — that is the design. An open model I can measure, break, and demote is worth more than a closed one I have to trust.

My Agent Session

(Optional. If you save the session with DevRelay and embed it here, this becomes an entry for Best Use of Entire as well. Otherwise delete this section.)

The Lazy Payoff

I ran it for real. Four nights, unattended, on the Pi:

Night 1: one duplicate binned. 3.2 s.
Night 2: one empty file binned. 3.4 s.
Night 3: one abandoned .tmp binned. 5.2 s.
Night 4: nothing to do, and it said so.

Total human attention: zero. Total time I spent looking at that folder: zero. The folder got four files lighter while I was asleep, and the one file I actually care about was never touched.

That is the trade I wanted. Not a tool that finishes the job, but a tool that keeps working on a job I will never start.

Prize Categories

No partner technology is used — this runs on a Raspberry Pi and the Python standard library. My Agent Session is the only partner-adjacent section, and it only applies if a DevRelay session is attached (Best Use of Entire). Remove this section if you do not attach one.

Your Front Desk Doesn't Need the Cloud: a local LLM attendant on a 942 MB Raspberry Pi 2

unotchi — Sat, 17 Oct 2026 12:15:18 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

A front-desk attendant that runs on a 2015-vintage Raspberry Pi 2 — 32-bit ARM, 942 MB of RAM, no internet connection, no subscription.

It does not chat. It sits inside a small menu on a desk display and, at fixed moments, says one short line in Japanese: a greeting when someone wakes it, an acknowledgement when they pick something, a "the workshop is closed now" at the end of the day. The line is generated by a 150M-parameter open-weight Japanese model running locally through llama.cpp.

The people I think would pay for this are not looking for a chatbot. They run a small workshop, a studio, a reception desk, a maker space. They want a screen near the door that feels attended rather than blank. They have one unreliable Wi-Fi router. And they are uncomfortable with the idea that the sounds and words around their front desk end up on a server owned by someone else.

That is the honest product argument: the constraint is the feature. A cloud assistant at the front desk means a monthly bill, a network dependency, and someone else's data retention policy. This thing means a one-time hardware cost, a box that keeps working when the router dies, and a text log that never leaves the building.

It also means I had to make an LLM work on hardware that was obsolete before most people had heard of an LLM. That turned out to be the interesting part — and the reason this post has numbers in it instead of adjectives.

Demo

Demo video: https://youtube.com/shorts/5tsycraxtRs (unlisted)

The footage is a screen recording of the deployment this repository was extracted from: the same board, the same menu, the same pattern — one short line, generated locally. It runs 64 seconds.

It opens with the wait: "Waiting for response…" while the local model comes up. Nothing is fetched over the network; the wait is the model being loaded into 942 MB of RAM.
The menu then appears with a greeting line already written by the model on the device.
Selecting an item asks the local model for a new line, and the terminal shows the result.
The interface is Japanese, and there is no narration. This is the device as its operator actually sees it, not a polished screencast.

I left the wait in, and I did not cut a "clean" version. A demo that only shows the happy path is not a demo of something that has to survive a power cut in a workshop.

One honest note on scope: the device in the video runs my existing terminal menu (v2.0.1), not this repository. This repository is the standalone, documented extraction of the pattern — written inside the challenge window — and it is the code I would hand to a customer. The measurements in docs/MEASUREMENTS.md come from that same device.

Code

Repository: https://github.com/sanpeita/offline-attendant

The repo is deliberately small. It is a reference implementation of one idea, not a framework:

attendant.py — starts the local inference engine, checks it is healthy, and asks it for exactly one line using a fixed template. No user input is ever passed into the prompt.
config.example.toml — where it lives, what it says, where the model file is, and a loopback-only port.
systemd/ — a unit example that documents, in comments, why it is configured not to restart automatically.
docs/BUILD_PI2.md — the build recipe for 32-bit ARM, including the trap that cost me the most time (-j1, -latomic in the right place, --jinja, -c 512).
docs/MEASUREMENTS.md — every number in this post, with the conditions it was measured under.
tests/ — 19 tests that need no engine. A fake resident server stands in for llama-server, so the fail-closed paths are exercised for real: unavailable engine, empty output, a slow answer, and a config that tries to bind somewhere public.

If you only read one file, read docs/BUILD_PI2.md. The interesting knowledge on this board is all in the negative results.

How I Built It

The three walls of a 942 MB machine

1. You cannot build in parallel. ninja with no explicit job count picks nproc + 2 = 6 on this board. A single -O3 cc1plus process peaks around 300 MB. One translation unit (common/arg.cpp) took about seven minutes on its own and pushed available memory down to 139 MiB. The build must be -j1. A full build of the completion binary is 70–80 minutes across 73–82 steps.

2. 32-bit ARM needs libatomic, and the link order matters. 64-bit atomics are not inline on armv7, so the executable link fails with undefined reference to __atomic_load_8 and friends. Passing -latomic through CMAKE_EXE_LINKER_FLAGS puts it before the objects, where Debian's default --as-needed discards it. Through CMAKE_CXX_STANDARD_LIBRARIES it lands at the end of the link line, where it actually resolves. Shared libraries link fine either way, which is why the failure only shows up at the very last step and looks like a mystery.

3. Measuring memory can stop the build entirely. My first attempt gated the build on per-process peak RSS from /proc/<pid>/status. On this kernel, VmRSS and VmHWM are simply absent for tasks that have no memory map. A fail-closed gate then cannot distinguish "cannot measure" from "over budget", so it refuses to proceed — and the build ran zero steps. The fix was not to relax the threshold but to move the safety valve to the cgroup v2 memory controller (MemoryHigh, a soft limit; MemoryMax invites the OOM killer) and let the build run detached in a transient unit:

systemd-run --unit=llama-build --collect --no-block \
  --property=MemoryAccounting=yes --property=MemoryHigh=650M \
  ninja -j1 bin/llama-completion

Peak for the whole build: 267 MiB, memory.swap.peak = 0, oom_kill = 0. Measuring the thing was harder than building the thing.

Getting one line out of it

Two obvious options, and the measurements decided between them.

One process per request. Load the model, generate, exit. Clean and simple:

Measurement	Value
Generation	8.72 / 9.62 / 9.68 tokens/s
Prompt evaluation	24.1 – 27.2 tokens/s
Model load	658 – 667 ms
A 24-token line, end to end	10.22 s
A 64-token greeting, end to end	15.39 s
Peak RSS	~209 MB
Real drop in free memory	~46 MB
Swap activity	none

Look at that gap. Generation itself is 1.3–2.0 seconds. The other eight seconds are loading and tearing down the model, every single time. On a desk, ten seconds is not a pause, it is a bug.

Free memory barely moves (~46 MB) even though RSS says 209 MB, because the weights are mmaped and most of that is reclaimable page cache. Swap never moved at all — no thrashing, the board is fine.

One resident process. Start the engine once, talk to it over loopback:

Measurement	Value
Cold start until it accepts requests	12 s (paid once)
Resident RSS	~207 MiB
Real drop in free memory	~54 MB
A 24-token line	3.096 s
A 40-token greeting	4.293 s
Swap activity	none

Same hardware, same model, 3.3× faster, because the only thing that changed is that we stopped re-reading the file. That is not an optimization, it is the difference between a product and a prototype.

It also set the design constraint: at ~9.5 tokens/s, a 5-second line budget allows about 40 tokens. So the greetings are 40 tokens and the acknowledgements are 24. I capped the length, not the timeout — a timeout would just turn a slow answer into a missing one.

The real bug was not in the model

Generation was still failing, and the cause was not inference at all. It was an admission gate: "do not use the model unless at least 512 MiB is available."

Free memory with the desktop running: 473–487 MB
Requirement: 512 MiB

The gate could never pass. Not sometimes — structurally, every single time, by 40–50 MB. The model had been "broken" for days because a number was wrong.

The tempting fix is to lower the threshold until it passes. That is not a fix, it is a coin flip. The measurements said something better:

Real drop in free memory: ~46 MB
Swap: flat

So "512 MiB" had never been measured on this board. It was inherited from a different machine, at a different time, with a different model. I replaced it with a floor derived from what I actually observe — peak RSS plus headroom, 256 MiB — and kept the fail-closed behaviour: if the floor is not met, the line is not generated.

Lowering a number by measuring is engineering. Lowering a number by hoping is how you get a device that works on your desk and not in a shop.

What the real device taught me

All of this was verified locally, and then the first three read-only preflight runs on the actual hardware failed in a row — all three with the same class of defect: a rename that was applied on one side only.

A payload key was renamed on the writer, and the reader still asked for the old key.
A release-name contract still required the previous version string, so every run on the device stopped on that line.
An identifier's group structure changed from three parts to two, and the validator kept requiring the field that no longer existed.

Why did local tests miss all three? Because they checked that a value was present, not that a real value was accepted and a wrong one rejected. Once I fixed the tests to pin one real fixture and assert both directions, the whole class became impossible.

The second lesson was about diagnostics. The device-side bootstrap catches exceptions and returns only the type — a reasonable safety decision that also deletes the message. So the exception body was gone. I solved it by writing a read-only diagnostic that replays the identical byte sequence locally and prints the traceback. Failing safe and failing legibly are different properties, and the second one does not come for free.

The last one is my favourite, because it is pure environment: the preflight requires that swap does not move for five seconds. It failed once, with 600 MB free. The culprit was leftover processes from the stopped desktop session — X server remnants touching swapped-out pages pulled them back in. Not memory pressure: just history. After confirming stillness, the same preflight passed 87/87.

Honest limits

I would rather write these down than have someone discover them.

A 150M model is not smart. It writes short lines. It gets facts wrong. In device acceptance it confidently wrote "as of 1:55 PM" when the actual time was 17:31. The design requires a correct clock line to be shown separately whenever the model does not state a time, so the screen still told the truth — but a wrong time inside the generated sentence is a new failure mode, and it is next on the list.
The first line after boot is never generated. A 12-second cold start cannot fit a 5-second line budget. It falls back to a fixed line, by design, and the second line onward is real inference.
Resident operation with the desktop running is unverified. All resident numbers above were taken with the desktop stopped.
Long-run thermals and stability are unmeasured. It has not been left running for a week.
There is no auto-restart, on purpose. If the resident process dies, the attendant goes quiet rather than silently grabbing resources back. For a device in a shop, quiet-and-honest beats alive-and-lying.

Why Does Open Innovation Matter?

This project could not exist as a closed-API product, for four concrete reasons.

1. The unit economics only work at zero marginal cost. Each generated line is a few seconds of one CPU. If every line cost an API call, a front-desk greeter would be a subscription — and a subscription is exactly what this customer is trying to avoid. Open weights plus local inference turns marginal cost into electricity.

2. The hardware constraint is only survivable if you can change the stack. Supporting a 942 MB board is not a configuration option in a closed API. I needed to control quantization (Q8_0 here, Q4_K_M when I want more headroom), context size (512 — and it must be explicit, or the server fails to start on this board), thread count, and whether the model stays loaded at all. That last one was the entire performance story, and it is an inference-server decision, not an API decision.

3. Privacy is structural, not a policy promise. The board has no route to the internet, and the inference server binds to loopback only. There is no outbound port to accidentally enable, no retention setting to get wrong, no vendor to trust. "Your customers' words never leave the building" is a property of the architecture here, not of a terms-of-service page.

4. Legacy hardware is a market, and open source is the only one serving it. There are millions of perfectly good 32-bit boards sitting in drawers. A closed stack would tell their owners to buy new hardware. llama.cpp builds on armv7 if you know about -j1 and -latomic, and the model is a 157 MiB file. That is a product someone can install in an afternoon and understand completely — including the parts that are slow, because you can read the source of the thing that is slow.

The open part is not a checkbox on this build. It is the reason the numbers in this post exist at all: I could measure the thing down to its page-cache behaviour because I own every layer of it.

Prior Work and Credits

Being explicit, because the rules ask for it and because it is the truth.

The build knowledge in this post — the -j1 requirement, the -latomic link-order trap, --jinja for the Japanese template, the 12-second cold start, the 3.096-second resident line — comes from my own private lab notes from earlier in October, where I first got llama.cpp running on this specific board and later measured resident mode against one-process-per-request.

The repository submitted here was written during the challenge window (October 17–18, 2026) and contains no code copied from those notes; what crossed over is the measurements and the lessons, which are reproduced in docs/ with their conditions attached. The prior work was not public, so nothing here is derived from someone else's project.

Standing on: llama.cpp (MIT) and an open-weight Japanese model from the llm-jp project (150M instruct, Q8_0 GGUF, 157 MiB). Neither was modified.

What I Would Charge For

A one-time install, not a subscription: the board, the display, the flashed image, and a configuration pass where we sit down and write the lines the shop actually wants to say. Support as an annual option, because the failure I would expect in the field is not the model — it is a power supply.

The next three things I want to fix, in order: stop the model from stating a time it cannot know; get the resident path verified with the desktop running so a normal machine works, not just a stripped one; and measure what happens after a week of being left on.

If you have a spare Pi 2 in a drawer and a front desk that feels too quiet, the build notes are the part worth stealing.