I built an offline invoice sorter for freelancers, then measured whether local inference is actually cheaper

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

InvoiceLens reads a folder of invoices and receipts, pulls the fields out of
each one with a local open-weight model, sorts them into categories from a rule
file you edit yourself, and writes a spreadsheet you can hand to an accountant.

It is built for one person: the freelancer who bills by the hour and files their
own taxes. In September a client invoice arrives, in October a hosting bill, and
in between a pile of mixed receipts. At tax time that pile has to become one
file that says what came in, from whom, and which of it is a business cost.

Nothing leaves the machine. No API key, no account, no network call after the
model is downloaded.

The challenge prompt lists ideas, and its first one is almost exactly this: "an
invoice and receipt sorter for freelancers that runs locally, so client data
never leaves their laptop." I took that literally and then spent most of the
build arguing with it, because the obvious way to do this on a laptop does not
work. More on that below.

What it does today, measured rather than claimed:

Capability	Status
Text-based .pdf and .txt input	Works
Field extraction (vendor, dates, amounts, line items)	Works
Expense categorization from a YAML rule file	Works
Deductibility flags	Works, rules are yours to edit
CSV + JSON + Markdown output	Works
Self-repair loop for malformed model replies	Works
Scanned PDFs and images (OCR)	Not in v1
A GUI	No. CLI only.
Tax advice	No. It sorts and flags. It does not decide your return.

Demo

There is no deployed link, and that is the point rather than an omission: a
tool whose entire argument is that your client data never leaves your laptop
cannot demo by uploading your client data. So here is the real thing, run on
three synthetic invoices on a 4-core i5-8350U:

$ python -m invoicelens scan fixtures --out report --rules rules/rules.example.yaml

InvoiceLens 0.1.0: local-only invoice triage
model : ~/.cache/invoicelens/models/gemma-2-2b-it-Q4_K_M.gguf (1708582752 bytes)
rules : rules/rules.example.yaml (7 categories)
files : 3

engine: llama.cpp, CPU only
[1/3] design-retainer-2026-09.txt
      vendor=NORTHWIND STUDIO date=2026-09-03 total=2400.0 USD
      category=professional_services deductible=True
      extraction: 1 attempt

[2/3] office-mixed-2026-09.txt
      vendor=MERIDIAN OFFICE SUPPLY date=2026-09-27 total=1124.82 USD
      category=office_supplies deductible=True
      extraction: 1 attempt

[3/3] saas-subscription-2026-10.txt
      vendor=GITHUB, INC. date=2026-10-02 total=4.0 USD
      category=software_subscriptions deductible=True
      extraction: 1 attempt

done in 223.3s: 3/3 extracted, 0 repaired, 0 failed

Three for three, first attempt, no repair prompts. The middle one is the
interesting file: a mixed receipt with a standing desk, a monitor arm, paper,
a chair, and two kilos of coffee beans, and the model got all five line items
right. The third file is the one that shows the schema holding up: a $4 GitHub
Team invoice whose second line is "Additional Actions minutes (2,000)", which
the model correctly read as quantity 2000 at unit price 0 rather than as
another $4 charge.

The spreadsheet that comes out:

file,vendor,invoice_number,invoice_date,due_date,currency,subtotal,tax,total,category,deductible,rule_note,extraction_attempts,extraction_repaired,error
design-retainer-2026-09.txt,NORTHWIND STUDIO,NS-2026-0917,2026-09-03,2026-10-03,USD,2400.0,0.0,2400.0,professional_services,True,Subcontractor and agency invoices are normally a business cost.,1,False,
office-mixed-2026-09.txt,MERIDIAN OFFICE SUPPLY,MOS-88921,2026-09-27,2026-10-27,USD,1041.5,83.32,1124.82,office_supplies,True,Office equipment may be deductible or may need to be capitalized. Check locally.,1,False,
saas-subscription-2026-10.txt,"GITHUB, INC.",GH-7714-2026,2026-10-02,,USD,4.0,0.0,4.0,software_subscriptions,True,Business software subscriptions are normally deductible.,1,False,

And the honest part, because the demo above hides it. That mixed receipt was
mis-filed. The model extracted everything correctly, but the keyword rules
classified the whole invoice as office_supplies, because two hits
(office from the vendor name, monitor from a line item) beat
meals_entertainment's one hit (coffee). The coffee landed in office
supplies, where nobody will look at it again. It is in the README's limitations
section, it is in the fixture, and fixing it needs either per-line-item
classification or a model pass over the rules. Neither is in v1.

Code

dekita-dev / invoicelens

Offline-first invoice triage for freelancers: local Gemma 2 2B through llama.cpp. No API key, nothing leaves the machine.

InvoiceLens

Local-first invoice and receipt triage for sole proprietors and freelancers. An open-weight model (Gemma 2 2B) runs on your own machine, reads your invoices, and turns them into a categorized, deductible-flagged spreadsheet No API key. No account. Nothing leaves the laptop.

Built for the Hacktoberfest 2026 Open-Source AI Challenge, Week 2: Side Hustle.

---

Who would pay for this

Anyone who bills by the hour and files their own taxes. A freelancer gets invoices from a designer in September, a hosting bill in October, and a pile of mixed receipts in between. At tax time they need one spreadsheet that says what came in, from whom, and which of it is a business cost.

The two things that make this hard with a hosted LLM API are the two things this project is built around:

• The data is sensitive. Client names, project names, amounts, sometimes addresses. Sending that…

View on GitHub

MIT licensed, 45 passing tests that run with no model download, and a
tools/smoke_engine.py that does a real end-to-end check when you do have the
model.

How I Built It

The model is Gemma 2 2B IT, quantized to Q4_K_M, about 1.6 GB, run through
the official llama.cpp release binary. InvoiceLens starts llama-server as
a subprocess, waits for /health, and talks to it over HTTP on 127.0.0.1.
The engine wrapper is roughly 40 lines of urllib against an
OpenAI-compatible surface.

That shape is not what I started with, and the reason it changed is the most
useful thing I can tell you about building on Windows.

llama-cpp-python is out. It publishes no win_amd64 wheels for any
Python version, and this machine has no C++ compiler.

gpt4all looked like the answer and was not. It ships a
py3-none-win_amd64 wheel with llama.cpp bundled in, which is exactly what you
want. Three things kill it on Windows, each verified rather than assumed:

1. The wheel contains only two backends, CUDA and Kompute/Vulkan. There is no CPU backend DLL at all, and device="cpu" is a no-op: gpt4all.py:222 hard-codes backend = "kompute" on every non-macOS platform.
2. The Kompute build aborts inside llama.cpp during model load. It does so identically at llama.cpp:11912 for every combination of n_ctx in {2048, 4096, 8192} and n_gpu_layers in {0, 1, 12, 100}. That matrix rules out context size and GPU memory as causes, which is what let me stop guessing.
3. The bundled llama.cpp is old. The architecture table inside the DLL contains gemma2 but not gemma3, llama4 or qwen3, placing it around early 2025, before GGUF v3 was fully supported. A current Gemma 2 GGUF (version 3) aborts in llama_model_load as a result.

I want to be clear about step 3, because I got it wrong first. I assumed it was
a CUDA problem, installed the CUDA runtime and cuBLAS wheels (which did fix a
real LoadLibraryExW error 0x7e), and the assert did not move. Fixing a real
bug did not fix the actual bug. The version mismatch was the cause, and the
only way to see it was to read the architecture table out of the DLL.

So: the official release binary. Current, a real CPU backend that picks the
right instruction-set variant at load time, and an HTTP surface that means no
ctypes binding against a moving C ABI.

Two other things worth stealing from this build:

The parse error becomes the next prompt. The model is asked for a strict
JSON object. If the reply does not parse, the exception text and the bad reply
are fed back as the next prompt. This is deliberately not a regex fallback: the
model still owns the extraction, and the retry only tells it what went wrong.
Every run reports how many attempts each file needed, so you can see the
failure rate instead of trusting a silent success.

Physical cores, not logical ones. Generation with a small quantized model is
memory-bandwidth bound, so hyperthreads cost more than they return. Measured on
this machine with tools/bench_backends.py, which reads the server's own
timings block instead of dividing wall clock by a guessed token count:

Threads	Prompt tok/s	Generation tok/s
2	18.7	5.15
4	23.0	7.44
8	33.6	5.84

Two things fall out of the split. Prompt processing keeps scaling all the way
to eight threads. Generation peaks at the four physical cores and then goes
backwards: with hyperthreads on it runs 22% slower than with four. And because
generation is roughly two thirds of a real invoice's time, four threads still
wins overall even though eight would get through the prompt 46% faster. The
default is now physical core count, detected per platform, capped at 8.

Then I tried the GPU, because the machine has three of them. The device
matrix is not what the spec sheet suggests. llama-bench, 512-token prompt
and 16 generated tokens, four threads, all 26 layers offloaded:

Device	Prompt tok/s	Generation tok/s
CPU, 4 threads	23.7	6.4
Intel UHD 620 (integrated)	18.5	4.3
AMD RX 6600	1142.6	42.1
Quadro P500	cannot allocate	-

The integrated GPU is slower than the CPU, so "offload to the GPU" is not
advice, it is a device-selection problem. Full offload to the RX 6600,
measured on the real three-invoice workload with the benchmark that produced
the thread table:

	CPU (default)	RX 6600, 26 layers
Prompt tok/s	23.0	604
Generation tok/s	7.44	38.7
Cold start	5.7 s	8.3 s
Three invoices, wall clock	223.3 s	26.1 s

That is 8.6x end to end, and it survives the honesty check: every field the
tool depends on came out bit-identical between the CPU run and the GPU run,
from vendor and dates through all nine line items to deductibility, one
attempt each, zero repairs. One optional free-text field did not survive:
notes missed a line it had caught on CPU, reproducibly, across two GPU runs.
Same model, different arithmetic, different answer on a field nobody should be
trusting anyway. That caveat ships in the README next to the flag.

The flag is opt-in (--n-gpu-layers 26 --device vulkan1) and the default is
untouched, because the fastest device is not something the tool can know for
you: --list-devices exists precisely because device 0 on this box is the
worst usable one.

How this differs from the neighbouring submission. Another entry in this
challenge, LedgerVault AI,
solves the same problem for the same person, and in two ways it is ahead of
mine: it has a real UI, and it does OCR through tesseract.js in a
WebAssembly worker, which means it reads scans and photos. InvoiceLens v1 does
neither.

The trade goes the other way on setup. LedgerVault requires you to install
Ollama separately and run llama3.2, qwen2.5, mistral or smollm2 through
it. InvoiceLens has no external runtime at all beyond
pip install -r requirements.txt: it drives llama-server from the official
llama.cpp release as a subprocess, and it uses Gemma. So it is one subprocess
to manage against one service to install, and a CLI against a browser app.
Which of those is the better trade depends entirely on whether the person
doing their taxes would rather run a command or open a page.

Why Does Open Innovation Matter?

I went into this expecting to write that running locally is cheaper. I measured
it, and on this evidence it is not, and the article is better for saying so.

Token counts, server-reported rather than estimated:

File	Prompt tokens	Completion tokens	Wall clock
design-retainer-2026-09.txt	583	273	57.0 s
office-mixed-2026-09.txt	573	400	72.9 s
saas-subscription-2026-10.txt	508	258	50.4 s
Total	1664	931	180.3 s

The token counts came out identical on a second, independent run. Worth
knowing: tokenizing a fixed prompt is deterministic even though generation is
not, so the cost side of this table is reproducible and the time side is not.

Priced against Gemini 2.5 Flash-Lite at $0.10 per million input and $0.40
per million output tokens (Google's published rate, page last updated
2026-10-14, the cheapest tier on that page), those 2595 tokens cost
$0.000539, about $0.00018 per invoice.

Now the same run as electricity. At an assumed 45 W whole-machine draw for
180.3 seconds and $0.15/kWh, that is $0.000338. The hosted API is 1.59×
the electricity. At 300 invoices a year the two differ by roughly 2 cents.

So the money argument is a wash, and anyone who tells you local inference is
free is selling something. What you actually buy is not cost:

The data does not leave, and the cheap option specifically does not keep it
private. Google's own pricing page states that on the free tier, inputs and
outputs are used to improve Google's products; the paid tier does not. So the
"just use the cheap API" path means a freelancer's client names, project names
and amounts go to a third party, either for money or for training data. For a
one-person business whose product is discretion, that is the whole argument.

No account, no key, no quota, no deprecation. Nothing to rotate, nothing to
be rate-limited by, no model version pulled out from under you mid-quarter.

It works on a plane, which matters more than it sounds for anyone who does
their books on a trip.

And the honest price of all that is time: one to one and a quarter minutes
per invoice on four cores (180.3 s and 223.3 s on two runs of the same three
invoices), against roughly two seconds for a hosted API. That is the real
trade, and it is a good trade for a few hundred invoices and a bad one for five
thousand. The README says so in those words, and it also says what the opt-in
GPU flag does to the number: the same three invoices in 26.1 s on an RX 6600,
about 9 seconds each.

One measured detail that surprised me: the system prompt is 986 characters, and
the measured prompt for the first file was 583 tokens. Roughly 40% of what you
send on every call is instructions, not invoice. That overhead is identical
whether you run it locally or pay per token, so it is not an argument for
either side. It is just the thing nobody tells you.

My Agent Session

This article was written with AI assistance. Every number in it was measured on the machine described in the demo above, and the scripts that produced them are linked below so you can rerun them.

This was built by an AI agent, and the interesting part of the session was not
the code, it was the dead ends. Every runtime claim above was falsified with
evidence before it was abandoned, and the full investigation is written down in
the repository rather than in a chat log that will be gone next week:

• README.md, "Runtime notes (why this shape)" is the public version of the gpt4all post-mortem, including the assertion matrix that ruled out context size and GPU memory.
• tools/measure_tokens.py is where the token counts in the cost section come from. It reads the server's own usage block instead of dividing characters by four, because a cost comparison built on a guessed token count is worthless.
• tools/bench_backends.py produced the thread table, split into prompt processing and generation, from the server's own timing block.

The thread table above has a history worth telling, because it was wrong twice
over and the second error is the part that matters. The first draft of this
post carried a thread table headed "seconds per token" from a script that
actually measured seconds per output line. Going back to reproduce the numbers
found the worse problem:
bench_threads.py
could not produce a number at all. Its worker unpacked two command-line
arguments and the caller passed one, so every run died with an IndexError
inside the worker and printed an empty row that looked like a measurement. The
mismatch had been sitting in the initial commit, which means the published
table cannot have come from the script as committed. Both are fixed
(66a6e9a,
e044102), the
table above is re-measured, and run-to-run spread is 10 to 20 percent: the
shape of the curve is the finding, not the third digit.

The GPU section has its own list, and it is longer. Upstream llama.cpp
changed the default of --n-gpu-layers from 0 to auto between the two
builds this project uses, which means a flag you do not pass is a flag you do
not control: the first "CPU control" run on the Vulkan build silently
offloaded to the integrated GPU and reported a generation rate the CPU cannot
physically reach, and that physical impossibility is the only reason it was
caught. On that same build --n-gpu-layers 0 does not even mean CPU compute,
llama-bench reports backend: Vulkan with the weights still in system RAM.
And omitting --device is not "use device 0": it measured half the
explicit-device rate and matched no single GPU on the machine. Each one is
written up with its raw logs in
docs/bench-vulkan.md.

I did not use DevRelay to save and embed the session, so the agent_session
tag is not in this post. Installing it means putting a third-party MCP gateway
on the machine, which was not a call I wanted to make on someone else's behalf;
the links above cover the same ground and will not expire.

Prize Categories

Best Use of Gemma (Featured, $200). The whole extraction path is Gemma 2 2B
IT, Q4_K_M, running locally through llama.cpp. The project would not work
without it, and the model choice is also why the privacy argument holds: a 2B
model that fits in 1.6 GB is what makes "runs on the laptop you already own"
true rather than aspirational.