DEV Community

Cover image for I Built a Receipt Scanner for Myself: 15 Bills, Zero Wrong Numbers
Melwin Siril Joel V
Melwin Siril Joel V

Posted on

I Built a Receipt Scanner for Myself: 15 Bills, Zero Wrong Numbers

Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle Submission 💸

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

LedgerLens turns a folder of receipt photos into an accounting CSV, then finds the bills you already paid.

Point it at a folder of .jpg/.png receipts. It runs OCR on your own machine, extracts the vendor, date, invoice number, line items and total from each one, writes one clean row per receipt to data/ledger.csv, and flags the things a human skimming a spreadsheet misses: a bill entered twice, a price that jumped, a total far outside what that shop normally charges, or items that do not add up.

It is for the freelancer or small shop owner who keeps a shoebox of receipt photos and no useful record of them. The information is all there, nobody has time to type it in, and the cloud services that would do it charge per page and upload every receipt to a server you do not control.

That last part is the whole point. A shop's receipts carry client names, GST numbers, addresses and staff details. LedgerLens never uploads an image anywhere. Tesseract runs as WebAssembly inside Node, so there is no Python, no native toolchain, no API key, and no per-image bill. After a first run that downloads about 11 MB of English language data, it works with the network switched off.

Demo

$ npm run pipeline

         78.0 conf          0.5s  receipt-01.png
         82.0 conf          0.2s  receipt-02.png
         78.0 conf          0.4s  receipt-03-duplicate.png
         76.0 conf          0.3s  receipt-04-stationery.png

4 image(s) -> data/raw
    ANNAMAIYA PROVISION STORES       2026-10-12    1451.34  ok  via total
    ANNAMAIYA PROVISION STORES       2026-10-14     575.52  ok  via total
    ANNAMAIYA PROVISION STORES       2026-10-12    1451.34  ok  via total
    SARASWATHI STATIONERY            2026-10-15     649.64  ok  via total

4 receipt(s), 4 with a resolved total -> data/ledger.csv
  receipts : 4
  findings : 1

  [HIGH] duplicate_invoice
      Invoice 2291 from ANNAMAIYA PROVISION STORES appears 2 times at an
      identical 1451.34. Likely a double scan or a double payment.
Enter fullscreen mode Exit fullscreen mode

Those are the synthetic samples the repo ships. On real photographs it reads the paper and hands back one row per bill.

Code

GitHub logo Emanuelmsjv / ledgerlens

Offline receipt OCR to accounting CSV with anomaly detection. Tesseract via WebAssembly in pure Node - no native toolchain, no cloud, no API keys.

ledgerlens

Photograph a receipt. Get an accounting CSV. Find the bill you already paid.

Runs entirely on your own machine. Tesseract OCR executes as WebAssembly inside Node, so there is no Python, no native toolchain, no API key, and no image is ever uploaded anywhere. After the first run — which downloads about 11 MB of English language data — it works with the network off.

Built for the Hacktoberfest Open-Source AI Challenge, Week 2: Side Hustle.

Why it exists

A small business or freelancer keeps a folder of receipt photos and no useful record of them. The information is all there; nobody has time to type it in Cloud OCR services do the reading and bill per page, which for a shop doing hundreds of small purchases is a real recurring cost — and they upload every receipt to a server the owner does not control.

Financial…

How I Built It

Plain Node.js, one dependency (tesseract.js), no build step. The pipeline is four small scripts:

script what it does
ocr.mjs images to text, with rotation recovery
parse.mjs text to one CSV row per receipt, plus a self-check
analyze.mjs flags duplicates, price jumps, outliers, bad arithmetic
make-fixtures.mjs redacts real OCR output into safe test fixtures

The design rule everything else follows from: if a field cannot be read confidently, it returns null, never a guess. In a ledger, a blank cell is a nuisance; a confidently wrong figure is a false expense that gets filed and never questioned again.

The bug that rule comes from

My first parser took the last line containing "Total". On one real bill, three lines contained the word. It returned 459.00 where the bill said 4600.00 — with no warning, no low confidence, nothing.

I did not trust my memory of how bad that was, so I rebuilt the original logic and ran it against every real bill I had ground truth for. It was worse than I expected:

bill true total original logic shipped
railway ticket 180 (nothing) 180.00
bakery 593 102.00 (wrong) 593.00
bakery (other photo) 593 102.00 (wrong) 593.00
grocery 88 88.00 88.00
credit sale 4600 459.00 (wrong) 4600.00

Original: 1 right, 3 wrong, 1 abstained. Shipped: 5 right, 0 wrong.

The fix was not a better regex. It was label priority: prefer a NetAmount or balance line, never take a tender or "savings" figure, and when two candidates disagree, abstain. Measured, not claimed.

What the benchmark says

Two suites, reported separately and never averaged together:

synthetic real
cases 7 8 photos, 4 distinct bills
ground truth fixed when written read off the paper by eye
totals correct 7 / 7 7 / 8
wrong 0 0
wrong-rate of produced totals 0% 0%

14 of 15 documents, and no wrong number anywhere. The single miss is the system working: on one photograph it could not read the total cleanly, so it reported nothing. Coverage is the secondary metric; a wrong number is the only real failure.

The same rule caught a GSTIN that OCR had turned from 33ZZQPX1234F1Z2 into 332ZQPX1234F122 — a shape that is no longer a valid GSTIN. The parser returned null and left it for a human, instead of filing a mangled tax number on a real invoice.

The suite is covered by 68 checks across three files (30/30, 12/12, 26/26), plus a 15-check privacy lint wired into CI.

And the part I got wrong about privacy

Halfway through, I checked whether the real receipts had leaked into the public repo. The photos had not — samples/real/ was gitignored from day one. But the redacted fixtures derived from them were incomplete: real shop names, a street address, a pincode and bill numbers were sitting in fixtures, a benchmark file, and even the CI workflow. Worse, the test suite depended on a gitignored file, so the privacy guard in CI had never actually run — it crashed before reaching the check.

I fixed it properly: rewrote the redaction rules, added a fixed-point lint that fails the build if a fixture is not idempotently redacted, made the tests hermetic so the guard runs on a clean clone, and purged the leaked content from git history. The public repo now contains only fictional vendors. The gate reports 15/15.

I am including this because a tool whose whole pitch is "your data stays yours" has to hold itself to that, and mine did not at first.

Why Does Open Innovation Matter?

For this project, open isn't a preference. It is the reason the thing works at all.

A closed receipt-OCR API bills per image. For a shop doing hundreds of small purchases a month, that is a recurring cost that eats the margin it is supposed to protect — and every receipt, each one a page of someone's financial life, gets uploaded to a server they do not own. Running the model locally removes both the metered cost and the data exposure in one move. That is the entire product.

It also means I can trust it. The extraction rules and the five detectors are plain code I can read, test and fix. When the "Total" bug turned up, I could rebuild the old logic and measure it against real bills the same afternoon — no black-box API to wait on, no behaviour changing under me. Everything runs offline, on CPU, with no key and no account. That combination — no metered cost, no third-party exposure, and rules you can audit — is what open innovation made possible here. I do not think the closed version of LedgerLens is worth buying, and I do not think that is a coincidence.

My Agent Session

I built this end to end with an AI coding agent, including the measurement above and the privacy remediation. The session below is the slice where I stopped trusting a claim, rebuilt the original parser, and found the bug was worse than I had said.

LedgerLens: measuring a receipt-total bug instead of claiming a fix
Agent

Benchmark done and committed. Stopping here as you asked.

Benchmark results

synthetic real
cases 7 8 photos / 4 distinct bills
ground truth fixed when written read off paper by eye
correct 7 / 7 7 / 8
abstained 0 1
wrong 0 0
coverage 100% 88%

14 of 15 documents, zero wrong numbers.

The headline isn't accuracy - it's wrong-rate of produced totals. A confidently wrong figure gets filed and never questioned; a blank one is just annoying. The one abstention is a photo OCR couldn't read, where the tool said nothing rather than guessing.

The most interesting result

The one synthetic "failure" is the system working:

GSTIN 33ZZQPX1234F1Z2   (rendered)
GSTIN 332ZQPX1234F122   (what OCR read)

OCR turned Z into 2 twice, breaking the shape - so the parser rejected it and returned null. A sloppier parser files 332ZQPX1234F122 as a tax registration number. So: GSTIN coverage 3/4, and every GSTIN this tool emits is well-formed.

Two judgement calls

I implemented GSTIN check-digit validation, then deleted it. It would recover that case - but the only real GSTIN I had was itself OCR'd from an image, so validating against it is circular. Four algorithm variants disagreed with it, which pointed at the bad input, not bad code. An unvalidated validator silently nulls good GSTINs - the exact failure this project fights. Not shipped.

I caught myself leaking. The redaction script named the towns, depots and branches it was stripping - publishing exactly what it was there to hide. And the synthetic cases were seeded with your real GSTIN, depot name and receipt number. Both fixed; a leak check that only inspects fixtures isn't a leak check.

State

Emanuelmsjv/ledgerlens - 4 commits, pushed, 69 tests green, ~48 h left.

Next when you're back: the DEV post. Everything it needs is measured and honest.

You

lets plan first whats the status , what are the next steps what should be done today to push it to complete

Agent

Let me gather the actual state before planning.

Agent

Everything's in place, including the session curator from Week 1. Let me check the deadline math precisely - this determines the whole plan:

Agent

Status

The build is done. Only the submission is left.

Emanuelmsjv/ledgerlens 4 commits, pushed, 33 files
Tests 69/69 green
Benchmark synthetic 7/7, real 7/8, 0 wrong
Repo privacy clean - leak-scanned every staged file
Time left 2 days, 0.2 hours (Sun 18 Oct, 23:59 PDT)

What's already built and needs nothing

The pipeline (ocr -> parse -> analyze), rotation recovery, the five anomaly detectors, 69 tests, CI, the benchmark, and the redaction pipeline. All measured, all honest, all pushed.

What's missing - everything submission-shaped

docs/dev-post-week2.md        MISSING   the write-up
session for the embed         MISSING   a new DEV agent session
PUBLISHED-BODY.md             MISSING   the paste-ready body

The constraint that shapes the plan

A real bug cost us Week 1. I ran --body "probe" against your published post and overwrote it. You were right to stop me.

So the rule for this week: I never write to your DEV account again. I produce files; you paste. That costs10 minutes and removes the only way this fails badly.

What I'd do today, in order

1. Commit the post draft (~90 min)
Write it against the Week 2 template - all 7 headings. The material is unusually strong because it's mostly measurements:

claim source
14/15 documents, 0 wrong docs/benchmark.json
the 102.00 savings-as-total bug real incident, real receipt
GSTIN rejected rather than mangled GSTIN 332ZQPX1234F122 -> null
69 tests, negative ones included npm test
OCR boundary is privacy, not cost "financial receipts never leave the machine"

2. Agent session embed (~40 min, optional)
The Week 1 curator still exists in temp. It scrubs, ASCII-folds (both bugs we hit), and has a security gate I proved fires. Needs: export this session -> curate -> submit -> get ID.

3. You paste and publish (~15 min, you)

4. Optional TabPFN ($200 category)
I'd skip this. It needs a cloud API key and license acceptance, and it would consume the margin for a "nice to have" on a 2-day clock. Your real differentiator is offline - a cloud API undercuts it. Claim GitHub Copilot ($100, you already use it) and Overall instead.


One thing I want your call on

The benchmark says 3/4 GSTINs and I decided not to ship check-digit validation because I couldn't validate it. That's an honest gap - but it's also the most obvious thing a judge will probe.

Do you want me to spend ~1 hour trying to source a trustworthy real GSTIN to validate the check digit against, or leave the gap documented and move to the post?

Agent
Agent
Agent

Plan - Week 2 "Side Hustle", 2 days left

Status

Repo Emanuelmsjv/ledgerlens, 4 commits, pushed, 33 files
Tests 69/69 green
Benchmark synthetic 7/7 . real 7/8 . 0 wrong totals
Privacy clean - every staged file leak-scanned
Missing the write-up, the session embed, the paste-ready body

The build is done. Nothing below touches the pipeline.

Your decisions, locked in

  • GSTIN check digit: spend the hour sourcing ground truth
  • Categories: Overall + Entire

Entire means the agent-session embed is not optional this week - it's the category you claimed. It moves from nice-to-have to required.

Ground rule for this week

I never write to your DEV account. Week 1 I clobbered your post with --body "probe". I produce files; you paste. That costs 10 minutes and removes the only failure mode that can actually hurt you.

Today's plan

1. GSTIN check digit -1 hour, self-killing

Find published GSTINs I can verify independently (GSTN lookup, vendor invoice samples), implement mod-36, and test against them.

Abort condition: if no trustworthy ground truth emerges, delete the implementation and document the gap - same as last time. The benchmark already shows the parser rejects malformed codes, so the risk of shipping is "nulls a valid GSTIN," which is worse than the gap.

Gate before it ships: benchmark must still show 0 wrong, and GSTIN must go 3/4 -> 4/4.

2. Agent session - 40 min, required for Entire

The Week 1 curator is still in temp and already solved the hard parts:
- scrubs email, MLH IDs, home path
- ASCII-folds, because both DevRelay and the DEV editor mangle UTF-8
- has a security gate I proved fires by planting a fake OpenAI key

Export this session -> curate -> submit -> get ID. I'll verify the stored copy server-side, not the local one - that's how I caught the mangling last week.

Risk: this session contains ledgerlens work plus a lot of Week 1 context. If curation looks messy, I'll cut a tight slice rather than submit5 MB of transcript.

3. Write the post - 90 min

All 7 template headings. The material is unusually strong because it's measurements, not claims:

Section Evidence
What I Built offline receipt OCR -> accounting CSV + anomaly detection
Demo real terminal output from npm run pipeline
Code {% embed %} the repo
How I Built It Tesseract WASM in pure Node, no Python/native/API
Why Open Innovation the 102.00 bug - savings line returned as the receipt total, with no error raised. That story is the argument
My Agent Session the embed
Prize Categories Overall + Entire

Strongest beat: OCR read a GSTIN's Z as 2, and the parser rejected it and returned null rather than filing a mangled tax ID.

4. You paste - 15 min

PUBLISHED-BODY.md -> https://dev.to/new -> tags devchallenge, hf26challenge.

I'll verify read-only afterwards: tags present, embed rendered as a card, not raw Liquid.

What I'm deliberately not doing

  • TabPFN - cloud API key undercuts the offline premise, and 2 days is thin
  • Deploying - a locally-run tool that needs no hosting is the better story
  • Publishing anything real from your receipts - not without asking you first

Sequence and timing

Block Time Gate
GSTIN ground truth 1 h 4/4 or delete
Session embed 40 m stored copy verified
Write post 90 m all 7 headings present
You paste 15 m -

Leaves ~20 h buffer against a Sunday deadline.

One thing I need before starting: step 2 puts a transcript on a public profile. It covers ledgerlens, but it also sweeps in earlier context from this week. Do you want me to curate tightly to ledgerlens work only, or include the broader session the way Week 1's embed did?

You

so only the weel 2 work and start

Agent

Starting. Tight ledgerlens-only slice for the embed.

Agent
Agent

914 messages now (was 577). Ledgerlens starts at 718 - so ~196 messages. Let me check that range's quality and size:

Agent

196 messages / 3.13 MB - too big to embed. I need a tight arc. Also, important: I need to verify a claim before I write it into the post. Let me locate the key moment:

Agent

Now let me verify a claim before it goes in the post - I asserted my first parser would return 102.00, but I never actually ran it on real data. Let me make that a measurement instead of a claim:

Agent
Agent

Measured, not claimed. And it's worse than I predicted:

file truth original shipped
railway ticket 180 ---- 180.00
bakery 593 102.00 x 593.00
bakery (other photo) 593 102.00 x 593.00
grocery 88 88.00 88.00
credit sale 4600 459.00 x 4600.00

Original: 1 right, 3 WRONG, 1 abstained. Shipped: 5 right, 0 wrong.

Agent

PowerShell mangled the commit message mid-way (the \" broke it). Let me check and fix with a file:

Agent
You

and lets run some couple of debugging sessions and security test

Agent

Fixing the message, then on to debugging and security testing.

Agent

Now the important part. Let me start with the highest-stakes check - whether real receipt data ever leaked into git history, not just the working tree:

Agent

samples/receipt-0*.png are the synthetic ones (safe). Now the real test - can anything personal be recovered from the committed fixtures? Let me audit them adversarially:

Agent

Now the adversarial part - let me scan the committed fixtures for anything that still identifies a real person or business:

Prize Categories

I'm entering:

  • Overall — a genuinely useful, fully offline side-hustle tool with measured correctness.
  • Entire — the agent session above shows how it was built and where it went wrong.

Top comments (0)