"BillBox: A Local AI That Reads GST Bills, and a Checksum That Doesn't Trust It"

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

Every month, small shops and freelancers in India end up with a pile of GST bills: phone photos, WhatsApp images, emailed PDFs. Before filing, somebody retypes them into Excel by hand. It's slow, and one misread digit can mean a wrong tax claim.

BillBox is a local-first bill reader. You drop in bill photos or PDFs, an open-weight vision model running on your own machine reads them, and plain code verifies every number before it reaches a spreadsheet. Anything doubtful gets flagged instead of silently accepted. The output is a colour-coded Excel file with a one-line summary and an expense category for each bill.

Who would pay: chartered accountants who handle dozens of small clients and lose hours every month to data entry, and small business owners who'd rather not pay for that typing. My pricing idea is a flat monthly fee for individuals and a per-batch fee for accountants. That's a hypothesis, not something I've validated.

The design rule fits in one sentence: the AI reads, the code decides. A vision model is good with messy photos and occasionally wrong about a digit, so I never let it decide whether a bill is correct.

Demo

The demo shows BillBox end to end: reading bills, the colour-coded review table where problem cells are highlighted, and the Excel export.

Code

ankitsingh0101 / BillBox

BillBox

Read GST bills on your own computer with an open-weight AI model, verify every number with plain code, and export a clean Excel file for your CA.

BillBox turns a pile of photographed bills and PDF invoices into a reviewed, colour-coded spreadsheet. A local vision model reads each bill, deterministic rules check the result, and anything doubtful is flagged for a human instead of slipping into a tax filing. Nothing is uploaded to any cloud.

Built for the Hacktoberfest Open-Source AI Challenge (Week 2: Side Hustle).

---

Table of contents

1. The problem and who pays for the fix
2. GST in 60 seconds
3. How it works (workflow)
4. The core idea: AI reads, code decides
5. What every field means
6. The checks, one by one
7. Project structure: what every file is for
8. The Excel output
9. Privacy and data handling
10. Install and run
11. Measuring accuracy
12. Tests
13. Configuration
14. Tech stack and why
15. API reference
16. Troubleshooting
17. …

View on GitHub

MIT licensed. The README walks through the workflow, every field and every check. There's also an optional Docker setup.

How I Built It

The stack: Python, FastAPI, a plain HTML and JavaScript front end, Pillow and PyMuPDF for images and PDFs, openpyxl for Excel, and Ollama running Gemma 3 (4B, vision) as the model. No cloud API is involved.

The pipeline:

bill photo/PDF -> local vision model -> clean and repair -> validators -> OK / CHECK / REVIEW -> Excel

1. The image goes to Gemma through Ollama with a JSON schema, so the answer comes back as structured fields: vendor, GSTIN, invoice number, date, taxable value, CGST/SGST/IGST, total, plus a short summary and a category.
2. Python normalises the output, then runs the checks. A GSTIN's last character is a checksum, so a misread character usually fails it. The tax amounts must add up to the total (within Rs 1 for round-off). CGST must equal SGST, a bill can't mix IGST with CGST/SGST, the effective rate must be a real GST slab, and duplicate invoices are caught even across separate uploads.
3. Each bill gets a status, and the review table highlights exactly the cells that caused it. Editing a cell re-checks everything instantly.

What the 4B model got wrong on my own machine:

• It skipped the vendor GSTIN. My first schema didn't mark the field as required, so the model simply left it out. Making every field required fixed that.
• It invented a tax. On a bill with only IGST, it returned an SGST amount that wasn't printed anywhere. The "taxable + taxes must equal total" check and the "never mix IGST with SGST" rule flagged it, and I tightened the prompt so a tax is only filled if its label is actually printed.
• It misread characters inside a GSTIN. Reading a Z as 2, or a letter O as a zero, breaks the checksum, which is exactly what the checksum is for. So I added a repair step: if swapping exactly one look-alike character (0/O, 1/I, 8/B, 5/S, 2/Z, 6/G) makes the checksum pass, the fix is accepted. Otherwise the GSTIN is left alone and flagged. The checksum acts as the judge, so the code can prove a repair but never guesses. For example, 27GOBOY6505B42U was corrected back to 27GOBOY6505B4ZU, and a note on the bill shows what changed.
• It added an extra character. One read came back as a 16-character GSTIN, and a GSTIN is always 15. That's more than one error, so the repair step correctly refused to guess and the bill stayed red. That made me change the pipeline so a malformed GSTIN, not only a missing one, triggers a second, narrower question to the model ("list every GSTIN on this bill with its label").

Every automatic repair shows up as a blue note on the bill, so nothing changes silently.

Using AI to build it: I built BillBox with Claude as a pair-programming assistant, and I tested and debugged everything on my own machine. The debugging story above comes from my own runs.

Why Does Open Innovation Matter?

Bills are some of the most private paperwork a small business has: GSTINs, client names, amounts. With a closed API, every bill would be uploaded to someone else's server, and every page would add to a running cost.

Running an open-weight model locally changed three things:

• Privacy. Bills never leave the laptop when BillBox runs on the user's own computer. For an accountant handling dozens of clients' paperwork, that's a real selling point, not a nice-to-have.
• Cost. There's no per-page fee, so a batch of 500 bills costs the same as 5. That's what makes a cheap monthly price workable for a small business.
• Control. I could constrain the model's output with a schema, swap models with one environment variable, and design around its weaknesses. A closed model doesn't let you inspect or adjust that. The same openness is what would make fine-tuning on Indian bill formats possible later, which a general-purpose API handles poorly.

The trade-off is honest: a 4B model on a laptop is slower and misreads more than a frontier API would. BillBox's design (checks, flags and repairs) exists because of that, and I think it makes the product safer, not weaker.

My First User

My first user was my maternal uncle, who runs a small shop. He doesn't have a laptop. Everything he does is on his phone, and that's the gap I built BillBox for: bills that live as photos on a phone, with nowhere organised to go.

There was an obvious problem. BillBox and the Gemma model run on a computer, and he only has a phone. Here's how I solved it for the test:

1. I ran BillBox and Ollama on my laptop, so the AI reading happens there.
2. I started a Cloudflare Tunnel with one command, cloudflared tunnel --url http://localhost:8000, which gives the app a temporary public web address.
3. I sent him that link. He opened it in his phone's browser and picked photos of his bills from his gallery. He didn't install anything.
4. My laptop read the bills and sent the results back to his screen. When we finished, I stopped the tunnel and the link stopped working.

His verdict: it was somewhat beneficial for him. I'll take that as an honest review from someone who has real bills and no spare time.

What I learned from watching a real shop owner on a phone:

• The screen layout is built for a laptop. The review table is wide, so on a phone it scrolls sideways. It works, but a proper phone-first layout is the most important next step.
• The speed depends on the machine doing the reading. All the AI work happens on my laptop, so he only waits as long as that machine takes.
• The privacy promise has a limit. BillBox's promise is that nothing leaves your own machine, and that holds when someone runs it on their own computer. While the tunnel was live, his photos travelled through Cloudflare to my laptop. That's the trade-off of sharing it this way, and it's why I don't recommend the tunnel for real client data. For a shop like his, the better answer is an app on his own device or a small local server in the shop.
• One relative is not proof of demand. I haven't tested pricing or whether he'd pay, so I can't claim that yet.

What I haven't proven

• My test bills are mostly synthetic, so accuracy on messy real-world bills is likely lower. I've only tried a small number of real ones.
• summary and category are AI-written and not verified by code. They never change a bill's status.
• Handwritten and very blurry bills are much less reliable than printed invoices.
• No line items or HSN codes yet, and it isn't tax advice. A qualified accountant should review anything before filing.

Prize Categories

• Best Use of Gemma: BillBox runs Gemma 3 locally through Ollama as the model that reads every bill.