Invosorter

I Built a Local Invoice Sorter With Gemma 4 — and Learned Why Validation Matters

This is my submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle.

What I Built

I built Invosorter, a small application that extracts information from Indian invoices and receipts using a locally running Gemma model.

The idea is simple: instead of manually typing every invoice into a spreadsheet, upload the document, extract its fields, review the result, and export it to CSV or Excel.

I had two kinds of users in mind: freelancers managing expenses and small shop owners keeping purchase records.

The application currently extracts fields such as:

• Vendor name, invoice number, date and GSTIN
• Taxable amount, CGST, SGST and IGST
• Gross invoice total, TDS and net payable amount when available

It also checks totals, flags suspicious GSTIN structures, distinguishes inferred values from extracted ones, and lets users correct records before exporting.

The goal isn't to replace an accountant or claim that an invoice is tax-compliant. It's to reduce repetitive data entry while making extraction errors easier to notice and correct.

Demo

https://youtu.be/EbeYEw0Z8TU

The demo shows the complete workflow: uploading an invoice, processing it locally, reviewing extracted fields, correcting a suspicious GSTIN, and downloading the resulting CSV and Excel files.

The current prototype processes only the first page of a PDF. Full multipage invoice processing isn't supported yet.

Code

https://github.com/Aneesh-0108/invosorter

The project is built with Python, Streamlit, Ollama and Gemma 4 E2B. The repository includes the extraction pipeline, validation logic, batch processing, an evaluation script and unit tests.

To run it locally, follow the setup instructions in the README.

How I Built It

I started with a synthetic Indian finance dataset containing invoices and a purchase register with known values. This gave me something important: a way to compare model output against expected answers instead of deciding whether an extraction looked plausible.

The main pipeline is:

1. Convert a PDF's first page to an image when needed.
2. Pass the invoice image to Gemma 4 E2B running locally through Ollama.
3. Request structured JSON using a defined schema.
4. Validate fields and check whether the tax amounts reconcile with the invoice total.
5. Flag suspicious records and allow manual corrections.
6. Export the results to CSV or Excel.

One of the most useful lessons was discovering that valid JSON doesn't mean correct information.

For example, Gemma confused the letter I with the digit 1 in a seller's GSTIN. The extracted string looked almost identical to the original. Instead of silently correcting it, I added structural validation to flag suspicious values while preserving the model's original output.

I also separated gross invoice total, TDS and net payable amount. They represent different financial concepts, and treating TDS as a negative additional tax would have produced misleading data.

What I found during testing

I evaluated the extractor on four synthetic invoices covering intra-state and inter-state GST cases.

The first three had no mismatches among their scored fields. The fourth matched 10 of 11 scored fields; the GSTIN's I versus 1 error was caught and flagged. Some comparisons remained unscorable because the purchase register didn't contain corresponding ground-truth fields.

I also experimented with Tesseract OCR instead of passing the image directly to Gemma.

On one invoice, Tesseract read CGST and SGST as ₹1,639 instead of ₹1,539. The OCR-assisted path also confused the buyer with the seller. Python's arithmetic validation caught the inconsistency in the tax total.

That experiment convinced me not to make OCR the default just because it is fast. It was useful to compare both approaches and understand how an early recognition error can propagate through an AI pipeline.

What I learned

• Multimodal models can extract structured information directly from document images.
• JSON schemas make model integration more predictable, but don't guarantee semantic correctness.
• Deterministic validation can catch inconsistencies that an AI model misses.
• A human review step is useful when an incorrect field has financial consequences.
• Ground-truth evaluation is much more informative than looking at a few successful outputs.

Why Does Open Innovation Matter?

I chose Gemma because I wanted to experiment with a model that I could run and integrate locally, rather than making every invoice dependent on a hosted AI API.

The application sends inference requests to the local Ollama endpoint. It doesn't need to send invoice contents to a remote model service, and it doesn't incur a per-invoice cloud inference API charge. The initial model and software downloads still require an internet connection.

Local inference also gave me room to experiment with the extraction pipeline, inspect failure cases, write my own validation rules and decide what happens when the model returns a suspicious value.

There is a trade-off. I'm running this on a CPU-only laptop, and inference currently takes roughly one to two minutes per invoice in my observed runs. This is a prototype, not a claim of production-grade speed.

For this project, open innovation meant being able to learn from the model's failures and build around them, not just use the model for a successful demo.

What Comes Next

I'd like to improve support for multipage PDFs, evaluate more varied invoices, and continue measuring whether OCR-assisted extraction is useful on different document layouts.

The most important takeaway for me was that getting an AI model to return data is only the beginning. Building a useful application also means measuring errors, handling uncertainty, validating outputs and letting the user make the final correction.