This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle
What I Built
Every April or end of financial year, self-employed people across Alberta sit down with a shoebox of faded receipts and a spreadsheet. I wanted the boring part, reading and sorting, to happen in minutes without handing a year of financial data to a cloud service.
Shoebox is a small app for self-employed people in Alberta. You drop in photos of your receipts and get back a year of expenses sorted the way CRA's T2125 form wants them, with the GST worked out. It runs entirely on your own computer.
- Drop in receipt photos. A local open-weight vision model, Google's Gemma 3 (4B) running through Ollama, reads the vendor, date, subtotal, GST, tip and total.
- Shoebox checks the AI's work with plain arithmetic, not more AI. It flags any receipt where subtotal + tax + tip doesn't equal the total, where the tax isn't Alberta's 5% GST (a BC receipt with PST, or Ontario HST), where the date is in the future, or where it looks like a duplicate.
- Each expense goes to a T2125 line. Gas goes to 9281 Motor vehicle, coffee with a client to 8523 Meals (counted at 50%), printer ink to 8811 Office supplies. Business-use percentages for your vehicle and phone are applied.
- GST is handled both ways. If you're GST-registered, Shoebox deducts the pre-GST cost and totals your input tax credits. If you're not, the GST is part of your expense.
- Export a CSV for your accountant, or for yourself at tax time.
Every flagged receipt opens side by side with its photo, so fixing a misread takes seconds:
Who would pay for it? Ride-share and delivery drivers, realtors, hairstylists, trades and consultants. These are people with a few hundred receipts a year who either pay a bookkeeper by the hour or lose deductions because the shoebox never gets sorted. Because it runs locally, there's no per-receipt cost to pass on. That means a one-time price works, instead of yet another monthly subscription.
Demo
Try it: download for Windows, macOS or Linux from the Shoebox 0.1.0 release. Every push also runs Gemma 3 on the sample receipts: see the CI runs.
Code
Shoebox 🧾
Sort a year of receipts for tax time, privately, on your own computer.
You drop in photos of your receipts. A local open-weight AI model (Google's Gemma 3, run through Ollama) reads the vendor, date, subtotal, GST and total. Shoebox then:
- checks the arithmetic itself, flagging any receipt where the numbers don't add up, the tax isn't Alberta's 5% GST, or the date looks wrong;
- sorts each expense into its CRA T2125 line;
- totals your deductible expenses and GST input tax credits;
- exports a CSV for you or your accountant.
It's built for Alberta freelancers and gig workers: drivers, realtors, stylists, trades, consultants.
Nothing leaves your computer. The web page is served from 127.0.0.1 only. The AI model runs on your own machine. Receipts are stored in a normal folder (~/Shoebox) that you can back up or delete.
What it does
…How I Built It
Stack: .NET 8 (an ASP.NET Core minimal API) serving one self-contained HTML page, with Ollama running Gemma 3 4B for vision. There are zero third-party NuGet packages and no CDN scripts or web fonts, so the whole thing works offline and there's very little supply chain to trust with financial data.
Reading receipts. Each photo goes to Ollama's /api/chat with a JSON schema in the format field (Ollama's structured outputs). The model must answer with vendor, date, subtotal, gst, other_tax, tip, total, items and a category drawn from a fixed list of T2125 codes. Temperature is 0. The parser is still lenient: it accepts "$12.50" as a string, month names in dates, and stray chatter around the JSON.
Not trusting the model. A 4B model will misread a digit sometimes. Rather than pretend otherwise, Shoebox sorts its findings into two kinds:
- Problems keep a receipt flagged until it's fixed: the numbers don't add up, there's no total, or the date is invalid or in the future.
- Cautions clear once you've looked and saved: PST on an out-of-province receipt, a GST rate under 5% (some items may be exempt), or a possible duplicate.
A misread total becomes a highlighted row to check, not a silent error in your tax numbers.
Categories: rules first, AI second. Keyword rules handle the obvious cases (Petro-Canada, Tim Hortons, Canada Post, Telus to name a few and many more) predictably and explainably. Gemma's suggestion is only used when no rule matches, and the UI shows whether a category came from a rule, the AI or you.
Tax maths in C#, not in the model. Meals are counted at 50%. Vehicle and phone use your business-use %. GST registrants get deductible = (total − GST) × share and ITC = GST × share. Non-registrants deduct total × share. There are 23 built-in self-tests covering this maths, the checks, category rules and CSV safety: text read off a receipt like =HYPERLINK(...) is neutralized so it can't run as an Excel formula.
Privacy by construction. The server binds to 127.0.0.1 only, rejects requests for any other host name (to block DNS-rebinding tricks), and requires a custom header on every change, so a website open in another tab can't upload, edit or delete anything.
Testing with a real model. On every push, GitHub Actions builds and self-tests on Windows, macOS and Linux, then installs Ollama, pulls gemma3:4b, and has it read six clearly-labelled sample receipts from fictional businesses. Each field is scored against the known values and the results are posted in the run summary:
| Field | Vendor | Date | Subtotal | GST | Total | Category |
|---|---|---|---|---|---|---|
| Correct (of 6) | 6 | 6 | 6 | 4 | 6 | 5 |
Each receipt took about 80 seconds on GitHub's CPU-only runner. A laptop with a decent GPU is much faster.
The two misreads are exactly why the checks exist:
- Courier receipt: Gemma read the GST as $25.00, copying the subtotal line, instead of $1.25. Shoebox flagged it twice: "Amounts don't add up (off by $23.75)" and "GST is 100% of the subtotal."
- Grocery receipt: the receipt clearly printed $0.00 GST (basic groceries are zero-rated). Gemma "helpfully" filled in $1.87, which is exactly 5%. The model assumed Alberta GST instead of reading it. Shoebox flagged it: "off by $1.87."
Both wrong amounts were caught; neither slipped into the totals. The one thing no check can catch is judgement: Gemma filed the groceries under "Other" instead of "Personal". That's why every category shows whether it came from a rule, the AI or you.
Why Does Open Innovation Matter?
The business model only works with an open model. A cloud vision API charges per image. That's fine for a demo, but it turns a sorting tool into a subscription that has to cover someone else's inference bill. With Gemma running locally, reading the 500th receipt costs the same as the first, which is nothing. That's what makes a one-time-price tool for a freelancer viable.
The data is the most sensitive kind. A year of receipts shows where you drive, what you buy, who your clients are and roughly what you earn. Shoebox's answer to "where does my data go?" is "a folder on your laptop". That isn't a privacy policy; it's how the software is built.
I can tune it for a niche general tools handle poorly. Alberta has GST but no PST, CRA has its own T2125 lines, and meals are counted at 50%. Owning the whole pipeline means the prompt, the categories and the checks all speak Canadian tax. I can swap in a bigger model (--model gemma3:12b) for someone with a faster computer, or fine-tune later on corrected receipts, without asking anyone's permission or changing pricing.
No lock-in. If Ollama, Gemma or I disappear tomorrow, the user still has a CSV and a folder of their own photos.
Honest limits: Shoebox gives estimates, not tax advice. It doesn't handle capital cost allowance, home-office expenses or receipts that mix personal and business items, and a small model on a laptop is slower than a cloud API (about 80 seconds per receipt on a CPU-only machine in my tests; faster with a GPU). For a once-a-month sorting session, that trade-off is worth it.
My First User
I haven't landed one yet: I'm new to Calgary's freelance scene and didn't have contacts to ask this week. So here's an open invitation. If you're self-employed in Canada, download the release, try it on a few receipts, and tell me in the comments what it got wrong and whether you'd pay for it. I'll update this post with what I hear.
Why someone would pay for this
The customer: an Alberta ride-share driver, realtor, stylist or contractor with a few hundred receipts a year. Today they either pay a bookkeeper by the hour to type them in, or they don't sort them at all and miss deductions. That's real money: meals count at 50%, gas at your business-use %, and GST input tax credits are free money for registrants who track them.
What they're paying for: an evening of sorting instead of a weekend, a CSV their accountant can use as is, and a check that catches the misread numbers a tired human (or an AI) would miss.
Why a one-time price works: because Gemma runs on their own laptop, my cost per receipt is zero. I don't have a cloud bill to pass on, so I can sell it once (I'm thinking [$29] as side hustle) instead of adding another monthly subscription to a freelancer's expenses. And "your receipts never leave your laptop" is a selling point no cloud tool can match.
My Agent Session
I built Shoebox with an AI coding agent (Claude), working from the CRA rules down to the arithmetic checks, a security pass on the local server, browser testing on desktop and mobile, and a CI run against a real local Gemma model.
Prize Categories
- Gemma: Gemma 3 (4B, vision) runs locally through Ollama and reads every receipt. It's the app's default model, and CI tests it on every push.
Top comments (0)