Confere: Local Receipt Reading With a Paper Trail

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle.

What I Built

A receipt photo is not yet an expense you should trust.

It might be the same photo sent twice. A number might be a subtotal rather than the amount paid. A perfectly readable total can still belong to a different date or currency. For a freelancer handing expenses to a bookkeeper, the useful outcome is a reviewed list with a way back to the source.

I built Confere, a bilingual receipt workspace for Brazilian freelancers. An open neural OCR model reads photos on the device. The user compares the original image, recognized text and suggested fields, then approves each expense. Only approved receipts count toward the total or appear in the exports.

The workflow is intentionally small:

1. Add PNG, JPG or WebP images.
2. Inspect the source, correct the merchant/date/BRL amount and choose a category.
3. Resolve possible duplicates before approval.
4. Export a CSV or a self-contained evidence packet containing the original images, recognition text, corrections, source-image hashes and review timestamps.

My background is in accounting, so I wanted the handoff to be inspectable instead of stopping at a neat-looking number. A freelancer or small bookkeeping practice could plausibly pay to reduce receipt re-entry and organize that handoff. The prototype is free; local deployment, tailored exports and support are possible service offerings. That is a business hypothesis, not evidence of paying demand. I have not claimed customers, revenue, interviews or measured time savings.

Demo

Open Confere — English and Portuguese interfaces, no account or API key required.

For a reproducible walkthrough, click Try 3 samples. That creates three clearly marked fictional receipt images and actually runs the recognition model on them. Two are distinct receipts; the third is a byte-identical copy of the café receipt. They are not prefilled OCR responses.

Try approving the duplicate first: it is blocked. Exclude the copy, inspect and approve the two originals, and the approved total becomes R$ 159,80. Download the evidence packet to see what remains attached to those numbers. The packet contains its own images and opens independently of the app.

Code

Source repository · Verification record · Actual sample exports

The application is MIT licensed. All substantive application code was started from scratch on October 13, inside this challenge's window. It is a new project, separate from my earlier submissions. Codex assisted implementation, documentation and testing; third-party code and model licenses are credited in the repository.

How I Built It

The open AI component is Tesseract's LSTM neural recognition engine, running through Tesseract.js 6.0.1 and tesseract.js-core 6.1.2 in a browser worker. The Portuguese and English model files are open and ship alongside the application. Pixels become text locally; there is no closed inference API.

The path is:

Receipt pixels → local neural OCR → conservative field suggestions → human review → approved expense → export with evidence.

The deterministic parts enforce the boundaries:

• A candidate amount needs a total label. Subtotals, discounts, tender and change are not substituted for the final total.
• Conflicting labeled totals leave the amount blank for review. Missing dates are not invented.
• Recognized foreign-currency markers block approval; this version handles BRL and does not convert currencies.
• Duplicate checks use the image's SHA-256 hash or the combination of merchant, date and amount. A match is a warning, not proof of fraud. Keeping both requires explicit acknowledgement and a reason.
• Amounts are integer cents. Pending and excluded records do not contribute to the sum.
• Editing an approved record invalidates its approval. It must be reviewed again before it counts.
• CSV export neutralizes spreadsheet formula prefixes; receipt text is escaped in the HTML packet.

What I actually checked

In real browser inference on clean synthetic images, the café's source total 34,90 and stationery shop's 124,90 were both read and parsed correctly. Recorded reading times were 0.76 s and 0.68 s on the test device, excluding worker/model startup. These are individual observations, not a performance or accuracy guarantee.

The duplicate failed approval until resolved. Two approved originals produced exactly 159,80. A one-cent edit removed the café's approval and reduced the approved total to 124,90; reviewing the original amount restored the total. The actual downloaded CSV had exactly two expense rows, and the HTML packet embedded both source images. Reloading restored the records.

I also prepared the offline cache, stopped the local app/model HTTP server, reloaded and ran the three images again. A new worker still recognized them. That verifies this cache path without a live app/model server; it is not an OS-wide airplane-mode test. Browser storage eviction can still require another download.

Eleven unit tests cover the calculation and validation boundaries. Those tests do not establish OCR accuracy. Real photographed receipts, glare, skew, faded thermal paper and handwriting need a separate evaluation before claims about production reliability. This version does not read PDFs, validate fiscal authenticity, connect banks or make tax decisions.

Why Does Open Innovation Matter?

Local recognition changes the operating model. Receipt images and text do not need to leave the device for inference, and each recognition does not incur a hosted-model fee. All runtime and model assets are served from the same origin; the first visit needs downloads, then the explicit offline preparation action can cache them.

This does not make the host anonymous: GitHub still receives normal page and asset requests. Nor does it make an export private after somebody shares it. The benefit is narrower and concrete: the app does not upload receipt contents to an inference service.

An open engine also leaves a replaceable boundary. A future model can improve recognition while the same review, duplicate and exact-cents rules remain. I chose OCR rather than a general-purpose text generator because the job is to read evidence, not invent a plausible expense.

The next validation would be consented real receipts from a freelancer, measuring field errors, correction effort and whether the exported packet fits their actual bookkeeping workflow. Until then, this is a working prototype with a business hypothesis and documented limits.

Prize Categories

Best Use of GitHub Copilot — GitHub Actions automation. The category description explicitly includes automating a project with GitHub Actions. My workflow installs locked dependencies, runs the unit tests and syntax checks, builds the self-hosted model/runtime bundle and publishes the resulting artifact to GitHub Pages. The deployment depends on the successful build job. I am claiming this Actions integration; I did not use Copilot itself.

I am also entering the overall category. No other partner technology is claimed.