Automating Receipt Tracking with Python OCR

I do most of my grocery shopping at Lidl, and like a lot of people I have a vague, unreliable mental model of what things actually cost. Was that block of cheese €2.49 last week or €3.29? Is the "special" actually special? The receipt has all of that information on it and it just evaporates the moment the paper hits the bin.

So I built a small app to capture it. Lidl Receipt Manager reads my Lidl (Ireland) receipts, builds a searchable price history of every product I've ever bought, and turns that history into tick-off shopping lists with a live running total. It runs on the desktop as a PyQt5 app and on my phone as a native Android .apk.

This post walks through what it does, how it's designed, and the part that ate the most of my time. As well as what it actually takes to package a Python app into an Android build.

What it does:

The feature set is deliberately small and focused on the one job: turning receipts into useful data.

• Import receipts three ways. Photograph the receipt and let on-device OCR read it, pick an existing image file, or just paste the receipt text in. The paste path always works, which matters more than you'd think (more on that later).
• A product price database. Every line item from every receipt is stored. Those rows get rolled up per product into an average, minimum, and maximum unit price, a count of how many times I've bought it, and when I last did.
• Search as you type. Filter the product list instantly.
• Shopping lists. Add a product and it comes pre-filled with its latest known price. Add manual items too. Tick things off as you walk the aisles and watch a live "remaining" total count down.
• Duplicate-safe imports. Receipts are de-duplicated by their transaction ID, so accidentally importing the same receipt twice won't double-count anything.
• Fully local and offline. Everything lives in a local SQLite database. Nothing is uploaded anywhere, not the photos, not the prices, nothing. Even the OCR runs on-device.

The design philosophy: a shared core, two skins

The most important design decision in the whole project is this: the desktop app and the Android app share the exact same brain. The parsing logic and the database schema are identical across both. Only the UI layer differs.

The repository reflects that split cleanly:

.
├── lidlReceiptManager.py     # Desktop app (PyQt5) a single file
└── android/                  # Android port (KivyMD)
    ├── main.py               # Touch UI (KivyMD bottom-nav)
    ├── core/
    │   ├── parser.py         # parse_receipt() is shared with desktop
    │   ├── db.py             # SQLite layer (same schema)
    │   └── ocr.py            # ML Kit on Android, pytesseract on desktop
    └── buildozer.spec        # APK build config

The desktop app is a single self-contained file. When I ported it to Android, I pulled the three things that have nothing to do with the UI, the parser, the database, and OCR, out into a small core/ package, and rebuilt only the presentation layer on top of them.

That gives a layered architecture where the dependency arrows all point inward:

   PyQt5 UI                 KivyMD UI
 (desktop main)          (android/main.py)
        \                     /
         \                   /
          v                 v
        core: parser · db · ocr
                   |
                   v
              SQLite file

Both UIs are "dumb" as they collect input, call into core, and render whatever comes back. The parser is pure standard library with no UI dependencies at all.

The parser

The parser is the heart of the project, and it's just careful regular expressions over the receipt's plain text. Lidl Ireland receipts have a predictable shape: a store header, a transaction ID (TRN-ID:), a date, a block of line items, and a TOTAL. Each line item is a name followed by a price and a single-letter VAT
class.

parse_receipt() does two passes. The first sweeps the whole receipt for the metadata such as store, transaction ID, date. The second walks line by line building up items: a price line starts a new item, a following 2 x 1.49 line attaches a quantity to it, and a trailing -0.50 applies a discount. Deposit lines and visual noise (rows of dashes, stray EUR/Copy tokens) are filtered out. At the end it computes a net unit price per item and returns a tidy dictionary:

return {'store': store, 'date': rdate, 'trn_id': trn_id,
        'total': total, 'items': products}

Because it's pure text in, pure dict out, it's trivial to reason about and it behaves identically no matter which app calls it. The Android copy is lifted verbatim from the desktop original, that's a deliberate choice so parsing behaviour can never silently drift between the two platforms.

The database

The data layer is a thin wrapper around SQLite with four tables: receipts, purchases (the individual line items, with a cascading foreign key back to the receipt), lists, and list_items. There's no ORM and no migrations machinery just CREATE TABLE IF NOT EXISTS and a handful of hand-written queries.

The price-history "rollup" that powers the product view isn't a stored table at all; it's a single GROUP BY query that aggregates on demand:

SELECT name,
       ROUND(AVG(unit_price),2) AS avg_price,
       ROUND(MIN(unit_price),2) AS min_price,
       ROUND(MAX(unit_price),2) AS max_price,
       COUNT(*) AS times,
       MAX(rdate) AS last_seen
FROM purchases p JOIN receipts r ON p.receipt_id = r.id
GROUP BY name ORDER BY name COLLATE NOCASE

Duplicate protection is enforced at the schema level as trn_id is UNIQUE and checked in code before insert, so re-importing a receipt returns a 'duplicate' status instead of doubling your data.

The one real change between platforms is where the database lives. On desktop it defaults to ~/.lidl_receipts/receipts.db, exactly as the original did. But on Android, ~ isn't writable, so the data directory is injectable: the Android app passes in its private user_data_dir and everything else stays the same.

class DB:
    def __init__(self, folder=None):
        folder = folder or default_data_dir()

There's also a small Android-specific concession, check_same_thread=False, on the connection because Kivy may touch the database from clock callbacks on a different thread than it was created on.

Pluggable OCR

OCR is the one piece of "core" that genuinely must differ by platform, so it's designed as a pluggable backend with a two-function public API: ocr_available() and image_to_text(path). The module sniffs the environment once at import time:

ON_ANDROID = 'ANDROID_ARGUMENT' in os.environ or hasattr(sys, 'getandroidapilevel')

• On the desktop, it uses Tesseract via pytesseract, even auto-detecting the Tesseract executable in the usual Windows install locations so you don't have to fiddle with PATH.
• On Android, it reaches into Google ML Kit's on-device text recognizer through pyjnius, which lets Python call Java/Android APIs directly. The text recognition model is bundled into the APK, so it runs fully offline with no Google account and no network round-trip.

That ML Kit call is a nice little window into how Python-on-Android actually works. You grab Java classes by name and call them as if they were Python objects:

from jnius import autoclass
InputImage = autoclass('com.google.mlkit.vision.common.InputImage')
TextRecognition = autoclass('com.google.mlkit.vision.text.TextRecognition')
...
task_await = getattr(Tasks, 'await')   # 'await' is a Python keyword!
result = task_await(recognizer.process(image))

There are real-world wrinkles baked in here: the gallery picker hands back a content:// URI while the camera returns a plain file path, so the code builds the right kind of Uri for each. And await is a reserved word in Python, so the Java Tasks.await() method has to be reached via getattr. Little things, but exactly the kind of friction you hit when bridging two runtimes.

The two UIs

The desktop app (PyQt5) is a classic two-tab desktop layout: a Products tab with an import toolbar, a search box, and a sortable table of every product with its price stats; and a Shopping Lists tab with a split view of lists on the left and their items on the right. It's styled with a bit of custom Qt stylesheet to look less like a 2005 application.

The Android app (KivyMD) rebuilds the same two screens for touch using a Material Design bottom navigation bar into Products and Lists. The receipt import buttons become big tappable "Photo / Image / Paste" buttons, list items become rows with a checkbox and a delete button, and the whole thing is laid out in KivyMD's declarative KV language right inside main.py.

The one genuinely tricky bit of the mobile UI is threading. OCR can take a second or two, and you must never block the UI thread, so image import spins the recognition work onto a background thread and then marshals the result back to the main thread to update the screen:

def worker():
    text = ocr.image_to_text(path)
    self._parsed_main(parse_receipt(text))   # @mainthread-decorated

threading.Thread(target=worker, daemon=True).start()

KivyMD's @mainthread decorator makes that hop back onto the UI thread clean.

Building the Android APK: the part nobody warns you about

Here's where things get real. Writing a Kivy app is the easy 20%. Packaging it into an .apk that installs and runs on a phone is the other 80%, and it's full of sharp edges. Here's the path I landed on after a fair amount of trial and error.

Test on the PC first

The single biggest time-saver was realising I didn't need an Android build to iterate. KivyMD runs on the desktop, so I could develop the entire UI and data layer with a plain python main.py and a fast feedback loop. Only camera capture is phone-only; everything else behaves identically.

But getting the desktop preview running surfaced two Windows gotchas worth knowing:

1. Python version matters. Kivy 2.3.0 has no Windows wheels for Python 3.13/3.14, so you have to pin to Python 3.11 or 3.12. (uv python install 3.12 makes this painless.)
2. KivyMD 1.1.1 is broken out of the box on modern setuptools. It's distributed only as an sdist, and modern setuptools silently drops all of its non-Python data files when building it, the .kv layouts, the GLSL shaders, the fonts. The result is cryptic FileNotFoundError: ... label.kv at import or ... header.frag at first render. I wrote a little helper, fix_kivymd_kv.py, that downloads the official sdist and copies every missing data file back into the installed package. One command and the preview runs.

The actual build: Buildozer on WSL2

The packaging tool is Buildozer, which orchestrates python-for-android (p4a) to cross-compile CPython, your code, and all the native dependencies into an APK. The catch: Buildozer only runs on Linux/macOS. On my Windows 10 machine, that means WSL2 (Ubuntu).

The one-time setup is a hefty apt install (JDK 17, the Android build toolchain, autoconf, cmake, libffi, libssl, and so on) plus a virtualenv with Buildozer and a pinned Cython. Then the build itself is one command:

buildozer -v android debug

The first run is a commitment: it downloads the entire Android SDK and NDK and compiles everything from scratch so about 20 to 40 minutes. Subsequent builds are minutes. One important tip: build inside the WSL filesystem (~), not on /mnt/c/..., because building across the Windows/Linux filesystem boundary is slow and hits path issues.

The buildozer.spec is where the real decisions live

The build is configured by buildozer.spec, and almost every line in mine exists because something broke without it. The highlights:

• Requirements: python3,kivy==2.3.0,kivymd==1.1.1,pyjnius,plyer,android. pyjnius is what lets me call ML Kit; plyer provides the camera and file picker; android provides the permissions API. Notably, Pillow is deliberately omitted as it's only used by the desktop OCR fallback, and on Android OCR is ML Kit, so PIL would just add a fragile native build for no benefit.
• ML Kit as a Gradle dependency. This is the elegant bit:

  android.gradle_dependencies = com.google.mlkit:text-recognition:16.0.0

That one line pulls Google's on-device text recognition into the APK at build time and bundles the Latin model so it works offline.

• Pinned versions everywhere. This is the hardest-won lesson. p4a is pinned to the v2024.01.21 release, which builds Python 3.11 because the p4a master default targets Python 3.14, and Kivy 2.3.0 does not compile against 3.14 (private CPython C-API functions it relies on, like _PyLong_AsByteArray, changed or were removed). The NDK is pinned to 25b to match. KivyMD is pinned to 1.1.1 in both the build spec and the desktop requirements, because KivyMD's API churns between releases. In this corner of the ecosystem, "just use the latest version" is how you lose a weekend.
• Permissions and targets: CAMERA plus storage read/write, target API 34, minimum API 24, building for both arm64-v8a and armeabi-v7a, and accept_sdk_license = True so the first build doesn't block on an interactive license prompt.

The APK lands in bin/, gets copied back across to Windows, and installs either over USB with adb or just by tapping the file on the phone. On first launch it asks for camera and storage permissions, and from then on it's a self-contained, offline little app.

Lessons learned

A few things I'm taking away from this project:

• Separate the brain from the skin early. Pulling the parser, DB, and OCR into a UI-agnostic core meant the entire Android port was "write a new UI" rather than "rewrite the app." The shared core is the reason two apps can stay in lockstep.
• A pluggable seam is worth it where platforms genuinely differ. OCR was the one thing that had to change per platform, and giving it a tiny two-function interface kept that difference from leaking everywhere else.
• Always leave an escape hatch. OCR is magic when it works and useless on a crumpled receipt in bad light. The "paste the text" import path costs almost nothing and means the app is never completely stuck.
• Mobile packaging is a versioning minefield. The code was the easy part. The real engineering was discovering the exact combination of Python, Kivy, KivyMD, p4a, and NDK versions that actually build together and then pinning every one of them so it stays buildable.

The end result is exactly what I wanted: I photograph my receipt on the way out of the shop, and over time I've built up a private, offline, searchable history of what everything actually costs and a shopping list that knows the price of
things before I get to the till.