I Built an AI Receipt Scanner That Tracks Your Spending (FastAPI + Amazon Textract)

#python #aws #ai #sideprojects

The Problem

I kept stuffing receipts in my pocket and forgetting about them. At the end of the month: "Where did all my money go?"

Spreadsheets are tedious. Expense apps want you to type everything manually. I wanted something simpler — snap a photo, done.

What I Built

SnapReceipt — take a photo of any receipt, AI reads it automatically, and tracks your spending.

How it works:

1. Upload a receipt photo

Any receipt from any store, any country. JPG or PNG.

2. AI extracts everything

{
  "store_name": "Sainsbury's",
  "date": "2026-04-18",
  "total": 11.80,
  "currency": "GBP",
  "category": "Groceries",
  "items": [
    {"name": "JS 2 WHITE BAGUETTES", "quantity": 1, "price": 1.50},
    {"name": "JW TUNA OIL 4X 125G", "quantity": 1, "price": 4.25},
    {"name": "JS FAIRTRD BANANA LS", "quantity": 1.215, "price": 1.15}
  ]
}

3. Auto-categorization

The app detects the store name and categorizes automatically:

🛒 Groceries (Tesco, Sainsbury's, Aldi, Walmart...)
🍕 Restaurant (McDonald's, Starbucks, Nando's...)
🚗 Transport (Shell, BP, Uber, airlines...)
🛍️ Shopping (Amazon, Argos, Primark...)
💊 Health (Boots, pharmacies, doctors...)

4. Currency detection

Reads £, €, $ symbols from the receipt. Also detects currency from the store name — Sainsbury's → GBP, Carrefour → EUR.

5. Spending analytics

Monthly summaries, category breakdowns, 6-month trends — all in the dashboard.

The Biggest Challenge: File Size

API Gateway has a ~5MB payload limit. Phone photos of receipts can be 4-8MB. My first approach (direct upload) failed for larger images.

The fix: Presigned S3 upload URLs.

Step 1: POST /receipts/upload-url → get presigned S3 URL
Step 2: PUT image directly to S3 (bypasses API Gateway)
Step 3: POST /receipts/process → Textract reads from S3

This works for any file size up to 10MB. It's the standard pattern for serverless file uploads — and it's what I'd recommend to anyone building file upload features on Lambda.

Tech Stack

Component	Service	Cost
API	FastAPI on Lambda	$0 (free tier)
Database	DynamoDB	$0 (free tier)
Image storage	S3	$0 (free tier)
Receipt OCR	Amazon Textract	~$0.01 per scan
API Gateway	HTTP API	$0 (free tier)

Total cost at 1000 scans/month: ~$10/month

What Amazon Textract Gets Right (and Wrong)

Gets right:

Store names — almost always correct
Line items — reads item names and prices accurately
Totals — usually finds the right total
Date — handles most formats (DD/MM/YYYY, MM/DD/YYYY, etc.)

Gets wrong sometimes:

Picks up phone numbers or reference numbers as the "total"
Crumpled or faded receipts reduce accuracy
Some handwritten receipts don't work well

My fix for wrong totals: When Textract returns multiple candidate totals, I pick the one closest to the sum of line items. This eliminates phone numbers and reference numbers that are far from the actual receipt total.

def _pick_best_total(candidate_totals, items):
    items_sum = sum(i["price"] for i in items)
    if items_sum > 0:
        best = min(candidate_totals, key=lambda t: abs(t[1] - items_sum))
        return best[1]
    return candidate_totals[0][1]

Revenue Model

Free: 20 scans/month (enough for casual use)
Pro: £5/month for unlimited scans

The scan counter resets monthly. When users hit the limit, they see an upgrade prompt. The free tier is generous enough to be useful but limited enough that regular users convert.