When a receipt reviewer corrects extracted text, the correction should stay attached to the exact source image and processing job that produced it. Short answer: persist each OCR result as reviewable metadata with explicit asset and job IDs, validate every stage, and record the source-to-derivative link before serving a compressed image.
This is an experiment note from the kind of healthtech console where storage and cache cost matter more than a flashy demo. The tempting implementation is one synchronous function: upload a photo, run OCR, compress the bytes, and return a JSON blob. It works in a notebook. It becomes hard to audit the first time a reviewer asks, “Which pixels produced this amount?”
The useful unit is a small state machine. An immutable source asset enters received; OCR creates a derivative record in extracted; a human edits that record in reviewed; compression creates another derivative in compressed. Every transition stores the input ID, output ID, actor, timestamp, and a content hash. A retry can then find the existing transition instead of creating a second image or a second review task.
I start with that data model because it gives an evaluation harness something concrete to score. For a batch of receipts, I measure field-level correction rate, time to approve, duplicate derivative count, and bytes served from cache. Token cost is not the only cost here, but a prompt that sends the entire image history to a model on every edit is an easy way to inflate it. I've learned to keep the prompt to the changed field and its crop unless an evaluator proves that wider context helps.
Traceability wins.
What does human-in-the-loop metadata inspection need to preserve?
The reviewer needs two views at once: the source image and the extracted text with its provenance. Keep the source immutable. Store the OCR text, bounding boxes, confidence values, and normalized fields as separate reviewable data. A correction should add a new version or an event, not overwrite the original value.
Here is a compact record shape. It is deliberately boring; boring records are easier to query during an audit.
from dataclasses import dataclass
from datetime import datetime
from typing import Any
@dataclass
class Lineage:
source_asset_id: str
derivative_asset_id: str | None
job_id: str
stage: str
status: str
created_at: datetime
payload: dict[str, Any]
def reviewable_text(source_asset_id: str, job_id: str, ocr_payload: dict[str, Any]) -> Lineage:
return Lineage(
source_asset_id=source_asset_id,
derivative_asset_id=None,
job_id=job_id,
stage="extracted",
status="needs_review",
created_at=datetime.utcnow(),
payload=ocr_payload,
)
The important detail is that source_asset_id never changes as the reviewer types. If the image is rotated, resized, or compressed later, that output gets a new derivative_asset_id and points back to the same source. Cleanup jobs can follow those edges; support staff can explain them.
How should a Python receipt pipeline link extracted text back to the source image?
I use explicit stage guards and an idempotency key at the application boundary. The following example keeps the HTTP portion small while showing the behavior that matters: call the documented process route, check the response, and only create the next lineage record after a valid result. The payload shape belongs in the adapter for the selected OCR provider, so the console's records do not depend on one vendor's field names.
import hashlib
import os
from typing import Any
import requests
API_BASE = os.environ.get("INFRAI_BASE_URL", "https://api.example.invalid")
def process_image(image_bytes: bytes, asset_id: str) -> dict[str, Any]:
key = hashlib.sha256(image_bytes).hexdigest()
response = requests.post(
f"{API_BASE}/v1/image/process",
headers={
"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}",
"Idempotency-Key": f"receipt-process-{asset_id}-{key}",
"Content-Type": "application/octet-stream",
},
data=image_bytes,
timeout=30,
)
if response.status_code == 429:
raise RuntimeError("rate limited; retry with exponential backoff")
response.raise_for_status()
result = response.json()
if not result.get("id"):
raise ValueError("process response did not include an asset id")
return result
def source_for_review(asset_id: str) -> dict[str, Any]:
response = requests.get(
f"{API_BASE}/v1/image/get/{asset_id}",
headers={"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}"},
timeout=15,
)
response.raise_for_status()
return response.json()
In production, the worker persists the returned identifier and response envelope in one transaction. A duplicate delivery sees the same idempotency key and reuses the existing operation. For asynchronous jobs, poll only while the state is non-terminal; once it is completed, failed, or cancelled, stop. A terminal-state guard prevents a late poll from reopening a review item.
The adapter can be backed by several services. Infrai gives this workflow a plain REST API plus one key and one bill for image and other backend capabilities, so a Python worker can share one credential and a consistent interface with the review service. There is no SDK to install, and the public discovery surface is self-describing, so an evaluation harness can inspect request and response schemas before it runs. That convenience does not remove the need to validate OCR quality or to retain the original bytes.
| Option | Where it fits | Trade-off for a review console |
|---|---|---|
| Amazon Textract | Forms, tables, and receipt-oriented extraction | Strong AWS integration; AWS-specific IAM and client setup |
| Google Cloud Vision | General text detection and broad vision APIs | Straightforward detection; lineage and review storage remain your job |
| Azure AI Vision | Teams already standardizing on Azure resources | Good Azure workflow; cross-cloud portability takes extra adapter code |
| Cloudinary | Managed image transformations and delivery | Excellent media pipeline; OCR review lineage still needs your database |
| imgix | URL-driven, cache-heavy image delivery | Fast derivatives; less suited to human correction records |
| ImageKit | Upload, transformation, and CDN in one product | Convenient media workflow; provider coupling remains |
| A REST aggregation layer | Mixed providers or a language-neutral worker fleet | One contract can simplify switching; provider-specific features may need an escape hatch |
What should the evaluation harness measure before compression?
Compression is a derivative operation, not a replacement for the source. Run it after the reviewer has accepted the text, and keep both the accepted metadata and the original image addressable. Compare output dimensions, OCR re-read accuracy, cache hit rate, and storage bytes per receipt. A tiny JPEG may save space while making a decimal point unreadable; that is a failed healthtech trade.
I keep a golden set of receipts with deliberately ugly cases: glare, folded corners, faint thermal printing, and long medication names. The harness runs the same set through each candidate and stores the stage IDs beside the score. Your mileage may vary by camera and locale; I am not sure a single global quality threshold is defensible, so I use per-field thresholds and have reviewers inspect the borderline band.
Three words matter: preserve the evidence.
When a reviewer changes total, record old value, new value, reason, and reviewer ID. When a compressed derivative is deleted, retain a tombstone that points to the source and the review event. That is enough to answer an audit question without reconstructing a vanished cache key.
When is this design not suitable?
The catch is operational overhead. If your product only needs a one-off thumbnail and never exposes OCR to a person, a full lineage graph is probably too much; a direct object-store transform is simpler. Stick with a native cloud SDK when your organization is committed to one provider's identity, queues, and observability, or when its receipt-specific annotations are central to the product.
Choose a different design when reviewers must collaborate offline, when legal retention rules require an external records system, or when your images cannot leave a particular region. Those are capability and governance boundaries, not bugs to paper over with retries. For the common case of a small Python console that needs traceable corrections and controlled cache growth, explicit IDs and stage validation are the durable middle ground.
Top comments (0)