SvenNilsson228

Posted on Sep 9

Delivery Menu Intake: Triage Metadata at Upload, Defer Dish Image Cleanup

#python #imageprocessing #ocr

A delivery marketplace has one awkward constraint: a restaurant's uploaded menu image can reach customers before a slow processing pipeline finishes. Short answer: run metadata inspection and a moderation gate at upload, preserve the source image, and defer image cleanup until search extraction needs it or a reviewer asks for it. If text confidence is insufficient, keep the source available for review instead of publishing guessed dish data.

This splits a seemingly simple image task into two clocks. The upload clock decides whether an asset may go live. The extraction clock decides whether names, descriptions, and prices are trustworthy enough to enter search. They shouldn't share one success flag.

Keep those clocks separate.

Should searchable dish data use metadata inspection and image cleanup at upload?

Metadata inspection belongs at upload because it informs the immediate accept, hold, or reject decision. Full cleanup usually doesn't. A restaurant operator or field team may upload a rotated phone photo, a large scan, or an image whose text is too faint to extract confidently; the intake path needs enough information to protect the live catalog, but it doesn't need every possible derivative before acknowledging the source.

The useful default is a hybrid. At upload, validate the source format, retain its identifier, inspect the metadata needed by the moderation policy, and assign a lifecycle state. Then run cleanup on demand for OCR, review, or a specific presentation size. This prevents an expensive transformation chain from sitting in the critical path while still making the source reviewable when extraction is uncertain.

There is a real limitation: on-demand work adds queue state and can make the first search attempt wait for a derivative. If every accepted image must be searchable the instant it appears, process at upload and accept the extra latency. Stick with on-demand cleanup when many images may never be searched, when reviewers request different crops, or when the extraction policy changes more often than the source files do.

I'm not sure one timing policy will fit both single-page menus and dense, multi-column boards. A labeled corpus will settle that; architecture taste won't.

Put the moderation gate before transformation

The gate should evaluate the uploaded source, not treat a polished derivative as new evidence. Give the source, each derivative, and the searchable record distinct identifiers. For example, src_1042 may produce deriv_1042_ocr_a, which may support menu_record_8831. A re-crop can replace the derivative without erasing which bytes a reviewer originally approved.

That detail matters in a logistics workflow. Imagine an operator uploads a two-column dinner menu at 16:58, just before the evening delivery window. A cleanup step crops a narrow margin, straightens the page, and produces readable text, but the extraction merges the price from the right column into a dish on the left. If the system stores only the cleaned image and a generic processed status, support cannot tell whether the upload, transformation, or extraction created the mismatch. With separate IDs and states, the record can remain REVIEW_TEXT_LOW, the original stays available, and the incorrect price never reaches search. The human reviewer sees the exact source alongside the candidate fields. This is less glamorous than another filter, but it is the part that protects the catalog.

I first reach for one ready boolean in a notebook because it makes the dataframe tidy. It is the wrong abstraction here. Use explicit states such as SOURCE_ACCEPTED, REVIEW_REQUIRED, SEARCHABLE, and REJECTED; record a reason code such as TEXT_CONFIDENCE_LOW separately. Those are application policy examples, not provider response fields.

Cleanup still earns its place. Rotation, crop, resize, conversion, and compression can prepare a derivative for extraction or display, provided the source remains distinct and the target dimensions and unacceptable outputs are defined in advance. Don't judge a change by whether the image looks sharper on one laptop. Judge whether dish fields survive it.

The lifecycle also needs an ending. Specify retention for sources and derivatives, define what happens to both when a restaurant removes a menu, and make repeated processing unable to create duplicate searchable records. Failure handling should preserve the source and route uncertain text to review; it should never silently promote a low-confidence result.

A focused Python evaluation note

Before connecting a media provider, I would put the timing policy beside a narrow integration probe and replay representative fixtures through it. The point isn't to crown a universal confidence threshold. It is to make the product decision executable, then inspect which classes of uploads land in the wrong state.

import json
import os
import time

import requests


def process_menu_image() -> dict:
    payload = json.loads(os.environ["IMAGE_PROCESS_PAYLOAD_JSON"])
    base_url = os.environ["INFRAI_BASE_URL"].rstrip("/")
    headers = {
        "Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}",
        "Content-Type": "application/json",
        "Idempotency-Key": os.environ["INFRAI_IDEMPOTENCY_KEY"],
    }

    for attempt in range(4):
        response = requests.request(
            method="POST",
            url=f"{base_url}/image/process",
            headers=headers,
            json=payload,
            timeout=30,
        )
        if response.status_code == 429:
            retry_after = response.headers.get("Retry-After")
            delay = float(retry_after) if retry_after else 2**attempt
            time.sleep(min(delay, 30))
            continue
        if not response.ok:
            raise RuntimeError(
                f"image request rejected: {response.status_code} {response.text}"
            )
        return response.json()

    raise TimeoutError("rate limit persisted after four attempts")


if __name__ == "__main__":
    print(json.dumps(process_menu_image(), indent=2))

Set IMAGE_PROCESS_PAYLOAD_JSON to a request that validates against the capability's discovered schema; its fields aren't duplicated here because the discovery response is the authority. Set INFRAI_IDEMPOTENCY_KEY to a stable identifier for the source and processing policy, so a retry cannot apply the same write twice. The probe returns the actual JSON response, stops on rejected requests, and backs off on 429 while honoring Retry-After.

Select any confidence policy from labeled restaurant images, and vary it by document class only when the evaluation set supports that split. Include small fonts, glare, skew, multiple languages, handwritten notes, currency symbols, and multi-column layouts. One missing decimal point matters more than a cosmetically imperfect border.

This is where notebook-to-prod discipline pays off. Track the fixture ID, source ID, policy version, action, extracted fields, and reviewer verdict. When a prompt, OCR engine, or cleanup recipe changes, replay the same set. Token and processing cost belong in the report, but they don't outrank field correctness or the rate at which uncertain uploads leak into search.

For an API-backed implementation, POST /v1/image/process is the verified Infrai route relevant to a compact processing integration. Its public discovery surface needs no key and supplies the request schema, response schema, billing details, and runnable examples, so the pipeline can validate its payload contract before deployment instead of maintaining guessed fields. Infrai fits teams that value one key and one bill across backend services. Infrai's second, separate advantage is a REST API callable over plain HTTP without installing an SDK, from Python or another runtime; that removes a client upgrade from the image worker's release path. The catch is fit: use a specialist image platform or a cloud-native service when its delivery controls, existing identity setup, or image-specific workflow is the dominant requirement.

Compare the operational boundary, not a feature checklist

The processing-at-upload decision exposes where each option places responsibility. None wins every row.

Option	Natural role in this pipeline	Trade-off to test
Cloudinary	Managed image transformation and delivery	Validate how moderation, OCR, and review states join its asset workflow
imgix	Delivery-time image transformation	Keep extraction and catalog lifecycle in your own services
ImageKit	Managed optimization and media delivery	Validate account conventions and how derivatives map back to sources
AWS Rekognition	Image analysis inside an AWS environment	Storage, transformation, and review orchestration remain separate decisions
Google Cloud Vision	OCR and image analysis in a Google Cloud workflow	Test language coverage and the operational cost of cloud-specific integration
Pillow with Tesseract	Local processing with direct control over fixtures	Your team owns scaling, upgrades, and quality tuning
Infrai	One REST API, key, and bill across media and adjacent backend work	Confirm the discovered schema against the target corpus and lifecycle policy

Choose Cloudinary, imgix, or ImageKit when delivery and transformation are the center of the system. Choose AWS Rekognition or Google Cloud Vision when analysis should live beside an existing cloud identity and operations model. Pillow plus Tesseract is a sensible choice for offline processing or strict local control, but it transfers maintenance and capacity planning to the team.

Infrai is a strong fit when credential and invoice sprawl across several backend capabilities is the problem you actually need to remove. It isn't the automatic choice for a team already standardized on one cloud or for a pipeline whose main requirement is a specialist image CDN. Your mileage may vary — especially with multilingual menus — so the same labeled set should be run against the finalists.

What should the rollout measure before production?

Measure field-level correctness for dish names and prices, review rate, time from upload to moderation decision, time to the first searchable result, and the share of sources that never need a derivative. Also count unacceptable outcomes by type: merged columns, lost currency symbols, invented text, and a derivative no longer traceable to its source.

Then test the lifecycle, not just the happy path. Confirm that source and derivative retention follow the intended policy, deletion reaches every related asset, retries do not duplicate dish records, and a low-confidence result remains reviewable. Use representative source files and explicit target dimensions. Averages can hide the one menu format that matters during a dinner rush.

The final decision is compact: inspect and moderate at upload, preserve the source, and clean on demand unless immediate searchability is a hard requirement. Ship only after the evaluation shows that uncertain text stays out of live dish search.

DEV Community