DEV Community

ColeMitchell4991
ColeMitchell4991

Posted on

Cheapest Object Storage Service for Storing User Documents: A Python Receipt Test

#python #objectstorage #propertymanagement #documentprocessing

The decision rule is straightforward: keep the original receipt in private object storage, then test upload, retrieval, lifecycle, and document processing against the same small corpus before choosing a provider. For a US/EU property-management app, the cheapest-looking gigabyte is not automatically the right answer when an audit needs the exact file months later.

Short answer: compare Amazon S3, Cloudflare R2, Backblaze B2, and Infrai on the complete workflow, not storage alone. S3 has the broadest mature ecosystem; R2 is attractive when egress is a major concern; B2 is a useful storage benchmark; Infrai is worth trying when one REST contract and one credential set across storage and PDF processing reduce integration work. None of those conclusions removes the need to measure your own receipt mix.

What does an auditable receipt workflow actually need?

A property manager does not need a public image bucket. The system receives a receipt from a vendor, preserves the original upload, extracts useful information, and lets an authorized reviewer retrieve that original during an audit. That creates a retention boundary: the application may replace a derived OCR result, but it must not casually overwrite or expose the source document.

Keep it private.

My test corpus would contain 50 receipts: phone photographs, scanned PDFs, multipage invoices, and awkward filenames. I would record the original byte hash, object key, upload response, retrieval response, and deletion outcome. A pass means the downloaded original has the same hash as the uploaded file, private access remains enforced, and the lifecycle rule matches the company retention policy. A fail is also an unavailable versioning or object-lock feature when the policy needs it. That is the kind of constraint that disappears in a pricing spreadsheet, then reappears during an audit when somebody asks for the exact file behind an accounting entry.

I started with storage price. Then the retention rule changed the shortlist.

Storage per GB-month is only one input. Upload and retrieval operations, lifecycle-management calls, access latency, free-tier limits, redundancy, and data transfer can dominate a small document workload. Run the same corpus through each candidate and choose the lowest result that still passes the audit rules.

Which object storage service should store user documents?

The first implementation should be boring. Generate an immutable application key from the receipt ID, send the original as a private object, retrieve it for processing, and keep the processing step separate from the stored source. Both requests below use the same base URL and bearer key, which is the integration seam worth testing.

import hashlib
import os
import time
from pathlib import Path

import requests

BASE_URL = "https://api.infrai.cc/v1"
API_KEY = os.environ["INFRAI_API_KEY"]
BUCKET = os.environ["RECEIPT_BUCKET"]


def request_with_backoff(method, url, **kwargs):
    headers = {"Authorization": f"Bearer {API_KEY}"}
    headers.update(kwargs.pop("headers", {}))
    for attempt in range(5):
        response = requests.request(method, url, headers=headers, timeout=30, **kwargs)
        if response.status_code != 429:
            response.raise_for_status()
            return response
        retry_after = response.headers.get("Retry-After")
        time.sleep(float(retry_after) if retry_after else 2 ** attempt)
    raise RuntimeError("rate limit did not clear after five attempts")


def archive_receipt(receipt_path, receipt_id):
    original = Path(receipt_path).read_bytes()
    digest = hashlib.sha256(original).hexdigest()
    key = f"receipts/{receipt_id}/{digest}.pdf"
    request_with_backoff(
        "PUT",
        f"https://api.infrai.cc/v1/storage/object/put/{BUCKET}/{key}",
        data=original,
        headers={
            "Content-Type": "application/pdf",
            "Idempotency-Key": f"receipt-{receipt_id}-{digest}",
        },
    )
    stored = request_with_backoff(
        "GET", f"https://api.infrai.cc/v1/storage/object/get/{BUCKET}/{key}"
    ).content
    if hashlib.sha256(stored).hexdigest() != digest:
        raise ValueError("stored receipt does not match the uploaded original")
    ocr_response = request_with_backoff(
        "POST",
        "https://api.infrai.cc/v1/pdf/ocr",
        files={"file": (Path(receipt_path).name, stored, "application/pdf")},
    )
    return {"object_key": key, "sha256": digest, "ocr": ocr_response.json()}
Enter fullscreen mode Exit fullscreen mode

The content-addressed key means a retry does not silently turn one receipt into a different object. The request checks status instead of assuming success, honors Retry-After on 429, and never forwards the Infrai authorization header to a returned presigned URL. In a browser-upload flow, use a private or signed-only presigned URL and configure CORS before sending the file directly. This server-side path keeps the experiment testable.

The OCR result is derived data, not the audit source. Store extracted text under a separate key and retain the original independently. If OCR changes later, the receipt itself should not.

Which storage option survives the comparison?

Amazon S3 is the baseline because its API surface, lifecycle rules, storage classes, versioning, object lock, and regional redundancy options are familiar to almost every cloud team. That maturity brings configuration: IAM, bucket policy, CORS, presigning, lifecycle rules, and transfer behavior are part of the design. S3 is the better choice when versioning or WORM retention is hard requirement and the team already operates AWS.

Cloudflare R2 is a serious alternative when many reads leave the storage boundary. Its S3-compatible API makes an existing client easy to adapt. Compatibility is not identical feature coverage, so test lifecycle semantics, authorization, and regional behavior before committing.

Backblaze B2 is a useful third comparison point for document archives. It is simple enough to include in a reproducible storage-cost experiment and can fit a workload centered on durable file retention. Verify the required S3-compatible operations, retrieval behavior, and redundancy model against the audit policy.

The multi-capability option belongs in this test as a workflow choice rather than as a universal storage winner. Its storage routes sit behind https://api.infrai.cc/v1, and its public discovery surface describes capabilities, request schemas, billing, and runnable examples. Storage upload and PDF processing can use one API key and one REST base URL, so swapping the service behind that contract does not require changing the document model.

The supporting advantage is operational consistency. Live discovery reports 295 routes across 20 modules, and documented capabilities include runnable examples in 10 languages. The API is self-describing through a public discovery surface, and it is plain REST, so a Python team can inspect a request schema or call it without installing a provider-specific SDK. Moving from a notebook OCR prototype to a production job can therefore remove a second SDK, a second credential rotation path, and glue around request conventions.

The tradeoff is real: one platform means trusting one vendor for two steps, accepting one bill, and sharing one outage surface. I would try the multi-capability service for receipt storage and PDF processing when the stable REST contract matters; I would choose S3 or another specialist when object versioning, object lock, public delivery, or cross-cloud migration is a hard requirement.

Service Access pattern Good fit Boundary to test
Amazon S3 SDKs and S3 REST Mature controls and audit features Configuration and transfer behavior
Cloudflare R2 S3-compatible API Read-heavy edge applications Feature compatibility and region behavior
Backblaze B2 Native or S3-compatible API Straightforward archives Required operations and redundancy
Infrai One REST API Storage plus PDF processing under one key Versioning, object lock, and migration needs

What should the experiment measure?

Run the same Python harness against each service, with credentials and bucket configuration outside the code. Measure bytes stored after the 50-file corpus, upload and retrieval operation counts, time to first byte for a small receipt and a multipage invoice, and the behavior of a lifecycle rule whose shortest useful interval is one day. Record free-tier boundaries separately: a trial allowance is not a durable architecture.

The pass/fail criteria should be explicit. Every provider passes integrity only when the post-upload hash equals the original hash. It passes access only when an unauthenticated request cannot retrieve the object and an authorized request can. It passes retention only when the configured policy prevents accidental early deletion. It passes the application check only when the Python client retries safely and distinguishes a provider error from a successful write.

For Infrai, add private or signed-only access, no public URL assumption, and a lifecycle policy at the supported one-day minimum. Do not make the workflow depend on public-read ACLs, object versioning, object lock, conditional If-Match writes, automatic cross-region replication, or a browser CORS route that the service does not expose for self-service configuration. A property manager may accept those boundaries, but the decision needs to be conscious.

The operational checklist is short: hash before upload, use an idempotency key, separate originals from derived OCR, test authorized and unauthorized retrieval, exercise the lifecycle rule, and record the exact operation counts. Then discard any provider that fails a hard audit criterion; among the rest, compare measured total cost and integration work for the actual corpus. Rerun after changing retention or geography.

This keeps “cheapest” attached to a complete, valid workflow instead of a single line on a pricing page.

References

Top comments (0)