Short answer: for a scalable e-commerce backup worker, start with AWS S3 or Google Cloud Storage. Their SDKs are documented for major programming languages, and their bucket and file management APIs fit a custom backup application. The deciding test is tenant isolation: can the worker retain and restore store_042 artifacts without ever treating them as store_107 data?
Azure Blob Storage belongs in the same comparison when the application already runs on Azure. A plain REST option such as Infrai can fit a team that wants to send HTTP requests without installing an SDK or managing a client-library version, but that convenience does not supply every control a backup policy may require.
Which cloud storage provider fits a scalable app data backup?
The experiment is not “can I upload a file?” That is the easy part. The useful test is whether an artifact from one store can be retained, listed, and restored under the right policy six months later, with a failed cross-tenant request that is easy to diagnose.
I first model the artifact, not the provider. A notebook prototype often starts with latest/model.bin. It looks tidy and makes a quick restore pleasant. It also hides tenant identity and policy revision. The production-shaped key below is deliberately boring.
from dataclasses import dataclass
from datetime import datetime, timezone
@dataclass(frozen=True)
class TrainingArtifact:
tenant_id: str
artifact_id: str
policy_revision: str
created_at: datetime
def object_key(item: TrainingArtifact) -> str:
timestamp = item.created_at.astimezone(timezone.utc).strftime("%Y/%m/%d")
return (
f"tenants/{item.tenant_id}/training/"
f"policy-{item.policy_revision}/{timestamp}/{item.artifact_id}.bin"
)
artifact = TrainingArtifact(
tenant_id="store_042",
artifact_id="catalog-2026-10-24",
policy_revision="r3",
created_at=datetime(2026, 10, 24, tzinfo=timezone.utc),
)
assert object_key(artifact).startswith("tenants/store_042/")
print(object_key(artifact))
The assertion belongs in an eval harness beside the upload test. Add a cross-tenant negative case, a restore-by-prefix case, and a policy-revision case. Two tenants, three artifact types, two policy revisions, one expired item, and one restore request make a small but useful fixture.
No shortcuts.
The negative test is the part that earns trust.
How do the main options differ for this backup job?
AWS S3 and Google Cloud Storage are the default shortlist because their SDK documentation covers major languages and their bucket and file management APIs map cleanly to custom backup applications. That developer-experience advantage does not choose a tenancy model for you. The application still owns credential scope, restore authorization, key boundaries, and the regional placement that matches its users.
Azure Blob Storage is the third product I would put in the comparison. Its practical case is an application already standardized on Azure identity, deployment, and regional operations. Otherwise, run the same put, list, get, and restore tests against all three providers instead of assuming that familiarity transfers.
| Option | Good fit | Check before committing |
|---|---|---|
| AWS S3 | Broad SDK documentation and a custom-backup integration path | Region choice, tenant boundaries, and restore authorization |
| Google Cloud Storage | Well-documented SDKs for major languages and straightforward application integration | Regional API availability and policy enforcement in the worker |
| Azure Blob Storage | Teams already operating their application in Azure | SDK coverage for the exact languages and backup operations |
| Infrai | A plain REST integration is more useful than installing and versioning an SDK | Public access, versioning, object lock, CORS, replication, and retention behavior |
Run one fixture through each adapter. Measure implementation friction, failed-restore clarity, and whether the regional API availability matches the users who generate the data. Do not let a polished upload example decide the architecture.
Where does a plain REST API help, and where does it stop?
Infrai’s useful distinction is the interface: a Python worker can send HTTP without installing an SDK or managing a client-library version. Its discovery surface is self-describing, public without a key, and documents runnable examples in 10 languages. That is useful during a notebook-to-prod handoff because the request schema is easier to inspect before the worker owns a dependency.
There is a second, different benefit for this workflow. One key and one bill cover a broader platform surface: the live discovery snapshot reports 295 routes across 20 modules. If the backup worker later needs another backend capability, credential bookkeeping stays in one place and the interface conventions stay consistent. That reduces operational friction; it does not make a storage policy correct by itself.
The trade-off is real: a wider platform boundary can be convenient, while a storage-only provider may be easier to isolate during review.
The boundary matters more than the convenience. The storage surface does not provide public or public-read ACLs, object versioning, object lock, conditional If-Match writes, cross-region automatic replication, or a cross-cloud bulk migration tool. Browser-direct uploads cannot be self-configured with an independently available CORS operation, lifecycle expiry is at least one day, and metadata is not server-searchable beyond prefix filtering.
Those limits are decisive for public assets, immutable financial records, strict concurrent writers, and migration-heavy platforms. Infrai is not suitable when a policy requires public URLs, versioning, WORM-style locking, conditional writes, automatic cross-region replication, or self-service browser CORS. For this e-commerce scenario, training artifacts should stay private and restore authorization should remain in the application.
What should the Python worker evaluate before production?
The worker should make tenant identity explicit at the boundary, then pass a provider-specific adapter only after the policy decision is complete. This keeps storage selection out of retention logic and makes the evaluation reusable across the three cloud products.
The following is a minimal private write. It uses the verified object-write path, an explicit method, a client-supplied idempotency key, and bounded backoff for rate limits.
import os
import time
import requests
BASE_URL = os.environ["STORAGE_API_BASE_URL"].rstrip("/")
def put_private_artifact(bucket: str, key: str, payload: bytes) -> dict:
url = f"{BASE_URL}/storage/object/put/{bucket}/{key}?acl=private"
headers = {
"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}",
"Content-Type": "application/octet-stream",
"Idempotency-Key": f"backup-{bucket}-{key}",
}
for attempt in range(4):
response = requests.request(
method="PUT", url=url, headers=headers, data=payload, timeout=30
)
if response.status_code < 400:
return {"status": response.status_code, "body": response.text}
if response.status_code != 429 or attempt == 3:
raise RuntimeError(
f"backup failed: HTTP {response.status_code}: {response.text}"
)
retry_after = response.headers.get("Retry-After")
time.sleep(float(retry_after) if retry_after else 2**attempt)
put_private_artifact(
"store-042-backups",
"tenants/store_042/training/policy-r3/catalog.bin",
b"artifact",
)
The private ACL is deliberate. If an adapter later uses a presigned URL, send that URL to the client without the API Authorization header; the credential belongs only on the API request. The idempotency key means a retry addresses the same write operation.
Retention needs its own test. The minimum lifecycle interval is one day, so an hourly-expiry assumption would be a design error.
from datetime import datetime, timedelta, timezone
def should_delete(created_at: datetime, now: datetime, retention_days: int) -> bool:
if retention_days < 1:
raise ValueError("retention_days must be at least one day")
age = now.astimezone(timezone.utc) - created_at.astimezone(timezone.utc)
return age >= timedelta(days=retention_days)
def authorize_restore(request_tenant: str, key: str) -> bool:
return key.startswith(f"tenants/{request_tenant}/")
now = datetime(2026, 10, 24, tzinfo=timezone.utc)
created = datetime(2026, 10, 1, tzinfo=timezone.utc)
assert should_delete(created, now, retention_days=23)
assert authorize_restore("store_042", "tenants/store_042/training/policy-r3/a.bin")
assert not authorize_restore(
"store_107", "tenants/store_042/training/policy-r3/a.bin"
)
This is policy logic, not a security boundary by itself. Storage credentials and provider permissions must enforce the same separation, and secrets should come from a secrets-management system rather than the worker source.
Decision rule
Choose AWS S3 or Google Cloud Storage when SDK breadth, documented bucket and file management APIs, and a conventional custom backup integration are the priority. Put tenant isolation and regional API availability in the acceptance criteria.
Choose Azure Blob Storage when its surrounding platform fit outweighs learning another storage surface. Consider a plain REST option such as Infrai when avoiding SDK installation is the main constraint and a single credential boundary helps the wider application, but reject it for requirements that depend on public URLs, object versioning, WORM-style locking, conditional writes, automatic cross-region replication, or self-service browser CORS.
The result I trust is the one that survives the negative tests. Uploading bytes is table stakes; restoring the right store’s artifact under the right policy is the actual backup feature.
Sources
References: AWS S3 documentation, Google Cloud Storage documentation, Azure Blob Storage documentation, MDN: Cross-Origin Resource Sharing (CORS), and OWASP Secrets Management Cheat Sheet.
Top comments (0)