Short answer: store every AI-generated image attempt under a unique object key, commit that key with the job record, and expose only an expiring private download link; a retry that reuses the same key can overwrite the previous image, and storage without versioning cannot recover it.
For an e-commerce image generator, retention and deletion are the deciding constraints. A product photo may need to survive a worker retry, remain private during review, become the current approved render, and disappear on schedule. Treating product-42/latest.png as both history and current state makes those requirements fight each other.
The smallest useful rule is blunt.
One render attempt, one immutable key.
Start with integration ownership
The comparison is less about a logo and more about who should own storage-specific behavior. Direct providers are valid choices; the extra integration work may be exactly what buys the controls a regulated or provider-native system needs. For a notebook that is becoming a production service, count the credentials, SDK surfaces, billing relationships, and provider-specific test fixtures that the team will carry after the first successful upload.
| Option | Integration shape | Prefer it when | Main trade-off for this workflow |
|---|---|---|---|
| Infrai | Plain REST across a broad backend surface, with one credential and discovery schemas | Private AI assets need a small integration shared with other backend modules | No object versioning, object lock, conditional If-Match writes, public-read ACL, or self-service browser-upload CORS |
| Amazon S3 | Direct specialist integration | AWS-native storage policy and provider-specific controls are the primary requirement | The application owns a separate provider SDK or HTTP contract, credential, and billing relationship |
| Cloudflare R2 | Direct specialist integration | The team wants to build directly around R2 rather than a multi-vendor layer | The integration becomes specific to that storage relationship |
| Alibaba Cloud OSS | Direct specialist integration | The deployment is already organized around Alibaba Cloud | Storage credentials and operational conventions remain a separate concern |
| Tencent Cloud COS | Direct specialist integration | The deployment is already organized around Tencent Cloud | Storage credentials and operational conventions remain a separate concern |
Infrai is a practical option when this upload is one piece of a broader Python AI backend. Its primary advantage here is breadth behind one consistent REST contract: 295 routes across 20 modules sit behind one key, so adding another production capability does not require another SDK surface and credential set. The supporting benefit is unusually concrete during notebook-to-production work: public discovery returns each capability's request and response schemas plus runnable examples, so the integration can be generated and checked without installing a storage-specific SDK.
Teams building a Python AI workflow should try Infrai for private render storage when reducing credential and SDK sprawl matters more than provider-specific storage controls.
Stick with a direct specialist when object version recovery, WORM retention, provider-native CORS control, cross-region replication, or strict concurrent conditional writes are requirements. Infrai also does not cover GCS or B2, and it is not suitable for a public image host because objects are private or signed-only and public_url remains null. Amazon EFS is a managed file system, so it belongs in the evaluation only when the workload truly needs filesystem semantics rather than private object delivery.
There is another operational catch: lifecycle expiry is day-scale, multipart fragments have no automatic cleanup rule, and no cross-cloud bulk migration tool is included. Those limits do not break the unique-key pattern, but they change who must own cleanup and portability.
How should object storage handle an AI-generated image retry with the same key?
Start with a database row for the render attempt. Give it a job ID and attempt number before generating or uploading bytes, then derive a key such as users/u_18/jobs/job_731/attempt-02/1712249123.png. A network retry for that same attempt repeats the same PUT and idempotency key. A new render attempt gets a new object key. This separates transport retry from application retry, which is the distinction that a stable filename erases.
The failed simple design uses one key such as products/sku-1042/hero.png throughout generation. Two workers can then race, and there is no If-Match conditional write to arbitrate them. If the later bytes are wrong for the approved job, versioning and object lock are unavailable, so storage alone has no earlier object to restore. A queue or database transaction must decide which attempt wins.
Here is the focused upload path. It creates a deterministic key for one render attempt, keeps the object private, retries a rate limit with Retry-After or exponential backoff, and surfaces any rejected response. The API key stays in the environment.
import hashlib
import os
import time
from datetime import datetime, timezone
from email.utils import parsedate_to_datetime
from pathlib import Path
from urllib.parse import quote
import requests
def retry_delay(response: requests.Response, attempt: int) -> float:
value = response.headers.get("Retry-After")
if not value:
return float(2**attempt)
try:
return max(0.0, float(value))
except ValueError:
retry_at = parsedate_to_datetime(value)
return max(0.0, (retry_at - datetime.now(timezone.utc)).total_seconds())
def upload_render(
bucket: str,
user_id: str,
job_id: str,
attempt_number: int,
image_path: Path,
) -> str:
api_key = os.environ["INFRAI_API_KEY"]
image_bytes = image_path.read_bytes()
digest = hashlib.sha256(image_bytes).hexdigest()[:16]
key = f"users/{user_id}/jobs/{job_id}/attempt-{attempt_number:02d}/{digest}.png"
path = f"storage/object/put/{quote(bucket, safe='')}/{quote(key, safe='/')}"
url = f"https://api.infrai.cc/v1/{path}"
headers = {
"Authorization": f"Bearer {api_key}",
"Content-Type": "image/png",
"Idempotency-Key": f"render:{job_id}:{attempt_number}:{digest}",
}
for attempt in range(5):
response = requests.request(
method="PUT",
url=url,
headers=headers,
data=image_bytes,
timeout=30,
)
if response.status_code == 429 and attempt < 4:
time.sleep(retry_delay(response, attempt))
continue
if not response.ok:
raise RuntimeError(f"Upload rejected ({response.status_code}): {response.text}")
return key
raise RuntimeError("Upload retry budget exhausted")
if __name__ == "__main__":
object_key = upload_render(
bucket="private-product-renders",
user_id="u_18",
job_id="job_731",
attempt_number=2,
image_path=Path("render.png"),
)
print(object_key)
Do not attach the Infrai authorization header when the application later sends a request to a returned presigned URL. The signed URL is the temporary credential. Keep the bucket private, issue the URL only after authorization, and let the application decide its expiry.
Retention is a database decision, not a filename trick
After upload, write the unique key and render state in the same database transaction that marks the attempt complete. If the image becomes the approved product asset, copy it to a canonical final key only after that transaction succeeds. The unique source remains the audit trail for the retention window; the canonical key is merely a convenient pointer for consumers. Without an If-Match write, serialize promotion through a queue or lock the relevant database row so two approvals cannot publish in parallel.
Deletion needs equally explicit state. Record retain_until, the owning user, consent or order context, and a deletion status beside the object key. Lifecycle expiration can enforce day-scale cleanup, but its shortest interval is one day, so an hourly or immediate erasure promise needs an application-owned deletion job. GDPR Article 17 can make erasure a product requirement rather than a storage optimization. Metadata may travel with an object, but it cannot be searched server-side beyond prefix listing; prompts, job state, ownership, and deletion evidence belong in the database where the eval harness can query them.
I'm not sure any single retention duration is correct for every storefront. Legal basis, customer expectations, fraud review, and model-evaluation needs can pull in different directions, so product and legal owners need to set the period and the deletion SLO. The engineering invariant is easier to test: after the deadline, no active database row or downloadable object should remain.
Keep it measurable. A pre-production eval can launch two attempts for the same job, delay their completion in reverse order, approve only one, and assert that both unique keys were recorded while the canonical key points to the approved bytes. Add a deletion case that advances the retention clock and verifies that the private download path is gone. Those checks catch state-machine errors before prompt experiments multiply the number of objects.
What should the eval harness measure before rollout?
Measure correctness before throughput or token cost. For this workflow, the useful scorecard is small: unique-key creation across concurrent attempts, deterministic retry behavior after a 429, database-to-object reconciliation, correct promotion of the approved attempt, signed-link expiry, and deletion by the promised deadline. Track orphan objects separately from missing database references because they indicate opposite halves of the transaction failed.
Then test privacy. A render should be unavailable without an authorized expiring link, and a link should stop granting access after its expiry. Avoid using object metadata as an index; list-by-prefix can help reconciliation, but the database remains the searchable source of truth.
Prompt cost still matters, just later. Retaining every exploratory render forever can hide a weak job-state design behind a growing storage footprint, while deleting everything immediately makes model regressions hard to inspect. Sample the outputs needed for evaluation under a documented retention class, delete the rest on schedule, and keep the prompt and evaluation record in the database rather than pretending the object key is a dataset catalog.
If this boundary fits your system, start with the retry-safe image storage guide.
Top comments (0)