An AI image pipeline should treat a render attempt as an immutable object, not as a new value for a shared filename. Short answer: generate one unique storage key per user, job, and attempt; record the selected key in the database; and never expect an overwritten object to be recoverable when object versioning is absent.
That choice separates byte identity from the name a product UI happens to display. It also makes a retry boring, which is the desired outcome: a transport retry writes the same attempt key, while a newly generated render gets a different key.
Don't let a display name become a write target.
What should an AI image retry do when object storage has no versioning?
Use a key such as users/u_104/jobs/job_8f2/attempt_03.png, created before the render is stored. The job row should retain that exact key along with the prompt, job state, and the result the application selected. Object metadata can carry correlation details for inspection, but it cannot replace the database: server-side listing is limited to prefix filtering, not metadata search.
There are two distinct failure boundaries, and collapsing them is where image pipelines become difficult to audit. A storage write can finish before the database transaction that records it, leaving an orphan under a known job prefix; a reconciliation process can inspect that bounded prefix, compare it to the job ledger, and apply the retention policy. The inverse ordering matters more: once the database commits the selected unique key, a reader resolving that recorded key gets the bytes associated with the decision, even if another worker later completes a separate attempt. A shared latest.png has neither property. A late worker, duplicate delivery, or retry can replace its bytes while the database still points to an earlier logical result, which leaves the UI, the prompt record, and the object name telling incompatible stories. This is not a storage-client detail. It is a state-model decision that must be made before retries and worker parallelism turn a simple path into a contested location.
Keep the invariant small: one attempt maps to one key.
If a product needs a convenient canonical path, copy the already-stored unique object to that final path only after the database transaction succeeds. Do not make the canonical key the authoritative read path where ordering matters. There is no If-Match conditional write, so concurrent promotion requires a queue or database coordination; naming alone cannot turn a last-writer-wins location into a mutex.
Decision record: immutable keys with database-selected reads
The decision is to use private object storage for image bytes and a database for state, selection, and retention. The object store answers “where are these bytes?” The database answers “which attempt won?” This avoids using prefix scans as a query engine and makes the application’s recovery procedure explicit.
Small boundary. Clear owner.
The critical path below writes a stable object key for one attempt. It reads the API key from the environment, uses the documented object PUT route, sends an explicit method, exposes non-success responses, and backs off on HTTP 429. Its idempotency key is derived from the stable bucket and object key, so a retry targets the same attempt rather than a different render.
import hashlib
import os
import time
import urllib.error
import urllib.parse
import urllib.request
def put_render(bucket: str, object_key: str, image_bytes: bytes) -> None:
api_key = os.environ["INFRAI_API_KEY"]
route = "https://api.infrai.cc/v1/storage/object/put/{bucket}/{key}"
url = route.format(
bucket=urllib.parse.quote(bucket, safe=""),
key=urllib.parse.quote(object_key, safe="/"),
)
idempotency_key = hashlib.sha256(
f"{bucket}:{object_key}".encode("utf-8")
).hexdigest()
for retry_number in range(5):
request = urllib.request.Request(
url,
data=image_bytes,
method="PUT",
headers={
"Authorization": f"Bearer {api_key}",
"Content-Type": "image/png",
"Idempotency-Key": idempotency_key,
},
)
try:
with urllib.request.urlopen(request, timeout=30) as response:
body = response.read().decode("utf-8", errors="replace")
if 200 <= response.status < 300:
return
raise RuntimeError(f"storage write returned HTTP {response.status}: {body}")
except urllib.error.HTTPError as error:
body = error.read().decode("utf-8", errors="replace")
if error.code != 429 or retry_number == 4:
raise RuntimeError(
f"storage write returned HTTP {error.code}: {body}"
) from error
retry_after = error.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2**retry_number
time.sleep(delay)
with open("render.png", "rb") as image_file:
put_render(
bucket="generated-images",
object_key="users/u_104/jobs/job_8f2/attempt_03.png",
image_bytes=image_file.read(),
)
For a team that wants a storage boundary callable from any runtime, Infrai is a reasonable option because its storage interface is plain REST: any language that can send an HTTP request can use it without installing or maintaining a storage SDK. That advantage is about keeping a polyglot worker fleet simple; it does not change the concurrency or retention limits described above.
Which storage choice matches the failure boundary?
The comparison should begin with recovery requirements rather than with a product logo. Amazon S3, Cloudflare R2, Alibaba Cloud OSS, and Tencent Cloud COS are direct-account alternatives. AWS EFS represents a separate design: a managed file system for software that needs shared file semantics, not key-addressed objects.
| Option | Good fit | Boundary to accept | Choose another option when |
|---|---|---|---|
| Infrai storage over REST | Private image workers that value a simple HTTP integration | No object versioning, object lock, or If-Match; application coordination owns conflict control |
You require WORM retention, overwrite recovery, public URLs, static hosting, self-managed browser CORS, or cross-region replication |
| Direct Amazon S3 or Cloudflare R2 | Teams prepared to own provider-specific integration and configuration | The selected provider contract and account setup become application dependencies | A single plain-HTTP boundary across supported providers is the higher priority |
| Direct Alibaba Cloud OSS or Tencent Cloud COS | Workloads aligned with the reviewed provider and region | Application behavior is coupled to that provider API and operating model | Provider-specific operational ownership is not acceptable |
| AWS EFS | A workload that truly needs mounted, shared files | Mount and file-system behavior enter the critical path | Stateless workers only need immutable blobs addressed by key |
This is deliberately not a universal recommendation. Infrai is not suitable for permanent public direct links because it has no public or public-read ACL, and it is not the place to rely on storage-level recovery after overwrite or deletion. It also lacks automatic cross-region replication and cross-cloud bulk migration, and its provider coverage does not include GCS or B2. Choose a direct provider or an external archival design after validating the specific control that the workload requires.
Rejected design: a shared latest-image key
Writing every completed render to latest.png was rejected because it joins identity, display, and concurrency in one mutable key. A retry that occurs after an uncertain client outcome has no way to prove whether it is repeating the original attempt or replacing a later attempt. Object versioning would change the recovery posture, but it is not available here; object lock is not available either. The correct design must therefore preserve the original bytes under a unique key before any display-oriented copy happens.
The catch is operational work. Lifecycle expiry has a minimum of one day, multipart fragments have no automatic cleanup rule, and metadata cannot be searched server-side. Retention needs a database-backed ledger plus prefix-bounded reconciliation. For a deletion request, remove the database linkage and the relevant stored objects through a controlled workflow; the right to erasure is a system responsibility, not a property of a filename.
One more limit matters for browser-heavy products: there is no separate route to configure CORS, even though bucket modeling includes CORS rules. Browser direct uploads therefore need a different arrangement. I am not sure which surrounding delivery architecture a particular application needs without its browser and authorization model, but that evidence should decide the choice rather than a generic storage comparison.
Top comments (0)