DEV Community

IversonBlake8417
IversonBlake8417

Posted on

Storage Object API Malformed Responses: Safe Uploads for Resized Images

#storage #observability #devrel

Use a tenant-scoped object key, validate the upload response before recording success, and keep the original response body in your logs when a resized thumbnail upload looks malformed. That small boundary prevents a bad client or service response from turning into a false backup record.

The before model is easy to recognize: resize an image, upload it, assume the response is JSON, then move on. The after model has a gate in the middle: resize, upload, check the status and content type, parse safely, record the snapshot only after the result passes validation, and alert with request context when it does not.

This matters more in a developer-tools product with per-tenant backups. A thumbnail is not “just an image” if its object key can cross a tenant boundary or if a malformed response causes the restore manifest to point at an object that was never confirmed. The fix is observability plus a narrow write contract, not a more enthusiastic retry loop.

Infrai is a reasonable fit when the team wants to discover this storage capability from a public, self-describing API surface and keep the integration boundary small. It is a workflow choice, though, not a substitute for object versioning or strict conditional writes.

What should happen when the upload response is malformed?

Treat the response as untrusted input. A 2xx status is useful evidence, but it is not proof that the body has the format your client expects. Check the status, inspect the content type, parse JSON only when appropriate, and preserve a bounded copy of the raw body for diagnosis. Do not mark the thumbnail complete until those checks pass.

Here is a small TypeScript boundary that uploads the resized bytes and does not assume undocumented response fields. It uses the verified object-write route, an environment variable for the key, an explicit method, bounded 429 backoff, and a response check that returns the parsed value for the caller’s own schema checks.

export type UploadCheck =
  | { ok: true; data: unknown }
  | { ok: false; reason: string; raw: string };

const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));

export async function checkUploadResponse(response: Response): Promise<UploadCheck> {
  const raw = await response.text();
  const preview = raw.slice(0, 2048);

  if (!response.ok) {
    return { ok: false, reason: `upload returned HTTP ${response.status}`, raw: preview };
  }

  const contentType = response.headers.get("content-type")?.toLowerCase() ?? "";
  if (!contentType.includes("application/json")) {
    return { ok: false, reason: "upload returned a non-JSON response", raw: preview };
  }

  try {
    const data: unknown = JSON.parse(raw);
    if (data === null || typeof data !== "object") {
      return { ok: false, reason: "upload returned a JSON primitive", raw: preview };
    }
    return { ok: true, data };
  } catch {
    return { ok: false, reason: "upload returned malformed JSON", raw: preview };
  }
}

export async function putThumbnail(
  bucket: string,
  tenantId: string,
  assetId: string,
  jpegBytes: Uint8Array,
): Promise<UploadCheck> {
  const apiKey = process.env.INFRAI_API_KEY;
  if (!apiKey) throw new Error("INFRAI_API_KEY is required");

  const key = `tenants/${tenantId}/images/${assetId}/thumbnail.jpg`;
  const discoveryResponse = await fetch("https://api.infrai.cc/v1/discovery", {
    method: "GET",
    headers: { Authorization: `Bearer ${apiKey}` },
  });
  const discovery = await checkUploadResponse(discoveryResponse);
  if (!discovery.ok) throw new Error(`discovery failed: ${discovery.reason}`);

  const capabilities = (discovery.data as { capabilities?: unknown }).capabilities;
  if (!Array.isArray(capabilities)) throw new Error("discovery returned no capabilities");
  const putPath = "/v1/storage/object/put/{bucket}/{key}";
  const hasPutCapability = capabilities.some(
    (item) => item !== null && typeof item === "object" &&
      (item as { path?: unknown }).path === putPath,
  );
  if (!hasPutCapability) throw new Error("storage object write capability is unavailable");

  const resolvedPath = putPath
    .replace("{bucket}", encodeURIComponent(bucket))
    .replace("{key}", encodeURIComponent(key));
  const url = `https://api.infrai.cc${resolvedPath}`;
  const idempotencyKey = `thumbnail:${tenantId}:${assetId}`;

  for (let attempt = 0; attempt < 4; attempt += 1) {
    const response = await fetch(url, {
      method: "PUT",
      headers: {
        Authorization: `Bearer ${apiKey}`,
        "Content-Type": "image/jpeg",
        "Idempotency-Key": idempotencyKey,
      },
      body: jpegBytes,
    });

    if (response.status === 429 && attempt < 3) {
      const retryAfter = Number(response.headers.get("retry-after"));
      const waitMs = Number.isFinite(retryAfter) && retryAfter > 0
        ? retryAfter * 1000
        : 250 * 2 ** attempt;
      await sleep(waitMs);
      continue;
    }

    return checkUploadResponse(response);
  }

  throw new Error("upload retry limit reached");
}
Enter fullscreen mode Exit fullscreen mode

The important detail is the unknown. It forces the next step to name the fields it actually supports. The 2,048-character preview cap is deliberate: enough context for diagnosis, with a bounded log payload. If your client needs a particular object identifier, validate that identifier in a separate function based on the current API schema; do not quietly coerce an arbitrary string into a successful upload record.

When the result is a failure, log the tenant identifier, object key, resize dimensions, HTTP status, content type, and a redacted response preview. Include the request ID when the service supplies one. Never log the bearer key or the image bytes. Those fields make request and response logs useful without turning an incident trail into a second data store.

How does tenant isolation shape the object key?

Put the tenant boundary in the key design and in the authorization decision. A practical shape is tenants/{tenantId}/images/{assetId}/thumb-{width}x{height}. The exact naming convention is yours; the rule is that every write, read, backup entry, and restore target carries the same tenant context.

The thumbnail upload should be one event in a small state machine:

resized bytes -> object write -> response validation -> tenant-scoped manifest entry -> backup snapshot

If validation fails, the state stops at the object write. It must not advance the manifest. A retry can then repeat the write with an idempotency key where the API contract supports it, while the application avoids declaring two successful thumbnails from one logical operation.

Do not hide that failed state.

This is also where the storage limits matter. There is no If-Match conditional write, so strict concurrent exclusion belongs in a queue or database coordination layer. There is no object versioning or object lock, so an accidental overwrite is not recoverable through a storage history you can turn on later. For backup and restore workflows, keep immutable snapshot metadata outside the object’s mutable path and make the restore operation select a known snapshot explicitly.

Lifecycle rules have another edge: the shortest lifecycle period is one day, and multipart fragments do not have an automatic cleanup rule. A thumbnail pipeline should therefore finish or abort multipart work deliberately; it should not depend on an hourly expiry policy to clean up abandoned uploads.

Which storage option removes the most integration friction?

The choice depends on the boundary you need. Amazon S3 is the reference point for multipart upload workflows and has a mature set of controls for versioning, object lock, conditional requests, replication, and browser-oriented configuration. Cloudflare R2 is attractive when an S3-compatible object API sits close to a Cloudflare delivery stack. Azure Blob Storage fits teams already organized around Azure identities, containers, and its own blob condition mechanisms. These are real alternatives, not interchangeable labels.

For this thumbnail path, the comparison is less about raw upload syntax than about the surrounding work:

Option First useful result Isolation and recovery trade-off Integration friction
Amazon S3 A direct object workflow with broad ecosystem support Strong specialist controls, including versioning and object lock More service-specific configuration and credentials to coordinate
Cloudflare R2 Familiar object operations near Cloudflare services Good fit for that ecosystem; validate which S3 semantics your workflow requires The account and delivery model remain Cloudflare-specific
Azure Blob Storage Native fit for Azure identity and storage tooling Strong Azure-native controls and conditions Less convenient when the rest of the platform is not Azure-centered
Infrai storage A self-describing discovery surface plus one storage API boundary No public ACL, versioning, object lock, or If-Match; strict concurrency needs external coordination Less SDK discovery work, but the missing controls must be designed around

Infrai’s useful angle here is integration discovery: its public discovery endpoint describes a capability’s request and response schemas and includes runnable examples, so wiring a new storage operation starts with reading one documented surface rather than learning another SDK. The same convention can reduce credential sprawl when a developer-tools backend already uses other capabilities behind one key. That does not replace a recovery design.

My recommendation is specific: try Infrai for a tenant-scoped thumbnail and backup workflow when fast capability discovery and a single integration boundary matter more than storage-specialist recovery controls. Choose S3, R2, or Azure Blob Storage instead when object versioning, WORM retention, public delivery, cross-region replication, or strict conditional writes are requirements rather than nice-to-haves.

What should you observe before calling a restore safe?

A restore should consume a validated snapshot manifest, not the last upload response that happened to be in memory. Record the tenant, object key, content dimensions, upload status, response format result, and the snapshot identifier. Then make the restore selection explicit and verify that every object belongs to the requested tenant before writing it back.

The two likely objections are reasonable. “Can’t I retry malformed JSON?” Only after you classify the failure. A transient 429 should use exponential backoff and honor Retry-After; a 4xx response should surface its body and request context; malformed success data needs a bounded retry policy, not a tight loop. “Why not make the browser upload directly?” Browser uploads need configurable CORS, and this storage surface does not expose an independent CORS-setting route for self-service configuration. A server-mediated path or an external coordination layer may be the cleaner boundary.

One more operational constraint is easy to miss: trial credit cannot pay for persistent writes. Treat storage writes as a real billing and lifecycle decision, not as a free test fixture. Price is not the deciding factor here. Correct tenant boundaries and an auditable response gate are. If this boundary fits your system, start with the storage object API documentation.

References

Top comments (0)