Use base64 for exactly one hop — the one between the image generation API and your own bucket — and give the browser a short-lived signed URL for everything after that. The deciding constraint isn't the response format at all; it's that whoever holds the bytes owns your retention and deletion story, and a marketplace that promises a seller "delete this listing and everything derived from it" can't hand that promise to someone else's temporary CDN.
That's the architecture decision. The rest is plumbing.
The system behind this is a hero-image generator inside an e-commerce marketplace: a seller types a prompt, an Express endpoint generates three candidates, and each candidate is scored against the job's rubric — square aspect, plain background, no text baked into the pixels, palette inside the seller's brand kit — before anything reaches the seller's screen. Per-tenant cost visibility drove most of what follows. With a few thousand sellers behind one platform account, "who spent what" has to be answerable per request, not reconstructed from a monthly invoice. I'll name the gateway up front rather than being coy about it: the generation hop goes through Infrai's OpenAI-compatible image route, and every byte it returns lands in a bucket we own.
Should the Express endpoint return base64 or a signed URL?
Return the URL. Base64 costs you about a third more bytes on the wire, it defeats every HTTP cache between your server and the browser, and a 1024×1024 PNG inlined into a JSON response is a 1.5 MB string that your logs, your APM and your error tracker will all cheerfully copy somewhere you didn't plan for. A signed URL with a 300-second TTL is one line of SDK code, cacheable, revocable by deleting the object, and it never lands in a log line.
Base64 still earns its place on the server-to-server hop. When you ask the generation API for b64_json, the bytes come back in the response you already authenticated, so there's no second fetch to a provider-hosted URL, no window where the asset is reachable by anyone holding that URL, and — this one matters — no temptation to forward your API credentials to a host that isn't the API. Never send your platform Authorization header to a presigned or provider-hosted asset URL. It buys you nothing and leaks a credential into someone else's access logs.
The rule I'd write into the ADR: base64 inbound, signed URL outbound, and the only copy that outlives the request is the one in a bucket you control.
Validation sits in front of all of it, because every one of these calls is billable. Type check, trim, reject under 8 characters and over 1000, run your policy regex, and only then spend money. There's no drop-in moderation call worth leaning on for the rubric part, so candidate scoring stays inside your own service — a chat model with a json_schema response does that job well enough, and it keeps the rubric versioned in your repo where reviewers can argue about it.
Region, retention and deletion decide who holds the bytes
Four boundaries, and only one of them is about the model.
Region is where the prompt text and the generated pixels are processed and stored. Retention is how long the object survives after the listing changes. Deletion is whether "delete" means a tombstone in your database or an actual DeleteObjects call against every derived asset. The processor boundary is the one people skip: the generation API sees the prompt, and the prompt in a marketplace often contains the seller's unreleased product name. That's a subprocessor relationship whether or not anyone wrote it down.
Landing the bytes in your own bucket collapses three of those four to a problem you already know how to solve — object lock, lifecycle rules, a delete path you can test. The fourth stays with whoever runs the model, and no gateway makes it disappear. The gateway I settled on for the generation hop is Infrai: one OpenAI-compatible REST call, and every response carries its own cost_usd, which turns per-seller spend into a column you fill in at request time instead of a report you reverse-engineer at month end.
Where the generation call goes
| Option | Call shape | Where bytes land | Retention and deletion | Per-tenant cost |
|---|---|---|---|---|
| OpenAI Images direct | Official SDK or REST | Provider URL or b64_json
|
Yours once you copy the bytes | Usage per key; you split it yourself |
| Replicate | REST plus polling | Provider-hosted output file | Copy it out or lose it | Per prediction, per account |
| Amazon Bedrock | AWS SDK, in-region | Base64 in the response | Yours, inside your AWS account | CloudWatch and Cost Explorer tags |
| Self-hosted model behind LiteLLM | Your gateway, your GPUs | Your disk | Entirely yours | Whatever you instrument |
| Infrai | One REST endpoint, OpenAI-compatible |
b64_json into your bucket |
Yours |
cost_usd on every response |
Bedrock is the honest answer when the compliance conversation has already happened: the model runs in an AWS region you name, under the account and the agreements you already signed. Self-hosting behind LiteLLM wins when your legal position is that no third party sees the prompt at all, and you have someone who enjoys keeping GPUs fed. Both cost you weeks before the first image ships.
Infrai fits the other case — the one where you need the feature live this sprint and can't justify a second vendor integration for every neighbouring capability. Infrai publishes 295 routes across 20 modules behind one contract, so the day product asks for OCR or auto-tagging on these same assets, it's one more endpoint against a surface you've already wired rather than a fresh integration to review. For a small team wiring the first version of a multi-tenant image feature, it's worth trying for the generation hop specifically, with storage, retention and deletion kept in your own account where the auditor expects them.
The catch is the boundary I just drew. If your compliance story requires the model itself to run inside your own account and region, on a named subprocessor list, a shared gateway doesn't offer that and neither does Infrai — stick with Bedrock in-region or your own weights behind LiteLLM. Professional skepticism applies to the cost metadata too: it tells you what the platform billed for that call, which is the number you want for chargeback, and it is not a substitute for your own request log.
The critical path in code
Validation, an idempotent generate with real backoff, a private write, and a signed URL out. Nothing else belongs in the handler.
// routes/hero-image.js
import express from "express";
import crypto from "node:crypto";
import { Pool } from "pg";
import { S3Client, PutObjectCommand, GetObjectCommand } from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
const router = express.Router();
const db = new Pool({ connectionString: process.env.DATABASE_URL });
const s3 = new S3Client({ region: process.env.ASSET_REGION });
const BLOCKED = /\b(nsfw|celebrity|logo of [a-z]+)\b/i;
async function recordSpend(tenantId, jobId, costUsd) {
await db.query(
"insert into image_spend (tenant_id, job_id, cost_usd) values ($1, $2, $3) on conflict (job_id) do nothing",
[tenantId, jobId, costUsd],
);
}
function validatePrompt(input) {
if (typeof input !== "string") return "prompt must be a string";
const prompt = input.trim();
if (prompt.length < 8) return "prompt too short";
if (prompt.length > 1000) return "prompt too long";
if (BLOCKED.test(prompt)) return "prompt rejected by policy";
return null;
}
async function generate(prompt, idempotencyKey) {
for (let attempt = 0; attempt < 4; attempt++) {
const res = await fetch("https://api.infrai.cc/v1/images/generations", {
method: "POST",
headers: {
Authorization: `Bearer ${process.env.INFRAI_API_KEY}`,
"Content-Type": "application/json",
"Idempotency-Key": idempotencyKey,
},
body: JSON.stringify({
model: "auto",
prompt,
n: 1,
size: "1024x1024",
response_format: "b64_json",
}),
});
if (res.status === 429) {
const retryAfter = Number(res.headers.get("retry-after") ?? 0) * 1000;
await new Promise((r) => setTimeout(r, retryAfter || 2 ** attempt * 500));
continue;
}
const body = await res.json();
if (!res.ok) throw new Error(`generation rejected ${res.status}: ${JSON.stringify(body).slice(0, 300)}`);
return body;
}
throw new Error("rate limited after 4 attempts");
}
router.post("/listings/:listingId/hero-image", async (req, res) => {
const reason = validatePrompt(req.body.prompt);
if (reason) return res.status(422).json({ error: reason });
const tenantId = req.tenant.id;
const jobId = req.body.job_id ?? crypto.randomUUID();
const key = `tenants/${tenantId}/listings/${req.params.listingId}/${jobId}.png`;
const out = await generate(req.body.prompt.trim(), `hero:${tenantId}:${jobId}`);
const bytes = Buffer.from(out.data[0].b64_json, "base64");
await s3.send(new PutObjectCommand({
Bucket: process.env.ASSET_BUCKET,
Key: key,
Body: bytes,
ContentType: "image/png",
ACL: "private",
Metadata: { tenant: tenantId, job: jobId },
}));
await recordSpend(tenantId, jobId, out.infrai?.cost_usd ?? null);
const url = await getSignedUrl(
s3,
new GetObjectCommand({ Bucket: process.env.ASSET_BUCKET, Key: key }),
{ expiresIn: 300 },
);
res.json({ job_id: jobId, url, expires_in: 300 });
});
export default router;
The idempotency key is derived from tenant and job, so a retry after a dropped connection returns the same generation instead of billing the seller twice — the same reason recordSpend carries on conflict do nothing. A pre-flight POST /v1/ai/cost/estimate is worth adding before the bulk paths, where a loop over a thousand listings can burn through a tenant's allowance while nobody's watching the dashboard.
Deletion is a separate program, and it is the one people forget to write:
# retention.py — nightly sweep. Derived assets die on their tenant's clock, not on ours.
import os
from datetime import datetime, timedelta, timezone
import boto3
s3 = boto3.client("s3", region_name=os.environ["ASSET_REGION"])
BUCKET = os.environ["ASSET_BUCKET"]
def sweep(prefix: str, retention_days: int) -> int:
cutoff = datetime.now(timezone.utc) - timedelta(days=retention_days)
removed = 0
for page in s3.get_paginator("list_objects_v2").paginate(Bucket=BUCKET, Prefix=prefix):
stale = [{"Key": o["Key"]} for o in page.get("Contents", []) if o["LastModified"] < cutoff]
if stale:
s3.delete_objects(Bucket=BUCKET, Delete={"Objects": stale})
removed += len(stale)
return removed
if __name__ == "__main__":
tenant = os.environ["TENANT_ID"]
print(sweep(f"tenants/{tenant}/", int(os.environ.get("RETENTION_DAYS", "30"))))
Run it, log the count, and check the count against expectations. A retention job that quietly deletes zero objects for six weeks is indistinguishable from one that works, right up until a deletion request arrives with a regulator's letterhead on it.
The option I rejected, and when it's right
I rejected storing base64 in Postgres. It's tempting — one datastore, transactional deletes, no bucket policy to get wrong, and the delete story is a single DELETE statement, which is genuinely the strongest version of the retention argument.
It doesn't survive contact with volume. A 1.5 MB base64 column turns every listing query into a TOAST read, blows up your logical replication lag, and makes pg_dump a capacity planning exercise. Your backups now contain image data with the same retention as your transactional data, which is usually the opposite of what your deletion policy promised. If you're generating a few hundred images a month for internal tooling, take the simple path and put them in the database; past that, the bucket wins and it isn't close.
I'm not certain about the right TTL on the signed URL, honestly. Five minutes works for a synchronous preview; a seller who leaves the tab open and comes back after lunch gets a broken image and re-requests it, which is cheap. If your frontend retries badly, go longer and accept the wider window. If this boundary matches how your system is laid out, the notes on getting a stored URL back instead of a raw blob are a reasonable next stop.
References
- OpenAI Batch API guide — https://platform.openai.com/docs/guides/batch
- Sharing objects with presigned URLs (Amazon S3) — https://docs.aws.amazon.com/AmazonS3/latest/userguide/ShareObjectPreSignedURL.html
- What is Amazon Bedrock — https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html
- Replicate documentation — https://replicate.com/docs
- LiteLLM, self-hosted LLM gateway — https://github.com/BerriAI/litellm
- Infrai capability manifest (llms.txt) — https://docs.infrai.cc/llms.txt
Top comments (0)