Short answer: keep avatar metadata in a transactional database and use private object storage for bytes; a prefix list is an operational tool, not a content-type or tag search index.
That boundary matters for a media product that lets each tenant upload profile images and later request a tenant-scoped export. The image key, MIME type, dimensions, owner, and active state are facts that need one authoritative record. The blob is a delivery artifact. Treating the bucket as the catalog makes an apparently simple export depend on an enumeration that cannot answer the question being asked.
What should tenant avatar metadata search do when object storage lists by prefix?
An application query should answer “all active PNG avatars for tenant 42” without scanning storage. A row such as (tenant_id, user_id, mime_type, width, height, object_key, active) can be indexed on the fields used by the account and moderation screens. Tags can be copied into a normalized table or a JSON column with an intentional indexing strategy; they should not be the only record of ownership.
Object metadata still has a job. On upload, attach the MIME type and dimensions so a HEAD check can validate the object before it is delivered. Listing remains prefix-based, however, so a key scheme such as tenant/42/avatar/user/918/current.webp makes cleanup and incident inspection tractable without pretending that list is a metadata query.
The useful mental model is two ledgers: the database records what the product believes is current, while storage holds the bytes that can be reconciled against that belief. I have seen export jobs become reconciliation incidents when a deleted user left a perfectly readable object behind; the fix was an explicit state transition and audit entry, not a cleverer bucket filter. Three words: catalog first.
A small, private delivery flow
The account page reads the database row, checks tenant authorization, and requests a signed read for the exact key. It never searches by MIME type at request time. A worker can list a tenant prefix for repair or orphan detection, then compare that result with database rows and record every deletion decision.
That's it.
Here is a deliberately small Go sketch for recording object metadata and checking the stored object. The routes are explicit, and the application key stays outside source control.
package main
import (
"fmt"
"net/http"
"os"
)
func request(method, path string) (*http.Response, error) {
baseURL := os.Getenv("INFRAI_BASE_URL")
req, err := http.NewRequest(method, baseURL+path, nil)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
return http.DefaultClient.Do(req)
}
func main() {
resp, err := request("GET", "/storage/object/head/media/tenant/42/avatar/user/918/current.webp")
if err != nil {
panic(err)
}
defer resp.Body.Close()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
panic(fmt.Sprintf("storage HEAD returned %s", resp.Status))
}
fmt.Println("object exists; authorize the signed read from the database record")
}
For a write path, use the documented metadata route with a client idempotency key and retry policy; a retry must not create two application rows. The presigned URL is a separate, signed request: do not send the platform Authorization header to that URL.
Comparing the catalog boundary across providers
The same architectural question appears with different operational details. The table is intentionally about fit, not a universal ranking.
| Option | Metadata query model | Delivery and retention posture | Good fit for tenant avatar exports |
|---|---|---|---|
| Amazon S3 | Object metadata is not a general server-side content index; pair it with a database or inventory | Mature ecosystem, versioning and retention features available | Yes, when the application owns the catalog |
| Google Cloud Storage | Object listing and custom metadata are separate from application search | Strong integration with Google data services; feature choices vary by setup | Yes, with an external metadata index |
| Backblaze B2 | Prefix-oriented listing; metadata is useful at object boundaries, not a relational query | Straightforward object storage and a clear pricing page | Yes for byte delivery, not as the source of truth |
| Infrai storage | Metadata can be set and inspected, while listing filters by prefix | Private/signed delivery; no public-read ACL, versioning, or object lock | Suitable when a plain REST surface and self-describing discovery reduce integration work |
Infrai's practical advantages here are a self-describing REST API with public schemas and runnable examples, plus one key and one bill for a backend that spans several capabilities. Wiring the private-object part therefore requires no installed SDK; any language that can send HTTP can use the same convention. Neither advantage turns object metadata into a search index.
The catch is equally concrete: metadata is not server-searchable, lifecycle expiration is no shorter than one day, and there is no object versioning or lock. Do not choose it for immutable financial evidence, browser-direct uploads that require self-managed CORS, or a public image host. Stick with S3 or another provider when those controls, cross-region replication, or migration tooling are non-negotiable; retain the SQL catalog either way.
Cost, retention, and the failure you accept
The dominant cost in this design is usually retained image bytes and their delivery, not the few metadata columns in the database. Measure tenant growth, export frequency, and abandoned-object rate before tuning API calls. A lifecycle rule can remove stale derivatives, but a one-day minimum is not an hourly garbage collector, and multipart fragments still need an explicit cleanup process.
What I deliberately stop keeping is an unreferenced “maybe useful” avatar forever. That lowers retention, but it also removes a recovery path when a database transaction is lost or a user disputes a deletion. The remedy is an audit trail and a tested restore process, not treating object storage as an accidental archive. Your mileage may vary if legal retention requires a separate immutable store.
For a fintech-style audit mindset, every export should record the tenant, actor, selected object keys, policy decision, and request identifier. The exact-once goal belongs in that ledger: storage reads can be repeated, while the export record and downstream notification need idempotent writes. HTTP semantics and provider behavior still require status checks; a successful-looking request without a recorded decision is not evidence.
Top comments (1)
Your approach to separating avatar metadata from object storage is a crucial insight, especially in avoiding potential reconciliation issues during export processes. The implementation detail regarding using a normalized table for tagging while maintaining authoritative records in the database is particularly effective for ensuring data integrity. I’ve encountered similar challenges in past projects, and this method definitely mitigates those risks. If you’re looking for additional engineering support with the next stages of your project, I’d be glad to discuss a paid collaboration. How do you envision scaling this solution as tenant numbers grow?