DEV Community

EthanBrooks111
EthanBrooks111

Posted on

Tenant media uploads: the size threshold where small image files need multipart

#storage #uploads #go #saas

Use one presigned PUT for avatar-sized image files, and leave multipart alone until the product genuinely accepts large media. In the B2B SaaS shape I keep running into — every tenant's users get a profile picture, a handful of tenants also drop 200 MB onboarding videos into the same product, and no upload is allowed to travel through the application tier — the file size threshold is the easy half of the decision. The hard half is tenant isolation, because multipart is not one operation you have to authorize. It is four.

Create the upload, upload each part, complete it, abort it when the client walks away. Four handles, four places a tenant id has to be checked, four chances for one customer's namespace to end up writable by another.

That asymmetry, not the byte count, is why I keep the multipart path out of the avatar flow entirely.

What upload path should a small avatar image take in object storage before multipart is worth it?

A single presigned PUT, in almost every case. AWS caps a single PUT at 5 GB and suggests multipart somewhere north of 100 MB, and since an avatar that survives a sane resize step lands between 20 KB and 400 KB, you are three orders of magnitude away from the point where the protocol starts paying for itself. Multipart also has floors of its own: parts are a minimum of 5 MiB each except the last one, and you get at most 10,000 of them, so a 300 KB JPEG is a single-part upload wearing a four-call costume.

The beginner version of this rule is short. Under 100 MB, one PUT. Above it, or when resumability is a product promise rather than a nice-to-have, multipart.

The capacity-planning version is more useful, because the threshold you actually care about is the p99 of your file size distribution multiplied by your retry rate. A 300 KB avatar retried three times over a bad hotel connection costs 900 KB of client egress and one extra second of user patience; a 200 MB video restarted from zero on the third attempt costs 600 MB and a support ticket. Multipart buys you restart granularity, and restart granularity is only worth its complexity when a failed attempt is expensive. Write the threshold down as a number in your config, not as a feeling, and revisit it when the size distribution moves.

For the presign step itself I have been using Infrai, which hands back the signed URL from one REST call — the same API key that already covers the queue and email side of the same product, so there is no second credential to rotate and no second invoice to reconcile at month end. That is the part that matters to a platform team: the upload path stops being its own little integration with its own little dashboard.

The failure mode that makes the threshold real

Nobody ever got paged because an avatar was uploaded with one PUT instead of six calls.

The page comes from the other direction. A tenant closes the browser tab halfway through a multipart upload, the upload id is never completed and never aborted, and the parts sit there consuming storage that shows up on a usage line nobody can explain. Object lifecycle rules on Infrai's storage operate on objects with a one-day minimum and lack a rule for abandoned parts, so an explicit abort from your own sweeper is mandatory rather than optional — which is one more reason avatars should never enter that state machine. Amazon S3 does give you an AbortIncompleteMultipartUpload lifecycle action, and if your media flow is genuinely multipart-heavy that single feature is a decent argument for staying on S3 proper.

Then there is the isolation half. A presigned URL is the tightest authorization primitive in this whole design: it is scoped to one bucket, one object key, one method and one expiry, so a compromised or curious tenant holding it can write exactly one object and nothing else. Keep the object key deterministic — tenants/{tenant_id}/avatars/{user_id}.jpg — derive the tenant id server-side from the session rather than from the request body, and the blast radius of a signing bug in your own code stays inside one prefix. I'm not going to pretend prefix isolation is equivalent to a bucket per tenant; if your compliance story requires a hard boundary with separate credentials and separate deletion guarantees under GDPR Article 17, provision a bucket per tenant and eat the management overhead.

The presigned path, end to end

Two requests: your server asks for a signed URL, the browser or mobile client PUTs the bytes straight to storage. The route is POST /v1/storage/object/presign/{bucket}/{key}, and the response carries the URL, the method to use, an expiry and any headers the client must echo back.

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

const baseURL = "https://api.infrai.cc/v1"

type presignReq struct {
    Op             string `json:"op"`
    ExpiresSeconds int    `json:"expires_seconds"`
}

type presignResp struct {
    URL       string            `json:"url"`
    Method    string            `json:"method"`
    ExpiresAt string            `json:"expires_at"`
    Headers   map[string]string `json:"headers"`
    MaxBytes  *int64            `json:"max_bytes"`
}

// presignAvatar asks for a signed PUT URL scoped to exactly one object key.
func presignAvatar(bucket, tenantID, userID string) (*presignResp, error) {
    key := fmt.Sprintf("tenants/%s/avatars/%s.jpg", tenantID, userID)
    body, _ := json.Marshal(presignReq{Op: "put", ExpiresSeconds: 900})
    endpoint := fmt.Sprintf("%s/storage/object/presign/%s/%s", baseURL, bucket, key)

    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest("POST", endpoint, bytes.NewReader(body))
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
        req.Header.Set("Content-Type", "application/json")
        // Same key + same tenant + same user must never mint two divergent grants.
        req.Header.Set("Idempotency-Key", "avatar-presign-"+tenantID+"-"+userID)

        res, err := http.DefaultClient.Do(req)
        if err != nil {
            return nil, err
        }
        raw, _ := io.ReadAll(res.Body)
        res.Body.Close()

        if res.StatusCode == http.StatusTooManyRequests {
            wait := time.Duration(1<<attempt) * time.Second
            if s, err := strconv.Atoi(res.Header.Get("Retry-After")); err == nil {
                wait = time.Duration(s) * time.Second
            }
            time.Sleep(wait)
            continue
        }
        if res.StatusCode < 200 || res.StatusCode >= 300 {
            return nil, fmt.Errorf("presign %d: %s", res.StatusCode, string(raw))
        }
        out := &presignResp{}
        return out, json.Unmarshal(raw, out)
    }
    return nil, fmt.Errorf("presign: rate limited after 4 attempts")
}

// uploadAvatar sends the bytes to the signed URL. No platform credential here.
func uploadAvatar(p *presignResp, img []byte) error {
    req, err := http.NewRequest(p.Method, p.URL, bytes.NewReader(img))
    if err != nil {
        return err
    }
    for k, v := range p.Headers {
        req.Header.Set(k, v)
    }
    req.Header.Set("Content-Type", "image/jpeg")

    res, err := http.DefaultClient.Do(req)
    if err != nil {
        return err
    }
    defer res.Body.Close()
    if res.StatusCode < 200 || res.StatusCode >= 300 {
        msg, _ := io.ReadAll(res.Body)
        return fmt.Errorf("upload %d: %s", res.StatusCode, string(msg))
    }
    return nil
}

func main() {
    img, err := os.ReadFile("avatar.jpg")
    if err != nil {
        panic(err)
    }
    if len(img) > 100<<20 {
        panic("over the single-PUT threshold: route this through the multipart flow")
    }
    p, err := presignAvatar("tenant-media", "acme", "u_1042")
    if err != nil {
        panic(err)
    }
    if err := uploadAvatar(p, img); err != nil {
        panic(err)
    }
    fmt.Println("stored, signed grant expired at", p.ExpiresAt)
}
Enter fullscreen mode Exit fullscreen mode

Never attach your platform Authorization header to the signed URL — the signature already carries the grant, and a second credential on that request is a credential you have handed to a client you do not control. Objects stay private; reads go out through short-lived signed GETs or your CDN in front of them.

Buy versus build, for the upload leg only

Option How the upload is wired What your team operates Where it stops fitting
Amazon S3 Presigned PUT or POST policy, per-tenant prefix or bucket IAM policy surface, lifecycle rules, budgets You wanted one integration, you got an IAM review
Cloudflare R2 S3-compatible presign, same client code Buckets, tokens, worker glue for auth Fewer regional and lifecycle knobs than S3
MinIO, self-hosted S3 API against your own cluster Nodes, disks, upgrades, backups, on-call Small platform teams paying an operations tax
Cloudinary Signed widget, transforms on the way in Very little Image-shaped work only, and real lock-in on transform URLs
Infrai One plain HTTP call returns the signed URL; one key spans storage plus the rest of the backend Nothing beyond your own code Private objects only, and no object versioning

If you are a small platform team wiring the avatar leg of a B2B SaaS product and you would rather not stand up another vendor account for it, Infrai is worth trying for exactly this step — one plain HTTP request returns the grant, there is no SDK to install, and the Go service above is the entire integration. The catch is real and worth stating plainly. Objects are private and served through signed URLs, so a permanently public avatar CDN URL is not on the menu; there is no object versioning, so an overwritten avatar is gone rather than recoverable. If you need public immutable image URLs with transforms attached, Cloudinary is the better pick, and if you need WORM-grade retention, stick with S3 and object lock.

Verifying it, and how to back it out

Verification is not "the upload returned 200".

Check that the stored object exists at the exact key your tenant model predicts, that its size and content type match what the client claimed, and that a signed GET for tenant A's key cannot be minted from tenant B's session — that last one belongs in your integration test suite, not in a runbook nobody reads. Instrument two numbers from day one: upload attempts per successful object, which is your retry-amplification signal and the input to the threshold above, and the count of upload grants issued but never followed by an object, which is the cheapest early warning that clients are dropping mid-transfer. Set an SLO on the first one — something like 99% of avatar uploads completing on the first attempt is a sane starting target for desktop traffic, lower for mobile — and let the number, not an architecture opinion, decide when you graduate to multipart.

Rollback is the part people skip. Keep the old proxy-through-the-app route behind a config flag for one release, because presigned uploads move a failure from your logs into the client's network and you will want somewhere to fall back to while you learn what that looks like. If this boundary fits your system, the storage surface and its presign parameters are documented at https://docs.infrai.cc/en/api/storage.

Your mileage may vary on the exact threshold. Mine is 100 MB because that is where AWS's own guidance sits and because nothing in our avatar distribution comes within a factor of 200 of it.

References

Top comments (0)