DEV Community

rasmusberg6592
rasmusberg6592

Posted on

Private files with a delete-by date: signed download links in multi-tenant object storage

#storage #security #multitenancy #go

A countersigned merchant agreement on an e-commerce platform arrives with a deletion deadline attached: keep it while the account trades, destroy it a fixed number of days after termination. That deadline, and not the signing algorithm, is what should shape the layout. Use one private object storage namespace per data class, put the tenant boundary in the credential that mints the link instead of in request-handling code, keep every temporary download URL scoped to a single key with a short life, and store the deadline on the object itself so deletion never depends on an application remembering to run. AI-generated product images belong in a second namespace with a cheaper policy, because they are regenerable and the agreements are not.

Two data classes, two lifetimes, one access path. That framing survives contact with an audit; a single bucket holding everything does not.

Three isolation options, and what actually enforces the tenant line

The question I keep asking on design review is not "how do we sign a link" but "what has to be wrong for tenant A to read tenant B's contract". If the answer is a template string or a route parameter, the boundary is one careless refactor from being gone, and you'll find out from a support ticket rather than from a metric.

Prefix-scoped credentials move that boundary into the storage layer, where it is enforced by policy evaluation instead of by whichever engineer last touched the handler. The signer holds a per-tenant key and a policy that can only address tenants/<id>/, so a bug that constructs the wrong key produces a denial rather than a leak. Bucket-per-tenant goes further and is genuinely attractive at low tenant counts, but the capacity plan turns ugly fast: providers cap buckets per account, they cap lifecycle rules per bucket, and every new merchant becomes a control-plane write that can fail halfway. On a storefront platform that onboards merchants continuously, that is a queue, a retry policy and an on-call runbook you did not previously own.

Layout What enforces isolation Blast radius of one bad key Per-tenant deletion deadline Ops load
Shared bucket, tenant prefix, checks in app code Application logic Every tenant Sweeper query over a database Low, until the first leak
Shared bucket, prefix-scoped credentials Storage-side policy One tenant Lifecycle rule per prefix Medium: policy per tenant
Bucket per tenant Account-level policy One tenant Lifecycle rule per bucket High: bucket and rule caps

I'd take the middle row for anything that looks like a marketplace, and I'd revisit it if a single enterprise tenant showed up with a contractual demand for a dedicated key and its own deletion schedule.

How long should a temporary signed download link for private files live?

Long enough for the slowest legitimate download, plus clock skew, and not one minute more.

Signed links are bearer tokens wearing a URL costume. They land in browser history, in proxy logs, in the Referer header of any page the browser navigates to next, in the screenshot a merchant pastes into a support thread, and in whatever CDN sits between you and the object. None of that is exotic — it's the normal life of a string that grants access to a private file, and it's the reason a five-minute link and a five-day link are different risk products even though the code that creates them is identical. For an interactive download in a merchant dashboard, five to fifteen minutes is the range I argue for; for a server-to-server fetch that starts immediately, sixty seconds is plenty. Anything measured in days is a permalink with extra steps, and it should be built as one, with a revocable record behind it.

Clock skew is the failure that bites short TTLs. Expiry is compared against a clock, and if the host issuing links drifts away from the host validating them, a link can be born expired; monitor the offset on the signing hosts and treat a drift alarm as a paging condition, not a dashboard curiosity. Then check one behaviour in your provider's documentation before you promise anything to a security reviewer: whether the signature is validated at request start or continuously, because a large image bundle that begins transferring one second before expiry may well finish long after it.

Issuance itself is cheap. It's an HMAC over a canonical string, no network call, so the SLO is boring by construction — I'd hold p99 link-mint latency under 20 ms and alert on the error rate instead, since a spike there means a credential or configuration problem rather than a capacity problem.

Retention that survives a deleted tenant record

Here is the pattern that fails audits. The database row for a terminated merchant is marked deleted, the retention job iterates over live merchants, and the objects belonging to the terminated one are now unreachable by the very query that was supposed to clean them up. They sit there past the deadline, correctly private and completely non-compliant.

Storage-side lifecycle expiration avoids that whole class of orphan, because the rule is evaluated against the bucket rather than against your notion of who exists. Tag the object at write time with its class and its delete-after date, let the lifecycle rule act on the tag or the prefix, and keep the database as an index rather than as the enforcement mechanism. Two details are worth wiring in on day one: if versioning is on, an ordinary delete leaves a noncurrent version behind and you need the matching noncurrent-version expiration rule, otherwise "deleted" means "invisible in the console"; and backups are copies with their own clock, so a legal hold that outlives the deadline needs an explicit owner and an expiry of its own.

AI-generated images get the easy end of this. They're derivative, they can be regenerated from the prompt and the product record, and a short expiration on the rendered variants is a cost lever rather than a compliance control.

Signing code for a link that carries its own deadline

The signer is small on purpose. One tenant, one key, one deadline, one key version — anything outside the canonical string is unprotected and can be tampered with, which is the single most common way a homegrown scheme goes wrong.

package files

import (
    "crypto/hmac"
    "crypto/sha256"
    "encoding/base64"
    "fmt"
    "net/url"
    "strconv"
    "strings"
    "time"
)

// Grant is a scoped, expiring download: one tenant, one object key, one deadline.
type Grant struct {
    Tenant  string
    Key     string // already prefixed with the tenant namespace
    Expires time.Time
    KeyID   string // signing key version, so one tenant can be revoked alone
}

func canonical(g Grant) string {
    return strings.Join([]string{
        g.Tenant, g.Key, strconv.FormatInt(g.Expires.Unix(), 10), g.KeyID,
    }, "\n")
}

func Sign(g Grant, secret []byte) string {
    mac := hmac.New(sha256.New, secret)
    mac.Write([]byte(canonical(g)))
    q := url.Values{
        "tenant":  {g.Tenant},
        "expires": {strconv.FormatInt(g.Expires.Unix(), 10)},
        "kid":     {g.KeyID},
        "sig":     {base64.RawURLEncoding.EncodeToString(mac.Sum(nil))},
    }
    return fmt.Sprintf("https://files.example.com/%s?%s", g.Key, q.Encode())
}

// Verify runs on the edge in front of the private namespace: deadline first,
// then a constant-time comparison. Skew covers clock drift between hosts.
func Verify(g Grant, sig string, secret []byte, skew time.Duration) error {
    if time.Now().After(g.Expires.Add(skew)) {
        return fmt.Errorf("grant for %s expired at %s", g.Tenant,
            g.Expires.UTC().Format(time.RFC3339))
    }
    mac := hmac.New(sha256.New, secret)
    mac.Write([]byte(canonical(g)))
    want := base64.RawURLEncoding.EncodeToString(mac.Sum(nil))
    if !hmac.Equal([]byte(want), []byte(sig)) {
        return fmt.Errorf("signature rejected for tenant %s", g.Tenant)
    }
    return nil
}
Enter fullscreen mode Exit fullscreen mode

That is the whole mechanism, and it ports directly: the Node.js storefront in front of this calls crypto.createHmac('sha256', secret).update(canonical).digest('base64url') over the exact same canonical string, which matters because the two services must agree byte for byte or every link fails closed. Providers that offer their own presigned URL scheme give you the same shape with the signature moved into their query parameters; the reason to own the canonical string here is the kid field, which no provider scheme will give you.

Test the expiry, then practice the revoke

A retention policy you have not tested is a plan, not a control. Run a synthetic auditor that samples objects whose deadline has passed and asserts they are gone, and run it on the same schedule as the lifecycle rules rather than once at launch.

// audit.go, same package as the signer above.
package files

import (
    "context"
    "net/http"
    "time"
)

// AuditExpired samples objects whose deadline has passed and asserts they are gone.
// It never deletes anything: a survivor is a page, not a cleanup task.
func AuditExpired(ctx context.Context, c *http.Client, base string, sample []Grant) ([]string, error) {
    var survivors []string
    for _, g := range sample {
        if time.Now().Before(g.Expires) {
            continue
        }
        req, err := http.NewRequestWithContext(ctx, http.MethodHead, base+"/"+g.Key, nil)
        if err != nil {
            return survivors, err
        }
        res, err := c.Do(req)
        if err != nil {
            return survivors, err
        }
        res.Body.Close()
        if res.StatusCode != http.StatusNotFound {
            survivors = append(survivors, g.Key)
        }
    }
    return survivors, nil
}
Enter fullscreen mode Exit fullscreen mode

Pair it with a negative test that signs a grant with tenant A's key for an object under tenant B's prefix and asserts the edge refuses it. That test is the only evidence that the isolation model still holds after six months of refactoring.

Rollback is where the kid field earns its place. Keep two key versions live so routine rotation is uneventful, and when a tenant's links are suspected of having leaked, drop the old version for that tenant only: every outstanding URL they issued dies within seconds, nobody else notices, and the tenant re-mints links on their next request. For the deletion side, run the sweeper in dry-run for one full cycle and diff what it would have removed against what the lifecycle rules already handled — deletion is the one operation with no undo, so the rehearsal is not optional.

The catch is that signed links buy you throughput by taking the request off your servers, which means you lose the per-download audit trail that a proxying endpoint gives you for free. Storage access logs are delayed and coarse. If a regulator wants to know which named user opened a specific agreement and when, stick with a download endpoint that streams the object through your service and writes an audit record synchronously; the same applies when a document has to be watermarked per viewer, or revoked mid-download. Signed links don't support any of that, and pretending otherwise is how a compliance requirement becomes an incident. For everything else on a merchant platform — product imagery, generated renders, invoices, contracts a merchant downloads for their own records — the short-lived, tenant-scoped link is the cheaper and safer default. I'm less certain about the right TTL for mobile clients on poor connections; that number should come from your own transfer duration histogram rather than from an article.

Sources

Top comments (0)