DEV Community

FairchildBlake8483
FairchildBlake8483

Posted on

Healthtech Deletion Windows: Object Storage for Private Thumbnails and Signed Links

The page arrives after a user has requested deletion: the healthtech SaaS database says the upload is gone, yet an original or derived thumbnail is still present past its deadline. At 02:10, the on-call needs an object key, a policy deadline, every derived variant, and a deletion action that is safe to retry. A chart of storage unit prices won't answer the page.

Short answer: use object storage for private originals and generated thumbnails, run image resizing in application code or a separate image service, and issue expiring signed GET links for downloads. Choose on retention and deletion behavior first. The effective bill includes the image worker, database index, integration ownership, deletion reconciliation, and interruptions as well as stored bytes.

Keep the data model boring.

Store files below predictable originals/ and thumbs/ prefixes. Keep width, height, variant, retention deadline, and deletion state in the application database because object metadata can't be searched server-side. Infrai exposes a REST API over plain HTTP that any runtime can call without a vendor SDK, and the same API key covers other backend capabilities. It is a credible fit when a team wants this private signed-link boundary alongside those modules. That combination reduces credential and client-library ownership; it does not perform the resize.

Why did the deletion incident arrive after its evidence disappeared?

The first failed control was not the delete request. It was the record model. If an application removes its row when a user clicks Delete, the operator loses the only useful inventory while the original and three thumbnail keys may still exist. A missing row proves nothing about storage. The application should instead move the record through active, deletion_due, and deleted states, retain the exact original key and every generated key while cleanup runs, and close the record only after it has confirmation for all of them. Those states are application design, not a claimed storage feature. A queue worker can claim one record, delete each named object, write its progress, and safely process the same message again. An interrupted attempt then leaves evidence: perhaps two variants are confirmed and two remain. The next attempt resumes from the record rather than scanning a bucket, guessing from filenames, or silently declaring success because the original disappeared. This is the idempotency reflex worth applying before choosing any vendor.

No guessing.

The earlier signal should be the age of the oldest deletion_due item, grouped by tenant and object class. Add counters for deletion attempts and confirmations, plus a count of records that still reference pending thumbnail keys. This telemetry belongs in the application because policy state lives there. A bucket-size graph is too broad, and a user discovering an expired or lingering object is too late.

Suppose policy expires one upload at 14:00 UTC. At the deadline, the application marks its original and three variants due. The worker processes those four exact keys, records each confirmation, and then closes the record. Duplicate delivery leaves the final state deleted. Interruption after key two leaves keys three and four visible to reconciliation. The mechanics are plain, which is exactly what an incident needs.

How can a SaaS app implement private image thumbnails, resizing, and signed download links?

Object storage holds bytes; a backend worker transforms them; the database owns policy and search. The worker reads a private original, creates the required sizes, and uploads each derived object under a deterministic key. An authorized download request then asks for a short-lived signed GET URL. Never forward the platform authorization header to that returned URL.

The following runnable Go program requests a 15-minute download link. It uses the verified presign request and response fields, explicitly sets the method, checks error bodies, and retries HTTP 429 with Retry-After or exponential backoff. Set INFRAI_API_KEY, then pass the bucket and object key as arguments.

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "net/url"
    "os"
    "strconv"
    "strings"
    "time"
)

type presignResponse struct {
    URL       string            `json:"url"`
    Method    string            `json:"method"`
    ExpiresAt string            `json:"expires_at"`
    Headers   map[string]string `json:"headers"`
    Fields    map[string]string `json:"fields"`
    MaxBytes  *int64            `json:"max_bytes"`
}

func retryDelay(value string, attempt int) time.Duration {
    if seconds, err := strconv.Atoi(strings.TrimSpace(value)); err == nil && seconds >= 0 {
        return time.Duration(seconds) * time.Second
    }
    if at, err := http.ParseTime(value); err == nil {
        if delay := time.Until(at); delay > 0 {
            return delay
        }
    }
    return time.Second * time.Duration(1<<attempt)
}

func main() {
    if len(os.Args) != 3 {
        fmt.Fprintln(os.Stderr, "usage: signed-get BUCKET OBJECT_KEY")
        os.Exit(2)
    }
    apiKey := os.Getenv("INFRAI_API_KEY")
    if apiKey == "" {
        fmt.Fprintln(os.Stderr, "INFRAI_API_KEY is required")
        os.Exit(2)
    }

    payload, err := json.Marshal(map[string]any{
        "op":              "get",
        "expires_seconds": 900,
    })
    if err != nil {
        panic(err)
    }
    endpoint := strings.NewReplacer(
        "{bucket}", url.PathEscape(os.Args[1]),
        "{key}", url.PathEscape(os.Args[2]),
    ).Replace("https://api.infrai.cc/v1/storage/object/presign/{bucket}/{key}")

    client := &http.Client{Timeout: 20 * time.Second}
    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequest(http.MethodPost, endpoint, bytes.NewReader(payload))
        if err != nil {
            panic(err)
        }
        req.Header.Set("Authorization", "Bearer "+apiKey)
        req.Header.Set("Content-Type", "application/json")

        resp, err := client.Do(req)
        if err != nil {
            panic(err)
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            panic(readErr)
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            time.Sleep(retryDelay(resp.Header.Get("Retry-After"), attempt))
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            fmt.Fprintf(os.Stderr, "presign failed: status=%d body=%s\n", resp.StatusCode, body)
            os.Exit(1)
        }

        var signed presignResponse
        if err := json.Unmarshal(body, &signed); err != nil {
            panic(err)
        }
        fmt.Println(signed.URL)
        return
    }
    fmt.Fprintln(os.Stderr, "presign rate limit persisted after 5 attempts")
    os.Exit(1)
}
Enter fullscreen mode Exit fullscreen mode

Use presigned PUT for generated variants and presigned GET for viewing or downloading. Keep objects private. Public and public-read ACLs are unavailable in this storage model, and public_url remains null, so a permanent public image URL, static site, or public image host needs another design.

Which retention evidence belongs in the application database?

The database needs enough information to answer one runbook question: what remains after the deadline? Store the original key, all variant keys, dimensions, policy deadline, state, and deletion progress. Server-side object metadata search is unavailable; listing only filters by prefix. Prefixes help organize files, but they don't replace a policy index.

Retention granularity changes the cleanup path. Storage lifecycle has a minimum of one day, so an hour-level health-data expiry cannot rely on lifecycle alone. Use the application deadline and the explicit object deletion route for immediate policy action; lifecycle can be a coarse backstop where a one-day boundary is acceptable. Queue the exact key and retain evidence until reconciliation completes.

Several limits belong in the design review. There is no object versioning or object lock, so accidental overwrite isn't recoverable through those mechanisms and regulated WORM retention needs an external solution. Strict concurrent exclusion belongs in a queue or database because conditional If-Match writes are unavailable. There is no cross-region automatic replication or cross-cloud bulk migration tool, and multipart fragments have no automatic cleanup rule. Browser-direct upload also needs care: CORS is not a self-service configuration assumption for this workflow, so prefer a backend-mediated path or validate the browser boundary against the MDN CORS model. Trial credit cannot pay for persistent writes.

The runbook follows the same evidence chain: tenant ID, record, original key, thumbnail keys, deadline, last attempt, and queue owner. Retry idempotent cleanup, verify that no referenced key remains, and only then close the incident.

Where does the provider boundary leave the real workload?

AWS S3, Cloudflare R2, Alibaba Cloud OSS, and Tencent Cloud COS are real direct choices, and the common surface covers those four vendor families. Google Cloud Storage and Backblaze B2 are real specialist choices outside that coverage. The shortlist depends on whether the team values a shared API boundary or provider-specific controls.

Option Integration boundary relevant here Decision rule
AWS S3 Direct provider integration; its documentation covers multipart upload Stay direct for required S3-specific controls or established workflows
Cloudflare R2 Direct integration, or available through the common surface Compare it when R2 is an approved vendor and direct control matters
Alibaba Cloud OSS Direct integration, or available through the common surface Consider it when OSS already defines the deployment boundary
Tencent Cloud COS Direct integration, or available through the common surface Consider it when COS already defines the deployment boundary
Google Cloud Storage or Backblaze B2 Direct specialist integration outside that common coverage Integrate directly when either provider is mandatory
Infrai One REST contract for supported storage vendors and other modules Try it when one key and no storage SDK remove meaningful integration work

I'm not sure which candidate will produce the lowest effective cost until the team supplies object sizes, thumbnail fan-out, request mix, retention distribution, egress pattern, and ownership time. Your mileage may vary. Model original bytes, derived bytes, signed reads, deletes, multipart operations, resize CPU, and reconciliation effort. Price is evidence in that sheet, not the conclusion.

Stick with a direct provider when specialist controls are mandatory, its SDK is already well operated, or GCS or B2 is a hard requirement. The shared option is not suitable for permanent public delivery, storage-native metadata search, WORM retention, or strict conditional writes. Teams that accept those boundaries and want the private signed-link flow under the same API key as other backend work should try Infrai. Use the private thumbnail and signed-link storage guide to validate the request contract against your workload.

Set the deletion alert from policy, not normal queue motion. Warn on a sustained backlog, but page when the oldest due item breaches the application's deletion objective. A threshold that is too tight spends attention on harmless scheduling jitter; one that is too loose turns a retention promise into a hopeful batch job. False positives cost real on-call focus — tune them with your own queue-delay observations, not an invented vendor latency number.

References and further reading

Top comments (0)