The page says thumbnail delivery is missing its availability target. On-call sees successful uploads, but game artwork grids show empty slots at several viewport widths. A hosted image processing API and Sharp in your own service can both produce thumbnails; neither explains why a required derivative is missing. Was it never produced, stuck in a queue, or requested for the first time when a player opened the grid?
Short answer: generate the bounded, predictable sizes at upload when a missing thumbnail would break publication; reserve on-demand processing for rare dimensions only when the first-view latency budget can absorb it. A hosted image processing API shifts native dependency packaging and worker operation across a boundary. A service running Sharp or another native transformer keeps that boundary under your control. Neither choice removes the need to measure time until every required size is readable.
What should have paged before the empty grid?
An upload success counter is an input signal, not proof that a usable responsive set exists. For each accepted original, enumerate required output widths, format policy, and access scope; mark the set ready only after the required objects can be read. Track the oldest unready set age, the fraction of accepted uploads ready within the target window, and failed transformations by reason. Page on sustained user-visible readiness failure against the service's SLO; queue age and retry growth are earlier warning signals.
A burst alone is not a page.
The missing size is the symptom.
Separate the clocks. Upload-to-ready latency measures the asynchronous path, while request-to-first-byte and cache hit rate describe the on-demand path. An aggregate image response success rate can look healthy while one unpopular screen width repeatedly misses; dimension and source asset class belong in diagnostic labels, although individual asset IDs are too high-cardinality for routine metrics. Correlate an upload ID through accepted, queued, transformed, stored, and served events in sampled logs or traces.
Should a hosted image processing API or a Sharp service handle uploads?
A service running native transformations inherits binary compatibility, memory limits, image-decoder updates, worker restarts, and releases for that dependency. A hosted API moves much of that maintenance to another operator, but introduces an external availability dependency, network latency, data-transfer boundaries, and contract drift. Its limitation is that restrictive data-residency rules may prohibit sending original artwork out of the team's trust boundary. An in-service transformer has the opposite limitation: it is a poor fit when the team cannot maintain decoder updates and worker isolation on-call. Preserve the original separately in either arrangement, and make derivative writes idempotent so retries cannot create an ambiguous ready state.
| Decision | Process at upload | Process on demand |
|---|---|---|
| Player experience | Readiness delay before publication; predictable reads afterward | Fast publication; first request may pay transform latency |
| Capacity planning | Size workers for upload bursts and backlog drain | Size transforms for cache misses and traffic spikes |
| Failure isolation | Failed jobs can block publication of required sizes | Transform outages surface on reads unless cached variants exist |
| Operational burden | Queue, retries, and derivative storage | Cache keys, stampede control, and miss-path SLO |
The buy-versus-build question cannot rest on an invoice comparison. A hosted boundary may reduce on-call work for binary distribution, yet the team still owns the readiness contract, source validation, observability, and fallback behavior. Running the transformer internally makes deployment reproducibility and resource isolation your responsibility; it may also keep image bytes within an existing trust boundary. Check where originals are allowed to travel before comparing implementations.
How do you trace the missing size back to its cause?
Start with one missing grid variant and its upload ID. Confirm that the original passed file-type validation and decoding; a filename extension is not a reliable description of bytes, and browser format support differs. Next check whether the width is in the required upload-time set. If so, inspect accepted-to-ready transitions and backlog age. If it is an on-demand size, check the cache key, simultaneous misses for the same key, and transformation latency. The same visible symptom can require two different pages.
Do not let each cache miss launch an unbounded decode. Bound concurrent work by memory as well as CPU, coalesce identical misses, and apply a deadline. A large source can consume substantial resources even when its output is small. Define the key from immutable source version, width, format, and transform policy. This Go example constructs a stable key; the actual transform still needs bounded execution and an atomic derivative write:
package thumbnail
import (
"crypto/sha256"
"encoding/hex"
"fmt"
)
func Key(sourceVersion string, width int, format, policyVersion string) (string, error) {
if sourceVersion == "" || width <= 0 || format == "" || policyVersion == "" {
return "", fmt.Errorf("invalid thumbnail key input")
}
sum := sha256.Sum256([]byte(fmt.Sprintf("%q|%d|%q|%q",
sourceVersion, width, format, policyVersion)))
return hex.EncodeToString(sum[:]), nil
}
Deploy a changed transform policy under a new key version, leaving previously served bytes stable until callers migrate. Exercise an actual decoder and encoder in the production container image, rather than merely verifying that the binary starts. Include malformed input, a high-resolution original, concurrent requests for one derivative, and a retry after a worker interruption. Verify the original remains intact and a derivative becomes visible only after its write completes.
One worker crash must not publish half a file.
When does the alert become noise?
A threshold that pages on every queue spike spends the on-call budget without protecting players. Compare backlog age with the remaining readiness window and expected drain rate; 100 queued jobs means little without job duration and worker capacity. Keep a warning for sustained growth, and page when the forecast breaches the user-facing readiness objective or when required derivatives already miss it. For on-demand variants, alert on sustained miss-path latency and errors where they affect actual reads. Review both thresholds after a burst test.
The false-positive cost is a needless wake-up and potentially rushed capacity changes when the pipeline would have drained in time. The false-negative cost is visible in the grid. Choose the upload-time set from viewport sizes the game actually ships, measure the long tail separately, and revisit the split when traffic or artwork dimensions change.
Top comments (0)