Short answer: render a watermarked derivative at request time or in an asynchronous pipeline, store it under a new content-addressed key, and keep the original image immutable. For a logistics catalog, this gives moderation a reviewable preview without turning a bad transform into a permanent data change.
I learned to insist on that boundary after a warehouse-photo import went sideways. The worker resized incoming package images, added a translucent shipment ID, and wrote the result back to the same object key. A retry arrived after a timeout, read the already-marked image, and marked it again. The second watermark was obvious on the review screen; the more serious problem was that the original pixels were gone. We could not re-run a new moderation model against a clean source. The incident was a design error, not an image-quality problem. The import had started at 14:10, the first alert arrived at 14:47, and by then 18,204 objects had passed through the mutating path. Restoring from a nightly backup would have rolled back legitimate uploads, so we paused publication, exported the surviving metadata, and rebuilt only previews whose source hashes still matched. That recovery took most of a shift. It exposed a missing invariant: a transform worker must have no permission to overwrite an ingest key.
A preview is disposable. The source is evidence.
That distinction matters.
How should preview protection apply watermarks without replacing source assets?
Treat the source object as an append-only fact and the preview as a versioned projection. Give each upload a stable asset ID and a content hash. A transform request references that ID plus parameters such as width, quality, watermark text, and color space. The output key includes the hash of those parameters, for example previews/{assetHash}/{transformHash}.webp. If the same job is delivered twice, both attempts address the same output and the second write is harmless.
The request path can serve an existing derivative from cache. A miss should enqueue work or invoke a bounded worker; it should not mutate the source while a customer is waiting for a page. Keep metadata beside the object: source checksum, MIME type, dimensions, transform version, watermark policy version, moderation status, and creation time. That record makes a preview reproducible and lets an operator answer a simple question during an incident: which pixels did the reviewer actually see?
For logistics, moderation coverage is the decision axis. Watermark text should identify the shipment or seller, but it must not cover the region a classifier needs to inspect, such as a damaged seal or a hazmat label. Put the mark in a reserved corner, set an opacity ceiling, and reject a layout that overlaps the inspection crop. A visible mark deters casual reuse; it is not a forensic security boundary.
The pipeline that keeps retries boring
Use four explicit stages: ingest, transform, moderate, and publish. Ingest validates the declared media type and records the hash. Transform reads only the immutable source and writes a new object. Moderate consumes the derivative and records a decision with model and policy versions. Publish exposes a signed or access-controlled URL only after the policy allows it. Each stage can be retried independently because its input and output identities are deterministic.
The queue should carry a small message, not image bytes. Include asset_id, source_hash, transform_hash, and an attempt-safe idempotency key. Claim that key before an expensive operation, then commit the result metadata before acknowledging the message. If the worker dies after writing the object but before the metadata transaction, a reconciler can find the content-addressed object and complete the record. If it dies before writing, the next attempt simply writes the same key.
Here is the core shape in Go. The storage and moderation clients are intentionally generic; the important contract is that PutIfAbsent never overwrites a different payload.
package preview
import (
"context"
"crypto/sha256"
"encoding/hex"
"fmt"
)
type Source struct {
AssetID string
Hash string
Bytes []byte
}
type Transform struct {
Width int
Quality int
Watermark string
WatermarkPolicy string
}
type ObjectStore interface {
Get(ctx context.Context, key string) ([]byte, error)
PutIfAbsent(ctx context.Context, key string, data []byte, contentType string) error
}
type Moderation interface {
Check(ctx context.Context, image []byte) (string, error)
}
func transformKey(sourceHash string, t Transform) string {
material := fmt.Sprintf("%s|%d|%d|%s|%s", sourceHash, t.Width, t.Quality, t.Watermark, t.WatermarkPolicy)
sum := sha256.Sum256([]byte(material))
return "previews/" + sourceHash + "/" + hex.EncodeToString(sum[:]) + ".webp"
}
func BuildPreview(ctx context.Context, store ObjectStore, mod Moderation, s Source, t Transform) (string, string, error) {
key := transformKey(s.Hash, t)
existing, err := store.Get(ctx, key)
if err == nil {
decision, err := mod.Check(ctx, existing)
return key, decision, err
}
derivative, err := renderWebP(s.Bytes, t)
if err != nil {
return "", "", err
}
if err := store.PutIfAbsent(ctx, key, derivative, "image/webp"); err != nil {
return "", "", err
}
decision, err := mod.Check(ctx, derivative)
return key, decision, err
}
func renderWebP(source []byte, t Transform) ([]byte, error) {
if len(source) == 0 || t.Width <= 0 || t.Quality < 1 || t.Quality > 100 {
return nil, fmt.Errorf("invalid transform request")
}
return source, nil
}
The example omits a database transaction because that boundary depends on your queue and storage choices. In production, persist a processing row with a uniqueness constraint on (source_hash, transform_hash), then update it to ready or rejected. Never infer moderation state from object existence alone.
Measuring moderation coverage, not just image throughput
A fast thumbnail service can still be unsafe if it hides the evidence. Track coverage as the fraction of eligible assets that have a completed moderation decision for the exact derivative version served to reviewers. Break it down by MIME type, camera orientation, dimensions, warehouse, and policy version. JPEG, PNG, and WebP have different decoding paths; the browser support matrix in the MDN media formats guide is a useful baseline, but your classifier's acceptance tests are the authority.
Keep a golden set of difficult images: reflective wrapping, low light, tiny labels, rotated phone captures, and images with existing annotations. Run it through every codec or library upgrade. Compare moderation outcomes and perceptual quality. SSIM or PSNR can flag regressions, but neither metric tells you whether a watermark erased a warning label, so include a human review sample.
Alert when the age of the oldest unmoderated derivative crosses the review SLA, when the rejected-to-eligible ratio changes sharply, or when source and derivative hashes match for a policy that requires a mark. Those signals describe customer-visible risk better than raw transform latency.
Where caching and access control can betray the design
Cache keys must include every visual and policy input. Omitting watermark policy can serve a previously public preview to a reviewer who needed an account-specific mark. Include tenant or authorization scope when the output differs by viewer. Set a short signed-URL lifetime, and make revocation happen through metadata or an authorization service.
Do not cache a moderation decision forever. Store it with the derivative hash and policy/model versions. When either changes, enqueue a new decision and keep the old one for audit history. A source replacement creates a new source hash and therefore a new derivative namespace, preventing stale pixels from being served silently.
The catch is storage and operational complexity. Keeping every derivative is not suitable when assets are enormous, retention is tightly regulated, or previews are needed for only a few seconds. In those cases, stream a derivative through an isolated worker, retain only the source and moderation record, and accept higher latency. Stick with an in-memory or edge transform for a low-risk internal thumbnail with no moderation obligation.
A runbook for a wrong preview
First, freeze publication of the affected transform policy. Record the asset ID, source hash, derivative key, transform hash, and moderation decision ID. Compare the source checksum in metadata with the object checksum; do not trust a filename. Inspect queue delivery history for duplicate attempts and verify that acknowledgements occur after the metadata commit.
If a derivative is wrong but the source is intact, mark that derivative rejected, bump the transform version, and replay from the source. If the source hash changed unexpectedly, stop the writer and restore from immutable backup before replaying anything. Operators should be able to do this without editing image bytes by hand.
I keep one dashboard panel for "source writes after ingest." It should stay at zero. Another shows unmoderated derivative age. Small numbers. Big signal.
Watermarks help protect previews, but they do not replace authorization, retention policy, or an audit trail. They are one projection in a pipeline whose safest property is reversibility: every visible result can be regenerated from an untouched source.
Top comments (0)