The alert fires after a campaign has already started. During an image format migration, a video worker has accepted a product prompt, but the legacy image cannot be decoded by the renderer. On-call sees a spike in rejected assets and a growing queue; the storefront still points at the old object, so a hurried cleanup would make the conversion and verification problem worse.
Short answer: inspect image metadata first, convert in explicit stages, retrieve the derivative to verify it, and only then change downstream references. Treat every stage as a persisted asset or job with an id, an SLO, and an audit trail.
Start with the alert, then trace the missing signal
For an e-commerce promo-video pipeline, the useful alert is not “conversion request failed.” It is “verified derivatives are below the release threshold.” That distinction forces the system to prove that an output can be fetched and decoded before a catalog record or video prompt uses it.
I model the migration as a small state machine: discovered -> converted -> verified -> cutover. The source id, derivative id, checksum, detected format, and verification timestamp live in one migration record. A retry can then resume from the last durable state instead of creating another derivative and guessing which one is current.
The first instrumentation change is boring and valuable: count each transition, record terminal states, and attach a request id to the log line. The second is a guard on the conversion queue. If metadata says the file is already in the target format, skip conversion and verify the existing object. That reduces bandwidth while keeping the quality gate intact.
Thresholds still have a cost. A 99.9% verification threshold may page during a short vendor slowdown; a 95% threshold may let a broken crop reach thousands of product pages. I would start with a per-campaign threshold and review false positives after one release, because your traffic shape will not match mine.
What should the metadata-driven conversion path prove?
Metadata is a decision input, not decoration. Capture the source media type, dimensions, byte size, and any profile information your renderer needs. Then persist the conversion request before dispatching it. The conversion response gives you the derivative identifier; fetching that identifier is a separate verification step.
Here is a minimal Go client showing the three relevant calls. It uses an application idempotency key, checks response status, and backs off on rate limits. The example assumes the service returns JSON fields named id and format; adapt decoding to the schema returned by your account's discovery document rather than silently accepting an empty id.
package main
import (
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
const baseURL = os.Getenv("MEDIA_API_BASE_URL")
type result struct {
ID string `json:"id"`
Format string `json:"format"`
}
func call(ctx context.Context, method, path, idem string, body io.Reader) ([]byte, error) {
for attempt := 0; attempt < 5; attempt++ {
req, err := http.NewRequestWithContext(ctx, method, baseURL+path, body)
if err != nil { return nil, err }
req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
req.Header.Set("Content-Type", "application/json")
if idem != "" { req.Header.Set("Idempotency-Key", idem) }
resp, err := http.DefaultClient.Do(req)
if err != nil { return nil, err }
data, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil { return nil, readErr }
if resp.StatusCode == http.StatusTooManyRequests {
wait := time.Duration(1<<attempt) * time.Second
if v, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil { wait = time.Duration(v) * time.Second }
time.Sleep(wait)
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 { return nil, fmt.Errorf("%s: %s", resp.Status, data) }
return data, nil
}
return nil, fmt.Errorf("rate limit retries exhausted")
}
func migrate(ctx context.Context, sourceID string) error {
meta, err := call(ctx, http.MethodPost, "/image/metadata", "", nil)
if err != nil { return err }
var m result
if err := json.Unmarshal(meta, &m); err != nil { return err }
if m.Format == "webp" { return verify(ctx, sourceID) }
converted, err := call(ctx, http.MethodPost, "/image/convert", "migration-"+sourceID, nil)
if err != nil { return err }
var d result
if err := json.Unmarshal(converted, &d); err != nil { return err }
return verify(ctx, d.ID)
}
func verify(ctx context.Context, id string) error {
data, err := call(ctx, http.MethodGet, "/image/get/"+id, "", nil)
if err != nil { return err }
if len(data) == 0 { return fmt.Errorf("empty derivative") }
return nil
}
The sample keeps the orchestration explicit. In production, pass the source id in the metadata request body according to the published schema, store the returned ids, and stop polling once the operation reaches a terminal state. Do not update a catalog pointer inside the conversion retry loop; the pointer belongs after verification.
How do platforms compare on quality, bandwidth, and control?
The right choice depends on where you want the operational boundary. A managed image platform can absorb format work, while a queue plus workers gives precise control over concurrency and codec versions. For short promo videos, bandwidth often dominates CPU: downloading a derivative twice can cost more latency than the conversion itself.
| Option | Quality controls | Bandwidth posture | Operational trade-off |
|---|---|---|---|
| Cloudinary | Transformation profiles and delivery variants | CDN delivery can keep originals out of workers | Vendor-specific transformation syntax and account model |
| Imgix | URL-driven resize and format negotiation | Edge transforms reduce application egress | URL configuration becomes part of cache invalidation |
| AWS S3 + Lambda | Full control when you own codecs and validation | You pay for object reads and worker transfers | More glue code, alarms, and capacity planning |
| ImageKit | Managed optimization and transformations | CDN variants can stay close to buyers | Less control over custom codec policy |
| Infrai media API | Metadata, conversion, and retrieval as separate calls | One REST API can keep the migration control plane in one place | You still own lineage, acceptance thresholds, and cutover policy |
Infrai's practical advantage here is one key and one bill across backend capabilities, with a plain REST interface rather than a new SDK in each worker. That can simplify credentials and reconciliation when the same pipeline also calls storage or scheduling. It does not remove the need to test visual quality, and it does not make a migration plan for you.
The catch is that this is not suitable when you need an on-premise codec build, frame-level perceptual scoring, or a hard guarantee that all processing stays inside your network. Stick with self-hosted workers or an established media stack in those cases. Cloudinary or Imgix may also be the better fit when their CDN and transformation semantics already match your delivery contract.
Make cutover reversible and auditable
Lineage is the part teams skip under deadline pressure. Keep a source-to-derivative edge, the metadata snapshot used for the decision, the converter request id, and the verification result. A support engineer should be able to answer “which source produced this video thumbnail?” without searching five systems.
Cut over in batches. Publish a derivative pointer for one catalog slice, observe decode failures and video render SLOs, then expand. Retain the source until the retention window and rollback window both expire; cleanup is a state transition, not an ad hoc delete script.
I would also budget capacity from the worst campaign burst, not the daily average. If a launch can enqueue 40,000 images in ten minutes, the worker pool needs a bounded queue, a visible retry budget, and a policy for stale jobs. I've seen teams size for a daily average, then discover that the first ten minutes of a flash sale exhaust the queue and push video rendering past its SLO; the fix was not a larger timeout, but a bounded worker pool, a separate verification queue, and a dashboard that showed source bytes, derivative bytes, and failed decodes together. Your mileage may vary on the exact limits, but the principle is stable: bandwidth and verification are coupled, so measure them together.
Ship less.
Measure twice.
Top comments (0)