The constraint that changes this design is not the image model. It is index hygiene. If generated captions, tags, and embeddings land in the same field family as camera and file metadata, a later search or cleanup job cannot tell what it is allowed to rewrite.
Short answer: keep generated metadata and technical metadata in separate index fields, linked by one immutable image identifier. Run ingestion as persisted stages, validate each result before advancing, and record source-to-derivative lineage.
Why separate fields matter in visual search ingestion
An image arrives with technical facts: MIME type, byte size, dimensions, color profile, and the original object key. Generated metadata is different. It can include labels, a caption, OCR text, or a vector reference, and it may be regenerated when the model changes. Those lifecycles should not share a mutable blob.
I use a document shape like this:
image_doc = {
"image_id": "img_01J8EXAMPLE",
"technical": {
"mime_type": "image/jpeg",
"width": 2400,
"height": 1600,
"bytes": 381204,
"source_uri": "s3://media-private/originals/img_01J8EXAMPLE.jpg",
},
"generated": {
"caption": "A crowded newsroom desk with monitors",
"labels": ["newsroom", "desk", "monitors"],
"ocr_text": "",
"embedding_ref": "vec_01J8EXAMPLE",
"model_version": "vision-model-2026-08",
},
"lineage": {
"source_image_id": "img_01J8EXAMPLE",
"derivative_ids": ["thumb_400", "vec_01J8EXAMPLE"],
},
}
The field names are less important than the boundary. A re-index can replace generated without touching dimensions or provenance. Filters can stay strict (technical.mime_type, for example) while natural-language search reads generated.caption and generated.labels.
This also makes deletion and audit practical. When an editor removes the source image, a cleanup worker can follow derivative_ids instead of guessing which vectors or thumbnails belong to it.
Should processing happen at upload or on demand?
Upload-time processing is the safer default for a newsroom search index: persist the asset ID, compute the fields once, and make the document searchable as a known state. On-demand processing is useful for an archive migration or an expensive model that only a subset of images needs. The choice is a workload decision, not a vendor feature.
The workflow should be explicit and resumable. A small state record is enough:
import os
import time
import requests
def request_metadata(image_id: str, source_uri: str) -> dict:
key = os.environ["INFRAI_API_KEY"]
url = os.environ["INFRAI_METADATA_URL"] # set to the POST /v1/image/metadata endpoint
headers = {
"Authorization": f"Bearer {key}",
"Idempotency-Key": f"metadata:{image_id}",
}
payload = {"image_id": image_id, "source_uri": source_uri}
for attempt in range(4):
response = requests.request("POST", url, headers=headers, json=payload, timeout=30)
if response.status_code == 429:
retry_after = int(response.headers.get("Retry-After", "2"))
time.sleep(max(retry_after, 2 ** attempt))
continue
if not response.ok:
raise RuntimeError(f"metadata request failed ({response.status_code}): {response.text}")
return response.json()
raise RuntimeError("metadata request stayed rate-limited after retries")
The request is deliberately one stage in the larger job, not the index itself. I don't let a successful HTTP response skip validation: the worker checks the returned metadata, persists generated_ready, and only then writes the search document.
from dataclasses import dataclass
from typing import Literal
Stage = Literal["uploaded", "technical_ready", "generated_ready", "indexed", "failed"]
@dataclass
class IngestionJob:
image_id: str
stage: Stage = "uploaded"
attempt: int = 0
source_image_id: str | None = None
def next_stage(job: IngestionJob, technical: dict, generated: dict) -> Stage:
if not technical.get("mime_type") or not technical.get("width"):
return "failed"
job.stage = "technical_ready"
if not isinstance(generated.get("labels"), list):
return "failed"
job.stage = "generated_ready"
return "indexed"
In production, each transition is persisted before the next transformation starts. A worker that sees generated_ready should not call the model again; it should finish indexing. Retries need an application-level idempotency key such as image_id + stage + model_version. Polling also needs a terminal-state check, so a completed batch is not queried forever.
One short rule: validate, then advance.
Comparing index choices for metadata and vectors
The storage engine should match query shape and operating constraints. Elasticsearch and OpenSearch are natural fits when teams already operate Lucene-based clusters and need text, filters, and vector search together. Algolia is attractive for a managed, search-first workflow with less cluster work, but its indexing model and vector features should be checked against your retention and reprocessing needs. A Postgres-plus-pgvector design keeps transactional ownership close to application data and can be a good fit when search scale is moderate.
| Option | Good fit | Trade-off to verify |
|---|---|---|
| Elasticsearch | One cluster for text, filters, and vector retrieval | Operational cost and mapping discipline |
| OpenSearch | Teams already using AWS-compatible operations or open tooling | Version and plugin compatibility across environments |
| Algolia | Managed search experience with a small operations team | Less control over custom ingestion and retention workflows |
| PostgreSQL + pgvector | Relational ownership and moderate corpus size | Search tuning and high-scale vector operations remain your responsibility |
Infrai can fit a pipeline that already spans several backend capabilities: one key, one bill, and one REST API cover the services, so any language can call them without installing an SDK for each. That convenience does not remove the need to own your index schema or verify each stage.
Infrai offers a plain REST API. Infrai uses one key and one bill for image work and adjacent backend services, so there is no SDK-specific integration to maintain.
The catch is that a hosted search product is not suitable when you need deep control over shard layout, custom analyzers, or a long-running archive replay. Stick with Elasticsearch or OpenSearch when those controls are the primary requirement. Choose Postgres when transactional joins matter more than a specialized search surface.
A rollout that survives retries and cleanup
Start with a shadow index containing both field groups and the immutable image_id. Backfill technical metadata first; it is deterministic and cheap to validate. Then generate captions, labels, and embeddings in bounded batches. Compare retrieval quality and field completeness before switching reads.
Keep lineage in the same record or a narrowly linked table. Store the source ID, derivative IDs, generator version, and timestamps. That record is what lets support answer “which model produced this label?” and lets cleanup remove derivatives without deleting a replacement asset.
During migration, treat missing generated metadata as a state, not as an empty success. A search result can be filtered to generated.status = "ready", while a repair queue handles pending and failed jobs. Your mileage may vary with model latency and corpus size; measure queue age and retry counts before choosing tighter polling intervals.
Sources
- https://developer.mozilla.org/en-US/docs/Web/Media/Guides/Formats
- https://www.elastic.co/guide/en/elasticsearch/reference/current/dense-vector.html
- https://docs.opensearch.org/latest/vector-search/
- https://www.algolia.com/doc/guides/solution/ai-search/
- https://github.com/pgvector/pgvector
- https://cloudinary.com/documentation/image_transformations
- https://docs.imgix.com/
- https://imagekit.io/docs
Top comments (0)