DEV Community: Sahib Alizada

Visual Search on pgvector + Gemini: Detect, Crop, Embed, Rank (and the Averaging Bug That Broke Exact Match)

Sahib Alizada — Fri, 14 Aug 2026 14:15:40 +0000

Visual Search on pgvector + Gemini: Detect, Crop, Embed, Rank (and the Averaging Bug That Broke Exact Match)

I run a fashion marketplace in Azerbaijan (solo founder + AI pair programming). Users upload a photo — a screenshot from Instagram, a shop window, an outfit — and get matching products from our catalog.

Last week I gave it the cruelest test I could design: I uploaded a screenshot of a product's own catalog photo. The system had this exact image in its database.

It missed. Score: 0.9259. Exact-match threshold: 0.93. And a blue dress (0.8165) ranked above the identical orange one (0.9259).

Both failures turned out to be textbook embedding-system mistakes that I suspect half the "add vector search in a weekend" tutorials out there quietly share. Here's the full pipeline — Django, PostgreSQL + pgvector, Gemini (flash-lite gate + gemini-embedding-2) — with the real fixes.

Live demo first, if you prefer:

1. The gate: one small-model call that structures everything

Every upload hits a Gemini Flash-Lite class model once. Not "is this a product?" — that's a waste of a multimodal call. We ask for everything ranking will need, as JSON:

{
  "is_product": true,
  "brand": "",
  "items": [
    {
      "name_az": "Narıncı don", "name_ru": "Оранжевое платье",
      "color_az": "Narıncı",
      "category_public_id": "cat_...", "subcategory_public_id": "sub_...",
      "variant_item_public_ids": ["vri_..."],
      "bbox": [120, 260, 890, 640],
      "partial": false,
      "primary": true
    }
  ]
}

Three decisions that mattered:

The gate reads our catalog. We inject a ~3K-token digest of the live taxonomy (categories, subcategories, variant vocabulary — Redis-cached 12h) into the prompt. The model returns our IDs, not free text. No fuzzy string-matching layer between detection and ranking, and item labels come back in all 4 UI languages for free.
primary beats "largest bbox". The model marks the star of the photo. A model holding a white bag → the bag is primary, not the bigger blouse behind it.
The ≥90% visibility rule. A trouser leg entering the frame corner is not a searchable item. Mostly-cut-off detections get partial: true — shown as a label, never cropped, never searched. This single rule killed a whole class of garbage results.

Fail-open everywhere: if the gate call dies, search continues without hints. Cost with the catalog digest: ~$0.002/search.

2. Crop + embed — twice

The primary bbox is cropped (5% padding, skip if it covers >85% of the frame — already tight). Then the asymmetry problem: catalog vectors come from full product photos; a cropped query is a zoomed fragment. Same dress, drifted fingerprints.

Cheapest fix in this article — embed both, search both, best score wins per product:

crop = crop_item_bytes(image_bytes, primary["bbox"])   # None if bbox > 85% of frame
vec_crop = embed(crop or image_bytes)
vec_full = embed(image_bytes) if crop else None

candidates = knn(vec_crop, pool=40)
if vec_full:
    candidates = merge_max_by_product(candidates, knn(vec_full, pool=40))

gemini-embedding-2 is $0.00012/image. The second embedding is insurance that costs a hundredth of a cent and rescues every catalog-screenshot query.

3. The averaging bug (read this section if you skim everything else)

Original design, the one every tutorial teaches: product has N photos → embed each → store the mean as the product vector.

What the mean actually is: a point somewhat close to every view and identical to none. My test dress had front + back photos. Uploading the back photo produced a query sitting exactly on "back photo" — measuring 0.9259 against the stored midpoint. Threshold 0.93. An identical image, diluted below my own exact bar by my own storage scheme.

And it fails intermittently: 1-photo products match perfectly, 4-photo products miss constantly. Users read that as "flaky," which is worse than "broken."

The fix — stop discarding per-photo vectors:

class VisualImageEmbedding(models.Model):
    product = models.ForeignKey(Product, on_delete=models.CASCADE)
    color_combination = models.ForeignKey(ColorCombination, null=True, ...)
    source = models.CharField(...)      # main | gallery | cc
    image_ref = models.CharField(...)   # storage path = natural dedupe key
    embedding = VectorField(dimensions=1536)

    class Meta:
        constraints = [models.UniqueConstraint(
            fields=["product", "image_ref"], name="uniq_vie_product_image_ref")]

Every photo is a row. Query-time kNN runs over views; a product's score = its best-matching view (max-merge, same pattern as the colorway vectors). The mean still gets derived — free, since the per-view vectors are already in hand — because category centroids and the PDP "similar products" rail want a prototype. Retrieval doesn't.

Re-embedding the catalog: one background job, 906 products + 301 colorways, 0 failures, ≈ $0.25 total. The row upsert checks (product, image_ref, model_id) first, so re-runs reuse stored vectors instead of calling the API again.

Post-fix, same cruel test: 0.9743. Exact, rank 1.

Rule of thumb: mean embeddings are for classification prototypes. If a user can query with any single view of an entity, store per-view vectors and max-merge. The tidy average is where your exact matches go to die.

4. pgvector notes that saved us

Plain Cloud SQL Postgres + pgvector, HNSW indexes, same instance as the shop. No second datastore to operate.
hnsw.iterative_scan matters: with a status='Published' filter, vanilla HNSW under-fills (asks for 40, returns 7). Iterative scan re-probes until the limit fills. We wrap it in a context manager that does SET LOCAL inside a transaction — PgBouncer-safe.
Guarded migrations: the HNSW index is created inside RunPython with a connection.vendor == "postgresql" check, so SQLite dev/test environments don't explode.
Per-view rows tripled the row count and changed nothing about latency worth graphing — fetch pool*3 view-rows, merge to distinct products in Python.

5. Ranking: centroids, a trust gate, and one golden rule

Cosine similarity is category-blind — it will rank a manicure set next to a face cream because both are "small object on white background." So candidates pass a reasoning stage:

Image centroids for classification. Every category/subcategory/gender keeps a centroid = mean of its product-image embeddings (averages are right here — we want prototypes). The query classifies against ~70 centroids in one pass. Leave-one-out over 583 production products: 84.7% category, 95.9% gender — ~7 points over the neighbor-vote baseline. Fun fact: our first classifier compared the image against embedded category names. Image-to-text cosine was so flat (~0.67 for right AND wrong answers) it couldn't separate hoodies from lace pajamas. Compare images with images.

The centroid build is deliberately boring — a weekly job, pure averaging, no training loop:

def build_centroids(min_samples=3):
    sums, counts = {}, {}
    for row in ProductEmbedding.objects.filter(product__status="Published"):
        for key in keys_for(row.product):          # ("category", cat_id),
            acc = sums.setdefault(key, [0.0] * DIMS)   # ("subcategory", sub_id),
            for i, v in enumerate(row.embedding):      # ("gender", g) — unisex excluded
                acc[i] += float(v)
            counts[key] = counts.get(key, 0) + 1

    for key, total in sums.items():
        if counts[key] < min_samples:      # sparse label → no centroid → falls back to vote
            continue
        VisualCentroid.objects.update_or_create(
            kind=key[0], ref_id=key[1],
            defaults={"embedding": [v / counts[key] for v in total],
                      "sample_count": counts[key]},
        )

New category added by the content team? Next weekly run picks it up. Zero maintenance, zero drift meetings. Note the irony: the classifier is built from averages — the same operation that broke retrieval. Averages are prototypes; prototypes are exactly what classification wants. Context is everything.

Trust gate: the centroid (or the gate's taxonomy) only hard-filters when confident (score + top1−top2 margin) and the strongest visual match agrees. Otherwise: plain weighted majority vote of the top candidates.

The golden rule, written in blood twice:

# nothing above the exact threshold may EVER be filtered out
pool = [c for c in pool
        if c["category"] in dominant or c["score"] >= exact_threshold]

A bikini-set photo taught us this: the gate saw "bikini top" as primary, the vote locked onto dresses, and the identical set — top score in the pool — got voted off the main grid into a side section. Exact-tier candidates are now exempt from the category vote, the taxonomy scope, color reordering, minority demotion — everything. A 93%+ match outranks every committee.

Also in this stage: color-match-first ordering (crediting the product's own color metadata, not just colorway-photo matches — the second bug from the intro), minority-subcategory demotion to a capped tail, and honest empty results. When there's no exact match, the header names the dominant subcategory of what's actually shown ("closest Dresses") — computed from the final ranked list, not from an upstream guess that ranking may have overruled.

6. Observability or it didn't happen

One structured log line per search:

[VS-RANK] hint=clothing src=gate rag_sub=48(Dresses) sub_score=0.845
          margin=0.019 gender=women(0.83) mode=focused demoted=0
          | n=12 | 19/48:0.8165, 19/48:0.9259, ...

Every incident in this article was diagnosed from these lines in minutes, not hours. Two boring-but-vital details: log the max score as top_score (we briefly logged the post-sort first element and silently poisoned two weeks of threshold-calibration data), and never log dynamic-keyed dicts to a BigQuery sink — json.dumps them into a single STRING column, or the auto-schema will mint a new column per key and jam the pipeline.

7. Failure modes: the boring table that makes it production

Three model calls, a cache and a queue sit in this request path. Every row below is implemented, not aspirational — and two of them are postmortems:

Failure	Behavior	Mechanism
Gate call dies / times out	Search continues, no chips/hints	Fail-open — assistants are never load-bearing
Catalog digest unavailable	Gate degrades to basic questions	Digest appended in try/except; structured IDs are a bonus
Embedding API fails	Clean 503, credit NOT charged	`consume_credit()` runs only after a successful embed
Junk upload (no fashion item)	Polite 422, credit IS charged	Deliberate anti-abuse: free rejections make quotas decorative
Nothing ≥ similarity floor (0.78)	Honest empty state	Never pad the grid
New code, old schema (rolling deploy)	Invisible	Per-view kNN wrapped in fallback to legacy vectors
Redis outage	Searches run; rate limits fail open	Availability > strictness for a cache tier
Dynamic dict keys in analytics	BigQuery sink jammed (once)	Since then: `json.dumps` into ONE string column, always
`top_score` logged from `scores[0]`	Two weeks of calibration data poisoned (once)	`max(scores)`, and metrics name their sources
Any committee vs an exact match	Committee loses	The golden rule from §5, enforced in every filter

Why not CLIP / a vector DB / Google's managed visual search?

Asked constantly, so, honestly:

Vertex AI Search for Commerce: ~$2.50/1k queries ≈ 4-6× our all-in cost, plus catalog sync, plus a ranker that can't learn our exemption rule. No.
Vision Product Search: maintenance mode per Google's own docs. You don't build on sunset infrastructure.
Dedicated vector DB: at ~10³ products, pgvector latency is indistinguishable and marginal ops is zero. One database, operated honestly.
Self-hosted CLIP-family: GPU serving + a second embedding space divorced from our text search and chatbot (which share this one). The single shared space is what makes "this bag, but in brown" a future merge instead of a migration.

Buy undifferentiated capability; build what encodes your catalog's truth (the gate prompt, the centroids, the exemption rules).

8. The human loop

Every visual search lands in a moderator queue: confirm/correct the AI's category, subcategory, gender and variants (against the real variant vocabulary), and tick which shown products were actually relevant. Reviewed rows are exempt from the 90-day purge — each one is a permanent human-verified training pair. Threshold calibration reads this pool today; centroid enrichment and fine-tuning read it next. The moderation UI is a labeling shop wearing a moderation costume.

Numbers

906 products + 301 colorways re-embedded per-view for ≈$0.25, zero failures
~70 centroids (17 cat / 50 subcat / 3 gender), rebuilt weekly by a scheduled job
Cruel test before → after: 0.9259 → 0.9743 (threshold 0.93)
~$0.002/search (gate + 2 embeddings); vector search marginal cost ≈ $0
≤4 detected pieces per photo, ≥90% visibility to be searchable, 15-min session, piece-searches free
~40 admin knobs, hot-reloaded, zero deploys — and every incident retired at least one knob

Takeaways

Per-view vectors + max-merge for retrieval; averages only as prototypes.
Embed noisy queries twice (full + detected object). $0.0001 insurance.
Classify images against image centroids, never label text.
Exact-threshold candidates are exempt from every filter. Write the rule before the incident.
Fail open on assistant calls, fail honest on results.
Log ranking decisions (scope, scores, demotions), not just outcomes — and log max(scores), not scores[0].
Ship the human-review queue with the feature, not after it. It's your training-data factory.

Questions about any layer — the gate prompt shape, iterative_scan, the centroid build — happy to go deeper in comments.

Search-engine prequel (4 languages, 2 alphabets, CIEDE2000 color science): see the canonical Medium article. Try the live feature at geyin.az — the camera button in the search bar.

Building a Search Engine for 4 Languages, 2 Alphabets, and 107 Messy Color Names (Elasticsearch + pgvector + CIEDE2000)

Sahib Alizada — Mon, 03 Aug 2026 19:30:25 +0000

Real production queries from my fashion marketplace in Azerbaijan:

гара йупка      # "black skirt" — Azerbaijani words, Cyrillic script
pembe yubka     # "pink skirt"  — half Turkish, half Russian
roziviy platya  # "pink dress"  — Russian translit, with a typo

My catalog is written in Azerbaijani (qara ətək, çəhrayı don). Vanilla Elasticsearch returns nothing — or confidently wrong results — for all three. Here's the full architecture that fixes it, with the actual techniques and the failures along the way. Stack: Django, Elasticsearch 8, PostgreSQL + pgvector, Gemini embeddings. Team: one founder + AI pair programming.

Prefer watching it in action first? 90-second demo of the live engine:

1. Script folding + synonyms (the floor)

Everything starts with character folding and transliteration at analysis time: ə→e, ş→s, Cyrillic→Latin (гара→qara). On top: 103 curated synonym groups mapping street language, not dictionary language — one slippers group reads tərlik, terlik, тапочки, шлёпки, slippers, səndəl (five languages/registers → one catalog concept), and the suede group carries the slang spellings users actually type (zamuj, zamıj, zamsha → zamşa).

Bonus mechanism: the synonym keys themselves are fuzzy-matched (threshold 0.78, tunable). "yupka" isn't in any group — "yubka" is; the bridge connects the misspelling to the key, the key expands to the catalog term: yupka → yubka → ətək. Two hops, zero user friction. Raise the threshold to 0.82 and the bridge silently dies — measured knob, not a constant.

One analyzer discipline that saves you months of "why is this matching": edge_ngram at index time only. Index-time custom_analyzer builds front-prefixes (autocomplete recall); query-time uses a separate search_analyzer with NO ngram — otherwise every 2-letter fragment of the query becomes a prefix matcher and your results fill with garbage. We learned this from a prod bug where the fragment "el" matched gloves, dresses, and a shop description simultaneously.

2. Query → structured intent

Every query passes a token classifier that tests each token against the live taxonomy in all four languages at once, priority-ordered: gender → category → subcategory → color/size → price.

"pembe yubka"  →  { color_family: "cehrayi", category: "Skirts", text: "" }

Multi-word subcategories get a sliding-window phrase matcher with token-coverage scoring:

for size in (3, 2):
    for window in windows(tokens, size):
        for sub, field, name_tokens in candidates:   # stopwords stripped
            if all(any(similarity(w, nt) >= 0.9 for nt in name_tokens)
                   for w in window):
                coverage = size / len(name_tokens)
                if coverage >= 0.5:
                    accept(sub, window)

So "ətək dəsti" (skirt set) matches the subcategory "Ətək və Üst Geyim Dəsti" as a phrase — a single ambiguous token can never hijack the filter.

Naive matching also needs guards: without them, "ayaqqabısı" (a long word for shoes) fuzzy-matches the size "S". Length-ratio rules and a size-keyword list protect real size queries while blocking accidental ones. Price hints parse too — "50 manatadək" becomes {price_max: 50}, "ucuz" becomes a cheap-range filter.

End-to-end, one mixed-language line — qadın pembe yubka 50 manatadək (AZ + TR + RU-translit + AZ price) — parses to:

{ gender: "women",            # AZ column match
  color_families: ["pink"],   # TR alias
  category: "Skirts",         # RU word → synonym pool → ətək
  price_max: 50 }             # parsed, not matched

Zero tokens left as free text. Swap pembe for English pink — identical parse; aliases carry all four languages, so one sentence can mix them freely.

3. Color science: 107 vendor strings → 15 families

Vendors wrote 107 distinct color values, including pipes ("Qırmızı | Açıq çəhrayı") and literal CSS:

radial-gradient(circle, #000000 20%, #C69258 20%)   # a leopard print

The fix is not more string matching. It's color science:

HEX_RE = re.compile(r'#([0-9a-fA-F]{6}|[0-9a-fA-F]{3})\b')  # finds ALL hexes,
                                                             # even inside gradients

def hex_to_lab(hex_code):        # sRGB → linear → XYZ(D65) → Lab
    ...

def delta_e(lab1, lab2):         # full CIEDE2000 (Sharma formulation),
    ...                          # ~80 lines, zero dependencies

Every family has anchor hexes; every vendor color maps to its nearest anchor by CIEDE2000. We validated against the official Sharma test vectors — 9/9 to four decimals, including the notorious 1.5381 pair. Upgrading from naive CIE76 (Euclidean in Lab) to CIEDE2000 re-classified nine boundary colors correctly: mustard→yellow (was: gold), ivory→beige (was: white), navy stopped bleeding into purple.

And 15 is today's taxonomy, not a ceiling. Families live in an admin table — adding a 16th is a database row, not a deploy. More importantly, the classifier has no color limit at all: whatever new shade a vendor invents tomorrow, its hex lands in Lab space and maps to the nearest family automatically. As sellers grow, the color vocabulary grows without permission — and the organization absorbs it.

Rules on top of the math:

Pipe/dual colors → both families (a red|pink set must appear in pink searches)
≥4 hex-derived families → collapse to "multicolor"
Pattern aliases (leopard, zebra) → multicolor, hex families skipped

Products with no color variant at all? One-shot backfill: product photo → Gemini Flash-Lite → "dominant product colors as hex, ignore background and skin" → CIEDE2000 → family. 76 products classified for under $0.01 total.

One war story: the first version set max_output_tokens=256 on the Gemini call. An AI review agent flagged it before deploy: thinking tokens share that budget, so the model could burn the whole ceiling thinking and return empty text — the entire backfill would have silently produced nothing. Ceiling raised to 2048, truncation logged.

4. Filters that degrade instead of failing

Hard filters are how you serve empty pages. Every filtered search descends a ladder; the first rung with hits wins:

yield 'full',      search_with(category, color, text)
if color:
    yield 'no_color',  search_with(category, text)
if subcategory:
    yield 'partial',   search_with(text_plus_category_only)
yield 'text_only', search_with(text)

A wrong constraint costs ranking quality, never a blank screen.

Second forgiveness mechanism: the taxonomy bridge. "yubka" detects the Skirts category — but skirt sets live in another category whose subcategory is literally named "Skirt & Top Set." A hard category.id filter silently hid them. Now the filter is:

Q('bool', should=[
    Q('term', **{'category.id': detected_id}),
    Q('match', **{'subcategory.title': {          # concept bridge
        'query': specific_tokens_only,            # generic tokens stripped
        'analyzer': 'search_analyzer',
    }}),
], minimum_should_match=1)

Generic tokens (clothing, set, top…) are excluded so the bridge can't over-widen — if only generic tokens remain, no bridge is built. (Credit: an AI code-review agent caught the over-widening risk before it shipped.)

5. Semantic layer: one embedding space, two doors

Text and image embeddings live in a single shared 1536-dim space (Gemini Embedding 2, multimodal), stored in pgvector — durable across index rebuilds.
When lexical results are weak (below a threshold), we run text-kNN and image-kNN and fuse with Reciprocal Rank Fusion:

def rrf_merge(*ranked_lists, k=60, limit=None):
    scores = defaultdict(float)
    for lst in ranked_lists:
        for rank, pid in enumerate(lst):
            scores[pid] += 1.0 / (k + rank + 1)
    ...

The same space powers visual search (photo → products). One space, two doors.
The taxonomy is embedded too. Every category/subcategory/gender has a centroid — the mean vector of its products — in pgvector. Visual search classifies the query photo against centroids before ranking:

rows = (VisualCentroid.objects
        .filter(embedding__isnull=False)
        .annotate(distance=CosineDistance('embedding', query_vector))
        .values_list('kind', 'ref_id', 'distance'))
# → "this photo is a women's earring" → type-constrain results
# (an earring photo can never return trousers on color similarity alone)

Crucially, semantic only fires on weak lexical results. When exact match works, don't dilute it with vibes.

Two production footnotes for the vector crowd:

SET LOCAL hnsw.iterative_scan = relaxed_order;

pgvector's HNSW index silently under-returns when you post-filter a kNN query (e.g. status='Published') — the index yields K candidates, the filter eats most of them, and you get 3 results where 40 exist. Every filtered kNN of ours runs inside that session setting.

And freshness: every product save dispatches a Cloud Task that re-syncs its ES document within seconds — no nightly rebuild, no Celery workers, just HTTP-triggered tasks on Cloud Run. A vendor lists a dress; it's searchable before they close the tab.

6. One brain, three doors: the chatbot runs on the same engine

Our AI assistant has no product knowledge of its own. It calls the search engine through 7 authenticated tool endpoints (API-key via constant-time compare, plus Google-signed JWT for service accounts):

search_products            # same intent parser, same filters
search_shops               # same product-evidence ranking
search_faq / search_blog   # RAG corpus: FAQ + docs + blog chunks,
                           # same embedding space, same RRF fusion
search_products_by_vibe    # semantic-only door
find_similar_products      # embedding neighbors
get_catalog_info           # taxonomy/variants/brands (cached)

Visual search is the third door — same embeddings, same centroids, same ranking rules. Fix a synonym once; the search bar, the camera, and the chatbot all improve in the same second.

7. The data flywheel: why BigQuery is the whole point

Every meaningful interaction emits a structured JSON event — 18 event types, 15 of them routed by the Cloud Logging sink into BigQuery (60K+ events in the last 30 days). The interesting ones aren't pageviews:

log_smart_upload_save(
    ai_suggestions=...,      # what the AI proposed
    vendor_choices=...,      # what the human decided
    changes=...,             # field-level diff: accepted/modified/rejected
    acceptance_rate=...,     # one number per upload
)

That's a labeled training pair manufactured by normal platform usage. Same pattern everywhere:

SearchHistory stores query → clicked product (text AND visual; visual queries keep their 1536-dim embedding) → query-relevance pairs
Moderator rejections carry categories and feed a negative-example index; new uploads get a kNN pre-check against it before any Gemini call — a lookalike of a rejected image is flagged in milliseconds and skips the API cost entirely
A moderator review queue grades visual search results and corrects categories → hand-labeled ground truth
Zero-result queries land in a dedicated BigQuery view → weekly synonym curation

The staircase: (1) dashboards and mining today → (2) behavior-driven ranking (CTR adjusts weights, semantic blend ratio set by measurement) → (3) fine-tuning our own retrieval/classification models on suggestion-vs-decision pairs and click-relevance pairs. Nobody else has this dataset, because nobody else's users type "гара йупка".

8. The control plane

Every AI behavior is admin-tunable with zero deploys — 47 knobs across two singleton config models (semantic thresholds, RRF k, color-filter toggle, visual similarity tiers, gate model, rate limits, session TTLs). Config resolution: DB override → settings → defaults, cache-invalidated on admin save.

Thresholds aren't vibes. A zero-API-cost eval command runs leave-one-out over every product embedding, compares three classification methods (neighbor-vote / centroid / text-RAG argmax), and prints the threshold grid that reaches 95% precision:

score ∈ {0.60..0.85} × margin ∈ {0..0.05} → precision/coverage table
→ recommended gate = first cell with precision ≥ 0.95

The admin knob gets set to what the benchmark says.

Cost is part of the control plane too: the Gemini catalog context lives in a server-side context cache (~94% of prompt tokens per AI-upload call served from cache), vision inputs are resized before sending, and the rejected-image kNN pre-check skips Gemini calls entirely — each skipped call is logged with its saved cost.

Production debugging, not guessing

When "pembe" (pink) started returning beige and denim skirts in production, we didn't tweak boosts blindly. We traced the parsed intent on the live system:

FILTERS = {'variants': [{'title': 'Açıq Mavi | Krem | Çəhrayı',
                         'matched_field': 'title_tr', ...}],
           'color_families': ['cehrayi', 'bej', 'mavi'], ...}

There it is: "pembe" exact-matched a pipe item's Turkish translation and inherited all three of its families, diluting the filter. Fix: word-aliases take precedence over item inheritance — pink means pink. One trace, one line changed, covered by two new tests.

Numbers

4 languages, 2 alphabets, ~100 synonym groups
15 color families; 116/116 colors auto-classified; 76 products colored from photos for <$0.01
CIEDE2000: 9/9 official vectors
18 analytics event types → BigQuery; 7 AI tool endpoints; 47 zero-deploy admin knobs
366 automated tests on the search stack
Built and reviewed by 1 founder + AI agents (code review, ES review, migration safety, perf)

Where this goes

The layered architecture is deliberately a template: the Turkic-language world (Turkey, Central Asia) shares the same script-mixing and transliteration chaos as our home market — each new language is a new folding table + synonym pack on the same brain. Next signals on the roadmap: user behavior (already streaming), then body measurements and personal style (opt-in, privacy-first) as ranking inputs. Target state: the search bar behaves like a personal stylist that happens to accept text, photos, and chat.

Takeaways

Multilingual search dies at the alphabet layer first. Fold scripts before you tune ranking.
A color word is a filter, not a keyword. Parse intent.
Lab + CIEDE2000 is 80 dependency-free lines that outperform any hand-maintained color mapping.
Build a relaxation ladder. Wrong constraints should cost ranking, never results.
Zero-result logs are free product management.
One brain, many doors: search bar, camera, and chatbot should share the same intent parser, embeddings, and ranking.
Capture every AI-suggestion-vs-human-decision pair from day one. That's your fine-tuning dataset compounding while you sleep.

Questions about any layer — analyzers, the phrase matcher, the CIEDE2000 port, RRF tuning — ask below. I'll share details.

I'm building geyin.az (Azerbaijan's first AI-powered fashion marketplace) solo, in public, with AI. Previous write-up: how we built a zero-hallucination AI content pipeline.

How We Built a Zero-Hallucination AI Content Army for Fashion E-Commerce

Sahib Alizada — Mon, 23 Mar 2026 10:25:55 +0000

Geyin.az AI Agent Command Center — 11 agents monitoring dashboard with zero hallucination pipeline
By Sahib Alizada — Founder, Geyin.az

In my first article, I shared how I built Azerbaijan’s first AI-powered fashion marketplace as a solo founder. Today, I want to go deeper into the hardest problem we solved: making AI-generated content trustworthy.

The AI industry has a $67.4 billion hallucination problem. According to the Suprmind Research Report 2026, 47% of enterprise AI users made at least one major business decision based on hallucinated content in 2024. Each enterprise employee costs $14,200 per year in hallucination-related mitigation alone.

At Geyin.az, we decided that “good enough” wasn’t good enough. When you’re publishing fashion content in four languages to hundreds of thousands of potential readers, every fabricated trend, every made-up brand detail, every fictional price point erodes trust — and trust is the only currency that matters in e-commerce.

This is the story of how we built an AI content army that doesn’t hallucinate.

The Scale of the Hallucination Problem
Let’s be honest about where the industry stands. Even the best models still hallucinate:

Gemini 2.0 Flash holds the current record at 0.7% hallucination rate on Vectara’s benchmark — the lowest ever recorded.

GPT-5 reduced hallucinations by 44% compared to GPT-4o, dropping from 12.9% to 9.6% on factual tasks (PMC peer-reviewed study).

Claude Sonnet 4.6 achieved a 91% success rate in detecting false information, with only a 3% rate of confidently accepting falsehoods (AnyAPI LLM Hallucination Index 2026).

OpenAI’s own research (September 2025) concluded that next-token training fundamentally rewards confident guessing over calibrated uncertainty (arXiv:2509.04664).

That last point is critical. The very architecture that makes language models powerful — predicting the next most likely token — is also what makes them hallucinate. They’re designed to sound confident, even when they shouldn’t be.

And here’s the uncomfortable truth: domain-specific hallucination rates are 10–20% or higher, even when standardized benchmarks show 1–3%. Fashion is a domain where product names, seasonal trends, brand histories, and pricing are hyper-specific. Generic AI benchmarks mean nothing here.

Press enter or click to view image in full size
AI hallucination rate comparison 2025–2026 — Gemini 0.7% to GPT-4o 12.9%

Our Approach: Architecture Over Prompting

Most teams try to solve hallucination by tweaking prompts. We solved it by designing an architecture where hallucination literally cannot survive the pipeline.

11 Agents, Each With One Job

Our AI army consists of 11 specialized agents running 24/7 on a dedicated server. Each agent has a narrowly defined role and data access pattern:

Blog Writer — Writes articles in English only. Receives verified research data as input — never generates facts from memory.

GEO/AEO Enricher — Adds FAQ sections, comparison tables, and citations. Only works with the Writer’s output plus verified external sources.

Translator — Converts EN → AZ/TR/RU in one pass. Translates existing content — no creative generation allowed.

Brand Tracker — Monitors 34 fashion brands. Scrapes official websites and Wikipedia — no speculation.

Trend Tracker — Analyzes fashion trends from 50+ RSS feeds daily — reports only what’s published.

Competitor Monitor — Tracks 10 competitors. Scrapes real pricing and catalog data.

SEO Monitor — Runs full-site SEO audits every morning. Crawls actual pages — reports measured metrics.

Chef Agent — Quality control commander. Scores all output on 5 criteria — rejects anything scoring below 7/10.

Health Monitor — Checks platform uptime every 30 minutes. Pings real endpoints — binary pass/fail.

Telegram Bot — Customer-facing AI chat using the Chef Agent’s personality with grounded context.

Topics Researcher — Generates new content topics by cross-referencing trend and competitor data — only suggests verified gaps.

The key insight: no single agent both generates and publishes content. Every piece goes through at least three agents before it reaches our platform.

Geyin.az AI agent service running on dedicated server — 11 tasks, 45.9MB memory, active since March 23

The 14 Anti-Hallucination Rules

Every content-generating agent in our system follows 14 hard-coded rules. Here are the most important ones:

Never state a fact without a data source. If the agent can’t cite where information came from (RSS feed, scraped website, brand report), it must not include it.
Web scraping is mandatory, not optional. Before writing about any brand or trend, the agent scrapes the official source. “I think Zara’s new collection features…” is forbidden. “According to zara.com, accessed March 23, 2026…” is required.
Numbers must be traceable. Every statistic in our content links back to a specific data source — a competitor’s catalog count, a trend report’s color frequency, a brand’s official pricing.
When uncertain, omit. Our agents are explicitly instructed: if you’re not 100% sure, leave it out. We’d rather publish a shorter, accurate article than a longer one with fabricated details.
Cross-agent verification. The Enricher cross-references the Writer’s claims against its own data sources. The Chef Agent independently validates factual claims before scoring.

Why RAG Alone Isn’t Enough (And What We Do Instead)

Retrieval-Augmented Generation (RAG) is the industry’s favorite answer to hallucination. And it works — RAG reduces hallucination rates by 71% when properly integrated (Suprmind, 2026). In clinical settings, self-reflective RAG lowered hallucinations to 5.8% (MDPI Electronics, peer-reviewed). LinkedIn integrated RAG with knowledge graphs and saw a 77.6% improvement in retrieval accuracy (arXiv survey).

But RAG has a fundamental limitation: it retrieves and summarizes. Our agents don’t just retrieve — they verify, cross-reference, and reject.

Our pipeline is closer to what I call the “Verified Intelligence Pipeline” (VIP):

Step 1: RESEARCH — Agent scrapes primary sources (official sites, Wikipedia, RSS)

Step 2: WRITE — Blog Writer uses ONLY the research data (never its own “knowledge”)

Step 3: ENRICH — GEO/AEO Enricher adds FAQ, tables, Schema.org — from verified data

Step 4: TRANSLATE — Translator converts EN → AZ/TR/RU (no creative liberty)

Step 5: REVIEW — Chef Agent scores on 5 criteria (SEO, quality, fashion relevance, multilingual, GEO/AEO)

Step 6: GATE — Score ≥ 7 = approved | Score < 7 = flagged | Score < 5 = rewritten

Step 7: PUBLISH — Only approved content reaches the platform

The critical difference from RAG: our system has a rejection mechanism. RAG retrieves and generates. Our pipeline retrieves, generates, verifies, and can reject. That rejection layer is what makes zero-hallucination possible.

Press enter or click to view image in full size
Verified Intelligence Pipeline — 7-stage content verification with quality gate and rejection mechanism
Standing on Giants: Google and Anthropic
We didn’t build this alone. Our stack leverages two of the most advanced AI platforms in the world:

Google Gemini 2.5 Flash — Powers our Smart Upload (product recognition) and content generation. Gemini 2.0 Flash currently holds the world’s lowest hallucination rate at 0.7%. The improvement trajectory is remarkable: from 21.8% in 2021 to 0.7% in 2025 — a 96% reduction in four years (SparkCo analysis). We use Gemini’s grounding features to connect our agents to real-time web content.

Anthropic Claude — Powers our development workflow and strategic planning. Claude’s approach to “epistemic humility” — the tendency to say “I don’t know” rather than fabricate — is unique in the industry. Anthropic is the only vendor showing a consistent upward trajectory in this behavior (Balbix analysis). Their 2025 interpretability research identified the actual internal circuits responsible for declining answers when the model lacks information — a breakthrough in understanding why hallucinations happen at the architectural level.

Google Vertex AI — Our multimodal embedding engine uses Vertex AI’s 1408-dimensional vector space for visual search, understanding both images and text in the same mathematical space. When a customer uploads a photo of a dress, our system finds visually similar products — no hallucination possible, just pure mathematical similarity.

We chose these platforms deliberately. In an industry where 76% of enterprises now run human-in-the-loop processes specifically to catch AI hallucinations (Suprmind, 2026), we wanted to minimize the need for human intervention by starting with the most reliable foundations.

Press enter or click to view image in full size
Real-time agent logs showing Health Monitor, Topic Researcher, and Blog Writer activity

What’s Next: Our Own ML Models

We’re not stopping at using other companies’ models. Here’s what’s coming:

Custom Model Fine-Tuning

We’re preparing to fine-tune our own models specifically for the fashion domain. Why? Because general-purpose models have general-purpose hallucination patterns. A model fine-tuned on verified fashion data — our own curated dataset of brand information, trend histories, material properties, and pricing patterns — will have dramatically lower hallucination rates in our specific domain.

Our data advantage: every piece of content our agents produce is verified before it enters our dataset. This means our fine-tuning data is clean by design, not cleaned after the fact. Most companies start with dirty web-scraped data and spend millions cleaning it. We start with verified, structured, multilingual fashion intelligence.

Gemini Embedding 2.0 Migration

We’re upgrading from 1,408-dimensional to 3,072-dimensional embeddings with Google’s latest Gemini Embedding 2 model. This means:

5 modalities (text, image, video, audio, PDF) instead of 3. 100+ language support (critical for our 4-language platform). Task-specific optimization for fashion product similarity.

BigQuery Intelligence Engine

All our agent data flows into Google BigQuery. The next step: building a feedback loop where our ML models learn from what content performs best — which articles get the most engagement, which product descriptions convert, which trend predictions prove accurate. This is the Shopify model applied to fashion content intelligence.

The Numbers That Matter

Press enter or click to view image in full size
Geyin.az AI Army metrics — 11 agents, $55 monthly cost, 4 languages, 20K+ daily API calls, zero fabricated facts
Our entire AI agent army:

11 agents running 24/7 on a single server. $55/month total infrastructure cost. Produces what would require 5–6 content specialists. 4 languages (Azerbaijani, Turkish, Russian, English). 20,000+ API requests/day capacity across 7 AI providers. Zero fabricated facts in published content (enforced by pipeline).

For comparison, industry data shows:

85.1% of AI users deploy it for blog content generation (AutoFaceless, 2026). But 92% of organizations will increase GenAI investment while only 1% say deployment reached maturity (Business of Fashion, State of Fashion 2026). The AI agent market is projected to grow from $7.84 billion (2025) to $52.62 billion by 2030 — a 46.3% CAGR (MEV analysis). Gartner predicts 40% of enterprise apps will feature task-specific AI agents by end of 2026, up from less than 5% in 2025.

We’re already there. Not planning to adopt AI agents — operating them in production, at scale, with verified output.

The Future Belongs to AI-Native Platforms

Here’s my thesis: the fashion platforms that win in 2027 and beyond won’t be the ones with the biggest catalogs or the most VC funding. They’ll be the ones with the most trustworthy AI systems.

Traditional search engine volume will drop 25% by 2026 due to AI chatbots and agents (Gartner via SearchEngineLand). Shopping-related generative AI searches grew 4,700% between July 2024 and July 2025 (Business of Fashion). AI-driven revenue per visit on US retail sites grew 84% in the same period.

The implication is clear: your content needs to be citable by AI. Not just indexable by Google — citable by ChatGPT, Perplexity, Google AI Overviews, and Claude. That requires structured data, verified facts, authoritative sources, and expert authorship signals.

This is exactly what our AI army produces — every article, every day, in four languages, with zero hallucination.

The AI in fashion market is worth $2.92 billion in 2025 and projected to reach $3.99 billion in 2026 — a 40.8% growth rate (Business Research Insights). We’re building at the intersection of two explosive trends: AI agents and fashion e-commerce.

For Builders: What I Learned

If you’re building AI systems that generate content, here’s what I’d tell you:

Architecture beats prompting. You can’t prompt your way out of hallucination. You need a pipeline with verification and rejection stages.
Separate generation from publishing. No agent should both create and deploy content. The gap between those two actions is where quality control lives.
Use real data, not AI “knowledge.” Your agents should never rely on their training data for facts. Feed them verified sources — scrape it, cite it, or don’t include it.
Build on the best foundations. Google and Anthropic are spending billions on reducing hallucination at the model level. Leverage their work — then add your own verification layer on top.
Clean data in, clean data out. If you’re planning to fine-tune models, start collecting verified data now. Your future competitive advantage is the quality of your training dataset, and that takes time to build.
The cost is lower than you think. Our entire 11-agent army runs on $55/month. The barrier to AI-native operations isn’t money — it’s architecture.

Sahib Alizada is the founder of Geyin.az, Azerbaijan’s first AI-powered fashion marketplace. He builds AI agent systems for fashion e-commerce and writes about the intersection of artificial intelligence, entrepreneurship, and the future of retail.

Follow me on LinkedIn for more on AI-native business building.