Shoppers rarely type the exact product name. They type "black leather ankle boots" or upload a photo and expect the app to find something close. This post shows how to build that with open models, then looks at what it means for a fashion app budget, using Australia as the example market.
Why it matters
Online spending in Australia hit AUD 21.9 billion in April–June 2026, up 14% year on year, according to Australia Post. Fashion catalogues are huge and visual, and keyword search alone misses a lot. Cost guides for ecommerce apps, such as Techugo's breakdown for an app like THE ICONIC, list AI-based search, recommendations and visual discovery among the features that raise development effort.
The core idea: one embedding space for text and images
A CLIP-style model maps images and text into the same vector space. Embed every product photo once, embed each query at request time, and return the nearest vectors.
python
from sentence_transformers import SentenceTransformer, util
from PIL import Image
model = SentenceTransformer("clip-ViT-B-32")
# 1. Offline: embed the catalogue images
paths = ["boot_01.jpg", "sneaker_02.jpg", "dress_03.jpg"]
img_emb = model.encode(
[Image.open(p) for p in paths],
convert_to_tensor=True,
normalize_embeddings=True,
)
# 2. Online: embed the shopper's text query
query = "black leather ankle boots"
q_emb = model.encode(query, convert_to_tensor=True, normalize_embeddings=True)
# 3. Retrieve the closest products
hits = util.semantic_search(q_emb, img_emb, top_k=3)[0]
for h in hits:
print(paths[h["corpus_id"]], round(h["score"], 3))
Passing an image as the query works the same way, which gives you "shop the look" for free.
Architecture in four parts
Ingestion job: embeds new or changed product images on a schedule and writes vectors to an index (FAISS, pgvector or a managed vector database).
Search API: embeds the query, runs a nearest-neighbour lookup, then filters by size, stock and price.
Re-ranking: blends similarity with business signals such as availability, margin or popularity.
App layer: a cross-platform client (Flutter or React Native) is usually enough, because the heavy lifting happens on the server.
Things that go wrong in practice
Domain gap: general-purpose models can confuse close categories such as "midi skirt" and "maxi skirt". Evaluate on your own catalogue and consider fine-tuning.
Stale vectors: if images change and embeddings don't, results drift. Version your index.
Cold start: new shoppers have no history, so fall back to visual similarity and trending items.
Measure it: track recall@k on a small labelled query set, plus click-through and add-to-cart rate online.
What it means for the budget
Reference ranges from the same cost guide, in AUD:
Tier - Budget - Timeline
MVP - 30K – 150K - 3–5 months
Mid-level - 150K – 250K - 5–8 months
Advanced - 250K – 350K - 8–12 months
My suggestion is to ship plain search in the MVP, then add visual search once you have real queries to evaluate against. Beyond the build, plan for:
Hosting: roughly AUD 300–3,000+ per month depending on traffic, with spikes in big sale seasons. Vector search and model inference add to this.
Maintenance: about 15–20% of the initial build cost per year.
Compliance: if you use shopper photos or behaviour data, the Privacy Act 1988 and the Australian Privacy Principles apply.
Takeaway
Visual search is a small, well-understood pipeline: embed once, search fast, re-rank with business rules. Start with an open model, measure on your own catalogue, and let the data decide whether fine-tuning is worth the cost.
Resources
Sentence Transformers: image search with CLIP
Techugo: cost to build an app like THE ICONIC in Australia
Australia Post quarterly ecommerce update (July 2026)
OAIC: Australian Privacy Principles
Cost figures are indicative estimates from the sources above and vary by vendor, scope and region.
Top comments (0)