The Recommendation System That Works Out of the Box

Starting from Nothing

You are building a marketplace or an ecommerce platform, something like Jumia or Jiji.com.
Your goals are clear: transactions, user interactions, a thriving ecosystem of buyers and sellers. Your roadmap is ambitious. But today, you are at ground zero.

Your database is empty. No catalog of products. No user base. No behavioral data. No ratings, no clicks, no purchase history. Just a wireframe or prototype and an idea.

And yet, your investors want to see engagement metrics. Your product manager wants to ship a recommendations feature. Your designer has mockups of "You
might also like" sections.

Traditional wisdom says wait. Collect data first. Let users interact. Then, and only then, deploy a recommendation system. But in a marketplace, users
decide in the first few seconds whether to stay or leave. If your recommendations section is empty or irrelevant on day one, you lose them before you ever had them.

This is the cold start problem, and it is a deadlock: you cannot serve recommendations until users interact, but users will not interact until they see relevant content.

Approaches That Won't Work

Recommendation systems, by design, are data-hungry.
Popular approaches:

1. Collaborative Filtering (Surprise, LightFM, implicit):user-item interaction matrix ( ratings, clicks, purchases, views).
   At zero users, the matrix is empty. The first customers see nothing.

2. Neural Recommenders (TensorFlow Recommenders, two-tower models): trains on labeled interaction sequences. No data equals no training. Weeks of model development yield nothing usable.

3. Cloud Managed Services (AWS Personalize, GCP Recommendations AI): requires you to upload datasets before they work. No interactions to upload means no service,they charge per recommendation tying your unit economics to a provider.

4. Graph-Based Recommendations (Neo4j GDS, graph embeddings): a pre-constructed graph of user-item relationships. With zero users, the graph has
   one node.

placeholder recommendations are not recommendations. They destroy trust immediately. A buyer sees generic products unrelated to their intent, clicks away, and never returns.
Placeholders are worse than nothing because they look like the product team did not care.

You need a recommendation system that works on day one, with zero data, and zero placeholder content.

A Data-Free Approach

When your entire problem is that you have no data, perhaps the solution is to not need data at all.

Instead of "what do similar users like?",lets go with "What looks similar to this item?"

This is Content-based visual search; no user interactions required.

How It Works

The approach is a three-stage pipeline:

1. Embed the image or text into a dense vector using a pre-trained model
2. Store catalog vectors in a vector-capable database
3. Rank by similarity score between query and catalog vectors

No training. No user data. No waiting. Just a single image/text, a pre-trained model, and a SQL query.

Option A: Visual-to-Visual Search (Image Only)

When you have product photos but no captions, text, or metadata:

Model	Purpose	Source	Embedding Dim
FashionCLIP ViT-B/32	Fashion image embeddings	huggingface.co/patrickjohncyh/fashion-clip	768
CLIP ViT-B/32	General image embeddings	huggingface.co/openai/clip-vit-base-patch32	512
OpenCLIP ViT-B/32	Open source CLIP variant	huggingface.co/laion/opensource-CLIP	512

FashionCLIP is trained specifically on fashion images, making it more accurate for apparel than general CLIP models. Export it to ONNX (Open Neural Network
Exchange) format for CPU-only inference with no dependencies on large ML frameworks at runtime.

You also need YOLOv8n for garment detection and cropping. A full-body photo contains background, accessories, and other items. You want the dress, not
the person holding it. A pre-trained garment detection model identifies the four classes: accessories, bags, clothing, shoes. Crop to the bounding box, then embed.

Note: Generic ONNX checkpoints for both visual and detection models are available on HuggingFace for production deployment.

Option B: Text-to-Text Similarity (No Images)

When you have product titles, descriptions, or captions but no images. Or when you want to combine textual and visual signals:

Model	Purpose	Source	Embedding Dim
all-MiniLM-L6-v2	Lightweight text embeddings	huggingface.co/sentence-transformers/all-MiniLM-L6-v2	384
text-embedding-3-small	OpenAI high-quality embeddings	platform.openai.com	512 or 1536
multilingual-e5-small	Cross-lingual text	huggingface.co/intfloat/multilingual-e5-small	384

Hybrid approach: You can also combine two signals like what an item looks like visually and what its description says.
This works especially well when sellers provide captions alongside product photos. Such as (color, style, cut) and attributes (price range, size, location).

The Code

The full pipeline is under 60 lines:

import numpy as np
import duckdb
from PIL import Image
import onnxruntime as ort
from pathlib import Path


def load_model(model_path: str) -> ort.InferenceSession:
    """Load FashionCLIP ONNX model once. Reuse for all inferences."""
    opts = ort.SessionOptions()
    opts.log_severity_level = 3
    return ort.InferenceSession(
        model_path,
        sess_options=opts,
        providers=["CPUExecutionProvider"],
    )


def preprocess(image_path: str) -> np.ndarray:
    """Convert image file to ONNX-ready tensor (1, 3, 224, 224) float32."""
    img = Image.open(image_path).convert("RGB")
    img = img.resize((224, 224), Image.BILINEAR)
    arr = np.array(img, dtype=np.float32) / 255.0
    return arr.transpose(2, 0, 1)[np.newaxis, ...]


def embed(session: ort.InferenceSession, image_path: str) -> list[float]:
    """Generate a 768-dim normalized embedding for one image.
    """
    blob = preprocess(image_path)
    raw = session.run(None, {"image": blob})[0].astype(np.float32)
    vec = raw.flatten()
    norm = float(np.linalg.norm(vec))
    return (vec / norm).round(6).tolist() if norm > 0 else [0.0] * 768


def search_similar(connection: duckdb.DuckDBPyConnection,
                   session: ort.InferenceSession,
                   query_path: str, limit: int = 5) -> list[dict]:
    """Embed a query image and return top-k similar items.

    Builds the vector as a SQL literal because DuckDB vss expects an
    inline array, not a bound parameter, for the similarity score
    function.
    """
    query_vector = embed(session, query_path)
    vec_literal = "[" + ",".join(str(v) for v in query_vector) + "]"
    rows = connection.execute(f"""
        SELECT item_id, image_url,
               array_cosine_similarity(vector, {vec_literal}::FLOAT[768]) AS score
        FROM catalog
        ORDER BY score DESC
        LIMIT {limit}
    """).fetchall()
    return [{"rank": i + 1, "item_id": r[0], "image_url": r[1],
             "score": round(float(r[2]), 3)} for i, r in enumerate(rows)]

Setup in Three Commands

pip install onnxruntime duckdb pillow numpy
python -c "import duckdb; c = duckdb.connect(); c.execute('INSTALL vss; LOAD vss;')"

Then create the catalog table, pre-compute embeddings for your image catalog,
and run searches:

conn = duckdb.connect("catalog.db")
conn.execute("INSTALL vss; LOAD vss;")
conn.execute("""
    CREATE TABLE IF NOT EXISTS catalog (
        item_id VARCHAR, image_url VARCHAR, vector FLOAT[768]
    )
""")
# Embed every image in your catalog once, then store in DuckDB
build_catalog(conn, "/path/to/images/", "/path/to/fashion_clip.onnx")

# Query with any image
results = search_similar(
    conn,
    load_model("/path/to/fashion_clip.onnx"),
    "/path/to/query.jpg",
)
for r in results:
    print(f"{r['rank']}. {r['item_id']} (score: {r['score']})")

The Outcome

This approach is a production-viable recommendation system that works at high scaling points:

Scale	Engine	What Changes
1 - 100K items	DuckDB + VSS	Nothing( single file, single query)

Above 100K items you can safely transition to qdrant cloud (cheap/free tier) or self-hosted (cheaper).

Where Content Comes From

So where does the catalog come from if you are starting from zero? You do not need a users signed up to have data.

Here are four strategies :

• Web scraping: Cloudflare Worker scrapers collect data from other platforms.

• User uploads: Guests upload photos through the web app. No account required.

• Public image datasets: Hugging Face or Kaggle provide labeled images with captions.

• Synthetic data: Generate catalog entries to create
  variations without new photography.

Conclusion

In real-world projects I have seen teams stall trying to build recommendation engines that depend entirely on user behavior like collecting clicks, ratings, and purchase history before they can do anything useful.
I hit the same wall on Twynon, my search engine for local fashion markets, and the solution was the same approach.

Twynon builds on this principle: a search engine for local fashion markets that works from day one, no user data required. If you are solving a similar problem, focus on content first and behavior second.

Follow for future posts..