Building Recommender Systems with LLM and Collaborative Filtering

Recommender systems have reached a saturation point where collaborative filtering alone cannot resolve cold-start ambiguity, and large language models alone cannot encode collective behavioral patterns. The pragmatic path forward is a hybrid architecture that uses matrix factorization or neural collaborative filtering to generate candidates, then employs an LLM to re-rank, explain, or augment those recommendations with contextual reasoning.

The Case for Hybrid Recommender Architectures

Collaborative filtering excels at uncovering latent patterns in user-item interaction matrices. It is computationally efficient, well understood, and scales horizontally. However, it fails when an item has no interaction history, and it offers no mechanism to leverage raw content metadata such as product descriptions, reviews, or multimodal attributes.

LLMs introduce complementary capabilities. They can parse unstructured content, reason over user profiles expressed in natural language, and generate structured output such as ranked lists or explanation strings. The challenge is not capability but cost and latency. When every user history and item description becomes a prompt, token-based billing scales unpredictably.

A hybrid pipeline isolates collaborative filtering for candidate generation, keeping the search space narrow, then applies the LLM only to a shortlist of items. This limits inference costs and reduces latency.

Collaborative Filtering as the Signal Foundation

Start with a standard matrix factorization approach. Using implicit feedback data, factorize the user-item matrix into low-dimensional embeddings.

import numpy as np
from implicit.als import AlternatingLeastSquares

# interactions: scipy sparse matrix (users x items)
model = AlternatingLeastSquares(factors=128, regularization=0.05)
model.fit(interactions)

# retrieve candidate item indices for a given user
user_factors = model.user_factors[user_id]
scores = np.dot(user_factors, model.item_factors.T)
candidate_indices = np.argpartition(scores, -50)[-50:]  # top-50 candidates

These candidates represent the statistically probable preferences based on historical behavior. They form the input set for the LLM re-ranking stage.

LLM Enrichment for Content and Context

Once candidates are retrieved, the LLM evaluates them against explicit user context. Construct a prompt that includes the user’s stated preferences, recent interactions, and structured metadata for each candidate item.

The prompt should be deterministic and structured. Use JSON mode or constrained decoding if the model supports it.

import json

from openai import OpenAI

client = OpenAI(

    base_url="https://api.oxlo.ai/v1",

    api_key="YOUR_OXLO_API_KEY"

)

def rank_candidates(user_context, candidates):

    prompt = f"""

    User preferences: {user_context}

Candidate items:
{json.dumps(candidates, indent=2)}

Return a JSON array of item_ids ordered by relevance, highest first.
"""

response = client.chat.completions.create(
    model="llama-3.3-70b",  # or Qwen 3 32B for multilingual workloads
    messages=[{"role": "user", "content": prompt}],
    response_format={"type": "json_object"},
    temperature=0.1
)
return json.loads(response.choices[0