Building Recommender Systems with LLMs: A Technical Guide

We are going to build a real-time article recommender that ranks candidate content with an LLM, skipping the heavy upfront work of training collaborative filtering models. This approach helps teams ship personalization fast by treating ranking as a reasoning task rather than a matrix factorization problem. The finished script is a drop-in Python module that calls Oxlo.ai through the OpenAI SDK and returns structured JSON you can serve from any backend.

What you'll need

• Python 3.10 or newer.
• The OpenAI SDK: pip install openai.
• An Oxlo.ai API key from https://portal.oxlo.ai. Because Oxlo.ai charges per request instead of per token, you can feed the model a full catalog of candidate abstracts without watching a meter run. See https://oxlo.ai/pricing for plan details.

Step 1: Set up the client and candidate catalog

First, point the OpenAI SDK at Oxlo.ai and define a small set of candidate articles. In production you would pull this from your CMS, but a hardcoded list is enough to validate the logic.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

CANDIDATES = [
    {
        "id": "a1",
        "title": "Understanding Transformer Architectures",
        "abstract": "A deep dive into self-attention mechanisms and positional encoding in modern NLP models.",
    },
    {
        "id": "a2",
        "title": "Rust Memory Safety for Python Developers",
        "abstract": "Learn how Rust's ownership model can inform safer Python extension modules.",
    },
    {
        "id": "a3",
        "title": "Kubernetes Observability at Scale",
        "abstract": "Patterns for collecting metrics, logs, and traces in clusters with over 10,000 pods.",
    },
    {
        "id": "a4",
        "title": "Diffusion Models for Audio Generation",
        "abstract": "Recent advances in applying diffusion-based generative models to music and speech synthesis.",
    },
    {
        "id": "a5",
        "title": "Vector Database Comparison 2025",
        "abstract": "Benchmarking retrieval latency and recall across pgvector, Milvus, and Weaviate.",
    },
]

Step 2: Capture implicit user signals

Next, model the user from recent reading history. I keep this simple, titles and tags only, but you could append dwell time or clickstream events if you log them.

USER_HISTORY = [
    {"title": "Introduction to PyTorch 2.0", "tags": ["machine learning", "python"]},
    {"title": "Scaling LLM Training on Kubernetes", "tags": ["infrastructure", "mlops"]},
    {"title": "Attention Is All You Need, Revisited", "tags": ["nlp", "transformers"]},
]

def build_user_profile(history):
    lines = []
    for item in history:
        lines.append(f"- {item['title']} (tags: {', '.join(item['tags'])})")
    return "\n".join(lines)

user_context = build_user_profile(USER_HISTORY)

Step 3: Write the recommender system prompt

The system prompt encodes the ranking logic. I force JSON output and lock the schema so parsing is trivial and safe.

SYSTEM_PROMPT = """You are a content recommender engine. Your job is to rank candidate articles for a specific user.

You will receive:
1. A list of articles the user recently read, with tags.
2. A list of candidate articles, each with an ID, title, and abstract.

Instructions:
- Analyze the user's interests based on their reading history.
- Rank the top 3 candidates from most to least relevant.
- For each ranked item, write a one-sentence explanation of why it matches the user's interests.
- Output strict JSON with no markdown formatting. Use this exact structure:
{
  "ranking": [
    {"rank": 1, "id": "candidate_id", "reason": "explanation"},
    {"rank": 2, "id": "candidate_id", "reason": "explanation"},
    {"rank": 3, "id": "candidate_id", "reason": "explanation"}
  ]
}

Be concise. Do not include any text outside the JSON object."""

Step 4: Rank candidates with Llama 3.3 70B

Now assemble the prompt and call Llama 3.3 70B through Oxlo.ai. I use JSON mode and a low temperature to keep the model focused on structure rather than creativity.

import json

def build_user_message(candidates, user_profile):
    candidate_lines = []
    for c in candidates:
        candidate_lines.append(
            f"ID: {c['id']}\nTitle: {c['title']}\nAbstract: {c['abstract']}"
        )
    candidate_block = "\n\n".join(candidate_lines)

    return f"""User reading history:
{user_profile}

Candidate articles:
{candidate_block}

Return the JSON ranking."""

user_message = build_user_message(CANDIDATES, user_context)

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": user_message},
    ],
    response_format={"type": "json_object"},
    temperature=0.2,
)

raw_output = response.choices[0].message.content
print(raw_output)

Step 5: Parse and serve the results

Finally, deserialize the JSON and surface the ranking. In your API you would return this list to the frontend or cache it for the session.

result = json.loads(raw_output)

for item in result["ranking"]:
    rank = item["rank"]
    article_id = item["id"]
    reason = item["reason"]
    print(f"{rank}. {article_id}: {reason}")

Run it

Copy the full script below into recommend.py, swap in your Oxlo.ai key, and run python recommend.py.

from openai import OpenAI
import json

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

CANDIDATES = [
    {
        "id": "a1",
        "title": "Understanding Transformer Architectures",
        "abstract": "A deep dive into self-attention mechanisms and positional encoding in modern NLP models.",
    },
    {
        "id": "a2",
        "title": "Rust Memory Safety for Python Developers",
        "abstract": "Learn how Rust's ownership model can inform safer Python extension modules.",
    },
    {
        "id": "a3",
        "title": "Kubernetes Observability at Scale",
        "abstract": "Patterns for collecting metrics, logs, and traces in clusters with over 10,000 pods.",
    },
    {
        "id": "a4",
        "title": "Diffusion Models for Audio Generation",
        "abstract": "Recent advances in applying diffusion-based generative models to music and speech synthesis.",
    },
    {
        "id": "a5",
        "title": "Vector Database Comparison 2025",
        "abstract": "Benchmarking retrieval latency and recall across pgvector, Milvus, and Weaviate.",
    },
]

USER_HISTORY = [
    {"title": "Introduction to PyTorch 2.0", "tags": ["machine learning", "python"]},
    {"title": "Scaling LLM Training on Kubernetes", "tags": ["infrastructure", "mlops"]},
    {"title": "Attention Is All You Need, Revisited", "tags": ["nlp", "transformers"]},
]

SYSTEM_PROMPT = """You are a content recommender engine. Your job is to rank candidate articles for a specific user.

You will receive:
1. A list of articles the user recently read, with tags.
2. A list of candidate articles, each with an ID, title, and abstract.

Instructions:
- Analyze the user's interests based on their reading history.
- Rank the top 3 candidates from most to least relevant.
- For each ranked item, write a one-sentence explanation of why it matches the user's interests.
- Output strict JSON with no markdown formatting. Use this exact structure:
{
  "ranking": [
    {"rank": 1, "id": "candidate_id", "reason": "explanation"},
    {"rank": 2, "id": "candidate_id", "reason": "explanation"},
    {"rank": 3, "id": "candidate_id", "reason": "explanation"}
  ]
}

Be concise. Do not include any text outside the JSON object."""

def build_user_profile(history):
    lines = []
    for item in history:
        lines.append(f"- {item['title']} (tags: {', '.join(item['tags'])})")
    return "\n".join(lines)

def build_user_message(candidates, user_profile):
    candidate_lines = []
    for c in candidates:
        candidate_lines.append(
            f"ID: {c['id']}\nTitle: {c['title']}\nAbstract: {c['abstract']}"
        )
    candidate_block = "\n\n".join(candidate_lines)

    return f"""User reading history:
{user_profile}

Candidate articles:
{candidate_block}

Return the JSON ranking."""

user_context = build_user_profile(USER_HISTORY)
user_message = build_user_message(CANDIDATES, user_context)

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": user_message},
    ],
    response_format={"type": "json_object"},
    temperature=0.2,
)

result = json.loads(response.choices[0].message.content)
for item in result["ranking"]:
    print(f"{item['rank']}. {item['id']}: {item['reason']}")

Example output:

{
  "ranking": [
    {"rank": 1, "id": "a1", "reason": "Directly extends the user's demonstrated interest in transformer architectures and NLP fundamentals."},
    {"rank": 2, "id": "a3", "reason": "Aligns with their DevOps reading history around Kubernetes and large-scale infrastructure."},
    {"rank": 3, "id": "a5", "reason": "Relevant to ML infrastructure, though more database-focused than their core interests."}
  ]
}

Wrap-up and next steps

This pattern gets you to production fast, but two moves will harden it. First, cache the user profile in Redis so you do not rebuild the history prompt on every request. Second, switch to a hybrid pipeline: retrieve a broad set of candidates with BGE-Large embeddings via Oxlo.ai's embeddings endpoint, then let Llama 3.3 70B or DeepSeek V3.2 re-rank the shortlist. Because Oxlo.ai pricing is request-based, expanding the candidate pool or the user history does not inflate the cost, which makes iterative tuning painless.