Using LLM for Recommender Systems and Personalized Marketing in E-commerce

Large language models are reshaping e-commerce infrastructure. Where classical recommender systems rely on collaborative filtering and sparse embeddings, LLMs can reason directly over product descriptions, user reviews, and behavioral narratives in natural language. This shift enables stronger cold-start performance, better coverage of long-tail inventory, and unified generation of personalized marketing copy. The barrier has been cost. Feeding rich product catalogs and lengthy user histories into token-based APIs causes inference spend to scale linearly with context length. Oxlo.ai removes that constraint with flat, per-request pricing and a broad catalog of long-context open-source models, making LLM-powered recommendation and personalization economically viable at production scale.

Why LLMs Change Recommendation Engines

Classical matrix factorization and two-tower models treat users and items as opaque IDs. They work well for head items with dense interaction signals, but fail on cold-start products and sparse behavioral histories. LLMs invert this. A model such as Llama 3.3 70B or Qwen 3 32B on Oxlo.ai can read a product description, infer attribute relationships, and connect them to a user's stated preferences or browsing narrative without requiring millions of implicit feedback events. The same model can also generate the subject line for a retention email, collapsing two separate pipelines into one unified inference layer.

Architecture Patterns

Most production systems keep a fast candidate-generation stage and use an LLM only for ranking or copy generation.

• Embedding retrieval. Use an embedding model like BGE-Large or E5-Large, available through Oxlo.ai's embeddings endpoint, to encode items and queries into a shared vector space. Retrieve the top-k candidates from a vector database.
• LLM re-ranking. Pass the retrieved candidates, along with the user's recent clicks, cart events, and profile, to an LLM in a structured prompt. The model returns a ranked list and optional explanations.
• Agentic personalization. For high-value sessions, an agent loop powered by DeepSeek R1 671B or Kimi K2.6 can ask clarifying questions, refine constraints, and iterate on recommendations.

Implementing Prompt-Based Ranking

The ranking prompt is a JSON schema or natural language instruction that lists candidates and user context. Because Oxlo.ai is fully OpenAI SDK compatible, you can drop in the base URL and use JSON mode to enforce structured output.

import openai
import json

client = openai.OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"
)

user_history = [
    {"item": "Trail Runner Shoes v2", "event": "viewed", "timestamp": "2025-06-01T14:00Z"},
    {"item": "Merino Wool Socks", "event": "purchased", "timestamp": "2025-05-30T09:00Z"}
]

candidates = [
    {"id": "P101", "title": "Waterproof Trail Runner", "attrs": "Gore-Tex, 8mm drop, 320g"},
    {"id": "P102", "title": "Cotton Crew Socks", "attrs": "Basic cotton, 3-pack, casual"}
]

prompt = f"""You are a ranking engine for an outdoor retailer.
Given the user history and candidate products, return a JSON object with:
1. 'ranked_ids': list of candidate IDs sorted by relevance (most relevant first).
2. 'reasoning': one sentence per candidate explaining the rank.

User history: {json.dumps(user_history)}
Candidates: {json.dumps(candidates)}"""

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[{"role": "user", "content": prompt}],
    response_format={"type": "json_object"}
)

result = json.loads(response.choices[0].message.content)
print(result)

The model reasons over attributes like "Gore-Tex" and connects them to the user's trail-running interest, while downranking generic cotton socks that do not complement the recent purchase.

The Long-Context Economics of User History

A serious limitation of token-based inference is that every product description and every click event added to the prompt increases cost. For a re-ranker that passes 50 detailed product cards to the model, input tokens can dominate spend. Oxlo.ai uses request-based pricing: one flat cost per API call regardless of prompt length. That means you can include full item descriptions, long session histories, and even structured catalog metadata without scaling costs.

This pricing model is especially effective with Oxlo.ai's long-context models. Kimi K2.6 offers a 131K context window for advanced reasoning and agentic coding, while DeepSeek V4 Flash supports 1M tokens for near state-of-the-art open-source reasoning. You can fit an entire product category or months of user behavior into a single request for the same flat price. For heavy long-context or agentic workloads, this architecture can be significantly cheaper than token-based alternatives. See the exact rates on the Oxlo.ai pricing page.

Personalized Marketing Generation

After ranking, the same LLM pipeline can generate channel-specific copy. Multilingual models such as Qwen 3 32B let you produce localized content for global storefronts without maintaining separate templates per language.

system_msg = ("You are a direct-response copywriter for an outdoor retailer. "
              "Return strictly JSON with keys 'subject' and 'body'.")

user_msg = """Segment: Trail runners who bought socks but no shoes.
Offer: Waterproof Trail Runner at 20% off.
Constraints: Subject line under 50 characters. Body under 120 words. Tone is technical and concise."""

response = client.chat.completions.create(
    model="qwen3-32b",
    messages=[
        {"role": "system", "content": system_msg},
        {"role": "user", "content": user_msg}
    ],
    response_format={"type": "json_object"}
)

campaign = json.loads(response.choices[0].message.content)
print(campaign["subject"])

Because Oxlo.ai supports streaming responses, you can return draft copy to marketing dashboards in real time.

Evaluation and Guardrails

LLM recommenders require disciplined evaluation. Offline, measure ranking quality with NDCG@k or MRR against held-out click logs. Online, run A/B tests that compare LLM re-ranking against your incumbent collaborative filtering baseline.

Latency is a common concern. Oxlo.ai serves popular models with no cold starts, so p99 latency stays predictable even when scaling from prototype to production load. For high-volume traffic, use smaller variants such as Oxlo.ai Coder Fast or Qwen 3 Coder 30B for ranking, and reserve larger models like DeepSeek R1 671B or GLM 5 for low-volume, high-value agentic sessions.

Guardrails matter. Use JSON mode to constrain output structure, and validate all returned IDs against your catalog before rendering them to users. If you generate marketing copy, add a secondary moderation pass or filter for disallowed claims.

Getting Started with Oxlo.ai

Oxlo.ai is a drop-in replacement for any OpenAI SDK client. Change the base URL and API key, and existing code for chat completions, embeddings, or JSON mode works without modification.

client = openai.OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"
)

The free tier includes 60 requests per day across 16+ models, with a 7-day full-access trial. DeepSeek V3.2 is available on the free tier for coding and reasoning experiments. When you move to production, the Pro and Premium plans offer fixed daily request volumes that simplify budgeting for variable traffic.

Conclusion

LLMs offer a unified, semantic layer for recommendation and personalized marketing, but only if inference economics support long contexts and high request volumes. Oxlo.ai's flat per-request pricing, long-context model catalog, and OpenAI-compatible API remove the financial and operational barriers that slow adoption. For e-commerce teams building the next generation of recommender systems, Oxlo.ai is a strong, relevant platform to prototype and scale on.