Building a Content Generation App with LLM

Content generation pipelines are now a standard workload for modern applications. Whether you are drafting marketing copy, personalizing outreach, or expanding product descriptions, a reliable LLM backend determines both output quality and operating cost. This guide walks through building a lightweight, production-ready content generation app that accepts prompts, enriches them with context, and streams structured copy back to the user.

What You Will Build

You will build a Python service that exposes an HTTP endpoint. It receives a topic, tone, and audience parameters, assembles a system prompt, and returns generated content via Server-Sent Events. The stack is FastAPI for the server, Jinja2 for prompt templating, and the OpenAI SDK for inference.

Architecture Overview

The pipeline has three stages: ingestion, prompt assembly, and generation. Ingestion validates user input against a Pydantic schema. Prompt assembly merges that input with a Jinja2 template to enforce style guidelines. Generation sends the final payload to an LLM endpoint and streams tokens back to the client. Keeping these stages separate makes it easy to swap models, adjust templates, or add post-processing without rewriting the core loop.

Setting Up the Environment

Create a virtual environment and install the dependencies.

python -m venv venv
source venv/bin/activate
pip install fastapi uvicorn jinja2 openai pydantic

Set your Oxlo.ai API key as an environment variable. You can generate one from the Oxlo.ai dashboard.

export OXLO_API_KEY="your_api_key_here"

Integrating the LLM API

Oxlo.ai exposes a fully OpenAI-compatible endpoint at https://api.oxlo.ai/v1. Because the platform uses request-based pricing, your cost per generation stays flat regardless of how long your system prompt or retrieved context grows. This is especially useful for content apps that inject long documents, style guides, or conversation history into every request.

Below is the client initialization. Point the base URL to Oxlo.ai and select a model suited for copywriting. Llama 3.3 70B is a strong general-purpose choice, while Qwen 3 32B handles multilingual content well.

import openai
import os

client = openai.OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.getenv("OXLO_API_KEY")
)

MODEL = "llama-3.3-70b"  # or qwen-3-32b for multilingual workloads

Building the Content Pipeline

Define a Pydantic model to validate incoming requests, then use Jinja2 to keep prompts maintainable.

from pydantic import BaseModel
from jinja2 import Template

class ContentRequest(BaseModel):
    topic: str
    tone: str
    audience: str
    word_count: int = 300

SYSTEM_TEMPLATE = Template("""
You are an expert copywriter. Write {{ word_count }} words about {{ topic }}.
Tone: {{ tone }}.
Audience: {{ audience }}.
Use clear structure with headings and bullet points where appropriate.
""")

def build_messages(req: ContentRequest):
    system_content = SYSTEM_TEMPLATE.render(
        topic=req.topic,
        tone=req.tone,
        audience=req.audience,
        word_count=req.word_count
    )
    return [
        {"role": "system", "content": system_content},
        {"role": "user", "content": "Generate the content."}
    ]

Handling Long Context and Batch Workloads

Content generation apps often need to ingest source material. A blog generator might feed in ten existing posts as examples, or a legal brief tool might attach case files. On token-based providers, those extra tokens inflate every request. Oxlo.ai charges a flat rate per API call, so adding retrieved documents or few-shot examples does not change your unit cost. This makes it practical to enrich prompts with full articles, PDF extracts, or multi-turn revision history without worrying about metered token burn.

If you need to process hundreds of articles in a batch, map your prompt list over an async client. Because Oxlo.ai has no cold starts on popular models, the first request in a batch returns as quickly as the last.

Putting It All Together

Wire the pipeline into a FastAPI app with streaming.

from fastapi import FastAPI
from fastapi.responses import StreamingResponse

app = FastAPI()

def generate_content(req: ContentRequest):
    messages = build_messages(req)
    response = client.chat.completions.create(
        model=MODEL,
        messages=messages,
        stream=True,
        temperature=0.7,
        max_tokens=2048
    )
    for chunk in response:
        if chunk.choices[0].delta.content:
            yield chunk.choices[0].delta.content

@app.post("/generate")
async def generate(req: ContentRequest):
    return StreamingResponse(
        generate_content(req),
        media_type="text/event-stream"
    )

Run the server with uvicorn main:app --reload and send a test request:

curl -X POST http://localhost:8000/generate \
  -H "Content-Type: application/json" \
  -d '{"topic":"API pricing models","tone":"professional","audience":"developers"}'

Deploying and Scaling

For production, containerize the FastAPI service and place it behind an async worker pool. Store prompt templates in version control, not hardcoded strings, so non-technical stakeholders can edit copy guidelines without redeploying code. Monitor latency at the 95th percentile and set request timeouts based on your longest expected generation, typically five to ten seconds for short-form copy.

If your workload grows beyond hobby scale, evaluate Oxlo.ai tiers that match your daily request volume. The flat per-request model means your forecast is based on API call count, not unpredictable token math. See the latest plans at https://oxlo.ai/pricing.

Why Oxlo.ai for Content Generation

Oxlo.ai fits content generation pipelines in three specific ways. First, request-based pricing removes the penalty for long system prompts and retrieved context, which are common in RAG-powered copywriting tools. Second, the OpenAI SDK compatibility means you can prototype with existing code and switch the base URL without rewriting your client logic. Third, the absence of cold starts on popular models keeps user-facing latency consistent, even when traffic is bursty.

Models like Llama 3.3 70B, Qwen 3 32B, and DeepSeek V3.2 cover most content tasks, from blog drafts to technical documentation. You can experiment with all of them under the free tier or a 7-day full-access trial, then scale to a Pro or Premium plan as your user base grows.