Building Language Generation Models with Oxlo

#learnai #oxlo #ai

We are going to build a structured markdown blog generator that turns a handful of bullet points into a full technical draft. It is useful for developer advocates and technical writers who need consistent first drafts without managing token budgets. Because Oxlo.ai charges a flat rate per request, you can feed it long system prompts and detailed context without watching the meter tick on every token.

What you'll need

Python 3.10 or newer, an Oxlo.ai API key from https://portal.oxlo.ai, and the OpenAI SDK installed with pip install openai.

Step 1: Scaffold the client and verify the endpoint

I always start by confirming the endpoint returns a valid completion before adding any application logic.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "user", "content": "Say hello."},
    ],
)

print(response.choices[0].message.content)

Step 2: Define the system prompt and generation rules

The system prompt constrains the model to emit only valid JSON with a markdown payload. Keeping it in a constant makes iteration easier.

SYSTEM_PROMPT = """You are a technical blog generator. 
The user will provide a topic, audience, tone, and key points.
You must respond with a JSON object containing exactly two keys: title (string) and markdown (string).
The markdown must include YAML frontmatter with title, date, and tags.
The body must have an introduction, one section per key point, and a conclusion.
Use code blocks where relevant. Output only the JSON object."""

Step 3: Build the input assembler

I wrap the user inputs into a structured message so the model receives consistent formatting every time.

def build_user_message(topic, audience, tone, points):
    points_str = "\n".join(f"- {p}" for p in points)
    return f"""Topic: {topic}
Audience: {audience}
Tone: {tone}
Key points:
{points_str}

Generate the full markdown article now."""

# Quick sanity check
user_msg = build_user_message(
    topic="Understanding MoE Architectures",
    audience="Senior backend engineers",
    tone="Conversational but precise",
    points=[
        "Sparse vs dense parameter usage",
        "Routing mechanisms and load balancing",
        "Inference cost trade-offs"
    ]
)
print(user_msg)

Step 4: Generate with JSON mode

Using JSON mode guarantees the output can be parsed downstream, splitting metadata from content cleanly.

import json

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": user_msg},
    ],
    response_format={"type": "json_object"},
)

result = json.loads(response.choices[0].message.content)
print(result["title"])
print(result["markdown"][:500])

Step 5: Wrap it in a reusable class

I add a thin wrapper that validates the response schema and writes the file to disk.

import os
from datetime import datetime
from openai import OpenAI

class BlogGenerator:
    def __init__(self, api_key, model="llama-3.3-70b"):
        self.client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key=api_key)
        self.model = model

    def generate(self, topic, audience, tone, points):
        user_msg = build_user_message(topic, audience, tone, points)
        resp = self.client.chat.completions.create(
            model=self.model,
            messages=[
                {"role": "system", "content": SYSTEM_PROMPT},
                {"role": "user", "content": user_msg},
            ],
            response_format={"type": "json_object"},
        )
        data = json.loads(resp.choices[0].message.content)
        if "title" not in data or "markdown" not in data:
            raise ValueError("Missing required fields in response")
        return data

    def save(self, data, out_dir="posts"):
        os.makedirs(out_dir, exist_ok=True)
        slug = data["title"].lower().replace(" ", "-")[:50]
        filename = f"{slug}.md"
        path = os.path.join(out_dir, filename)
        with open(path, "w", encoding="utf-8") as f:
            f.write(data["markdown"])
        return path

Run it

Here is the full script wired together. I run it against a real topic to produce a draft.

import os
from openai import OpenAI

# Paste your system prompt and build_user_message from above here

if __name__ == "__main__":
    generator = BlogGenerator(api_key=os.environ.get("OXLO_API_KEY"))

    article = generator.generate(
        topic="Request-Based Pricing for LLM Inference",
        audience="Engineering managers evaluating AI infrastructure",
        tone="Direct and analytical",
        points=[
            "Why token counting creates unpredictable bills",
            "How flat per-request pricing simplifies forecasting",
            "When long-context workloads shift the cost equation"
        ]
    )

    path = generator.save(article)
    print(f"Saved to {path}")
    print("\nPreview:\n")
    print(article["markdown"][:800])

Example output:

Saved to posts/request-based-pricing-for-llm-inference.md

Preview:

---
title: "Request-Based Pricing for LLM Inference"
date: 2025-01-15
author: "AI Generator"
tags: ["llm", "infrastructure", "pricing"]
---

## Introduction

Engineering managers evaluating AI infrastructure quickly discover that not all pricing models behave the same way. Token-based billing, while granular, introduces volatility that makes monthly forecasting a headache. This post breaks down why a flat per-request model can remove that uncertainty, especially for workloads with variable context sizes.

## Why Token Counting Creates Unpredictable Bills

...

Wrap-up

Swap in qwen-3-32b or kimi-k2.6 if you need deeper reasoning for research-heavy posts. You can also extend the generator with a CLI using argparse so your team can pipe topics from a CSV and bulk-generate drafts. See https://oxlo.ai/pricing to compare plans if you start running high-volume batch jobs.

DEV Community