LLM for Text Generation Tasks: A Beginner's Guide

#learnai #oxlo #ai

We will build a topic-to-draft generator that takes a one-line brief and returns a structured blog post with a headline, summary, and body. It is useful for content pipelines, internal documentation, or any workflow where you need clean prose on demand without surprises in your bill.

What you'll need

Python 3.10 or newer
The OpenAI SDK: pip install openai
An Oxlo.ai API key from https://portal.oxlo.ai
A virtual environment (optional but recommended)

Step 1: Configure the Oxlo.ai client

I start by importing the OpenAI SDK and pointing it at Oxlo.ai. Because Oxlo.ai is fully OpenAI-compatible, this is a drop-in replacement. No custom clients or extra wrappers are needed.

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.environ.get("OXLO_API_KEY")
)

Step 2: Define the system prompt

The system prompt is the only training the model gets. I keep it explicit about format, tone, and length so the output is predictable. I store it as a constant so I can iterate without touching the rest of the code.

SYSTEM_PROMPT = """You are a technical writing assistant.
When given a topic and a target audience, produce a structured draft with exactly three sections:
1. HEADLINE: a clear, specific title.
2. SUMMARY: one paragraph, max three sentences.
3. BODY: two to three paragraphs of plain, concrete prose. No filler, no bullet points unless requested.
Tone: helpful and direct. Avoid hype and superlatives."""

Step 3: Create the generation function

I wrap the API call in a small function so the rest of my script stays clean. I use Llama 3.3 70B, which is Oxlo.ai's general-purpose flagship. Because Oxlo.ai uses request-based pricing instead of token-based pricing, I do not need to worry about the prompt length eating my budget if I decide to add examples later.

def generate_draft(topic: str, audience: str) -> str:
    user_message = f"Topic: {topic}\nAudience: {audience}"

    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_message},
        ],
        temperature=0.7,
        max_tokens=1024,
    )
    return response.choices[0].message.content

Step 4: Add JSON mode for structured parsing

Parsing free text in production is fragile. I switch to JSON mode so the model returns a machine-readable object. I update the system prompt to request valid JSON, then set response_format={"type": "json_object"}. Oxlo.ai supports this on compatible models, so the integration stays simple.

SYSTEM_PROMPT_JSON = """You are a technical writing assistant.
When given a topic and a target audience, return a single JSON object with these keys:
- headline: string
- summary: string, max 3 sentences
- body: string, 2 to 3 paragraphs, plain prose
Tone: helpful and direct. Avoid hype."""

def generate_draft_json(topic: str, audience: str) -> dict:
    user_message = f"Topic: {topic}\nAudience: {audience}"

    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT_JSON},
            {"role": "user", "content": user_message},
        ],
        response_format={"type": "json_object"},
        temperature=0.7,
    )
    import json
    return json.loads(response.choices[0].message.content)

Step 5: Build a small CLI runner

I add a simple guard block so I can run the script directly, pass in a topic, and print the result. This keeps the tutorial self-contained.

if __name__ == "__main__":
    topic = "Using request-based pricing for long-context LLM workloads"
    audience = "senior platform engineers evaluating inference providers"

    draft = generate_draft_json(topic, audience)

    print("HEADLINE:")
    print(draft["headline"])
    print("\nSUMMARY:")
    print(draft["summary"])
    print("\nBODY:")
    print(draft["body"])

Run it

Save the full script as draft_generator.py, set your key, and run python draft_generator.py. You should see output similar to this:

HEADLINE:
Why Per-Request Pricing Cuts Costs for Long-Context LLM Workloads

SUMMARY:
Token-based billing penalizes prompts that include large context windows or multi-turn agent traces. A flat per-request model keeps costs predictable even as input length grows.

BODY:
Platform teams running retrieval-augmented generation pipelines often embed thousands of tokens into every prompt. Under token-based pricing, that context window becomes a meter that never stops running. Oxlo.ai uses request-based pricing so your bill scales with API calls, not with word count.

For agentic workflows that chain multiple tool calls and reasoning steps, the savings compound quickly. You can pass full conversation histories, documentation chunks, and system prompts without rewriting your code to shave off tokens. If you want to test the difference, Oxlo.ai offers a 7-day full-access trial on the free tier.

Wrap-up

Two concrete ways to extend this. First, add a second pass with DeepSeek R1 671B on Oxlo.ai to review the draft for technical accuracy before publishing. Second, wrap the generator in a FastAPI endpoint and deploy it behind a simple queue so your content team can submit topics over HTTP. See https://oxlo.ai/pricing for plan details if you need higher daily limits.