DEV Community

shashank ms
shashank ms

Posted on

Using LLM for Text Generation: Best Practices and Techniques

Today we are building a release note generator that turns raw git commits into polished Markdown summaries. I use this to draft weekly changelogs without staring at a blank page. Because Oxlo.ai uses flat per-request pricing, loading an entire week of verbose commit history does not inflate the cost.

What you'll need

Step 1: Instantiate the Oxlo.ai client

First, I import the OpenAI SDK and point it at Oxlo.ai. The base URL and key are the only differences from the standard setup.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Say 'Connection OK'"},
    ],
)

print(response.choices[0].message.content)

Step 2: Define the system prompt

The system prompt is the contract. I keep it in a dedicated constant so I can iterate without touching the rest of the code.

SYSTEM_PROMPT = """You are a technical writer drafting release notes.
Rules:
- Group commits into Added, Fixed, and Changed sections.
- Write in Markdown with H2 headers.
- Ignore merge commits and minor dependency bumps.
- Keep each bullet under 120 characters.
- Do not mention commit hashes."""

Step 3: Feed raw commits into the context window

I format the raw data as a plain text list. Because Oxlo.ai bills per request rather than per token, I can pass the full week of commits in one shot without worrying about context length.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

commits = [
    {"hash": "a1b2c3d", "message": "feat: add OAuth2 login flow", "author": "dev1"},
    {"hash": "e5f6g7h", "message": "fix: resolve race condition in worker queue", "author": "dev2"},
    {"hash": "i8j9k0l", "message": "chore: bump pytest to 8.2", "author": "dev3"},
]

user_message = "Here are the commits since last release:\n"
for c in commits:
    user_message += f"- {c['message']} ({c['author']})\n"

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": user_message},
    ],
)

print(response.choices[0].message.content)

Step 4: Lock the output shape with JSON mode

To make the result machine readable, I force valid JSON by setting response_format. This lets me pipe the output into other tools or a static site generator.

from openai import OpenAI
import json

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

SYSTEM_PROMPT = """You are a technical writer drafting release notes.
Return strictly valid JSON with this shape:
{
  "added": ["string"],
  "fixed": ["string"],
  "changed": ["string"]
}
Rules:
- Group items logically.
- Ignore merge commits and minor dependency bumps.
- Keep each bullet under 120 characters."""

commits = [
    {"hash": "a1b2c3d", "message": "feat: add OAuth2 login flow", "author": "dev1"},
    {"hash": "e5f6g7h", "message": "fix: resolve race condition in worker queue", "author": "dev2"},
    {"hash": "i8j9k0l", "message": "chore: bump pytest to 8.2", "author": "dev3"},
]

user_message = "Here are the commits since last release:\n"
for c in commits:
    user_message += f"- {c['message']} ({c['author']})\n"

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": user_message},
    ],
    response_format={"type": "json_object"},
)

result = json.loads(response.choices[0].message.content)
print(json.dumps(result, indent=2))

Step 5: Package the generator into a CLI script

I wrap the logic in a small script that reads commit lines from stdin. This is the version I drop into CI pipelines.

import sys
import json
from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

SYSTEM_PROMPT = """You are a technical writer drafting release notes.
Return strictly valid JSON with this shape:
{
  "added": ["string"],
  "fixed": ["string"],
  "changed": ["string"]
}
Rules:
- Group items logically.
- Ignore merge commits and minor dependency bumps.
- Keep each bullet under 120 characters."""

def generate_notes(commit_lines, model="llama-3.3-70b"):
    user_message = "Here are the commits since last release:\n" + "".join(commit_lines)

    response = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_message},
        ],
        response_format={"type": "json_object"},
    )
    return json.loads(response.choices[0].message.content)

if __name__ == "__main__":
    raw = sys.stdin.readlines()
    notes = generate_notes(raw)
    print(json.dumps(notes, indent=2))

Run it

Save the file as notes.py, create a small input file, and pipe it through.

$ printf "feat: add OAuth2 login flow (dev1)\nfix: resolve race condition in worker queue (dev2)\nchore: bump pytest to 8.2 (dev3)\nfeat: introduce caching layer for configs (dev1)\nfix: patch SQL injection in search endpoint (dev2)\n" | python notes.py

Example output:

{
  "added": [
    "OAuth2 login flow for secure third-party integrations.",
    "In-memory caching layer for hot configuration paths."
  ],
  "fixed": [
    "Race condition in worker queue that caused duplicate jobs.",
    "SQL injection vulnerability in the search endpoint."
  ],
  "changed": []
}

Next steps

Pipe the JSON output directly into a Jinja template to generate static HTML for your docs site. If you need deeper reasoning on security-related commits, route only those items through deepseek-v3.2 for a detailed impact paragraph.

Top comments (0)