We are going to build DocuLM, a documentation-writing language model that runs on top of Oxlo.ai's inference API. Rather than training weights from scratch, we will compose a specialized system prompt, few-shot examples, and a lightweight retrieval layer to turn a general-purpose LLM into a domain-specific tool. This approach is practical for teams that need consistent output without a months-long training pipeline.
What you'll need
- Python 3.10 or newer
- The OpenAI SDK:
pip install openai - An Oxlo.ai API key from
https://portal.oxlo.ai
Step 1: Connect and smoke-test the client
I always verify the connection before adding complexity. The snippet below initializes the OpenAI-compatible client against Oxlo.ai and asks Llama 3.3 70B for a quick confirmation.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[{"role": "user", "content": "Reply with exactly: connection OK"}],
)
print(response.choices[0].message.content)
Step 2: Define the system prompt
This prompt acts as the model card for our custom language model. It constrains tone, format, and output structure so the assistant behaves like a dedicated documentation generator.
SYSTEM_PROMPT = """You are DocuLM, a specialized language model that converts rough Python code into polished Markdown documentation.
Rules:
- Use a level-2 heading for the function name.
- Include a parameter table with columns: Name, Type, Description.
- Include a Returns section describing the return type and meaning.
- Use present tense and active voice.
- Output only the Markdown. Do not add conversational filler."""
Step 3: Teach style with few-shot examples
Next we give the model concrete examples of input code and desired output. Because Oxlo.ai uses flat per-request pricing, these long context blocks do not raise the cost the way token-based billing would.
FEW_SHOTS = [
{
"role": "user",
"content": "def fetch_user(user_id, include_deleted=False):\n # gets user from db\n pass",
},
{
"role": "assistant",
"content": "## fetch_user\n\nRetrieves a user record from the database.\n\n| Name | Type | Description |\n|------|------|-------------|\n| user_id | int | The unique identifier of the user. |\n| include_deleted | bool | Whether to include soft-deleted records. Defaults to False. |\n\n### Returns\n\n`dict | None`: The user record if found, otherwise None.\n",
},
{
"role": "user",
"content": "def moving_average(data, window=14):\n # calculate sma\n if len(data) < window:\n return None\n return sum(data[-window:]) / window",
},
{
"role": "assistant",
"content": "## moving_average\n\nCalculates the simple moving average over a given window.\n\n| Name | Type | Description |\n|------|------|-------------|\n| data | list[float] | A sequence of numerical values. |\n| window | int | The number of periods to average. Defaults to 14. |\n\n### Returns\n\n`float | None`: The computed average, or None if data is insufficient.\n",
},
]
Step 4: Add a retrieval context layer
To keep the model grounded in our team's style guide, we prepend a short retrieved context block. In production you might use a vector database, but a dictionary lookup is enough to show the pattern.
STYLE_GUIDES = {
"python": "Follow the Google Python Style Guide. Use type hints in descriptions when available.",
"typescript": "Follow TSDoc. Use @param and @returns tags instead of Markdown tables.",
}
def get_style_context(language: str) -> str:
guide = STYLE_GUIDES.get(language, "Write clear, concise documentation.")
return f"Style guide to follow:\n{guide}"
Step 5: Assemble the reusable class with streaming
Now we package everything into a small class. It constructs the message list and calls Oxlo.ai with streaming enabled so the user sees tokens as they arrive.
class DocuLM:
def __init__(self, api_key: str, model: str = "llama-3.3-70b"):
self.client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key=api_key)
self.model = model
def document(self, code: str, language: str = "python"):
context = get_style_context(language)
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
*FEW_SHOTS,
{"role": "user", "content": f"{context}\n\nCode to document:\n{code}"},
]
stream = self.client.chat.completions.create(
model=self.model,
messages=messages,
temperature=0.2,
stream=True,
)
for chunk in stream:
delta = chunk.choices[0].delta
if delta.content:
yield delta.content
Run it
The script below creates an instance and documents a rough function. I use Llama 3.3 70B here because it follows structured instructions reliably, but you can swap in Qwen 3 32B or DeepSeek V3.2 if you prefer.
doculm = DocuLM(api_key="YOUR_OXLO_API_KEY")
rough_code = '''def calculate_bollinger_bands(prices, window=20, num_std=2):
# compute bollinger bands
if len(prices) < window:
return None
sma = sum(prices[-window:]) / window
variance = sum((p - sma) ** 2 for p in prices[-window:]) / window
std = variance ** 0.5
return sma, sma - num_std * std, sma + num_std * std
'''
print("--- Generated Documentation ---")
for token in doculm.document(rough_code, language="python"):
print(token, end="")
print()
Example output:
## calculate_bollinger_bands
Calculates Bollinger Bands for a given price series.
| Name | Type | Description |
|------|------|-------------|
| prices | list[float] | A sequence of historical price values. |
| window | int | The lookback period for the SMA. Defaults to 20. |
| num_std | int | The number of standard deviations for the bands. Defaults to 2. |
### Returns
`tuple[float, float, float] | None`: A tuple containing the middle band (SMA), lower band, and upper band, or None if insufficient data.
Next steps
Try swapping the model string to deepseek-v3.2 or qwen-3-32b to see which architecture produces the most consistent tables for your codebase. If you deploy this into a CI pipeline, consider adding a caching layer for repeated snippets, and evaluate outputs against a small golden dataset to measure drift over time.
Top comments (0)