LLM Models for Code Generation and Completion

#engineering #oxlo #ai

I needed a single script that could generate boilerplate, complete half-written functions, and explain legacy code without burning through token budgets on long files. In this tutorial, I will walk through building a lightweight code assistant that routes tasks to different Oxlo.ai models using flat per-request pricing, which keeps costs predictable even when I feed it entire modules.

What you'll need

Before starting, grab Python 3.10 or newer and install the OpenAI SDK. You will also need an Oxlo.ai API key from https://portal.oxlo.ai.

pip install openai

Step 1: Initialize the Oxlo.ai client

I always start by verifying the connection with a simple completion call. I use DeepSeek V3.2 because it handles coding tasks well and sits on the free tier, so experimenting costs nothing.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

response = client.chat.completions.create(
    model="deepseek-v3.2",
    messages=[
        {"role": "system", "content": "You are a terse coding assistant."},
        {"role": "user", "content": "Complete this Python function:\n\ndef fib(n):"},
    ],
)

print(response.choices[0].message.content)

Step 2: Define task-specific system prompts

Next, I write explicit system prompts for each mode. Keeping them in a dictionary makes it easy to tune behavior without touching the rest of the logic.

SYSTEM_PROMPTS = {
    "complete": (
        "You are an expert code completion engine. "
        "Continue the provided code snippet with minimal explanation. "
        "Output only the code that should be inserted, preserving the original indentation."
    ),
    "explain": (
        "You are a senior engineer documenting a codebase. "
        "Explain the provided code in plain English, noting any edge cases or performance issues."
    ),
    "debug": (
        "You are a meticulous debugger. "
        "Identify bugs in the provided code, explain the root cause, and provide a corrected version."
    ),
    "document": (
        "You are a technical writer generating docstrings and inline documentation. "
        "Add concise Google-style docstrings and type hints where appropriate."
    ),
}

Step 3: Build the model router

Different Oxlo.ai models excel at different tasks. I route completion requests to DeepSeek V3.2, explanations to Llama 3.3 70B, debugging to Kimi K2.6, and documentation to Qwen 3 32B. This mapping lives in a simple dictionary.

MODEL_ROUTER = {
    "complete": "deepseek-v3.2",
    "explain": "llama-3.3-70b",
    "debug": "kimi-k2.6",
    "document": "qwen-3-32b",
}

def run_task(task: str, code: str) -> str:
    if task not in MODEL_ROUTER:
        raise ValueError(f"Unknown task: {task}")

    model = MODEL_ROUTER[task]
    system_prompt = SYSTEM_PROMPTS[task]

    response = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": f"

```{code}```

"},
        ],
        temperature=0.2,
    )

    return response.choices[0].message.content

Step 4: Add streaming for real-time output

Waiting for the full response slows down the loop. I switch on streaming so tokens arrive as they are generated, which is especially useful for long completions.

def run_task_stream(task: str, code: str):
    if task not in MODEL_ROUTER:
        raise ValueError(f"Unknown task: {task}")

    model = MODEL_ROUTER[task]
    system_prompt = SYSTEM_PROMPTS[task]

    stream = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": f"

```{code}```

"},
        ],
        temperature=0.2,
        stream=True,
    )

    for chunk in stream:
        content = chunk.choices[0].delta.content
        if content:
            print(content, end="", flush=True)
    print()

Step 5: Wire everything into a CLI

Finally, I add a small argparse interface so I can pipe code directly from editors or version control diffs. The script reads from stdin and prints the result.

import argparse
import sys
from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

SYSTEM_PROMPTS = {
    "complete": (
        "You are an expert code completion engine. "
        "Continue the provided code snippet with minimal explanation. "
        "Output only the code that should be inserted, preserving the original indentation."
    ),
    "explain": (
        "You are a senior engineer documenting a codebase. "
        "Explain the provided code in plain English, noting any edge cases or performance issues."
    ),
    "debug": (
        "You are a meticulous debugger. "
        "Identify bugs in the provided code, explain the root cause, and provide a corrected version."
    ),
    "document": (
        "You are a technical writer generating docstrings and inline documentation. "
        "Add concise Google-style docstrings and type hints where appropriate."
    ),
}

MODEL_ROUTER = {
    "complete": "deepseek-v3.2",
    "explain": "llama-3.3-70b",
    "debug": "kimi-k2.6",
    "document": "qwen-3-32b",
}

def run_task_stream(task: str, code: str):
    if task not in MODEL_ROUTER:
        raise ValueError(f"Unknown task: {task}")

    model = MODEL_ROUTER[task]
    system_prompt = SYSTEM_PROMPTS[task]

    stream = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": f"

```{code}```

"},
        ],
        temperature=0.2,
        stream=True,
    )

    for chunk in stream:
        content = chunk.choices[0].delta.content
        if content:
            print(content, end="", flush=True)
    print()

def main():
    parser = argparse.ArgumentParser(description="Oxlo.ai code assistant")
    parser.add_argument("--task", choices=MODEL_ROUTER.keys(), default="complete")
    args = parser.parse_args()

    print("Reading code from stdin...", file=sys.stderr)
    code = sys.stdin.read()

    run_task_stream(args.task, code)

if __name__ == "__main__":
    main()

Run it

Save the full script as assistant.py, replace the placeholder key, and pipe a partial function into it.

$ cat <<'EOF' | python assistant.py --task complete
def fetch_user(session, user_id):
    # TODO: implement
EOF

DeepSeek V3.2 streams back the implementation in real time:

Reading code from stdin...
    stmt = select(User).where(User.id == user_id)
    result = session.execute(stmt)
    return result.scalar_one_or_none()

Wrap-up

The assistant is already useful, but two upgrades stand out. First, add Oxlo.ai's embedding models to index your internal codebase, then prepend the most relevant snippets as context for each request. Second, wire the script into a Git pre-commit hook to auto-generate docstrings with Qwen 3 32B on every diff. Both are cheap to experiment with because Oxlo.ai charges per request, not per token, so long files do not inflate the bill. See https://oxlo.ai/pricing for the latest plan details.

DEV Community