DEV Community

shashank ms
shashank ms

Posted on

Building a Language Model with LLM: An Oxlo Perspective

#learnai #oxlo #ai

We're going to build a lightweight research agent that ingests technical excerpts about LLM architectures and returns structured JSON analysis. It is useful for engineers who need to digest papers or documentation without managing a full retrieval pipeline. You will end up with a script you can point at any text file.

What you'll need

Step 1: Connect to Oxlo.ai

Instantiate the OpenAI SDK pointing at Oxlo.ai. Because Oxlo.ai is fully OpenAI-compatible, this is the only setup required. I use Qwen 3 32B here because it handles reasoning and agent instructions accurately.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"
)

# Verify connectivity
response = client.chat.completions.create(
    model="qwen-3-32b",
    messages=[{"role": "user", "content": "Say OK"}],
    max_tokens=10
)
print(response.choices[0].message.content)

Step 2: Lock in the system prompt

The system prompt scopes the model to strict JSON output and prevents hallucination outside the provided text. Lock this into a constant so it stays consistent across calls.

SYSTEM_PROMPT = """You are a research assistant that analyzes excerpts about language model architectures.
Read the user excerpt and produce a single JSON object with exactly these keys:
- summary: one sentence describing the main idea.
- key_innovation: the core technical contribution.
- limitations: a list of limitations mentioned or implied.
- confidence: a float from 0.0 to 1.0 reflecting how certain you are based only on the text.
Do not include markdown formatting, only raw JSON."""

Step 3: Ingest a technical document

To keep the script self-contained, I embed a short excerpt as a string. In production you would read from a file, but this lets you run the tutorial immediately.

PAPER_EXCERPT = """
Low-Rank Adaptation (LoRA) freezes pre-trained model weights and injects trainable rank decomposition matrices into each layer of the Transformer architecture. This reduces the number of trainable parameters for downstream tasks by 10,000x and lowers the hardware barrier to entry for fine-tuning large models.
"""

Step 4: Enforce structured output with JSON mode

Now I combine the system prompt and the excerpt into a chat completion. I enable JSON mode so the response is guaranteed parseable. Oxlo.ai's flat per-request pricing is especially useful here because you can pass long excerpts in full without scaling costs by token count.

import json

def analyze_excerpt(text: str) -> dict:
    response = client.chat.completions.create(
        model="qwen-3-32b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": text},
        ],
        response_format={"type": "json_object"},
        temperature=0.1,
    )
    raw = response.choices[0].message.content
    return json.loads(raw)

result = analyze_excerpt(PAPER_EXCERPT)
print(json.dumps(result, indent=2))

Step 5: Batch process a reading list

Finally, wrap the call in a loop so you can process multiple inputs. I add a small delay between requests to avoid hitting rate limits on the free tier.

import time

EXCERPTS = [
    """Low-Rank Adaptation (LoRA) freezes pre-trained model weights and injects trainable rank decomposition matrices into each layer of the Transformer architecture. This reduces the number of trainable parameters for downstream tasks by 10,000x and lowers the hardware barrier to entry for fine-tuning large models.""",
    """Mixture of Experts (MoE) architectures route each input token to a subset of specialized feed-forward networks. This decouples total parameter count from active computation, allowing models to scale to hundreds of billions of parameters without proportional increases in inference latency."""
]

for i, excerpt in enumerate(EXCERPTS):
    try:
        out = analyze_excerpt(excerpt)
        print(f"Document {i+1}:", json.dumps(out, indent=2))
    except Exception as e:
        print(f"Document {i+1} failed: {e}")
    time.sleep(0.5)

Run it

Save the full script as analyzer.py, replace YOUR_OXLO_API_KEY, and run python analyzer.py. You should see output similar to this.

Document 1: {
  "summary": "LoRA freezes pre-trained weights and injects trainable low-rank matrices to reduce fine-tuning parameters.",
  "key_innovation": "Rank decomposition matrices reduce trainable parameters by 10,000x.",
  "limitations": [],
  "confidence": 0.95
}
Document 2: {
  "summary": "MoE architectures route tokens to subsets of parameters to scale models without linear latency growth.",
  "key_innovation": "Decoupling total parameters from active computation via token routing.",
  "limitations": ["Routing overhead", "Load balancing complexity"],
  "confidence": 0.92
}

Wrap-up and next steps

From here, you can wire the analyze_excerpt function into a FastAPI endpoint to power a documentation chatbot, or chain multiple Oxlo.ai models together by routing summaries through DeepSeek R1 for deeper critique. Because Oxlo.ai charges per request rather than per token, expanding the context window to full paper sections is a straightforward configuration change, not a pricing surprise. See https://oxlo.ai/pricing for current plan details.

Top comments (0)