Language Understanding Systems with LLM

#learnai #oxlo #ai

We are building a natural language command parser that turns unstructured DevOps requests into structured JSON. This is the core of any language understanding system: accurate intent classification and slot filling without brittle regex. Teams use this to power internal CLI assistants, Slack bots, and workflow automation.

What you'll need

Python 3.10 or newer
The OpenAI SDK installed with pip install openai pydantic
An Oxlo.ai API key from https://portal.oxlo.ai

Step 1: Define the schema

First, declare the structured output we expect. A Pydantic model acts as both documentation and runtime validation, so we fail fast if the LLM returns malformed data.

# models.py
from pydantic import BaseModel, Field
from typing import Literal

class DevOpsCommand(BaseModel):
    intent: Literal["deploy", "scale", "restart", "query_metrics", "rollback", "unknown"]
    target_service: str = Field(description="The microservice or cluster name")
    environment: Literal["staging", "production", "dev"] = "production"
    parameters: dict = Field(default_factory=dict, description="Flags, time ranges, or resource limits")
    confidence: float = Field(ge=0.0, le=1.0, description="Model certainty about this parse")

Step 2: Write the system prompt

The system prompt is the only training we need. It tells the model exactly how to map free text to our schema and warns it to return unknown when the input is ambiguous.

SYSTEM_PROMPT = """You are a precise language understanding engine for a DevOps assistant.
Your job is to parse the user's natural language command into a JSON object matching this schema:

- intent: one of deploy, scale, restart, query_metrics, rollback, unknown
- target_service: the specific service or cluster mentioned
- environment: staging, production, or dev (default to production if not specified)
- parameters: a dictionary of extra flags (e.g., {"canary": "10%", "duration": "4h"})
- confidence: a float between 0.0 and 1.0 representing your certainty

Rules:
1. If the intent is unclear or the service name is missing, set intent to "unknown" and confidence below 0.5.
2. Do not guess service names. Use exactly what the user wrote.
3. Return only valid JSON. No markdown, no explanations."""

Step 3: Build the extraction client

Now we wire the prompt to Oxlo.ai. I use llama-3.3-70b because it follows instructions reliably for structured JSON. Because Oxlo.ai uses flat per-request pricing, parsing verbose log messages costs the same as short ones, which keeps batch costs predictable.

import json
from openai import OpenAI
from models import DevOpsCommand

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

def parse_command(user_message: str) -> DevOpsCommand:
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_message},
        ],
        response_format={"type": "json_object"},
        temperature=0.1,
    )
    raw = json.loads(response.choices[0].message.content)
    return DevOpsCommand(**raw)

Step 4: Add validation and fallbacks

Real systems cannot trust every parse. If confidence is low or validation fails, we retry once with kimi-k2.6, which handles advanced reasoning and edge cases well, then fall back to unknown so the calling app can escalate to a human.

def parse_command_safe(user_message: str) -> DevOpsCommand:
    try:
        result = parse_command(user_message)
        if result.confidence >= 0.7:
            return result
    except Exception:
        pass

    # Retry with a stronger reasoning model on Oxlo.ai
    response = client.chat.completions.create(
        model="kimi-k2.6",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_message},
        ],
        response_format={"type": "json_object"},
        temperature=0.1,
    )
    raw = json.loads(response.choices[0].message.content)
    return DevOpsCommand(**raw)

Step 5: Test interactively

A quick CLI loop lets us verify behavior against real commands before integrating into a larger system.

if __name__ == "__main__":
    print("DevOps NLU parser. Type a command or 'quit'.")
    while True:
        user_input = input("> ").strip()
        if user_input.lower() in ("quit", "exit"):
            break
        parsed = parse_command_safe(user_input)
        print(parsed.model_dump_json(indent=2))

Run it

Save the files and run python parser.py. Here is a sample session with a complex, multi-part command:

$ python parser.py
DevOps NLU parser. Type a command or 'quit'.
> Deploy the payment service to production with canary rollout and 10% traffic
{
  "intent": "deploy",
  "target_service": "payment service",
  "environment": "production",
  "parameters": {
    "canary": "10%",
    "rollout_type": "canary"
  },
  "confidence": 0.95
}

Wrap-up

This parser is now a drop-in module for any automation pipeline. Two concrete next steps: expose it as a FastAPI endpoint so your Slack bot can call it, and add a retrieval step to resolve fuzzy service names against your internal service catalog before executing the command.

DEV Community