Building a Language Understanding System using LLM

#learnai #oxlo #ai

We are building a structured intent parser that turns raw customer support messages into typed JSON objects. It is useful for any team that needs to route tickets, trigger automations, or analyze trends without maintaining legacy NLP pipelines.

What you'll need

Python 3.10 or newer installed locally. The OpenAI SDK, which you can install with pip install openai. An Oxlo.ai API key from https://portal.oxlo.ai. Because Oxlo.ai charges a flat rate per request rather than per token, you can iterate with long prompts and large batch inputs without input length costs driving up your bill. See https://oxlo.ai/pricing for plan details.

Step 1: Configure the Oxlo.ai client

I start every project by setting up the client in its own block so the rest of the script can reuse it. I use llama-3.3-70b because it handles structured extraction reliably, but Oxlo.ai also hosts qwen-3-32b and deepseek-v3.2 if you want to experiment with different reasoning styles.

import json
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.getenv("OXLO_API_KEY", "YOUR_OXLO_API_KEY")
)

Step 2: Define the system prompt

The system prompt is the only schema definition I need. I keep it strict, give one inline example, and remind the model to avoid markdown so I do not have to parse fences later.

SYSTEM_PROMPT = """You are an NLU engine. Extract the following from the user's message and return ONLY a JSON object with no markdown formatting.

Fields:
- intent: one of [refund_request, technical_issue, billing_question, general_inquiry]
- entities: an object with any relevant keys such as order_id, product_name, or email
- sentiment: one of [positive, neutral, negative]
- urgency: one of [low, medium, high]

Example:
{"intent": "refund_request", "entities": {"order_id": "ORD-12345"}, "sentiment": "negative", "urgency": "high"}
"""

Step 3: Build the extraction function

Next I write the extraction wrapper. It calls the model, strips any accidental markdown fences, and returns a Python dict.

def parse_message(user_message: str) -> dict:
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_message},
        ],
    )
    raw = response.choices[0].message.content.strip()
    if raw.startswith("

```"):
        raw = raw.split("\n", 1)[1].rsplit("```

", 1)[0].strip()
    return json.loads(raw)

Step 4: Harden with retries and validation

In production I have seen malformed JSON on edge cases, so I add a retry loop and a fallback dictionary. This keeps the pipeline from crashing on a single bad response.

def parse_message(user_message: str, retries: int = 1) -> dict:
    for attempt in range(retries + 1):
        try:
            response = client.chat.completions.create(
                model="llama-3.3-70b",
                messages=[
                    {"role": "system", "content": SYSTEM_PROMPT},
                    {"role": "user", "content": user_message},
                ],
            )
            raw = response.choices[0].message.content.strip()
            if raw.startswith("

```"):
                raw = raw.split("\n", 1)[1].rsplit("```

", 1)[0].strip()
            data = json.loads(raw)
            assert "intent" in data and "entities" in data
            return data
        except (json.JSONDecodeError, AssertionError, KeyError):
            if attempt == retries:
                return {
                    "intent": "unknown",
                    "entities": {},
                    "sentiment": "neutral",
                    "urgency": "low",
                    "error": True,
                }
            user_message = user_message + "\n\nReturn only raw JSON."

Step 5: Batch process inputs

Finally I wire it to a small batch runner so I can process a list of tickets in one go.

if __name__ == "__main__":
    tickets = [
        "I was charged twice for my subscription this month and I need the extra payment refunded immediately.",
        "How do I reset my password? I can't log in.",
        "Love the new dashboard, but I noticed a small bug in the export button.",
    ]

    for t in tickets:
        result = parse_message(t)
        print(f"Input: {t[:50]}...")
        print(json.dumps(result, indent=2))
        print()

Run it

Save the full script as nlu.py, export your OXLO_API_KEY, and run python nlu.py. The output should look similar to this.

Input: I was charged twice for my subscription this...
{
  "intent": "billing_question",
  "entities": {
    "issue": "double charge"
  },
  "sentiment": "negative",
  "urgency": "high"
}

Input: How do I reset my password? I can't log in....
{
  "intent": "technical_issue",
  "entities": {
    "topic": "password reset"
  },
  "sentiment": "neutral",
  "urgency": "medium"
}

Input: Love the new dashboard, but I noticed a smal...
{
  "intent": "general_inquiry",
  "entities": {
    "topic": "export button bug"
  },
  "sentiment": "positive",
  "urgency": "low"
}

Next steps

Expose the parser as a FastAPI endpoint and use Oxlo.ai's flat per-request pricing to keep inference costs predictable even as input volume grows. Collect mismatches between predicted and actual intent, then run eval batches against kimi-k2.6 or deepseek-v3.2 on Oxlo.ai to see if a stronger reasoning model improves accuracy on ambiguous phrasing.

DEV Community