shashank ms

Posted on Jul 8

Building a Language Understanding Platform with LLM and NLP

#learnai #oxlo #ai

We are building a structured feedback analysis service that turns raw customer messages into JSON with intent, sentiment, product mentions, and urgency. This gives support and product teams a clean API they can drop into their stack without managing model infrastructure.

What you'll need

Python 3.10 or newer
pip install openai fastapi uvicorn
An Oxlo.ai API key from https://portal.oxlo.ai

Step 1: Bootstrap the Oxlo.ai client

I always start by verifying the connection. The Oxlo.ai endpoint is a drop-in replacement for the OpenAI SDK, so the only change is the base URL and model name.

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.environ.get("OXLO_API_KEY", "YOUR_OXLO_API_KEY")
)

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[{"role": "user", "content": "Say OK"}],
    max_tokens=10
)
print(response.choices[0].message.content)

Step 2: Write the system prompt

The system prompt is the contract between our platform and the model. It forces strict JSON output so downstream code never has to parse prose.

SYSTEM_PROMPT = """You are a language understanding engine. Analyze the user text and return a single JSON object with these keys:
- intent: the primary user intent, one of [feedback, complaint, question, praise, bug_report]
- sentiment: a float from -1.0 to 1.0
- product_mentions: a list of product or feature names mentioned
- urgency: low, medium, or high
- summary: a one-sentence summary
Do not include markdown fences or explanation. Output raw JSON only."""

Step 3: Build the extraction function

This function wraps the SDK call, strips any accidental markdown fences, and returns a Python dict. I keep temperature low because we want deterministic extraction.

import json

def analyze_text(text: str) -> dict:
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": text},
        ],
        temperature=0.1,
        max_tokens=512
    )
    raw = response.choices[0].message.content.strip()
    if raw.startswith("

```"):
        raw = raw.split("\n", 1)[1].rsplit("```

", 1)[0].strip()
    return json.loads(raw)

Step 4: Process long feedback threads

Because Oxlo.ai uses flat per-request pricing, passing a long conversation thread does not inflate cost the way token-based providers do. We can analyze the full context in one shot instead of building a chunking pipeline. See https://oxlo.ai/pricing for details.

long_thread = """
Sarah (Customer): The export button has been failing since Tuesday.
Tom (Support): Thanks for reporting. Which browser are you using?
Sarah: Chrome 124. I also tried Firefox and it hangs.
Tom: Can you check the console for errors?
Sarah: It says "Network timeout" and this is blocking our month-end reports.
"""

result = analyze_text(long_thread)
print(json.dumps(result, indent=2))

Step 5: Batch process multiple items

In production you will ingest more than one message at a time. This loop handles each item, catches parsing errors, and collects the results.

feedback_batch = [
    "The dashboard loads slowly after the last update.",
    "How do I connect my Stripe account?",
    "Love the new dark mode, great work team!"
]

outputs = []
for item in feedback_batch:
    try:
        out = analyze_text(item)
        outputs.append({"input": item, "analysis": out})
    except Exception as e:
        outputs.append({"input": item, "error": str(e)})

print(json.dumps(outputs, indent=2))

Step 6: Expose a FastAPI endpoint

A single POST endpoint turns the script into a service. I use Pydantic to validate the incoming payload so bad requests never reach the model.

from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()

class FeedbackRequest(BaseModel):
    text: str

@app.post("/analyze")
def analyze_feedback(req: FeedbackRequest):
    return analyze_text(req.text)

Run it

Start the server with Uvicorn and send a request with curl. You should get back a strict JSON object that matches the schema in the system prompt.

# terminal 1
uvicorn platform:app --reload

# terminal 2
curl -X POST http://localhost:8000/analyze \
  -H "Content-Type: application/json" \
  -d '{"text":"The export button is broken and I need this fixed before month-end."}'

Expected response:

{
  "intent": "bug_report",
  "sentiment": -0.8,
  "product_mentions": ["export button"],
  "urgency": "high",
  "summary": "User reports a broken export button and needs it fixed urgently before month-end."
}

Wrap up

You now have a working language understanding API. A solid next step is to persist the JSON results in Postgres and build a Grafana dashboard on top. Another is to pipe the summaries through Oxlo.ai embeddings to cluster similar feedback by semantic meaning.

DEV Community