Introduction to LLM Engineering: Fundamentals and Applications

#learnai #oxlo #ai

We are going to build a support ticket triage agent that reads an unstructured customer message, extracts structured fields like issue type and urgency, and drafts a contextual reply. This two-step pattern, extraction then generation, is the foundation of most production LLM systems. We will run the whole pipeline against Oxlo.ai using the OpenAI SDK so you can deploy it without changing client code.

What you'll need

Python 3.10 or newer.
The OpenAI SDK: pip install openai.
An Oxlo.ai API key from https://portal.oxlo.ai. Oxlo.ai uses flat per-request pricing, so experimenting with long prompts and multi-step chains does not inflate your cost the way token-based billing does. See https://oxlo.ai/pricing for current rates.

Step 1: Setup and System Prompt

Start by defining the system prompt and initializing the Oxlo.ai client. Keeping the prompt in a dedicated variable makes it easy to iterate without touching business logic.

SYSTEM_PROMPT = """You are a support ticket analyst for a hardware store.
When the user provides a customer message, extract the following fields and return them as a JSON object:
- issue_type: one of Shipping, Returns, ProductQuestion, or Complaint
- urgency: Low, Medium, or High
- order_id: the order number if present, otherwise null
- sentiment: Positive, Neutral, or Negative
Do not include markdown formatting or explanation outside the JSON."""

from openai import OpenAI
import json

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

Step 2: Structured Extraction with JSON Mode

Call the model with JSON mode to turn the raw message into structured data. Oxlo.ai supports the standard response_format parameter, so the output is guaranteed to be valid JSON that we can parse immediately.

def extract_ticket(raw_message: str):
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": raw_message},
        ],
        response_format={"type": "json_object"},
    )
    content = response.choices[0].message.content
    return json.loads(content)

raw = "I ordered 500 screws (order #44921) two weeks ago and they still have not arrived. This is holding up my entire project and I need a refund immediately."
structured = extract_ticket(raw)
print(json.dumps(structured, indent=2))

Step 3: Draft the Response

Feed the extracted fields into a second prompt to generate a reply that references the real order ID and matches the customer sentiment. Because Oxlo.ai has no cold starts on popular models, the second call returns just as fast as the first.

def draft_response(structured: dict, raw_message: str):
    prompt = f"""Draft a short, professional support reply for the message below.
Use a tone that matches the customer's sentiment ({structured['sentiment']}).
Acknowledge order {structured['order_id']} if applicable.
The issue type is {structured['issue_type']} and urgency is {structured['urgency']}.

Customer message: {raw_message}"""

    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": "You are a helpful customer support representative."},
            {"role": "user", "content": prompt},
        ],
    )
    return response.choices[0].message.content

reply = draft_response(structured, raw)
print(reply)

Step 4: Wire the Pipeline Together

Combine both steps into a single function that any service can import. I added a small guard so JSON parsing failures return a clean error instead of crashing the process.

def process_ticket(raw_message: str):
    try:
        structured = extract_ticket(raw_message)
    except json.JSONDecodeError:
        return {"error": "Failed to parse extraction", "structured": None, "reply": None}

    reply = draft_response(structured, raw_message)
    return {"structured": structured, "reply": reply}

if __name__ == "__main__":
    raw = "I ordered 500 screws (order #44921) two weeks ago and they still have not arrived. This is holding up my entire project and I need a refund immediately."
    result = process_ticket(raw)
    print("=== STRUCTURED ===")
    print(json.dumps(result["structured"], indent=2))
    print("\n=== REPLY ===")
    print(result["reply"])

Run It

Save the file as agent.py, replace YOUR_OXLO_API_KEY, and run python agent.py. You should see output similar to this:

=== STRUCTURED ===
{
  "issue_type": "Shipping",
  "urgency": "High",
  "order_id": "44921",
  "sentiment": "Negative"
}

=== REPLY ===
I am sorry to hear about the delay with order #44921. I completely understand how frustrating this is, especially when it is blocking your project. I have escalated this to our shipping team and initiated a refund. You should see the credit within 2 business days.

Next Steps

Replace the hardcoded extraction with function calling so the agent can query your order database in real time. Then expose the process_ticket function through a FastAPI endpoint and host it on Oxlo.ai for a fully integrated support stack.

DEV Community