Engineering LLM for Agentic Workloads: Lessons Learned and Best Practices

#engineering #oxlo #ai

I recently shipped a delivery-support agent that handles WISMO queries using a multi-turn tool-calling loop. The agent extracts an order ID, calls a mock warehouse API, and writes a contextual response. Below is the exact code I run in production, adapted to use Oxlo.ai so you can reproduce it in under ten minutes.

What you'll need

Python 3.10 or newer
pip install openai
An Oxlo.ai API key from https://portal.oxlo.ai
No third-party services or vector databases required

Step 1: Configure the Oxlo.ai client

I use the OpenAI SDK because Oxlo.ai exposes a fully compatible API. One client handles chat, tools, and streaming if I need it later. Drop in your API key from the Oxlo.ai portal.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

Step 2: Write the system prompt and tool schema

The system prompt is the contract. It tells the model when to ask for missing data, when to call the tool, and how to format the final answer. I keep it explicit to reduce hallucinated arguments.

SYSTEM_PROMPT = """You are a delivery support agent for a furniture store.
Your job is to answer "Where is my order?" questions.

Process:
1. If the user did not provide an order ID, ask for it.
2. Once you have an order ID, call get_order_status with the exact ID.
3. Use the tool result to write a concise, helpful response.

Rules:
- Order IDs must match the format ORD- followed by 6 alphanumeric characters.
- Do not invent tracking numbers, carriers, or dates.
- If the user is frustrated, acknowledge the inconvenience and stay factual."""

TOOLS = [
    {
        "type": "function",
        "function": {
            "name": "get_order_status",
            "description": "Look up the current status of a customer order",
            "parameters": {
                "type": "object",
                "properties": {
                    "order_id": {
                        "type": "string",
                        "description": "The order ID, e.g. ORD-7A8B9C"
                    }
                },
                "required": ["order_id"]
            }
        }
    }
]

Step 3: Add the mock backend and validator

I do not want the agent to hit a real warehouse API during testing, so I wrote a small stub. I also added a regex validator that runs before any tool call. This prevents malformed IDs from leaking into downstream systems.

import json
import re

def is_valid_order_id(oid: str) -> bool:
    return bool(re.match(r"^ORD-[A-Z0-9]{6}$", oid))

def get_order_status(order_id: str) -> dict:
    # Simulated warehouse response
    if order_id == "ORD-123456":
        return {
            "status": "shipped",
            "carrier": "FastFreight",
            "eta_days": 2,
            "last_location": "Memphis, TN"
        }
    return {
        "status": "processing",
        "eta_days": 5,
        "note": "Awaiting final packaging at warehouse"
    }

Step 4: Build the agent loop

This is the engine. I send the conversation to kimi-k2.6 on Oxlo.ai because it handles tool use and multi-turn reasoning reliably. If the model returns a tool call, I execute the function, append the result, and call the model again so it can synthesize the final answer. I cap the loop at five turns to avoid runaway recursion.

def run_agent(user_message: str) -> str:
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": user_message}
    ]

    for turn in range(5):
        response = client.chat.completions.create(
            model="kimi-k2.6",
            messages=messages,
            tools=TOOLS,
            tool_choice="auto"
        )

        message = response.choices[0].message

        assistant_msg = {
            "role": "assistant",
            "content": message.content or "",
        }
        if message.tool_calls:
            assistant_msg["tool_calls"] = [
                {
                    "id": tc.id,
                    "type": tc.type,
                    "function": {
                        "name": tc.function.name,
                        "arguments": tc.function.arguments
                    }
                }
                for tc in message.tool_calls
            ]
        messages.append(assistant_msg)

        if not message.tool_calls:
            return message.content

        for tc in message.tool_calls:
            fn_name = tc.function.name
            args = json.loads(tc.function.arguments)

            if fn_name == "get_order_status":
                if not is_valid_order_id(args.get("order_id", "")):
                    result = {"error": "Invalid order ID format."}
                else:
                    result = get_order_status(args["order_id"])
            else:
                result = {"error": f"Tool {fn_name} not found."}

            messages.append({
                "role": "tool",
                "tool_call_id": tc.id,
                "content": json.dumps(result)
            })

    return "Agent reached the maximum number of turns."

Run it

Test the agent with a typical customer query. The first call extracts the intent and order ID, the second call receives the warehouse payload and writes the reply.

if __name__ == "__main__":
    query = "Where is my order? My ID is ORD-123456."
    print(f"User: {query}")
    reply = run_agent(query)
    print(f"Agent: {reply}")

Example output:

User: Where is my order? My ID is ORD-123456.
Agent: Your order ORD-123456 has shipped via FastFreight and is currently in Memphis, TN. You should receive it within 2 days.

Wrap-up and next steps

That loop is the minimal viable agent. Two concrete improvements I plan to add next:

Persistent memory. Replace the in-memory messages list with a thread store backed by Redis or SQLite so returning customers do not have to repeat their order IDs.
Long-context evaluation logs. I log every turn for offline analysis. Because Oxlo.ai uses flat request-based pricing, those logs can include the full conversation history without inflating cost per turn. See https://oxlo.ai/pricing for details.

DEV Community