What we are building
We are going to build a support ticket classifier that takes a general-purpose LLM and adapts it to a domain-specific routing task without any fine-tuning. This is transfer learning in practice: the model has already learned language and reasoning from broad pre-training, and we transfer that capability downstream using nothing but prompt design and a few examples. If you run a support queue and want to stop manually tagging tickets, this saves hours each week.
What you'll need
- An Oxlo.ai API key from https://portal.oxlo.ai
- Python 3.10 or newer
- The OpenAI SDK:
pip install openai
Step 1: Establish a baseline with the raw model
First, let us see what the base model does when we ask it to classify a ticket with no guidance. It understands English, but it has not been transferred to our schema yet, so the output is usually verbose and inconsistent.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "user", "content": "Classify this support ticket: I was charged twice for my subscription this month."},
],
)
print(response.choices[0].message.content)Running this gives a paragraph explaining billing concepts rather than a label. That is expected. The knowledge is in the model, but it has not been aimed at our task.
Step 2: Transfer with a system prompt and domain schema
Transfer learning at inference time starts with a clear system prompt. We define the categories, the rules, and the output format so the model can map its general language understanding onto our specific taxonomy.
SYSTEM_PROMPT = """You are a support ticket classifier. Your job is to read a customer message and assign exactly one category.
Categories:
- Billing: payment issues, invoices, refunds, charges
- Technical: bugs, errors, integrations, API issues
- Account: login, password, user management, access
- Feature Request: product suggestions, improvements
Respond with only the category name."""Now we send the same ticket again, this time with the system prompt in place.
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": "I was charged twice for my subscription this month."},
],
)
print(response.choices[0].message.content)The model now returns a single word like Billing. The general knowledge has been transferred to our domain, but the result is still brittle on edge cases.
Step 3: Ground the task with few-shot examples
To make the transfer robust, we provide a handful of input-output pairs inside the conversation. This is in-context learning, the fastest way to adapt a pre-trained model to a new distribution without updating weights.
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": "The API returns a 500 error when I submit a batch larger than 1000 items."},
{"role": "assistant", "content": "Technical"},
{"role": "user", "content": "Can you add dark mode to the dashboard?"},
{"role": "assistant", "content": "Feature Request"},
{"role": "user", "content": "I was charged twice for my subscription this month."},
]
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=messages,
)
print(response.choices[0].message.content)With these examples in context, the model catches nuances like the difference between a bug report and a feature ask. Because Oxlo.ai charges a flat rate per request, adding these extra tokens for few-shot context does not inflate your cost the way token-based pricing would.
Step 4: Lock the output shape with JSON mode
A classifier is only useful if downstream code can parse it. We extend the system prompt to request JSON, then set response_format to enforce valid output. We also add confidence scores and short reasoning so we can audit decisions.
import json
SYSTEM_PROMPT_JSON = SYSTEM_PROMPT + """
Respond with a JSON object containing keys: category, confidence (high|medium|low), reasoning."""
few_shot = [
{"role": "user", "content": "The API returns a 500 error when I submit a batch larger than 1000 items."},
{"role": "assistant", "content": '{"category": "Technical", "confidence": "high", "reasoning": "500 errors are server-side bugs."}'},
{"role": "user", "content": "Can you add dark mode to the dashboard?"},
{"role": "assistant", "content": '{"category": "Feature Request", "confidence": "high", "reasoning": "This is a product suggestion."}'},
]
def classify_ticket(text: str) -> dict:
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT_JSON},
*few_shot,
{"role": "user", "content": text},
],
response_format={"type": "json_object"},
)
return json.loads(response.choices[0].message.content)Run it
Here is a tricky ticket that blends account access with a billing context. The transferred model should disambiguate it correctly.
ticket = "My team cannot access the billing portal after the SSO update. It says unauthorized."
result = classify_ticket(ticket)
print(json.dumps(result, indent=2))Example output:
{
"category": "Account",
"confidence": "high",
"reasoning": "SSO login issues relate to user access and authentication, not the billing system itself."
}Wrap-up and next steps
You now have a working classifier that leverages transfer learning to turn a general LLM into a domain-specific routing engine. Because Oxlo.ai uses flat per-request pricing, you can stuff long few-shot contexts into every call without watching token meters spin up. See https://oxlo.ai/pricing for plan details.
Two concrete ways to extend this. First, wire the classify_ticket function into a webhook so new tickets are tagged automatically as they arrive. Second, swap in qwen-3-32b or kimi-k2.6 if you need stronger multilingual reasoning or vision support for screenshots attached to tickets.
Top comments (0)