DEV Community

shashank ms
shashank ms

Posted on

Transfer Learning with LLM: Best Practices and Applications

#learnai #oxlo #ai

We are going to build a support ticket classifier that transfers a general LLM's language understanding to a company's private support taxonomy using in-context learning. Instead of fine-tuning, we will inject labeled examples into the prompt so the model learns the mapping from raw text to categories on the fly. This is the fastest way to ship a domain-specific NLP system without managing training infrastructure.

What you'll need

Step 1: Configure the Oxlo.ai client

First, instantiate the client pointing at Oxlo.ai's OpenAI-compatible endpoint. I use Llama 3.3 70B because it follows complex instructions and handles the long system prompt we will build.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

MODEL = "llama-3.3-70b"

Step 2: Curate the transfer examples

Transfer learning here means giving the model enough labeled examples to infer the pattern. I collected six representative support tickets mapped to departments and urgency levels. These examples carry the domain knowledge into the model without any weight updates.

TRANSFER_EXAMPLES = """
Example 1:
Ticket: "I was charged twice for my monthly subscription on April 2nd. I need a refund."
Category: Billing
Urgency: High

Example 2:
Ticket: "How do I reset my password? I forgot it."
Category: Account
Urgency: Low

Example 3:
Ticket: "The API returns a 500 error every time I call /v1/batch. This started today."
Category: Technical
Urgency: High

Example 4:
Ticket: "Can I get an invoice for last quarter?"
Category: Billing
Urgency: Low

Example 5:
Ticket: "My dashboard loads but the export button does nothing."
Category: Technical
Urgency: Medium

Example 6:
Ticket: "I want to add two more seats to my team plan."
Category: Account
Urgency: Medium
"""

Step 3: Build the system prompt

The system prompt embeds the transfer examples and enforces a strict JSON output schema. This turns the general model into a structured classifier for our domain.

SYSTEM_PROMPT = f"""You are a support ticket classifier. Your job is to read a raw customer message and output a JSON object with exactly two keys: "category" and "urgency".

Allowed categories: Billing, Technical, Account.
Allowed urgency levels: Low, Medium, High.

Use the following labeled examples to learn the mapping:

{TRANSFER_EXAMPLES}

Rules:
- Output only valid JSON. No markdown, no explanation.
- If the ticket mentions payment, invoice, charge, or refund, classify as Billing.
- If the ticket mentions API errors, crashes, or broken features, classify as Technical.
- If the ticket mentions passwords, users, seats, or plans, classify as Account.
- Assign High urgency for outages, security issues, or duplicate charges.
"""

Step 4: Write the classifier function

This function takes a ticket string, sends it to Oxlo.ai with the system prompt, and parses the JSON response. Because Oxlo.ai uses flat per-request pricing, adding all these examples does not increase the per-call cost the way token-based providers would. You can expand the prompt with dozens more examples and still pay the same single request fee.

import json

def classify_ticket(ticket_text: str):
    response = client.chat.completions.create(
        model=MODEL,
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": f"Ticket: {ticket_text}"},
        ],
        response_format={"type": "json_object"},
    )
    
    content = response.choices[0].message.content
    return json.loads(content)

Step 5: Add batch processing

In production you will classify more than one ticket. This loop handles many tickets and prints results in a readable table.

tickets = [
    "Our webhook endpoint stopped receiving events 30 minutes ago. Is there an outage?",
    "I need to downgrade from Pro to Free before the next billing cycle.",
    "Two-factor authentication SMS is not arriving on my phone.",
]

results = []
for t in tickets:
    result = classify_ticket(t)
    results.append({"ticket": t, **result})

for r in results:
    print(f"[{r['urgency']:6}] {r['category']:10} | {r['ticket'][:50]}...")

Run it

Run the script from your terminal. Here is the output I get back from Oxlo.ai using Llama 3.3 70B.

$ python classify.py

[High  ] Technical  | Our webhook endpoint stopped receiving event...
[Medium] Billing    | I need to downgrade from Pro to Free before ...
[High  ] Account    | Two-factor authentication SMS is not arrivi...

Next steps

Connect this classifier to a Slack or email ingestion script so new tickets are classified automatically as they arrive. If you need stronger multilingual performance or advanced reasoning on ambiguous tickets, swap in Qwen 3 32B or Kimi K2.6. Because Oxlo.ai offers flat per-request pricing, experimenting with longer context windows and different models does not penalize you for long prompts. Check the latest model list and request rates at https://oxlo.ai/pricing.

Top comments (0)