Introduction to LLM Transfer Learning: Concepts and Applications

#learnai #oxlo #ai

We are going to build a support ticket triage agent that demonstrates transfer learning at inference time. By repurposing a general foundation model with a strict system prompt and a few labeled examples, we can create a domain-specific classifier without training any new weights. The entire pipeline runs against Oxlo.ai's flat per-request API, so you can pack the context window with examples and still pay the same cost per call.

What you'll need

Python 3.10+
An Oxlo.ai API key from https://portal.oxlo.ai
The OpenAI SDK: pip install openai

Step 1: Establish the baseline

I always start by testing the raw model. If the base model cannot roughly guess the category without help, prompt-based transfer may not be enough and you will need actual fine-tuning.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

ticket = "I was charged twice for my subscription this month. The second charge appeared on March 15th but I only have one account."

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "user", "content": f"Classify this support ticket: {ticket}"},
    ],
)
print(response.choices[0].message.content)

Step 2: Transfer behavior with a system prompt

Now we repurpose the model from open-ended chat to structured triage. The system prompt constrains the output to our taxonomy and encodes the domain rules that define our support queue.

SYSTEM_PROMPT = """You are a support ticket classifier for a SaaS billing platform.
Your job is to read the ticket and return a JSON object with exactly these keys:
- category: one of [Billing, Account, Technical, Feature-Request]
- urgency: one of [Low, Medium, High, Critical]
- summary: a one-sentence summary of the issue

Rules:
- Be concise.
- If the ticket mentions duplicate charges, fraud, or refunds over $500, set urgency to Critical."""

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

ticket = "I was charged twice for my subscription this month. The second charge appeared on March 15th but I only have one account."

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": ticket},
    ],
)
print(response.choices[0].message.content)

Step 3: Refine with few-shot examples

Edge cases are where zero-shot transfer usually fails. By inserting labeled examples into the message list, we teach the exact decision boundaries of our domain without updating any model weights.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

SYSTEM_PROMPT = """You are a support ticket classifier for a SaaS billing platform.
Return a JSON object with keys: category, urgency, summary.
Categories: [Billing, Account, Technical, Feature-Request]
Urgency: [Low, Medium, High, Critical]
Set urgency to Critical for duplicate charges, fraud, or refunds over $500."""

ticket = "I was charged twice for my subscription this month. The second charge appeared on March 15th but I only have one account."

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": "Ticket: Login page keeps refreshing after I enter 2FA code."},
        {"role": "assistant", "content": '{"category": "Technical", "urgency": "High", "summary": "Login page infinite refresh after 2FA entry."}'},
        {"role": "user", "content": "Ticket: Can you add CSV export to the analytics dashboard?"},
        {"role": "assistant", "content": '{"category": "Feature-Request", "urgency": "Low", "summary": "Request for CSV export in analytics dashboard."}'},
        {"role": "user", "content": "Ticket: I need to update the credit card on file before tomorrow's renewal."},
        {"role": "assistant", "content": '{"category": "Billing", "urgency": "Medium", "summary": "User needs to update credit card before renewal."}'},
        {"role": "user", "content": f"Ticket: {ticket}"},
    ],
)
print(response.choices[0].message.content)

Step 4: Lock the format with JSON mode

A classifier is only useful if its output is machine readable. We enable JSON mode so the response is guaranteed valid JSON and can be fed directly into a webhook or database.

from openai import OpenAI
import json

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

SYSTEM_PROMPT = """You are a support ticket classifier for a SaaS billing platform.
Return a JSON object with keys: category, urgency, summary.
Categories: [Billing, Account, Technical, Feature-Request]
Urgency: [Low, Medium, High, Critical]
Set urgency to Critical for duplicate charges, fraud, or refunds over $500."""

ticket = "I was charged twice for my subscription this month. The second charge appeared on March 15th but I only have one account."

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": "Ticket: Login page keeps refreshing after I enter 2FA code."},
        {"role": "assistant", "content": '{"category": "Technical", "urgency": "High", "summary": "Login page infinite refresh after 2FA entry."}'},
        {"role": "user", "content": "Ticket: Can you add CSV export to the analytics dashboard?"},
        {"role": "assistant", "content": '{"category": "Feature-Request", "urgency": "Low", "summary": "Request for CSV export in analytics dashboard."}'},
        {"role": "user", "content": "Ticket: I need to update the credit card on file before tomorrow's renewal."},
        {"role": "assistant", "content": '{"category": "Billing", "urgency": "Medium", "summary": "User needs to update credit card before renewal."}'},
        {"role": "user", "content": f"Ticket: {ticket}"},
    ],
    response_format={"type": "json_object"},
)
result = json.loads(response.choices[0].message.content)
print(json.dumps(result, indent=2))

Run it

Here is the complete script wrapped in a reusable function. I use it inside a FastAPI route in production, but you can call it from a notebook just as easily.

from openai import OpenAI
import json

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

SYSTEM_PROMPT = """You are a support ticket classifier for a SaaS billing platform.
Return a JSON object with keys: category, urgency, summary.
Categories: [Billing, Account, Technical, Feature-Request]
Urgency: [Low, Medium, High, Critical]
Set urgency to Critical for duplicate charges, fraud, or refunds over $500."""

FEW_SHOTS = [
    {"role": "user", "content": "Ticket: Login page keeps refreshing after I enter 2FA code."},
    {"role": "assistant", "content": '{"category": "Technical", "urgency": "High", "summary": "Login page infinite refresh after 2FA entry."}'},
    {"role": "user", "content": "Ticket: Can you add CSV export to the analytics dashboard?"},
    {"role": "assistant", "content": '{"category": "Feature-Request", "urgency": "Low", "summary": "Request for CSV export in analytics dashboard."}'},
    {"role": "user", "content": "Ticket: I need to update the credit card on file before tomorrow's renewal."},
    {"role": "assistant", "content": '{"category": "Billing", "urgency": "Medium", "summary": "User needs to update credit card before renewal."}'},
]

def triage(ticket_text: str) -> dict:
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        *FEW_SHOTS,
        {"role": "user", "content": f"Ticket: {ticket_text}"},
    ]
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=messages,
        response_format={"type": "json_object"},
    )
    return json.loads(response.choices[0].message.content)

if __name__ == "__main__":
    ticket = "I was charged twice for my subscription this month. The second charge appeared on March 15th but I only have one account."
    print(json.dumps(triage(ticket), indent=2))

Example output:

{
  "category": "Billing",
  "urgency": "Critical",
  "summary": "Customer reports duplicate subscription charge in March."
}

Wrap-up

You now have a working transfer learning pipeline that adapts a general model to your support queue through prompting and in-context examples. Because Oxlo.ai pricing is flat per request, you can expand the few-shot set or switch to a larger context model without re-budgeting for token volume.

Two concrete next steps: integrate the classifier with Oxlo.ai function calling to auto-create Zendesk tickets, or test kimi-k2.6 on the same pipeline if you need stronger reasoning for tickets that span multiple categories.

DEV Community