A Comprehensive Guide to Using Transfer Learning with LLMs

#learnai #oxlo #ai

We are going to build a support ticket classifier that adapts a general LLM to a private company taxonomy using in-context transfer learning. Instead of fine-tuning weights, we will transfer domain knowledge by feeding labeled examples directly into the prompt at inference time. This helps teams that need accurate routing today without setting up a training pipeline.

What you'll need

Python 3.10 or newer
The OpenAI SDK: pip install openai
An Oxlo.ai API key from https://portal.oxlo.ai

Step 1: Set up the Oxlo.ai client

First, import the OpenAI SDK and point it at Oxlo.ai. This gives us a fully compatible client for every model on the platform.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

Step 2: Prepare the transfer set

Next, define the target labels and a small transfer set of labeled tickets. These examples carry the domain knowledge we want the model to adopt.

LABELS = ["Account Access", "Billing", "Product How-To", "API Issue", "Bug Report"]

transfer_examples = [
    {"text": "I forgot my password and the reset email never arrives.", "label": "Account Access"},
    {"text": "My credit card was charged twice this month.", "label": "Billing"},
    {"text": "How do I schedule a report to export every Monday?", "label": "Product How-To"},
    {"text": "All my POST requests return 401 after the key rotation.", "label": "API Issue"},
    {"text": "The dashboard crashes when I open the analytics tab in Safari.", "label": "Bug Report"},
    {"text": "I need to add a new seat to my team plan.", "label": "Billing"},
    {"text": "Two-factor authentication codes are delayed by ten minutes.", "label": "Account Access"},
]

Step 3: Build the few-shot assembler

We need a helper that injects the transfer examples into the user prompt. Keeping the assembler separate makes it easy to swap in dynamic retrieval later.

def build_few_shot(examples, query):
    blocks = []
    for ex in examples:
        blocks.append(f"Ticket: {ex['text']}\nLabel: {ex['label']}")
    blocks.append(f"Ticket: {query}\nLabel:")
    return "\n\n".join(blocks)

Step 4: Define the system prompt and classifier

Now we write the system prompt and the classifier function. I keep temperature low and cap tokens so the model returns only the label.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

SYSTEM_PROMPT = """You are a support triage model.
Read the ticket below and classify it into exactly one label.
Allowed labels: Account Access, Billing, Product How-To, API Issue, Bug Report.
Respond with only the label, no explanation."""

def classify_ticket(query):
    few_shot_text = build_few_shot(transfer_examples, query)

    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": few_shot_text},
        ],
        temperature=0.1,
        max_tokens=20,
    )
    return response.choices[0].message.content.strip()

Run it

Here is how the finished agent handles three unseen tickets. The few-shot context transfers the taxonomy without any weight updates.

unseen = [
    "Where is my invoice for last quarter?",
    "Getting 502 Bad Gateway on the webhook endpoint.",
    "Locked out after I changed my phone number.",
]

for ticket in unseen:
    label = classify_ticket(ticket)
    print(f"Ticket: {ticket}\nLabel: {label}\n")

Example output:

Ticket: Where is my invoice for last quarter?
Label: Billing

Ticket: Getting 502 Bad Gateway on the webhook endpoint.
Label: API Issue

Ticket: Locked out after I changed my phone number.
Label: Account Access

Wrap-up

You now have a working transfer learning pipeline that runs entirely through Oxlo.ai inference. Because Oxlo.ai uses request-based pricing, adding more few-shot examples to the context does not raise the per-request cost, which makes this approach especially cheap compared to token-based providers for long-context transfer sets.

Two concrete next steps: expand the transfer set to twenty or thirty examples for better coverage, or swap the model to qwen-3-32b if you need to classify multilingual tickets.

DEV Community