Optimizing LLM Model Training Data for Better Performance

#learnai #oxlo #ai

Most fine-tuning projects fail because of noisy training data, not model choice. I recently built a small pipeline that scores, filters, and rewrites raw instruction-response pairs into a clean dataset ready for supervised fine-tuning. I run the evaluator on Oxlo.ai because its flat per-request pricing means I can pass in long JSON documents and full context windows without token costs ballooning.

What you'll need

Python 3.10 or newer
pip install openai
An Oxlo.ai API key from https://portal.oxlo.ai

Step 1: Prepare the raw dataset

I start with five messy instruction-response pairs. Two are decent, one is off-topic, and two are underwhelming. I save them as raw_data.jsonl so the pipeline has something realistic to chew on.

import json

raw_examples = [
    {"instruction": "Explain Python list comprehensions.", "response": "List comprehensions are a way to make lists. [x for x in range(10)]"},
    {"instruction": "What is the capital of France?", "response": "Paris is the capital city of France and it is known for the Eiffel Tower."},
    {"instruction": "Write a haiku about clouds.", "response": "Clouds are white and fluffy, they float in the sky, I like clouds a lot"},
    {"instruction": "Debug this code: print(x)", "response": "You need to define x first."},
    {"instruction": "Summarize the attached 10-page legal contract.", "response": "The weather is nice today."},
]

with open("raw_data.jsonl", "w") as f:
    for ex in raw_examples:
        f.write(json.dumps(ex) + "\n")

print(f"Wrote {len(raw_examples)} examples to raw_data.jsonl")

Step 2: Define the scoring agent and system prompt

The core of the pipeline is an agent that returns strict JSON scores. I use Qwen 3 32B on Oxlo.ai because its reasoning ability is solid for evaluation tasks, and the flat per-request pricing keeps costs predictable even when I batch-process large files.

Here is the system prompt I use:

SYSTEM_PROMPT = """You are a training-data quality evaluator. You rate instruction-response pairs on three axes: relevance (does the response answer the instruction?), correctness (is the information accurate?), and clarity (is the text well-formed and unambiguous?). Return ONLY a JSON object with keys: relevance, correctness, clarity, each an integer 1-10. Do not add markdown or explanation."""

And the scoring function:

from openai import OpenAI
import json

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

def score_example(instruction: str, response: str) -> dict:
    user_message = f"Instruction: {instruction}\nResponse: {response}"
    response = client.chat.completions.create(
        model="qwen-3-32b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_message},
        ],
        temperature=0.1,
    )
    content = response.choices[0].message.content
    return json.loads(content)

# Quick sanity check
print(score_example("What is 2+2?", "4"))

Step 3: Score the dataset and filter

I process every example and keep only the ones with an average score of 8 or higher. I also stash the rejects so I can try to save them in the next step.

def load_jsonl(path):
    with open(path) as f:
        return [json.loads(line) for line in f]

raw = load_jsonl("raw_data.jsonl")
cleaned = []
rejected = []

for ex in raw:
    scores = score_example(ex["instruction"], ex["response"])
    avg = sum(scores.values()) / len(scores)
    ex["scores"] = scores
    ex["avg_score"] = avg
    if avg >= 8:
        cleaned.append(ex)
    else:
        rejected.append(ex)

print(f"Kept {len(cleaned)}, rejected {len(rejected)}")
for r in rejected:
    print(f"  - {r['instruction'][:50]}... (avg {r['avg_score']:.1f})")

Step 4: Rewrite borderline rejects

Some rejects are fixable. I run a second pass with Llama 3.3 70B to rewrite low-quality pairs into clear, accurate versions, then re-score them. Anything that hits the 8-point threshold gets promoted back into the clean set.

def rewrite_pair(instruction: str, response: str) -> dict:
    user_message = f"Rewrite this into a high-quality instruction-response pair.\nInstruction: {instruction}\nResponse: {response}"
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": "You are a data-cleaning assistant. Rewrite low-quality instruction-response pairs into clear, accurate versions. Return ONLY JSON with keys: instruction, response."},
            {"role": "user", "content": user_message},
        ],
        temperature=0.3,
    )
    return json.loads(response.choices[0].message.content)

recovered = []
for ex in rejected:
    try:
        rewritten = rewrite_pair(ex["instruction"], ex["response"])
        new_scores = score_example(rewritten["instruction"], rewritten["response"])
        avg = sum(new_scores.values()) / len(new_scores)
        if avg >= 8:
            rewritten["scores"] = new_scores
            rewritten["avg_score"] = avg
            recovered.append(rewritten)
    except Exception:
        continue

cleaned.extend(recovered)
print(f"Recovered {len(recovered)} examples. Final clean count: {len(cleaned)}")

Step 5: Export to JSONL for fine-tuning

The last step formats the cleaned data into the standard chat-completion JSONL structure that training frameworks expect.

def to_chat_jsonl(examples, out_path):
    with open(out_path, "w") as f:
        for ex in examples:
            messages = [
                {"role": "user", "content": ex["instruction"]},
                {"role": "assistant", "content": ex["response"]},
            ]
            f.write(json.dumps({"messages": messages}) + "\n")

to_chat_jsonl(cleaned, "training_data.jsonl")
print("Exported training_data.jsonl")

Run it

I execute the script and watch the pipeline do its work.

$ python optimize_data.py
Wrote 5 examples to raw_data.jsonl
{'relevance': 10, 'correctness': 10, 'clarity': 10}
Kept 2, rejected 3
  - Explain Python list comprehensions... (avg 7.3)
  - Write a haiku about clouds... (avg 5.0)
  - Summarize the attached 10-page le... (avg 2.3)
Recovered 2 examples. Final clean count: 4
Exported training_data.jsonl

The final training_data.jsonl now contains four high-quality turns. The off-topic legal-contract example was too far gone to recover, which is exactly what I want.

Wrap-up and next steps

This pipeline gives you a repeatable way to improve training data without manual review. Two concrete moves from here: first, add semantic deduplication by generating embeddings with Oxlo.ai's BGE-Large endpoint and dropping near-duplicates before fine-tuning. Second, wrap the scorer into an evaluation loop for your fine-tuned model so you can keep iterating on data quality and model outputs together.

DEV Community