I needed a small, clean instruction dataset to fine-tune a model on internal API docs. Writing thousands of examples by hand was not an option, so I built a generator that uses a large model to produce and verify training pairs. The pipeline runs entirely on Oxlo.ai, which keeps costs flat because I pay per request instead of per token.
What you'll need
- Python 3.10 or newer
- An Oxlo.ai API key from https://portal.oxlo.ai
- The OpenAI SDK:
pip install openai
Step 1: Configure the Oxlo.ai client
I always verify connectivity before adding logic. This snippet hits Oxlo.ai with Llama 3.3 70B to confirm the key and base URL are correct.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
system_prompt = "You are a helpful assistant."
user_message = "Say 'Connection OK' and nothing else."
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_message},
],
)
print(response.choices[0].message.content)
Step 2: Define the generator prompt
This system prompt locks the output format. It forces strict JSON so I do not have to wrestle with markdown or explanations.
SYSTEM_PROMPT = (
"You are a training data generator. Given a topic, output a single JSON object "
"with no markdown formatting. The object must contain keys: 'instruction', 'input', "
"and 'output'. The 'instruction' is a clear task. The 'input' provides brief context. "
"The 'output' is the correct, concise answer. Do not include any text outside the JSON."
)
Step 3: Generate raw examples
I loop over topic seeds and call DeepSeek V3.2. It is available on Oxlo.ai's free tier, so I can experiment without burning budget.
import json
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = (
"You are a training data generator. Given a topic, output a single JSON object "
"with no markdown formatting. The object must contain keys: 'instruction', 'input', "
"and 'output'. The 'instruction' is a clear task. The 'input' provides brief context. "
"The 'output' is the correct, concise answer. Do not include any text outside the JSON."
)
topics = [
"Python list comprehensions",
"HTTP status codes 401 vs 403",
"Regular expression capture groups",
]
raw_examples = []
for topic in topics:
user_message = f"Generate one training example about: {topic}"
response = client.chat.completions.create(
model="deepseek-v3.2",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user_message},
],
)
raw_examples.append(response.choices[0].message.content)
print(f"Generated {len(raw_examples)} raw examples")
Step 4: Validate and filter
Raw generations sometimes drift. I parse each response and run it past Qwen 3 32B in a second pass to catch bad JSON or weak answers.
import json
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
VALIDATOR_PROMPT = (
"You validate training data. Given a JSON training example, reply with only VALID or INVALID. "
"Reply INVALID if the JSON is malformed, the answer is factually wrong, or the instruction is ambiguous. "
"Otherwise reply VALID."
)
clean_dataset = []
for raw in raw_examples:
try:
parsed = json.loads(raw)
check = client.chat.completions.create(
model="qwen-3-32b",
messages=[
{"role": "system", "content": VALIDATOR_PROMPT},
{"role": "user", "content": json.dumps(parsed)},
],
)
if "VALID" in check.choices[0].message.content:
clean_dataset.append(parsed)
except json.JSONDecodeError:
continue
print(f"Kept {len(clean_dataset)} validated examples")
Step 5: Export to JSONL
Most training frameworks expect newline-delimited JSON. I dump the verified list to disk.
with open("training_data.jsonl", "w") as f:
for ex in clean_dataset:
f.write(json.dumps(ex) + "\n")
print("Saved to training_data.jsonl")
Run it
I save the full script as generate_data.py and run it from the terminal. The result is a file ready for Axolotl or Unsloth.
import json
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = (
"You are a training data generator. Given a topic, output a single JSON object "
"with no markdown formatting. The object must contain keys: 'instruction', 'input', "
"and 'output'. The 'instruction' is a clear task. The 'input' provides brief context. "
"The 'output' is the correct, concise answer. Do not include any text outside the JSON."
)
VALIDATOR_PROMPT = (
"You validate training data. Given a JSON training example, reply with only VALID or INVALID. "
"Reply INVALID if the JSON is malformed, the answer is factually wrong, or the instruction is ambiguous. "
"Otherwise reply VALID."
)
topics = [
"Python list comprehensions",
"HTTP status codes 401 vs 403",
"Regular expression capture groups",
]
raw_examples = []
for topic in topics:
user_message = f"Generate one training example about: {topic}"
response = client.chat.completions.create(
model="deepseek-v3.2",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user_message},
],
)
raw_examples.append(response.choices[0].message.content)
clean_dataset = []
for raw in raw_examples:
try:
parsed = json.loads(raw)
check = client.chat.completions.create(
model="qwen-3-32b",
messages=[
{"role": "system", "content": VALIDATOR_PROMPT},
{"role": "user", "content": json.dumps(parsed)},
],
)
if "VALID" in check.choices[0].message.content:
clean_dataset.append(parsed)
except json.JSONDecodeError:
continue
with open("training_data.jsonl", "w") as f:
for ex in clean_dataset:
f.write(json.dumps(ex) + "\n")
print(f"Generated {len(clean_dataset)} examples. Saved to training_data.jsonl")
Example output:
$ python generate_data.py
Generated 3 examples. Saved to training_data.jsonl
$ cat training_data.jsonl
{"instruction": "Explain what a Python list comprehension is and give an example.", "input": "Topic: Python list comprehensions", "output": "A list comprehension is a concise way to create lists in Python. Example: squares = [x**2 for x in range(10)]"}
{"instruction": "Distinguish between HTTP 401 and HTTP 403 status codes.", "input": "Topic: HTTP status codes 401 vs 403", "output": "401 Unauthorized means authentication is required or has failed. 403 Forbidden means the server understood the request but refuses to authorize it."}
{"instruction": "Describe what a capture group is in regular expressions.", "input": "Topic: Regular expression capture groups", "output": "A capture group is a part of a regex pattern enclosed in parentheses that stores the matched substring for later retrieval."}
Next steps
Swap the topic list for your internal wiki pages, or replace the validator with a programmatic JSON Schema check to speed things up. If you scale to thousands of examples, Oxlo.ai's request-based pricing means the cost does not balloon as your prompts grow longer. You can view the latest plans at https://oxlo.ai/pricing.
Top comments (0)