What a Fine-Tune's Data Format Requirements Are Across Providers

#ai #machinelearning #llm #programming

Moving a tuning job is not a file conversion. The container is the same on every provider — JSONL, one JSON object per line — and the object inside it encodes assumptions about roles, system prompts and turn order that do not all have a counterpart on the other side.

JSONL everywhere, different object inside

OpenAI’s supervised fine-tuning guide specifies a line whose top level is a messages array of objects carrying role and content, the same shape the chat endpoint takes at inference time. Function-calling examples add tools alongside the messages, and assistant turns that call a tool carry tool_calls. The file is uploaded through the Files API with purpose set to fine-tune, and the guide states that the minimum number of examples is ten while recommending you start at around fifty and evaluate before adding more. See OpenAI’s supervised fine-tuning guide.

Google’s supervised tuning for Gemini on Vertex AI specifies a line whose top level is contents, each entry carrying role and a parts array whose entries hold text, with an optional systemInstruction sitting beside contents rather than inside it. Roles are user and model, and turns must alternate. See Google Cloud’s page on preparing supervised fine-tuning data.

Read those two paragraphs as a diff and the work looks like a rename. It is not, because three of the differences are structural and two of them are semantic.

The five things that do not survive

The system prompt changes category. On one side it is a message in the same list as everything else; on the other it is a sibling field of the conversation. A dataset with exactly one system message at the front converts cleanly in either direction. A dataset with two system messages, or one that appears mid-dialogue because somebody re-stated the instructions after a tool result, has no faithful target representation. You will merge them, and merging changes what the model is trained to condition on.
Role names are a rename; role ordering is not. assistant to model is mechanical. Enforced alternation is not: if your source data contains two consecutive assistant turns — common when a turn was split, or when a tool result sits between them — the target rejects the line. Concatenating the two turns produces a training example that no longer shows the model where it was supposed to stop.
Content is a string on one side and a list of parts on the other. Text-only examples convert without a decision. Anything else — an image, an audio clip, a typed block — needs a per-block-type rule, and a block type with no counterpart means dropping the example rather than degrading it.
Tool-calling examples may not be expressible. The representation of a tool call inside a training example differs more than the chat shapes do, and not every tuning surface accepts tool-calling examples at all. Check that before you write the converter, not after.
Per-turn loss masking disappears quietly. If your dataset marks some assistant turns as context rather than as targets — so they are shown to the model but not trained on — confirm whether the target format has any equivalent. If it does not, that instruction is silently dropped and the turn is trained on. This is the one that produces a tuned model that has learned to imitate the wrong half of its own transcripts.

The requirement that is not a format

Before any of that, establish that the target tunes at all, and that it tunes the model you want. Google’s Gemini API documentation records that with the deprecation of Gemini 1.5 Flash-001 in May 2025 there is no longer a model in the Gemini API or AI Studio that supports fine-tuning, with tuning available through a different surface. See Google’s model tuning page. That is a whole class of migration failure that no amount of schema work reaches: the format question was never the binding constraint.

Two more capability questions belong in the same check. Which base models in the target family accept a tune, and for how long — a fine-tuned model is a derivative of a base model, and the base model’s retirement date is your tuned model’s retirement date. And whether the tuning surface is in the region and account structure you actually deploy into, which is frequently a different project from the one you experiment in.

A converter that refuses to guess

Write the conversion as source format to a neutral intermediate to target format, not source to target directly. The intermediate is worth the extra hour because it is where you put the assertions: a direct converter has one place to be lenient and it will be lenient there, whereas an intermediate makes every unrepresentable case an explicit branch.

The rule for every branch is the same. When the target has no faithful representation, the converter raises and names the line number. It does not pick a reasonable default. A tuning dataset is one of the few artifacts where a silently-wrong record costs a whole training run before anyone notices, and the run is the expensive part.

# one line of the intermediate, before any target is chosen
{
  "system": "You are a claims triage assistant.",   # None if absent
  "turns": [
    {"speaker": "user",      "text": "..."},
    {"speaker": "assistant", "text": "...", "train_on": True},
  ],
  "source_line": 4127,
}

def to_target(rec):
    if rec["system"] is not None and target_system_field is None:
        raise Unrepresentable(rec["source_line"], "target has no system slot")
    for a, b in zip(rec["turns"], rec["turns"][1:]):
        if a["speaker"] == b["speaker"] and target_requires_alternation:
            raise Unrepresentable(rec["source_line"], "consecutive same-role turns")
    if any(not t["train_on"] for t in rec["turns"]) and not target_supports_masking:
        raise Unrepresentable(rec["source_line"], "loss masking would be dropped")
    ...

Run it once over the whole dataset and read the exception counts as a report before fixing anything. Nine hundred unrepresentable lines out of a thousand is a signal to reconsider the migration; nine is a signal to hand-edit nine records.

Validating before you spend money

Provider-side validation runs after upload and catches schema errors. It does not catch the errors that matter, so run these locally first. Assert that every line parses independently — a JSONL file that is accidentally pretty-printed JSON fails at line one and the message rarely says so. Assert the turn ordering rule the target enforces. Assert that every conversation ends on the turn the model is meant to produce, because a dataset ending on user turns trains nothing useful. Count tokens with the target’s own tokenizer rather than the source’s: the same text yields different counts under different vocabularies, which is the mechanism behind token count mismatches generally, and here it decides whether examples are silently truncated.

Then hold back a validation split before you upload anything, and hold back the same records you held back on the source provider. A tuned model on a new provider evaluated against a freshly-drawn split is not comparable to the old one, and comparability is the entire reason you are doing this. On sizing that split and the training set around it, how many examples a fine-tune actually needs is the general treatment.

Minimum example counts, per-file size caps and token caps are set by each provider and change. The ten-example minimum above is what OpenAI’s guide states at the time of writing; the file and token caps are not stated on that page, so read the current limits from the provider’s own documentation before sizing a dataset around them.