Inference with Fine-Tuned Models: Delivering the Message

During WWII, after tirelessly sorting through mountains of letters, the women of the Six Triple Eight ensured that each piece of mail reached its correct recipient. They didn’t simply sort the letters and stop there; they completed the job by handing them off for delivery. In the world of AI, inference is that critical final stage. Once our model has been fine-tuned, we need to test it on actual input to verify that it can accurately perform the tasks it was trained for—whether that’s classification, summarization, or another specialized function.

Below is a simple example of how to perform inference using a fine-tuned OpenAI model in Python. In this scenario, we’re classifying an article into one of several categories: business, tech, entertainment, politics, or sport.

fine_tuned_model = "ft:gpt-3.5-turbo:xxxxxxxxxxxx"


system = "Classify this article into this category : business,tech,entertainment,politics,sport,tech"

user = "A new mobile phone is launched"

try:
    response = openai.completion.create(
        model=fine_tuned_model,
        messages=[
{'role':'system','content':system},
{'role':'user','content':user}
],

    )
    print("Model Response:")
    print(response.choices[0].message)
except Exception as e:
    print("Error during inference:", e)

Parallels to History

Just as the Six Triple Eight verified each letter’s destination, our inference checks that the fine-tuned model correctly identifies the category of new, incoming data. It’s the final check to ensure our “mail” (in this case, user queries) arrives at the correct destination (i.e., the correct classification).

By completing this last step, we confirm that our refined model can handle real-world tasks reliably—a testament to the time and effort spent fine-tuning the data in earlier stages.