DEV Community

Cover image for Inference with Fine-Tuned Models: Delivering the Message
es404020
es404020

Posted on

1

Inference with Fine-Tuned Models: Delivering the Message

During WWII, after tirelessly sorting through mountains of letters, the women of the Six Triple Eight ensured that each piece of mail reached its correct recipient. They didn’t simply sort the letters and stop there; they completed the job by handing them off for delivery. In the world of AI, inference is that critical final stage. Once our model has been fine-tuned, we need to test it on actual input to verify that it can accurately perform the tasks it was trained for—whether that’s classification, summarization, or another specialized function.

Below is a simple example of how to perform inference using a fine-tuned OpenAI model in Python. In this scenario, we’re classifying an article into one of several categories: business, tech, entertainment, politics, or sport.




fine_tuned_model = "ft:gpt-3.5-turbo:xxxxxxxxxxxx"


system = "Classify this article into this category : business,tech,entertainment,politics,sport,tech"

user = "A new mobile phone is launched"

try:
    response = openai.completion.create(
        model=fine_tuned_model,
        messages=[
{'role':'system','content':system},
{'role':'user','content':user}
],

    )
    print("Model Response:")
    print(response.choices[0].message)
except Exception as e:
    print("Error during inference:", e)




Enter fullscreen mode Exit fullscreen mode

Parallels to History

Just as the Six Triple Eight verified each letter’s destination, our inference checks that the fine-tuned model correctly identifies the category of new, incoming data. It’s the final check to ensure our “mail” (in this case, user queries) arrives at the correct destination (i.e., the correct classification).

By completing this last step, we confirm that our refined model can handle real-world tasks reliably—a testament to the time and effort spent fine-tuning the data in earlier stages.

Image of Timescale

🚀 pgai Vectorizer: SQLAlchemy and LiteLLM Make Vector Search Simple

We built pgai Vectorizer to simplify embedding management for AI applications—without needing a separate database or complex infrastructure. Since launch, developers have created over 3,000 vectorizers on Timescale Cloud, with many more self-hosted.

Read more →

Top comments (0)

A Workflow Copilot. Tailored to You.

Pieces.app image

Our desktop app, with its intelligent copilot, streamlines coding by generating snippets, extracting code from screenshots, and accelerating problem-solving.

Read the docs

👋 Kindness is contagious

Please leave a ❤️ or a friendly comment on this post if you found it helpful!

Okay