DEV Community

Cover image for Inference with Fine-Tuned Models: Delivering the Message
es404020
es404020

Posted on

Inference with Fine-Tuned Models: Delivering the Message

During WWII, after tirelessly sorting through mountains of letters, the women of the Six Triple Eight ensured that each piece of mail reached its correct recipient. They didn’t simply sort the letters and stop there; they completed the job by handing them off for delivery. In the world of AI, inference is that critical final stage. Once our model has been fine-tuned, we need to test it on actual input to verify that it can accurately perform the tasks it was trained for—whether that’s classification, summarization, or another specialized function.

Below is a simple example of how to perform inference using a fine-tuned OpenAI model in Python. In this scenario, we’re classifying an article into one of several categories: business, tech, entertainment, politics, or sport.




fine_tuned_model = "ft:gpt-3.5-turbo:xxxxxxxxxxxx"


system = "Classify this article into this category : business,tech,entertainment,politics,sport,tech"

user = "A new mobile phone is launched"

try:
    response = openai.completion.create(
        model=fine_tuned_model,
        messages=[
{'role':'system','content':system},
{'role':'user','content':user}
],

    )
    print("Model Response:")
    print(response.choices[0].message)
except Exception as e:
    print("Error during inference:", e)




Enter fullscreen mode Exit fullscreen mode

Parallels to History

Just as the Six Triple Eight verified each letter’s destination, our inference checks that the fine-tuned model correctly identifies the category of new, incoming data. It’s the final check to ensure our “mail” (in this case, user queries) arrives at the correct destination (i.e., the correct classification).

By completing this last step, we confirm that our refined model can handle real-world tasks reliably—a testament to the time and effort spent fine-tuning the data in earlier stages.

Heroku

Build apps, not infrastructure.

Dealing with servers, hardware, and infrastructure can take up your valuable time. Discover the benefits of Heroku, the PaaS of choice for developers since 2007.

Visit Site

Top comments (0)

A Workflow Copilot. Tailored to You.

Pieces.app image

Our desktop app, with its intelligent copilot, streamlines coding by generating snippets, extracting code from screenshots, and accelerating problem-solving.

Read the docs

👋 Kindness is contagious

Dive into an ocean of knowledge with this thought-provoking post, revered deeply within the supportive DEV Community. Developers of all levels are welcome to join and enhance our collective intelligence.

Saying a simple "thank you" can brighten someone's day. Share your gratitude in the comments below!

On DEV, sharing ideas eases our path and fortifies our community connections. Found this helpful? Sending a quick thanks to the author can be profoundly valued.

Okay