Token Factory series: Multiple messages as input

Hello 👋, this is in continuation to Token Factory series: Getting started. In this post, we shall try sending multiple messages in our input, instead of one message. The typical message types we use in generative AI are system, user and assistant. Think of a system message as a stable message that likely will not change often, where as a user message is like a dynamic message which changes in each turn.

Let's say we are chatting for the first time, our first message might have a system message plus a user message which is our first prompt, we would get a response for that message, so in the next turn the system message, our old user message, the response message as well as the new user message might be present in the input. This way in a multi turn conversation, one input can have mutiple messages of each type (system, user, assistant), depending on how much recent history we want to persist, and typically system messages appear at the beginning of the conversation.

Note that we will just try using a system/user message combination for this post and not a multiturn conversation.

The typical format for a text message looks like below:

message = {
    "role" : role,
    "content": [
        {
            "type": "text",
            "text": "some text",
        }
    ]
}

The role is either system, user or assistant. And if you note, content is an array that acts like multiple chunks with in one message, i.e. all within the same message id.

Let's form our system message:

sys_msg = {
    "role" : "system",
    "content": [
        {
            "type": "text",
            "text": "The response should be approx. 300 chars and it should have exactly 3 emojis.",
        }
    ]
}

And then our user message:

user_msg = {
    "role" : "user",
    "content": [
        {
            "type": "text",
            "text": "Give me one interesting fact about wildlife, which kids can easily understand.",
        }
    ]
}

We can use the chat completions api to send multiple messages in our input, with the messages attribute.

response = client.chat.completions.create(
    model="nvidia/Nemotron-3_5-Lightning",
    messages = [ sys_msg, user_msg ]
)

The to_json method for the response gives a string which can be converted to dict further with the json package.

import json

response_dict = json.loads(response.to_json())

This dict has a choices key, the first index of it contains the output message, if you recall, each message should have a role and content. In this case the role is assistant.

output_message = response_dict["choices"][0]["message"]

We can print the role and message content.

print(output_message["role"])
print(output_message["content"])

So our final code would be:

$ cat main.py 
import json
import os

from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()

client = OpenAI(
    base_url="https://api.tokenfactory.nebius.com/v1/",
    api_key=os.environ.get("NEBIUS_API_KEY"),
)

sys_msg = {
    "role" : "system",
    "content": [
        {
            "type": "text",
            "text": "The response should be approx. 300 chars and it should have exactly 3 emojis.",
        }
    ]
}

user_msg = {
    "role" : "user",
    "content": [
        {
            "type": "text",
            "text": "Give me one interesting fact about wildlife, which kids can easily understand.",
        }
    ]
}

response = client.chat.completions.create(
    model="nvidia/Nemotron-3_5-Lightning",
    messages = [ sys_msg, user_msg ]
)

response_dict = json.loads(response.to_json())

output_message = response_dict["choices"][0]["message"]

print("Role: ", output_message["role"])
print("Content: ", output_message["content"])

Let's run it.

$ uv run main.py 
Role:  assistant
Content:  Did you know octopuses have three hearts and blue blood? They’re smart—they can open jars, solve puzzles, and use tools! Kids love their blue blood and how they squeeze through tiny holes. Octopuses have nine brains, one in each arm. They can regrow lost arms and fit in tiny spaces! 🐙💙🧠

That's it for the post, hope it was useful and Thank you for reading!

Image credit: Photo by Serena Repice Lentini on Unsplash