Token Factory series: Getting started

Hey 👋, in this beginner friendly post, we shall see how to make an api call to Nebius Token Factory and get a response to our prompt. We shall also cover some python basics.

Token factory is a platform where we can use opensource models with API calls, alongside several other features.

Let's get started!

We would be using uv which is a package and project manager for python. The installation instructions for uv are here

Initialize a uv project, and navigate to its directory.

uv init try-tokenfactory
cd try-tokenfactory/

Let's see the contents uv generated, -1 is used to display contents in a linear way for readability and a to show hidden files(starting with .) as well, . stands for current directory and .. for previous directory.

$ ls -1a
.
..
.git
.gitignore
.python-version
.venv
main.py
pyproject.toml
README.md
uv.lock

Generate a token factory API key and add it to .env.

$ cat .env 
NEBIUS_API_KEY="<REDACTED>"

Add the openai library for making api calls to tokenfactory.

uv add openai==3.46.0

Add the dotenv library for loading env vars from a file, in our case it's .env.

uv add python-dotenv==1.2.4

Let's write code in main.py, let's see it line by line.

Import the inbuilt os module, to interact with the underlying operating system.

import os

Tip: this is a module as it contains only one file. You can check with print(hasattr(os, "__path__")) and it should give False as it's a module, it would be True for a package with multiple modules/files.

Import the load_dotenv function from the dotenv package.

from dotenv import load_dotenv

Tip: In python, variables and functions are usually defined in snake_case. When we use this function in our code, we would add parantheses, which tells it's a function.

Import the OpenAI class from the openai package. Tip: In python, class names are usually defined in PascalCase.

from openai import OpenAI

Load the env variable from the .env file.

load_dotenv()

Initialize an object/instance with the OpenAI class. We can call it client.

client = OpenAI(
    base_url="https://api.tokenfactory.nebius.com/v1/",
    api_key=os.environ.get("NEBIUS_API_KEY"),
)

Nebius Token Factory API is compatible with the openai API hence we can use it just like we would use it openai endpoints. The base url is the tokenfactory endpoint and the api key is obtained through the os module as we have loaded it already with dotenv package.

Let's try to prompt. Before that let's get a model endpoint from here. You may select one of the public endpoints there.

I am choosing Nemotron-3.5-Lightning, we can click on that card and get the routing key, which is nvidia/Nemotron-3_5-Lightning.

We can use this router key in our api call, as the model attribute.

response = client.responses.create(
    model="nvidia/Nemotron-3_5-Lightning",
    input="Give me one interesting fact about wildlife, which kids can easily understand.",   
)

We have set the prompt, we can access the response text as follows.

print(response.output_text)

Tip: This response would have some attributes, we can check those this with print(dir(response)), output_text is one of those.

Here is a sample response.

$ uv run main.py 
One cool wildlife fact: **Butterflies taste with their feet!** 

They have tiny taste sensors on their legs, so when they land on a leaf or flower, they can immediately tell if it's the right, yummy plant to eat or lay their eggs on — just by standing on it! 🦋👣

Cool fact, indeed!

Note that we just ran the code with uv. And here is the complete code.

$ cat main.py 
import os

import dotenv
from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()

client = OpenAI(
    base_url="https://api.tokenfactory.nebius.com/v1/",
    api_key=os.environ.get("NEBIUS_API_KEY"),
)

response = client.responses.create(
    model="nvidia/Nemotron-3_5-Lightning",
    input="Give me one interesting fact about wildlife, which kids can easily understand.",   
)

print(response.output_text)

In this post, we just gave one message as input. But usually, it would be multiple messages per input, we shall see that in the next post.

I think the post was easy to understand for beginners, let me know if I need to modify something. We can gradually increase the complexity as we go to the next post in the series.

Thank you for reading!

Cover image: Photo by Boris Workshop on Unsplash