Syeed Talha

Posted on May 20

Running Nvidia Nemotron on LangChain via OpenRouter

#ai #llm #python #tutorial

Nvidia's Nemotron models are powerful, free-to-use AI models available through OpenRouter. In this guide, you'll learn how to set up a LangChain agent that uses Nemotron — from installing packages to running your first tool-calling agent.

What Are We Building?

A simple LangChain agent that:

Uses Nvidia Nemotron as its brain (via OpenRouter's free tier)
Has access to a custom tool (a weather function)
Answers questions by calling that tool automatically

Prerequisites

Python 3.10 or higher
A free OpenRouter account and API key
uv (recommended) or pip

Step 1 — Get Your OpenRouter API Key

Go to openrouter.ai/settings/keys
Click Create Key
Copy the key — you'll need it shortly

OpenRouter gives you free access to many models, including Nvidia's Nemotron family, with no credit card required.

Step 2 — Set Up Your Project

Create a new project folder and initialize it:

mkdir my-nemotron-agent
cd my-nemotron-agent
uv init

Or if you're using pip, just create a folder and a virtual environment:

mkdir my-nemotron-agent
cd my-nemotron-agent
python -m venv .venv
.venv\Scripts\activate   # Windows
# source .venv/bin/activate  # Mac/Linux

Step 3 — Install Dependencies

uv add langchain langchain-openrouter python-dotenv

Or with pip:

pip install langchain langchain-openrouter python-dotenv

These three packages are all you need:

langchain — the agent framework
langchain-openrouter — connects LangChain to OpenRouter's API
python-dotenv — loads your API key from a .env file safely

Step 4 — Store Your API Key

Create a .env file in your project folder (never commit this to Git!):

OPENROUTER_API_KEY=your-key-here

Also create a .gitignore to protect it:

.env
.venv

Step 5 — Choose a Nemotron Model

OpenRouter hosts several free Nvidia Nemotron models. Here are the main ones:

Model	ID	Best For
Nemotron 3 Nano 30B	`nvidia/nemotron-3-nano-30b-a3b:free`	Fast, general tasks
Nemotron 3 Super 120B	`nvidia/nemotron-3-super-120b-a12b:free`	Complex reasoning
Nemotron Nano 9B V2	`nvidia/nemotron-nano-9b-v2:free`	Lightweight tasks

Note: The :free suffix is required. Without it, OpenRouter will look for a paid endpoint.

For beginners, nvidia/nemotron-3-nano-30b-a3b:free is a great starting point — it's fast and capable.

Step 6 — Write Your Agent

Create a file called main.py:

from dotenv import load_dotenv
from langchain.agents import create_agent

# Load the OPENROUTER_API_KEY from your .env file
load_dotenv()

# Define a tool — any plain Python function works!
def get_weather(city: str) -> str:
    """Get weather for a given city."""
    return f"It's always sunny in {city}!"

# Create the agent
agent = create_agent(
    model="openrouter:nvidia/nemotron-3-nano-30b-a3b:free",
    tools=[get_weather],
    system_prompt="You are a helpful assistant",
)

# Run the agent
result = agent.invoke(
    {"messages": [{"role": "user", "content": "What's the weather in San Francisco?"}]}
)

# Print the final response
print(result["messages"][-1].content)

How It Works

load_dotenv() reads your .env file and sets the OPENROUTER_API_KEY environment variable
get_weather is a regular Python function — LangChain uses its docstring to describe the tool to the model
create_agent wires everything together: the model, the tools, and the system prompt
agent.invoke(...) sends the user's message and returns a result dict
result["messages"][-1].content gets the final text response from the last message

Step 7 — Run It

uv run main.py

Or with pip/venv:

python main.py

You should see something like:

The weather in San Francisco is sunny!

Step 8 — Add More Tools

You can give your agent multiple tools — just add more functions to the list:

def get_weather(city: str) -> str:
    """Get the current weather for a city."""
    return f"It's always sunny in {city}!"

def get_population(city: str) -> str:
    """Get the population of a city."""
    populations = {
        "San Francisco": "870,000",
        "New York": "8,300,000",
        "Dhaka": "21,000,000",
    }
    return populations.get(city, "Population data not available.")

agent = create_agent(
    model="openrouter:nvidia/nemotron-3-nano-30b-a3b:free",
    tools=[get_weather, get_population],
    system_prompt="You are a helpful assistant",
)

The agent will automatically decide which tool to call based on the user's question.

Common Errors & Fixes

Error	Cause	Fix
`OPENROUTER_API_KEY must be set`	Missing API key	Add it to your `.env` file and call `load_dotenv()`
`is not a valid model ID`	Wrong model name	Use the exact ID from openrouter.ai/models, with `:free` suffix
`No endpoints found`	Model unavailable	Try a different Nemotron model ID
`cannot import name 'create_agent'`	Wrong Python / old langchain	Use `uv run` instead of `python`, or upgrade langchain

Project Structure

When you're done, your project should look like this:

my-nemotron-agent/
├── .env              ← Your API key (never share this!)
├── .gitignore        ← Excludes .env from Git
├── main.py           ← Your agent code
└── pyproject.toml    ← Dependencies (if using uv)

Next Steps

Once you're comfortable with the basics, here's what to explore next:

Real tools — connect to real APIs (weather, search, databases) instead of mock functions
Memory — give your agent conversation history so it remembers past messages
Streaming — stream the response token by token for a ChatGPT-like feel
LangSmith — trace and debug your agent's reasoning visually
Other free models — try deepseek/deepseek-r1:free or google/gemma-3-27b-it:free for comparison

Happy building!

DEV Community