DEV Community

Cover image for Run LangChain Locally in 15 Minutes (Without a Single API Key)
Fabio Hiroki
Fabio Hiroki

Posted on

Run LangChain Locally in 15 Minutes (Without a Single API Key)

#python #tutorial #ai #langchain

You don’t need an API key.

That’s the secret everyone seems to forget.

I wanted to dive into LangChain, the powerful framework for building with Large Language Models. But every guide, even the official docs, felt like a trap. They were outdated, incomplete, or pushed me toward a paid Anthropic key.

I wrestled with Python version errors. I hunted for missing dependencies. It was a frustrating maze designed to make me quit.

So I carved my own path.

This is the guide I wish I had. It’s your shortcut to running a powerful LLM on your own machine, completely for free, in the next 15 minutes.

P.S.: If you want to skip the article, you can check the code directly on Github.

1. First, understand why this is a game-changer.

Running an LLM locally is about one thing: speed.

Not processing speed, but the speed of learning.

When you're just trying to get a feel for a new framework, the last thing you want is a roadblock. You don't need to sign up for an API, enter a credit card, or wait for an approval email just to run a "hello, world" prompt.

This local-first approach removes all that friction.

And as a bonus, it comes with three other superpowers:

  • Zero Cost: It’s completely free, no matter how many prompts you run.
  • Total Privacy: Your data never leaves your machine.
  • Full Control: You choose the model, you own the process.

This isn't just about saving money. It's about creating the fastest path to actually building and learning.

2. Next, get your environment right (this is critical).

Set Up Your Python Version.

This is where 99% of tutorials fail. They give vague instructions, but the AI world moves so fast that packages break between minor versions. We will avoid that entirely.

This tutorial was built and verified using Python 3.13.0. Sticking to this exact version removes any guesswork. For managing Python versions like a pro, I recommend pyenv:

pyenv install 3.13.0
pyenv local 3.13.0
Enter fullscreen mode Exit fullscreen mode

Install Ollama.

Ollama is the magic that runs the LLM on your machine. Go to https://ollama.com/download/ and download it for your system. Install it like any other application. It will run quietly in the background, ready when you need it.

3. Now, start Ollama and download your AI model.

With Ollama installed, you need to make sure its server is running before you can download or use a model.

Open a new, separate terminal window and run this command. Leave this terminal running in the background.

ollama serve
Enter fullscreen mode Exit fullscreen mode

Note: If you get an error that the server is already running, that's fine—it means the app started automatically.

Now, in your original terminal, pull the llama3.2 model.

ollama pull llama3.2
Enter fullscreen mode Exit fullscreen mode

This might take a few minutes. You only have to pull once.

4. Then, install the exact package versions.

To guarantee this works, we will install the exact library versions used in this tutorial.

First, create a project folder and set up a virtual environment (a critical best practice):

python -m venv venv
source venv/bin/activate
Enter fullscreen mode Exit fullscreen mode

Now, install the pinned dependencies:

pip install langchain==1.0.3 langchain-ollama==1.0.0
Enter fullscreen mode Exit fullscreen mode

This ensures you have the precise code that we know works. No surprises, no cryptic error messages.

5. Finally, write less than 20 lines of Python code.

You have the environment, the model, and the libraries. All that's left is to connect them.

Create a new Python file named main.py and paste in the following code.

from langchain.agents import create_agent
from langchain_ollama import ChatOllama

def get_weather(city: str) -> str:
    """Get weather for a given city."""
    return f"Return the weather in {city} in a joke."

agent = create_agent(
    model=ChatOllama(model="llama3.2"),
    tools=[get_weather],
    system_prompt="You are a helpful assistant",
)

if __name__ == "__main__":
    # Run the agent
    result = agent.invoke(
        {"messages": [{"role": "user", "content": "what is the weather in sf"}]}
    )
    print(result["messages"][-1].content)
Enter fullscreen mode Exit fullscreen mode

The code creates the model, defines a simple instruction, chains them together, and asks a question. That’s the core of LangChain in action.

In your terminal, run the script:

python main.py
Enter fullscreen mode Exit fullscreen mode

You should see something like:

Fog-get about it, it's always raining in San Francisco!
Enter fullscreen mode Exit fullscreen mode

That's it.

You just ran a state-of-the-art LLM on your local machine using LangChain. You've bypassed the confusing docs, avoided all API keys, and built something that is 100% free and private.

Now, what will you build next?

Top comments (0)