Call a local model from plain Python: no SDK, no dependencies

#ai #python #ollama

Most "use a local model" tutorials start with a virtualenv and a dozen packages. For a small tool you may not need any of them. If a local Ollama server is running, Python's standard library is enough to talk to it.

I used this approach in Sportello chiaro, a small helper I built to explain Swiss administrative letters in plain Italian. It is Python 3.10, standard library only, and about 306 lines in total. No web framework, no HTTP client package, no SDK.

What Ollama gives you

Ollama runs a local HTTP server on port 11434. Its API documentation describes a POST /api/generate endpoint, and a "stream": false option that returns one JSON object instead of a stream of chunks. The generated text is in the response field. That is all the plumbing you need.

The call, in plain Python

This is an illustrative example written for this article, not an excerpt from the project:

import json
import urllib.request

def ask_local_model(prompt, model="gemma3:1b"):
    body = json.dumps({"model": model, "prompt": prompt, "stream": False}).encode()
    req = urllib.request.Request(
        "http://localhost:11434/api/generate",
        data=body,
        headers={"Content-Type": "application/json"},
    )
    with urllib.request.urlopen(req, timeout=300) as resp:
        return json.loads(resp.read())["response"]

Two details matter. First, set a generous timeout: a small model on a CPU is slow. Second, keep stream off unless you actually want to print tokens as they arrive. Without streaming, the function is a plain "string in, string out" call that is easy to test.

What a small model on a CPU is like

In my own test of the project, Gemma 3 1B on a CPU took about 40 seconds to explain a Zahlungsbefehl (a Swiss payment order). That is not fast, but for a letter you read once it is acceptable, and nothing leaves the machine. For documents like these, privacy was the reason to go local in the first place.

Speed is the price. If you need answers in a second, a 1B model on a CPU is the wrong tool.

Why drop the dependencies

Nothing to install. If Python 3.10 and Ollama are there, it runs.
Less to break. There is no package that changes its API under you.
Easy to read. 306 lines can be read in one sitting, which matters when a tool handles personal letters.

The standard library is not always the better choice. If you need retries, connection pooling or async calls, a proper client library pays for itself. For a one-shot call per document, urllib was enough.

Takeaway

Before adding a dependency for a local model, try the plain HTTP call. For a small, single-purpose tool it may be all you need.

DEV Community