One model download and one chat request, followed through Ollama's local server, its runners, the files on disk and the 5 minute memory timer.
A note on how this was written: an AI writing assistant drafted it from the sources listed at the end, and every claim was checked against those sources before it went out.
At a glance
- One local server at 127.0.0.1:11434 answers every app on the machine.
- Models are stored as fingerprinted layers and stay loaded 5 minutes after use.
- Each model answers one request at a time by default; the rest wait in line.
What Ollama does, and who it is for
Ollama runs open models (models whose files you can download and run yourself) on your own computer. It downloads a model once, keeps it in memory while you use it, and answers on a local web address that apps call the way they would call a cloud API (a fixed way for one program to ask another for something).
The quickstart lists three uses: desktop apps, AI coding tools, and apps you build. Running locally keeps the text on the machine, and each call costs nothing. The model has to fit in your computer's memory, so your machine sets how big a model you can run. Ollama also offers cloud models; cloud requests need an API key (a key that says who is asking), and local requests need none. This post stays local.
The core idea is one shared server (a program that waits for requests and answers them). Think of the shared printer in an office: every computer sends jobs to it, whatever program made them, and the printer decides the order. Ollama works the same way for models: one server takes every app's requests and decides which models sit in memory.
Step one: the download
Say I want gemma4:e2b, the local model in the quickstart. I'd type ollama pull gemma4:e2b. That command is just another app: it asks the local server, which fetches the model from the ollama.com library. The quickstart suggests 8 GB of free graphics card memory, or shared memory on a Mac.
Picture a warehouse that labels each box with a fingerprint of its contents and keeps a packing list for each order. Two orders that need the same box share one on the shelf. Ollama stores models that way. A download is a set of layers (the parts a model is made of), each named by its SHA-256 (a fingerprint of a file: change one byte and the fingerprint changes). A manifest (a small file that lists the layers) says which ones make up gemma4:e2b. The OLLAMA_MODELS environment variable (a setting the server reads when it starts) sets the folder where they live.
The other way to get a model is a Modelfile (Ollama's recipe file: the base model, its settings and its prompt template). The command ollama create turns it into a new model.
Step two: one question, lane by lane
Now an app asks gemma4:e2b to say hello in one sentence (the quickstart's test). What comes in: the request goes to the path /api/chat on the server. The server listens on 127.0.0.1 (an address only this computer can reach) at port 11434 (a numbered door on the computer), written localhost:11434 in the docs. Next to it sit /api/generate, /api/embed and /v1/chat/completions, an OpenAI compatible path (one that accepts a subset of OpenAI's request format).
What does the work: a server written in Go (a programming language) with a scheduler (the part that decides which model loads where, and holds requests while a model loads). The model is not in memory yet, so the scheduler holds my request while it loads into a runner (a separate program, one per loaded model). For GGUF models (a single file format for weights, the numbers a model learned), the runner is the server from llama.cpp (an open source engine that runs language models on ordinary computers). For safetensors models (another weights format), it is an engine using MLX (Apple's open source library for running models on its own chips), on by default on those chips. Then the chat template (a pattern that turns a list of messages into one block of text) builds the prompt (the exact text the model reads).
What it keeps: think of a hallway light on a motion timer. It stays on while people pass and switches off a while after the last one. A loaded model does the same. It stays in memory 5 minutes after the last request, then unloads. The setting is keep_alive (how long a model stays loaded). It can be longer, shorter or forever, and 0 unloads the model right after the answer.
What goes out: tokens (the pieces of text a model reads and writes) stream back as they are made. The same server returns embeddings (text turned into lists of numbers so similar meanings sit close together) from /api/embed. When the server is busy, new requests wait in a queue. Past 512 waiting (the OLLAMA_MAX_QUEUE setting), a new one gets a 503 (the web's code for busy, try again later).
Why it is built this way
First, a local server, not a library (code each app loads into itself). Any app in any language can call it, and one server decides what stays in memory and shares loaded models. The cost is one more program running. It listens only on the machine itself unless OLLAMA_HOST (the setting for the address it listens on) says otherwise.
Second, layers named by fingerprint, not one big file per model. Downloads can be checked, and parts two models share are stored once. The cost is that the files on disk have no friendly names.
Third, unload after 5 idle minutes, rather than keep models loaded always or load one fresh for every request. Follow up requests are fast, and memory comes back when you stop. The cost is that the first request after a pause waits for the load.
Fourth, reuse proven engines, llama.cpp and MLX, instead of writing a new one. That brings wide hardware support and quantized models (stored with fewer bits per number, so the model is smaller and faster, with a small loss in quality). The cost is keeping up with the engines' changes: Ollama pins a version and patches it.
What changes at ten times the load
Two apps want two big models on one laptop. If both fit in memory, both load. If not, the new request waits while an idle model is unloaded to make room. By default up to 3 models can load per GPU (graphics chip), or 3 on the CPU (the main chip), if they fit.
Six hundred requests arrive at once. Each model works on one request at a time by default (the OLLAMA_NUM_PARALLEL setting), and the rest wait their turn. Raising it costs memory, which the FAQ says grows with that number. Past 512 waiting requests, the next ones get the 503.
Other machines on the network want in. Set OLLAMA_HOST to listen beyond this computer, and put your own access control (a check on who may come in) in front. The local API has no sign in of its own.
Check it yourself
Here is how I'd watch one download and one call. First, install Ollama from the quickstart and open the app. On Linux, start the server with ollama serve if it is not running. Second, open a terminal (the text window where you type commands) and run ollama pull gemma4:e2b, about 7.2 GB.
Third, open the models folder the FAQ lists: ~/.ollama/models on macOS, /usr/share/ollama/.ollama/models on Linux, or C:\Users\%username%.ollama\models on Windows. Then run ollama show --modelfile gemma4:e2b, which prints the model's Modelfile. In the docs' example, the FROM line (where the recipe names its base) points to a file under models/blobs named sha256 plus a long fingerprint.
Fourth, run ollama run gemma4:e2b and ask one question (type /bye to leave). An app could instead send the quickstart's request to localhost:11434/api/chat, with no API key. Fifth, run ollama ps. It shows what is loaded, its size, a PROCESSOR column (GPU, CPU or both) and an UNTIL column, which reads 4 minutes from now in the FAQ's example. After 5 quiet minutes, run it again: the model should be gone. ollama stop gemma4:e2b unloads it at once.
The full sheet
The full sheet, with every box and decision, is on the Atlas. If you want one for a project you use or are building, ask for a sheet.
How long do you keep your models loaded, and why?
Sources
- Ollama docs: FAQ
- Ollama docs: Quickstart
- Ollama docs: API introduction
- Ollama docs: OpenAI compatibility
- Ollama docs: Modelfile reference
- Ollama on GitHub
More free tools for data and AI: tabrizianalytics.com
Top comments (0)