DEV Community: Lorena Dávila Ermus

2.Self-Hosted AI: n8n + Ollama, local AI workflows on your Mac

Lorena Dávila Ermus — Sat, 08 Aug 2026 03:38:53 +0000

If you want AI agents running on your own machine, with your own models, and no data leaving your computer, this is the article :).

This is part three of the series. In part one we set up PostgreSQL, and in part two we covered the LLM concepts (models, parameter, quantization, context, capabilities, VRAM). Today we put them to work: n8n for the workflows and Ollama for the models.

One prerequisite: Docker. If you do not have it yet, install Docker Desktop for Mac following the official guide [Docker docs].

Quick setup: n8n

The fastest path is n8n's official Self-hosted AI Starter Kit, a Docker Compose template that ships n8n, Ollama, Qdrant (a vector store) and PostgreSQL preconfigured to talk to each other [n8n docs].

git clone https://github.com/n8n-io/self-hosted-ai-starter-kit.git
cd self-hosted-ai-starter-kit
cp .env.example .env   # file where your passwords are stored

The .env file is hidden by default. In Finder, press Command + Shift + Period to show hidden files, or just edit it from the terminal. Update the credentials, for example:

POSTGRES_USER=admin
POSTGRES_PASSWORD=root
POSTGRES_DB=n8n

Also replace the N8N_ENCRYPTION_KEY and N8N_USER_MANAGEMENT_JWT_SECRET values with your own random strings.

Now one Mac-specific detail. Docker on Apple Silicon cannot use the Mac's GPU, so the kit's README recommends running Ollama natively on your Mac for speed and letting the containers connect to it [starter kit README]. That is what we'll do. Set this in your .env:

OLLAMA_HOST=host.docker.internal:11434

Then start everything:

docker compose up

Open http://localhost:5678 to create your n8n account (once), and http://localhost:5678/home/workflows is where your workflows and agents live.

If you only want n8n without the rest of the kit, this single command works too [n8n docs]:

docker run -it --rm --name n8n -p 5678:5678 -v n8n_data:/home/node/.n8n docker.n8n.io/n8nio/n8n

Quick setup: Ollama

On the Mac side (from part two, condensed):

brew install ollama
ollama pull gemma4   # download a model
ollama run gemma4    # chat with it in the terminal
ollama list          # see all downloaded models

Ollama serves an API at http://localhost:11434, and that address is what n8n will call. Not sure this model fits your machine or your task? Part two covers how to check with the VRAM calculator and when a task belongs on a local model versus a cloud one.

Connecting Ollama to n8n

Now the fun part. In n8n, Ollama appears as a model node you can plug into chats and agents.

Step 1: create the credential. In n8n go to Credentials, create an "Ollama" credential, and set the base URL. Because n8n runs inside Docker and Ollama runs on your Mac, localhost will not work from inside the container. Use this instead [n8n docs]:

http://host.docker.internal:11434

The starter kit already includes a "Local Ollama service" credential. Edit it and set the same base URL [starter kit README]. No API key is needed, your local Ollama has no authentication.

Step 2: use it in a chat. Create a workflow, add a Chat Trigger node, then an AI Agent (or Basic LLM Chain) node. In its model slot pick "Ollama Chat Model", select your credential, and choose gemma4 from the model dropdown. Hit the Chat button at the bottom of the canvas and you are talking to your local model inside n8n [Ollama docs].

Step 3: give the agent tools. The AI Agent node accepts tool sub-nodes (HTTP Request, Gmail, a Qdrant vector store, another workflow). The model decides when to call them. This only works well if the model supports tool use, which llama3.1 does. Check the model's page on ollama.com for the "tools" tag before relying on it.

If the connection refuses, the base URL is almost always the culprit. The rule: n8n in Docker and Ollama on the host means host.docker.internal, both in the same compose network means the service name (http://ollama:11434), n8n installed natively means plain localhost [n8n docs].

What about the API keys page?

You may have seen https://ollama.com/settings/keys and wondered if you missed a step. You did not. Those keys are for Ollama's cloud, where ollama.com hosts large models and acts as a remote Ollama server [Ollama docs]. If you ever want to call a cloud model (for example when a task needs more power than your Mac has), you create a key there and add it to the n8n Ollama credential with the base URL https://ollama.com. For everything local, no key, no account.

Wrapping up

That is a complete private AI stack: n8n orchestrating, Ollama serving the model, and your data staying on your machine. Next I want to build a real workflow on top of this, a document assistant using the Qdrant vector store that came with the kit. See you there :).

Sources: Docker: get Docker, n8n Self-hosted AI Starter Kit docs, starter kit README, n8n Docker install, n8n Ollama credentials, n8n Ollama common issues, Ollama n8n integration, Ollama cloud, LLM VRAM Calculator.

1. Self-Hosted AI: the LLM concepts you need to run models effectively

Lorena Dávila Ermus — Sat, 08 Aug 2026 03:02:43 +0000

If you want to run AI models on your own machine and learn the basic concepts with me to do it effectively, then this is the right article :).

This is part one of the series. In the next one we build local AI workflows with n8n and Ollama. This article is the vocabulary we need to pick a model, load it without crashing your machine, and know what each setting does.

The pieces

Running a model locally involves a few separate things that people mix up all the time.

The model is a file on your disk. The server (Ollama, LM Studio) loads that file and runs the math. The API is a local address the server opens so other apps can reach it. The clients are whatever you point at it: a chat window, your code editor, an n8n workflow. You never talk to the model directly, you always go through the server.

Model and inference

The models we run are large language models (LLMs): machine learning models trained on massive text datasets to understand and generate language, built on a neural network type called a transformer[Cloudflare]. In practice a model is a file of learned numbers, called weights, and it works by predicting the next chunk of text over and over.

Running a trained model to get answers is called inference, the step we do on our hardware, as opposed to training, which is how the model was built[Cloudflare]. Everything below is about making inference run well.

Server and API

The server loads and runs the model, and it opens a local API. An API is a set of rules that lets two pieces of software communicate [AWS]. Here it means an address on your own machine that apps send prompts to:

Ollama: http://localhost:11434
LM Studio: http://localhost:1234

localhost means the request never leaves your computer. Both servers copy the OpenAI API format, so any tool built for ChatGPT's API can point at this address and work without changes. One running model can feed your editor, a chat app, and an n8n workflow at the same time.

GPU, VRAM, and RAM

This trio decides whether a model runs fast, slow, or not at all.

A GPU runs many calculations in parallel, which fits the matrix math LLMs depend on. VRAM is the fast memory built into the GPU, and the model's weights need to load into it, because the GPU reads them for every token it generates[BentoML]. Whatever does not fit spills into system RAM and runs on the CPU, which works but is slower.

On Apple Silicon Macs there is no separate VRAM. The CPU and GPU share one pool called unified memory, so your RAM is effectively your VRAM.

Parameters, and how to read a model name

Parameters are the internal variables a model learns during training, and they store the patterns it uses to make predictions[IBM]. More parameters generally means more capable, heavier, and slower. The count is written with a "B" for billions.

Size	Runs on	Good for
1 to 3B	Any laptop	Quick tasks, autocomplete
7 to 9B	8 to 16 GB laptop	A strong default for general use
13 to 32B	16 to 24 GB GPU or Mac	Stronger coding and reasoning
70B and up	Workstation	Top quality, heavy hardware

Model names encode this. In llama3.1:8b, the part before the colon is the family and version (Llama 3.1), and the tag after it is the variant, usually the size. Tags can also carry the quantization or the purpose, like qwen2.5-coder:7b.

One more naming detail worth knowing: a base model is the raw next-word predictor straight out of training, while an instruct model was further tuned to follow instructions and hold conversations. For chat and workflows you always want the instruct version. Ollama's library serves instruct variants by default, but on Hugging Face you will see both, so check the name.

Quantization and formats

Quantization converts a value from a high precision format to one that takes less space and is slightly less precise, so the model runs faster and on cheaper hardware while ideally giving up very little accuracy[Cloudflare]. For us it means the same model ships in smaller versions that need much less memory.

Labels like Q4_K_M or Q8_0 tell you how many bits each weight keeps. Fewer bits, smaller and faster file, slightly less accurate.

Quant	Size vs full	Quality	Notes
Q8_0	~50%	~99%	High quality, still heavy
Q5_K_M	~35%	~98%	Good balance
Q4_K_M	~29%	~96%	What I pick when unsure
Q3_K_M	~22%	~90%	Only if memory is very tight

Two file formats cover most of what you will meet. GGUF packs a quantized model into a single file and is the native format of llama.cpp, the engine under Ollama and LM Studio. Ollama stores and runs its models as GGUF, and you can import any GGUF from Hugging Face with a one-line Modelfile[Ollama docs, Hugging Face]. Safetensors is the format Hugging Face uses for full precision weights, which you will mostly meet when a model has not been quantized yet.

Tokens

A token is the smallest unit of text a model processes, usually a word, part of a word, or punctuation[NVIDIA]. The model reads input as tokens and produces output one token at a time.

A useful rule: one token is about four characters, or three quarters of a word. Tokens are the unit behind two numbers you will see constantly: speed in tokens per second (a 7 to 8B model does about 30 to 50 on a typical laptop GPU), and the context limit below.

Context window

The context window is the amount of text, in tokens, the model can consider at once, its working memory[IBM]. It holds everything simultaneously: the system prompt, the conversation, pasted files, and the room left for the reply.

When it fills up, the oldest tokens drop out and the model forgets the start of a long chat. And a bigger window costs more memory, since the model keeps a growing cache of the conversation. I keep it at 8K for chat and coding, and raise it only when feeding long documents.

Temperature

Temperature controls the randomness of generated text[IBM]. Lower values push the model toward the most likely next token, higher values spread the choice wider.

In practice: 0.1 to 0.3 for code and facts, around 0.7 for chat, above 1.0 for creative variety at the cost of focus. If code answers feel vague, lower it. If writing feels flat, raise it.

GPU offload

GPU offload is how many of the model's layers sit on the GPU versus the CPU. All layers on the GPU is fastest and happens automatically when the model fits in VRAM. When it does not fit, the leftover layers run on the CPU and everything slows down.

Ollama decides this for you, and ollama ps shows the resulting GPU/CPU split. LM Studio exposes a slider (n_gpu_layers) to set it by hand. Short version: fit it all in VRAM and it is fast, spill over and it crawls.

Will it fit? Use a calculator

Your real memory use is more than the model file:

Total ≈ weights + context + a bit of overhead

As a reference, an 8B model at Q4 takes about 4.6 GB of weights, context adds roughly 0.1 GB per 1,000 tokens, and overhead is 0.5 to 1 GB. Instead of doing this math per model, use the LLM Model VRAM Calculator: give it a model, a quant, and a context size, and it tells you whether your hardware can take it. When the total goes over your memory, the fix is a smaller quant, a smaller model, or a smaller context.

Capabilities: what a model can and cannot do

Abilities are fixed during training, so check the model's page before relying on one. The tags on ollama.com mark them.

Chat / text generation: the baseline. Every instruct model does this.
Vision: accepts images as input, so it can read screenshots, photos, and scanned PDFs. Needed for anything visual; a text-only model simply cannot see the file.
Tool use (function calling): the model can decide to call a function or API you describe to it, and return the arguments for the call. This is what makes agents possible, including n8n AI Agent workflows.
Reasoning: the model works through a problem step by step before answering. Better at math, logic, and planning, but slower and more verbose.
Structured output: the model is forced to answer in a strict format like JSON. Essential for automation, where the next node expects clean data instead of prose.
Code completion (fill-in-the-middle): completing code at the cursor given what is before and after it. This is what autocomplete models do, and it is a different job from explaining code in a chat.
Embeddings: a separate model type. An embedding is a numeric vector representing text, which lets you measure similarity and search by meaning[Cloudflare]. This powers RAG, the technique behind chatting with your own documents.

A coding model usually cannot see images, and a vision model may be weak at tool use. Match the model to the task.

Knowledge cutoff and hallucination

Two limits to keep in mind with any model, local or not.

A model's knowledge freezes at its training cutoff date. Ask a local model about something from last month and it either does not know or, worse, guesses. Which leads to the second limit: models sometimes state wrong things fluently and confidently, called hallucination. It happens because the model predicts plausible text rather than checking facts. Lower temperature and giving the model the relevant documents (RAG) reduce it, but nothing removes it. Treat any factual claim from a model as a draft to verify.

Local or cloud?

Not every task belongs on a local model. Local wins on privacy and cost, since your data never leaves the machine and tokens are free after download, while cloud models pull ahead on raw capability, huge contexts, and fresh information via web search[WebCraft]. My checklist:

Question	Points to local	Points to cloud
Privacy, NDA, client data?	Yes, nothing leaves your machine	Only if provider terms allow it
Context per request?	Small to medium fits fine	Very large documents
Need info updated in 2026?	No, cutoffs apply	Yes, cloud adds web search
Hardware?	You have the memory for it	You do not
Cost at scale?	Free after download	Pay per token

Quick glossary

System prompt: hidden instructions that set the model's behavior for the whole conversation.
Modelfile (Ollama) / Preset (LM Studio): a saved bundle of settings so you do not redo them.
MoE (Mixture of Experts): a model that activates only part of itself per token, large but faster than its size suggests.
RAG (retrieval augmented generation): fetching relevant documents (via embeddings) and giving them to the model with the question.
top_p and repeat penalty: finer randomness knobs, fine at their defaults.

You are ready

With PostgreSQL from part one and these concepts, you can set up a local model right now. On a Mac:

brew install ollama
ollama pull gemma4   # download a model
ollama run gemma4    # chat with it in the terminal
ollama list          # see all downloaded models

That model now sits behind http://localhost:11434, waiting for clients. In the next article we connect it to n8n and build AI workflows on top. See you there :).

Sources: Cloudflare: LLMs, Cloudflare: inference vs training, Cloudflare: quantization, Cloudflare: embeddings, AWS: what is an API, IBM: parameters, IBM: temperature, IBM: context window, NVIDIA: tokens, BentoML: GPU memory, Ollama: importing GGUF, Hugging Face: Ollama and GGUF, LLM VRAM Calculator, WebCraft: Ollama vs ChatGPT vs Claude.

Quick Guide to Setting Up PostgreSQL on Mac via CLI with Homebrew

Lorena Dávila Ermus — Thu, 14 May 2026 21:34:45 +0000

Hey, I'm excited to do a series of short articles to set up Free AI tools such as n8n and Ollama on your local machine, or to setup the environment for your personal projects.

One of the first issues you may encounter is that you may not have postgres running on local.

To solve that with the minimal setup follow these exact steps.

Step 1: Install and Start the Service

Open your Mac Terminal and run these commands to install Postgres and keep it running in the background

brew install postgresql # Installs the latest version

brew services start postgresql # Keeps it running in the background

(Optional) You can check if it is running or stop it later using these commands:

brew services list

brew services stop postgresql

Step 2: Connect for the First Time

Homebrew automatically creates a default superuser matching your Mac username with a blank password. You do not need a password yet. Connect directly to the default postgres template database by running:

psql -U $(whoami) -d postgres

Your terminal prompt will change to postgres=#. You are now inside the database engine.

Step 3: Useful Navigation Shortcuts

Inside the Postgres prompt, use these quick slash commands to inspect your setup:
\du -> View all database users.
\l -> View all existing databases (including the default postgres).
\q -> Exit Postgres and return to your regular terminal.

Step 4: Create Your Project Database and User

It is best practice not to use your superuser account for applications. Instead, run these SQL commands inside the prompt to set up a clean environment. Remember the semicolons at the end of each line:

CREATE USER my_project_user WITH PASSWORD 'my_secure_password';

CREATE DATABASE my_project_db;

GRANT ALL PRIVILEGES ON DATABASE my_project_db TO my_project_user;

Note: If you are on Postgres 15 or higher, you might also need to grant schema permissions after connecting to the new DB:
GRANT ALL ON SCHEMA public TO my_project_user;

That’s it! Type \q to exit. You can now connect your application to localhost using your new database, username, and password.