How I Use Grok Build to Run Free Models Alongside Grok-4.6

I really like Grok Build as a harness. The terminal UI is polished without too much glitter, the tools are decent, and there’s a certain way it sticks to a project. What I like less is having access only to the Grok models lineup. What if I can have the same rich layer, but talking to open weights models?

So I did what every AI obsessed person does these days: started to dig into config files, squeezing every ounce of juice from everywhere I can. It turned out that Grok Build talks to anything that speaks an OpenAI-compatible chat API. It can be a model on your machine, or a free model behind OpenRouter. You use the same interface and you flip models with /model and keep working.

The whole customization is in:

~/.grok/config.toml

Below is my actual setup. There are two different options, you can pick just one, or keep both.

Option 1: Local models

For this you need something serving the model on localhost. llama.cpp, LM Studio, Ollama — whatever you already like, as long as it exposes /v1/chat/completions. I configured my server to listen on port 8080, but you can choose whatever you want, it's local anyway.

Then you tell Grok Build about it in the config.toml file:

[model.gemma-4-12b-local] 
model = "gemma-4-12b-it-qat-q4_0" 
base_url = "http://127.0.0.1:8080/v1" 
name = "Gemma 4 12B QAT Q4_0 (local)" 
api_backend = "chat_completions" 
context_window = 16384 
[model.qwen-3-8-27b-local] 
model = "Qwen3.8-27B-UD-IQ2_S" 
base_url = "http://127.0.0.1:8080/v1" 
name = "Qwen 3.8 27b QT 2 (local)" 
api_backend = "chat_completions" 
context_window = 16384

On my M1 16GB MacBook Pro I can run heavily quantized models in the Gemma / Qwen layer, nothing above that. Performance isn’t great, but it’s local. A few notes that can save you some time:

• model must match the id your local server expects, not the actual marketing name.
• base_url ends at /v1. Grok Build appends the rest.
• context_window should be kept minimal if you're low on RAM (like I am). If you aim for 128k and the quantized model only holds 16k, you get compaction at nearly every prompt and long sessions are completely amnesic.
• No API key needed for localhost.

Start the server, which loads the weights, restart Grok Build, then in the new session:

/model gemma-4-12b-local

or you can start grok directly with the model as an argument:

grok -m gemma-4-12b-local

And that’s the whole local option. It’s offline, you don’t pay anything, and it’s private by default. Like I said, quality depends on your hardware, specifically RAM, and how aggressively you quantized. As a rule of thumb, meaningful work can be done if you have over 32GB of RAM, 16GB, like I do now, is mainly for experiments.

Option 2: OpenRouter free models

Local is really great, but most of the time you want a bigger model than your machine can hold. OpenRouter has a free tier for a bunch of open models. We will use the same harness, but with a different endpoint.

You will need an OpenRouter API key for this. Generate one in your dashboard, then add it to your environment:

export OPENROUTER_API_KEY="sk-or-..."

Then define the provider once, so you don’t repeat yourself for every model:

[model_providers.openrouter] 
base_url = "https://openrouter.ai/api/v1" 
env_key = "OPENROUTER_API_KEY" 
api_backend = "chat_completions" 
[model_providers.openrouter.extra_headers] 
HTTP-Referer = "https://x.ai" 
X-Title = "Grok Build"

Then add the free models you care about. Here’s my non-exhaustive list:

[model.glm-5-2-free] 
model = "z-ai/glm-5.2:free" 
name = "GLM 5.2 (OpenRouter free)" 
model_provider = "openrouter" 
base_url = "https://openrouter.ai/api/v1" 
env_key = "OPENROUTER_API_KEY" 
api_backend = "chat_completions" 
context_window = 256000`

[model.openrouter-free] 
model = "openrouter/free" 
name = "OpenRouter free router" 
description = "Routes to whatever free OpenRouter model is available" 
model_provider = "openrouter" 
base_url = "https://openrouter.ai/api/v1" 
env_key = "OPENROUTER_API_KEY" 
api_backend = "chat_completions" 
context_window = 200000 
[model.gemma-4-31b-free]

model = "google/gemma-4-31b-it:free" 
name = "Gemma 4 31B (OpenRouter free)" 
model_provider = "openrouter" 
base_url = "https://openrouter.ai/api/v1" 
env_key = "OPENROUTER_API_KEY" 
api_backend = "chat_completions" 
context_window = 262144

The :free suffix is very important. Without it you hit the paid route. openrouter/free is also an interesting option - it picks whatever free model is available that day. I find it really cool for experiments. But for real work I pin a specific one, usually GLM 5.2 free or Gemma 4 31B free, so behavior stays somewhat consistent.

In the Grok Build harness you switch the same way:

/model glm-5-2-free

And you can also check what Grok Build can see:

grok models

How I Actually Use This

Almost 90% of my work sessions are on a paid Grok model, I maximize the full harness and its tools. I switch to local for private or offline sessions. And I choose OpenRouter free when I want a bigger open model without drawing from my Grok usage. Sometimes I go for models like Nemotron or DeepSeek Flash. I didn’t include the configs for those in this article, I just leave this as a little bit of homework for you.

The important part is the config file. Once ~/.grok/config.toml knows about a model, the rest of Grok Build treats it like any other: tools, sessions, /model, Ctrl+M picker. You're never starting a second app onto your workflow. You're pointing the same app at a different brain.

If something fails to connect, curl the endpoint first. For local:

curl -s http://127.0.0.1:8080/v1/models

For OpenRouter:

curl -s https://openrouter.ai/api/v1/models \ -H "Authorization: Bearer $OPENROUTER_API_KEY"

If curl is happy and Grok Build isn’t, it’s almost always a typo in model, base_url, or env_key.

That’s my entire setup: one harness, two free paths and endless choices. All you have to do is edit the toml, restart or switch with /model, and keep building.