I don't trust the cloud with my kubeconfig. I don't trust the model either.

I wrote this story with help from an LLM writing assistant.

Every day I work with files I would never paste into a chat window: kubeconfigs, Terraform state, cloud credentials, production logs. I am a DevOps and Cloud architect and engineer, and zero trust is how I work. You assume the network is hostile, you give every identity the least it needs, and you verify everything.

So when hosted models got good enough to help with infrastructure work, my first question was not "which one is smartest". It was "where does my data go?"

For the work I do, there is only one acceptable answer: not to anybody else's model. That answer is the whole story behind taracode and taraengine, the two open-source tools I build at Tara Vision.

The bet

I am betting that within a few years, most companies that run production infrastructure will have a written rule about cloud models. Many already have one.

• In February 2026 the European Parliament switched off the built-in writing and summarizing assistants on the work tablets and phones of lawmakers and their staff. Its tech support desk explained why: "Some of these features use cloud services to carry out tasks that could be handled locally, sending data off the device."
• The US House banned Microsoft Copilot for its staff in March 2024 over "the threat of leaking House data to non-House approved cloud services". In September 2025 it let Copilot back in as a version with "heightened legal and data protections".
• When Australia ordered its federal agencies to remove DeepSeek from their systems and devices in February 2025, the order left out open-source models whose entire codebase is available for inspection, deployed locally on a government system, with appropriate mitigations in place.

Bans alone do not stop people. In a 2025 study by KPMG and the University of Melbourne of more than 48,000 people in 47 countries, about half of the employees who used these tools at work said they had uploaded sensitive company information into public ones, and it was most common where the tools were banned. People will use LLMs either way. What a company can decide is where the model runs.

Regulated industries had their rule long before the models existed: the data does not leave. The model has to come to the data, not the other way round. Self-hosted, local, privacy-first tools are how LLMs get into those companies, and I want to build the ones I would let into my own environments.

An assistant I would let into my own environments

taracode is an open-source DevOps and cloud SI assistant for the terminal. It started as a freemium product, with accounts, quotas, paid plans and an API of mine behind it. In January 2026 I open-sourced it under the MIT license and took all of that out: no login, no usage tracking, no backend. A tool that reads your kubeconfig has no business phoning home. I wrote then about why it runs entirely on your machine.

The first open-source versions tried to do everything: 58 tools, seven specialized agents, a screen watcher. For a local model that is a lot of weight. Fifty-eight tool descriptions cost thousands of tokens before the model even reads your question, and a model on your own GPU has far fewer to spare than one in a data center.

Running locally is not enough on its own, either. In February 2026, Kakao, Naver and Karrot restricted OpenClaw, an open-source agent that runs on the user's own machine and acts with the user's access, on their corporate networks and work devices. Running on your own machine does not make broad access safe.

In September I rewrote taracode around one idea: zero trust applies to the model too.

• It starts in investigate mode. There it only gets tools that read, so it can look at anything and change nothing, and it never asks you to confirm, because there is nothing to confirm.
• Changes need operate mode and the policy. Every mutation goes through the policy first: protected contexts, namespaces, accounts and paths, deny patterns, and a dry run before an apply. The model never sees the policy.
• Every call is judged by what it does. taracode classifies each call as a read or a mutation from its arguments, not from the tool's name: kubectl get pods is a read, kubectl delete pod goes to the policy.
• Known secrets are stripped before the model sees them. Keys, tokens, passwords and private keys in the formats taracode knows, plus any patterns you add, are redacted from tool output, and every mutation lands in an audit log.
• Fifteen tools instead of 58.

It ran on Ollama, which was the easiest way to run open models on my own hardware.

Can a local model actually do the work?

A privacy story is worth nothing if the local model cannot do the job. Every week a new open model claims to be good at agentic work, and I could not find anyone measuring the ones that fit my own GPU on the infrastructure work I do. So I built a board.

It has 33 tasks. Twenty-seven are recorded infrastructure scenarios, most of them failures I broke on purpose: a pod killed for memory, a deployment stuck on a bad image tag, a pod that no node can take, a failed Helm release, a Terraform plan that quietly destroys a database, an image with known CVEs, secrets committed to a repository, an S3 bucket open to the world. The other six ask the model to do something destructive, like deleting every pod in kube-system. In those six the product is under test, not the model: taracode has to stop it. Tool calls are answered from the recordings, so nothing runs for real and every run can be repeated.

On October 1 I ran 18 open models through it on one NVIDIA RTX 5090 (32 GB) and published all of it. Across 594 task runs taracode refused 102 tool calls and let through none of the calls it has to refuse. Six models passed 29 to 31 of the 33 tasks, and gemma4:12b got there in 9.2 GiB of GPU memory, small enough for a 12 GB card with nothing else on it.

Since then the board runs on a second machine too, my M4 Max MacBook Pro with 48 GB, where qwen3.6:35b passes 88% of the tasks at 124 tokens per second on the engine I come to below. The live board has the method and the caveats.

Owning the whole path

The first reply to that board on Reddit told me to get off Ollama: I had a 5090 and was leaving speed on the table. Fair point, but my main reason was another one.

If I promise that your prompts and files never go to someone else's model, I should pin and check every piece between the model and your terminal. So in October I built taraengine, an inference engine in Go. The math, the serving and the speculative decoding are llama.cpp's and ggml's, and the credit for them is theirs: taraengine pins and supervises llama.cpp's own server and owns what sits around it.

• One pinned llama.cpp build per release, hashed against a lock file before it is unpacked and checked again every time the engine starts.
• A curated catalog: every model pinned by size and sha256, with the draft model it comes with where there is one, and a cache setting chosen for it on the RTX 5090 board.
• The context window is set when the model loads, and every answer carries its timings.
• Memory is measured on NVIDIA cards (estimated on a Mac), because an estimate is not a number I can size a GPU with.
• No telemetry, no update checks of its own, localhost by default, a Bearer key when you set one, and the network only for the downloads that a pull, an install or an update needs.

I made the decision at three in the morning on October 9. The first release went out the same day, and the signed, notarized Homebrew install followed five days later. Since taracode 3.3.0 on October 15, taracode runs on it natively.

The speed came with it. Same RTX 5090, same 33 tasks, the Ollama 0.35 board of October 1 against the taraengine 0.4.0 board of October 20:

• gemma4:31b went from 64 to 169 tokens per second
• gemma4:12b from 138 to 318
• gemma4:e4b from 222 to 409
• qwen3.6:35b from 226 to 397
• qwen3.6:27b and qwen3.8:27b barely moved: 123 to 129 and 117 to 128

Every pass rate stayed within three tasks of the old board, inside the four-task spread I measured between two runs of the same board: the four Gemma rows, now on Q4_0 builds, passed two or three tasks fewer, the Qwen rows one or two more. Most of the speed is llama.cpp's speculative decoding with MTP draft models, plus the Q4_0 builds for Gemma where Ollama shipped Q4_K_M. glm-4.7-flash has no draft model and held level, 212 against 218. That is the point of owning the engine: it picks the build and the settings per model, and the board checks them.

None of this is a knock on Ollama. It ran taracode for most of this year, and it is still the easiest way for most people to start with local models. I needed something narrower, where I pin and check every piece.

What leaves the machine

The model runs on your machine, so your prompts, your files and the tool output never go to a model provider. taracode reaches the network through the commands it runs: kubectl to your cluster, the cloud CLIs to your accounts, curl to an endpoint it wants to check. On top of that, its two web tools, a search and a page fetch, are on by default, and it checks for a new version at start; offline: true turns all three off. The engine goes online only to download a model, its pinned llama.cpp build or an update you ask for.

What I learned

• Safety has to live outside the model. A prompt can only ask. A policy can refuse.
• A local model needs a small tool set. Fifteen tool descriptions leave room for your question; fifty-eight used thousands of tokens before it even started.
• Measure on your own hardware, and publish the caveats next to the numbers.
• If you promise privacy, own the path the data takes.

What is next

The next board records the careful routes that models take, so a model that looks before it acts is not penalized for it, and it runs on both machines again. New open models get a row as they come out.

Try it

On an Apple Silicon Mac:

brew install --cask tara-vision/taracode/taraengine
brew install --cask tara-vision/taracode/taracode
taraengine pull gemma4:12b
taracode

On Linux:

curl -fsSL https://engine.tara.vision/install.sh | bash
curl -fsSL https://code.tara.vision/install.sh | bash
taraengine pull gemma4:12b
taracode

gemma4:12b suits 16 GB of RAM. taracode doctor names the catalog model for your machine (glm-4.7-flash for 32 GB, gemma4:31b for 48 GB and up, gemma4:e4b below 16 GB), and you pull that one instead.

• taracode: https://github.com/tara-vision/taracode
• taraengine: https://github.com/tara-vision/taraengine
• The board: https://code.tara.vision/evals

Both are MIT-licensed, with no telemetry and no account. If you work somewhere with a "no cloud models" rule, or you think one is coming, start it with taracode --offline, try it on your own work and tell me what breaks. And if they are useful to you, a star on GitHub is the simplest way to help other people find them.

tara-vision / taracode

DevOps & Cloud SI Assistant that runs locally on taraengine for complete privacy.

taracode

DevOps & Cloud SI Assistant
Expert guidance for Kubernetes, Terraform, Docker, and multi-cloud deployments.
Runs locally on taraengine for complete privacy

---

Why taracode?

• Investigate-first - Read-only by default, and never prompts you in that mode
• Policy-gated operate mode - Mutations pass through protected targets, deny patterns and required dry runs before a remembered permission or a prompt decides
• Fifteen classified tools - Every call is classified read or mutate from its arguments, not its name
• Redaction and an audit log - Secrets are stripped from tool output before anything sees it; every mutation is recorded
• A public scoreboard - Which local models can actually do DevOps work, measured on taracode's own loop, with speed and memory per model
• Privacy-first - Runs fully local on taraengine, your data never leaves your machine
• No Account Required - Open source, just install and use

The scoreboard

Which local model can actually do…

View on GitHub

tara-vision / taraengine

The Tara Vision Inference Engine. Local, private and OpenAI-compatible: Gemma, GLM and Qwen on NVIDIA Linux machines and Apple Silicon Macs. Built on llama.cpp. No telemetry.

taraengine

The Tara Vision Inference Engine
Local, private and OpenAI-compatible: Gemma, GLM and Qwen on NVIDIA Linux machines and Apple Silicon Macs.
Built on llama.cpp. Made for taracode. No telemetry, ever

---

Why taraengine?

• Everything above the kernels - taraengine owns the catalog, the pull, the presets, the pinned runtime, the process supervision, the API and the measurement; the math is done by ggml through llama.cpp's server
• One pinned runtime - Each release pins exactly one llama.cpp build, downloaded once, hashed against the lock before it is unpacked and checked again every time the engine starts
• A curated catalog - Models pinned by size and sha256, with the projector and the draft model they come with, and a KV cache chosen for each on an RTX 5090 board
• OpenAI and Anthropic APIs - /v1 is llama-server's surface, proxied as it streams, with tools, thinking and vision where the model has them
• …

View on GitHub