DEV Community: QVAC by Tether

VisionPsy-Nano: state-of-the-art vision AI in its weight class, small enough to run on your phone

Thomas — Wed, 29 Jul 2026 12:14:46 +0000

Tether AI Research* is releasing VisionPsy-Nano, a family of ~460M-parameter vision-language models built to run on the device in your pocket.

There are two variants: VisionPsy-Nano-460M when accuracy is the priority, and VisionPsy-Nano-460M-Flash when time-to-first-token, and memory are what matters most.

Download VisionPsy-Nano now on HuggingFace: https://huggingface.co/collections/qvac/visionpsy

Best in its weight class

Against every other ~0.5B vision model tested (LFM2.5-VL-450M, SmolVLM2-500M, and its own nanoVLM-460M-8k base), VisionPsy-Nano-460M leads on 16 of 17 benchmarks, with the highest overall normalized score in its class:

Model	Overall normalized score
VisionPsy-Nano-460M	62.3
LFM2.5-VL-450M	59.6
nanoVLM-460M-8k	54.9
SmolVLM2-500M	52.5

It also leads all four capability categories, not just the average: document understanding and OCR (73.9), visual perception (61.5), reasoning and knowledge (52.2), and instruction following and reliability (65.1). Its widest margins are the two that usually break small models: reasoning and knowledge (+7.4% over the next best ~0.5B model) and visual perception (+4.6%).

It punches above its size class

The more interesting result is what happens when you stop comparing it to its own weight class. On three full benchmarks, a 460M model beats every one of the 0.75B to 1B models tested outright, including Apple's FastVLM-0.5B (759M), Qwen3.5-0.8B (873M), and InternVL3.5-1B (1061M), which are 1.6x to 2.3x its size:

Benchmark	VisionPsy-Nano-460M	FastVLM-0.5B	Qwen3.5-0.8B	InternVL3.5-1B
ScienceQA	86.5	84.7	71.6	79.7
MM-IFEval (instruction following)	42.3	21.1	36.9	33.6
POPE (hallucination robustness)	87.9	86.5	87.3	86.0

MM-IFEval matters more than it looks: it measures whether the model actually does what you asked. Leading every larger model tested on that axis is the difference between a model you can build a product on and one you have to babysit.

Flash: the first token, on a real phone

The cost that decides whether on-device vision feels usable is not the model size, it is how long you wait after pointing the camera. That wait is driven by the number of visual tokens the model has to read. Flash uses 64 visual tokens where the full model uses 1088, and it keeps the quality: within about 1% of the full model (61.4 vs 62.3 normalized).

The result, measured on real phones with quantized GGUF builds at 512x512:

Device	Flash, time to first token
iPhone 15	0.3s
Galaxy S25 Ultra	2.6s
Galaxy S23	5.9s
Pixel 9	6.1s

That is 11.9x to 25.1x faster to the first token than nanoVLM-460M and SmolVLM2-500M on the same runtime, and still 1.3x to 2.4x faster than LFM2.5-VL-450M and 2.3x to 3.5x faster than Qwen3.5-0.8B across all four devices. Peak memory drops 24% to 39% on Android and 41% to 65% on iPhone 15 against the full-token baselines.

Built to run locally, not shrunk to fit

VisionPsy-Nano is not a cloud model trimmed down until it could run on a phone. Running on the device was the design constraint from the start, and it shaped every choice: the parameter budget, the visual-token budget that Flash pulls on, the decision to optimize for a single image per query, and the quantized builds that ship next to the full-precision weights. A phone hands one app a small slice of RAM and no dedicated VRAM, so a model that is going to live there has to be small, quantizable, and fast to the first token. That is what it was built to be.

What that buys is what local AI is for. The image is understood where it was taken, so it never has to leave the device to be read. The model keeps working with no signal. And it fits the situations it was scoped for: asking questions about what the camera is pointed at, reading documents, charts and diagrams, pulling text out of a scene, and following light visual instructions.

How it got there: Failure-driven, not scale-driven

The result did not come from scaling data. It came from finding out precisely where the model was wrong and fixing that.

Tether AI Research built an agentic failure-discovery harness: a larger teacher model interrogates the small model with realistic questions on real images, grades every answer with the image in view, then clusters what it found into a weakness catalogue that steers the next round. Over 300 rounds and roughly 18,000 probes, it mapped concrete failure modes: misread prices and license plates, reading a chart value correctly but failing to sum a column, confusing left and right, and degenerate repetition loops.

Those clusters became the training signal. A five-stage post-training pipeline then turns a good base model into a best-in-class one: broad supervised fine-tuning, capability-targeted fine-tuning, weakness-targeted fine-tuning on teacher-verified synthetic data, merging the resulting capability specialists into a single checkpoint with TIES so their strengths accumulate at no extra inference cost, and a final preference-alignment stage that kills the repetition loops.

Get it

Open weights under Apache 2.0, these models are intended to support researchers and for educational purposes, with three ways to run them today: full precision via Transformers, quantized GGUF builds for on-device inference through llama.cpp, and vLLM for high-throughput server inference. Support in the QVAC SDK is coming.

Every model in the report was scored under one VLMEvalKit harness with the eval configs and judge setup published, so the benchmark numbers are reproducible.

* References to Tether AI Research are references to Tether Data, S.A. de C.V.

References:

Hugging Face VisionPsy-Nano Collection: https://huggingface.co/collections/qvac/visionpsy

Hugging Face Blog Article: https://huggingface.co/blog/qvac/visionpsy

Run OpenCode locally with QVAC: a coding agent with no cloud

Thomas — Wed, 29 Jul 2026 09:22:43 +0000

What OpenCode is

OpenCode is an open-source coding agent. You give it a goal in plain language, and it plans, reads and writes files, runs commands, and reports back. It lives in your terminal or in a browser interface, and it works the way a teammate would: open the repo, make the change, run it, fix what broke.

The detail that matters for this guide is how OpenCode talks to a model. It speaks to any endpoint that exposes an OpenAI-compatible API. That means it is not tied to a single cloud vendor. You point it at whatever model endpoint you want, including one running on your own machine.

That is the whole idea here. OpenCode for the agent loop, a local model for the intelligence, both on hardware you control.

// Detect dark theme var iframe = document.getElementById('tweet-2069412641416843506-783'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=2069412641416843506&theme=dark" }

Why local is the right default for a coding agent

When a model only chats, where it runs is mostly a question of privacy. When a model drives a coding agent that reads your repository and runs commands, it becomes a question of control. The agent acts on your machine. If the model behind it lives in someone else's datacenter, every file it reads and every command it runs depends on a system you do not own: its uptime, its latency, its terms, and its access policy, any of which can change without notice.

Running the model locally changes the property, not just the policy:

Your code stays on the machine. The files the agent reads, the code it writes, the commands it runs: none of it leaves your disk to reach the model.
It works offline. Once the model is downloaded, you can disconnect entirely and keep working. Turn off the network and the agent still answers.
No per-token bill. A long coding session that would run up a metered API costs nothing locally.
No silent model swaps. The weights are a file on your disk. They do not change under you between sessions.

For a coding agent, that adds up to a simple property: the work, and the intelligence doing it, are both yours.

QVAC, and the tools you already use

QVAC is the piece that makes the local half practical. It is an open-source SDK and CLI from Tether that runs models on your own device across every major GPU backend (NVIDIA, AMD, and Intel through Vulkan, and Apple Silicon through Metal), and it ships an OpenAI-compatible server. That server is the bridge a coding agent plugs into.

It is also not a one-tool trick. QVAC is a first-class local provider for OpenCode and OpenClaw, and because the server speaks the standard OpenAI API, it works with the other coding tools developers already use, including Cline, Aider, Continue, and Roo. For OpenCode specifically there is a dedicated plugin, @qvac/opencode-plugin, that wires the whole thing up from one line of config (below). You bring your own model, running locally, to the agent you already like. QVAC is Apache 2.0, with no API keys, no rate limits, and no per-token cost.

What it looks like

Running Qwen3.6-35B-A3B locally through QVAC (downloaded once, then with the network off), OpenCode handled three ordinary developer tasks back to back:

Build from a prompt. From one sentence, it wrote a self-contained page animating eight hundred glowing particles that drift and react to the cursor, then opened it in the browser. No libraries, every line generated on the machine.
Explain an unfamiliar repo. Asked what a repository does, it read the code and explained it: a solar-system simulation with planets orbiting on circular paths. Opening the page confirmed it.
Find and fix a bug. Handed a shopping cart whose total rendered as garbled text instead of a number, it located the cause, edited the file, and the total added up correctly.

All of it ran locally, on an open and free stack.

Which model to run

Coding agent work (reading a repo, generating a file, fixing a bug) asks more of a model than a chat does, so it pays to run a capable one. You select a model by id in the plugin config (step 2 below), and that id flows straight through to OpenCode's model picker.

That id is a QVAC model reference, not a path to a file you supply. On first run, QVAC downloads that model once into its own local store and serves it locally from then on, with no network. Because it pulls QVAC's own copy, a .gguf you may already have from another tool is not reused.

Use case	Model	RAM	Disk	Config id
Recommended default, runs on most recent laptops	Qwen3.5-4B (4-bit)	~8 GB	~3 GB	`QWEN3_5_4B_MULTIMODAL_Q4_K_M`
Balanced, the plugin's out-of-box default	Qwen3.5-9B (4-bit)	~12 GB	~6 GB	`QWEN3_5_9B_MULTIMODAL_Q4_K_M`
Strongest coding, used in the demo	Qwen3.6-35B-A3B (4-bit)	~32 GB	~21 GB	`QWEN3_6_35B_A3B_MULTIMODAL_Q4_K_M`

The demo uses Qwen3.6-35B-A3B: 35 billion parameters of knowledge, only about 3 billion active per token, so it stays fast despite its size. For most laptops, Qwen3.5-4B or the 9B default is the better starting point.

A GPU helps but is not required. QVAC uses Apple Metal on Apple Silicon and Vulkan on NVIDIA, AMD, and Intel; on a CPU-only machine it still runs, just slower. To check what your hardware supports before you start, run npx -y @qvac/cli doctor (no install needed).

Set it up

No second terminal and no manual server. OpenCode reads the QVAC plugin from your project config, brings up a managed QVAC server by itself, points itself at it, and shuts it down when you quit.

Requirement: Node.js 22.17 or newer (node --version, from nodejs.org). Disk and RAM depend on the model you pick: the demo's Qwen3.6-35B-A3B needs about 21 GB of disk and 32 GB of RAM; the lighter qwen3.5-9b default needs much less.

1. Install OpenCode.

npm install -g opencode-ai

2. Add the QVAC plugin to your project. In the folder you want to code in, create an opencode.json. The whole file is one line, using the plugin's default model (qwen3.5-9b):

{
  "$schema": "https://opencode.ai/config.json",
  "plugin": ["@qvac/opencode-plugin"]
}

To match the demo with the strongest model instead (heavier: about 21 GB of disk and 32 GB of RAM), name it explicitly:

{
  "$schema": "https://opencode.ai/config.json",
  "plugin": [["@qvac/opencode-plugin", { "model": "QWEN3_6_35B_A3B_MULTIMODAL_Q4_K_M" }]]
}

3. Run OpenCode.

opencode web        # browser UI, the most visual
opencode            # terminal UI
opencode run "..."  # one-shot

That is the whole setup. On first run the plugin installs itself, spawns a managed QVAC server in the background (downloading the model once), wires OpenCode to it as the local qvac provider, and reaps it when you quit. There is no qvac serve to run yourself, no provider block to write, and no toolsMode to set: the plugin handles all of it, and loads on startup whatever interface you use.

A good first test: ask it whether it is running in the cloud or on your machine, then turn off your network and ask again. It keeps answering, because nothing was being sent anywhere.

What to try next

From here, the same setup handles the rest of what a coding agent does. Point OpenCode at a project folder and have it read and edit code, generate a small app from a prompt, explain an unfamiliar repository, or track down a bug. The agent loop is identical to the cloud experience. The only thing that changed is that the model behind it is yours, on your hardware, with nothing leaving the machine.

Notes and limits

A model that fits on your machine is not a frontier cloud model. Expect it to need clearer, more focused instructions, and give it single-goal tasks rather than long open-ended ones.
Local responses can take longer than a cloud API today. On a single local worker, a tool-using turn with the 35B model runs in roughly 20 to 30 seconds; a smaller model is snappier. That gap is closing fast as local hardware speeds up and small models improve.
One QVAC worker runs per machine, and multiple OpenCode windows share it. If the OpenCode desktop app is open it can hold locks the terminal needs, so quit it when running opencode from the terminal.
QVAC is Apache 2.0 and free. The model is downloaded once and cached locally. Source and docs: github.com/tetherto/qvac and docs.qvac.tether.io.

How to (easily) run a fully local, private AI assistant with OpenClaw and QVAC

Thomas — Wed, 29 Jul 2026 09:22:29 +0000

What an agent harness is, and what OpenClaw does

A language model on its own can only produce text. It cannot open a file, run a command, call an API, or remember what it did five minutes ago. An agent harness is the layer that closes that gap. It takes the model's text output, turns it into real actions (read this file, run this command, search this folder), feeds the results back to the model, and loops until the task is done. The model is the brain. The harness is the hands.

OpenClaw is one of these harnesses, and right now it is the most widely used one. It crossed a large install base in a few months and sits at the top of agent usage charts. You give it a goal in plain language, and it plans, calls tools, and reports back. It speaks to any model that exposes an OpenAI-compatible API, which is the detail that matters for this guide: you are not locked to a single cloud vendor. You point it at whatever model endpoint you want, including one running on your own machine.

That is the whole idea here. OpenClaw for the agent loop, QVAC for the model, both on hardware you control.

// Detect dark theme var iframe = document.getElementById('tweet-2067624599270166813-984'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=2067624599270166813&theme=dark" }

What people use OpenClaw for

The harness is general, so the use cases are broad. The common ones:

Coding tasks. Read a repo, write a function, run the tests, fix what broke. The agent edits files and runs commands directly.
File and system chores. Rename a batch of files, summarize a folder of documents, reorganize a directory, pull a number out of a spreadsheet.
Local research and drafting. Read a set of notes and produce a summary, a draft, or a structured table.
Glue work. Chain a few steps together that would otherwise be a manual sequence: generate something, save it, open it, move it.
Always-on assistants. Run it on a small machine that stays on, and reach it from your laptop or phone.

In every one of these, the agent is taking actions on real files and a real system. Which is exactly why where the model runs starts to matter.

Why local AI is the right default for an agent

When a model only chats, where it runs is mostly a question of privacy. When a model drives an agent that reads your files and runs commands, it becomes a question of control. The agent acts on your machine, and if the model behind it lives in someone else's datacenter, every file it reads and every command it runs depends on a system you do not own: its uptime, its latency, its terms, and its access policy, any of which can change without you. Running the model locally keeps the agent answerable to your hardware, not to a service that can throttle it, change it, or cut it off.

Running the model locally changes the property, not just the policy:

Your data stays on the machine. The files the agent reads, the code it writes, the commands it runs: none of it leaves your disk to reach the model. There is no prompt log on a server you cannot see.
It works offline. Once the model is downloaded, you can disconnect entirely and the agent keeps working. Enable airplane mode and ask it a question. It still answers.
No per-token bill. The model runs on hardware you already own. A long agent session that would cost real money against a metered API costs nothing locally. And no risk of seeing the bill inflate over time due to rising costs for serving inference at scale.
No silent model swaps. The weights are a file on your disk. They do not change under you between sessions, a practice that some providers run to decrease their cost.

The trade is latency and raw capability: a model that fits on a laptop is smaller than a frontier cloud model, so it is slower and less sharp. For a large class of real work, that trade is worth it, and it keeps getting better as small models improve.

QVAC is the piece that makes the local half practical. It is an open-source SDK and CLI from Tether that runs models on your own device across every major GPU backend (NVIDIA, AMD, Intel, Adreno (Qualcomm) and Mali on Linux, Windows and Android through Vulkan, and Apple Silicon through Metal), and it ships an OpenAI-compatible server. That server is the bridge OpenClaw plugs into.

How it fits together

Three pieces, two of them long-running:

The QVAC server holds the model in memory and answers OpenAI-compatible requests on your local port of your own machine. You start it once and leave it running.
The OpenClaw gateway runs the agent loop. You point it at the QVAC server during setup, then leave it running too.
Your commands go to the gateway. You ask a question or hand it a task, and it works against the local model.

Nothing in this chain calls out to the internet for inference. The model file is downloaded once, then everything happens on the machine.

Which model to run

The setup uses Qwen3-8B, which is the best balance for most laptops. If you have less memory, drop to the 4B. If you have a workstation and want stronger results on multi-step coding tasks, step up to the Qwen3.6-27B multimodal model. To switch, change the model name in the config file (step 2 of the setup) to the constant in the last column.

Use case	Model	Recommended RAM	Required storage	Config name
Light and fast, mostly conversation	Qwen3-4B (4-bit)	8 GB	~3 GB	`QWEN3_4B_INST_Q4_K_M`
Recommended default, good all-round balance	Qwen3-8B (4-bit)	16 GB	~6 GB	`QWEN3_8B_INST_Q4_K_M`
Heavier coding, multi-step agents, and vision	Qwen3.6-27B multimodal (4-bit)	48 GB	~18 GB	`QWEN3_6_27B_MULTIMODAL_Q4_K_XL`

A GPU helps a lot but is not required. QVAC uses Apple Metal on Apple Silicon and Vulkan on NVIDIA, AMD, and Intel. On a CPU-only machine it still runs, just slower. Run qvac doctor to see what your hardware supports.

Set it up

The walkthrough below is pure copy-paste. Run each step in order, and you will have a local coding agent in a few minutes. The only thing that takes real time is the first model download. The setup uses Qwen3-8B quantized at 4 bits (about 4.7 GB), which downloads once and is cached after that.

There are two paths through it, and the steps below handle both:

If you do not have OpenClaw yet, run the optional install step.
If you already have OpenClaw, skip that one step. Everything else is the same.

Requirement: Node.js 22.17 or newer (check with node --version, get it from nodejs.org). About 5 GB of free disk for the model.

1. Install the QVAC CLI (all platforms)

npm install -g @qvac/cli @qvac/sdk

2. Create a model config. Makes a folder and writes one small config file.

macOS and Linux:

mkdir -p ~/qvac-openclaw && cd ~/qvac-openclaw
cat > qvac.config.json <<'EOF'
{
  "plugins": ["@qvac/sdk/llamacpp-completion/plugin"],
  "serve": {
    "models": {
      "qwen3-8B-Q4-chat": {
        "model": "QWEN3_8B_INST_Q4_K_M",
        "type": "llamacpp-completion",
        "preload": true,
        "config": { "tools": true, "toolsMode": "static", "ctx_size": 16384, "gpu_layers": -1, "reasoning_budget": 0 }
      }
    }
  }
}
EOF

Windows (PowerShell):

mkdir $HOME\qvac-openclaw; cd $HOME\qvac-openclaw
@'
{
  "plugins": ["@qvac/sdk/llamacpp-completion/plugin"],
  "serve": {
    "models": {
      "qwen3-8B-Q4-chat": {
        "model": "QWEN3_8B_INST_Q4_K_M",
        "type": "llamacpp-completion",
        "preload": true,
        "config": { "tools": true, "toolsMode": "static", "ctx_size": 16384, "gpu_layers": -1, "reasoning_budget": 0 }
      }
    }
  }
}
'@ | Set-Content -Encoding utf8 qvac.config.json

3. Start the QVAC server (all platforms). Run it in that folder and leave the terminal open. qwen3-8B-Q4-chat is the alias you defined in step 2; it serves Qwen3-8B at 4-bit (Q4_K_M). First run downloads the model once (about 4.7 GB). Ready when you see QVAC API server listening.

qvac serve openai --model qwen3-8B-Q4-chat

4. Install OpenClaw (optional). Skip if you already have it.

npm install -g openclaw

5. Point OpenClaw at QVAC (new terminal). The first command connects OpenClaw to the local server. The next three keep the agent fast and reliable on a local model.

openclaw onboard --auth-choice custom-api-key --custom-base-url http://127.0.0.1:11434/v1 --custom-model-id qwen3-8B-Q4-chat --custom-api-key "qvac" --non-interactive --accept-risk --skip-channels --skip-daemon --skip-search --skip-ui --skip-skills --skip-health

Two values here tie back to step 2. --custom-model-id qwen3-8B-Q4-chat must match the model alias in your config (the key under serve.models). The --custom-api-key "qvac" is only a placeholder: the local QVAC server does not require a key, but OpenClaw's setup needs the field filled, so any non-empty value works.

openclaw config set tools.profile coding
openclaw config set tools.allow '["write","read","exec"]' --strict-json
openclaw config set models.providers.custom-127-0-0-1-11434.timeoutSeconds 600

6. Start the agent gateway (new terminal, leave open). Ready when you see ready.

openclaw gateway run

7. Talk to your local agent (new terminal). Try turning off your network and asking again.

openclaw agent --agent main --message "Are you running in the cloud or on my machine? Answer in one sentence."

8. Have it build something (optional). Asks the agent to write a small animated web page and open it. On a local model it takes a minute or two. The open command differs per platform: open on macOS, xdg-open on Linux, start on Windows.

openclaw agent --agent main --message "Create an HTML file at ~/openclaw_lobster.html showing a large lobster emoji at 140px pulsing with a CSS scale animation on a dark #0f1410 background, with the text 'openclaw running on local with QVAC' in teal #16E3C1 monospace below it, fully visible immediately. Then run the shell command: open ~/openclaw_lobster.html"

What you just built, and what to try next

You now have a coding agent running entirely on your own machine. A good first test is to confirm the obvious: ask it whether it is running in the cloud or on your machine, then turn off your network and ask again. It keeps answering, because nothing was ever being sent to a server in the first place.

For something you can see, hand it a small build task and watch it write a file and open it. A coding task like generating a small web page runs in a minute or two on a typical laptop, fully offline, because the model is doing real work locally rather than streaming from a datacenter. The setup includes a ready-made example: it asks the agent to write a tiny animated web page and open it in your browser, with no further input from you.

From here, the same setup handles the rest of what OpenClaw does. Point it at a project folder, ask it to read and edit code, have it summarize a directory, or give it a multi-step chore. The agent loop is identical. The only thing that changed is that the intelligence behind it is yours.

Notes and limits

A model that fits on a laptop is not a frontier cloud model. Expect it to be slower and to need clearer instructions. For agent work, give it focused, single-goal tasks rather than long open-ended ones.
Depending on your hardware, a local response can take longer than a cloud API today. Treat that as a temporary gap, not a permanent cost: local hardware keeps getting faster and capable models keep getting smaller while holding their quality, so the gap is closing fast. The physics also runs in local's favor. A request to a datacenter is bounded by the speed of light, a round trip no provider can engineer away. For anything that has to react in real time, a robot, a control loop, a live interface, that round trip is a hard floor, and local is the only option that stays reliable.
QVAC is Apache 2.0 and free. The model you run is downloaded once and cached locally. Source and docs: github.com/tetherto/qvac and docs.qvac.tether.io.

How to run Hermes, a self-improving personal AI agent, fully local with QVAC

Thomas — Fri, 24 Jul 2026 14:49:18 +0000

What Hermes is, and why it is different

Most AI agents you have seen are task tools. You give them a job, they do it, they forget you. Hermes Agent, from Nous Research, is built on a different idea: an agent that is yours, that remembers you, and that gets better the longer you use it.

Three things make it stand out:

It has a persona. A single file, SOUL.md, defines how it talks. Edit it and the agent changes character on the next message.
It remembers. Conversations, facts, and context persist across sessions in a local memory, so it is a companion, not a one-shot.
It learns. Hermes has a library of skills (research, notes, GitHub, documents, smart home, and many more) and a background curator that creates and refines skills from experience. The agent that finishes your week is not the one that started it.

All of that personality, memory, and learned skill is data about you. Which raises the obvious question: where does the model behind it run?

// Detect dark theme var iframe = document.getElementById('tweet-2077391578310791482-199'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=2077391578310791482&theme=dark" }

Why local is the right default for a personal agent

When a model only answers questions, where it runs is mostly about privacy. When a model is your personal agent, holding your persona, your memory, and skills it learned from your own habits, it becomes about ownership. If the brain lives in someone else's datacenter, then your assistant's memory of you, and everything it learns from you, flows through a system you do not control: its uptime, its terms, its access policy, all of which can change without you.

Running the model locally changes the property, not just the policy:

Your context stays on the machine. Your persona, your memory, the skills the agent builds: none of it has to leave your disk to reach the model.
It works offline. Once the model is downloaded, you can disconnect entirely and keep talking to your agent. Turn on airplane mode and it still answers.
No per-token bill. A personal agent you talk to all day, that runs scheduled jobs in the background, would meter up fast against a cloud API. Locally it costs nothing per token.
No silent model swaps. The weights are a file on your disk. Your agent's "brain" does not change under you between sessions.

The trade is latency and raw capability: a model that fits on a laptop is smaller than a frontier cloud model. For a personal agent that lives with your data, that trade is easy, and it keeps getting better as small models improve.

QVAC is the piece that makes the local half practical. It is an open-source SDK and CLI from Tether that runs models on your own device across every major GPU backend (NVIDIA, AMD, and Intel through Vulkan, Apple Silicon through Metal), and it ships an OpenAI-compatible server. That server is the bridge Hermes plugs into.

How it fits together

Two long-running pieces and you:

The QVAC server holds the model in memory and answers OpenAI-compatible requests on a local port. Start it once and leave it running.
Hermes runs the agent: persona, memory, skills, tools. You point it at the QVAC server once, then just talk to it.
Your messages go to Hermes, which thinks using the local model and acts with its tools.

The model inference never leaves the machine. (Some skills, like web research, still use the internet for that specific tool, the same way a browser does. The brain stays on-device.)

Which model to run

Hermes is tool-heavy: it sends a large set of tools on every turn, so it pays to run a capable model. The setup below uses Qwen3.6-35B-A3B, a mixture-of-experts model: 35 billion parameters of knowledge, only about 3 billion active per token, so it stays fast for its size. To switch models, change the constant in the config file (step 2 of the setup).

Use case	Model	Recommended RAM	Required storage	Config name
Lightest, runs almost anywhere	Qwen3.5-4B (4-bit)	8 GB	~3 GB	`QWEN3_5_4B_MULTIMODAL_Q4_K_M`
Balanced	Qwen3.5-9B (4-bit)	16 GB	~6 GB	`QWEN3_5_9B_MULTIMODAL_Q4_K_M`
Recommended for the full agent	Qwen3.6-35B-A3B MoE (4-bit)	36 GB	~22 GB	`QWEN3_6_35B_A3B_MULTIMODAL_Q4_K_M`

Hermes leans on tool calls, and smaller models are weaker at that, so on the 4B or 9B keep the active skill set lean (see "What to try next"). The 35B-A3B MoE handles the full toolset most reliably.

Be realistic: these are small local models, and tool-heavy agent work is where they struggle most. The 4B is for trying the setup, the 9B mainly adds reliability over the 4B rather than smarts, and the 35B-A3B MoE is the only one that drives the full toolset dependably, so it is what we recommend. It is a capable on-device assistant, not a frontier cloud model.

A GPU helps a lot but is not required. QVAC uses Apple Metal on Apple Silicon and Vulkan on NVIDIA, AMD, and Intel. Run qvac doctor to see what your hardware supports. On the 35B-A3B, expect the first turn to take around 20 seconds (Hermes sends a large system prompt plus its tools), and faster turns after. For a snappier agent, narrow the active skills (see "What to try next").

Set it up

The walkthrough is copy-paste. Run each step in order. The only slow part is the first model download, which is cached after that.

Requirement: Node.js 22.17 or newer (node --version, from nodejs.org). Disk and RAM per the model table above.

1. Install the QVAC CLI (all platforms)

npm install -g @qvac/cli

2. Create a model config. Makes a folder and writes one small config file.

macOS and Linux:

mkdir -p ~/qvac-hermes && cd ~/qvac-hermes
cat > qvac.config.json <<'EOF'
{
  "plugins": ["@qvac/sdk/llamacpp-completion/plugin"],
  "serve": {
    "models": {
      "qwen3.6-moe": {
        "model": "QWEN3_6_35B_A3B_MULTIMODAL_Q4_K_M",
        "type": "llamacpp-completion",
        "default": true,
        "preload": true,
        "config": { "tools": true, "toolsMode": "static", "ctx_size": 32768, "gpu_layers": -1, "reasoning_budget": 0 }
      }
    }
  }
}
EOF

Windows (PowerShell):

mkdir "$HOME\qvac-hermes"; Set-Location "$HOME\qvac-hermes"
@'
{
  "plugins": ["@qvac/sdk/llamacpp-completion/plugin"],
  "serve": {
    "models": {
      "qwen3.6-moe": {
        "model": "QWEN3_6_35B_A3B_MULTIMODAL_Q4_K_M",
        "type": "llamacpp-completion",
        "default": true,
        "preload": true,
        "config": { "tools": true, "toolsMode": "static", "ctx_size": 32768, "gpu_layers": -1, "reasoning_budget": 0 }
      }
    }
  }
}
'@ | Out-File -Encoding utf8 qvac.config.json

The two settings that matter: "tools": true and "toolsMode": "static". Static is what lets an external client like Hermes run the model's tool calls itself. (With "dynamic" the tool calls never execute, which looks like the model is broken.)

3. Start the QVAC server (all platforms). Run it in that folder and leave the terminal open. First run downloads the model once (about 22 GB for the MoE). Ready when you see QVAC API server listening.

qvac serve openai --model qwen3.6-moe

It listens on http://localhost:11434/v1. That is the whole command: --model qwen3.6-moe is the alias you defined in step 2, and running from the ~/qvac-hermes folder lets QVAC auto-detect that qvac.config.json (which is also what sets toolsMode: static). Run it somewhere without that config and it will error that the alias is unknown.

4. Install Hermes (new terminal). The installer sets up Python, the global hermes command, and its dependencies.

curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash

On Windows, run that installer inside WSL (Windows Subsystem for Linux). The QVAC server from step 3 runs natively on Windows, and Hermes in WSL reaches it over localhost either way.

5. Point Hermes at QVAC. Tell Hermes to use your local server as a custom OpenAI-compatible provider.

hermes config set model.provider custom
hermes config set model.base_url "http://localhost:11434/v1"
hermes config set model.default qwen3.6-moe
hermes config set model.api_key qvac

qwen3.6-moe must match the alias in your config (the key under serve.models). The api-key value is a placeholder: the local QVAC server does not check it, but Hermes wants the field filled, so any non-empty value works. (You can also run the interactive hermes model wizard and pick "custom" instead of these four commands.)

6. Talk to your local agent.

hermes

That opens an interactive chat. Try turning off your network and asking it whether it runs in the cloud or on your machine. It keeps answering.

What you just built, and what to try next

You now have a personal AI agent running entirely on your own machine. A good first test is the obvious one: ask it whether it is in the cloud or local, then enable airplane mode and ask again. It keeps answering, because the inference was never leaving your device.

From there, the things that make Hermes Hermes all run on the same local model:

Give it a personality. Edit ~/.hermes/SOUL.md, then send a message. The persona is loaded fresh each time, so the change is immediate.
Let it remember. Tell it something about how you work, start a new session later, and it recalls it. The memory lives in ~/.hermes/.
Use a skill. Run hermes skills to see the library, then ask it for something real: take structured notes, organize a project on its task board, summarize a folder of documents. Pick local skills (notes, files, board) to stay fully offline; web skills will use the internet for that tool.
Let it improve. Over time the curator refines skills from what you do. That learning happens on your hardware and stays there.

For speed, narrow the active skills to what you actually use (hermes skills, hermes tools); a smaller toolset means a smaller prompt and faster turns on a local model.

Notes and limits

A model that fits on a laptop is not a frontier cloud model. Expect it to need clearer instructions and to be slower than a cloud API today. That gap is closing as local hardware speeds up and small models improve.
"Local" here means the model runs on your device. A given skill may still call the internet for its own job (web research, for example), the same as any app would. The intelligence, your persona, and your memory stay on the machine.
QVAC is Apache 2.0 and free. Hermes Agent is from Nous Research (github.com/NousResearch/hermes-agent). The model is downloaded once and cached locally. QVAC source and docs: github.com/tetherto/qvac and docs.qvac.tether.io.

Meet MedPsy: a private medical AI model small enough for your phone

Thomas — Fri, 24 Jul 2026 07:05:58 +0000

QVAC MedPsy is a free, open-source medical AI model that can run entirely on your own device, with no cloud and no data leaving your phone. It ships in two sizes, 1.7B and 4B. On standard medical benchmarks, the 4B version matches or beats Google's MedGemma-27B, a model nearly seven times larger, while the 1.7B version is small enough to run on an ordinary smartphone.

The headline is simple: better training beat raw size. MedPsy reaches the level of models two to seven times bigger, not by being huge, but by being trained more carefully on higher-quality medical data. And because you can run locally, the most sensitive data there is, your health, never has to travel to someone else's server.

A private medical AI model built to run on your device

MedPsy is a family of small medical language models developed by Tether Data's AI Research group as part of the QVAC platform, Tether's open-source, local-AI ecosystem. It answers everyday medical questions and walks through clinical reasoning in clear, plain text. There are two sizes: MedPsy-1.7B for phones, and MedPsy-4B for high-end phones and laptops. Both are built on open Qwen3 models and released under the Apache 2.0 license for research and educational use. Both models are available on Hugging Face, in their full version and as smaller compressed versions that run efficiently with llama.cpp or the QVAC SDK. The QVAC SDK provides out-of-the-box optimal execution support, offering a simple library that runs locally on any device, from a phone to a laptop or server with seamless cross-platform behavior, delegated inference and peer-to-peer (P2P) capabilities.

Disclaimer: MedPsy is a general medical model for question-answering and clinical reasoning. It is not a mental health, therapy, or crisis-support tool, and has not been trained or validated for that.

Better training, not raw size

In AI, how you train a model matters as much as how big it is. The QVAC team focused on the quality of the medical training data and a careful, multi-stage training process aimed squarely at medicine, instead of simply scaling up parameters. A model that specializes deeply in one field can outperform a much larger model that tries to know everything.

The training data was built using QVAC Genesis II synthetic medical datasets, which cover diverse medical domains such as biology and chemistry along with publicly available open-source medical QA prompts. The datasets were used as seeds by a strong teacher model (Baichuan-M3-235B) to generate chain-of-thought traces, extended rationales, and decision-oriented answers to improve the model's quality. For more information about the training methodology, see here.

There is a practical bonus. MedPsy-4B reaches its answers using about 3.2 times fewer tokens than the general model it is based on. Fewer tokens means faster replies and less battery used, which is exactly what you want on a phone.

How MedPsy compares to Google's MedGemma

On the average across closed-ended medical benchmarks, MedPsy-4B scores 70.54 and edges out MedGemma-27B at 69.95, despite being nearly seven times smaller. It leads most clearly on the hardest, most realistic tests: HealthBench, HealthBench Hard and the expert-level MedXpertQA. On a few academic multiple-choice tests it runs a point or two behind. Here is the honest side-by-side.

Medical benchmark	MedPsy-4B	MedGemma-27B
Average (closed-ended)	70.54	69.95
HealthBench (realistic care)	74.00	65.00
HealthBench Hard	58.00	42.00
MedXpertQA (expert level)	30.61	25.18
MedQA-USMLE	84.39	83.29
PubMedQA	75.00	71.93
MedMCQA	72.15	72.77
AfriMedQA	71.50	73.07
MMLU Health	89.70	90.48
MMLU-Pro Health	70.45	72.94

The smaller model tells the same story. MedPsy-1.7B scores 62.62 on average and beats Google's MedGemma-1.5-4B (51.20) by more than 11 points, while being less than half its size and small enough to run on a normal smartphone.

Why running on your device matters

Health data is the most sensitive data most of us will ever generate. The usual way medical AI works is to send your question to a company's servers in the cloud. MedPsy flips that: it runs 100% on your device, so your symptoms, questions and notes stay with you. There is no account, no subscription, and nothing to leak.

Running locally also makes it work where the cloud cannot. It keeps working with no internet, on a plane, in a remote clinic, or anywhere bandwidth is scarce. And because it fits on a phone people already own, it does not need an expensive server or a data center to be useful, which matters most in the places that have the least access to care.

How to try MedPsy

The simplest path is the QVAC SDK. Install it, pull a MedPsy model from Hugging Face, and run it on your device:

npm install @qvac/sdk

No API key, no cloud bill. Get the models for free and get started.

MedPsy on Hugging Face

All MedPsy models, GGUF files, quantized variants, and resources in one place: open the collection.

MedPsy-4B: higher-quality edge model. Surpasses MedGemma-27B-text-it on closed-ended medical benchmarks at about 7x smaller. Model card.
MedPsy-1.7B: smartphone-class medical model. Beats MedGemma-1.5-4B-it by +11.42 points on closed-ended; matches Qwen3-4B-Thinking-2507. Model card.
MedPsy-4B-GGUF: GGUF repo with an unquantized BF16 export and seven quantized files. Q5_K_M (3.16 GB) adds a high-quality 5-bit tier; Q4_K_M (2.72 GB) remains the recommended size/quality trade-off. GGUF repo.
MedPsy-1.7B-GGUF: smartphone-ready GGUF repo with an unquantized BF16 export and seven quantized files. Q5_K_M (1.47 GB) is nearly lossless; Q4_K_M (1.28 GB) is the best mobile trade-off. GGUF repo.

Important: MedPsy is not a substitute for professional medical judgment, diagnosis, or treatment, and must never be used in emergencies or as a sole decision-making tool. Like any language model, it can produce wrong or incomplete answers. It is text-only, English-only, and released for research and educational purposes. Always consult a qualified healthcare professional.

Frequently asked questions

Is QVAC MedPsy free?

Yes. MedPsy is open source under the Apache 2.0 license and free to download from Hugging Face, released for research and educational purposes.

Does my data leave my device when I use MedPsy?

No. MedPsy is designed to run entirely on your own device through the standard llama.cpp inference engine and the QVAC SDK. There is no cloud call, so your prompts and any health information you type never leave the device.

Is MedPsy a mental health or therapy AI?

No. MedPsy is built for general medical question-answering and clinical reasoning. It is not trained or validated for mental health, therapy, or crisis support.

Can MedPsy replace a doctor?

No. MedPsy is not a substitute for professional medical judgment, diagnosis, or treatment, and it should never be used for emergencies. Its answers can contain mistakes and should always be checked with a qualified clinician.

How does MedPsy compare to Google MedGemma?

On the closed-ended medical benchmark average, MedPsy-4B scores 70.54 versus 69.95 for MedGemma-27B, despite being nearly seven times smaller. It leads clearly on the hardest realistic health tests, such as HealthBench (74 vs 65) and HealthBench Hard (58 vs 42).

What is the difference between MedPsy-1.7B and MedPsy-4B?

MedPsy-1.7B is small enough to run on an ordinary smartphone and beats Google's MedGemma-1.5-4B by over 11 points. MedPsy-4B is more capable, targets high-end phones and laptops, and matches models nearly seven times its size.

References

Nothing is more private than a thought: how QVAC runs a brain-to-text model fully on-device

Thomas — Fri, 24 Jul 2026 06:48:02 +0000

TL;DR

Nothing is more private than a thought. So when a model can decode speech directly from the brain, where it runs matters as much as how well it works. QVAC, Tether's open-source on-device AI stack, now runs exactly that kind of model fully on the device, with nothing sent to a cloud. The model is BrainWhisperer, built by Tether Evo's research team and now available in the QVAC SDK as a proof-of-concept capability. The version in the QVAC SDK is a small, end-to-end model: on real neural recordings it decodes more than 90% of words correctly for a single participant, an 8.7% word error rate, while running in under 2 GB of memory. A larger, more complex version of the same research placed 4th of 466 teams in an international challenge. It is early, but it puts a brain-to-text engine in developers' hands, locally, as a foundation for the products that could one day give people their voice back.

QVAC and the model

QVAC is Tether's open-source stack for running AI on your own device, with no cloud. Most of what it runs is familiar: language models, speech, vision, image generation. Its newest and most experimental addition is a brain-computer interface: it can take neural signals and decode them into text, on-device, through a single SDK call.

The model behind that capability is BrainWhisperer, built by Tether Evo's research team. It is integrated into the QVAC SDK at proof-of-concept level, and that is the important framing. It is not a product. It is a working engine that developers can run locally today, and a foundation for the assistive products that could be built on top of it tomorrow. The rest of this article is what it does, how well it does it, and why running it on-device is the whole point.

The hardest kind of silence

Imagine knowing exactly what you want to say and not being able to say it.

For people living with ALS and similar conditions, that is daily life. As the disease progresses, it takes control of the muscles: the arms, the hands, and eventually the face and the voice. Some can still try to form sounds, but the words come out unintelligible. Others cannot produce sound at all.

What does not change is the mind. The thoughts are still there. The language is still there. The person still composes full sentences, still has things to tell the people they love. The connection between the brain and the world is what breaks, not the brain itself. The result is a particular kind of isolation: present, aware, and unable to be heard.

For decades, the workarounds have been slow. Communication by tracking eye movements, letter by letter, is exhausting and error-prone. A full sentence can take minutes. That is the problem Tether Evo set out to change.

Reading the words from the source

Traditionally, people who cannot speak communicate through eye-tracking, choosing letters one at a time, which is slow and error-prone. BrainWhisperer goes straight to the source and reads the brain directly.

When a person tries to speak, a specific part of the brain, the speech motor cortex, lights up with electrical activity, the same way it would if the muscles were responding. A small implant placed there picks up that activity. BrainWhisperer takes those neural signals and translates them into text. You do not have to move your face. You think the words, and the model works as a translation layer, turning the language of the brain into English.

The clever part is where the model comes from. BrainWhisperer is a redesign of Whisper, the same family of speech-recognition AI that powers modern transcription, trained on hundreds of thousands of hours of human speech. Tether Evo rebuilt it to take neural data as input instead of audio. So instead of learning to turn sound into text from scratch, with the tiny amount of brain data that exists, it inherits everything a large speech model already knows about how language works, and applies it to the brain.

Once the words are decoded, they can go anywhere text can go: read aloud in a synthetic voice, or handed to an AI assistant to act on.

What the results actually show

This is the part that matters, because it is measured, not promised.

Tether Evo tested BrainWhisperer on real recordings from people implanted with a 256-channel array in the speech motor cortex, using a public research dataset, and then entered it into the Brain-to-Text '25 challenge.

It decoded a large majority of words correctly. On many sentences the transcription is exact, word for word: "i just bought a new house", "do you believe in the dallas cowboys", "that was the point", all decoded perfectly.
Measured as word error rate, the standard accuracy metric, the version in the SDK reaches 8.7% for a single participant in an open vocabulary setting. That is well below the 10 percent mark researchers treat as the threshold for real-world usefulness, and it comes from a fully end-to-end model.
A separate, more complex version of the model placed 4th out of 466 teams in the Brain-to-Text '25 challenge. It scores higher, but it is not end-to-end, so the SDK ships the smaller end-to-end model, which is the direction we want to build in.
And it does this efficiently. The fast version runs in under 2 GB of memory with roughly 50 milliseconds of delay, compared to the hundreds of gigabytes of memory older approaches needed.

That last point is not a detail. It is the difference between a system that lives in a data center and one that can run on the device in front of the person.

Why local matters here more than anywhere

A thought is the most private thing a person has.

A speech decoder reads intention straight from the brain. If that has to travel to a server to be processed, you are sending the most intimate data a person has across a network. Because BrainWhisperer is light enough to run on a local device, the decoding can happen where the person is, with nothing leaving their hands. That is the role of QVAC, Tether's open-source on-device AI stack: it is what lets a model like this run anywhere, privately, with no cloud in the loop.

The control stays with the person in another way too. The system only decodes speech the user is actively trying to produce. A person can simply choose not to, or think about something else, and there is nothing to read. Further ahead, ideas like "mental passwords" could ensure these systems always answer to their user and no one else.

Where this stands today, honestly

This is research, and it is important to be clear about that.

BrainWhisperer is a proof of concept, now available in the QVAC SDK as an experimental brain-computer-interface capability. It shows two things: that intended speech can be decoded from the brain accurately, and that it can run on ordinary local hardware thanks to QVAC. What it is not, yet, is a product you can use. What it is, is a working engine a developer can run locally today and build toward those products with.

The honest limits: the kind of brain implant this needs is rare, and getting one involves real medical, safety, and regulatory hurdles. The models are trained on very few people, because very few people have these implants and only pre-recorded research data was available. The main model was trained on 6 participants; the smaller version in the SDK on a single one. It generalizes across the participants it has seen, which is itself a milestone, but a brand-new person would still need an implant and a short calibration period to record their own data before the model could adapt to them. There is real work between here and a usable system.

We would rather say that plainly than oversell it.

The frontier ahead

Today, the target is speech. But speech is one signal among many that the brain produces.

The same idea, reading the brain's own activity and translating it into something we can share, points at a much wider horizon. The team sees a path toward decoding not just words, but the images a person pictures in their mind, the sounds they imagine, one day perhaps intended movement. We are not putting a date on any of it. It is the frontier we are walking toward, one careful result at a time.

A team at the edge of what is possible

BrainWhisperer comes out of Tether Evo, Tether's research team working where technology, fundamental science, and medicine meet. Bringing their work into the QVAC SDK is what turns a research result into something a developer can run on a device today, and build the next assistive product on tomorrow. The thread is simple: use technology to make a life better, starting with the people who need it most.

Read the research: BrainWhisperer, arXiv 2603.13321.

FAQ

What is BrainWhisperer?

A research model that decodes intended speech directly from brain activity and turns it into text. It was built by Tether Evo on top of a large speech-recognition model adapted to read neural signals, and is now available in the QVAC SDK as a proof-of-concept capability.

Who is it for?

People who have lost the ability to speak, for example through ALS or severe paralysis, but whose ability to think and form language is intact.

How accurate is it?

On real neural recordings, the model in the QVAC SDK reaches an 8.7% word error rate for a single participant in an open vocabulary setting, so more than 90% of words are decoded correctly. A separate, more complex version placed 4th of 466 teams in the Brain-to-Text '25 challenge; it scores higher but is not end-to-end, which is why the SDK ships the smaller end-to-end model.

Is my brain data sent to the cloud?

No. The model is small enough to run on a local device, so decoding can happen without sending neural data anywhere, and the system only reads speech the user is actively trying to produce.

Can I use it today?

As a developer, yes. The brain-to-text capability ships in the QVAC SDK as a proof of concept you can run and experiment with locally today. What it is not yet is a finished product for end users: real-world use by a patient still depends on a specialized brain implant, which carries significant medical and regulatory hurdles, plus a short per-person calibration.

Key takeaways

BrainWhisperer reads intended speech from the brain and turns it into text, for people who cannot speak.
It decodes more than 90% of words correctly for a single participant, an 8.7% word error rate. A separate, more complex version ranked 4th of 466 teams in an international challenge.
It runs in under 2 GB on a local device, so the most private data a person has never has to leave their hands.
It is research, not a product. The frontier ahead includes imagined images, sounds, and movement, with no fixed timeline.

BrainWhisperer is a research model from Tether Evo, now a proof-of-concept capability in the QVAC SDK. QVAC is Tether's open-source on-device AI stack. Apache 2.0.

Quick-start: build your first local AI app with your coding agent

Thomas — Wed, 22 Jul 2026 14:20:56 +0000

TL;DR

You do not need to learn an SDK by heart to build a local AI app. You need to understand what local AI is good and bad at, know what is possible, and hand your coding agent the right context so it writes the code for you. This is that orientation: the trade-offs of running AI on your own hardware, the 12 things the QVAC SDK can do, a set of app ideas, and how to point your agent so it builds correctly. Almost no code here, on purpose.

Building this way? QVAC.md is the one file that makes your agent write QVAC code that runs the first time. Get it here.

How this guide works

This is not a copy-paste tutorial. You will not be pasting large code blocks. So this guide gives you the three things a builder actually needs:

The trade-offs of local AI, so you pick the right project for it.
What the SDK can do, so you know what is on the menu.
How to set up your agent, so the code it writes actually works.

What "local AI" actually means

Local AI means the model runs on your hardware (or your user's device), not behind a cloud API. That single fact creates real constraints and real advantages. Knowing both is how you choose good projects.

The constraints (know these going in)

Your hardware sets the ceiling. RAM and GPU decide which models you can run and how fast. A small model runs fine on a laptop CPU. Bigger and faster wants a GPU and more memory. As a rough guide, a 4B model needs a few GB; a 30B-class mixture-of-experts model needs around 20 GB or more.
A local model is not a frontier model. The largest cloud models are far bigger than anything that fits on a device. On the hardest reasoning, the broadest world-knowledge, and very long context, a local model is weaker. Use it for the jobs it is good at, not for everything.
The first run downloads the model (hundreds of MB to several GB), then it is cached and runs offline forever after.

The advantages (why it is worth it)

Private by construction. After the download, nothing leaves the device. No prompt logging, no data shipped to a third party.
No API key, no per-token bill. The cost is hardware you already own. Run it as much as you like.
Works offline. On a plane, in a clinic, on a factory floor, in the field, with no signal.
Can run 24/7. An always-on background agent, a daemon, a scheduled job. No metered API to drain your budget, no rate limits to throttle you.
Low latency. No network round-trip to a data center.
You own it. The weights are on your disk. Nothing changes under you.

The rule of thumb

Reach for local AI when privacy, offline use, always-on operation, high volume, or cost are what matter. Reach for a frontier cloud model when you need maximum reasoning or breadth and the data is not sensitive. Many of the best products use both: local for the private, high-frequency path, cloud for the occasional hard question.

What you can build: the 12 capabilities

QVAC is Tether's open-source local-AI SDK. One npm install gives you twelve AI tasks behind one consistent API, all running on-device:

Text and chat - a local LLM to assist, summarize, extract, or classify by prompt.
Vision - ask questions about an image, describe a photo, read a chart.
Embeddings - turn text into vectors for search and similarity.
RAG - search your own documents and answer with citations, locally.
Speech to text - transcribe audio and meetings, multilingual.
Text to speech - generate spoken audio in many languages, including voice cloning.
Translation - the Mozilla Bergamot engine, many language pairs.
Image generation - text to image, on-device.
OCR - read text out of scans and photos.
Classification - sort inputs into labels.
Image upscaling - enhance images on-device.
On-device fine-tuning - adapt a small model to a person or domain, trained on the device itself.

The point is the consistency: your agent learns the pattern once and can reach for any of the twelve.

Example apps to spark ideas

You do not build "an AI feature", you build a product. A few that play to local AI's strengths, each one you would simply describe to your coding agent or harness:

A private meeting transcriber and summarizer (speech to text + chat). Recordings never leave the laptop. Good for lawyers, doctors, journalists.
Chat with your notes (RAG + embeddings). A local "second brain" that answers from your own files, with citations, offline.
A private medical Q&A assistant (chat). Answer clinical questions on-device with MedPsy, a medical model small enough for a phone - nothing sent to a server.
An offline travel translator (translation + speech to text + text to speech). Works with no signal abroad.
An always-on document sorter (OCR + classification). A daemon that files incoming scans around the clock, with no per-page API cost.
An on-device image studio (image generation + upscaling). No cloud render bills, no usage caps.
A kiosk or in-car voice assistant (speech to text + chat + text to speech). Fully offline and private.
A personal-style writing model (fine-tuning). Adapt a small model to your own voice, trained on your machine, never uploaded. (See how it works in LoRA fine-tuning on edge GPUs via QVAC Fabric.)

Want to see real builds on QVAC? OpenClaw runs a fully local, private AI assistant, and you can even run your own coding agent on a local model with OpenCode. Pick an idea, describe its first slice to your agent, and iterate.

How to build it with your agent

The whole workflow is short:

Start an empty project and install the SDK (npm install @qvac/sdk; you need Node 22.17 or newer, which is above the usual LTS).
Give your agent QVAC.md. This is the step that matters. Coding agents are unreliable on SDKs they have not seen much of: they invent functions that do not exist. QVAC.md is a single file you drop into your agent's rules (a CLAUDE.md, an AGENTS.md, a Cursor or Cline rules file) that gives it the real API. With it, the agent writes code that runs.
Describe one capability at a time. For example: "Using the QVAC context, build a CLI that transcribes an audio file passed on the command line, fully locally."
Run it and iterate with your agent.

Cross-platform: what to keep in mind

The same QVAC code runs on desktop (macOS, Linux, Windows via Node) and on mobile (iOS and Android, native, via Expo). A few things worth telling your agent up front:

Match the model to the target device. A workstation can run a large model; a phone needs a small one. The model has to fit the device's memory, so choose per platform.
The GPU differs by platform (Apple Metal, Vulkan, or a CPU fallback). The SDK picks the right backend automatically, but speed varies, so test on the real device, not just your laptop.
The first download can be large. On mobile especially, show a progress state so users are not staring at a frozen screen on first launch.
Node 22.17+ on desktop. Below that, setup fails in confusing ways.

Get QVAC.md

Everything above is the human's half. The agent's half is QVAC.md: a single drop-in file with the real API surface, the model-type rules, the gotchas that stop hallucinations, and verified snippets. Paste it into your agent and it builds local AI apps that run.

Get QVAC.md for your coding agent on the original guide. Drop your email and we send you the drop-in file. Paste it into your agent and start building.

FAQ

Which coding agent does this work with?

Any of them. Cursor, Claude Code, Cline, Aider, OpenCode, or any harness that reads project rules or takes a system prompt. QVAC.md tells you where to paste it for each.

Do I need a GPU?

No. QVAC uses your GPU when one is available (Metal, NVIDIA, AMD, Intel) and falls back to CPU otherwise. Small models run fine on a laptop CPU; larger models are much faster with a GPU.

How does a local model compare to ChatGPT or Claude?

A model that fits on your device is smaller, so on the hardest reasoning and the broadest knowledge it is weaker than a frontier cloud model. In exchange it is private, offline, free to run, and always on. Pick the right tool for the job, and feel free to combine both.

Is my data sent anywhere?

No. After the model downloads once, inference happens entirely on-device. No API call, no prompt logging at run time.

Can this run on a phone?

Yes. The same code runs natively on iOS and Android through Expo, on the device hardware. Use a smaller model to fit the phone's memory. MedPsy is one example of a model built to run there.

Is it free? What is the license?

The SDK is Apache 2.0 and free, including commercial use. It is JavaScript and TypeScript, on Node 22.17+, Bare 1.24+, and Expo 54+.

Key takeaways

You build local AI apps by directing your coding agent, not by memorizing an SDK.
Local AI trades raw frontier capability for privacy, offline use, zero per-call cost, and 24/7 operation. Choose projects that want those.
One npm install gives you twelve on-device AI tasks behind one API.
QVAC.md is what makes your agent write correct QVAC code.
The same code runs on desktop and mobile; match the model to the device.

Next steps

Grab QVAC.md and paste it into your agent.
Pick one app idea above and describe its first slice to your agent.
Read the SDK docs and the model registry when you want to go deeper.

None of this is a big commitment. You are not adopting a framework, you are giving your agent the right context and letting it build. The intelligence in the app you ship will not be rented from a server you do not control; it will run where your code runs, on hardware someone already owns. Pick one small idea, have it run entirely on-device, and you will feel the difference the first time it answers with the wifi off.

Originally published on the QVAC blog. QVAC is Tether's open-source local-AI SDK. Apache 2.0.