Quick-start: build your first local AI app with your coding agent

Thomas — Wed, 22 Jul 2026 14:20:56 +0000

TL;DR

You do not need to learn an SDK by heart to build a local AI app. You need to understand what local AI is good and bad at, know what is possible, and hand your coding agent the right context so it writes the code for you. This is that orientation: the trade-offs of running AI on your own hardware, the 12 things the QVAC SDK can do, a set of app ideas, and how to point your agent so it builds correctly. Almost no code here, on purpose.

Building this way? QVAC.md is the one file that makes your agent write QVAC code that runs the first time. Get it here.

How this guide works

This is not a copy-paste tutorial. You will not be pasting large code blocks. So this guide gives you the three things a builder actually needs:

The trade-offs of local AI, so you pick the right project for it.
What the SDK can do, so you know what is on the menu.
How to set up your agent, so the code it writes actually works.

What "local AI" actually means

Local AI means the model runs on your hardware (or your user's device), not behind a cloud API. That single fact creates real constraints and real advantages. Knowing both is how you choose good projects.

The constraints (know these going in)

Your hardware sets the ceiling. RAM and GPU decide which models you can run and how fast. A small model runs fine on a laptop CPU. Bigger and faster wants a GPU and more memory. As a rough guide, a 4B model needs a few GB; a 30B-class mixture-of-experts model needs around 20 GB or more.
A local model is not a frontier model. The largest cloud models are far bigger than anything that fits on a device. On the hardest reasoning, the broadest world-knowledge, and very long context, a local model is weaker. Use it for the jobs it is good at, not for everything.
The first run downloads the model (hundreds of MB to several GB), then it is cached and runs offline forever after.

The advantages (why it is worth it)

Private by construction. After the download, nothing leaves the device. No prompt logging, no data shipped to a third party.
No API key, no per-token bill. The cost is hardware you already own. Run it as much as you like.
Works offline. On a plane, in a clinic, on a factory floor, in the field, with no signal.
Can run 24/7. An always-on background agent, a daemon, a scheduled job. No metered API to drain your budget, no rate limits to throttle you.
Low latency. No network round-trip to a data center.
You own it. The weights are on your disk. Nothing changes under you.

The rule of thumb

Reach for local AI when privacy, offline use, always-on operation, high volume, or cost are what matter. Reach for a frontier cloud model when you need maximum reasoning or breadth and the data is not sensitive. Many of the best products use both: local for the private, high-frequency path, cloud for the occasional hard question.

What you can build: the 12 capabilities

QVAC is Tether's open-source local-AI SDK. One npm install gives you twelve AI tasks behind one consistent API, all running on-device:

Text and chat - a local LLM to assist, summarize, extract, or classify by prompt.
Vision - ask questions about an image, describe a photo, read a chart.
Embeddings - turn text into vectors for search and similarity.
RAG - search your own documents and answer with citations, locally.
Speech to text - transcribe audio and meetings, multilingual.
Text to speech - generate spoken audio in many languages, including voice cloning.
Translation - the Mozilla Bergamot engine, many language pairs.
Image generation - text to image, on-device.
OCR - read text out of scans and photos.
Classification - sort inputs into labels.
Image upscaling - enhance images on-device.
On-device fine-tuning - adapt a small model to a person or domain, trained on the device itself.

The point is the consistency: your agent learns the pattern once and can reach for any of the twelve.

Example apps to spark ideas

You do not build "an AI feature", you build a product. A few that play to local AI's strengths, each one you would simply describe to your coding agent or harness:

A private meeting transcriber and summarizer (speech to text + chat). Recordings never leave the laptop. Good for lawyers, doctors, journalists.
Chat with your notes (RAG + embeddings). A local "second brain" that answers from your own files, with citations, offline.
A private medical Q&A assistant (chat). Answer clinical questions on-device with MedPsy, a medical model small enough for a phone - nothing sent to a server.
An offline travel translator (translation + speech to text + text to speech). Works with no signal abroad.
An always-on document sorter (OCR + classification). A daemon that files incoming scans around the clock, with no per-page API cost.
An on-device image studio (image generation + upscaling). No cloud render bills, no usage caps.
A kiosk or in-car voice assistant (speech to text + chat + text to speech). Fully offline and private.
A personal-style writing model (fine-tuning). Adapt a small model to your own voice, trained on your machine, never uploaded. (See how it works in LoRA fine-tuning on edge GPUs via QVAC Fabric.)

Want to see real builds on QVAC? OpenClaw runs a fully local, private AI assistant, and you can even run your own coding agent on a local model with OpenCode. Pick an idea, describe its first slice to your agent, and iterate.

How to build it with your agent

The whole workflow is short:

Start an empty project and install the SDK (npm install @qvac/sdk; you need Node 22.17 or newer, which is above the usual LTS).
Give your agent QVAC.md. This is the step that matters. Coding agents are unreliable on SDKs they have not seen much of: they invent functions that do not exist. QVAC.md is a single file you drop into your agent's rules (a CLAUDE.md, an AGENTS.md, a Cursor or Cline rules file) that gives it the real API. With it, the agent writes code that runs.
Describe one capability at a time. For example: "Using the QVAC context, build a CLI that transcribes an audio file passed on the command line, fully locally."
Run it and iterate with your agent.

Cross-platform: what to keep in mind

The same QVAC code runs on desktop (macOS, Linux, Windows via Node) and on mobile (iOS and Android, native, via Expo). A few things worth telling your agent up front:

Match the model to the target device. A workstation can run a large model; a phone needs a small one. The model has to fit the device's memory, so choose per platform.
The GPU differs by platform (Apple Metal, Vulkan, or a CPU fallback). The SDK picks the right backend automatically, but speed varies, so test on the real device, not just your laptop.
The first download can be large. On mobile especially, show a progress state so users are not staring at a frozen screen on first launch.
Node 22.17+ on desktop. Below that, setup fails in confusing ways.

Get QVAC.md

Everything above is the human's half. The agent's half is QVAC.md: a single drop-in file with the real API surface, the model-type rules, the gotchas that stop hallucinations, and verified snippets. Paste it into your agent and it builds local AI apps that run.

Get QVAC.md for your coding agent on the original guide. Drop your email and we send you the drop-in file. Paste it into your agent and start building.

FAQ

Which coding agent does this work with?

Any of them. Cursor, Claude Code, Cline, Aider, OpenCode, or any harness that reads project rules or takes a system prompt. QVAC.md tells you where to paste it for each.

Do I need a GPU?

No. QVAC uses your GPU when one is available (Metal, NVIDIA, AMD, Intel) and falls back to CPU otherwise. Small models run fine on a laptop CPU; larger models are much faster with a GPU.

How does a local model compare to ChatGPT or Claude?

A model that fits on your device is smaller, so on the hardest reasoning and the broadest knowledge it is weaker than a frontier cloud model. In exchange it is private, offline, free to run, and always on. Pick the right tool for the job, and feel free to combine both.

Is my data sent anywhere?

No. After the model downloads once, inference happens entirely on-device. No API call, no prompt logging at run time.

Can this run on a phone?

Yes. The same code runs natively on iOS and Android through Expo, on the device hardware. Use a smaller model to fit the phone's memory. MedPsy is one example of a model built to run there.

Is it free? What is the license?

The SDK is Apache 2.0 and free, including commercial use. It is JavaScript and TypeScript, on Node 22.17+, Bare 1.24+, and Expo 54+.

Key takeaways

You build local AI apps by directing your coding agent, not by memorizing an SDK.
Local AI trades raw frontier capability for privacy, offline use, zero per-call cost, and 24/7 operation. Choose projects that want those.
One npm install gives you twelve on-device AI tasks behind one API.
QVAC.md is what makes your agent write correct QVAC code.
The same code runs on desktop and mobile; match the model to the device.

Next steps

Grab QVAC.md and paste it into your agent.
Pick one app idea above and describe its first slice to your agent.
Read the SDK docs and the model registry when you want to go deeper.

None of this is a big commitment. You are not adopting a framework, you are giving your agent the right context and letting it build. The intelligence in the app you ship will not be rented from a server you do not control; it will run where your code runs, on hardware someone already owns. Pick one small idea, have it run entirely on-device, and you will feel the difference the first time it answers with the wifi off.

Originally published on the QVAC blog. QVAC is Tether's open-source local-AI SDK. Apache 2.0.

DEV Community: Thomas