DEV Community: Ondrej Machala

How to Set Up Diction: The Self-Hosted Speech-to-Text Alternative to Wispr Flow

Ondrej Machala — Sat, 18 Apr 2026 10:44:08 +0000

This article is about getting your own private speech-to-text on your iPhone. Tap a key, speak, watch the words land in whatever app you're in. No cloud in the middle, no subscription, no company on the other end reading what you said. The keyboard is Diction. This post is the full setup, start to finish, blank machine to working dictation in under thirty minutes.

I built the server side for myself:
https://github.com/omachala/diction

I talk to my AI agents all day. Claude in the terminal, my Telegram bot OpenClaw, a handful of others. Voice for everything. Long prompts, half-formed plans, emails I want rewritten, code I want reviewed. Every word used to pass through someone else's transcription cloud before my own agents ever heard it. Not anymore.

A small Docker stack on a box at home now handles the transcription. An optional cleanup step scrubs filler words and fixes punctuation using any LLM you want: OpenAI, Groq, a local Ollama model, anything OpenAI-compatible.

Every command is below.

What You'll End Up With

A box at home running the speech model, 24/7
Your iPhone sending audio to it over your home WiFi
Optional: an LLM of your choice for cleaning up filler words and fixing punctuation (OpenAI, Groq, Anthropic, a local Ollama model, anything with an OpenAI-compatible API)
Total running cost with cleanup on: depends on the LLM you pick. Roughly a cent per hour of dictation on gpt-4o-mini, zero if you run a local model.

The speech part is free forever. The cleanup part costs whatever your LLM provider charges. Use a local model and pay nothing. More on that at the end.

What You Need

Any machine that can run Docker: Mac mini, an old laptop, a home server in a closet, a NUC, a home lab box. Apple Silicon or any modern x86 works fine. Raspberry Pi is a stretch for the speech part. Anything newer is comfortable.
An iPhone running iOS 17 or newer
Both on the same WiFi network
Optional: an API key for any OpenAI-compatible LLM (OpenAI, Groq, Together, Anthropic via a proxy, Ollama running locally, etc.) if you want AI cleanup

I'll assume you know what Docker is and how to open a terminal. That's it.

Step 1: Install Docker

You need Docker Engine plus Docker Compose. Both come bundled in Docker Desktop on Mac and Windows. On Linux you install them separately (they're both free and open source).

macOS (Intel or Apple Silicon): Download Docker Desktop, open the .dmg, drag the whale icon to Applications, launch it. The first run asks for admin credentials (it needs to install a helper tool and set up networking). When the whale icon in the menu bar stops animating and says "Docker Desktop is running", you're ready.

Windows: Download Docker Desktop. The installer will enable WSL2 if it's not already on - this is required, and needs a reboot. After the reboot, launch Docker Desktop. Same whale icon in the system tray tells you when it's ready.

Linux: Either install Docker Desktop (same download page) or go with the native packages:

# Ubuntu / Debian
sudo apt update
sudo apt install docker.io docker-compose-plugin

# Fedora / RHEL
sudo dnf install docker docker-compose-plugin

# Arch
sudo pacman -S docker docker-compose

Start the service and add your user to the docker group so you don't need sudo every time:

sudo systemctl enable --now docker
sudo usermod -aG docker "$USER"

Log out and back in (or reboot) so the group change takes effect. Yes, you really need to log out. Running newgrp docker works too but only in the current shell.

Verify it's all working:

docker --version
docker compose version
docker run --rm hello-world

The last command pulls a tiny test image and prints a greeting. If it fails with "permission denied" on Linux, you skipped the log-out-and-back-in step.

Apple Silicon users, one extra thing: open Docker Desktop → Settings → General and make sure "Use Rosetta for x86/amd64 emulation" is enabled. This is the default on recent Docker Desktop builds. The Diction gateway image is built for amd64 (multi-arch is on the roadmap), so Docker needs Rosetta to run it on your M1/M2/M3/M4. Performance impact is negligible - the speech model image is multi-arch and runs natively on arm64, so Rosetta is only handling the small Go binary in front of it.

While you're in Settings, also check Resources → Memory. The default Docker Desktop VM ships with 2 GB, which is tight for medium (~2.1 GB) and will OOM silently. Bump to 4 GB if you're running anything above small.

Step 2: Create a Project Folder

Pick a home for the compose file and any supporting config. Anywhere works. I use ~/diction:

mkdir -p ~/diction && cd ~/diction

Everything in the rest of this article assumes you're sitting in that folder. Docker Compose looks for docker-compose.yml in the current directory, so all the docker compose commands Just Work as long as you cd ~/diction first.

If you're setting this up on a remote server (Linux box in a closet, NUC, etc.), SSH in and run the same command there. Where you edit the file is up to you: nano docker-compose.yml on the server, VSCode Remote-SSH, or editing locally and scp-ing the file over. All fine.

Step 3: Write the Compose File

Here's what we're about to spin up. Two containers working together:

Diction Gateway. The open-source Go service at the front of the stack. On the outside it speaks the standard OpenAI transcription API (POST /v1/audio/transcriptions), which is what the Diction iPhone app talks to. On the inside it routes your audio to whichever speech model you've loaded, and optionally passes the transcript through an LLM for cleanup. The source is on GitHub, MIT licensed. Small, boring Go. Read it, fork it, bend it to your needs.
A voice model. The engine that actually turns audio into text. For this starter stack we're using faster-whisper - a compact, battle-tested open-source model that ships in sizes tiny, base, small, medium, large-v3, and large-v3-turbo. Bigger means more accurate and slower. We'll run small. It's the sweet spot for CPU-only machines: accurate enough for real dictation, transcribes a 5-second clip in 1 to 2 seconds on a modern Mac mini or NUC.

If you've got an NVIDIA GPU sitting in the machine, you can skip small and run something far better (Parakeet or large-v3-turbo). Jump to the "Got an NVIDIA GPU Sitting Idle?" section below before you paste the compose file. Otherwise continue here.

Paste this into ~/diction/docker-compose.yml:

services:
  whisper-small:
    image: fedirz/faster-whisper-server:latest-cpu
    container_name: diction-whisper-small
    restart: unless-stopped
    volumes:
      - whisper-models:/root/.cache/huggingface
    environment:
      WHISPER__MODEL: Systran/faster-whisper-small
      WHISPER__INFERENCE_DEVICE: cpu

  gateway:
    image: ghcr.io/omachala/diction-gateway:latest
    platform: linux/amd64
    container_name: diction-gateway
    restart: unless-stopped
    ports:
      - "8080:8080"
    depends_on:
      - whisper-small
    environment:
      DEFAULT_MODEL: small

volumes:
  whisper-models:

What each line does

Quick tour so you know what you're pasting.

whisper-small service:

image: fedirz/faster-whisper-server:latest-cpu. The voice model engine. faster-whisper is a C++/CTranslate2 reimplementation of the original open-source voice model from OpenAI, running 4x faster with less memory. fedirz/faster-whisper-server wraps it in a small Python server that speaks the OpenAI transcription API. The -cpu tag is the CPU build. There's also a -cuda tag for NVIDIA users (see the GPU section below).
container_name: diction-whisper-small. Just a friendly name so docker ps shows something readable instead of a random string.
restart: unless-stopped. If the container crashes or the host reboots, Docker brings it back. The only thing that stops it is you explicitly running docker compose down.
volumes: - whisper-models:/root/.cache/huggingface. The model weights are downloaded on first start (about 500MB for small). This volume persists them across container rebuilds, so you don't re-download every time you pull a newer image.
WHISPER__MODEL: Systran/faster-whisper-small. The specific voice model to load. It's a HuggingFace repo ID. You can swap this for any CT2-compatible voice model.
WHISPER__INFERENCE_DEVICE: cpu. Tells it to run on CPU. Swap to cuda if you've got an NVIDIA card (full example in the GPU section below).

gateway service:

image: ghcr.io/omachala/diction-gateway:latest. The Diction gateway from GitHub Container Registry.
platform: linux/amd64. The current published image is amd64-only. On Apple Silicon, Docker will run it under Rosetta transparently. Drop this line on a native x86 host if you want the error message to be slightly tidier on docker compose config.
ports: - "8080:8080". Maps port 8080 on the host to 8080 in the container. This is the one your iPhone will talk to. If 8080 is already in use on your machine, change the left side: "18080:8080" and use http://your-ip:18080 from the phone.
depends_on: - whisper-small. Docker starts the whisper container first so the gateway doesn't throw connection-refused on startup. Not strictly required (the gateway retries), but makes logs cleaner.
DEFAULT_MODEL: small. The model the gateway routes to when the iPhone sends a request without specifying one. The gateway has a built-in mapping of short names (small, medium, large-v3-turbo, parakeet-v3) to backend service URLs. Setting DEFAULT_MODEL: small makes it expect a service named whisper-small on port 8000. This is why the first service is named whisper-small and not whisper.

volumes: block at the bottom: declares the named volume Docker uses for the model cache. Named volumes are managed by Docker itself and survive container rebuilds.

Model sizes and what to pick

small is the starter. It's accurate enough for everyday dictation and fits comfortably on any modern laptop or NUC. If you want something else, swap WHISPER__MODEL in the compose file:

Model	Parameters	RAM	CPU latency (5s clip)	Notes
`Systran/faster-whisper-tiny`	39M	~350 MB	1-2s	Fast, lower accuracy
`Systran/faster-whisper-small`	244M	~850 MB	3-4s	Sweet spot for CPU
`Systran/faster-whisper-medium`	769M	~2.1 GB	8-12s	More accurate, slow on CPU
`deepdml/faster-whisper-large-v3-turbo-ct2`	809M	~2.3 GB	<2s on GPU	Best with NVIDIA

The latency numbers are from my own homelab (AMD Ryzen 9 7940HS, CPU-only). Apple Silicon is in the same ballpark: fast enough for small to feel instant, slow enough that medium will make you wait.

Two rules when switching models:

Also change DEFAULT_MODEL on the gateway to match one of: tiny, small, medium, large-v3-turbo.
Rename the service to the one the gateway expects: whisper-tiny, whisper-small, whisper-medium, or whisper-large-turbo. The gateway looks up its backend by service hostname.

Skip either and the gateway will give you a 404 when the app asks for a model.

One caveat for Mac mini / Apple Silicon users

Docker on macOS runs everything inside a Linux VM. That VM can't reach Apple's GPU or Neural Engine. Containers are CPU-only regardless of how nice your M4's GPU is. Sounds bad on paper, but for dictation workloads you won't feel it: the small voice model handles a short sentence well under five seconds on an M-series CPU. Longer dictations scale linearly. If you want GPU speed, either (a) run a Linux box with an NVIDIA card and keep the Mac as a client, or (b) use Diction's on-device mode on the iPhone itself (Core ML on the Neural Engine).

Step 4: Start Everything

Make sure you're in the project folder, then:

docker compose up -d

The -d flag runs the containers in the background (detached mode).

On the first run this takes a minute or two. Docker pulls two images from their registries:

fedirz/faster-whisper-server:latest-cpu - about 1.7 GB, includes the Python runtime and CTranslate2 binaries
ghcr.io/omachala/diction-gateway:latest - about 210 MB, a compiled Go binary plus ffmpeg for audio conversion

After the pulls finish, the voice model container does one more thing on first boot: it downloads the model weights from HuggingFace into the whisper-models volume (~500 MB for small). Subsequent restarts skip this step - the volume is persistent. That's why there's a volumes: block in the compose file.

Check everything is healthy

docker compose ps

You should see both services:

NAME                     STATUS
diction-gateway          Up 30 seconds
diction-whisper-small    Up 30 seconds (health: starting)

health: starting on the whisper container is normal for the first couple of minutes. It's loading the model into RAM. Once that's done, the status will flip to Up (healthy) or just Up.

Watching logs

If something looks wrong, look at the logs:

docker compose logs -f

-f follows them in real time. Ctrl+C to detach.

You can also tail a single service:

docker compose logs -f gateway
docker compose logs -f whisper-small

What healthy logs look like (abbreviated):

Gateway:

{"level":"info","msg":"gateway starting","port":"8080"}
{"level":"info","msg":"backend registered","name":"small","url":"http://whisper-small:8000"}

Whisper:

INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000

Common early errors:

pull access denied on the gateway image. A stale GitHub Container Registry token is cached in your Docker config (on macOS, usually in the login keychain from a past docker login). Run docker logout ghcr.io - yes, even if you don't think you're logged in - and try again.
exec format error on Apple Silicon. Rosetta isn't enabled. Go back to Docker Desktop → Settings → General and flip the Rosetta option on.
The voice model container stuck on health: starting for more than 3 minutes. Usually means it's still downloading weights on a slow connection. Check docker compose logs -f whisper-small to see the download progress.

Stopping and restarting

docker compose stop        # stop containers, keep their state
docker compose start       # start them again
docker compose down        # stop and remove containers (volumes survive)
docker compose down -v     # stop, remove containers AND volumes (re-downloads weights)
docker compose pull        # get newer images
docker compose up -d       # apply pulls / config changes

The model cache in the whisper-models volume is shared across rebuilds, so docker compose pull && docker compose up -d to upgrade is a ~30-second operation.

Step 5: Test It

Before you go anywhere near the iPhone, prove the server itself works. A broken stack is easier to debug from a terminal than from a keyboard extension.

Get an audio file

The quickest path: use your phone's built-in Voice Memos app. Record yourself saying "Hello from my home server." Hit stop. Share → Save to Files, or AirDrop to your Mac, or email it to yourself. You want the .m4a file on the same machine that's running the containers.

On Linux without a phone handy, record with arecord or sox:

# 5 seconds of 16-bit mono WAV at 16 kHz - whisper's native format
arecord -f S16_LE -r 16000 -c 1 -d 5 voice-memo.wav

On macOS, skip recording altogether and let the system generate a clip with say:

say -o voice-memo.aiff "Hello from my home server"

That gives you an .aiff the gateway accepts directly. Handy for scripted testing where you don't feel like holding a microphone.

No microphone and no speech synth? Grab any short speech clip you have lying around. MP3, WAV, M4A, AIFF, FLAC, Ogg - they all work. The voice model handles re-encoding internally.

Hit the gateway

curl -X POST http://localhost:8080/v1/audio/transcriptions \
  -F "file=@voice-memo.m4a" \
  -F "model=small"

You'll get back something like:

{"text":"Hello from my home server."}

That's the whole speech pipeline. Running on your hardware. Your audio never left the box.

Ask for different response formats

The same endpoint supports response_format=text if you'd rather have a plain string (useful if you're piping it into a shell):

curl -X POST http://localhost:8080/v1/audio/transcriptions \
  -F "file=@voice-memo.m4a" \
  -F "model=small" \
  -F "response_format=text"
# → Hello from my home server.

Check the response headers

The gateway adds timing info to the response headers - useful for benchmarking without reading logs:

curl -sS -D - -o /dev/null -X POST \
  http://localhost:8080/v1/audio/transcriptions \
  -F "file=@voice-memo.m4a" -F "model=small"

Look for:

X-Diction-Whisper-Ms - how many milliseconds the speech model took
X-Diction-LLM-Ms - appears only if you've enabled the cleanup step in Step 7

Talk to it from Python

Since the gateway speaks the OpenAI transcription API, the official openai Python SDK works against it directly. Useful if you want to script transcriptions from a laptop:

from openai import OpenAI

client = OpenAI(
    base_url="http://192.168.1.42:8080/v1",
    api_key="anything",   # the gateway doesn't check this by default
)

with open("voice-memo.m4a", "rb") as f:
    result = client.audio.transcriptions.create(
        file=f,
        model="small",
        response_format="text",
    )

print(result)

Same story with the Node SDK, LangChain, or any other tool that expects OpenAI's speech API. Diction becomes a drop-in local replacement for api.openai.com/v1/audio/transcriptions.

If the test fails

Connection refused. The gateway container isn't running. docker compose ps to confirm.
504 Gateway Timeout. The whisper container is still starting (model loading into RAM). Give it another 60 seconds.
400 Bad Request: "invalid audio file". Your file is corrupted or in a format whisper doesn't understand. Try a freshly recorded clip.
404 Not Found. You probably have a typo in the URL. The path is exactly /v1/audio/transcriptions - plural, with /v1/ prefix.
Empty response / hang. The voice model container crashed out of memory mid-transcription. Check docker compose logs whisper-small. small should be fine on any machine with 2GB of free RAM; if you upgraded to medium and the host doesn't have 3GB free, it'll OOM.

Step 6: Find Your Server's LAN IP

Your iPhone needs an address to reach this. Your server probably has two kinds: a public IP (facing the internet, you don't want to use that) and a private LAN IP (on your home WiFi, that's the one).

macOS:

ipconfig getifaddr en0

en0 is usually Wi-Fi on laptops and the built-in Ethernet on desktops. If it prints nothing (you're wired via a USB-C dongle, or on a Mac mini with Wi-Fi off), the right interface is somewhere else - try en1, en4, en5. Quickest catch-all:

ifconfig | grep 'inet ' | grep -v 127.0.0.1

Pick the 192.168.x.x or 10.x.x.x address. Ignore anything starting with 100. - that's Tailscale, not your LAN.

Linux:

hostname -I | awk '{print $1}'

Or, if you want a specific interface:

ip -4 addr show wlan0 | grep inet

Windows:

ipconfig | findstr IPv4

You'll get something like 192.168.1.42. Write it down. This is what you'll paste into the Diction app in Step 8.

Pin it so it doesn't drift

Your router hands out IPs via DHCP, which means the one you just wrote down might change next time the server reboots (or when the lease expires). Two ways to keep it stable:

DHCP reservation. Log into your router's admin page (usually 192.168.1.1, 192.168.0.1, or 10.0.0.1). Find the DHCP client list, locate your server by hostname or MAC address, and click the "reserve" / "static" option. From then on, your router will always hand out that same IP to that machine.
Static IP on the machine. On Linux, edit /etc/netplan/ or use your distro's network manager. On macOS, System Settings → Network → Wi-Fi → Details → TCP/IP → Configure IPv4 → Using DHCP with manual address. More work, more fragile. The router method is better.

If you'd rather not deal with IPs at all and your setup is more portable (laptop moving between networks, for example), skip ahead to the "Reach It From Anywhere" section. Tailscale gives every machine a stable private address that follows it around.

Step 7: Add AI Cleanup (Optional but Nice)

Skip this step and your dictation still works. You'll get raw transcription, which is usually 95% right. The remaining 5% is filler words ("um", "like"), missing commas, misheard homophones ("their" vs "there"), and sometimes a full sentence with no punctuation. AI cleanup fixes all of that before your agent ever sees it.

What it does

You say:

so um basically the meeting went well and uh they agreed to the timeline

The gateway hands that to the LLM, which returns:

The meeting went well. They agreed to the timeline.

That's the whole feature. Any OpenAI-compatible LLM works - OpenAI's own models, Groq, Anthropic (via a compatibility proxy), Together, Fireworks, a local Ollama install, anything that speaks POST /chat/completions.

The flow

iPhone → gateway → voice model → raw transcript
                              ↓
                    your LLM (chat/completions)
                              ↓
                    cleaned text → back to the iPhone

The iPhone sends ?enhance=true on the request when the app's AI Companion toggle is on. The gateway hits {LLM_BASE_URL}/chat/completions with your system prompt + the transcript. Whatever comes back gets sent to the iPhone instead of the raw transcript. If the LLM errors out or times out, the gateway falls back to raw - your dictation doesn't break because of a downstream hiccup.

Config reference

Four environment variables on the gateway:

Variable	Required	What it is
`LLM_BASE_URL`	yes	OpenAI-compatible endpoint, e.g. `https://api.openai.com/v1`
`LLM_MODEL`	yes	Model identifier, e.g. `gpt-4o-mini`
`LLM_API_KEY`	no	Bearer token (your provider's API key). Not needed for local Ollama.
`LLM_PROMPT`	no	System prompt. Literal string, or a file path starting with `/` if you want a longer one mounted as a volume.

Both LLM_BASE_URL and LLM_MODEL must be set for cleanup to turn on. Miss either one and the feature silently stays off.

Option A: OpenAI (or any OpenAI-compatible provider)

Easiest first step. Get a key at platform.openai.com/api-keys and add $5 of credit. For cleanup that's hundreds of hours of dictation.

Create ~/diction/.env:

echo "OPENAI_API_KEY=sk-your-key-here" > ~/diction/.env

Update the gateway service in docker-compose.yml:

  gateway:
    image: ghcr.io/omachala/diction-gateway:latest
    platform: linux/amd64
    container_name: diction-gateway
    restart: unless-stopped
    ports:
      - "8080:8080"
    depends_on:
      - whisper-small
    environment:
      DEFAULT_MODEL: small
      LLM_BASE_URL: "https://api.openai.com/v1"
      LLM_API_KEY: "${OPENAI_API_KEY}"
      LLM_MODEL: "gpt-4o-mini"
      LLM_PROMPT: "Clean up this voice transcription. Remove filler words (um, uh, like). Fix punctuation and capitalization. Return only the cleaned text, nothing else."

Docker Compose reads ${OPENAI_API_KEY} from the .env file in the same folder automatically. No extra flags needed.

Not tied to OpenAI. Every major LLM provider exposes the same OpenAI-compatible /chat/completions endpoint. Swap the three LLM_* URLs and keys and you're done. A few that work out of the box:

Anthropic - Claude models via the OpenAI SDK
Groq - fastest inference on the market, generous free tier
Together AI - broad open-model catalog
Fireworks - tuned Llama and Mixtral hosting
DeepInfra - pay-per-token open models
OpenRouter - one key, hundreds of models from every provider
Mistral - native OpenAI-compatible endpoint

Pick one, drop its LLM_BASE_URL and LLM_MODEL into the compose file, same shape.

Option B: Local with Ollama (zero cost, fully private)

If you've got enough RAM and want nothing leaving your house - not even the transcribed text - run the LLM locally.

Add a third service to your compose file:

  ollama:
    image: ollama/ollama:latest
    container_name: diction-ollama
    restart: unless-stopped
    ports:
      - "11434:11434"
    volumes:
      - ollama-models:/root/.ollama

And update the gateway service:

  gateway:
    image: ghcr.io/omachala/diction-gateway:latest
    platform: linux/amd64
    container_name: diction-gateway
    restart: unless-stopped
    ports:
      - "8080:8080"
    depends_on:
      - whisper-small
      - ollama
    environment:
      DEFAULT_MODEL: small
      LLM_BASE_URL: "http://ollama:11434/v1"
      LLM_MODEL: "gemma2:9b"
      LLM_PROMPT: "Clean up this voice transcription. Remove filler words. Fix punctuation and capitalization. Return only the cleaned text, nothing else."

Add the Ollama volume to the bottom of the file:

volumes:
  whisper-models:
  ollama-models:

Bring it up and pull a model:

docker compose up -d
docker exec diction-ollama ollama pull gemma2:9b

LLM_API_KEY isn't needed - Ollama doesn't check it.

Which Ollama model?

Sizes below are memory footprint - system RAM if you run Ollama on CPU, VRAM if you pass a GPU through to the container. Either way the number is the same.

Model	Params	Memory	Notes
`gemma2:9b`	9B	~6 GB	Best editing quality at this size. My pick.
`qwen2.5:7b`	7B	~5 GB	Strong at following cleanup instructions.
`llama3.1:8b`	8B	~5 GB	Most popular, well-tested.
`gemma3:4b`	4B	~3 GB	For tighter machines. Still OK for basic cleanup.

Under 7B tends to fail in a specific, annoying way: the model treats your transcript as a question and tries to answer it, instead of cleaning it up. Stick to 7B+ if you can spare the memory.

If you have an NVIDIA GPU, pass it through to the Ollama container (same reservation block as the voice model GPU example further down) and you'll get 5-10x faster cleanup.

Apply the changes

Once your compose file has the LLM_* variables set, restart the gateway so it picks them up:

docker compose up -d

Docker Compose detects the env change and recreates only the gateway container. The voice model container (and its loaded model) keeps running.

Test the cleanup

Same voice memo as before, with ?enhance=true appended:

curl -X POST "http://localhost:8080/v1/audio/transcriptions?enhance=true" \
  -F "file=@voice-memo.m4a" \
  -F "model=small"

Without ?enhance=true you get the raw transcription. With it, the gateway sends the transcript through the LLM before returning. Quickest sanity check: record yourself saying some filler words ("um, this is uh a test like") and watch them disappear.

To confirm the LLM is actually running (and wasn't silently disabled because of a missing env var), check the response headers for X-Diction-LLM-Ms:

curl -sS -D - -o /dev/null -X POST \
  "http://localhost:8080/v1/audio/transcriptions?enhance=true" \
  -F "file=@voice-memo.m4a" -F "model=small" | grep -i diction

You should see both X-Diction-Whisper-Ms and X-Diction-LLM-Ms in the output.

Dialing in the prompt

The default prompt above is fine for generic cleanup. Adjust it to your taste. Some real prompts I've tried:

Conservative cleaner (preserves your voice, just fixes obvious errors):

Clean up this voice transcription. Fix punctuation and obvious typos only.
Do not rephrase or change word choice. Return only the cleaned text.

Email-ready rewriter (turns rambling into something you could actually send):

Rewrite this voice note as a short professional email. Keep the meaning intact.
Return only the rewritten text, no greeting or sign-off.

Bullet-pointer (for dumping meeting notes):

Convert this voice note into a bulleted list of the key points.
One bullet per idea. Return only the list.

Translator (I dictate in English, send in German):

Translate this English voice note into natural German. Return only the translation.

Long prompts via a file

If your prompt is more than a one-liner, mount it as a file. Create ~/diction/cleanup-prompt.txt:

You are a transcript cleaner.

Rules:
- Remove filler words (um, uh, er, like, you know).
- Fix grammar and punctuation.
- Preserve the speaker's voice and meaning.
- Common speech-to-text errors: "there / their / they're", "affect / effect".
- Do not add a preamble.
- Return only the cleaned text.

Mount it into the container and point LLM_PROMPT at the file path:

  gateway:
    image: ghcr.io/omachala/diction-gateway:latest
    # ... rest of config
    volumes:
      - ./cleanup-prompt.txt:/config/cleanup-prompt.txt:ro
    environment:
      LLM_BASE_URL: "https://api.openai.com/v1"
      LLM_API_KEY: "${OPENAI_API_KEY}"
      LLM_MODEL: "gpt-4o-mini"
      LLM_PROMPT: "/config/cleanup-prompt.txt"

If LLM_PROMPT starts with /, the gateway reads it as a file path. Otherwise it uses the string directly.

Why gpt-4o-mini or a 7B local model instead of something bigger

Cleanup is a simple task. The LLM only needs to polish, not reason. A frontier-tier model is overkill and slower. gpt-4o-mini (cloud) or gemma2:9b (local) hit the sweet spot for this workload. Save the expensive models for your actual conversations with the agent downstream.

Step 8: Install Diction and Point It at Your Server

Server's ready. Time to put the keyboard in front of it.

Install the app

On your iPhone, open the App Store and install Diction. It's free to download, and the modes you need for self-hosting (the entire point of this article) are free forever.

First run

Open the app. It walks you through three things:

Add the keyboard. iOS requires you to manually add any third-party keyboard. The app sends you to Settings → General → Keyboard → Keyboards → Add New Keyboard → Diction. Tap "Diction", then go back.
Allow Full Access. Back in Keyboards, tap "Diction" in the list and flip "Allow Full Access" on. iOS will show a scary-sounding warning. It's required for any keyboard that makes network requests, which Diction has to do (it sends audio to your server). Diction has no QWERTY input, no text logging, and no analytics - there's nothing to capture even if it wanted to. Only the mic audio leaves the phone, and only to the endpoint you configure below. The source for the gateway is on GitHub, so you can audit exactly what the server does with the audio.
Grant microphone access. Back in the app, it asks for mic permission. Yes.

Point it at your server

Inside the Diction app:

Go to Settings (gear icon, top right).
Tap Mode. Choose Self-Hosted.
Tap Endpoint. Enter http://192.168.1.42:8080 (substituting your server's IP from Step 6).
Scroll down. If you configured AI cleanup in Step 7, toggle AI Companion on.
Tap Test connection. You should see a green check within a second or two. If not, see the troubleshooting below.

Take it for a spin

Open any app that accepts text - Telegram, Messages, Notes, Mail, the Safari address bar, whatever. Tap to bring up the keyboard. Long-press the globe icon (bottom-left of the default keyboard) to switch keyboards. Pick Diction.

You'll see one big mic button. Tap it, talk, release. The audio streams to your server. The transcription arrives back in about as much time as it takes for you to take your finger off the button.

On a local network, end-to-end latency for a short sentence is typically under a second. Good enough that you stop thinking about it.

If it doesn't connect

Server not running? docker compose ps on the server.
iPhone not on the same WiFi as the server.
IP address typo - re-check what Step 6 returned.
Firewall blocking port 8080. On Linux with ufw: sudo ufw allow from 192.168.0.0/16 to any port 8080. On macOS, System Settings → Network → Firewall. Docker Desktop adds itself to the allow list on install, so inbound on published ports normally works - but if you've previously clicked "Deny" on a firewall prompt for Docker, that choice sticks. Flip it back under "Options…", or temporarily turn the firewall off to confirm that's the cause.

Quickest sanity check: open Safari on the iPhone and try http://192.168.1.42:8080/health. If the browser can't reach it, the app can't either.

Now dictate into your agent

Open Telegram. Tap your agent's chat. Tap the globe to switch to the Diction keyboard. Tap the mic. Talk. Release. Your server transcribes, the LLM cleans it up, and the message lands in the composer ready to send. Hit send. Your agent replies. Loop.

That's the whole point of the exercise.

Reach It From Anywhere (Not Just Home WiFi)

Right now your dictation only works on your home network. The moment you walk out the door, the iPhone can't reach 192.168.1.42 anymore. Three clean ways to fix this.

Tailscale (my pick)

Tailscale builds a private mesh network between your devices over WireGuard. Install it on the server and on the iPhone, sign in to the same account on both, and your phone gets a stable 100.x.x.x address it can use to reach the server from anywhere - cellular, coffee shop WiFi, a plane with WiFi, wherever.

Server side (Linux):

curl -fsSL https://tailscale.com/install.sh | sh
sudo tailscale up

On macOS, download the app and run it.

iPhone side: install the Tailscale app from the App Store, sign in.

On the server, grab the tailnet IP:

tailscale ip -4
# → 100.64.1.42

Back in the Diction app, change the Endpoint from http://192.168.1.42:8080 to http://100.64.1.42:8080. Your dictation now works wherever you've got signal. Free for personal use (up to 100 devices).

Cloudflare Tunnel (public URL, no port forwarding)

If you'd rather have a pretty URL and don't want to install anything on the phone, Cloudflare Tunnel gives you an outbound tunnel from your server to Cloudflare's edge. No router config, no exposed ports.

Add this service to your compose file:

  cloudflared:
    image: cloudflare/cloudflared:latest
    container_name: diction-cloudflared
    restart: unless-stopped
    command: tunnel --no-autoupdate run
    environment:
      TUNNEL_TOKEN: "${CLOUDFLARE_TUNNEL_TOKEN}"

Create the tunnel in the Cloudflare Zero Trust dashboard, grab the token, paste it into your .env, set the public hostname to route to http://gateway:8080. Done. Dictate over https://dictation.yourdomain.com.

Free tier. Works great. Only caveat: your transcriptions pass through Cloudflare's network on the way. That's not plaintext (HTTPS all the way), but if "no third party in the path" is the whole reason you set this up, stick to Tailscale.

ngrok (testing / temporary)

For quick testing, ngrok gives you a public URL in one command:

ngrok http 8080

It prints a https://xxx.ngrok-free.app URL. Paste that into the Diction app. Good for a demo or a five-minute test. Free tier URLs change every restart, which is annoying for permanent use. Also adds latency because your audio makes a round trip through ngrok's edge.

Which one?

Personal use, only you reach it: Tailscale. Fast, private, no external hostnames.
Family / small team reaches the same server: Cloudflare Tunnel. Pretty URL, TLS, one password.
Just testing: ngrok.

Already Have a Voice Model Server?

If you've already got a voice model server running somewhere - a self-hosted faster-whisper-server, a colleague's LocalAI instance, your employer's internal speech API - keep it. You don't need the voice model container from Step 3.

What you still need is the Diction Gateway. The iPhone app talks to it for WebSocket streaming and the end-to-end encryption handshake - neither of which a plain OpenAI-compatible transcription server exposes. Point the gateway at your existing server with CUSTOM_BACKEND_URL:

services:
  gateway:
    image: ghcr.io/omachala/diction-gateway:latest
    platform: linux/amd64
    container_name: diction-gateway
    restart: unless-stopped
    ports:
      - "8080:8080"
    environment:
      CUSTOM_BACKEND_URL: http://your-existing-server:8000
      CUSTOM_BACKEND_MODEL: Systran/faster-whisper-small
      # Optional LLM cleanup (Step 7):
      LLM_BASE_URL: "https://api.openai.com/v1"
      LLM_API_KEY: "${OPENAI_API_KEY}"
      LLM_MODEL: "gpt-4o-mini"
      LLM_PROMPT: "Clean up this voice transcription..."

Two extra knobs the CUSTOM_BACKEND_* path supports if you need them:

CUSTOM_BACKEND_AUTH: "Bearer sk-whatever". Sent as the Authorization header to your backend. For instances you've put an auth proxy in front of, or anything hosted that requires a token.
CUSTOM_BACKEND_NEEDS_WAV: "true". Some backends (Canary, Parakeet) only accept WAV. The gateway transparently converts incoming audio with ffmpeg before forwarding.

Point the iPhone at the gateway (http://your-server:8080), leave your existing voice model server where it is, and get streaming plus LLM cleanup on top.

Swap the Speech Model

The starter compose file runs small. That's a choice, not a commitment. Swapping to a different voice model size is two lines in your compose file plus a docker compose up -d. The gateway has a short name for each model it knows how to route to:

Short name (`DEFAULT_MODEL`)	Service hostname	Full model ID
`tiny`	`whisper-tiny`	`Systran/faster-whisper-tiny`
`small`	`whisper-small`	`Systran/faster-whisper-small`
`medium`	`whisper-medium`	`Systran/faster-whisper-medium`
`large-v3-turbo`	`whisper-large-turbo`	`deepdml/faster-whisper-large-v3-turbo-ct2`
`parakeet-v3`	`parakeet`	`nvidia/parakeet-tdt-0.6b-v3`

To swap from small to medium, rewrite your compose file so the whisper service is named whisper-medium, uses WHISPER__MODEL: Systran/faster-whisper-medium, and the gateway's DEFAULT_MODEL is medium.

If the service name doesn't match the short name the gateway expects, you'll see 404 model not found on every request. That's the #1 reason people get stuck when upgrading.

Running multiple models at once? Add more services (whisper-small + whisper-medium side by side) and the app can switch between them per-request by setting the model field in the request body. DEFAULT_MODEL only applies when the request doesn't specify one.

What This Actually Cost Me

The machine: whatever you already have idling at home
Electricity: the speech model at idle is effectively zero. Spikes briefly when you dictate.
OpenAI: gpt-4o-mini is the cheap model. An hour of dictation costs roughly a cent. Five dollars of credit lasts months.

Got an NVIDIA GPU Sitting Idle?

If the box you're setting this up on has an NVIDIA card in it, you can skip the small model and run something that's genuinely state of the art. CPU-only is fine for dictation. GPU unlocks the models that the paid services are running - often faster than those services, because there's no network round trip.

Two options. Pick one.

	Parakeet TDT 0.6B v3	large-v3-turbo
Best at	Speed + accuracy on European languages	Multilingual breadth (99 languages)
WER (English)	~6.3%	~7.4%
Latency	Sub-second	Under 2s on consumer GPU
VRAM (INT8)	~2 GB	~2.3 GB
Languages	25 European	99
Audio format	WAV only (gateway converts)	Anything (voice model handles it)

Option A: Parakeet (fastest, 25 European languages)

NVIDIA's Parakeet TDT 0.6B v3. On a recent consumer GPU (think RTX 3060 or better) it transcribes a 5-second clip in well under a second. Accuracy on clean English audio beats the large-v3 voice model on most benchmarks, at a fraction of the size and latency.

Supported languages: English, Bulgarian, Croatian, Czech, Danish, Dutch, Estonian, Finnish, French, German, Greek, Hungarian, Italian, Latvian, Lithuanian, Maltese, Polish, Portuguese, Romanian, Slovak, Slovenian, Spanish, Swedish, Russian, Ukrainian. If you dictate in any of these, Parakeet is the better engine.

If you dictate in Japanese, Mandarin, Arabic, Korean, or anything outside that list, use Option B.

Replace the whisper-small service in docker-compose.yml with this:

services:
  parakeet:
    image: ghcr.io/achetronic/parakeet:latest-int8
    container_name: diction-parakeet
    restart: unless-stopped
    ports:
      - "5092:5092"
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

  gateway:
    image: ghcr.io/omachala/diction-gateway:latest
    platform: linux/amd64
    container_name: diction-gateway
    restart: unless-stopped
    ports:
      - "8080:8080"
    depends_on:
      - parakeet
    environment:
      DEFAULT_MODEL: parakeet-v3

The gateway already knows how to speak to a service named parakeet on port 5092. No extra wiring needed. Test it exactly the same way as before.

You'll need the NVIDIA Container Toolkit installed on the host so Docker can pass the GPU through. One-line install if you haven't done it yet.

Option B: large-v3-turbo voice model (multilingual, frontier-tier)

The biggest model in this family, GPU-accelerated. This is what the paid cloud transcription services charge real money for. Runs great on any GPU with 6GB+ of VRAM.

services:
  whisper-large:
    image: fedirz/faster-whisper-server:latest-cuda
    container_name: diction-whisper-large
    restart: unless-stopped
    volumes:
      - whisper-models:/root/.cache/huggingface
    environment:
      WHISPER__MODEL: Systran/faster-whisper-large-v3-turbo
      WHISPER__INFERENCE_DEVICE: cuda
      WHISPER__COMPUTE_TYPE: float16
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

  gateway:
    image: ghcr.io/omachala/diction-gateway:latest
    platform: linux/amd64
    container_name: diction-gateway
    restart: unless-stopped
    ports:
      - "8080:8080"
    depends_on:
      - whisper-large
    environment:
      DEFAULT_MODEL: large-v3-turbo

volumes:
  whisper-models:

First boot pulls about 1.6GB of model weights. After that it's warm and fast.

What About NVIDIA Canary 1B?

If you've been reading up on speech models recently, you've probably seen Canary 1B at the top of the accuracy benchmarks. Yes, it's better than both options above on paper. The catch: NVIDIA ships it through NeMo, not as a turnkey OpenAI-compatible container. Getting it wrapped in the API the gateway expects is real work. You'll end up writing a small serving layer yourself. I run one of those internally for the Diction cloud, but I'm not going to pretend you can copy-paste a compose block for it. If you're willing to build that wrapper, point the gateway at it via CUSTOM_BACKEND_URL (see the next section) and you're set.

For everyone else: Parakeet or large-v3-turbo is already better than what most cloud services give you.

The OpenAI-Compatible API You Just Installed

The gateway speaks the OpenAI audio transcription API. That means anything that knows how to talk to api.openai.com/v1/audio/transcriptions also knows how to talk to your server. You spun up the iPhone keyboard client of this API, but you can also point laptops, scripts, or other services at the same URL.

Quick Python example using the official OpenAI SDK:

from openai import OpenAI

client = OpenAI(
    base_url="http://192.168.1.42:8080/v1",
    api_key="anything",   # not checked by default
)

with open("meeting.m4a", "rb") as f:
    text = client.audio.transcriptions.create(
        file=f,
        model="small",
        response_format="text",
    )

print(text)

Same thing works for the Node SDK, LangChain, Flowise, n8n, anything. Treat it as a local stand-in for OpenAI's hosted API.

What's supported

POST /v1/audio/transcriptions with file, model, language, prompt, response_format=json|text
GET /v1/models - lists the speech engines and models the gateway can route to. Response shape is Diction's own ({"providers": [{"id": "whisper", "models": [...]}, ...]}), not OpenAI's flat data array, so OpenAI SDK .models.list() calls won't parse it cleanly. Hit it directly with curl if you want to see what's available.
Multiple short-name aliases: small, medium, large-v3-turbo, parakeet-v3
HuggingFace-style IDs: Systran/faster-whisper-small, nvidia/parakeet-tdt-0.6b-v3, etc.

What's not supported

Text-to-speech (/v1/audio/speech). This is transcription only.
response_format=verbose_json | srt | vtt. No word-level timestamps.
Server-Sent Events streaming on the REST endpoint. Use the WebSocket /v1/audio/stream for streaming.
OpenAI's Realtime API (/v1/realtime).

Authentication

By default the gateway has AUTH_ENABLED=false. Pass any non-empty string as the API key - nothing's checked. If you want to lock it down (e.g. exposing via Cloudflare Tunnel), set AUTH_ENABLED=true and configure the token in your gateway env. The server/docker-compose.yml in the public repo has a more elaborate example if you want to see it.

Caveat: error response shape

Diction's gateway returns errors as {"error":"message"}, not OpenAI's nested {"error":{"message":"...","type":"..."}}. Most SDKs surface these as a raw HTTPError rather than a parsed APIError. Catch both if you're writing something defensive.

Privacy: What Actually Happens to Your Audio

The whole reason most people set this up is not paying a random SaaS to process their voice. Worth being precise about what this stack does and doesn't do:

What leaves your iPhone: raw audio, encoded as Opus (over WebSocket stream) or WAV (over REST), heading to the server endpoint you configured.

In transit: HTTP by default. Plain text audio over your LAN. That's fine on a trusted home network. If you expose the gateway over the internet (Cloudflare Tunnel, ngrok, your own reverse proxy), put TLS in front of it. Tailscale wraps everything in WireGuard so you don't need to think about TLS at all - that's part of why I prefer it.

What your server does with the audio: feeds it to the voice model container. The voice model transcribes. Returns text. Audio gets thrown away - neither the gateway nor faster-whisper-server persists audio anywhere. docker compose logs contains request metadata (latency, model used, text length) but not the audio or the transcript. You can verify yourself: docker exec diction-whisper-small ls -la /tmp is essentially empty between requests.

If cleanup is enabled: the transcript (plain text, no audio) gets sent to your configured LLM endpoint. That's the only point where data leaves your server. If you pick a local Ollama, nothing leaves the house at all. If you pick OpenAI/Groq/whatever, the transcript passes through their infrastructure. Their data policies apply to that leg - read them if it matters.

What the Diction app does with your audio: nothing. The keyboard's only job is to stream to your endpoint and insert the response. No analytics, no tracking, no background uploads. The app has no QWERTY input, so there's literally nothing to log even if it wanted to. Source for the server-side code is on GitHub (the iOS app itself isn't open source, but the data flow on the wire is straightforward: one POST per dictation, to the endpoint you configured).

Full Access permission: iOS requires this for any keyboard that touches the network. It's a coarse switch that also grants things like pasteboard access. Diction uses the network part and nothing else - again, no typed input, no pasteboard monitoring. If you'd rather not trust that claim, run the setup from this article and point Wireshark at the gateway's port. You'll see exactly one connection per dictation, to your endpoint.

One Small Thing About "AI Companion"

If you dig around the Diction app's settings you'll find an "AI Companion" toggle with its own prompt field. Worth knowing how that interacts with what you just built.

The toggle is what tells the app to ask for cleanup (?enhance=true in the request). It's the on/off switch. But the actual prompt the LLM sees is whatever you put in LLM_PROMPT in your compose file. The in-app prompt field is used by the hosted Diction Cloud setup. On your own server, your env var wins. Every time.

So: flip AI Companion on in the app if you want cleanup to run. Tune the prompt by editing docker-compose.yml and running docker compose up -d again. Nothing else to configure.

It's Open Source. Go Wild.

The gateway is on GitHub at omachala/diction under an open-source license. If there's a behavior you want that it doesn't have, fork it. If you hit a bug or add something other people would benefit from, I'd love a pull request. The codebase is small and deliberately boring Go. You don't need to be an expert to find your way around.

Some things I know people want and haven't built yet: per-app routing (different models for different apps), a richer context API, swappable post-processing pipelines. If any of those scratch your itch, the code's right there.

Heard of Speaches?

Speaches is the nearest neighbor - an OpenAI-compatible self-hosted speech server with transcription, TTS, and a realtime API. Good project for a general-purpose endpoint. It won't drive the Diction keyboard, though: the app opens a WebSocket at /v1/audio/stream and does an X25519 + AES-GCM handshake on every request, and Speaches streams transcription over SSE on the REST endpoint with no knowledge of that handshake. That's why I wrote Diction Gateway - the keyboard's protocol baked in, end-to-end encrypted transcripts by default, BYO LLM cleanup in a single env var, and a thin wrapper mode (CUSTOM_BACKEND_URL) so you can put it in front of any existing speech server. Even outside the keyboard use case, if you want a minimal OpenAI-compatible speech gateway with an LLM cleanup step wired in, reach for this one.

Where to Go Next

Some directions once the base setup is working:

Ditch the cloud LLM for a local model. You already saw the Ollama option in Step 7. Uncomment it in your compose file, ollama pull gemma2:9b, done. Nothing leaves your house. I've got a full walkthrough of the Ollama side here.
Move off home WiFi. Tailscale (Reach It From Anywhere section above) is the easy answer. Five minutes to set up, dictation works at the café.
Upgrade the speech model. Start with small, move to medium once you notice misheard words, jump to large-v3-turbo if you've got a GPU. Model accuracy climbs noticeably between each tier.
Dictate in another language. The voice model autodetects, so you don't have to do anything. If you're mostly in a European language and have a GPU, switch to Parakeet - it's meaningfully more accurate for those.
Tune the cleanup prompt. The default prompt fixes filler words and punctuation. Try the email-ready rewriter, the bullet-pointer, or your own variant. See the prompt library in Step 7.
Add a second gateway. Run one on your home server (high quality, slow connection over VPN) and one on a dev laptop (lower quality, instant local). Switch per-network.
Plug the gateway into other things. It's an OpenAI-compatible speech endpoint. Any transcription workflow - meeting notes, voice memos pipeline, automatic subtitling - can point at it instead of OpenAI.
Contribute. If you build something useful on top of this, PR it to omachala/diction. Better prompts, better docs, new backends, whatever.

The keyboard is in the App Store. You can self-host, use the Diction Cloud, or both. The app lets you switch per-app - self-host your Telegram dictation, use the cloud when you're offline from your tailnet, on-device only mode for the really sensitive stuff. Mix and match.

Closing the Thread

What I like about this setup: I can talk to OpenClaw and the rest of my agents without worrying about who else is listening on the way in. The keyboard's as fast as the built-in one. Short dictations land in under a second. The only thing I pay is whatever my cleanup LLM costs - pennies on OpenAI, zero on local Ollama. The rest stays on my hardware.

The project is still quite new, but the feedback from people using it daily has been genuinely amazing. I'm adding features almost every week and making the whole thing more rock solid with each release. If there's something missing for your workflow, say so - good chance it's on its way or can be.

If you found this useful, a GitHub star on omachala/diction would be a lovely token of appreciation - it's the easiest way to tell me this stuff is worth building more of. Try the app, tell someone else who'd find it useful, and if you hit something that's broken or confusing in this walkthrough, ping me. I'll fix it.

Happy dictating.

I Plugged Ollama Into My iPhone Keyboard. Here's the Full Self-Hosted Stack.

Ondrej Machala — Thu, 16 Apr 2026 09:00:00 +0000

You've got Ollama running on your home server. Your iPhone keyboard is still phoning home.

That gap bothered me enough to close it. Diction is an open-source iOS keyboard that speaks the standard OpenAI transcription API, POST /v1/audio/transcriptions. Its gateway is open source Go. You configure it entirely with environment variables. The latest update adds something I wanted from the start: plug in your own LLM for post-processing. Ollama, OpenAI, anything with an OpenAI-compatible endpoint.

Here's the full stack.

What Each Piece Does

The speech engine does the transcription. I ran Whisper for months. But there's a faster option if you dictate in English or another European language: NVIDIA's Parakeet model, available via the achetronic/parakeet Docker image. It supports 25 languages. On CPU it's roughly 10x faster than Whisper, uses about 2GB RAM, and edges out Whisper large-v3 on accuracy for English. Models are baked into the image, so no first-run download.

If you need Asian languages, Arabic, or anything outside those 25, use Whisper instead. Everything below works with either.

The gateway sits in front of the speech engine. It handles WebSocket streaming, so your phone streams audio live while you're still talking. By the time you stop, the transcript is mostly ready. It also handles the new LLM post-processing step.

Ollama cleans up the transcript after transcription. The gateway calls it with a system prompt you write, and the cleaned text is what gets inserted into the app. Your model, your prompt.

The Stack

Three containers. No GPU required. Under 3GB RAM without Ollama, 8-10GB with a 9B model loaded.

services:
  parakeet:
    image: ghcr.io/achetronic/parakeet:latest-int8
    ports:
      - "9006:5092"

  gateway:
    image: ghcr.io/omachala/diction-gateway:latest
    ports:
      - "8080:8080"
    depends_on:
      - parakeet
    environment:
      DEFAULT_MODEL: parakeet-v3
      LLM_BASE_URL: http://ollama:11434/v1
      LLM_API_KEY: ollama
      LLM_MODEL: gemma2:9b
      LLM_PROMPT: "Clean up this voice transcription. Remove filler words (um, uh, like). Fix punctuation and grammar. Return only the cleaned text."

  ollama:
    image: ollama/ollama
    volumes:
      - ollama-data:/root/.ollama

volumes:
  ollama-data:

docker compose up -d
docker compose exec ollama ollama pull gemma2:9b

Skip the Ollama block entirely if you don't want AI cleanup. The gateway checks for LLM_BASE_URL on startup. If it's not set, transcriptions come back raw.

If you already have Ollama running on a different machine, point LLM_BASE_URL at it. Works with any model you've already pulled.

For Whisper instead of Parakeet, swap the parakeet service for fedirz/faster-whisper-server:latest-cpu and set DEFAULT_MODEL: small (or medium, large-turbo) on the gateway.

Connecting the App

Install Diction from the App Store
In iPhone Settings: General → Keyboard → Keyboards → Add New Keyboard → Diction
Open the Diction app, switch to Self-Hosted
Paste your server address: http://192.168.1.100:8080
A green dot confirms the connection
Enable AI Enhancement in app settings (requires a Diction One subscription to unlock the toggle — the processing runs on your server, not Diction's)

The keyboard is now using your server. Audio goes from your phone to your server, Parakeet transcribes it, Ollama cleans the result, text lands in whatever app you're typing in. Nothing leaves your network.

If you're away from home, Tailscale or a Cloudflare Tunnel connects your phone without opening router ports.

Writing a Prompt That Works

The LLM_PROMPT env var is a single system prompt. The gateway sends it with every transcription request. The transcript is the user message. You control both.

A few starting points:

# General dictation
Remove filler words (um, uh, like, you know). Fix punctuation and grammar.
Preserve meaning and tone. Return only the cleaned result.

# Technical / developer notes
Fix transcription errors. Preserve technical terms, command names, and file paths
exactly as spoken. Remove filler words. Return cleaned text only.

# Medical or domain-specific
Fix transcription errors. Preserve all domain-specific terminology exactly as spoken.
Fix grammar and punctuation only. Return the corrected text.

One practical note: models under 7B parameters often answer the transcript rather than clean it. Gemma2 9B is reliable. Qwen2.5 7B is borderline on this task. Anything 9B+ behaves predictably.

What You Get

Audio from your phone to your server. Parakeet or Whisper transcribes it. Ollama cleans it. Text inserted. No third party, no word limits. The Diction gateway is fully open source on GitHub — inspect every line that runs in your network.

Full setup docs at diction.one/self-hosted. If you're already running a homelab with Ollama, the marginal effort is a single compose file and 10 minutes.

What's New in Diction 5.0

Ondrej Machala — Tue, 14 Apr 2026 21:18:46 +0000

Diction 5.0 is out. Three things changed in a meaningful way: the cloud is rebuilt from scratch, AI Companion can now edit text by voice, and the app is fully localized in 13 languages. Everything else is polish on top of that foundation.

Diction One is a different thing now

I spent several weeks working with physical hardware for this. Dedicated speech models for each language family, better audio processing, faster response across the board. If you tried cloud mode in an earlier version and found it slow or inaccurate, I'd ask you to try it again.

The mic is always warm now. Tap and start talking immediately. No noticeable gap between the button and your voice.

Your voice can now edit

AI Companion in v4 could clean up what you said. In v5, it can edit what's already written.

You're writing an email. You wrote "let's meet Tuesday" but the meeting moved. Place your cursor anywhere in that sentence and say "change Tuesday to Thursday." No selecting, no deleting, no retyping.

Or select a paragraph that's too stiff and say "make this more casual." Diction rewrites the selection. The action bar turns indigo when text is selected so you always know what mode you're in.

Long-press the action bar to rewrite text around your cursor without selecting anything first. Say what's wrong, Diction fixes it.

13 languages, properly localized

Every screen now adapts to your system language: Settings, History, Insights, everything. Switch the UI language live from the picker without restarting the keyboard.

Before this release, everyone got English regardless of what their iPhone was set to.

Other changes worth knowing

Insights is redesigned. Your typing speed multiplier is the main number, with daily average, words per minute, days used, and time saved all visible at once.

New mic release options. "After dictation" drops the mic the moment the transcription finishes. Ten-second and 30-second options for music and podcast listeners.

AirPods work correctly now. Music stays in stereo while Diction uses the built-in mic. Nothing ducks.

AI Companion is smarter about keeping your voice. It preserves natural speech patterns, writes numbers as digits, and doesn't drop sentences on longer recordings.

For self-hosters: support for bringing your own language model for AI Companion, a one-command setup covering 25 European languages, and smart routing that picks the right speech model per language with a health-checked fallback. Open source at github.com/omachala/diction.

Diction 5.0 is on the App Store now: https://apps.apple.com/app/id6759807364

How Diction handles privacy

Ondrej Machala — Thu, 02 Apr 2026 09:00:00 +0000

Voice keyboards sit in an uncomfortable position. Every app you use, every message you send, every search you type — the keyboard is there. It sees all of it.

Most people install a keyboard and never think about this. I did, because I was building one.

Earlier this year, someone reverse-engineered a popular voice keyboard and posted their findings. The app was collecting full browser URLs, names of focused apps, on-screen text scraped via the Accessibility API, clipboard contents including data copied from password managers, and sending it all back to a server. There was a function in the binary called sendTrackResultToServer. None of this was in the privacy policy.

This is not a hypothetical. It happened. And the only reason anyone found out is because the app was installed on a machine where someone was curious enough to look.

That is the problem with closed-source software and privileged access: you cannot verify the claims. A privacy policy is a document. The code is what runs.

Full Access and what it actually enables

When iOS asks if you want to allow Full Access for a keyboard, the permission is broader than most people realise. It enables network access (how keyboards send audio for transcription or sync dictionaries). But in the wrong hands it also means the keyboard code runs in a context where it could read clipboard data, monitor app usage patterns, or transmit information alongside its legitimate function.

Diction has no QWERTY keys. There is nothing to type into it, so nothing to log in that sense. But I wanted to go further than just "we don't do the bad thing." I wanted to build it so you can verify we don't.

How I built Diction with this in mind

There are three ways Diction can process your audio, and I picked each one with this threat model in mind.

On-device is the cleanest answer. Your audio never leaves your iPhone. A local speech model handles transcription, the result comes back, and that is it. No server, no transmission, no policy to read. If you want absolute certainty, this is the mode for you.

Self-hosted is for people who want cloud-quality transcription but on infrastructure they control. You point the app at your own server. Your audio goes there and nowhere else. I have no access to what you say or what gets transcribed. The server software is open source. You can read exactly what it does before you run it.

Diction One is the hosted cloud option. Here I had to think carefully. Audio is processed in memory and discarded immediately after transcription. Nothing is written to disk. No transcriptions are stored or logged. And every transcription is encrypted with AES-256-GCM using a fresh X25519 key per request. The same standards WireGuard uses. I am not asking you to trust the policy. The implementation is in the open-source server code.

The app itself

The Diction app contains no analytics and no tracking code. No device identifiers, no usage events, no behavioural monitoring. The App Store privacy label reads "Data Not Collected." I can say that confidently because I wrote every line and there is nothing there.

What you can actually verify

The server code is public at github.com/omachala/diction. You can read the transcription handler and confirm that audio is not written anywhere. You can read the encryption implementation. If you run on-device mode, you can point a network inspector at the app and confirm no requests leave it.

I built it this way because I wanted to use this keyboard myself. And I was not willing to just trust a policy page written by someone else.

Download on the App Store · diction.one/privacy-first

What's New in Diction 4.0

Ondrej Machala — Wed, 01 Apr 2026 15:04:42 +0000

Diction 4.0 is the biggest update since launch. The theme is straightforward: do more with your voice, with fewer rough edges. This release took hundreds of commits and more testing rounds than I want to count. Here is what landed.

Speak to Edit

This is the headline feature. Select any text in any app, tap the mic, and say what you want changed.

You are editing an email. You select "Wednesday works for me" and say "Thursday actually." Diction replaces the selection.

It also handles instructions. Select a paragraph and say "translate to Czech." Or "make this shorter." Or "more formal." Diction figures out whether you are giving a literal replacement or an editing instruction and acts accordingly.

Before this, voice keyboards were append-only. You could dictate new text, but editing meant switching to the regular keyboard. Now you stay in voice the whole time.

Custom Words Improve Transcription Directly

In 3.0, custom words (My Words) only helped during AI Enhancement cleanup. Now they feed directly into the speech model as vocabulary hints.

Your coworker's name is Kaelith. Your product is called Nexaro. You added both to My Words. Now the raw transcription gets them right on the first pass, even with AI Enhancement turned off.

This matters most for anyone dictating technical terms, brand names, or anything the base model has never seen.

Long Recordings That Actually Finish

Previous versions could cut off or lose the end of longer dictations. 4.0 rewrites the recording pipeline to handle long sessions without dropping audio.

If you are dictating meeting notes, a long email, or journal entries, the transcript comes back complete.

Profile

Tell Diction who you are and how you write. "I'm a software engineer. I write in short, direct sentences. I use American English."

AI Enhancement uses your profile to match your style. Instead of generic cleanup, it produces text that sounds like you actually wrote it. The profile persists across all your dictations, so you set it once.

Guided Onboarding

First launch used to throw permission dialogs at you and hope you figured it out. Now there is a step-by-step walkthrough: keyboard installation, permissions, first dictation. You know exactly where you are and what to do next.

Better On-Device Setup

Downloading speech models should not be confusing. The download flow is smoother now, preparation is faster, and the model is ready to use as soon as it finishes. No extra steps.

No More Phantom Orange Dot

Opening the Diction app used to activate the microphone, which lit up the iOS orange dot even though you were not dictating. Fixed. The mic only activates when you actually start a dictation.

Under the Hood

AI Enhancement accuracy improved across apps
UI polish across the keyboard, history, tones, and settings
Stability improvements throughout. 4.0 is a significantly more stable release than 3.0.

Diction is a voice keyboard for iPhone. Tap the mic, speak, text appears wherever your cursor is. On-device, cloud, or self-hosted.

App Store / GitHub / Website

I Use NanoClaw in Telegram All Day. I Stopped Typing to It.

Ondrej Machala — Sat, 28 Mar 2026 09:00:00 +0000

I have NanoClaw connected to my Telegram. Throughout the day I send it things. Translate this. Summarise that article. What time is it in Tokyo. Draft a reply to this message. It responds in the same thread, without me leaving the app.

It runs on my own machine, inside a container. The agent only has access to what you explicitly give it. Setup took about fifteen minutes: clone the repo, run Claude Code, type /setup.

What I didn't anticipate: I was still typing everything. Long questions. Multi-sentence requests. Context I had to spell out carefully. The assistant was right there in Telegram, but slow input was still slowing me down.

Where Diction comes in

I built Diction to fix exactly this. It's an iOS keyboard extension. In Telegram, you switch to it, tap the mic, speak, and the text appears in the compose field. Send it like any message.

I dictate to NanoClaw now. "Can you draft a short reply to this email, keep it friendly but firm." Things that would take a minute to type take ten seconds to say. NanoClaw gets the same message either way.

Diction has an on-device mode that runs locally on your iPhone. Nothing leaves the device. For a setup where the whole point is keeping your data on your own hardware, that felt like the right match.

The setup, end to end

NanoClaw:

Clone github.com/qwibitai/nanoclaw
Run Claude Code in the repo directory
Type /setup — Claude Code handles everything
Connect your Telegram bot token when prompted

Diction:

Install from the App Store
Settings → General → Keyboard → Add New Keyboard → Diction
Switch to it in Telegram, tap the mic

That's the whole stack. A personal assistant on your own hardware, voice input on your own device.

NanoClaw on GitHub | Diction on the App Store

Self-Host Speech-to-Text and Use It as Your iPhone Keyboard in 3 Commands

Ondrej Machala — Thu, 26 Mar 2026 09:00:00 +0000

If you're running a homelab, you've probably already got speech-to-text somewhere in your stack.

Maybe you use it for Home Assistant voice commands. Or local LLM integrations. Or just transcribing meeting recordings.

Here's something you might not have considered: you can use that same transcription server as a keyboard on your iPhone.

The 3 Commands

git clone https://github.com/omachala/diction
cd diction
docker compose up -d

That's the server running. Now install Diction on your iPhone, point it at your server URL, and you have a voice keyboard backed by your own speech-to-text instance.

What's Actually Running

The Docker Compose setup spins up two services:

services:
  gateway:
    image: ghcr.io/omachala/diction-gateway:latest
    ports:
      - "8080:8080"

  whisper-small:
    image: fedirz/faster-whisper-server:latest-cpu
    environment:
      WHISPER__MODEL: Systran/faster-whisper-small
      WHISPER__INFERENCE_DEVICE: cpu

whisper-small: the transcription engine — runs open-source Whisper via a REST API. CPU works fine for real-time dictation.

gateway: a small open-source Go service that handles communication between the iOS app and the transcription backend. It accepts WebSocket connections from the phone, buffers audio frames, and forwards them to Whisper. This is what makes dictation feel instant instead of "record, upload, wait."

The gateway exposes port 8080. That's the URL you put into the Diction app.

Making It Accessible From Your Phone

Your phone needs to reach your server. A few options depending on your setup:

Tailscale (easiest): Install Tailscale on both your server and iPhone. You get a private IP accessible from anywhere. No port forwarding, no firewall rules.

http://100.x.x.x:8080

Reverse proxy (for existing homelabbers): If you're already running Caddy, nginx, or Traefik, add a route to port 8080.

https://diction.yourdomain.com

Direct LAN (simplest for home-only use): Just use your server's local IP. Works on home WiFi, not outside.

http://192.168.1.100:8080

Choosing a Model

Swap the transcription model by changing WHISPER__MODEL. The model downloads automatically on first use.

Size	RAM	Speed (CPU)	Notes
Tiny	~350MB	~1-2s	Lower accuracy, great for low-power hardware
Small	~800MB	~3-4s	Good default for everyday dictation
Medium	~1.8GB	~8-12s	Better with accents and background noise
Large	~3.5GB	~20-30s	Highest accuracy, benefits from GPU

For most home servers, the small model hits the sweet spot — fast enough to feel real-time, accurate enough for messages and notes.

Swap it in your compose file:

  whisper-small:
    image: fedirz/faster-whisper-server:latest-cpu
    environment:
      WHISPER__MODEL: Systran/faster-whisper-medium
      WHISPER__INFERENCE_DEVICE: cpu

Running on Lower-Power Hardware

The small model runs well on modern CPUs. For a NAS or Raspberry Pi, try tiny — less RAM (~350MB), faster responses, some accuracy trade-off. For real-time keyboard use, aim for sub-3 second round-trip. Small on a modern CPU or tiny on lower-power hardware gets you there.

Already Running a Whisper Server?

If you already have a speech-to-text container running, you don't need to spin up another one. Just run the gateway and point it at your existing server:

services:
  gateway:
    image: ghcr.io/omachala/diction-gateway:latest
    ports:
      - "8080:8080"
    environment:
      CUSTOM_BACKEND_URL: http://your-server:8000
      CUSTOM_BACKEND_MODEL: your-model-name

More details on connecting to existing servers in this post.

The server and gateway are fully open source: github.com/omachala/diction

You Already Have a Speech Server. Your iPhone Keyboard Should Use It.

Ondrej Machala — Wed, 18 Mar 2026 09:36:35 +0000

Someone posted on our GitHub Discussions this week. They'd been running a speech-to-text container on their homelab for months. Found Diction, an open-source iOS voice keyboard. Pointed the app at their server. Got a server error. The settings screen even said "endpoint reachable."

Here's what was going wrong, and how two lines of config fixes it.

Why direct connection fails

Diction doesn't talk directly to speech servers. It connects through a lightweight gateway first.

The reason is WebSockets. When you tap the mic, the app opens a WebSocket and streams raw PCM audio to the gateway in real time as you speak. When you're done, the gateway POSTs the full audio to your speech server, gets the transcript, and sends it back. The whole exchange happens in the time it takes to stop speaking.

Without this, the alternative is: record the whole thing, send a file, wait. You'd feel every pause. The WebSocket is what makes it feel instant.

The "endpoint reachable" check passes because the iOS app pings /health or /v1/models. Most speech servers expose these. But the actual transcription uses the WebSocket endpoint, which only the gateway handles. No gateway, no streaming.

The fix

You don't need to run our speech containers. Just the gateway, pointed at yours.

If your server is at http://192.168.1.50:8000, this is your entire docker-compose.yml:

services:
  gateway:
    image: ghcr.io/omachala/diction-gateway:latest
    ports:
      - "8080:8080"
    environment:
      CUSTOM_BACKEND_URL: http://192.168.1.50:8000
      CUSTOM_BACKEND_MODEL: your-model-name-here

Start it:

docker compose up -d

Open Diction, go to Self-Hosted, paste http://192.168.1.50:8080. Done.

Your model stays where it is. The gateway handles the WebSocket layer, audio buffering, and forwarding. Audio still only goes to your server.

CUSTOM_BACKEND_MODEL

One thing to get right: the model name.

Most speech servers that follow the OpenAI-compatible API format expect a model field in the transcription request to know which model to load. Without it, some return an error.

Set CUSTOM_BACKEND_MODEL to whatever name your server expects. Check your server's docs or the model you started it with. If your server only runs one model and ignores the field, you can omit it entirely.

WAV-only servers

Some speech servers only accept WAV audio input. The gateway handles conversion automatically:

environment:
  CUSTOM_BACKEND_URL: http://192.168.1.50:5092
  CUSTOM_BACKEND_NEEDS_WAV: "true"

With this set, the gateway converts audio to 16kHz mono WAV via ffmpeg before forwarding. Your server gets the format it expects.

API key protection

If your server is behind an API key:

environment:
  CUSTOM_BACKEND_URL: http://my-server:8000
  CUSTOM_BACKEND_MODEL: my-model
  CUSTOM_BACKEND_AUTH: "Bearer sk-your-key"

The gateway injects the Authorization header on every request to your backend.

Latency on a local network

I tested this end-to-end: generated a speech WAV, sent it through the gateway to a real speech container, got the transcript back correctly. On a local network with a CPU-only container, the round trip was under 5 seconds. With a dedicated GPU, it's near instant.

What the gateway actually does

The gateway is open source at github.com/omachala/diction. It's a small Go service that:

Accepts WebSocket connections from the iOS app
Buffers incoming PCM audio frames
Wraps them in a WAV header and POSTs to your speech backend
Returns the transcript over the WebSocket

No cloud calls. No telemetry. The full source is in /gateway/core/.

If you're running a Whisper server and want to get started from scratch, the 3-command setup guide covers the full stack.

You Wrote 14 Playwright Scripts Just to Screenshot Your Own App

Ondrej Machala — Tue, 17 Mar 2026 14:00:00 +0000

It started simple. One Playwright script to capture the homepage.

const browser = await chromium.launch();
const page = await browser.newPage();
await page.goto('https://myapp.com');
await page.screenshot({ path: 'homepage.png' });
await browser.close();

Then the team needed the pricing page. So I added another script. Then the dashboard (which needs login first). Then the settings page (which needs a specific tab clicked). Then mobile versions.

Two months later I had 14 Playwright scripts. Some shared a login helper. Some had hardcoded waits. One had a try-catch that silently swallowed errors because the cookie banner sometimes loaded and sometimes didn't.

I was maintaining a bespoke test suite, except it wasn't testing anything. It was just taking pictures.

Config, not code

Here's what those 14 scripts look like as config:

{
  "hiddenElements": {
    "myapp.com": [".cookie-banner", ".chat-widget"]
  },
  "screenshots": [
    { "name": "homepage", "url": "https://myapp.com", "selector": ".hero" },
    { "name": "pricing", "url": "https://myapp.com/pricing", "selector": ".pricing-grid" },
    { "name": "dashboard", "url": "https://myapp.com/dashboard", "selector": ".dashboard" },
    {
      "name": "settings",
      "url": "https://myapp.com/settings",
      "selector": ".settings-panel",
      "actions": [
        { "type": "click", "selector": "[data-tab='notifications']" }
      ]
    }
  ]
}

npx heroshot

All 14 screenshots. One command. No scripts to maintain.

The cookie banner problem

In my scripts, I had this pattern everywhere:

try {
  await page.click('.cookie-banner .dismiss', { timeout: 3000 });
} catch { /* maybe it didn't show */ }

With heroshot, you define hidden elements once per domain:

{
  "hiddenElements": {
    "myapp.com": [".cookie-banner"],
    "docs.myapp.com": [".cookie-banner", ".announcement-bar"]
  }
}

Every screenshot on that domain hides those elements automatically. No try-catch. No timeouts. No "maybe it showed, maybe it didn't."

Actions for complex state

The settings page needed a tab clicked first. In Playwright, that's 5 lines of setup. In config:

{
  "actions": [
    { "type": "click", "selector": "[data-tab='notifications']" },
    { "type": "wait", "text": "Email preferences" }
  ]
}

There are 14 action types: click, type, hover, select_option, press_key, drag, wait, navigate, evaluate, fill_form, handle_dialog, file_upload, resize, and hide. Covers pretty much every pre-screenshot setup I've needed.

When to use Playwright directly

If you need complex conditional logic, dynamic data generation, or integration with a test framework, raw Playwright is the right tool.

But if you're just taking pictures of known pages at known states, config is simpler, more readable, and doesn't break when someone renames a helper function.

I Stopped Paying $15/Month for Wispr Flow. Here's the Open-Source Replacement.

Ondrej Machala — Mon, 16 Mar 2026 08:58:03 +0000

I paid for Wispr Flow for five months.

A monthly subscription. Every month. For voice-to-text on my iPhone.

It's a good product. The AI editing layer is genuinely impressive — it strips filler words, fixes grammar, adapts to how you write. That part works. If you want the best cloud-based dictation and don't mind paying, Wispr delivers.

But every time I used it, the same thought: my voice is going to their cloud. Not my cloud. Theirs.

I already run a home server. Docker Compose, Tailscale, the usual homelab stack. I had faster-whisper running for other things. The transcription engine was already there. I just didn't have a way to use it from my phone.

So I built one.

What the switch actually looked like

The server side was easy. I already had the transcription container. I wrote a small Go gateway to handle WebSocket streaming from the phone, and wrapped both in a compose file:

services:
  transcription:
    image: fedirz/faster-whisper-server:latest-cpu
    volumes:
      - models:/root/.cache/huggingface

  gateway:
    image: ghcr.io/omachala/diction-gateway:latest
    ports:
      - "8080:8080"
    environment:
      DEFAULT_MODEL: small
    depends_on:
      - transcription

volumes:
  models:

docker compose up -d and it's running.

The hard part was the iOS keyboard. Keyboard extensions on iOS run in a sandbox with a 48MB memory ceiling, no direct mic access without Full Access, and a text proxy that behaves differently in every app. That took months, not hours.

The result is Diction — a voice keyboard that connects to whatever transcription server you point it at.

What's honestly worse

Wispr's AI editing layer is better than raw transcription. It doesn't just transcribe — it rewrites. Filler words vanish, punctuation lands correctly, and it matches your tone. Diction transcribes what you say. It has optional AI cleanup now, but Wispr's has had years of refinement.

Wispr also has a personal dictionary that learns your vocabulary over time. Diction has custom dictionaries too, but they're newer and simpler.

If you don't want to think about infrastructure and just want the best cloud experience, Wispr is still a strong choice.

What's better

My audio stays on my network. I can verify that because the server code is open source — there's nothing to trust on faith.

No word limits. Wispr's free tier caps you at 1,000 words/week on iOS. Self-hosted Diction has no caps, no subscription, no catch.

Latency on a local network is excellent. The small Whisper model on a modern CPU returns transcriptions in 2-4 seconds. With a GPU, it's near instant.

And when my internet goes down, on-device mode keeps working. Wispr is cloud-only — no connection, no transcription.

The honest trade-off

I traded polish for control. Wispr is more refined. Diction gives me ownership of the entire pipeline, from the mic to the model, and it's getting better with every release.

If you're already running Docker at home and the idea of sending every word you speak to someone else's server bothers you, the self-hosted setup takes about 10 minutes.

github.com/omachala/diction

Astro Docs Without a Single Manual Screenshot

Ondrej Machala — Sat, 14 Mar 2026 14:00:00 +0000

I set up an Astro docs site last week. Content collections, MDX pages, Starlight theme. Beautiful.

Then I needed screenshots. The dashboard page, the settings panel, the onboarding flow. Three screenshots, light and dark mode each, so six images total.

I spent an hour taking them by hand. Opened the app, resized the browser, captured, cropped, saved, repeated. Six times.

The next sprint, the onboarding flow changed. Screenshots were already stale.

Config-driven screenshots

Instead of capturing manually, define what you need:

{
  "outputDirectory": "src/assets/screenshots",
  "screenshots": [
    {
      "name": "dashboard",
      "url": "https://myapp.com/dashboard",
      "selector": ".dashboard-grid"
    },
    {
      "name": "settings",
      "url": "https://myapp.com/settings",
      "selector": ".settings-panel"
    },
    {
      "name": "onboarding",
      "url": "https://myapp.com/onboarding",
      "selector": ".onboarding-wizard"
    }
  ]
}

npx heroshot

Three config entries, six files. Light and dark variants are included automatically.

Using them in Astro

In your MDX file:

import { Image } from 'astro:assets';
import dashboard from '../../assets/screenshots/dashboard-light.png';

<Image src={dashboard} alt="Dashboard overview" />

Or if you want automatic dark mode switching, use a <picture> tag:

<picture>
  <source
    srcset="/screenshots/dashboard-dark.png"
    media="(prefers-color-scheme: dark)"
  />
  <img src="/screenshots/dashboard-light.png" alt="Dashboard" />
</picture>

Heroshot has a shortcut for this:

npx heroshot snippet

It generates the <picture> markup for every screenshot.

Astro + Starlight

If you're using Starlight (Astro's docs theme), screenshots go in src/assets/ so Astro optimizes them at build time. Set the output directory accordingly:

{
  "outputDirectory": "src/assets/screenshots"
}

Starlight handles responsive images, lazy loading, and format conversion automatically. You just provide the source PNGs.

Keeping them fresh

Add to your workflow:

# After every deploy
npx heroshot

# Check if anything changed
git diff --name-only src/assets/screenshots/

If screenshots changed, commit them. If nothing changed, move on.

Heroshot is open source and works with any static site generator. Astro, Next.js, VitePress, whatever. The config is framework-agnostic.

Keep Your MkDocs Screenshots Up to Date (Material Theme)

Ondrej Machala — Thu, 29 Jan 2026 08:51:37 +0000

You've got MkDocs with Material theme. Dark mode works. Code blocks adapt. But your screenshots? Still stuck in light mode.

Here's how to fix that with Heroshot - a CLI that captures screenshots and handles light/dark variants automatically.

Step 1: Install the CLI

Pick your preferred method:

curl (standalone binary):

curl -fsSL https://heroshot.sh/install.sh | sh

Homebrew:

brew install omachala/heroshot/heroshot

npm:

npm install -g heroshot

Docker:

docker pull heroshot/heroshot

Step 2: Install the Python package

The CLI captures screenshots. The Python package provides the MkDocs macro:

pip install heroshot

Step 3: Configure output directory

MkDocs serves from docs/. Create .heroshot/config.json:

{
  "outputDirectory": "docs/heroshots"
}

Your structure:

my-project/
├── docs/
│   ├── index.md
│   └── heroshots/    # screenshots go here
├── mkdocs.yml
└── .heroshot/
    └── config.json

Step 4: Capture screenshots

Start your MkDocs dev server:

mkdocs serve

In another terminal, run heroshot:

heroshot

(Or npx heroshot / docker run --rm -v $(pwd):/work heroshot/heroshot depending on install method)

A browser opens with a visual picker. Navigate to localhost:8000, click on elements you want to capture, name them. Close when done.

You'll get two files per screenshot:

dashboard-light.png
dashboard-dark.png

Step 5: Add the macro

Update your mkdocs.yml:

plugins:
  - macros:
      modules: [heroshot]

Use it in your markdown:

{{ heroshot("dashboard", "Dashboard overview") }}

The macro expands to Material's #only-light / #only-dark syntax. When readers toggle the theme, screenshots swap automatically.

Step 6: Keep them fresh

When your UI changes:

heroshot

All configured screenshots regenerate. No manual cropping, no file hunting.

Bonus: Screenshots without theme variants

For diagrams or architecture images that don't need dark mode:

{{ heroshot_single("architecture", "System architecture") }}

Renders a simple <img> tag.

Full docs: heroshot.sh/docs/integrations/mkdocs

Example repo: github.com/omachala/heroshot/tree/main/integrations/examples/mkdocs