How Tilt's Live Update Fixed My Kubernetes Dev Loop and Gave Me an Hour a Day Back

#devops #automation #kubernetes #docker

Change one line of Python. docker build. docker push. kubectl rollout restart. Wait for the pod. Check the logs. Repeat, forty times a day. I actually did the arithmetic on what that ritual was costing me one afternoon, and the number was ugly enough that I nearly gave up developing on Kubernetes entirely. Instead I found Tilt's Live Update, rebuilt my Kubernetes dev loop around it, and got roughly an hour a day back. This is how that went.

Why this bugged me for weeks

My team had just moved local development onto a k3d cluster for parity with prod. Good call in principle - and in practice my day looked like this for every single code change to our FastAPI service myapp:

docker build -t k3d-registry.localhost:5000/myapp .
docker push k3d-registry.localhost:5000/myapp
kubectl rollout restart deploy/myapp -n myapp
kubectl logs -f deploy/myapp -n myapp   # wait, watch, hope

A minute or two, minimum, per edit. Ten edits and half an hour of my life had dissolved into progress bars. But the raw time wasn't even the worst of it - the context switching was. By the time the pod finally came up, I'd forgotten what I was checking in the first place. Kubernetes had taken my tight, genuinely joyful edit-run loop and turned it into molasses, and I could feel my patience for the whole platform draining by the day.

I was one bad afternoon away from ripping the app back out of the cluster - and throwing away all the parity we'd just bought - when I came across a write-up on using Tilt for a fast local Kubernetes dev loop. It described the exact pain I was drowning in, and it gave me a way out.

The three-line Tiltfile that started it

Tilt's config is a file called Tiltfile (no extension) written in Starlark, which is basically a trimmed-down Python. Run tilt up and it executes top to bottom, builds a graph of what to build and deploy, then watches your files and rebuilds only what changed. My first working version was almost embarrassingly short:

# Tiltfile

# How to build the myapp image from the current directory
docker_build('k3d-registry.localhost:5000/myapp', '.')

# What to deploy — our existing manifests
k8s_yaml(['k8s/deployment.yaml', 'k8s/service.yaml'])

# Fine-tuning: forward the container's port 8080 to localhost:8080
k8s_resource('myapp', port_forwards='8080:8080')

Three functions carry almost any Tiltfile, all documented in Tilt's API reference: docker_build for how to build, k8s_yaml for what to deploy, and k8s_resource for the fine-tuning like port-forwards and grouping. The clever bit is how Tilt ties them together - it scans the YAML, finds the workload, matches the built image to the manifest by tag, swaps in a uniquely-tagged fresh build, and deploys. The one rule I had to internalize immediately: the tag in docker_build must exactly match the image: in deployment.yaml. Mismatch it and Tilt cheerfully builds one image while the Deployment asks for another - instant ImagePullBackOff, and a confusing ten minutes before you realize what you did.

After tilt up, my service was live at localhost:8080 with no manual kubectl port-forward. Already better. But the real prize came next.

Live Update: the feature that gave me my day back

This is the part that actually changed things. The normal rebuild-image-then-redeploy-pod cycle takes tens of seconds even for a tiny service - Docker layers, registry push, cluster pull, pod restart, all of it. Live Update skips the whole dance: instead of rebuilding, Tilt copies changed files directly into the running container and, if needed, runs commands there. Seconds, not minutes.

You configure it inside docker_build via live_update, and the steps run in a strict order:

docker_build(
    'k3d-registry.localhost:5000/myapp',
    '.',
    live_update=[
        # if deps change, just rebuild the whole image
        fall_back_on(['requirements.txt']),
        # sync code instantly into /code/app (the WORKDIR from our Dockerfile)
        sync('./app', '/code/app'),
        # reinstall deps only if that file changed
        run('pip install -r requirements.txt', trigger=['requirements.txt']),
    ],
)

Tilt's decision tree is refreshingly simple, and the Live Update reference spells out the exact ordering: a fall_back_on file changed means a full rebuild; a file matched by a sync gets a fast live update; a file in the context but covered by no sync triggers a full docker build; an untracked file does nothing. Two rules tripped me up until I read them twice. sync paths must live inside the build context - if Tilt is watching it, you can sync it - and run() cannot come before sync(), because you have to put the files in place before you act on them. Also worth knowing: the very first deploy is always a full one, since Live Update needs an already-running container to copy into.

Hot reload closes the loop

Live Update drops the new file into the container, but how does the running process actually pick it up? For us, delightfully, FastAPI via uvicorn hot-reloads on its own. The launch command:

uvicorn app.main:app --host 0.0.0.0 --port 8080 --reload --reload-dir app

Per uvicorn's settings docs, --reload runs an internal watcher that restarts the process when .py files change, and --reload-dir app narrows the watch so it doesn't twitch on every temp file. So when your framework self-reloads, sync alone is enough: files land in the container, uvicorn's watcher notices, the app reloads, and there's nothing else to wire up.

If your stack can't hot-reload - a Go binary, or uvicorn without --reload - the synced files just sit there while the old process runs stale code, which is the classic "my changes didn't apply" trap that'll cost you an afternoon of confusion. For that case, Tilt's restart_process extension gives you docker_build_with_restart, which re-runs your entrypoint after each sync. For our FastAPI service plain --reload was simpler, and since it's dev-only, we never ship --reload in the production image.

The dashboard I didn't know I needed

tilt up also brings up a web UI at localhost:10350, and it quietly ended my tab-juggling. On one screen I get every resource with two statuses - did the build and deploy succeed, and what is the pod doing right now - along with the Pod ID behind a copy button, endpoints as clickable links, and the part I use constantly: filterable logs, by build versus runtime, by level, by keyword. When a pod won't start, I stopped grep-ing through kubectl logs across three terminals and just filtered the dashboard. There's a Trigger Update button for manual rebuilds, and a manual mode if you'd rather save often and deploy on demand:

trigger_mode(TRIGGER_MODE_MANUAL)
k8s_resource('myapp', trigger_mode=TRIGGER_MODE_AUTO)  # but keep this one automatic

Is Tilt the only option? My honest take

Tilt isn't alone here - it usually gets compared with Skaffold, from Google, and DevSpace. All three build, deploy, and sync for a fast loop, and the real differences come down to UI-first versus CLI-first and Starlark versus YAML. Tilt bets on the visual dashboard and Live Update; the Starlark config is flexible but a steeper climb than YAML, and it shines for beginners and mixed-experience teams precisely because you can see the whole cluster state. Skaffold is CLI-only with YAML config, file sync, profiles, and a dedicated skaffold debug - familiar and declarative, but no visual panel. DevSpace is a YAML CLI with two-way sync, reverse port-forward, and dev containers.

If your team lives in YAML and loves the terminal, Skaffold or DevSpace is a great fit, and I won't pretend that choice is anything but subjective. For getting a newcomer developing comfortably on Kubernetes, though, the dashboard won it for me - less blind fumbling in the terminal, more actually understanding what's happening.

How it feels now

The per-edit time went from a minute or two down to a second or two, and the list of things I run by hand collapsed from build-push-restart-logs to simply saving the file. The image only rebuilds when dependencies change now, not on every keystroke. Logs and status live on one dashboard instead of scattered across N terminals. The port-forward is declared once in the Tiltfile instead of manually re-established every time a pod restarts. And my focus, which used to get shredded on every deploy, stays intact.

The arithmetic that sold me was blunt: forty edits times roughly ninety seconds saved is about an hour a day I got back, and that's before you count the context-switching tax, which I suspect was the bigger number anyway. The whole setup took a single afternoon.

What lingers, though, isn't the hour. It's that the pain was never Kubernetes - it was my loop. A tiny Tiltfile, Live Update, and uvicorn's --reload turned the whole experience from molasses back into the tight edit-run rhythm I'd been quietly grieving, and I got to keep cluster parity while doing it. If you're still rebuilding images by hand and watching progress bars, that's the trade I'd make again in a heartbeat: an afternoon of setup for a loop that finally respects your attention.