How I Cut Python Docker Image Size From 1.2 GB to a Lean, Secure Container

#python #docker #kubernetes #devops

When we moved our tracking API to Kubernetes, the container turned out to be the weakest link in the whole thing. It was a 1.2 GB image that took two full minutes to rebuild on every code change and then, adding insult to injury, flat-out refused to run in our local k3d cluster. If you've ever wanted to reduce a Python Docker image size and wondered why your "working" image won't load into a cluster, this is the afternoon where I fixed all three problems at once - and stopped being quietly embarrassed by my own Dockerfile.

Why this bugged me for weeks

Our service, myapp, is a Python 3.12 and FastAPI HTTP API on port 8080 that talks to PostgreSQL. The first Dockerfile I ever wrote for it is the first Dockerfile anyone writes, and it technically works:

FROM python:3.12
WORKDIR /code
COPY . .
RUN pip install -r requirements.txt
CMD python -m uvicorn app.main:app --host 0.0.0.0 --port 8080

Here's the uncomfortable truth: every single line is a small mistake. The full-fat python:3.12 base is enormous. COPY . . before pip install means a one-character code edit invalidates the dependency layer and reinstalls the world. The shell-form CMD quietly breaks signal handling. And it runs as root. I lived with all of this for longer than I'd like to admit, treating the slow rebuilds as just the cost of doing business, until I got fed up and went hunting for a canonical, production-minded reference. Someone had written up a walkthrough on writing a small, fast, secure Dockerfile for a FastAPI service and loading it into k3d, and honestly it restructured how I think about every instruction in the file.

The cheapest 90% win: fix your layer order

Each Dockerfile instruction is a layer, Docker caches them, and - this is the part that matters - if a layer changes, every layer after it is invalidated too. I was editing application code dozens of times a day and almost never touching dependencies, yet my ordering forced a full pip install on every build. The fix is embarrassingly simple: install what rarely changes before copying what changes constantly.

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt   # heavy, rarely changes
COPY ./app ./app                                      # light, changes often

That one reorder dropped my rebuild from around two minutes to a few seconds on a code-only change, because the dependency layer now comes straight from cache. It's the single most common Dockerfile anti-pattern in existence, and I'd been cheerfully living inside it for months.

The multi-stage build

The naive image ships everything into the final result - compilers, dev headers, pip caches - all of it dead weight at runtime. A multi-stage build installs dependencies in a build stage and copies only the finished virtualenv into a clean runtime image:

# --- Stage 1: build ---
FROM python:3.12-slim AS build
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1 PIP_NO_CACHE_DIR=1
WORKDIR /code
RUN python -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# --- Stage 2: runtime ---
FROM python:3.12-slim
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1 PATH="/opt/venv/bin:$PATH"
WORKDIR /code
COPY --from=build /opt/venv /opt/venv
COPY ./app ./app
EXPOSE 8080
CMD ["fastapi", "run", "app/main.py", "--port", "8080"]

How much you save depends entirely on your dependencies - with pure wheels the gain is modest, but we had a couple of packages that compiled from source, and dropping the build toolchain shrank the image substantially. Switching from python:3.12 to python:3.12-slim did the rest.

The small details that make it production-grade

The write-up drilled a handful of these in, and every one of them has bitten someone I know. Use fastapi run, not a bare uvicorn --reload - the reload flag is dev-only overhead in a cluster image, and FastAPI's own container guide uses fastapi run, which starts Uvicorn with sane production settings. Write CMD in exec form, as an array, so the app becomes PID 1 and receives SIGTERM from Kubernetes directly and shuts down gracefully; shell form wraps it in /bin/sh, the signal never arrives, and the Pod gets hard-killed on timeout. Set PYTHONUNBUFFERED=1, or your logs get stuck in a buffer and never surface in kubectl logs - I lost an hour to "why is my container silent" before I understood that one. Add an unprivileged user with adduser --disabled-password --uid 10001 appuser and then USER appuser, because root-by-default violates least privilege and it pairs later with securityContext.runAsNonRoot: true in the manifest. And add a .dockerignore, because without one I was shipping .git, a local .venv, and nearly a .env straight into the build context.

The bug that stole a full day: ImagePullBackOff

With a beautiful new image built, I ran docker build, applied the manifest, and the Pod sat in ImagePullBackOff forever. It seems completely obvious that a freshly built local image would be visible to the cluster. It is not, and this cost me a full day of my life. k3d nodes run their own containerd, isolated from your Docker daemon - an image sitting in Docker is invisible to the cluster, and the kubelet just keeps failing to pull it from a remote registry that doesn't have it.

The fastest fix for a one-off test is a direct import:

docker build -t myapp:dev .
k3d image import myapp:dev -c dev

Pair that with imagePullPolicy: IfNotPresent in the manifest so Kubernetes doesn't reach for the network anyway. For ongoing work we moved to k3d's built-in registry, where the same name k3d-registry.localhost:5000/myapp:dev works for both push from the host and pull from inside the cluster. The full delivery model - both paths, plus why *.localhost resolves - is covered in the companion write-up linked in Sources below.

Two more touches that paid off

First, a HEALTHCHECK that doesn't need curl. I wanted Docker to know whether the service was alive during local runs, but python:3.12-slim has no curl, and neither does distroless. So I did the check with Python, which is guaranteed to be in the image:

HEALTHCHECK --interval=30s --timeout=10s --start-period=60s --retries=3 \
  CMD ["python", "-c", "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://localhost:8080/healthz').getcode()==200 else 1)"]

Second, actually understanding what EXPOSE does and doesn't do. For an embarrassingly long time I thought EXPOSE 8080 published the port. It doesn't - the Dockerfile reference is explicit that EXPOSE doesn't actually publish anything; it's pure metadata, documentation for whoever reads the file. Real publishing happens with -p in docker run, or with Service and Ingress objects in Kubernetes. Internalizing that killed a whole afternoon of "why can't I reach the port" confusion, purely because I stopped expecting EXPOSE to do a job it was never designed for.

I also standardized on running a single worker per Pod and scaling with replicas, rather than cramming Gunicorn plus multiple Uvicorn workers into one container. In Kubernetes the cluster is the process manager, and letting it own concurrency kept my image simpler and my resource limits meaningful. For local development it's not even a question - one process is plenty.

How it feels now

The difference is night and day, and it's satisfying in a way that's hard to overstate. The base went from the full python:3.12 to a multi-stage python:3.12-slim. A rebuild on a code change went from roughly two minutes of full pip install to a few seconds served from cache. The container runs as uid 10001 appuser instead of root. Signal handling went from broken - the shell-form CMD swallowing SIGTERM - to correct, with the app as PID 1 receiving signals directly. And the image, which used to greet me with ImagePullBackOff every time, now imports and pushes into k3d cleanly.

For local dev I deliberately stayed on slim, because it's genuinely easy to debug. When I hardened the image for production later, I moved the runtime to distroless (gcr.io/distroless/python3-debian12) - no shell, no package manager, far fewer CVEs - building the deps on slim and copying them across. The trade-off is real: docker exec ... sh no longer works, so you debug with kubectl debug and ephemeral containers instead. That's a fair price for the reduced attack surface, but it's a choice worth making consciously rather than by accident.

What stuck with me most is how tangled up all of this had felt when it was really one thing. I'd been treating "make it smaller," "make it faster," "make it secure," and "make it actually run in the cluster" as four separate chores I'd get to someday. They were never four projects. They were one Dockerfile, done with a little more care than the copy-paste version I'd been shrugging past for months - and the version of me who kept restarting slow builds would not believe how good a well-ordered Dockerfile feels to live with.