The Most Expensive Docker Problem Usually Isn't Docker
A few years ago, I was helping investigate a CI pipeline that had gradually become one of the biggest complaints inside an engineering organization.
Developers were waiting 10–15 minutes for builds.
Pull requests piled up.
Deployments slowed down.
Teams started discussing larger build agents, more CPU, more memory, and more parallel runners.
The assumption was simple:
Docker had become slow.
After spending an afternoon looking through the build logs, the real problem became obvious.
Docker wasn't slow.
The Dockerfile was.
The application itself hadn't changed much over the years, but the container image had quietly accumulated everything imaginable:
- Build tools
- Compilers
- Development dependencies
- Package managers
- Source code
- Debugging utilities
- Test frameworks
The image had grown beyond a gigabyte.
Every build downloaded and rebuilt far more than production actually needed.
Once we cleaned up the Dockerfile, build times dropped dramatically and image sizes shrank by more than 90%.
No new hardware.
No expensive CI upgrades.
Just better container engineering.
This article explores three concepts that consistently deliver the biggest improvements:
- Understanding why Docker builds become slow
- Using multi-stage builds correctly
- Building ultra-small containers using static binaries and scratch images
These techniques improve more than build speed.
They reduce attack surface, lower infrastructure costs, accelerate deployments, and make supply chain security easier to manage.
Containers Were Never Meant To Be Tiny Virtual Machines
One misconception still appears surprisingly often.
People treat containers like lightweight servers.
They install operating system packages.
They add debugging tools.
They leave package managers inside production images.
They bundle build systems with runtime applications.
Eventually the image becomes a miniature Linux distribution carrying far more software than the application actually needs.
Containers work best when they have a single responsibility:
Run the application and nothing else.
Every extra component increases:
- Image size
- Build time
- Pull time
- Vulnerability count
- Maintenance burden
Optimizing containers starts with recognizing that production only needs a fraction of what was required during development.
The Silent Cost of Large Images
Most teams notice slow builds.
Fewer notice the ripple effects.
When an image grows from 80 MB to 1.2 GB, the impact appears everywhere.
The CI system spends more time pushing artifacts.
Nodes spend more time pulling images.
Autoscaling takes longer.
Disaster recovery takes longer.
Cross-region deployments consume more bandwidth.
Security scanners have more software to inspect.
SBOM generation becomes larger and more complex.
What starts as "just a bigger image" eventually affects nearly every stage of software delivery.
Stop Shipping Your Build Environment To Production
One of the biggest container anti-patterns is deploying the same image that was used to build the application.
The build environment contains tools needed during compilation.
Production rarely needs any of them.
Consider a typical Node.js application.
During development, we may need:
- npm
- TypeScript compiler
- ESLint
- Testing frameworks
- Build tooling
- Source code
Once the application is compiled, production generally needs only:
- Runtime
- Application artifacts
- Runtime dependencies
Everything else becomes baggage.
This is exactly the problem multi-stage builds solve.
The Build → Test → Runtime Pattern
Instead of creating one giant image, multi-stage builds separate responsibilities.
A typical flow looks like this:
Build
↓
Test
↓
Runtime
Every stage serves a purpose.
The final image receives only the artifacts it needs.
Everything else gets discarded.
Think of it like manufacturing.
You don't ship the factory to the customer.
You ship the finished product.
Containers should work the same way.
A Typical Single-Stage Dockerfile
Many projects start with something like this:
FROM node:22
WORKDIR /app
COPY . .
RUN npm install
RUN npm run build
CMD ["npm","start"]
At first glance this looks fine.
The application builds.
The container runs.
Nothing appears wrong.
The issue is that production now contains everything required to perform the build.
That includes tools that production never actually uses.
Those tools still consume:
- Storage
- Network bandwidth
- Security review effort
- Vulnerability scanning time
The image becomes larger than necessary.
A Better Approach
Multi-stage builds separate concerns.
FROM node:22 AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build
FROM node:22-slim
WORKDIR /app
COPY package*.json ./
RUN npm ci --omit=dev
COPY --from=builder /app/dist ./dist
CMD ["node","dist/server.js"]
The final image contains:
- Runtime dependencies
- Compiled application
Nothing else.
This simple pattern often produces dramatic reductions in image size.
Security Benefits That Nobody Talks About Enough
Most discussions around multi-stage builds focus on performance.
The security gains are equally important.
Every package inside a container is another component that can contain vulnerabilities.
Compilers.
Package managers.
Build systems.
Developer utilities.
All increase attack surface.
Removing unnecessary software follows one of the oldest security principles:
If you don't need it, don't install it.
The fewer components present in production, the fewer opportunities an attacker has to abuse them.
This is security by reduction.
And it works remarkably well.
Chasing Tiny Images
Once teams start removing unnecessary components, a natural question appears.
How small can a container actually become?
The answer is:
Much smaller than most people expect.
Especially for languages that can produce static binaries.
The Power of Static Binaries
Languages like:
- Go
- Rust
- Statically linked C
can often produce self-contained executables.
The application carries everything it needs inside a single binary.
No runtime installation.
No package manager.
No interpreter.
No dependency downloads.
Just the executable.
At that point, an entire operating system becomes unnecessary.
When Even Alpine Is Too Large
Many engineers stop at Alpine Linux.
Alpine is excellent.
But for static binaries, we can go further.
Docker provides a special base image:
FROM scratch
Scratch isn't really a Linux distribution.
It's essentially empty.
No shell.
No package manager.
No utilities.
No operating system tooling.
Only whatever files you copy into it.
Building A Scratch Image
A Go application is a classic example.
FROM golang:1.22 AS builder
WORKDIR /app
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -o app .
FROM scratch
COPY --from=builder /app/app /app
ENTRYPOINT ["/app"]
The resulting image contains almost nothing beyond the application itself.
The difference can be astonishing.
Images measured in hundreds of megabytes can become images measured in single-digit megabytes.
In some cases, only a few hundred kilobytes.
Tiny Images Aren't Always Better
One lesson learned from production systems:
Smaller is not automatically better.
Scratch images introduce operational challenges.
Common surprises include:
- Missing CA certificates
- Missing timezone data
- No shell access
- More difficult debugging
Teams often discover these limitations during incidents.
The right question isn't:
How small can we make this image?
The better question is:
What is the smallest image that remains practical to operate?
Engineering is always about trade-offs.
Containers Should Carry Intent
The best Dockerfiles reveal how a team thinks about software delivery.
Large images often indicate years of accumulated convenience.
Small, focused images usually indicate intentional engineering decisions.
Multi-stage builds force us to separate development concerns from production concerns.
Scratch images force us to understand exactly what our application requires.
Both practices lead to a valuable outcome:
Containers that are faster to build, faster to deploy, cheaper to operate, and harder to attack.
And in modern platform engineering, those goals are usually aligned far more often than people realize.
Top comments (0)