The moment our EKS cluster stopped sweating
The Problem: When Your Container Image Is Bigger Than Your Application
It started with a simple observation during a deployment review. Our team was staring at deployment logs, watching the same 47-second pattern repeat:
text
Pulling image: 45.3s
Starting container: 1.2s
Forty-five seconds of pull time. Every. Single. Deployment.
Our Docker image had ballooned to 1.2GB โ and honestly, we had no one to blame but ourselves. We were shipping node_modules like they were going out of style, including every dev dependency, build tool, and a full test suite in our production images.
The worst part? This wasn't just a "wait a bit longer" problem. It was a:
๐จ Security issue โ larger attack surface with unnecessary packages
๐ธ Cost problem โ more storage in ECR and slower scaling events
๐ค Developer experience killer โ impatient engineers waiting for deployments
The Goal: Fast Pulls, Lean Images
We set three clear objectives:
Image size under 200MB
ECR pull time under 10 seconds in EKS
Zero compromises on runtime performance or security
Here's exactly how we got there.
Step 1: Switch to pnpm (Game Changer)
We were using npm, and our node_modules folder was thicc. Around 800MB of dependencies, including 400MB of dev dependencies we didn't need in production.
We switched to pnpm for three killer benefits:
๐ฅ Hard Links = Disk Space Savings
pnpm stores packages globally and hard-links them into projects. Our node_modules went from 800MB to 180MB in the build stage.
๐ Faster Installation
No more re-downloading packages on every build. pnpm's content-addressable storage means if a package version exists, it's reused.
โก๏ธ Native Workspace Support
Our monorepo setup became significantly simpler.
Here's what our Dockerfile looked like before:
dockerfile
FROM node:18-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --production=false # ๐ฌ brings ALL dependencies
COPY . .
RUN npm run build
And after:
dockerfile
FROM node:18-alpine AS builder
WORKDIR /app
COPY package.json pnpm-lock.yaml ./
RUN npm install -g pnpm && pnpm install --frozen-lockfile
COPY . .
RUN pnpm run build
RUN pnpm prune --prod # ๐ฏ strip dev dependencies
Step 2: Multi-Stage Builds Done Right
We were already using multi-stage builds, but poorly. Our final stage was copying everything from the builder stage โ including source code, tests, and build artifacts we didn't need.
Before: Copying Too Much
dockerfile
FROM node:18-alpine AS runner
WORKDIR /app
COPY --from=builder /app . # ๐จ copies EVERYTHING
CMD ["node", "dist/main.js"]
After: Selective Copying
dockerfile
FROM node:18-alpine AS runner
WORKDIR /app
ENV NODE_ENV=production
Copy only what we need
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/package.json ./
RUN addgroup --system --gid 1001 nodejs && \
adduser --system --uid 1001 nodejs
USER nodejs
CMD ["node", "dist/main.js"]
This single change cut our image size by 60%.
Step 3: Alpine Base Image + Build Optimizations
We were already using Alpine, but we weren't taking full advantage of it. We added:
๐ฆ .dockerignore that actually works
text
node_modules
.git
.md
.env
.DS_Store
coverage
.nyc_output
logs
tmp
๐งน Layer Cleanup
When installing native dependencies, we made sure to clean apt caches:
dockerfile
RUN apk add --no-cache --virtual .build-deps python3 make g++ && \
pnpm install --frozen-lockfile && \
apk del .build-deps
๐๏ธ Build-time optimizations
dockerfile
RUN pnpm run build && \
pnpm prune --prod && \
rm -rf /root/.npm /root/.cache
Step 4: ECR Lifecycle Policies + EKS Pull Optimization
We also optimized the infrastructure side:
ECR Lifecycle Rules
We set up automatic cleanup for untagged and old images:
Keep last 30 images
Expire untagged images after 14 days
Tag important images with prod-* to bypass cleanup
EKS Node Optimization
Enabled container image caching on our nodes
Configured containerd with aggressive image GC settings
Pre-pulled base images using a DaemonSet
Here's the containerd config we use:
toml
[plugins."io.containerd.gc.v1.scheduler"]
pause_threshold = 0.02
deletion_threshold = 0
mutation_threshold = 100
schedule_delay = "0s"
startup_delay = "100ms"
The Results: Numbers That Made Us Smile
Metric Before After Improvement
Image Size 1.2 GB 118 MB 90.1% reduction
ECR Pull Time 45.3s 7.8s 82.8% faster
Deployment Time 2.1 min 28s 78% reduction
Monthly ECR Cost $47 $9 81% savings
Build Time 4.2 min 1.8 min 57% faster
Before and After: The Docker Image Breakdown
Before (1.2GB)
text
๐ฆ node_modules/ 850MB # dev + prod deps
๐ src/ 12MB
๐ tests/ 28MB # ๐ฑ tests in production image!
๐ coverage/ 15MB # seriously? yes
๐ .git/ 250MB # ๐คฆโโ๏ธ
๐ other files 45MB
After (118MB)
text
๐ฆ node_modules/ 85MB # prod deps only (pnpm)
๐ dist/ 15MB # built application
๐ package.json 2KB
๐ other files 16MB # minimal essentials
Key Lessons Learned
pnpm isn't just a package manager โ it's a strategy
The hard-linking approach fundamentally changes how you think about dependency management in containers.Copy selectively, live happily
COPY --from=builder /app . is the enemy. Be explicit about what you're copying.Test environments stay out of production images
Your tests don't need to ship. Neither does your test framework, linter, or build tools.Layer ordering matters
Put frequently changing files (source code) at the bottom, infrequently changing files (dependencies) at the top.Measure everything
We couldn't have optimized without metrics. Use docker images, docker history, and tools like Dive to analyze your image layers.
Try This Yourself
Here's a template you can adapt:
dockerfile
Stage 1: Builder
FROM node:18-alpine AS builder
WORKDIR /app
RUN npm install -g pnpm
COPY package.json pnpm-lock.yaml ./
RUN pnpm install --frozen-lockfile
COPY . .
RUN pnpm run build && pnpm prune --prod
Stage 2: Runner
FROM node:18-alpine
WORKDIR /app
ENV NODE_ENV=production
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/package.json ./
RUN addgroup -g 1001 nodejs && adduser -u 1001 -G nodejs nodejs
USER nodejs
EXPOSE 3000
CMD ["node", "dist/main.js"]
What's Next?
We're not stopping here. Our next targets:
~50MB with distroless images
~3s ECR pull with image pre-fetching
15s deployment times with rolling updates
Full SBOM (Software Bill of Materials) scanning in CI/CD
Final Thoughts
Going from 1.2GB to 118MB wasn't magic โ it was discipline. Every byte we removed improved our security, speed, and developer experience.
The best part? We spent about 4 hours total on these optimizations. That's 4 hours to save our team hours every single week.
If you're sitting on a bloated Docker image, start small:
Check what's in your image (docker history)
Move to pnpm
Audit every COPY command
Use Alpine as your base
Measure your improvements
Your EKS cluster will thank you. Your wallet will thank you. And most importantly, your team will thank you when deployments fly.
Have you optimized your container images? What's the craziest thing you've found in a production image? Share your war stories in the comments! ๐
Follow me for more DevOps deep dives ๐
Top comments (0)