tosane932

Posted on Jul 28 • Edited on Aug 2 • Originally published at qiita.com

My Multi-Stage Docker Build Reduced the Image by Only 3 MB

#docker #flask #python #beginners

This article was originally published in Japanese on Qiita and has been translated and adapted for DEV Community.

🍞 What I Learned by Rewriting the Dockerfile for sales_data_app and Measuring the Before and After Results

Introduction

Hello from Japan! 🇯🇵

I am tosane932, a professional truck driver with more than seven years of field experience who is teaching himself Python.

At the time of writing, I have completed approximately 125 hours of programming study.

In previous articles, I have written about topics such as:

“Loading mistakes” in my Flask code
Defensive programming and truck stopping distance
Dockerization
Production deployment

This time, I rewrote the Dockerfile for my application, sales_data_app, and measured whether introducing a multi-stage build actually reduced the Docker image size.

I will reveal the result first:

The reduction was much smaller than I expected.

However, the process of investigating why the image barely became smaller turned out to be more valuable than the size reduction itself.

🐋 The Original Dockerfile

sales_data_app is built with:

Flask
PostgreSQL
Gemini API

The original Dockerfile looked like this:

FROM python:3.12-slim

WORKDIR /app

COPY requirements.txt .

RUN pip install --no-cache-dir -r requirements.txt

COPY . .

EXPOSE 5000

CMD ["python", "app.py"]

This is a simple Dockerfile.

However, depending on the dependencies, build tools such as compilers may remain in the final production image.

In logistics terms, I imagined it like this:

The heavy toolboxes and packaging waste from the warehouse are still loaded onto the truck that will drive on the production highway.

The tools needed to prepare the cargo should remain at the packing facility.

They should not necessarily travel with the final shipment.

📦 Multi-Stage Builds as Cargo Transfer

To separate the build environment from the production environment, I introduced a multi-stage build.

# Stage 1: Packing area
FROM python:3.12-slim AS builder

WORKDIR /app

RUN apt-get update && apt-get install -y --no-install-recommends \
    gcc \
    libpq-dev \
    && rm -rf /var/lib/apt/lists/*

COPY requirements.txt .

RUN pip install --no-cache-dir --user -r requirements.txt


# Stage 2: Production truck
FROM python:3.12-slim

WORKDIR /app

RUN apt-get update && apt-get install -y --no-install-recommends \
    libpq5 \
    && rm -rf /var/lib/apt/lists/*

COPY --from=builder /root/.local /root/.local

COPY . .

ENV PATH=/root/.local/bin:$PATH

EXPOSE 5000

CMD ["python", "app.py"]

The basic idea is simple.

Stage 1: Packing Area

The builder stage contains the tools required to build Python packages:

gcc
libpq-dev

The dependencies are installed there.

Stage 2: Production Truck

The final stage receives only the completed Python packages from:

/root/.local

The compiler and development headers are not copied into the production image.

In theory, this means the production image carries only what is required to run the application.

📊 Measuring the Actual Result

I compared the image sizes using:

docker images

The result was:

Build Method	Content Size
Before: Single-stage build	212 MB
After: Multi-stage build	209 MB

The image became smaller by only:

3 MB

Honestly, part of me wanted to write an article titled:

I Dramatically Reduced My Docker Image Size!

But I have decided to report my development process based on facts.

The measured reduction was only 3 MB.

❓ Why Did the Image Size Barely Change?

After investigating the dependencies, I found an important clue inside requirements.txt.

sales_data_app uses:

psycopg2-binary

As the name suggests, psycopg2-binary is distributed as a precompiled binary package, usually in wheel format.

That means Python can often download and install the already compiled files without compiling the package from source.

In other words, the installation may not require tools such as:

gcc
PostgreSQL development headers
A local compilation process

In this project, preparing the “heavy toolbox” in the builder stage may not have been particularly useful in the first place.

The builder tools were installed, but the main PostgreSQL dependency was already available as a compiled wheel.

A multi-stage build would likely show a clearer effect when using dependencies that must be compiled from source.

For example:

psycopg2

instead of:

psycopg2-binary

may require a local compiler and PostgreSQL development files during installation.

In that type of project, removing the build tools from the final stage could produce a more noticeable reduction.

This test taught me that:

A multi-stage build is not a magical technique that always produces a much smaller image.

Its effectiveness depends on the nature of the dependencies.

That was something I could not fully understand until I measured the result myself.

Multi-Stage Builds Solve More Than One Problem

Even though the size reduction was small, the multi-stage structure still has value.

It creates a clear separation between:

Tools needed to build the application
Files needed to run the application

This can improve:

Image structure
Security
Maintainability
Dependency control
Reproducibility

The final image does not need to contain every tool used during the build process.

So even when the image-size reduction is small, the architecture may still be cleaner.

However, the result should not be exaggerated.

In this particular application, the measurable size benefit was limited.

🔐 An Unexpected Benefit: Reviewing How I Handle API Keys

After building the new image, I started the container.

The Gemini API key was not detected.

The following error appeared:

[ERROR] GEMINI_API_KEY is missing from environment variables.

The cause was simple.

The .env file on the host machine is not automatically passed into a Docker container.

I started the container again using:

docker run \
    -p 5000:5000 \
    --env-file .env \
    sales-after

This solved the problem.

However, the incident reminded me of an important principle:

The Docker image and secret credentials should be treated as separate things.

A Docker image may be:

Shared with another developer
Uploaded to a registry
Used in CI/CD
Deployed to a hosting platform
Distributed across multiple environments

If an API key is embedded directly into the image, the secret may travel with it.

That creates risks such as:

Giving the API key to anyone who receives the image
Exposing the secret through an image layer
Leaving the key visible in build history
Accidentally publishing the credential to a registry

For example, even if a secret is later removed from a Dockerfile, it may remain in an earlier image layer.

Tools such as:

docker history

can reveal information about how the image was built.

This follows the same principle as excluding .env from Git using .gitignore.

The root idea is identical:

Separate the code and container from the secret key.

The application image should contain the application.

The environment should provide the credentials at runtime.

Code and Secrets Have Different Lifecycles

The Docker image may be rebuilt many times.

The API key may also be:

Rotated
Revoked
Replaced
Different between development and production
Different between users

If the key is embedded in the image, every credential change requires rebuilding the entire image.

If it is provided through environment variables, the credential can be changed independently.

This separation creates a cleaner operational model:

Docker image
=
Application code and runtime dependencies

Environment variables
=
Deployment-specific configuration and secrets

This was not the main purpose of the experiment, but it became one of the most important lessons.

Before and After

Before

FROM python:3.12-slim

WORKDIR /app

COPY requirements.txt .

RUN pip install --no-cache-dir -r requirements.txt

COPY . .

EXPOSE 5000

CMD ["python", "app.py"]

After

FROM python:3.12-slim AS builder

WORKDIR /app

RUN apt-get update && apt-get install -y --no-install-recommends \
    gcc \
    libpq-dev \
    && rm -rf /var/lib/apt/lists/*

COPY requirements.txt .

RUN pip install --no-cache-dir --user -r requirements.txt


FROM python:3.12-slim

WORKDIR /app

RUN apt-get update && apt-get install -y --no-install-recommends \
    libpq5 \
    && rm -rf /var/lib/apt/lists/*

COPY --from=builder /root/.local /root/.local

COPY . .

ENV PATH=/root/.local/bin:$PATH

EXPOSE 5000

CMD ["python", "app.py"]

Measured Result

Before: 212 MB
After:  209 MB
Difference: 3 MB

The structure changed significantly.

The image size did not.

That difference between structural improvement and measurable size reduction was the main result of this experiment.

What I Would Investigate Next

This test raised several additional questions.

1. Are the Build Tools Necessary at All?

Because psycopg2-binary is precompiled, I may be able to remove:

gcc
libpq-dev

from the builder stage entirely.

If the other dependencies also provide wheels, the builder stage may not need a compiler.

2. Which Layers Are Actually Large?

The following command can help inspect the image layers:

docker history sales-after

This may reveal whether the largest parts come from:

The Python base image
Installed dependencies
Application files
Operating-system packages

3. Could a Dependency Audit Produce a Larger Reduction?

Removing one unused large Python package may have a greater effect than changing the Dockerfile structure.

The next useful step may be to inspect:

requirements.txt

and confirm whether every dependency is actually imported and required.

4. Would Another Base Image Be Smaller?

A different base image might reduce the size further.

However, a smaller base image can also introduce:

Build complexity
Compatibility problems
Missing system libraries
More difficult troubleshooting

The smallest possible image is not automatically the best image.

The correct choice depends on the application's needs.

Summary

I introduced a multi-stage build into sales_data_app
The image size decreased from 212 MB to 209 MB
The measured reduction was only 3 MB
One likely reason is that psycopg2-binary is already distributed as a compiled wheel
Multi-stage builds are more effective when dependencies require source compilation
The technique is not guaranteed to dramatically reduce every image
The experiment also reminded me to keep API keys separate from Docker images
Secrets should be supplied through environment variables at runtime
Structural improvements and image-size improvements are not always the same thing

The most important lesson was not:

Multi-stage builds make images smaller.

It was:

Measure first, then investigate why the result occurred.

A technique should not be judged only by what documentation says it can do.

It should be tested against the actual application and dependency structure.

My next article will cover the work I am currently doing with:

pytest
CI/CD
The errors and misunderstandings I encountered while introducing them

This article is based on actual measurements.

Results may vary depending on the operating system, Docker version, base image, CPU architecture, and dependency configuration.

DEV Community