DEV Community: Alexis M.

Docker : Change the build behavior with multi-contexts builds

Alexis M. — Thu, 16 May 2024 06:17:09 +0000

Why do I need to change my build behavior ?

In corporate settings, security tools often intercept and encrypt each request with their own certificates, falling under the category of MitM tools. While these tools can be intrusive during local development, there are solutions to address the issues. However, in Docker, the build environment differs from your local machine, rendering these solutions ineffective, leading to build failures. An elegant solution is required to ensure compatibility between these two tools.

You may have different scenario, but I will use this one to explain how we build our images with this kind of tool.

How do these tools block our Docker builds ?

These tools function as Man-in-the-Middle (MitM) tools. Every request we send is intercepted and encrypted with the tool's certificate. By default, this certificate is not trusted, and it is added to the machine's certificate store during installation.

Docker is unaware of these tools, and it performs requests as it would in a standard environment when building our image. We must modify the behavior for building within a restrictive environment.

💡for NPM, we must define NODE_EXTRA_CA_CERTS environment variable to point to the tool's certificate

💡for NuGet, we just need to add the certificate to the /etc/ssl/certs/ folder

⚠️ all runtimes may have a different way to handle certificate

For this post, I will use NPM as package manager for simplicity.

What's wrong ?

I believe a Dockerfile ought to encompass all the essential steps for constructing our application, rather than just copying a pre-built artifact from our machine. Additionally, it should be environment-agnostic, ensuring that the Dockerfile remains consistent whether the build occurs locally or on a CI system.

So, how should we address this?

Upon the tool's installation, the IT team offered three potential solutions; regrettably, none met our prerequisite (no changes in the Dockerfile).

This is our default Dockerfile for the post

# file: Dockerfile
FROM node:20 as build
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . ./
RUN npm build # output to /app/dist

FROM nginx:alpine
COPY --from=build /app/dist /usr/share/nginx/html

Solution 1 : One Dockerfile for local build, another one for CI build

I won't waste time detailing why it's ill-advised. Duplicating code with the intention of achieving identical results only leads to self-inflicted issues.

Solution 2 : Copy certificates and use ARG/IF to condition the steps

If duplication is not advisable, could we share code ?

# file: Dockerfile
ARG BUILD_ENV=remote
FROM node:20 as build
COPY rootcacert.pem /etc/ssl/certs/securitycert.pem
# ENV NODE_EXTRA_CA_CERTS=/etc/ssl/certs/securitycert.pem
RUN <<EOC
if [ "$BUILD_ENV" = "local" ]; then
  # for NPM, we just need to set an env
  # it's here for the purposes of this post, but it belongs above
  export ENV NODE_EXTRA_CA_CERTS=/etc/ssl/certs/securitycert.pem
  # run specifics command here
  # update the store certificate or other commands required by your tools
fi
EOC
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . ./
RUN npm build # output to ./dist

FROM nginx:alpine
COPY --from=build /app/dist /usr/share/nginx/html
...

Utilizing ARG and bash conditions to determine the necessity of a certificate increases the complexity of our Dockerfile.

Why is this approach problematic?

It complicates the Dockerfile.
The certificate needs to be included in the git repository.
Adding the certificate to the .gitignore file will cause the CI build to fail.

Locally, we must specify that we want to build on a restricted environment

docker build -t repo/image:1.0.0 --build-arg BUILD_ENV=local .

Solution 3 : Use a `compose.yaml` definition

The solution should be divided into two distinct parts:

Build using tools that only require the file to be correctly positioned or accompanied by the appropriate environment variables.
Build using tools that necessitate additional commands for managing certificates.

3.1 : build with tools that require a file or an ENV

With this solution, in the given context, there was no need to update our Dockerfile, which is the good news. We only need to add a compose.yaml file to specify certain configurations.

# file: compose.yaml
version: '3.7'
services:
  application:
    image: repo/image:${TAG}
    build:
      context: .
      dockerfile: Dockerfile
    environment:
      NODE_EXTRA_CA_CERTS: /etc/ssl/certs/securitycert.pem
    volumes:
      - /path/to/your/certificate:/etc/ssl/certs/securitycert.pem

3.2 : build with tools that require specific command

In this case, we need to update our Dockerfile to handle certificate

# file: Dockerfile
ARG BUILD_ENV=remote
ARG CERT_PATH=/etc/ssl/certs/securitycert.pem
FROM node:20 as build
# ENV NODE_EXTRA_CA_CERTS=/etc/ssl/certs/securitycert.pem
RUN <<EOC
if [ "$BUILD_ENV" = "local" ]; then
  # for NPM, we just need to set an env
  # it's here for the purposes of this post, but it belongs above
  export ENV NODE_EXTRA_CA_CERTS=$CERT_PATH
  # run specifics command here
  # update the store certificate or other commands required by your tools
fi
EOC
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . ./
RUN npm build # output to /app/dist

FROM nginx:alpine
COPY --from=build /app/dist /usr/share/nginx/html
...

And we need to define a compose.yaml file to specify some configurations

# file: compose.yaml
version: '3.7'
services:
  application:
    image: repo/image:${TAG}
    build:
      context: .
      dockerfile: Dockerfile
    args:
      BUILD_ENV: local
      CERT_PATH: /etc/ssl/certs/securitycertpem
    volumes:
      - /path/to/your/certificate:/etc/ssl/certs/securitycert.pem

In both cases, to build locally, we need to run the build with compose

TAG=1.0.0 docker compose -f compose.yaml build

On our CI system, we don't change how we build

docker build -t repo/image:1.0.0 .

As mentioned, this solution is suitable for a simple use case; however, for more complex setups, we will need to introduce additional complexity into our Dockerfile.

How multi-contexts build can help ?

We have observed that the three (or four) solutions mentioned are not satisfactory. They either add complexity or fail to work in all scenarios.

If duplication or code sharing don't serve as convincing solutions, perhaps we could isolate the problematic component?

In object-oriented programming, the strategy pattern enables us to alter our code's behavior without modifying the code itself. Why not apply it here?

In a Dockerfile, multiple contexts exist. The most common is the default build context (the '.' at the end of the 'docker build' command), but there are others, and we utilize them each time we write or build an image.

FROM node:20 as build-stage
COPY . ./
...

COPY --from=build-stage . ./

FROM node:20 as build-stage define a context based on node:20 context and named build-stage
COPY . ./ use the default build context to copy files in the current one
COPY --from=build-stage . ./ copy file from a context named build-stage into the current context

How can it help ?

If node:20 serves as a context, it is possible to update this context at build time using the multi-contexts build feature introduced in Docker. If there is a context that contains all the necessary requirements to build our image in a restricted environment, we should be able to replace the node:20 context with it.

To carry out the build in a restrictive context, we must construct a restricted context. We will create a new Dockerfile where we will specify all the requirements and proceed with the build.

# file: Dockerfile.securitycert
FROM node:20
COPY rootcacert.pem /etc/ssl/certs/securitycert.pem
# ENV NODE_EXTRA_CA_CERTS=/etc/ssl/certs/securitycert.pem
RUN <<EOC
# for NPM, we just need to set an env
# it's here for the purposes of this post, but it belongs above
export ENV NODE_EXTRA_CA_CERTS=/etc/ssl/certs/securitycert.pem
# run specifics command here
# update the store certificate or other commands required by your tools
EOC

docker build -t node:20-securitycert -f Dockerfile.securitycert /path/to/your/certificates/folder

Now, we have a restricted context with all requirements built on a new docker image node:20-securitycert.

Without any modifications on our base Dockerfile, we can now build our application container using this image as base.

docker build -t repo/image:1.0.0 --build-context node:20=docker-image://node:20-securitycert .

Docker will update the node:20 base we use with our newly created image at build time.

That sounds good, but it adds an extra step to build the solution, doesn't it?

Yes, we added an extra command to build an intermediate image, but our base Dockerfile remains unchanged. In our CI system, we continue to run the same command to build our image.

docker build -t repo/image:1.0.0 .

Indeed, the developer experience has been affected. To address this, we could utilize BAKE.

Improve developer experience with Bake

Docker buildx bake offers a novel approach to constructing our images, leveraging parallelization and orchestration of builds.

It necessitates the addition of a configuration file called docker-bake.hcl and a modification in our image-building method.

// file: docker-bake.hcl
variable "_BASE_IMAGE" {
  default = "node:20"
}

variable "TAG" {
  default = "latest"
}

target "_securitycert" {
  context = "/path/to/your/certificates/folder"
  dockerfile-inline = <<EOF
  FROM ${_BASE_IMAGE}
  COPY rootcacert.pem /etc/ssl/certs/securitycert.pem
  ENV NODE_EXTRA_CA_CERTS=/etc/ssl/certs/securitycert.pem
  EOF
}

target "default" {
  context = "."
  tags = [
    "repo/image:${TAG}"
  ]
}

target "securitycert" {
  inherits = [ "default" ]
  contexts = {
    "${_BASE_IMAGE}" = "target:_securitycert"
  }
}

_BASE_IMAGE variable is used internally by our targets to share the value (it can be change by the user, but I use _ as a convention for internal usage)
TAG variable can be change by user to change the tag used at build time
_securitycert target define how we build our restricted context image with an inline-docker definition (we also can define it on a separate file)
default target define how we build our image on a standard environment (not restricted one)
securitycert target defined how we build our image in a restricted environment. It inherits properties from default target, so every change on the default target is replicated on the securitycert one.

In the securitycert target, we replace the context _BASE_IMAGE with the _securitycert target. Docker will build the _securitycert target prior to executing the final build due to its dependency on it.

At build time, if we do not specify which target to build, bake will use the default target.

To build locally, we need to run

TAG=1.0.0 docker buildx bake securitycert

On CI, or non-restrictive environment

TAG=1.0.0 docker buildx bake

# you can also run the old one if you want
# docker build -t repo/image:1.0.0 .

Conclusion

Docker and Dockerfile are potent tools for building container images with ease, yet they must remain straightforward, even in complex contexts. Adding complexity to Dockerfiles may deter developers from maintaining them, which is understandable.

Multi-context builds are effective in addressing numerous issues related to build contexts and ought to be utilized more frequently.

To enhance the developer experience and documentation, 'buildx bake' appears to be a beneficial tool, providing numerous advantages, especially when used in conjunction with multi-context builds.

Docker : Build complex Solution with multi-contexts and Bake

Alexis M. — Thu, 16 May 2024 06:14:17 +0000

In the .NET world, Solutions serve as a structural mechanism to organize multiple projects, such as applications and libraries, under a single umbrella. Typically, a Solution includes all the projects that make up a full application, streamlining the process of sharing DLLs. Instead of publishing them to a NuGet feed, they are directly referenced in the .csproj files within the Solution.

Enabling Docker support for a project prompts VisualStudio to generate a Dockerfile in that project. However, the image must be built from the Solution directory, not the individual project directory. This approach ensures that all necessary code, including that which resides outside the project directory within the Solution, is accessible during the build.

docker build -t repo/image:1.0.0 -f Project/Dockerfile .
# or, build from the project folder
# docker build -t repo/image:1.0.0 ..

It's fine if you have only 2 projects on the solution (The domain, and the API for example).

What if you have a big solution and you want to build one project that depends only on one of these projects ?

Imagine building an IoT application. You need to develop two modules to retrieve data from different data sources, with one module dedicated to each source. These modules will convert the data into a format that the rest of your application can understand and then transmit it to a queue system. On the receiving end of the queue, there are two modules: one for aggregating and storing the data, and another for generating alerts and dispatching them to a notification channel. Additionally, there is a fifth module that exposes the aggregated data through an API.

For the data ingestion modules, you establish an external library that encompasses the domain of each data source. To facilitate data exchange between your ingestion and processing modules, you create another library that defines the object structures. Lastly, you develop a separate library for your domain, which is utilized by your API.

We have 5 applications, and 4 libraries

flowchart TB
  IngestionA(IngestionA)
  IngestionA.Domain{{IngestionA.Domain}}
  IngestionB(IngestionB)
  IngestionB.Domain{{IngestionB.Domain}}
  Exchange{{Exchange}}
  Data(Data)
  Alerts(Alerts)
  API(API)
  Domain{{Domain}}

  IngestionA --> IngestionA.Domain
  IngestionA --> Exchange
  IngestionB --> IngestionB.Domain
  IngestionB --> Exchange
  Data --> Exchange
  Alerts --> Exchange
  API --> Domain

Let's build our 5 applications

docker build -t repo/ingestionA:1.0.0 -f IngestionA/Dockerfile .
docker build -t repo/ingestionB:1.0.0 -f IngestionB/Dockerfile .
docker build -t repo/data:1.0.0 -f Data/Dockerfile .
docker build -t repo/alerts:1.0.0 -f Alerts/Dockerfile .
docker build -t repo/api:1.0.0 -f API/Dockerfile .

Docker build context

When executing the docker build command, Docker uses the specified folder as the build context and transfers it to the Docker daemon. It omits files and folders specified in the .dockerignore file but includes all other files (the complete codebase).

For building our API, Docker transfers the entire solution to the build context, as denoted by the . in our command, despite the fact that we might only require the API and the domain code.

✏️ Docker, with BuildKit is smart enough to lazy send files to context with only required files. The purpose of this still apply, but not for context optimizations. It's apply for codebase that is not in the same folder (context) as your default one

Docker build mutli-contexts

To avoid sending the entire codebase, we can limit the context to only the essential parts of the code. However, it's crucial to define the context for the dependent code.

This can be achieved with the multi-contexts build feature, which necessitates an update to the Dockerfile to accommodate this new approach.

This is our full Dockerfile base for the IngestionA module

# file: Dockerfile
FROM mcr.microsoft.com/dotnet/runtime:8.0 AS base
USER app
WORKDIR /app

FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build
ARG BUILD_CONFIGURATION=Release
WORKDIR /src
COPY IngestionA.Domain/IngestionA.Domain.csproj ./IngestionA.Domain/
COPY Exchange/Exchange.csproj ./Exchange/
COPY IngestionA/IngestionA.csproj ./IngestionA/
RUN dotnet restore ./IngestionA/IngestionA.csproj
COPY . .
WORKDIR /src/IngestionA
RUN dotnet build ./IngestionA.csproj -c $BUILD_CONFIGURATION -o /app/build --no-restore

FROM build AS publish
ARG BUILD_CONFIGURATION=Release
RUN dotnet publish ./IngestionA.csproj -c $BUILD_CONFIGURATION -o /app/publish /p:UseAppHost=false --no-restore

FROM base AS final
EXPOSE 5000
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "IngestionA.dll"]

Let's focus on the build stage to use multi-contexts

# file: Dockerfile
...

FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build
ARG BUILD_CONFIGURATION=Release
WORKDIR /src
COPY --from=domain IngestionA.Domain.csproj ./IngestionA.Domain/
COPY --from=exchange Exchange.csproj ./Exchange/
COPY IngestionA.csproj ./IngestionA/
RUN dotnet restore ./IngestionA/IngestionA.csproj
COPY --from=domain . ./IngestionA.Domain/
COPY --from=exchange . ./Exchange/
COPY . ./IngestionA/
WORKDIR /src/IngestionA
RUN dotnet build ./IngestionA.csproj -c $BUILD_CONFIGURATION -o /app/build --no-restore

...

To build our image, we need to define each context

# from the IngestionA project folder
docker build -t repo/ingestionA:1.0.0 \
  --build-context domain=../IngestionA.Domain/ \
  --build-context exchange=../Exchange/ \
  -f Dockerfile .

⚠️ Each context must define it's own .dockerignore

Now, to build all 5 applications, we must write quite long commands

docker build -t repo/api:1.0.0 \
  --build-context domain=./Domain
  API/
docker build -t repo/ingestionA:1.0.0 \
  --build-context domain=./IngestionA.Domain/ \
  --build-context exchange=./Exchange/ \
  IngestionA/
docker build -t repo/ingestionB:1.0.0 \
  --build-context domain=./IngestionB.Domain/ \
  --build-context exchange=./Exchange/ \
  IngestionB/
docker build -t repo/data:1.0.0 \
  --build-context exchange=./Exchange/ \
  Data/
docker build -t repo/alerts:1.0.0 \
  --build-context exchange=./Exchange/ \
  Alerts/

Improve developer experience with Bake

Docker buildx bake is a new way to build our images while drawing parallelization and build's orchestration.

We need to add one configuration file named docker-bake.hcl and to change the way we build images.

// file: docker-bake.hcl

variable "TAG" {
  default = "latest"
}

target "api" {
  context = ./API/
  contexts = {
    domain = ./Domain/
  }
  tags = [
    "repo/api:${TAG}"
  ]
}

target "ingestionA" {
  context = ./IngestionA/
  contexts = {
    domain = ./IngestionA.Domain/
    exchange = ./Exchange
  }
  tags = [
    "repo/ingestionA:${TAG}"
  ]
}

target "ingestionB" {
  context = ./IngestionB/
  contexts = {
    domain = ./IngestionB.Domain/
    exchange = ./Exchange
  }
  tags = [
    "repo/ingestionB:${TAG}"
  ]
}

target "data" {
  context = ./Data/
  contexts = {
    exchange = ./Exchange
  }
  tags = [
    "repo/data:${TAG}"
  ]
}

target "alerts" {
  context = ./Alerts/
  contexts = {
    exchange = ./Exchange
  }
  tags = [
    "repo/alerts:${TAG}"
  ]
}

group "default" {
  targets = [
    "data",
    "alerts",
    "api",
    "ingestionA",
    "ingestionB"
  ]
}

group "ingestions" {
  targets = [
    "ingestionA",
    "ingestionB"
  ]
}

group "data-pipeline" {
  targets = [
    "ingestionA",
    "ingestionB",
    "data",
    "alerts"
  ]
}

To build our 5 applications, we have to run a single command

docker buildx bake

By default, Bake will search for a group default (or target if group isn't defined). In this case, the group "default" contains all our targets

If we want to build only the ingestion and processing's modules, we must target data-pipeline group

docker buildx bake data-pipeline

To build only the API, we must call bake with the target api

docker buildx bake api

All the images we build will be tagged with latest, we can change this behavior in different ways.

Override the TAG value using ENV

TAG=1.0.0 docker buildx bake api

Override the tag's image using --set argument

docker buildx bake api --set api.tags=api:1.0.0

Define a variable for each target and override them in a parameter file

// file: docker-bake.hcl
variable "API_TAG" = {
  default = "latest"
}

target "api" { 
 context = ./API/ 
 contexts = {
   domain = ./Domain/
 }
 tags = [ 
   "repo/api:${API_TAG}" 
 ] 
}
...

// file: bake-parameters.hcl
API_TAG = "1.0.0"

docker buildx bake api -f docker-bake.hcl -f bake-parameters.hcl

Conclusion

Docker's multi-context builds are particularly beneficial for large solutions with numerous projects, as they help to reduce build context's size.
While they introduce complexity by requiring the definition of each context, this is mitigated when combined with Bake.

Bake streamlines the building process of the entire solution, allowing for the simultaneous construction of all images with a single command.

DEV Community: Alexis M.

Docker : Change the build behavior with multi-contexts builds

Why do I need to change my build behavior ?

How do these tools block our Docker builds ?

What's wrong ?

So, how should we address this?

Solution 1 : One Dockerfile for local build, another one for CI build

Solution 2 : Copy certificates and use ARG/IF to condition the steps

Solution 3 : Use a compose.yaml definition

3.1 : build with tools that require a file or an ENV

3.2 : build with tools that require specific command

How multi-contexts build can help ?

How can it help ?

Improve developer experience with Bake

Conclusion

Docker : Build complex Solution with multi-contexts and Bake

Docker build context

Docker build mutli-contexts

Improve developer experience with Bake

Conclusion

Solution 3 : Use a `compose.yaml` definition