Packaging ASP.NET applications into containers deployed on Kubernetes.

Let's take a look at different options for deploying an ASP.NET application in a container on Linux, along with the benefits and tradeoffs. The example application I'll be using is an F# ASP.NET demo application:
github.com/tyauvil/fsharp-beats

Container deployment is converging on some best practices for building application containers. Specifically, minimizing or eliminating dependencies and optimizing for size. This has two major benefits, less attack surface and faster deployments. To accomplish this with Rust or Go, the standard practice is to compile an application as a statically compiled binary and copying the resulting artifact to a scratch container so the end result is a small container with a minimal attack surface. That is not possible in .NET Core 3 but is a planned feature of .NET Core 5, however that feature is not available as of the currently available .NET Core 5.0 Preview 1.

The smallest possible deployment image uses the PublishSingleFile parameter in the build process. This creates a compressed executable file that packages all dependencies into a zip file and decompresses them to a temporary directory in the filesystem on first execution. The container will still need a handful of dynamic libraries in order to run. One needs to be aware that the optional Alpine container uses the musl C library which is different from the GNU C library used in Debian, the standard container OS. A single file executable compiled for Alpine will not run on Debian and vice versa.

.NET Core Runtime parameters:

Alpine: linux-musl-x64
Debian: linux-x64

Two stage container build

Unlike a Python container for example, an ASP.NET container benefits greatly from a two stage build. The first image has all of the .NET Core compiler components and during the build several hundred megabytes of temporary files are created. These are discarded when the compiled application is copied over to the runtime container.

FROM mcr.microsoft.com/dotnet/core/sdk:3.1 AS build-env

WORKDIR /app

# Copy fsproj and restore as distinct layers
COPY *.fsproj ./
RUN dotnet restore

# Copy everything else and build
COPY . ./
RUN dotnet publish -c Release --runtime linux-x64 -p:PublishReadyToRun=true \
    -o out


# Build runtime image
FROM mcr.microsoft.com/dotnet/core/aspnet:3.1

WORKDIR /app
COPY --from=build-env /app/out .

# install tini to act as init 
RUN apt-get update && apt-get install -y tini &&\
    rm -rf /var/lib/apt/lists/*

ENTRYPOINT ["tini", "--", "dotnet", "fsharp-beats.dll"]

The build adds about 111MB to the mcr.microsoft.com/dotnet/core/aspnet:3.1 container image for a total of 322MB. The main benefit of the two stage build is a savings of ~1.2GB over the intermediate container which grows to 1.52GB during the build process.

$ podman images
REPOSITORY           TAG      IMAGE ID       CREATED       SIZE
fsharp-beats         latest   1d1fe4deb61a   2 hours ago   322 MB
<none>               <none>   b3786230b0b1   2 hours ago   1.52 GB
dotnet/core/sdk      3.1      336698ad1713   7 hours ago   703 MB
dotnet/core/aspnet   3.1      9ac62e540b12   7 hours ago   211 MB

Build using the PublishSingleFile parameter the build process creates a single compressed executable that can be packaged in a container with minimal dependencies. Microsoft publishes the mcr.microsoft.com/dotnet/core/runtime-deps container image expressly for the purpose of running these single file executables. It won't run conventionally compiled .NET Core applications. In this example I use the 3.1-alpine tag which is only 10.5MB, one tenth the size of the Debian based runtime-deps container.

$ podman images | grep runtime-deps
REPOSITORY     TAG          IMAGE ID       CREATED        SIZE
runtime-deps   3.1-alpine   cd1cb1f0cfc3   24 hours ago   10.5 MB
runtime-deps   3.1          b52283a2b18f   24 hours ago   114 MB

FROM mcr.microsoft.com/dotnet/core/sdk:3.1-alpine AS build-env

WORKDIR /app

# Copy fsproj and restore as distinct layers
COPY *.fsproj ./
RUN dotnet restore

# Copy everything else and build
COPY . ./
RUN dotnet publish -c Release -r linux-musl-x64 -p:PublishSingleFile=true -o out


# Build runtime image
FROM mcr.microsoft.com/dotnet/core/runtime-deps:3.1-alpine

COPY --from=build-env /app/out/fsharp-beats /usr/local/bin/

RUN apk add --no-cache tini

ENTRYPOINT ["tini", "--", "fsharp-beats"]

There are pretty big image size savings by using the PublishSingleFile parameter. This build is also using an Alpine container with a few additional C libraries that are required because the binary is not statically linked, a feature coming in .NET Core 5.0.

$ podman images | grep singleFile
REPOSITORY     TAG          IMAGE ID       CREATED          SIZE
fsharp-beats   singleFile   4c781f411c21   15 seconds ago   74.5 MB

Kubernetes

Now to deploy to Kubernetes. There are a few considerations here, since we are definining a port other than 80 because we are running the conatiner as a non-root user, we need to pass in an environment variable to the container to configure ASP.NET to listen on a different port with the ASPNETCORE_URLS variable. This is similar to FLASK_URL in a Python flask application. We also specify a liveness probe to check a route that returns 200, which confirms the application is running. Bypassing the / route lowers the performance impact of the liveness probe. The liveness probe can also use custom headers which can be used to perform certain functions in the application or make it easy to filter them out while looking at application logs.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: fsharp-beats-deployment
spec:
  selector:
    matchLabels:
      app: fsharp-beats
  replicas: 2
  template:
    metadata:
      labels:
        app: fsharp-beats
    spec:
      securityContext:
        runAsUser: 65534 # run as the nobody/nogroup user
      containers:
      - name: fsharp-beats
        image: "tyauvil/fsharp-beats:latest"
        ports:
        - containerPort: 5000
        env:
        - name: ASPNETCORE_URLS
          value: "http://+:5000"
      - name: liveness
        image: k8s.gcr.io/liveness
        args:
        - /server
        livenessProbe:
          httpGet:
            path: /healthz
            port: 5000
          initialDelaySeconds: 10
          periodSeconds: 15

Since this application is designed to run in Kubernetes, logging is done directly to standard out (or the console). This allows one to view logs from the cli.

let configureLogging (builder : ILoggingBuilder) =
    // Set up the Console logger
    builder.AddConsole() |> ignore

After deploying the container we can see that the liveness probe is working as expected and we get the correct output from the root path of the demo application. This also shows that ASP.NET applications can run perfectly happy in a Linux container running in Kubernetes without any workarounds. The "cloud native" single binary deploy is coming in the next version however .NET Core is perfectly suitable for deploying in containers today.

$ kubectl apply -f k8s/deployment.yml

$ kubectl expose deployment fsharp-beats-deployment --type=NodePort

$ curl http://192.168.39.213:31654
@035.beats

$ kubectl logs fsharp-beats-deployment-7c5d9c4dcb-j9dww fsharp-beats | tail -8
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
      Request starting HTTP/1.1 GET http://192.168.39.213:31654/  
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
      Request finished in 0.6899ms 200 text/plain; charset=utf-8
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
      Request starting HTTP/1.1 GET http://10.1.0.38:5000/healthz  
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
      Request finished in 0.2705ms 200 text/plain; charset=utf-8

I hope this blog post shows how running .NET Core applications on Linux/Kubernetes is relatively straightforward and has very few caveats. There is a big opportunity here to migrate .NET applications from Windows to Linux for both reliability and cost savings reasons. This is no longer Steve Ballmer's Microsoft.

References:
Making a tiny .NET Core 3.0 entirely self-contained single executable