Here's How I Deployed My University Project with Traefik & Docker

And how you can deploy containerized applications with ease and confidence, too

I'm currently studying Applied Computer Science at the Cooperative State University Karlsruhe in Germany and about to finish yet another semester (the exceptional 2020 edition).

For the last two semesters, I've been working on a project for our Software Engineering lecture with a few friends of mine. The project consolidates many of our university's online services into a central hub which was due for finals last week.

One of our end goals is to deploy an environment that will be used as a live demo. Here's how I did it and what I can recommend for beginners, working with Traefik for the first time.

For the scope of this article, I will also provide some insights and best practices that I accumulated throughout my journey.

Why Traefik?

In the past, I deployed my web projects in various ways. From IIS (back when .NET Core wasn't even a thing), Apache (lol), to NGINX over to Traefik.

I've learned a lot during these phases, but I remember that I've spent more time configuring and fixing my reverse proxies than I probably spent on improving my code. Configuring networks and unfamiliar tools can become cumbersome, especially when the end-results don't meet your expectations.

For IIS, I remember the struggle with file permissions when a file is served from a local system path other than the default IIS document path. You don't want to spend many hours for administrating and fixing file permissions (that are not obvious in the first place), that's not your primary task as a developer. This, but also the complex IIS modules that required specific configuration are a huge pain.

Me configuring IIS for the 60th time

Also, let's not talk about configuring HTTPS for these tools.

As a software developer who chases clean code development, you want things to be as KISS and DRY as possible without dealing with every single configuration. Work smarter, not harder. Use the tools that simplify your life. 

These aspects have massively changed with the rise of DevOps related tasks and the rise of containerized applications as a new platform.

Traefik changes this. It's easy to wire up your reverse proxy for your containerized applications with features like HTTPS, SSL certificate renewal, middlewares, load balancing, etc. in a matter of minutes.

Discover Traefik

Traefik is an open-source state-of-the-art edge router by Containous which is designed to work with cloud-native applications. They claim to make networking boring. That's true, in a good way. 

I discovered Traefik version 1.0 in late 2018 at work, at the beginning of my studies when I needed to deploy my work project for demo purposes. I can't exactly recall how I stumbled upon it (I think it was Reddit), but the second I saw the features I was like: I need to try this out.

Ever since that, I've used it for all other projects, from v1 over to the current version v2.

How does it work?

Traefik works with a variety of providers such as Docker, Kubernetes, AWS, and many others. It's up to you which provider you use. I chose Docker because it's lightweight in comparison to many other providers and because I use it for local development, too.

It automatically discovers your services from the selected provider and matches it with your configuration. For all individual services, you can configure your endpoints, load balancing, path rewrites, basic auth, and even automatic HTTPS with SSL certificates powered by Let's Encrypt.

There's not much to say anymore because all the magic already happens for you automatically. So let's get our hands on it.

Getting started

If you'd like to refer to the project with the working Docker compose files, check out our university project (WIP).

Prepare Traefik

Before we can run Traefik, it's required that we have Docker fully up and running. I always recommend running Docker in Swarm mode when you're in production or staging. This makes deployment within a cluster easy (yes, Traefik even works on Docker Swarm across nodes out of the box).

To initialize Swarm, run docker swarm init and follow the instructions.

Once you're in swarm mode, you can start crafting your first Docker compose file for Traefik:

version: '3.8'

services:
  traefik:
    image: traefik:v2.2
    ports:
      - "80:80"
      - "443:443"
    command:
      - --api.insecure=true # set to 'false' on production
      - --api.dashboard=true # see https://docs.traefik.io/v2.0/operations/dashboard/#secure-mode for how to secure the dashboard
      - --api.debug=true # enable additional endpoints for debugging and profiling
      - --log.level=DEBUG # debug while we get it working, for more levels/info see https://docs.traefik.io/observability/logs/
      - --providers.docker=true
      - --providers.docker.swarmMode=true
      - --providers.docker.exposedbydefault=false
      - --providers.docker.network=unidash_traefik
      - --entrypoints.web.address=:80
      - --entrypoints.web-secured.address=:443
      - --certificatesresolvers.defaultlts.acme.httpChallenge.entrypoint=web
      - --certificatesresolvers.defaultlts.acme.email=[e-mail]
      - --certificatesresolvers.defaultlts.acme.storage=/letsencrypt/acme.json
    volumes:
      - letsencrypt:/letsencrypt
      - /var/run/docker.sock:/var/run/docker.sock
    networks:
      - traefik
    deploy:
      placement:
        constraints:
          - node.role == manager
      labels:
        - "traefik.enable=true"
        - "traefik.http.services.api.loadbalancer.server.port=8080"
        - "traefik.http.routers.api.rule=Host(`traefik.dev.unidash.top`)"
        - "traefik.http.routers.api.entrypoints=web-secured"
        - "traefik.http.routers.api.tls.certresolver=defaultlts"

networks:
  traefik:
    driver: overlay

volumes:
  letsencrypt:

I hope you are familiar with Docker compose. I will quickly run through some noteworthy configuration entries for Traefik:

• --api-dashboard=true: Enables a nice-looking dashboard where you can inspect your routes, services, and configuration.

• --providers.docker.swarmMode: Traefik runs on non-swarm mode enabled Docker hosts by default, but we need to instruct that we are using Docker Swarm.

• --providers.docker.exposedbydefault: By default, all discovered containers are automatically exposed to arbitrary endpoints, but we need to suppress this behavior.

In addition to that, we instruct Traefik that it should only discover containers that joined the designated Docker network as described in providers.docker.network and the networks entry at the end of the compose file.

Best Practice #1: Your network must be an overlay network for Docker Swarm instances. I wasted so many hours when I worked with it in Traefik v1 until I figured out what the hell the overlay driver is. Don't make the same mistake. (To create an overlay network manually, type: docker network create proxy -d overlay)

In the volumes section, we specify a Docker volume for our Let's Encrypt (LE) certificates store so we can persist it across tear-downs during development.

Best Practice #2: Make sure that you persist your LE certificates, otherwise you will face issues when deploying your host from scratch for the fifth time or so within a specific period. The certificate authority has a rate limit in place and won't allow you to renew the certificate for the given DNS name. You are essentially doomed then, especially on full production (true story, this has happened to a past QA environment of mine - no good).

For Docker Swarm deployments, we set a constraint for the Traefik container because we always want to run it on the master Docker node.

Now, the majority has been configured and from now on, we only need to configure the labels that configure Traefik for our containerized application. In the compose file, we will route our network traffic to Traefik's dashboard.

---

Before we do that, I want to explain some basic Traefik terminology.

Router

A router attributes how a specific request is handled. You instruct the router how to route the request to its destination (service) by providing a hostname, an entry point, middlewares, and much more. For now, just focus on the hostname.

Service

Service is the application that handles the incoming traffic. In the realms of Docker, that's your containerized application.

Entrypoint

We have already configured two entry points in our first compose file. An entry point defines the port that will receive the network packets. You don't need to worry about it after you configured the Traefik instance, it just works.

---

Okay, so now with the basic terminology out of the way, the configuration should be pretty much semantic & self-explanatory:

labels:
  - "traefik.enable=true"
  - "traefik.http.services.api.loadbalancer.server.port=8080"
  - "traefik.http.routers.api.rule=Host(`traefik.dev.unidash.top`)"
  - "traefik.http.routers.api.entrypoints=web-secured"
  - "traefik.http.routers.api.tls.certresolver=defaultlts"

• Since we tell Traefik to not auto-discover services, we need to set traefik.enable to true.
• In the services section, we tell Traefik where to connect to our containerized application. The dashboard uses port 8080.
• For the routers section, we specify the hostname, desired entry points and the certificate resolver.

Best Practice #3: We can also use complex host rules with tailing paths, but only do that if your app is compatible with different path bases than /, otherwise you'll face non-obvious issues.

Prepare the Application

Now that Traefik is configured, you will need to compose your application's compose file. This is a case by case basis, but I'll show you how I did it for my project.

Best Practice #4: For the containerized application, I have two separate Docker compose files. The first one is used locally and for remote deployments, the second one is used for specific environment overrides, for both local and remote deployments individually.

version: '3.8'

services:
  gateway:
    image: unidash/gateway
    build:
      context: .
      dockerfile: Unidash.Gateway/Dockerfile

  auth:
    image: unidash/auth
    build:
      context: .
      dockerfile: Unidash.Auth.Application/Dockerfile
    depends_on:
      - mssql

  # Trimmed for brevity...

  mongodb:
    image: mongo
    volumes:
      - mongodb_data:/data/db

  mssql:
    image: mcr.microsoft.com/mssql/server:2019-latest
    volumes:
      - mssql:/var/opt/mssql

  rabbitmq:
    image: rabbitmq:3


volumes:
  mongodb_data:
  mssql:

The clear advantage of this is that I can use this as a template for all environments. For local builds, it will build my images from scratch based on the build: property. For Docker stack deployments, it will use the image from Docker Hub instead and simply ignore the build property.

Now we need the environment's specific overrides. That will be our dev/prod environment.

version: '3.8'

services:
  gateway:
    networks: 
      - traefik
      - default
    deploy:
      labels:
        - "traefik.enable=true"
        - "traefik.http.services.gateway.loadbalancer.server.port=80"
        - "traefik.http.routers.gateway-secured.rule=Host(`tinf18b4.dev.unidash.top`)"
        - "traefik.http.routers.gateway-secured.entrypoints=web-secured"
        - "traefik.http.routers.gateway-secured.tls.certresolver=defaultlts"
        - "traefik.http.middlewares.gateway.headers.accesscontrolallowmethods=OPTIONS,GET,PATCH,PUT,DELETE"
        - "traefik.http.middlewares.gateway.headers.accesscontrolalloworiginlist=*"
        - "traefik.http.middlewares.gateway.headers.accesscontrolmaxage=100"
        - "traefik.http.middlewares.gateway.headers.addvaryheader=true"

  auth:
    networks:
      - default
    environment:
      - "ConnectionStrings:AuthDbContext=Server=mssql, 1433;Database=Auth;User=sa;Password=yourStrong!Password123;"

  timetable:
    networks: 
      - default
    environment:
      - "ConnectionStrings:TimeTableDbContext=Server=mssql, 1433;Database=TimeTable;User=sa;Password=yourStrong!Password123;"
      - "TimeTable:UpstreamICalUrl=https://rapla.dhbw-karlsruhe.de/rapla?page=ical&user=eisenbiegler&file=TINF18B4"

  canteen:
    networks: 
      - default

  chat:
    networks: 
      - default

  mssql:
    networks:
      - default
    environment:
      - ACCEPT_EULA=Y
      # This is for **demo purposes** only: don't store your secrets in compose files.
      # Instead, use Docker secrets or an equivilant secret manager.
      - SA_PASSWORD=yourStrong!Password123 

When developing microservices, it's advised using a centralized API gateway. Whether or not you should use it is an entirely different topic and not part of this article, but you can read more about it here if you are feeling adventurous.

As you can probably see, I used a similar configuration like the first one when we configured Traefik.

I instruct Traefik where to find the containerized application (the service) and how to route to it (the router). For further progress, we need to add new terminology to our disposal.

---

 Middlewares

A middleware extends the behavior of routers. It can fine-tune requests before they are transmitted to the service or back to the client. Commonly used middlewares are headers, rate limits, redirect schemes, and many more.

---

Since our project consists of a front-end and a back-end, I added a header middleware to the gateway which deals with CORS. The configuration entries are self-explanatory if you are already familiar with CORS, but if you are curious for more, check out Traefik's documentation.

Best Practice #5: Make sure that your containerized application supports forwarded headers. Traefik automatically forwards headers to your service. You might also need to adjust CORS at the application-level.

Also, don't forget to add the traefik network we created in the first compose file to the stack, otherwise, Traefik won't discover the service as per our initial configuration.

Okay, so now that was quite a lot of configuration. Let that sink in and grab a cup of coffee before we deploy our stack.

Release the Kraken

The deployment is the easiest yet most satisfying part. Let's quickly recap all files we've created before we deploy:

• docker-compose.yml This is the bare-bones configuration file that serves as our parent. It has all the details about where to find our Docker images.
• docker-compose.traefik.yml This compose file contains the Traefik instance and configuration within the commands and labels section. Remember that this is the first time we configured a Traefik service.
• docker-compose.dev.yml or docker-compose.{env}.yml This compose file is additive to our base docker-compose.yml file. It contains the application's and Traefik's specific configuration.

Now, let's copy these files to our main hosts with our favorite tool.

Best Practice #6: This type of hierarchy in our configuration also makes it reliable for CI/CD. Suppose you have a release pipeline that deploys to prod or dev. You can chuck the base files and then add the environment-specific file (the third one in the list) as an additive overwrite. This keeps it so organized and maintainable for future changes.

Once they are up on our server, we just need to run:

docker stack deploy <your project's name> -c docker-compose.yml -c docker-compose.traefik.yml -c docker-compose.dev.yml

… and voila, our app is now up and running, fully bootstrapped with SSL, too.

To verify this, you can try accessing the Traefik dashboard and your app's endpoint. To access the dashboard, look at your Traefik specific compose file again, then copy the hostname, add a /dashboard at the end of it and boom, you're now on the dashboard.

I hope that you enjoyed this article and find this helpful. Comment if you have feedback or question, I'd be happy to answer them.

Also, please feel free to follow me on dev.to and Twitter, or if you're feeling extra generous, buy me a coffee. 

Thanks for reading, and as always nowadays, stay safe.

---

Thanks Ryan for the coffee!

---

Hero photo by Brendan Church on Unsplash.