Disclaimer
This article introduces Traefik, a modern reverse proxy and load balancer for deploying micro-services. We will cover its basics, key features, configuration, and integration with Docker. By the end, you should understand how to set up and use Traefik in your projects.
The final example in this article builds on concepts discussed in my previous article: How to Automate Deployment with Ansible and GitHub Actions. In that article, I explained how to streamline your deployment process using Ansible for configuration management and GitHub Actions for continuous integration and deployment. If you haven't read it yet, I highly recommend doing so to fully grasp the advanced section of this article. Understanding the automation techniques covered there will be crucial for implementing the final example effectively.
Introduction
Traefik is a modern open-source reverse proxy and load balancer. It’s built with simplicity and flexibility in mind. Traefik excels in a containerized setup, especially with micro-services.
(Image source: Traefik Documentation)
Key features of Traefik proxy
Traefik comes with amazing features, including:
Dynamic configurations
Automatic SSL/TLS
Load balancing
Middleware support
Integration with other platforms
Configuring Traefik
Traefik configurations are written in familiar syntax i.e TOML or YAML .
Traefik gives you flexibility on the type of configuration you are comfortable with. It support the following types of configurations;
Static configuration where every or some configurations are defined in a single file;
traefik.yml
-
Dynamic configuration where different providers are supported and used to automatically detect or create Traefik configurations.
For example, with the Docker provider, configurations are defined as labels on Docker containers and automatically detected as Traefik configurations.
Other supported providers include Swarm, Nomad, Kubernetes, Consul, and many more.
Traefik & Docker
In this article, our focus is on Traefik’s Docker provider. We are going to install traefik as a docker container and the use it to proxy a python flask application ( also a docker container ).
A Demo project
For this first demo, I’ll explain a simple setup using docker-compose. In the next demo, we will integrate with GitHub Actions and Ansible.
Here is the project structure.
traefik-demo\
|-- docker-compose.yml
|-- traefik\
| |-- certs\
| |-- traefik.yml
To get started, we need to define a Traefik configuration file. Here is what it should look like:
traefik.yml
global:
checkNewVersion: true
sendAnonymousUsage: false # true by default
# (Optional) Log information
# ---
# log:
# level: ERROR # DEBUG, INFO, WARNING, ERROR, CRITICAL
# format: common # common, json, logfmt
# filePath: /var/log/traefik/traefik.log
# (Optional) Accesslog
# ---
# accesslog:
# format: common # common, json, logfmt
# filePath: /var/log/traefik/access.log
# (Optional) Enable API and Dashboard
# ---
api:
dashboard: true # true by default
insecure: false # Don't do this in production!
# Entry Points configuration
# ---
entryPoints:
web:
address: :80
# (Optional) Redirect to HTTPS
# ---
http:
redirections:
entryPoint:
to: websecure
scheme: https
websecure:
address: :443
# Configure your CertificateResolver here...
# ---
certificatesResolvers:
staging:
acme:
email: "{ EMAIL }"
storage: /etc/traefik/certs/acme.json
caServer: "https://acme-staging-v02.api.letsencrypt.org/directory"
tlsChallenge: {}
httpChallenge:
entryPoint: web
production:
acme:
email: "{ EMAIL }"
storage: /etc/traefik/certs/acme.json
caServer: "https://acme-v02.api.letsencrypt.org/directory"
tlsChallenge: {}
httpChallenge:
entryPoint: web
providers:
docker:
exposedByDefault: false # Default is true
file:
# watch for dynamic configuration changes
directory: /etc/traefik
watch: true
For automatic SSL to work, make sure to add a valid email. Replace “{ EMAIL }” with your own email address.
Let’s see how the docker compose file would look like.
docker-compose.yml
# Run traefik webserver, portainer, watchtower
services:
traefik:
container_name: "traefik"
image: "traefik:v2.5"
ports:
- "80:80"
- "443:443"
# - "59808:8080" # Uncomment this to expose the traefik dashboard
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- traefik-ssl-certs:/ssl-certs
- ./traefik:/etc/traefik
networks:
- traefik_network
restart: always
api:
image: ghcr.io/jackkweyunga/auto-deploy-flask-to-server:main
environment:
- FLASK_ENV=production
- DEBUG=${DEBUG}
expose:
- "5000"
network:
- traefik_network
labels:
- traefik.enable=true
- traefik.http.routers.api.rule=Host(`mydomain.com`)
- traefik.http.routers.api.entrypoints=websecure
- traefik.http.services.api.loadbalancer.server.port=5000
- traefik.http.routers.api.tls.certresolver=production
restart: always
networks:
traefik_network:
external: true
volumes:
traefik-ssl-certs:
Notice that we have defined a traefik_network
in the docker-compose file as external. We need to create this network before running the file. Every other Docker container that will be proxied by Traefik must join the traefik_network
.
sudo docker network create traefik_network
Notice the labels we added to the API service and that it is also part of the traefik_network
. The labels define Traefik configurations. Below is a detailed explanation of each label.
-
traefik.enable=true
Tells Traefik to proxy this container/service.
-
traefik.http.routers.api.rule=Host(
mydomain.com
)Tells Traefik to route all traffic from mydomain.com to this container/service. Ensure proper DNS records are set up to point the domain name to the server running Traefik.
-
traefik.http.routers.api.entrypoints=websecure
Tells Traefik to use a secure entry point for this container/service, as defined in the traefik.yml configuration file. This means using port 443, the secure port.
-
traefik.http.services.api.loadbalancer.server.port=5000
Tells Traefik to direct traffic to port 5000 of the container.
-
traefik.http.routers.api.tls.certresolver=production
Tells Traefik to use the production Let's Encrypt endpoints when fetching SSL certificates. This is also referenced in the traefik.yml configuration file.
NOTE: For each service, the router name must be unique from router names in other services. The same rule applies to service names. The router name is the part that comes after the dot, like traefik.http.routers.{router name}. If router names of different services are the same, proxying will fail.
When the setup runs on the server, Traefik will listen on ports 80 and 443. Any traffic on port 80 will be redirected to the secure port 443. Also, when someone visits mydomain.com
, a response from the Flask application will be returned.
Add traefik to your CI/CD workflow
Referring to the article How to Automate Deployment with Ansible and GitHub Actions, where we set up our CI/CD workflow with Ansible and GitHub Actions, we will now build on that and add the Traefik proxy. This will let us assign public domain names to our services.
Follow this GitHub repository: https://github.com/jackkweyunga/auto-deploy-flask-to-server
Traefik Ansible role
First, we add a Traefik Ansible role. In this role, we define the automation needed to configure and run Traefik on a remote server. Below is the file structure, which can be added to the structure mentioned in the previous article.
--- previous ---
.github\
workflows\
deploy-proxy.yml
ansible\
proxy.yml
traefik\
tasks\
main.yml
templates\
traefik.yml.jinja2
templates/traefik.yml.jinja2
global:
checkNewVersion: true
sendAnonymousUsage: false # true by default
# (Optional) Log information
# ---
# log:
# level: ERROR # DEBUG, INFO, WARNING, ERROR, CRITICAL
# format: common # common, json, logfmt
# filePath: /var/log/traefik/traefik.log
# (Optional) Accesslog
# ---
# accesslog:
# format: common # common, json, logfmt
# filePath: /var/log/traefik/access.log
# (Optional) Enable API and Dashboard
# ---
api:
dashboard: true # true by default
insecure: false # Don't do this in production!
# Entry Points configuration
# ---
entryPoints:
web:
address: :80
# (Optional) Redirect to HTTPS
# ---
http:
redirections:
entryPoint:
to: websecure
scheme: https
websecure:
address: :443
# Configure your CertificateResolver here...
# ---
certificatesResolvers:
staging:
acme:
email: {{ EMAIL }}
storage: /etc/traefik/certs/acme.json
caServer: "https://acme-staging-v02.api.letsencrypt.org/directory"
tlsChallenge: {}
httpChallenge:
entryPoint: web
production:
acme:
email: {{ EMAIL }}
storage: /etc/traefik/certs/acme.json
caServer: "https://acme-v02.api.letsencrypt.org/directory"
tlsChallenge: {}
httpChallenge:
entryPoint: web
serversTransport:
insecureSkipVerify: true
providers:
docker:
exposedByDefault: false # Default is true
file:
# watch for dynamic configuration changes
directory: /etc/traefik
watch: true
This Traefik configuration template allows us to pass variables to it while saving it on the remote server. In this case, an EMAIL will be passed.
tasks/main.yml
---
- name: Preparing required files and Directories in /etc/traefik
become: true
block:
- name: Create directory
file:
path: /etc/traefik
state: directory
- name: Create directory2
file:
path: /etc/traefik/certs
state: directory
- name: Copy config file
ansible.builtin.template:
src: templates/traefik.yml.jinja2
dest: /etc/traefik/traefik.yaml
- name: Configuring traefik
become: true
block:
- name: Create ssl-certs Volume
community.docker.docker_volume:
name: traefik-ssl-certs
register: v_output
ignore_errors: true
- name: Debug output
ansible.builtin.debug:
var: v_output
- block:
- name: Create traefik_network
become: true
community.docker.docker_network:
name: traefik_network
register: n_output
ignore_errors: true
- name: Debug output
ansible.builtin.debug:
var: n_output
when: v_output
- block:
- name: Deploy Traefik
community.docker.docker_container:
name: traefik
image: "traefik:v2.10"
ports:
- "80:80"
- "443:443"
- "59808:8080"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- traefik-ssl-certs:/ssl-certs
- /etc/traefik:/etc/traefik
networks:
- name: "traefik_network" # required. Name of the network to operate on.
restart_policy: always
labels:
com.centurylinklabs.watchtower.enable: "false"
register: d_output
ignore_errors: true
- name: Debug output
ansible.builtin.debug:
var: d_output
when: n_output
In this task, where the magic happens, we create the required folders, files, Docker volumes, and Docker networks (traefik_network). Then, we use Ansible’s docker_container community plugin to run Traefik on the remote server(s). If you look closely, you'll notice the similarity to the docker-compose.yml we used in the first demo.
ansible/proxy.yml
---
- hosts: webservers
vars_files:
- secret
roles:
- traefik
vars:
EMAIL: "{{ lookup('ansible.builtin.env', 'EMAIL') }}"
This is the main Ansible playbook we’ll run to configure Traefik. Notice how it references the Traefik role. We also read the EMAIL variable from the environment variable.
Now that the Ansible configurations are ready, let's add a GitHub workflow to run Ansible on demand with GitHub Action runners.
Add the proxy GitHub Workflow
.github/workflows/deploy-proxy.yml
name: proxy
on:
workflow_dispatch:
inputs:
REMOTE_USER:
type: string
description: 'Remote User'
required: true
default: 'ubuntu' # Edit here
EMAIL:
type: string
description: 'Email for fetching certs'
required: true
default: '<put a valid email here>' # Edit here
HOME_DIR:
type: string
description: 'Home Directory'
required: true
default: '/home/ubuntu' # Edit here
TARGET_HOST:
description: 'Target Host'
required: true
default: "< ip / domain >" # Edit here
jobs:
ansible:
runs-on: ubuntu-latest
env:
EMAIL: "${{ inputs.EMAIL }}"
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Add SSH Keys
run: |
cat << EOF > ansible/devops-key
${{ secrets.SSH_DEVOPS_KEY_PRIVATE }}
EOF
- name: Update devops private key permissions
run: |
chmod 400 ansible/devops-key
- name: Install Ansible
run: |
pip install ansible
- name: Adding or Override Ansible inventory File
run: |
cat << EOF > ansible/inventory.ini
[webservers]
${{ inputs.TARGET_HOST }}
EOF
- name: Adding or Override Ansible Config File
run: |
cat << EOF > ./ansible/ansible.cfg
[defaults]
ansible_python_interpreter='/usr/bin/python3'
deprecation_warnings=False
inventory=./inventory.ini
remote_tmp="${{ inputs.HOME_DIR }}/.ansible/tmp"
remote_user="${{ inputs.REMOTE_USER }}"
host_key_checking=False
private_key_file = ./devops-key
retries=2
EOF
- name: Run main playbook
run: |
sh ansible/create-sudo-password-ansible-secret.sh ${{ secrets.SUDO_PASSWORD }}
ANSIBLE_CONFIG=ansible/ansible.cfg ansible-playbook ansible/proxy.yml --vault-password-file=ansible/vault.txt
Make sure to edit the EMAIL and TARGET_HOST inputs.
We can now edit the flask-api ansible role to include traefik configurations via container labels.
Update the flask-api Ansible role
Edit ansible/flask-api/files/docker-compose.yml
volumes:
portainer-data:
# Added networks
networks:
traefik_network:
external: true
services:
portainer:
image: portainer/portainer-ce:alpine
container_name: portainer
command: -H unix:///var/run/docker.sock
expose:
- "9000"
volumes:
# Connect docker socket to portainer
- "/var/run/docker.sock:/var/run/docker.sock"
# Persist portainer data
- "portainer_data:/data"
restart: always
# Added networks
networks:
- traefik_network
# Added lables
labels:
- "traefik.enable=true"
- "traefik.http.routers.portainer.rule=Host(`${PORTAINER_DOMAIN}`)"
- "traefik.http.routers.portainer.entrypoints=websecure"
- "traefik.http.routers.portainer.tls.certresolver=production"
- "traefik.http.routers.portainer.tls=true"
- "traefik.http.services.portainer.loadbalancer.server.port=9000"
- "traefik.docker.network=traefik_network"
watchtower:
container_name: "watchtower"
image: "docker.io/containrrr/watchtower"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
# To enable docker authentication, uncomment the line below.
# You also need to make sure you are logged in to docker in the server
# E.g by running: sudo docker login ghcr.io
# - /root/.docker/config.json:/config.json:ro
restart: always
environment:
TZ: Africa/Dar_es_Salaam
WATCHTOWER_LIFECYCLE_HOOKS: "1" # Enable pre/post-update scripts
command: --debug --cleanup --interval 30
web:
image: ghcr.io/jackkweyunga/auto-deploy-flask-to-server:main
environment:
- FLASK_ENV=production
- DEBUG=${DEBUG}
expose:
- "5000"
restart: always
# Added networks
networks:
- traefik_network
# Added labels
labels:
- com.centurylinklabs.watchtower.enable=true
- traefik.enable=true
- traefik.http.routers.api.rule=Host(`${FLASK_API_DOMAIN}`)
- traefik.http.routers.api.entrypoints=websecure
- traefik.http.services.api.loadbalancer.server.port=5000
- traefik.http.routers.api.tls.certresolver=production
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
deploy:
resources:
limits:
memory: "256m"
cpus: "0.50"
Check all places with the comment #Added ... These sections are due to adding the Traefik proxy configurations.
Notice new variables:
FLASK_API_DOMAIN: The domain name of our Flask API, e.g., api.mydomain.com
PORTAINER_DOMAIN: The domain name of our Portainer instance, e.g., portainer.mydomain.com
As a result, we need to add these variables to ansible/deploy.yml
so that we can read them from the environment in GitHub Actions runners.
Edit: ansible/deploy.yml
---
- hosts: webservers
# an encrypted ansible secret file containing the sudo password
vars_files:
- secret
roles:
- services
environment:
DEBUG: "{{ lookup('ansible.builtin.env', 'DEBUG') }}"
# Added new variables
FLASK_API_DOMAIN: "{{ lookup('ansible.builtin.env', 'FLASK_API_DOMAIN') }}"
PORTAINER_DOMAIN: "{{ lookup('ansible.builtin.env', 'PORTAINER_DOMAIN') }}"
Update the GitHub workflow for deployment
Finally, we edit the deploy.yml
GitHub actions workflow file to include the domains
name: ansible-deploy
on:
workflow_dispatch:
inputs:
REMOTE_USER:
type: string
description: 'Remote User'
required: true
default: 'ubuntu'
HOME_DIR:
type: string
description: 'Home Directory'
required: true
default: '/home/ubuntu'
TARGET_HOST:
description: 'Target Host'
required: true
default: "example.com" # Change this to your server IP or Domain
jobs:
ansible:
runs-on: ubuntu-latest
env:
DEBUG: 0
# Added new variables
FLASK_API_DOMAIN: "api.mydomain.com" # Add your domain
PORTAINER_DOMAIN: "portainer.mydomain.com" # Add your domain
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Add SSH Keys
run: |
cat << EOF > ansible/devops-key
${{ secrets.SSH_DEVOPS_KEY_PRIVATE }}
EOF
- name: Update devops private key permissions
run: |
chmod 400 ansible/devops-key
- name: Install Ansible
run: |
pip install ansible
- name: Adding or Override Ansible inventory File
run: |
cat << EOF > ansible/inventory.ini
[webservers]
${{ inputs.TARGET_HOST }}
EOF
- name: Adding or Override Ansible Config File
run: |
cat << EOF > ./ansible/ansible.cfg
[defaults]
ansible_python_interpreter='/usr/bin/python3'
deprecation_warnings=False
inventory=./inventory.ini
remote_tmp="${{ inputs.HOME_DIR }}/.ansible/tmp"
remote_user="${{ inputs.REMOTE_USER }}"
host_key_checking=False
private_key_file = ./devops-key
retries=2
EOF
- name: Run deploy playbook
run: |
sh ansible/create-sudo-password-ansible-secret.sh ${{ secrets.SUDO_PASSWORD }}
ANSIBLE_CONFIG=ansible/ansible.cfg ansible-playbook ansible/deploy.yml --vault-password-file=ansible/vault.txt
Make sure to add working domain, that have DNS records directing them to the target server.
Operation
To install traefik on a remote server, run the proxy
GitHub workflow. After that run the deploy
GitHub workflow to update your infrastructure with the newly added traefik configurations and Domain names.
After both runs are successful, you can now visit your domains securely (with SSL). If SSL has not activated yet, give it some time and monitor Traefik’s logs in Portainer in case there is an issue.
Happy Configuring !
Conclusion
If you have reached this point, it is evident that Traefik significantly enhances your workflow. The Traefik proxy is an outstanding project that integrates seamlessly with containers and can be easily incorporated into your CI/CD pipeline.
With the addition of Traefik, our setup is nearly complete. Stay tuned for my next article.
Seeking expert guidance in Ops, DevOps, or DevSecOps? I provide customized consultancy services for personal projects, small teams, and organizations. Whether you require assistance in optimizing operations, improving your CI/CD pipelines, or implementing strong security practices, I am here to support you. Let's collaborate to elevate your projects. Contact me today | LinkedIn | GitHub
Top comments (0)