DEV Community: Paul Knulst

Self-Host Umami Analytics With Docker Compose

Paul Knulst — Thu, 27 Mar 2025 07:16:26 +0000

How To Track Website Analytics And Respect Data Privacy With Umami

Paul Knulst in Self-Hosted • Feb 9, 2024 • 8 min read

Introduction
Prerequisites
1. Folder Structure
2. The Environment File
Deploy Umami Using Compose For Single Instance Deployment
1. Run the Compose Services
Deploy Umami Using Compose For Docker Swarm Deployment
1. Run the Docker Swarm Stack
Post Installation Tasks
Closing Notes

Introduction

Umami is an open-source, privacy-centric, and lightweight web analytics service built with JavaScript (NextJS) running in a NodeJS environment. It offers a fantastic alternative for those who want to be free from conventional analytics platforms that track your data and more importantly your visitors.

Another thing that makes Umami really special is its user-friendly design, making it the ideal choice for your self-hosting alternative to Google Analytics. In this tutorial, you will learn how to unleash Umami's potential in a straightforward setup using Docker Compose. There is no need for complicated configurations or complex processes because Umami is designed about simplicity to ensure an easy self-hosted experience.

Furthermore, Umami is open-source and grants you full visibility and control, allowing you to customize it according to your unique requirements.

If you think about analytics you are often concerned about privacy, aren't you? Luckily, Umami has got your back, prioritizing the protection of user data while delivering the analytics insights you desire.

Now, get ready to dive into this tutorial and learn how to set up Umami with Docker Compose. If you follow this and deploy it in your own cluster, you will not only have a privacy-focused analytics solution in your toolkit but also the satisfaction of being the master of your data.

Let's hope this tutorial empowers you to elevate your web analytics game with Umami – the go-to open-source option for privacy-conscious developers like yourself.

Prerequisites

This tutorial will deploy Umami either on a server running Docker (with Docker Compose) or within a server cluster utilizing Docker Swarm.

💡Note: Whenever you read “Docker Swarm” we are actually talking about “*Docker Swarm mode*”. (not the deprecated product Docker swarm)

To deploy this the tutorial assumes that we already:

have any kind of website to monitor
have Docker and Docker Compose installed
optionally have a server cluster utilizing Docker Swarm
have a domain to publish the deployed Umami instance

Unfortunately, this tutorial will not show how to configure a domain with a TLS certificate because I normally use Traefik to automatically create SSL-secured domains based on configuration in the Compose file. Luckily, I have created several tutorials about how to set up your own Traefik Proxy on your server, or your server cluster:

On your single instance server using Docker:

How to setup Traefik v2 with automatic Let’s Encrypt certificate resolver

On your server cluster using Docker Swarm:

4 Important Services Everyone Should Deploy In A Docker Swarm

Folder Structure

The first step is to create a folder that will contain all files needed to deploy Umami either on a single server or on a server cluster. The structure should look like this containing two files .env and docker-compose.yml.

umami/
|-- .env
|-- docker-compose.yml

The Environment File

Switch to the umami folder and create a new .env file containing the following snippet:

DATABASE_URL=postgresql://umami_user:umami_pass@db:5432/umami_db
DATABASE_TYPE=postgresql
HASH_SALT=generate_a_random_salt

POSTGRES_DB=umami_db
POSTGRES_USER=umami_user
POSTGRES_PASSWORD=umami_pass

In this snippet, we define the DATABASE_URL which Umami will use. As we can see it uses the same values as POSTGRES_DB, POSTGRES_USER, and POSTGRES_PASSWORD. Keep this in mind if you want to change it. Additionally, you should generate a random salt and put it into the HASH_SALT variable. To generate something just hit your keyboard or use this command:

openssl rand -base64 64

Deploy Umami Using Compose For Single Instance Deployment

Now, we can create our Docker Compose file which will be used to deploy the service. As a starting point for our configuration, we can download the Compose file from Umami's GitHub repository which will get some modifications:

We will remove the environment variables for the database because we extract them already into an .env file
We add Traefik configuration to use automatic TLS certificate generation for a URL
We add TRACKER_SCRIPT_NAME to avoid getting blocked by Ad-Blocker

The resulting Compose file to deploy a single instance Docker using Traefik Proxy looks like:

version: '3.7'
services:
  umami:
    image: ghcr.io/umami-software/umami:postgresql-latest
    env_file: .env
    environment:
      TRACKER_SCRIPT_NAME: getinfo
      API_COLLECT_ENDPOINT: all
      APP_SECRET: replace-me-with-a-random-string
    labels:
      - traefik.enable=true
      - traefik.docker.network=traefik-public
      - traefik.constraint-label=traefik-public
      - traefik.http.routers.umami-http.rule=Host(`umami.${PRIMARY_DOMAIN}`)
      - traefik.http.routers.umami-http.entrypoints=http
      - traefik.http.routers.umami-http.middlewares=https-redirect
      - traefik.http.routers.umami-https.rule=Host(`umami.${PRIMARY_DOMAIN}`)
      - traefik.http.routers.umami-https.entrypoints=https
      - traefik.http.routers.umami-https.tls=true
      - traefik.http.routers.umami-https.tls.certresolver=le
      - traefik.http.services.umami.loadbalancer.server.port=3000
    networks:
      - default
      - traefik-public
    depends_on:
      - db
    restart: always
  db:
    image: postgres:15-alpine
    env_file: .env
    networks:
      - default
    volumes:
      - db:/var/lib/postgresql/data
    restart: always
volumes:
  db:
networks:
  traefik-public:
    external: true
  default:
    external: false

Some important information:

In this Compose file, we added several labels that are needed to deploy the service using a Traefik Proxy which is installed within our Docker environment like it was described in this tutorial:

How to setup Traefik v2 with automatic Let’s Encrypt certificate resolver

By adding TRACKER_SCRIPT_NAME we can change the filename of the tracking script. This allows us to avoid being blocked by many ad blockers. I just called it getinfo to have some very general name. This setting is totally optional but I recommend doing it. See this GitHub discussion for more information
The database container is not accessible from the public network and only communicates with the umami web container through the db network
umami web container is part of the external network traefik-public (see previous post) and the default network to communicate with the db container.
All data is persisted within a named volume called db

Run the Compose Services

After all files are created we have to define our PRIMARY_DOMAIN environment variable by exporting it:

export PRIMARY_DOMAIN=paulsblog.dev

Then, we can start the container by executing:

docker-compose up -d

💡 Note: If using a newer version of Docker and Docker Compose the command could be docker compose up -d

Switch to https://umami.PRIMARY\_DOMAIN and log in with the default credentials: admin:umami

To stop the container at any time we have to switch to the umami folder and execute the following:

docker-compose down

Deploy Umami Using Compose For Docker Swarm Deployment

To set up Umami within a Docker Swarm environment we have to adjust the previously explained Docker Compose file by adding the deploy keyword. Additionally, we have to add placement constraints for the database service to guarantee that it is deployed on the correct server within the Docker Swarm cluster.

The resulting Docker Compose file will be:

version: '3.7'
services:
  umami:
    image: ghcr.io/umami-software/umami:postgresql-latest
    env_file: .env
    environment:
      TRACKER_SCRIPT_NAME: getinfo
      API_COLLECT_ENDPOINT: all
      APP_SECRET: replace-me-with-a-random-string
    deploy:
      labels:
        - traefik.enable=true
        - traefik.docker.network=traefik-public
        - traefik.constraint-label=traefik-public
        - traefik.http.routers.umami-http.rule=Host(`umami.${PRIMARY_DOMAIN}`)
        - traefik.http.routers.umami-http.entrypoints=http
        - traefik.http.routers.umami-http.middlewares=https-redirect
        - traefik.http.routers.umami-https.rule=Host(`umami.${PRIMARY_DOMAIN}`)
        - traefik.http.routers.umami-https.entrypoints=https
        - traefik.http.routers.umami-https.tls=true
        - traefik.http.routers.umami-https.tls.certresolver=le
        - traefik.http.services.umami.loadbalancer.server.port=3000
    networks:
      - default
      - traefik-public
    depends_on:
      - db
    restart: always
  db:
    image: postgres:15-alpine
    env_file: .env
    networks:
      - default
    deploy:
      placement:
        constraints:
          - node.labels.umami.db == true
    volumes:
      - db:/var/lib/postgresql/data
    restart: always
volumes:
  db:
networks:
  traefik-public:
    external: true
  default:
    external: false

In this file, we have moved all labels below the deploy keyword to enable Traefik configuration while utilizing a Docker Swarm environment. Furthermore, we have added the deploy keyword containing placement constraints in the db service:

    deploy:
      placement:
        constraints:
          - node.labels.umami.db == true

This constraint will ensure that the db service is always deployed on the worker node within the Docker Swarm which has the corresponding label. To set the label (if not already done) we have to execute the following command on the Docker Swarm Manager Node:

docker node update --label-add umami.db=true ID_OF_NODE_TO_USE

💡 Note: ID_OF_NODE_TO_USE should be the node of the server that will be used. Use docker node ls on the manager node to find the IDs.

Run the Docker Swarm Stack

To deploy the service in our Docker Swarm we have to export the PRIMARY_DOMAIN by using export PRIMARY_DOMAIN=paulsblog.dev in our CLI and then execute:

docker-compose config | docker stack deploy -c - NAME_OF_STACK

This command will use the Compose config command to load the .env file settings into the Compose file before using docker stack deploy to deploy the stack in our Docker Swarm environment. Change NAME_OF_STACK to an appropriate name lime umami

💡Note: If you for any reason change the name of the Compose file from docker-compose.yml to something like docker-compose.umami.yml (or anything) the deploy command will fail. *It only works correctly if you keep the name!*

To stop the stack use docker stack rm NAME_OF_STACK

Post Installation Tasks

Now, as our Umami instance is running we should log in and instantly change our Umami credentials from admin:umami to something more secure.

Then we use the Add Site button in the top right corner to add our website to track

Adding a new website to Umami analytics

Now, we can switch to Tracking code in the tab layout.

The tab layout menu in the top of Umami analytics website

The tracking code looks like this:

<script async src="https://umami.paulsblog.dev/getbooks" data-website-id="sd2dasa-5231-7h7f-9jj8-15ab565f8af9"></script>

I would recommend adding the defer keyword to it and then insert it in the header section of your website. Afterward, tracking of our website will start and we can instantly see our visitors in the real-time view in Umami

Umami analytics realtime dashboard for paulsblog.dev

Closing Notes

To sum up, Umami really impresses me with its straightforward approach to website analytics, particularly when compared with other self-hosted alternatives like Countly, which I had previously used and found to be overly bloated. Furthermore, one of Umami's standout features is its straightforward setup process using Docker, making it accessible even to those Developers less familiar with advanced technical concepts.

Another important aspect of Umami is its trustworthy option, prioritizing user data protection without sacrificing functionality. As a software developer, it is paramount to understand that selecting analytics tools that not only provide valuable insights but also prioritize the security and privacy of user information is mandatory. By choosing Umami, we can ensure that our analytics remain both effective and ethically correct in today's digital landscape.

Do you have any questions regarding this tutorial? I would love to hear your thoughts and answer your questions. Please share everything in the comments.

Feel free to connect with me on Medium, LinkedIn, Twitter, and GitHub.

Thank you for reading, and happy analyzing your data!

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Photo by Stephen Dawson / Unsplash

My newest article :)

Paul Knulst — Fri, 21 Mar 2025 09:13:02 +0000

How To Manage Docker Logfiles with Logrotate

Paul Knulst ・ Mar 21

#programming #docker #webdev #softwareengineering

How To Manage Docker Logfiles with Logrotate

Paul Knulst — Fri, 21 Mar 2025 07:32:59 +0000

It is essential to configure log rotation for Docker containers. Log rotation is not performed by default, and if it’s not configured, logs on the Docker host can build up and eat up disk space. This guide will teach us how to set up Docker log rotation.

Paul Knulst in Docker • Dec 2, 2022 • 5 min read

Introduction
The Problem
A Brief On Docker Logs
Where are Docker logs stored?
Docker Log Rotation To The Rescue
Configuring Log Rotation For Specific Containers
1. Log Rotation With Docker Run
2. Log rotation in Docker Compose
Closing Notes

Introduction

Logs are important and are an essential piece of telemetry data. Logs contain all kinds of events that take place in your application and can be used to debug or troubleshoot issues. Often it is possible to identify the root cause of a problem by having good logs.

Using Docker and Docker Compose to containerize and scale applications is very easy nowadays. Unfortunately, operation complexity has increased, and frequently changing container-based environments are challenging to monitor.

Luckily Docker can display container logs that are generated through stdout and stderr with different logging drivers (local, json-file, syslog, awslog, etc). As default, Docker uses the json-file logging driver. This logging driver caches container logs and saves them to the disk.

But before deep-diving into Docker logs, adjusting the default Docker log settings, and enabling Docker log rotation I will briefly describe what happened to me and why it could lead to big problems for you if you do not enable Logrotate for your container.

The Problem

It is 10:30 pm and I am working on my private Docker Swarm cluster. I just finished a cool new service and start to build and deploy it. I already closed every terminal, exited the IDE, and want to turn off my computer to finally relax. But, THE BUILDS ARE FAILING. Oh no...

I cannot push or pull to my self-hosted private docker registry. But the containers are still running without any problem.

I asked myself why it is not possible to read/write to my registry until I realized that the disk is full. I opened a terminal and connect to my machine where I checked how much space is left. Unfortunately, nothing...

As I did not have any clue I searched on my disk which folder is using so much space and after some time it turns out the Docker folder is big. Nearly 400GB big. Why is it so big?

The answer: logs. a LOT of logs. Really, a LOT!

Unfortunately, I was not able to simply delete the Docker folders because it could harm my services so I was forced to research Docker logs and how I am able to fix my problem.

In the end, I fixed it on the same day (But, it was really close to midnight)

A Brief On Docker Logs

In Docker are two types of log files.

Docker daemon logs: Generated by the Docker daemon and saved on the host. This log file provided information about the state of the Docker platform.
Docker container logs: Docker container logs contains every log entry related to any application running in a container.

Unfortunately, Docker does not have any size restriction on logging files. This means they will increase over time and consume more and more storage if left unhandled. Every service that stays online and produces a large number of log messages, will consume more disk space over time.

Where are Docker logs stored?

Normally, logs are kept on the Docker host because the containers are stateless which means they won't save data if they are rebuilt/restarted. As default, Docker uses the json-file logging driver which records all stdout/stderr output in JSON format and create a file for each container at:

/var/lib/docker/containers/[container-id]/[container-id]-JSON.log

With this information in mind, I was able to delete the biggest log files and free up 200GB of disk space. Funnily, it was only 1 container that had this amount of logs but it was running for nearly one year and I totally forgot about it.

Docker Log Rotation To The Rescue

To be able to rotate logs produced by the Docker container I first had to configure the Docker daemon to use a particular log driver. This could be done by editing the daemon.json on the Docker host.

Step 1: Switch to the Docker daemon configuration file:

On Linux: /etc/docker/
On Windows: C:\ProgramData\docker\config\daemon.json

Step 2: If no logging driver is set, use the following command to set it up:

{  "log-driver": "json-file"}

You can also use another log driver but I prefer json-file. Docker recommends local because it performs log rotation by default.

Step 3: As I choose json-file I have to add an additional configuration for log rotation:

{  
    "log-driver": "json-file",  
    "log-opts": {    
        "max-size": "15m",    
        "max-file": "5",    
    }
}

Step 4: Save the file and restart the docker daemon:

sudo systemctl restart docker

Although the daemon.json was updated and the Docker daemon was restarted it will not work for any running container because the default logging driver will only impact containers that are created after modifying it. Existing containers will still work with the initial configuration which was json-file without any log rotation!

In order to enable log rotating for existing containers, they have to be recreated. Afterward, they will use log rotation!

Configuring Log Rotation For Specific Containers

Instead of adjusting the Docker daemon's default logging driver, it is possible to configure a single Docker container to use a different logging driver and enable log rotation. This can be done while creating the container with Docker run and within a Docker Compose file.

Log Rotation With Docker Run

To run a new container with another logging driver you need to include the --log-driver flag while using docker run:

docker run -it --log-driver json-file nginx

This snippet defines json-file as the logging driver and does not configure log rotation.

To find out which logging driver is currently used by any container you can use the following command:

$ docker inspect -f '{{.HostConfig.LogConfig.Type}}' <CONTAINER>

To set up a json-file logging driver with log rotation enabled use this command:

docker run  --log-opt max-size=15m --log-opt max-file=5 nginx:latest

Simplifying the above command:

--log-opt max-size=15m: Set the maximum file size for your Docker log file.

--log-opt max-file=5: Sets the maximum amount of log files to use. If the limit is reached, Docker will delete the oldest one and starts from the beginning.

nginx:latest: The container image name

Log rotation in Docker Compose

If you are using Docker Compose to run your Docker containers you can open up any Compose file and add a new section containing the logging driver and log rotate settings:

  webserver:
    image: nginx:latest
    logging:
      driver: "json-file"
      options:
        max-file: 5
        max-size: 15m

Simplifying the Compose section:

image: nginx:latest: The container image name

driver: Sets the logging driver to use

max-size: 15m: Set the maximum file size for your Docker log file.

max-file: 5: Sets the maximum amount of log files to use. If the limit is reached, Docker will delete the oldest one and starts from the beginning.

Closing Notes

In this guide, I showed how a small incident in my Docker environment forced me to learn about Docker log rotation.

Luckily, it was very easy to set up log rotation for my Docker environment and have every new container use this approach. For existing containers, I manually had to adjust the Compose file and redeploy them as I use a Docker Swarm environment.

I hope this article gave you a quick and neat overview of how to enable log rotation in your own Docker environment.

Keep in mind that Docker container environments are highly dynamic with multiple layers of abstraction and it's hard to debug such environments. Because of this, many applications create an enormous amount of logs which play a critical role in providing information to identify issues. You will always need these logs and you should not deactivate them!

How about you? Do you use log rotation for your Docker containers? Do you have any other approach? Also, do you have any questions regarding Docker logs? I would love to hear your thoughts. Please share everything in the comments.

This article was originally published on my blog at https://www.paulsblog.dev/how-to-manage-docker-logfiles-with-logrotate/

Feel free to connect with me on my blog, Medium, LinkedIn, Twitter, and GitHub.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Image by Freepik

How To Unhide Titlebars on Maximised Windows in KDE Plasma 6

Paul Knulst — Tue, 19 Mar 2024 11:40:49 +0000

Maybe Unimportant For You But Mandatory For Me

Since I updated my Garuda Linux some days ago it switched from Plasma 5 to Plasma 6 which removed a Widget that I was using in my top panel: "Application Menu"

This led to a major problem (for me) if I maximized my applications because instead of an application menu the Global Menu Widget was used. This can be seen in the following screenshot while maximizing the Brave browser:

The problem was that the Global Menu Widget missed some adjustments that I needed:

Extend the Menu to use the whole panel
Enable Double Click to reset Full Screen
Enable Click And Drag to move the Window to another screen maybe

Normally, this wouldn't be a problem because every application has its own title bar. Unfortunately, the default behavior for my installation of Garuda Linux was configured that if I maximized any Window it would hide the title bar which led to the problem that for some applications I couldn't minimize, move or close them through the top bar. See the marked spots on the following screenshot:

Note: If using a Panel with the Global Menu Widget and Window Buttons Widget (see screenshot) I would have the functionality to minimize, and close if using the buttons.

Fortunately, I found out that there exists a hidden setting in KWin that can be set to enable and also disable title bars and window borders if the application window is maximized.

To do this, I had to open the following file:

~/.config/kwinrc

In this file, I searched for the section [Windows] and changed the setting of BorderlessMaximizedWindows to false so that it looked like this:

[Windows]
BorderlessMaximizedWindows=true

After logging out and logging in back again the title bars were available all the time and I could maximize on double click and move the window without any problems.

As this was triggering me the last few days and I searched some days to fix this problem I decided to create a small post for everyone who will ever face this problem.

Also if you are one of these guys that do not want to have title bars at all, you can use this setting to remove them.

So I guess, it is important for everyone :)

This article was originally published on my blog at https://www.paulsblog.dev/how-to-unhide-titlebars-on-maximised-windows-in-kde-plasma-6/

Feel free to connect with me on my blog, Medium, LinkedIn, Twitter, and GitHub.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Photo by master1305 / Freepik

How To Set Up A Mailserver Within A Docker Swarm

Paul Knulst — Fri, 19 Jan 2024 07:48:26 +0000

Ever wanted to have your own mail server? Learn how to set up your own personal mail server with this step-by-step guide

I run my own mail server to have generalized email addresses for different services.

While I searched for a nice solution to put my mail server into a dockerized mail server I found the famous docker-mailserver. Unfortunately, I could not use this mail server because I had many errors while configuring it.

Because of this, I searched for an alternative and found hardware-mailserver which is something like an optimized usage of a docker-mailserver with multiple predefined functionalities.

If I had a normal docker environment without any other services I could use this docker-compose.yml and start it by executing:

docker-compose up -d

Because I run a Docker Swarm environment with a Traefik load balancer that creates SSL certificates for my domains I have to make some adjustments within the Compose file to set up and configure the mail server. Every adjustment will be explained later on service by service.

Mailserver

mailserver:
    image: hardware/mailserver:1.1-stable
    restart: ${MAILSERVER_RESTART_MODE}
    domainname: ${MAILSERVER_DOMAIN}
    hostname: ${MAILSERVER_HOSTNAME} 
    deploy:
        placement:
          constraints:
            - node.role == manager
        labels:
          - traefik.enable=true
          - traefik.docker.network=traefik-public
          - traefik.constraint-label=traefik-public
          - traefik.frontend.rule=Host:mail.${MAILSERVER_DOMAIN}
          - traefik.http.routers.mailserver-spam-http.rule=Host(`${MAILSERVER_FQDN?Variable not set}`) || Host(`pop3.${MAILSERVER_DOMAIN?Variable not set}`) || Host(`smtp.${MAILSERVER_DOMAIN?Variable not set}`) || Host(`imap.${MAILSERVER_DOMAIN?Variable not set}`) || Host(`${MAILSERVER_DOMAIN_RSPAMD?Variable not set}`)
          - traefik.http.routers.mailserver-spam-http.entrypoints=http
          - traefik.http.routers.mailserver-spam-http.middlewares=https-redirect
          - traefik.http.routers.mailserver-spam-https.rule=Host(`${MAILSERVER_FQDN?Variable not set}`) ||  Host(`pop3.${MAILSERVER_DOMAIN?Variable not set}`) || Host(`smtp.${MAILSERVER_DOMAIN?Variable not set}`) || Host(`imap.${MAILSERVER_DOMAIN?Variable not set}`) || Host(`${MAILSERVER_DOMAIN_RSPAMD?Variable not set}`)
          - traefik.http.routers.mailserver-spam-https.entrypoints=https
          - traefik.http.routers.mailserver-spam-https.tls=true
          - traefik.http.routers.mailserver-spam-https.tls.certresolver=le
          - traefik.http.services.mailserver-spam.loadbalancer.server.port=11334
    ports:
      - "25:25"         # SMTP                - Required
      - "110:110"     # POP3           STARTTLS - Optional - For webmails/desktop clients
      - "143:143"     # IMAP           STARTTLS - Optional - For webmails/desktop clients
      - "465:465"     # SMTPS         SSL/TLS  - Optional - Enabled for compatibility reason, otherwise disabled
      - "587:587"     # Submission  STARTTLS - Optional - For webmails/desktop clients
      - "993:993"     # IMAPS          SSL/TLS  - Optional - For webmails/desktop clients
      - "995:995"     # POP3S          SSL/TLS  - Optional - For webmails/desktop clients
      - "4190:4190" # SIEVE            STARTTLS - Optional - Recommended for mail filtering
    environment:
      - FQDN=${MAILSERVER_FQDN}
      - DOMAIN=${MAILSERVER_DOMAIN}
      - DBPASS=${MAILSERVER_DATABASE_USER_PASSWORD}       # MariaDB database password (required)
      - RSPAMD_PASSWORD=${MAILSERVER_RSPAMD_PASSWORD}     # Rspamd WebUI password (required)
    #- ADD_DOMAINS=aa.tld, www.bb.tld...      # Add additional domains separated by commas (needed for dkim keys etc.)
      - ENABLE_POP3=true                       # Enable POP3 protocol
    #
    # Full list : https://github.com/hardware/mailserver#environment-variables
    #
    volumes:
      - mail:/var/mail
      #- ./cert:/etc/letsencrypt/live/${MAILSERVER_FQDN}
    depends_on:
      - mariadb
      - redis
    networks:
      - traefik-public
      - default

This is the main service used by the mail server suite. The most important thing I had to add was the environment variables. Because I run a Docker Swarm setup hostname does not work correctly and I have to develop something else. Within a pull request on the GitHub page, I found a possible solution.

I have to add FQDN and DOMAIN as environment variables. Another section I have to change is the labels section. I added the deploy section and created two important properties:

placement-constraint and
labels.

placement-constraints are used to always deploy the mail server on my manager node and labels is filled with all information that is needed for my Traefik instance.

Finally, I have to change the volume entries. I want to use docker volumes instead of a local shared folder (which is achieved by adding a ./ in front of the volume name)

Also, there is one very important entry within the volumes section:

      #- ./cert:/etc/letsencrypt/live/${MAILSERVER_FQDN}

I will explain what it does and why its comment out later in detail

Postfix-admin

  postfixadmin:
    image: hardware/postfixadmin
    restart: ${MAILSERVER_RESTART_MODE}
    domainname: ${MAILSERVER_DOMAIN}
    hostname: ${MAILSERVER_HOSTNAME}
    deploy:
        placement:
          constraints:
            - node.role == manager
        labels:
          - traefik.enable=true
          - traefik.docker.network=traefik-public
          - traefik.constraint-label=traefik-public
          - traefik.http.routers.mailserver-postfixadmin-http.rule=Host(`${MAILSERVER_DOMAIN_POSTFIXADMIN?Variable not set}`)
          - traefik.http.routers.mailserver-postfixadmin-http.entrypoints=http
          - traefik.http.routers.mailserver-postfixadmin-http.middlewares=https-redirect
          - traefik.http.routers.mailserver-postfixadmin-https.rule=Host(`${MAILSERVER_DOMAIN_POSTFIXADMIN?Variable not set}`)
          - traefik.http.routers.mailserver-postfixadmin-https.entrypoints=https
          - traefik.http.routers.mailserver-postfixadmin-https.tls=true
          - traefik.http.routers.mailserver-postfixadmin-https.tls.certresolver=le
          - traefik.http.services.mailserver-postfixadmin.loadbalancer.server.port=8888
    environment:
      - DBPASS=${MAILSERVER_DATABASE_USER_PASSWORD}
      - FQDN=${MAILSERVER_FQDN}
      - DOMAIN=${MAILSERVER_DOMAIN}
    depends_on:
      - mailserver
      - mariadb
    networks:
      - traefik-public
      - default

Like I did before I have to add environment variables FQDN and DOMAIN, adjust the placement-constraint, labels and updated the volumes.

Rainloop

  rainloop:
    image: hardware/rainloop
    restart: ${MAILSERVER_RESTART_MODE}
    deploy:
        placement:
          constraints:
            - node.role == manager
        labels:
          - traefik.enable=true
          - traefik.docker.network=traefik-public
          - traefik.constraint-label=traefik-public
          - traefik.http.routers.mailserver-rainloop-http.rule=Host(`${MAILSERVER_DOMAIN_RAINLOOP?Variable not set}`)
          - traefik.http.routers.mailserver-rainloop-http.entrypoints=http
          - traefik.http.routers.mailserver-rainloop-http.middlewares=https-redirect
          - traefik.http.routers.mailserver-rainloop-https.rule=Host(`${MAILSERVER_DOMAIN_RAINLOOP?Variable not set}`)
          - traefik.http.routers.mailserver-rainloop-https.entrypoints=https
          - traefik.http.routers.mailserver-rainloop-https.tls=true
          - traefik.http.routers.mailserver-rainloop-https.tls.certresolver=le
          - traefik.http.services.mailserver-rainloop.loadbalancer.server.port=8888
    volumes:
      - rainloop:/rainloop/data
    depends_on:
      - mailserver
      - mariadb
    networks:
      - traefik-public
      - default

I only add the deploy section and change volumes.

MariaDB

  mariadb:
    image: mariadb:10.2
    hostname: mariadb.db
    restart: ${MAILSERVER_RESTART_MODE}
    # Info : These variables are ignored when the volume already exists (if databases was created before).
    environment:
      - MYSQL_RANDOM_ROOT_PASSWORD=yes
      - MYSQL_DATABASE=postfix
      - MYSQL_USER=postfix
      - MYSQL_PASSWORD=${MAILSERVER_DATABASE_USER_PASSWORD}
    deploy:
      placement:
        constraints:
          - node.role == manager
    volumes:
      - sql:/var/lib/mysql
    networks:
      - default

The MariaDB container gets a placement-constraint so I don't lose data and docker volumes that are used. Additionally, it is very important that the MYSQL_PASSWORD is the same as defined within the mailserver-service.

Redis

  redis:
    image: redis:4.0-alpine
    hostname: redis.db
    restart: ${MAILSERVER_RESTART_MODE}
    command: redis-server --appendonly yes
    deploy:
        placement:
            constraints:
              - node.role == manager
    volumes:
      - redis:/data
    networks:
      - default

The last service is the Redis container which gets a placement-constraint and adjusted volumes.

Going Further

After the services were adjusted within one file (docker-compose.mailserver.yml) the next step was to deploy the Docker Swarm stack. Because I declared several environment variables I had to export them first (and I have to export them every time I recreate the service):

export MAILSERVER_DOMAIN=$PRIMARY_DOMAIN
export MAILSERVER_HOSTNAME=mail
export MAILSERVER_DOMAIN_POSTFIXADMIN=postfixadmin.$MAILSERVER_DOMAIN
export MAILSERVER_DOMAIN_RSPAMD=mail-spam.$MAILSERVER_DOMAIN
export MAILSERVER_DOMAIN_RAINLOOP=emails.$MAILSERVER_DOMAIN
export MAILSERVER_FQDN=$MAILSERVER_HOSTNAME.$MAILSERVER_DOMAIN
export MAILSERVER_DATABASE_USER_PASSWORD=lkhsaf98oihn53oysf9usaih5nb3rysa98fh3lnoiushdfisdugfbn32thsd9gpiesgs
export MAILSERVER_RSPAMD_PASSWORD=wsdofytesdf#*TYFkjhasd
export MAILSERVER_DOCKER_TAG=1.1-stable
export MAILSERVER_RESTART_MODE=unless-stopped

After exporting these variables I could deploy the mail server to my Docker Swarm by executing:

docker stack deploy -c docker-compose.mailserver.yml mailserver

Now the mail server was deployed BUT it does not use SSL. This is because the certificates are generated by traefik the first time AFTER the website was visited for the first time. Because I declared within the mailserver-service that there is a domain called mail.$PRIMARY_DOMAIN. I opened this website and it showed rspam BUT it also creates the SSL certificate which is needed for the mail server.

How To Use SSL?

For the prior created SSL certificates I need a function to transfer them to the mail server. Unfortunately, the automatic transfer process from the mail server could not be used because I installed the second version of Traefik.

I had to create a small script that uses dumpcerts (from the mailserver GitHub) for extracting certs from traefik-acme.json and storing them into a file that the mail server can use to have an SSL certificate:

#!/bin/sh

mkdir -p /tmp/dump

./dumpcerts.traefik.v2.sh ./acme.json /tmp/dump
cp /tmp/dump/certs/$MAILSERVER_FQDN.crt cert/fullchain.pem
cp /tmp/dump/private/$MAILSERVER_FQDN.key cert/privkey.pem

This script assumes that the acme.json file that traefik uses to store certificates is located in the same folder as the mail server. The dumpcert script can be found within the GitHub repository.

If the scripts exit without error the mail server can be turned down and redeployed (because SSL data is only checked at the start of the mail server stack ).

Additionally, I created an update script that uses all commands together:

./update-cert
docker stack rm mailserver
docker stack deploy -c docker-compose.mailserver.yml mailserver

Unfortunately, you have to do this every three months because traefik generated certs only have a life span of three months.

After install stuff

If everything is started the next task is configuring postfix-admin and rainloop. There are two very easy info pages for this where you can find all information about configuring these services: rainloop-initial, postfix-initial

Furthermore, the DNS setup is required so that the mail server works! This step is very important.

Notes:

Make sure that the PTR record of your IP matches the FQDN (default: mail.domain.tld) of your mail server host. This record is usually set in your web hosting interface.
DKIM, SPF, and DMARC records are recommended to build a good reputation score.
DMARC record can be created on Dmarcian.com
SPF needs the public IP of the mail server!: v=spf1 a mx ipYOURMAINHERE ~all
The DKIM public (mail._domainkey) key will be available on the host after the container startup here:

/var/lib/docker/volumes/mailserver_mail/_data/dkim/domain.tld/public.key

Closing Notes

Now the mail server is running and you can use postfix admin (postfixadmin.yourdomain.de) to create accounts that can be accessed with Rainloop (webmail.yourdomain.de) or another email client (Thunderbird/Betterbird).

I hope you find this tutorial helpful and are now able to add a mail server to your Docker Swarm environment.

Also, if you have any questions, ideas, recommendations, or want to share cool Docker commands or tools, please contact me. I try to answer your question if possible and will test your recommendations.

This article was originally published on my blog at https://www.paulsblog.dev/how-to-set-up-a-mailserver-within-a-docker-swarm/

Feel free to connect with me on my blog, Medium, LinkedIn, Twitter, and GitHub.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Photo by Wayhomestudio / Freepik

4 Important Services Everyone Should Deploy In A Docker Swarm

Paul Knulst — Sat, 29 Jul 2023 08:28:58 +0000

Enhance your Docker Swarm with four important services that you will love: Traefik, Portainer, Registry, FTP

In a previous article, I showed how a Docker Swarm is set up in ~15 minutes.

Remember: Whenever you read “Docker Swarm” we are talking about “Docker in Docker Swarm mode”

Within this article, I will show and explain four services everyone should use in their Docker Swarm: traefik, Portainer, Docker-Registry, FTP.

Traefik

Makes Networking Boring
Cloud-Native Networking Stack That Just Works.

One important service in any Docker Swarm is traefik. I use this for assigning domains/subdomains to every docker service. A simple docker-compose.yml can be found on my FTP here. To use this file it is necessary to update the labels for your docker swarm by executing:

export NODE_ID=$(docker info -f '{{.Swarm.NodeID}}')
docker node update --label-add traefik-public.traefik-public-certificates=true $NODE_ID

These two commands assume that you want to have traefik on your manager node! To understand what a manager node have a look at the docker documentation.

Another VERY important step is creating a file named acme.json within the folder where the docker-compose will be stored and do:

chmod 600 acme.json

Furthermore, you have to define a TRAEFIK_USERNAME, TRAEFIK_HASHED_PASSWORD, TRAEFIK_DOMAIN and TRAEFIK_SSLEMAIL:

export TRAEFIK_USERNAME=admin
export TRAEFIK_HASHED_PASSWORD=$(openssl passwd -apr1 testpassword)
export TRAEFIK_DOMAIN=dashboard.YOUR_DOMAIN.tld
export TRAEFIK_SSLEMAIL=your_email@address.de

The last command which has to be executed before you can deploy the service is creating the traefik-public network which will be used across all containers:

docker network create --driver=overlay traefik-public

After this is done it is possible to deploy traefik to your docker swarm:

docker stack deploy -c docker-compose.traefik.yml

Just test it by launching https://dashboard.YOUR_DOMAIN.tld. This domain will use an SSL certificate and you can log in with admin/testpassword if you followed the above instructions.

Portainer

Portainer is a powerful, GUI-based Container-as-a-Service solution that helps organizations manage and deploy cloud-native applications easily and securely.

My docker-compose.yml for Portainer is not special. You can find it here if you need it. Normally there are no differences between my and the one you find if you google for Portainer service.

If you use mine you have to *add a label to your manager!* *This is very important* because Portainer needs a connection to your docker socket. You can add the label with these commands:

export NODE_ID=$(docker info -f '{{.Swarm.NodeID}}')
docker node update --label-add portainer.portainer-data=true $NODE_ID

Furthermore declare the PORTAINER_DOMAIN which will be used.

export PORTAINER_DOMAIN=portainer.$PRIMARY_DOMAIN

After you have done this you can deploy Portainer:

docker stack deploy -c docker-compose.portainer.yml portainer

Timing Note: Make sure you log in and create your credentials soon after Portainer is ready, or it will automatically shut down itself for security. If you didn’t create the credentials on time and it shut down itself automatically, you can force it to restart with:

docker service update portainer_portainer --force

Docker Registry

The Registry is a stateless, highly scalable server side application that stores and lets you distribute Docker images. The Registry is open-source, under the permissive Apache license.

The docker registry is important to have in a swarm environment if you don't want to upload your code/data to a public registry. If you don't have any private registry for yourself you have to upload the resulting image to docker-hub each time you extend an official image with your application code. While this is great if you only want to extend functionality in a general way it is not advisable if you copy your website with your closed source code into the image.

Imagine you have a modified Nginx image where the HTML folder is copied into the image:

FROM nginx
COPY html /usr/share/nginx/html

In many cases the html folder will contain your website which you don't want someone to use. If you have a private registry you can build the image and upload it but if you don't have one you have to use a public registry.

That's why I think that every swarm needs to have access to a private registry. And that's why I created one. Normally you can set up a registry with a simple docker run command but I wanted to have a registry that can be reached from everywhere. Because of that, I created a registry that could be accessed by username and password.

My personal docker-compose.yml can be downloaded here. It will run within my Traefik environment and create a new registry that I can use from every node in my swarm. To deploy the service you have to declare some environment variables which are used while deploying: REGISTRY_USERNAME, REGISTRY_HASHED_PASSWORD and REGISTRY_HOST:

export REGISTRY_USERNAME=reg_adm
export REGISTRY_HASHED_PASSWORD=$(openssl passwd -apr1 regsupersecret)
export REGISTRY_HOST=reg.YOUR_DOMAIN.tld

With these variables you can deploy the service:

docker stack deploy -c docker-compose.registry.yml registry

To use the registry you have to do two more things. The first thing is a quality of life feature to easily change the domain of the registry without affecting every container which uses an image from the private registry. *Add DOCKER_REGISTRY to .profile for* ****root**** *or whatever user you are using so it is known if docker-compose files should be pushed/downloaded*. The second task is very important. You have to execute docker login on every node of your swarm so that every node is allowed to pull images.

Now you can start using your private registry in docker-compose.yml. If you created a DOCKER_REGISTRY environment variable you can use it like this in your docker-compose.yml:

myapp:
    image: ${DOCKER_REGISTRY}/simple-app
    build:
      context: ./
      dockerfile: Dockerfile

If you want to deploy a service that contains the above part you have to build and push your image to deploy it correctly:

docker-compose build
docker-compose push
docker stack deploy -c docker-compose.yml www

If you only build and deploy it other nodes in your swarm cannot pull the image and so the service cannot be deployed.

FTP

Another important service is an FTP server for saving files you want to use anywhere in any app or service you create while working with the swarm.

I decided to use “pure-ftp” because it was the one I found while googling for a nice FTP server that runs within a docker environment. Because I want to keep it simple I created an FTP server without any traefik configuration. BUT I did a simple trick in putting the FTP server together with a website in a docker-compose.yml. I have done this because I want to have the possibility to download files from the server over https which I uploaded with ftp.

After configuration, I came up with this docker-compose.yml. Within the file, you can see that I use a custom Dockerfile for the web service which contains:

FROM nginx
COPY html /usr/share/nginx/html

The trick I describe is just the defined volume within the docker-compose.yml. As you can see I defined data in both services. Within web-service I just have an extra folder within the Nginx HTML folder so that I can access it from the web. And within the ftp-service I define data as the place where user can upload their data.

Before it is possible to deploy this service you have to declare environment variables: FTP_USERNAME, FTP_PASSWORD, FTP_DOMAIN_FOR_CERT, FTP_ORG_FOR_CERT, FTP_COUNTRYCODE_FOR_CERT and WEBSERVICE_DOMAIN:

export WEBSERVICE_DOMAIN=www.MYDOMAIN.tld
export FTP_USERNAME=SUPERUSER
export FTP_PASSWORD=clearTextPW
export FTP_DOMAIN_FOR_CERT=$WEBSERVICE_DOMAIN
export FTP_ORG_FOR_CERT=mybusiness
export FTP_COUNTRYCODE_FOR_CERT=DE

Furthermore, you have to add a label to any node of your swarm. To achieve this use docker node ls to find out the ID from every node and execute:

docker node update --label-add www.ftp-data=true ID_OF_NODE_TO_USE

After this is done you can safely deploy your website with enabled FTP

docker stack deploy -c docker-compose.web.yml webandftp

Now it is possible to connect with an FTP client to your WEBSERVICE_DOMAIN and upload a file (test.txt) which then can be accessed by this URL: WEBSERVICE_DOMAIN/data/test.txt

The whole FTP docker service can be downloaded from my GitHub:

https://github.com/paulscode-de/ftp-server

It is very important to say that you only can connect with your WEBSERVICE_DOMAIN if this domain also has an A record to your manager node!

5. Closing Notes

I hope you find this article helpful and can use my provided files to set up these services within your own Docker Swarm.

In my humble opinion, these four services should be present in every Docker Swarm environment because they are mandatory (or exchanged with function-like services).

This article was originally published on my blog at https://www.paulsblog.dev/services-you-want-to-have-in-a-swarm-environment/

Feel free to connect with me on my blog, Medium, LinkedIn, Twitter, and GitHub.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Photo by frank mckenna / Unsplash

Deploy Free Figma Alternative Penpot With Docker

Paul Knulst — Tue, 06 Jun 2023 11:58:49 +0000

Paul Knulst in Docker • Oct 6, 2022 • 8 min read

As we all probably heard Adobe has acquired the popular design tool Figma for a whopping $20 billion. Unfortunately, this "strategy" of eliminating competition by acquiring businesses is common for big tech companies.

But, luckily, there is a free and open-source design tool that also does some things better. Also, it was inspired by Figma.

Penpot: Free & Open-Source Design Tool

Penpot is an open-source project that is actively developed and totally free to use for everyone.

Here are some major features that make Penpot interesting

Free and open-source (of course).
Option to Self-host (will be covered here primarily).
Cross-platform.
Using SVG as the native format.
Web-based.
Featuring industry-standard features (inspired by Figma).

Watch this official Penpot video to learn about the basics of Penpot:

A very important feature of Penpot is the use of SVG as their native format instead of PNG/JPG because it enables you to be compatible with many vector graphic editing tools.

This is especially useful because you will not get locked by being forced to use a proprietary file format that no other application can use.

Also, Penpot aims to use the absolute best of open standards that already exist. The CEO of Penpot, Pablo Ruiz-Múzquiz, mentions more about it:

If you go for SVG (open standard, web, mobile, etc) at the storage level, you can suddenly integrate all your Penpot designs with your code repos. You could make changes to the actual representation of the design itself thanks to SVG and not yet another closed format. That opens the door to massive opportunities for designers AND devs. Also, SVG means we are low-code ready for free. You can pick any element in Penpot and ask for its SVG (and CSS) representation knowing it's actually what it is, no translation. That brings a more trustworthy relationship between designers and devs and allows frontend devs to try out their design skillset.

As mentioned before a very important feature of Penpot is the ability to self-host it in a Docker container on your local machine and also on any server (that runs Docker). The following steps will show how easily you can use Docker to host your own Penpot to replace Figma as your design tool.

If you want to test Penpot before installing it as a self-hosted version you can go to https://design.penpot.app/ log in with GitHub/Gitlab/Google account or create n new account to test it. But I would avoid doing this and sharing your personal data because deploying it locally is done super fast (if you already have Docker installed)

Deploy Penpot locally with Docker

Install Docker

To deploy Penpot locally (or remotely) you need to have Docker installed. To install Docker on your system follow the official tutorial on docker.com. If you are using Windows and aren't allowed to install Docker Desktop you can follow this guide:

How To Install Docker Without Docker Desktop On Windows

Deploy Penpot

If Docker is installed it is very easy to start up a Penpot instance locally. To do this you can simply retrieve the latest Compose and config file from the official Penpot Github project by download them:

    wget https://raw.githubusercontent.com/penpot/penpot/main/docker/images/docker-compose.yaml
    wget https://raw.githubusercontent.com/penpot/penpot/main/docker/images/config.env

The config file from the GitHub project is quite big and contains several variables that you will never use working locally with Penpot. It would be sufficient to use the following config as it only contains mandatory values:

    # Standard database connection parameters (only postgresql is supported):
    PENPOT_DATABASE_URI=postgresql://penpot-postgres/penpot
    PENPOT_DATABASE_USERNAME=penpot
    PENPOT_DATABASE_PASSWORD=penpot

    # Redis is used for the websockets notifications.
    PENPOT_REDIS_URI=redis://penpot-redis/0

    ASSETS_STORAGE_BACKEND=assets-fs
    PENPOT_STORAGE_ASSETS_FS_DIRECTORY=/opt/data/assets

    PENPOT_TELEMETRY_ENABLED=false

    # Enable or disable external user registration process.
    PENPOT_REGISTRATION_ENABLED=false

Now that you downloaded (or created) both files switch to the folder and start the Docker service with:

    docker-compose -p penpot up -d

If the process finishes you can open http://localhost:9001 and will see the following screen if Penpot was installed correctly:


Login screen for Penpot after deploying it to localhost with Docker

Create a Penpot user

Although you deployed Penpot on your local machine you cannot create a user using the Web interface because you started Penpot without configuring an SMTP account to receive registration emails. However, this is no problem because you can simply use this Docker command to manually create an already-activated user:

    docker exec -it penpot-penpot-backend-1 ./manage.sh create-profile

IMPORTANT: Check if penpot-penpot-backend-1 is the correct name of your backend container. You can do this by executing docker ps.

After executing the command you could get the following output which is confusing:

    [2022-10-06 10:18:13.523] I app.config - hint="flags initialized", flags="backend-api-doc,secure-session-cookies,login,backend-worker,registration"
    [2022-10-06 10:18:14.295] I app.metrics - action="initialize metrics"
    Email:  [2022-10-06 10:18:14.334] I app.db - hint="initialize connection pool", name="main", uri="postgresql://penpot-postgres/penpot", read-only=false, with-credentials=true, min-size=0, max-size
    =30

Ignore everything and just type an email that you want to use in your local installation, then set your name and password. The resulting log will look like this:

    [2022-10-06 10:22:33.900] I app.config - hint="flags initialized", flags="backend-api-doc,secure-session-cookies,login,backend-worker,registration"
    [2022-10-06 10:22:34.636] I app.metrics - action="initialize metrics"
    Email:  [2022-10-06 10:22:34.675] I app.db - hint="initialize connection pool", name="main", uri="postgresql://penpot-postgres/penpot", read-only=false, with-credentials=true, min-size=0, max-size
    =30
    user@local.de
    Full Name:  Paul Knulst
    Password:
    User created successfully.

After your user was created you can log in to your Penpot installation at http://localhost:9001/ and start using Penpot.

Deploy Penpot remotely with Docker and Traefik

Having a running installation of Penpot locally is very useful if you are the only person working with it. Normally, you will work in teams where multiple designers, developers, or others collaborate together on different designs. To allow doing this with Penpot you can deploy Penpot on any server using Docker and Traefik.

Installing Traefik

To install Traefik on your server and use it to forward URLs to Docker service and automatically assign SSL certificates you can follow this guide:

How to setup Traefik v2 with automatic Let’s Encrypt certificate resolver

Keep in mind that you need at least one URL that points to the server where your Traefik instance will run. If you do not have any URL you have to buy one and map it to your server.

Updating Config

Deploying Penpot on a server with a specific URL where multiple users can work together needs some changes to the prior shown config file because the registration process should be enabled. Therefore, you have to add an email account that will be used by Penpot to send registration emails. A simple Google/GMX/Apple Mail account will be sufficient.

Change the PENPOT_REGISTRATION_ENABLED environment variable to true and add the following variables (with correct values) to the config file:

    PENPOT_REGISTRATION_DOMAIN_WHITELIST=""

    #Your public penpot url
    PENPOT_PUBLIC_URI=

    # Enable Email 
    PENPOT_SMTP_ENABLED=true
    PENPOT_SMTP_DEFAULT_FROM=
    PENPOT_SMTP_DEFAULT_REPLY_TO=
    PENPOT_SMTP_HOST=
    PENPOT_SMTP_PORT=
    PENPOT_SMTP_USERNAME=
    PENPOT_SMTP_PASSWORD=
    PENPOT_SMTP_TLS=
    PENPOT_SMTP_SSL=

An important setting here is PENPOT_REGISTRATION_DOMAIN_WHITELIST="" where you can add a comma-separated list of allowed email domains to register. If you leave it empty every domain is allowed. Find the full config in this gist on Github.

Adjust Compose file for Traefik usage

After updating the config file you need to update the Compose file and add the Traefik network and the labels for Traefik to automatically generate an SSL certificate after deploying Penpot. Copy the content of the following Compose file into yours:

    version: "3.5"

    networks:
      default:
        external: false
      traefik-public:
        external: true

    volumes:
      postgres_data:
      assets_data:

    services:
      frontend:
        image: "penpotapp/frontend:latest"
        hostname: frontend.penpot
        volumes:
          - assets_data:/opt/data
        env_file:
          - config.env
        depends_on:
          - penpot-backend
          - penpot-exporter
        networks:
          - default
          - traefik-public
        labels:
          - traefik.enable=true
          - traefik.docker.network=traefik-public
          - traefik.constraint-label=traefik-public
          - traefik.http.routers.penpot-http.rule=Host(`${PENPOT_URL}`)
          - traefik.http.routers.penpot-http.entrypoints=http
          - traefik.http.routers.penpot-http.middlewares=https-redirect
          - traefik.http.routers.penpot-https.rule=Host(`${PENPOT_URL}`)
          - traefik.http.routers.penpot-https.entrypoints=https
          - traefik.http.routers.penpot-https.tls=true
          - traefik.http.routers.penpot-https.tls.certresolver=le
          - traefik.http.services.penpot.loadbalancer.server.port=80
      backend:
        image: "penpotapp/backend:latest"
        hostname: backend.penpot
        volumes:
          - assets_data:/opt/data
        depends_on:
          - postgres
          - redis
        env_file:
          - config.env
        networks:
          - default

      exporter:
        image: "penpotapp/exporter:latest"
        environment:
          # Don't touch it; this uses internal docker network to
          # communicate with the frontend.
          - PENPOT_PUBLIC_URI=http://frontend.penpot
        networks:
          - default

      postgres:
        image: "postgres:13"
        restart: always
        stop_signal: SIGINT
        environment:
          - POSTGRES_INITDB_ARGS=--data-checksums
          - POSTGRES_DB=penpot
          - POSTGRES_USER=penpot
          - POSTGRES_PASSWORD=penpot
        volumes:
          - postgres_data:/var/lib/postgresql/data
        networks:
          - default

      redis:
        image: redis:6
        restart: always
        networks:
          - default

Deploy Penpot

Before you finally can deploy the Docker service and use Penpot with your coworkers/friends you have to set your URL. This has to be the same URL that you used in the config (without HTTPS). Also, the URL has to point to your server!

    export PENPOT_URL=your-website.com

Now, switch to your folder and deploy your Penpot service:

    docker-compose up -d

Switch to your URL and register a new user. You should get an email from Penpot and can use it with your coworkers.

Keep in mind that you still can deactivate registration by setting the appropriate environment variable within the config and redeploying the server. If you do this you can create new users by executing the following command and manually set username/password:

    docker exec -it penpot-penpot-backend-1 ./manage.sh create-profile

With this approach, you will not need an email account and still be able to work with others.

Closing Notes

Congratulations if you followed my approach you have just installed Penpot as your own Figma replacement on either your local machine or on a server to use with collaborators.

The full Compose file (and config) for remote deployment can be found within the GitHub Gist I created for this article.

This is the end of this tutorial. Hopefully, you are now able to set up your personal installation. If you still have questions about setting up Penpot as a replacement for Figma you can just ask in the comment section. Also, if you enjoyed reading this article consider commenting with your valuable thoughts! I would love to hear your feedback about my tutorial or Penpot in general.

Furthermore, share this article with fellow designers to help them to replace Figma with Penpot.

This article was originally published on my blog at https://www.paulsblog.dev/deploy-free-figma-alternative-penpot-with-docker/

Feel free to connect with me on my blog, Medium, LinkedIn, Twitter, and GitHub.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Photo by GuerrillaBuzz Crypto PR / Unsplash

Optimize Android App Development With Docker, SonarQube, Detekt, and MobSF

Paul Knulst — Mon, 08 May 2023 09:03:55 +0000

Improve the Security and Code Quality of Android apps by using SonarQube, Detekt, and MobSF combined in the Android software-quality-chain!

Paul Knulst in Android App Development • 13 min read

As an Android app developer, you should always want to implement the best code that is possible. Also, your app should be as secure as possible.

Code quality is very important for any kind of software and you should always try to optimize your implementation. This article shows and explains different quality tools that will be combined in an Android software-quality-chain to achieve better code quality, a more secure app, and improved maintainability.

tl;dr: Clone this GitHub repository into your Android project folder and execute the full chain with: sh software-quality-chain.sh. Afterward, open the URL that will be logged in to the console. Log in with admin:admin12345 and see your Android project report.

Prerequisites

Docker

To run the script Docker needs to be installed on your operating system.

Docker is a widely used platform for developing, shipping, and running all kinds of applications. It enables you to separate infrastructure from your applications to quickly deliver software from one machine to another.

Implementing software while using Docker often negates infrastructure problems like the common "works on my machine" problem:


Created with memegenerator - Disaster Kid

To install Docker on your system follow the official tutorial on docker.com. If you are using Windows and aren't allowed to install Docker Desktop you can follow this guide:

https://www.paulsblog.dev/how-to-install-docker-without-docker-desktop-on-windows/

Detekt

Detekt is

__a static code analysis tool for the Kotlin programming language. It operates on the abstract syntax tree provided by the Kotlin compiler.__

It can be implemented in any Kotlin-based Android app and can be used with a self-defined set of rules to check an app. Detekt is a mandatory prerequisite to executing the script and has to be installed within your Android App. Fortunately, this procedure is very easy and can be done in four simple steps:

1. Add Detekt to project build.gradle

dependencies { 
    [...]
    classpath("io.gitlab.arturbosch.detekt:detekt-gradle-plugin:1.17.0")
}

2. Create a detekt.yml with a defined set of rules.

Download a sample gradle detek.yml file here and store it within project_folder/detekt/detekt.yml

3. Apply the detekt plugin in app build.gradle

apply plugin: "io.gitlab.arturbosch.detekt"

4. Add detekt block to app build.gradle

detekt {
    toolVersion = "1.17.0"
    buildUponDefaultConfig = false
    allRules = false
    config = files("../detekt/detekt.yml")
    baseline = file("../detekt/baseline.xml")
    input = files("src/main/java/com")
    debug = false
    reports {
        html {
            enabled = true
            destination = file("build/reports/detekt.html")
        }
        xml {
            enabled = true
            destination = file("build/reports/detekt.xml")
        }
        txt.enabled = false
        sarif.enabled = false
    }

Other Tools Used

MobSF

Mobile Security Framework (MobSF)

is an automated, all-in-one mobile application (Android/iOS/Windows) pen-testing, malware analysis and security assessment framework capable of performing static and dynamic analysis.

Before continuing, it is necessary to understand mobile security and how to create an android app in a secure way. Luckily, OWASP (Open Web Application Security Project), a popular nonprofit organization that is known for community-led open-source software projects and for improving software security created the "OWASP Mobile Top Ten" to improve mobile security. These include the most critical security issues that should be avoided in any Android app:

Fortunately, MobSF can help you to identify many possible security issues in your Android app in an automated way. The security framework works best for:

local environment
performing a quick security test
implemented in CI/CD with mobsfscan (will be used here)

To test MobSF you can switch to https://mobsf.live and upload any Android APK that will then be analyzed for common security issues.

My advice: all developers should use MobSF to identify many security vulnerabilities during development by doing a static code analysis. This analysis will check the source code without running the application. It can be used during mobile application development and should be carried out regularly. Normally, it should be executed before every app release, update and Pull Request submission.

Keep in mind that using MobSF for static code analysis does not guarantee that your mobile application is safe! But it will help to identify the most obvious security flaws.

SonarQube

__is an open-source platform developed by SonarSource for continuous inspection of code quality to perform automatic reviews with static analysis of code to detect bugs, code smells, and security vulnerabilities on 20+ programming languages.__

For your projects, SonarQube analyzes source code (not only Android/Kotlin apps), evaluates the quality, and generates reports. It allows for the continuous monitoring of quality throughout time and combines static and dynamic analytic methods. SonarQube inspects and evaluates everything that has an impact on our code base, from inconspicuous styling decisions to serious design flaws.

As a result, developers can access and track code analysis information on everything from styling mistakes, potential bugs, and code defects to design inefficiencies, code duplication, inadequate test coverage, and excessive complexity. The Sonar platform examines source code from a variety of angles; as a result, it digs down to your code layer by layer, progressing from the module level to the class level. Additionally, it generates metrics and data at each level, highlighting problematic regions in the source that need to be examined or improved. It automatically detects what type of software you use: Kotlin-based Android app.

Other Features:

SonarQube does more than merely highlight issues. It also provides quality-management tools to proactively assist you in making corrections.

SonarQube provides information on a dashboard on code standards, test coverage, duplications, API documentation, complexity, and architecture in addition to issues.
It provides you with a current snapshot of the quality of your Android app code as well as trends in trailing (what has previously gone wrong) and leading (what is most likely to go wrong in the future) quality indicators.
It offers measurements to guide you in making the best choice for your Android app.

SonarScanner

The SonarScanner CLI is a scanner that can be used every time if there is no specific scanner for your build system. Additionally, the SonarScanner CLI can be run within a Docker container to test the Android app source code.

It is a simple command line tool that scans a provided folder (the Android app folder) with SonarQube-specific roles. After a successful analysis, it uploads all data to a SonarQube server where the results can be viewed.

Prepare Dockerfiles

Now, after explaining the prerequisites and tools they will be combined in Docker Compose files to be then used in a single bash script that will result in the software-quality-chain.

SonarQube/mobsfscan Docker Compose File

As described earlier SonarQube and mobsfscan will be used within the Android software-quality-chain. To be able to use them they will be deployed with Docker Compose as a service onto your operating system.

The following Compose file contains the SonarQube server, a PostgreSQL database for SonarQube, and the mobsfscan container to scan your Android app:

version: "3"

services:
  sonarqube:
    image: sonarqube:9.6-community
    depends_on:
      sonarqube_db:
        condition: service_healthy
    environment:
      SONAR_JDBC_URL: jdbc:postgresql://sonarqube_db:5432/sonar
      SONAR_JDBC_USERNAME: sonar
      SONAR_JDBC_PASSWORD: sonar
    healthcheck:
      test: wget --no-verbose --tries=1 --spider http://localhost:9000/api/system/status || exit 1
      start_period: 10s
      interval: 30s
      timeout: 10s
      retries: 10
    volumes:
      - sonarqube_data:/opt/sonarqube/data
      - sonarqube_extensions:/opt/sonarqube/extensions
      - sonarqube_logs:/opt/sonarqube/logs
    ports:
      - "9000:9000"
  sonarqube_db:
    image: postgres:12
    environment:
      POSTGRES_USER: sonar
      POSTGRES_PASSWORD: sonar
    healthcheck:
      test: [ "CMD-SHELL", "pg_isready" ]
      interval: 10s
      timeout: 5s
      retries: 5
    volumes:
      - postgresql:/var/lib/postgresql
      - postgresql_data:/var/lib/postgresql/data
  mobsfscan:
    image: opensecurity/mobsfscan
    depends_on:
      sonarqube:
        condition: service_healthy
    volumes:
      - ./../app/src:/src
    entrypoint:
      - mobsfscan
      - --sonarqube
      - -o
      - /src/result.json
      - /src
volumes:
  sonarqube_data:
  sonarqube_extensions:
  sonarqube_logs:
  postgresql:
  postgresql_data:

Within this file three services are defined: SonarQube, PostgreSQL, and mobsfscan. While SonarQube and PostgreSQL will just use a standardized Docker service that gets started, mobsfscan will have the project root bound as a volume and an overwritten entry point that executes instantly after the SonarQube container is started correctly and marked healthy (see healthcheck).

SonarScanner Compose File

Luckily, SonarScanner utility is also available as Docker Image and can be used and configured in a Compose file:

version: "3.7"
services:
  sonarscanner:
    image: sonarsource/sonar-scanner-cli
    network_mode: "host"
    env_file:
      - .sq.env
    command:
      - -Dsonar.projectKey=YourAppName
      - -Dsonar.exclusions=**/*.java
      - -Dsonar.externalIssuesReportPaths=./output/result.json
      - -Dsonar.kotlin.detekt.reportPaths=./app/build/reports/detekt.xml
    volumes:
      - "./../:/usr/src"

As you can see in this file we define a projectKey (YourAppName), set some exclusions (comma-separated list), and also add additional reports to the execution of SonarScanner (relative path from /usr/src). Also, we add the Android project root as volume and map it to /usr/src.

Additionally, this Compose file needs an environment file that will contain the SonarQube URL and a User Token which will be used to upload the analysis:

SONAR_HOST_URL=http://localhost:9000
SONAR_LOGIN=squ_3d9c193d3699d18db4f539725090d67caef0b964

Combining the Tools

Now, all Dockerfiles will be combined within the bash script called software-quality-chain which consists of these steps:

Increasing the heap count of the Docker environment
Install, configure, and start SonarQube and mobsfscan
Waiting until mobsfscan finish the analysis
Running complete Gradle checks
Importing a clean PostgreSQL database.
Executing SonarScanner and combining results before uploading to SonarQube

This bash script contains the final solution which will execute all steps:

#!/bin/bash

echo "Removing old output"
rm -f ../output/detekt.xml
rm -f ../output/result.json

echo "Increasing heap count for SonarQube"
sysctl -w vm.max_map_count=262144

echo "Starting SonarQube, mobsfscan"
DOCKER_BUILDKIT=1 docker-compose -f docker-compose.sq.yml up -d

echo "SonarQube is started, running mobsfscan"

echo "Waiting until scan is succesfully finished"
until [ -f ../app/src/result.json ]
do
     sleep 5
done

mv ../app/src/result.json ../output/result.json

echo "finished mobsfscan"

echo "running gradle detekt"
cd ..
sed -i 's/baseline = file("..\/detekt\/baseline.xml")/\/\/baseline = file("..\/detekt\/baseline.xml")/g' app/build.gradle
./gradlew detekt
sed -i 's/\/\/baseline = file("..\/detekt\/baseline.xml")/baseline = file("..\/detekt\/baseline.xml")/g' app/build.gradle
cd docker
cp ../app/build/reports/detekt.xml ../output/detekt.xml
echo "finished gradle detekt"

echo "preparing files to import into SonarQube"
sed -i 's/\/src\/main\//app\/src\/main\//g' ../output/result.json
sed -i 's/"filePath": null/"filePath": "app\/src\/main\/AndroidManifest.xml"/g' ../output/result.json

echo "Restoring SonarQube database with default user"
docker exec -i docker-sonarqube_db-1 //bin//bash -c "PGPASSWORD=sonar psql --username sonar sonar" < dump.sql

echo "Starting SonarScanner"
DOCKER_BUILDKIT=1  docker-compose -f docker-compose.scan.yml up -d

echo "SonarScanner started, will scan code now and upload data"
docker logs -f docker-sonarscanner-1

Increasing the heap count of the Docker environment:

To execute the SonarQube server without any problems you have to increase the max_map_count of the operating system running the Docker container because the default value is not sufficient.

You can read about it in the SonarQube documentation.

Install, configure, and start SonarQube and mobsfscan:

As the Docker Compose files already contain every important setting the only command that has to be executed here is running the Compose file in the background

Waiting until mobsfscan finish the analysis:

Unfortunately, the Docker Compose service of mobsfscan is only started by running the Compose file and the analysis is executed afterward. Because of this, the script has to block until the mobsfscan analysis finishes and successfully creates a result file. This "blocking" is done in lines 16 - 21 where the script waits until the result.json is created and then moves it to the output folder.

Running complete Gradle Detekt checks:

The next step is executing a complete gradle detekt task. It's called complete because sometimes developers use a baseline.xml file in their detekt configuration to mark legacy code as "ignored". Within this step, the baseline.xml file will be removed by commenting it out with sed and then running the gradle detekt task. Afterward, the detekt report will be copied to the output folder, and the usage of baseline.xml file will be restored.

Preparing files to import into SonarQube:

There are two small bugs if combining mobsfscan utility and SonarQube while creating and importing the result.json.

mobsfscan also finds issues in the project that is not suited to specific source files. Unfortunately, these issues cannot be imported correctly into SonarQube because SonarQube always needs a file to add an issue. Luckily, it is easy to identify these issues because they all have filePath=null.
mobsfscan lists all file paths of files from the app folder while SonarQube lists all files from the project folder.

Both issues will be fixed in line 35 and 36 where sed is used to replace the wrong entries with the correct one:

Line 35: src/main -> app/src/main
Line 36: filePath=null -> filePath="app/src/main/AndroidManifest.xml"

Importing a clean PostgreSQL database:

Before starting the SonarScanner to analyze the source code and upload the results to the SonarQube instance a clean PostgreSQL database backup will be imported into the SonarQube instance (Line 39). This has to be done because the freshly installed SonarQube instance does not contain a User Token that can be used to upload data. The provided PostgreSQL backup contains a User (admin:admin12345) and a User Token that already is set within the SonarScanner environment variables.

Without this step, you would have to switch to your SonarQube instance and log in with the default user (admin:admin). Then you have to switch to your account configuration and create a new User Token that can be used for analysis.

Executing SonarScanner and combining results before uploading to SonarQube:

The last step will be executing the Compose file for the SonarScanner (Line 42) and opening the docker logs for this instance to be notified if the analysis is complete.

Afterward, you will get an URL within the terminal that you can open in the browser, log in with admin:admin12345, and see the results of the executed analysis.

Run the Chain

Now, that everything is set upped and explained you can start software-quality-chain. If you followed the tutorial and created the files you still need this PostgreSQL database dump that you can save as dump.sql within the docker folder. If you did not manually create every file you can clone the complete GitHub repository into your Android project root.

Then you can run it by executing:

sh software-quality-chain.sh

Grab a coffee and wait until the chain finishes. There will be an URL within the CLI that you can copy to open the SonarQube instance. Log in with admin:admin12345 and start checking your improvements, security issues, and other stuff.

Possible Errors

If you are using this software-quality-chain you maybe encounter some errors that can be fixed easily!

SonarQube does not start

As told before, to run SonarQube you have to increase the max_map_count. Normally, this will be done by the command in Line 8. If you are working with Windows or do not have permission you have to update the command:

For Windows:

wsl sysctl -w vm.max_map_count=262144

Not able because of permissions:

sudo sysctl -w vm.max_map_count=262144

Cannot run docker-compose (Windows)

If you do not have Docker Desktop (or Rancher Desktop) it could happen that docker-compose cannot be executed from your terminal.

Sometimes it happens because you have to add wsl in front of every docker-compose command (c.f. Line 11):

DOCKER_BUILDKIT=1 wsl docker-compose -f docker-compose.sq.yml up -d

Cannot import PostgreSQL backup (Windows)

If you try to run the software-quality-chain within the Powershell or Git-bash while working with windows you normally will encounter an error while the script tries to restore the PostgreSQL database.

This happens because of a problem with the file encoding of the PostgreSQL dump that is used. To fix this problem connect to your WSL and restart the software-quality-chain. Alternatively, you can log into the running SonarQube instance with default user admin:admin, switch to accounts settings, and create a new user token. Then you can update the sq.env file to use this new user token. If you do this you have to remove the part from the software-quality-chain where the PostgreSQL backup will be inserted.

Software-Quality-Chain Improvements/Changes

As this chain is mainly suited for developers that want to run a quick full test for their android apps on their own computer, this chain could also be changed in some ways.

An important feature of the described chain is that it will run on ANY device running Docker because it automatically inserts a database while executing. This means that every time you run this chain you will have a freshly installed SonarQube that only contains results from the executed analysis.

But if you want to have a curve where you can see improvements while developing you have to persist the data from every consecutive analysis. Also, you can work with several members on the same SonarQube instance and use this chain within your CI/CD pipeline to automatically check after a Pull Request is merged into your release branch.

The following chapters cover changes that have to be made to use the chain in any of these ways.

Persisting Analysis Data

To persist data between consecutive analyses you simply have to adjust the software-quality-chain after the first run.

After your first successful run of the chain switch to lines 38 and 39 where the database import takes place.

You still need the first run because it will populate the database to have a working user token and the user to log in admin:admin12345.

Use a different SonarQube instance

If you are using a different SonarQube instance within the software-quality-chain you have to adjust the two files:

docker-compose.sq.yml: Remove the SonarQube and PostgreSQL container and only keep the mobsfscan. The Compose file will look like this:

version: "3"

services:
  mobsfscan:
    image: opensecurity/mobsfscan
    volumes:
      - ./../app/src:/src
    entrypoint:
      - mobsfscan
      - --sonarqube
      - -o
      - /src/result.json
      - /src

sq.env: Replace the URL with your SonarQube instance URL and update the Token to your user token

Also, remove the PostgreSQL database backup (Line 38/39).

Use it in CI/CD chain

To use this script in a CI/CD chain you have to combine previous sections. Remove the SonarQube instance by editing the Compose file, changing the URL/Token of sq.env, and removing the PostgreSQL backup.

Additional Tools

To further improve the quality of Android apps it is possible to enhance your Android project with two additional extra tools/commands that work well in combination with the software-quality-chain.

SonarLint Plugin for Android

The SonarLint plugin can be downloaded and installed manually from the JetBrains marketplace or you can easily install it from within Android Studio by opening Settings->Plugins and installing it.

Pre-Push Hook For Executing Gradle Detekt

To guarantee that all pushed files did not contain any error that will be found with Gradle you can install a pre-push hook into your .git folder. To do this open .git/hooks, create a new file called pre-push, make it executable (chmod +x), and paste the following content:

#!/bin/bash
echo "Starting detekt check"
./gradlew detekt

Closing Notes

I hope you enjoyed reading this article and will now use my software-quality-chain to significantly improve your code quality and make your app more secure. Keep in mind that software that has high quality can be maintained and enhanced more easily!

All files that were explained in this tutorial can be found in this GitHub repository. Download the whole folder, put it into your Android project root, and execute the software-quality-chain with:

sh software-quality-chain.sh

Keep in mind that on Windows you have to be connected to the WSL!

This is the end of this tutorial. Hopefully, you are now able to use my Android software-quality-chain. If you still have questions about anything that is not fully described you can just ask in the comment section. Also, if you enjoyed reading this article consider commenting with your valuable thoughts! I would love to hear your feedback about my developed chain.

This article was originally published on my blog at https://www.paulsblog.dev/optimize-android-app-development-with-docker-sonarqube-detekt-and-mobsf/

Feel free to connect with me on my blog, Medium, LinkedIn, Twitter, and GitHub.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Photo by Jamison McAndie / Unsplash

Use NestJS, MongoDB and Docker to Create an URL Shortener

Paul Knulst — Wed, 19 Apr 2023 05:52:28 +0000

Implement a Simple URL Shortener With NestJS, Docker, MongoDB, And Deploy It to Production With SSL Enabled Using Traefik. Also, Docker Swarm Setup

Paul Knulst in Programming • Sep 6, 2022 • 10 min read

This tutorial will cover all work to build a simple URL shortener API with NestJS and MongoDB. Additionally, I will show how to deploy it to a (Docker and Docker Swarm) production environment using Docker.

The source code is published on GitHub and can be used freely:

https://github.com/paulknulst/paulsshortener

Create the NestJS Project

First, create a NestJS project which will work as a baseline for the URL shortener. To do this, you need to install the Nest-CLI on your system which can be done by:

npm install -g @nestjs/cli

Then switch to your projects folder and use the Nest-CLI to create a new NestJS project by executing:

nest new paulsshortener

During this process, you will be asked which package manager you want to use. Within this tutorial npm will be used.

Now, you can start the NestJS project in "watch-mode" to instantly see all changes you will make during this tutorial by executing:

npm run start:dev

To test if everything is started correctly hit http://localhost:3000 which will just show "Hello World" within the browser.

Within the project open the AppService (/src/app.service.ts) and change return 'Hello World!'; to return 'This will be your URL shortener';. After reloading http://localhost:3000 the updated response will be seen.

Set Up MongoDB database

For simplicity, MongoDB will be used during this tutorial. To avoid trouble setting up the correct version, replacing an old installation, and so on you should use Docker to deploy it. Save this minimal Compose file into your project root and name it docker-compose.local.yml:

version: '3.6'

services:
  mongo:
    image: mongo
    restart: always
    environment:
      MONGO_INITDB_ROOT_USERNAME: root
      MONGO_INITDB_ROOT_PASSWORD: supersafe

If you are not familiar with Docker Compose and do not want to use it to set up a MongoDB you can have a look at the official MongoDB documentation to learn how you can install it on your machine. Keep in mind that Docker is also used to deploy this URL shortener later within this tutorial!

Now, to deploy a MongoDB, switch to your project root within a terminal and execute:

docker-compose -f docker-compose.local.yml up -d

Afterward, you have successfully set up MongoDB and your machine and can start using it.

Connect Your NestJS project to MongoDB

To connect the project to MongoDB you should use Mongoose which is the most popular MongoDB object modeling tool. Start by installing the required dependencies into your project:

npm i @nestjs/mongoose mongoose

Once you have installed the dependency you can import the Mongoose module into the project by editing the AppModule (/src/app.module.ts) that it looks like this:

import { Module } from '@nestjs/common';
import { AppController } from './app.controller';
import { AppService } from './app.service';
import { MongooseModule } from '@nestjs/mongoose';

@Module({
  imports: [MongooseModule.forRoot('mongodb://localhost:12345/paulsshortener')],
  controllers: [AppController],
  providers: [AppService],
})

export class AppModule {}

The connection will now be automatically established and you can create the schema that will be used to work with the database.

Now use the Nest-CLI to create a new module within your NestJS project which will handle everything related to URLs:

nest g res url

The routine will ask two questions that you should answer.

What transport layer do you use? -> REST API
Would you like to generate CRUD entry points? -> No

Switch to the newly generated url folder, create a new folder schema, and add a file url.schema.ts. Then create a class (Url) add two properties (url, shortenedUrl) to the file. Also, add exports for Document and Schema. Your file should look like this:

import { Prop, Schema, SchemaFactory } from '@nestjs/mongoose';
import { Document } from 'mongoose';

export type UrlDocument = Url & Document;

@Schema()
export class Url {
  @Prop()
  url: string;

  @Prop()
  shortenedUrl: string;
}

export const UrlSchema = SchemaFactory.createForClass(Url);

Open up url.module.ts, add two new imports, and modify the @Module annotation:

import {Module} from '@nestjs/common';
import {UrlService} from './url.service';
import {UrlController} from './url.controller';
import {Url, UrlSchema} from "./schemas/url.schema";
import {MongooseModule} from "@nestjs/mongoose";

@Module({
    imports: [MongooseModule.forFeature([{name: Url.name, schema: UrlSchema}])],
    controllers: [UrlController],
    providers: [UrlService]
})
export class UrlModule {
}

Implement the URL Link Shortner Function

To shrink an URL you will be going to use the CRC32 hash algorithm which has to be added to the project to use it:

$ npm install crc-32 --save

Open the UrlService (/src/url/url.service.ts) and replace the content of the file with the following snippet:

import {Injectable} from '@nestjs/common';
import {InjectModel} from "@nestjs/mongoose";
import {Url, UrlDocument} from "./schemas/url.schema";
import {Model} from "mongoose";
import * as CRC from "crc-32";

@Injectable()
export class UrlService {

    constructor(@InjectModel(Url.name) private urlModel: Model<UrlDocument>) {
    }

    private shrink(url: string) {
        return CRC.str(url).toString(16)
    }

    async create(url: string) {
        const createdUrl = new this.urlModel({url: url, shortenedUrl: this.shrink(url)});
        await createdUrl.save();
        return createdUrl.shortenedUrl;
    }

    async find(shortenedUrl: string) { //-3c666ac
        const url = await this.urlModel.findOne({shortenedUrl: shortenedUrl}).exec();
        return url.url;
    }
}

This snippet contains three important functions:

shrink: Converts the given URL into an 8-character string by using the CRC32 algorithm. Then returns only the 8-character string.
create: Use the shrink function to create an 8-character string and saves a new UrlSchema document into the MongoDB
find: Retrieves the saved URL from the MongoDB.

Add Routes to NestJS API

To use the URL shortener in a Client we need to create three REST endpoints/functions.

GET /shrink: Create a new shortened URL using the HTTP Get method. The path parameter contains the unshortened URL. This can be done within any browser.
POST /shrink: Create a new shortened URL using the HTTP Post method. The body contains the unshortened URL. You need an API or Postman to use this.
GET /s: Takes the 8-character string as a path parameter and returns the unshortened URL. In the end, it will automatically forward to the unshortened URL.

As you are working with NestJS you can easily implement these endpoints by adding three new functions to the UrlController (url.controller.ts) within the URL module and annotate it with @Get and @Post. Open the UrlController and replace the content with:

import {Body, Controller, Get, Param, Post} from '@nestjs/common';
import {UrlService} from './url.service';

@Controller('')
export class UrlController {
    constructor(private readonly urlService: UrlService) {
    }

    @Get('/shrink/:url')
    getShrink(@Param('url') url: string) {
        return this.urlService.create(url)
    }

    @Post('/shrink')
    postShrink(@Body() body: { url: string }) {
        return this.urlService.create(body.url)
    }

    @Get('/s/:shortenedUrl')
    unshrink(@Param('shortenedUrl') shortenedUrl: string) {
        return this.urlService.find(shortenedUrl)
    }
}

Within this snippet, two new GET resources (/shrink/ and /s/) and one POST resource that calls the previously created functions from the UrlService are created. Also, the annotation above the class is changed from @Controller('url') to @Controller('') to further shrink the resulting URL.

Testing the URL shortener

To test the functionality you can simply use your browser because we have implemented both resources as GET calls.

Keep in mind that as you use GET for providing an URL as a path parameter you have to encode the URL! This means that an URL like this:

https://www.paulsblog.dev/manage-time-more-efficiently-with-the-pomodoro-technique/

will become this:

https%3A%2F%2Fwww.paulsblog.dev%2Fmanage-time-more-efficiently-with-the-pomodoro-technique%2F

With this information, you can create a shortened URL by opening the following URL in our browser:

http://localhost:3000/shrink/https%3A%2F%2Fwww.paulsblog.dev%2Fmanage-time-more-efficiently-with-the-pomodoro-technique%2F

With this GET call your API will return the 8-character string: -5f1a8349 (It should be the same within your project)

Append this string to your GET /s/ call to receive the unshortened version of the URL by opening: http://localhost:3000/s/-5f1a8349

The result in the browser will be the unshortened version of the previously provided URL.

Implement Forwarding to Unshortened URL

Now, that you have developed an API that can shorten URLs with help of the CRC-32 algorithm you should enable the functionality to forward the request and automatically open the URL it finds within the database.

With NestJS this is easy because you only have to adjust the unshrink function within the UrlController (src/url/url.controller.ts). Change the previously created function to this implementation:

@Get('/s/:shortenedUrl')
async unshrink(@Res() res, @Param('shortenedUrl') shortenedUrl: string) {
    const url = await this.urlService.find(shortenedUrl)
    return res.redirect(url)
}

Deploy The URL Shortener With Docker

Let's assume you want to deploy the URL shortener in a Docker environment and have to develop a Compose file that can be used to do this.

The first step to do will be to create a new Compose file (docker-compose.prod.yml) and copy the content from the previously created MongoDB file into it. Then add a new service called backend which will be used to install, compile and run the NestJS project within the Docker environment.

As you have developed a custom piece of software there will not be a suitable image on DockerHub and you have to create one from scratch. Because the project is based on NestJS, which is working in a NodeJS environment, you can create a new Dockerfile and use the latest version of node as a base image. Then simply copy the source code, install, build and run the project. The following Dockerfile will be sufficient and should be created in the project root:

FROM node:latest

WORKDIR /usr/src/app
COPY package*.json ./

RUN npm install
COPY . .
RUN npm run build

CMD [ "node", "dist/main.js" ]

This Dockerfile can now be used as the image for the backend service in the Compose file.

version: '3.6'
services:
  db:
    image: mongo
    restart: always
    environment:
      MONGODB_USER: paul
      MONGODB_DATABASE: paulsshortener
      MONGODB_PASS: paulspw
  backend:
    image: paulsshortener
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - 3001:3000
    depends_on:
      - db
    restart: unless-stopped

Unfortunately, running this Compose file will not work because the URL for the links within our project are hard-coded and will not automatically adjust. Also, the DB hostname and port within the AppModule (app.module.ts) are static.

To fix these problems, you have to add and change something within the AppModule (app.module.ts) and the UrlService (url.service.ts). To be specific, add three new variables that hold the database port, the database URL, and the base URL of the resulting service.

Switch to the AppModule (app.module.ts) and add these two variables above the class definition:

const DB_HOST = process.env.DB_HOST || 'localhost'
const DB_PORT = process.env.DB_PORT || '12345'

Additionally, change the Mongoose part within the imports from the module to correctly use these variables:

MongooseModule.forRoot('mongodb://' + DB_HOST + ':' + DB_PORT + '/paulsshortener')

Then open the UrlService (url.service.ts) and add the baseurl variable above the class definition:

const basepath = process.env.BASE_URL || 'http://localhost:3000/'

Lastly, change the create function to return the complete shortened URL using the newly introduced base variable.

async create(url: string) {
    const createdUrl = new this.urlModel({url: url, shortenedUrl: this.shrink(url)});
    await createdUrl.save();
    return basepath + "s/" + createdUrl.shortenedUrl;
}

Now, that you applied these changes you should adjust your Compose file by adding the available environment variables and adjusting them to your needs:

version: '3.6'
services:
  db:
    image: mongo
    restart: always
    environment:
      MONGODB_USER: paul
      MONGODB_DATABASE: paulsshortener
      MONGODB_PASS: paulspw
  backend:
    image: paulsshortener
    environment:
      - BASE_URL=https://locahost:3001
      - DB_HOST=db
      - DB_PORT=27017
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - 3001:3000
    depends_on:
      - db
    restart: unless-stopped

Finally, deploy it on your localhost:

docker-compose up -d --build

Your URL shortener is now correctly installed and can be used from your local environment.

Deploy to Production Using Traefik

Now that you have a running Docker Service that can be deployed anywhere you can use it to deploy it in a production environment using Traefik. To do this you will edit the Compose file, add all Traefik-related keywords (labels, networks), and adjust it to your needs.

If you are not familiar with deploying Docker Services using Traefik I can recommend the following tutorials covering basic installation on a single server and a server cluster installation using Docker Swarm:

To deploy your URL shortener using Traefik adjust the labels section. The following part will show and explain how it is done in a single server setup and additionally there will be a Docker Swarm configuration to download:

version: '3.6'
services:
  db:
    image: mongo
    restart: always
    environment:
      MONGODB_USER: paul
      MONGODB_DATABASE: paulsshortener
      MONGODB_PASS: paulspw
    volumes:
      - db:/data/db
    networks:
      - default
  backend:
    image: paulsshortener
    environment:
      - BASE_URL=https://at0m.de/
      - DB_HOST=db
      - DB_PORT=27017
    build:
      context: .
      dockerfile: Dockerfile
    networks:
      - default
      - traefik-public
    depends_on:
      - db
    restart: unless-stopped
    labels:
      - traefik.enable=true
      - traefik.docker.network=traefik-public
      - traefik.constraint-label=traefik-public
      - traefik.http.routers.pauls-shortener-http.rule=Host(`at0m.de`) || Host(`www.at0m.de`)
      - traefik.http.routers.pauls-shortener-http.entrypoints=http
      - traefik.http.routers.pauls-shortener-http.middlewares=https-redirect
      - traefik.http.routers.pauls-shortener-https.rule=Host(`at0m.de`)
      - traefik.http.routers.pauls-shortener-https.entrypoints=https
      - traefik.http.routers.pauls-shortener-https.tls=true
      - traefik.http.routers.pauls-shortener-https.tls.certresolver=le
      - traefik.http.services.pauls-shortener.loadbalancer.server.port=3000
      - traefik.http.middlewares.redirect-pauls-shortener.redirectregex.regex=^https://www.at0m.de/(.*)
      - traefik.http.middlewares.redirect-pauls-shortener.redirectregex.replacement=https://at0m.de/$${1}
      - traefik.http.middlewares.redirect-pauls-shortener.redirectregex.permanent=true
      - traefik.http.routers.blogs-knulst-https.middlewares=redirect-pauls-shortener
volumes:
  db:

networks:
  traefik-public:
    external: true

Before you can successfully deploy this service with Compose you have to adjust the Host: Use your own BASE_URL and update the traefik configuration (For me, it is at0m.de). After changing both values deploy it with:

docker-compose -f docker-compose.prod.yml up -d

Using a Docker Swarm you can use this Compose file. But, you have to adjust BASE_URL and the placement constraints for the MongoDB service. Then build, push the image to our registry, and deploy it onto your Docker Swarm:

docker-compose -f docker-compose.prod-swarm.yml build
docker-compose -f docker-compose.prod-swarm.yml push
docker stack deploy -c docker-compose.prod-swarm.yml paulsshortener

Additional Adjustments for Live Version

Because you deployed it publicly on your server you should add rate limiting by following this approach: https://docs.nestjs.com/security/rate-limiting.

tl;dr:

Install needed package within the project:

npm i --save @nestjs/throttler

In AppModule (app.module.ts) extend imports-array:

ThrottlerModule.forRoot({
    ttl: 60,
    limit: 10,
})

Then, add ThrottlerGuard to the providers array:

{
    provide: APP_GUARD,
    useClass: ThrottlerGuard,
}

Add the SkipThrottle annotation to the Get /s/ endpoint within the UrlController (url.controller.ts) to ignore rate limiting this specific call:

@SkipThrottle()

Finally, redeploy your Docker service. Don't forget to rebuild before deploying!

Closing Notes

I hope you enjoyed reading my tutorial and are now able to create, build, and deploy your URL shortener website within a Docker container.

Keep in mind that this is a very basic example without any error handling, and no URL checking. However, this tutorial should be a starting point for developing your version.

If you enjoyed reading this article consider commenting your valuable thoughts in the comments section. I would love to hear your feedback about this URL shortener. Furthermore, if you have any questions about implementing your own version, please jot them down below. I try to answer them if possible. Also, share this article with your friends and colleagues to show them how to use NestJs, MongoDB, and Docker to create their own URL shortener.

This article was originally published on my blog at https://www.paulsblog.dev/use-nestjs-mongodb-and-docker-to-create-an-url-shortener/

Feel free to connect with me on my blog, Medium, LinkedIn, Twitter, and GitHub.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Image by Vectorarte on Freepik

How To Host Your Website For Free

Paul Knulst — Fri, 24 Mar 2023 12:50:36 +0000

Learn How To Host Your Website On Multiple Different Sites For Free

Paul Knulst in Programming • Jun 24, 2022 • 3 min read

If you are a student or a professional, you typically work on some projects that you create in your free time for fun, for learning something new, or for your portfolio. Often, you also want that the projects could be shown to your fellow developers so that they can benefit from them. But only showing the code on GitHub is not sufficient because you want to show a running version that can be tested out by everyone before downloading your source code.

Luckily, today exist multiple services that you can use to show your work to the public. Although there are some paid options like AWS, Azure, and GCP (if you are out of the free tier), I will introduce some services that you can use without spending any money on them.

1. Github Pages⁣

GitHub Pages is a feature from GitHub that enables every user or company to host a public webpage directly from your GitHub repository. You can edit, push, and your changes will be live.

Pros of this solution:

Widely used
Free hosting
Fast and easy to setup
Great customer service
Great documentation
Security and reliable

Cons of this solution:

GitHub pages are limited to static websites
Special measures should be done for SEO
No support for any problem. You have to rely on forums and documentation

2. Netlify

Netlify is an American cloud computing company that offers hosting and serverless backend services for web applications and static websites. Normally, you have to pay for their services but they provide a free plan for personal websites. You can build, deploy, and host any static website or app with a drag and drop interface. Also, it can automatically deploy from GitHub or Bitbucket.

Pros of this solution:

Provides continuous integration and continuous deployment
Drag and drop functionality is intuitive
Customer support service is good
Easy deploy service and use
Support Custom domains
SSL support
Ability to use plugins to customize build workflow

Cons of this solution:

Dynamic content cannot be displayed
Supports only static websites

3. Vercel

A cloud platform for serverless deployment. It enables developers to host websites and web services that deploy instantly, scale automatically, and require no supervision, all with minimal configuration. Vercel is a tool in the Static Web Hosting category of a tech stack.

Pros of this solution:

Simple deployment
Easy, production-ready deployment with one command.
Simple scaling, including auto-scaling
SSR (Server-Side Rendering) support
Free tier
Free SSL

Cons of this solution:

The platform is a bit confusing
Lack of guides and manuals

4. Heroku

Heroku is a cloud platform with a free tier that allows up to 1000 dyno hours per month and includes deployment from Git and Docker. Normally, it is used to build, deliver, monitor, and scale all kinds of apps in different languages/frameworks: Node.js, Ruby, Python, Java, and much more.

Furthermore, with Heroku, you are allowed to use custom domains, container orchestration, and automatic OS patching.

Pros of this solution:

Deployments with Git and Docker
Easy to start
Custom domains
Free Tier
Container orchestration
Automatic OS patching

Cons of this solution:

Cannot control exact configuration of applications
Could be too overwhelming
No SSL
Could stop working/has some downtime

5. Firebase

Firebase from Google is primarily an app development platform that could be used as a backend for apps and games. It provides several features like Authentication, Remote configuration, A/B testing, and analytics out of the box within the free tier (and more).

Pros of this solution:

Authentication
Analytics
Test Lab
Free Tier
Free SSL

Cons of this solution:

Limitations of real-time database
More complicated

Closing Notes

All of the mentioned services could be easily used to host your private project. Although some of them have more features than others you could pick the best service for a specific project. Furthermore, you can use every service if you have multiple projects that you want to show.

Use them wisely to spread your work to the public crowd!

If you enjoyed reading this article consider commenting your valuable thoughts in the comments section. I would love to hear your feedback about my tutorial. Furthermore, share this article with fellow developers to help them get hosting their own services for free!

This article was originally published on my blog at https://www.paulsblog.dev/how-to-host-your-website-for-free/

Feel free to connect with me on my blog, Medium, LinkedIn, Twitter, and GitHub.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Image by 8photo on Freepik

How To Add Comments To Your Blog With Docker

Paul Knulst — Mon, 27 Feb 2023 08:30:35 +0000

Having Comments On Your Own Blog Is One Of The Most Engaging Features. Learn How To Integrate Isso Comments To Have Privacy-Focused Comment Feature In Your Blog.

Paul Knulst in Docker • Jul 1, 2022 • 7 min read

Having comments on your blog is one of the most engaging features! Unfortunately, Ghost Blogging Service does not support any comments out of the box. Although they are many different services like Disqus or Discourse which are kind of "free" (they have inbuilt Ads in the "free" version. And Ads are EVIL) there also exist some really free services like Commento, Schnack, CoralProject Talk, and Isso.

I tested all of the free services for my blog and come to the conclusion that Isso is the best service to use in a Ghost Blog. Within this article, I will describe why Isso is the best software, how it can be installed in a Docker environment, and how Isso comments can be integrated into any Blog (not only Ghost Blogging software).

Why Isso?

Isso is a commenting server written in Python and JavaScript and aims to be a drop-in replacement for Disqus or Discourse.

It has several features:

It is a very lightweight commenting system
Works with Docker Compose
Uses SQLite because comments are not Big Data!
A very minimal commenting system with a simple moderation system
Privacy-first commenting system
Supports Markdown
It's free
Similar to native WordPress comments
You can Import WordPress or Disqus
Embed it everywhere in a single JS file; 65kB (20kB gzipped)

All of these features are good reasons to choose Isso as your backend comment system instead of one of the others. If you want to install it in a Docker (or Docker Swarm) environment you can follow my personal guide.

Install Isso Comments With Docker

Prerequisite

Docker (optional Docker Swarm): To fully follow this tutorial about installing Isso for your blog you need to have a running Docker environment. I will also provide a Docker Swarm file at the end.
Traefik: Traefik is the load balancer that forwards my request to the Docker container. You need to have one installed to access the comments with an URL. If you do not have a running Traefik you will learn within this tutorial how you can set one up.

If you don't want to use Docker and Traefik please follow the official Installation Guide on the Isso website.

Set Up Docker Compose File

If you prepared everything you start installing Isso. First, you have to download the latest version of Isso from the GitHub page: https://github.com/posativ/isso/. You can either download the zip file containing the master branch and extract it or clone it:

git clone git@github.com:posativ/isso.git

Then switch into the isso folder, delete the docker-compose.yml, create a new docker-compose.yml, and paste the following code snippet into it:

version: "3.4"

services:
  isso:
    build:
      context: .
      dockerfile: Dockerfile
    image: isso
    environment:
      - GID=1000
      - UID=1000
    volumes:
      - db:/db
    networks:
      - traefik-public
    labels:
      - traefik.enable=true
      - traefik.docker.network=traefik-public
      - traefik.constraint-label=traefik-public
      - traefik.http.routers.isso-http.rule=Host(`YOUR_DOMAIN`)
      - traefik.http.routers.isso-http.entrypoints=http
      - traefik.http.routers.isso-http.middlewares=https-redirect
      - traefik.http.routers.isso-https.rule=Host(`YOUR_DOMAIN`)
      - traefik.http.routers.isso-https.entrypoints=https
      - traefik.http.routers.isso-https.tls=true
      - traefik.http.routers.isso-https.tls.certresolver=le
      - traefik.http.services.isso.loadbalancer.server.port=8080
    healthcheck:
      test: wget --no-verbose --tries=1 --spider http://localhost:8080/info || exit 1
      interval: 5s  # short timeout needed during start phase
      retries: 3
      start_period: 5s
      timeout: 3s
volumes:
  db:

networks:
  traefik-public:
    external: true

Within Line 5 - 8 you can see within this Compose file that the Isso image will be built from the Dockerfile that you downloaded from the official GitHub repository and will be named isso.

In Line 12-13 (and 34-35), I defined a volume in which all comments will be saved within an SQLite DB.

The labels section in Line 16 - 27 contains important information for Traefik. Replace YOUR_DOMAIN with the domain where the Isso backend should be accessible. This is important for the moderation of comments.

In Line 28 - 33, I also added a health check that will check every 5s if the Isso backend is running correctly. To learn more about Docker health checks you should read this tutorial:

https://www.paulsblog.dev/how-to-successfully-implement-a-healthcheck-in-docker-compose/

Configuration of Isso Backend

Before you can run your Isso backend you have to adjust the Isso configuration file to your needs.

The following file shows the most important settings that you HAVE to update in order to have a working Isso backend. I will explain them afterward.

[general]
dbpath = /db/comments.db
host =
    https://www.knulst.de
notify = smtp

[admin]
enabled = true
password = yourSuperSafePasswordThatYouNeedInAdminMenu

[moderation]
enabled = true

[server]
listen = http://localhost:8080
public-endpoint = https://isso.knulst.de

[smtp]
username = YOUR_EMAIL_USER
password = YOUR_EMAIL_USERPASS
host = YOUR_EMAIL_HOST
port = 587
security = starttls
to = USER_WHO_SHOULD_RECEIVE_MAIL
from = YOUR_EMAIL_USER

[general]: Set the dbpath, the URL of your blog, and that you want to get Emails if new comments are submitted

[admin]: Enable administration and set your admin password. This is needed to login into the admin interface.

[moderation]: Enable moderation of new comments. If set to false every comment will be shown instantly. Not recommended!

[server]: Set your public endpoint for your Isso backend. Should be equal to the value within the Compose file.

[smtp]: Set your Email server settings and account data of your Email.

Run Isso Backend

After adjusting the Compose and isso.cfg file to your needs, you can start the Isso backend by executing:

docker-compose up -d --build

After some seconds your Isso backend should be running at the specified domain. You can check it by going to https://your-chosen-domain.com/info.

Integrate Isso To Ghost CMS (or any equivalent Blog)

Adjust The Source Code Snippet

Now that you have installed your Isso backend you can integrate comments into your blog. To do this, all you have to do is insert the following snippet in your source code. I will explain how it is done in Ghost Blogging software afterward.

<script data-isso="//isso.knulst.de/" src="//isso.knulst.de/js/embed.min.js"></script>
<section id="isso-thread"></section>

First of all replace isso.knulst.de with the domain, you used with your Isso bac

kend. Furthermore, you should carefully read the source code snippet because there are two important settings:

The URL in data-isso starts with two /.
The URL in src also starts with two /.

If you change the URLs to your needs keep the two / in front of the Isso backend URL. This is needed because the Isso backend is deployed on a different host than the blog.

Add Snippet To Your Page

With the adjusted source code snippet you can now add comments to your blog. If you use the Ghost Blogging platform you should first download the design that you are using on your instance and extract it into a folder.

Locate the post.hbs file and check where the article ends. The best place for comments will be below the Tag List that is often shown at the end of an article. After you found the closing </div> insert the snippet in front of that </div>.

Zip your theme, reupload it to your Ghost Blogging Software, and activate it within the Design interface.

If you use another blogging software you should also be able to edit the post html/php/js file and add the snippet where you want to see comments.

Now, switch to any article on your blog and you should see the comments section:

If you set up the Isso backend as I explained in the previous section anyone can now comment on every post and you will be informed by email about a new comment and can instantly activate or delete it within the email:

After getting some cool comments they will be displayed below your post. Because activated the Gravatar feature and set the type to robohash, I have cool robot pictures for every user that creates a comment:

Deploy In Docker Swarm

To deploy Isso into your Docker Swarm you can use my Isso Docker Compose file that has the same settings as the plain Compose file. But, this will only work as intended if you set up your Docker Swarm as I describe in this tutorial:

https://www.paulsblog.dev/docker-swarm-in-a-nutshell/

Furthermore, you need a Traefik Load Balancer in your Docker Swarm that will be used by the Compose file. I explained how to enhance your Docker Swarm with a Traefik in this tutorial:

https://www.paulsblog.dev/services-you-want-to-have-in-a-swarm-environment/#traefik

Closing Notes

While many writers argue that you do not need a comment system integrated into your blog I completely deny this argument! I personally think that comments are a great way to interact with your users, answer questions and build your community. Also, if users comment on your articles you could use this interaction to improve your content.

This article was originally published on my blog at https://www.paulsblog.dev/how-to-add-comments-to-your-blog-with-docker/

Feel free to connect with me on my blog, Medium, LinkedIn, Twitter, and GitHub.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Photo by Volodymyr Hryshchenko / Unsplash

Advanced React UI Components To Optimize Development Process

Paul Knulst — Sun, 12 Feb 2023 19:52:05 +0000

Power-Charge Your React App Like Never Before With Mantine, TipTap, Syntax-Highlighter, and more

Paul Knulst in JavaScript • Jul 13, 2022 • 2 min read

Today, I will bring your attention to 5 lightweight free UI components that can be used to enhance the user experience on your website very easily.

1. Mantine

Mantine is a fully-featured React components library that can be used to build fully functional accessible web applications. Furthermore, it includes several customizable React components and hooks with native Dark mode support to cover you in any situation. It is focused on usability, accessibility, and developer experience. Mantine is TypeScript based.

Find it on GitHub

2. react-easy-crop

react-easy-crop can be used to crop images/videos with easy interactions. It supports drag, zoom, and rotating images in any format or base64 string. Also, it supports every video format that is supported in HTML5.

Furthermore, react-easy-crop is a mobile-friendly UI component.

Find it on GitHub

3. react-colorful

react-colorful provides a tiny (only 2.8 KB gzipped) color picker component for React apps (also Preact). With absolute no dependencies, it can still be used to have a cross-browser workable, mobile-friendly color picker that is built with hooks and functional components only. Also, the color picker is written in TypeScript and has all types included.

Find it on GitHub

4. React Syntax Highlighter

React Syntax Highlighter is a React UI component that enables syntax highlighting in any React app by integrating/combining two popular projects: lowlight and refractor.

Highlighting the code is done with Prism.js and highlight.js by using inline styles instead of altering the DOM manually or using dangerouslySetInnerHTML.

Find it on GitHub

5. TipTap

TipTap is a headless, framework-agnostic rich text editor that is extendable and based on ProseMirror. It enables full control over every aspect of a text editor experience. Also, it is very customizable, comes with a lot of extensions, and is fully documented.

Furthermore, TipTap works with Vanilla JavaScript and is also integrated into several TypeScript-based frameworks (React, Next.js, Vue, etc…).

Find it on GitHub

Closing Notes

I hope you like these UI components and will use them in your next React project.

Also, if you have any questions, ideas, recommendations, or want to share your own awesome UI component, please jot them down below. I try to answer your question if possible and will test your recommendations.

This article was originally published on my blog at https://www.paulsblog.dev/advanced-react-ui-components-to-optimize-development-process/

Feel free to connect with me on my blog, Medium, LinkedIn, Twitter, and GitHub.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Photo by Lautaro Andreani / Unsplash