DEV Community: Sergej Brazdeikis

TIL: Docker Logging with Loki and Grafana on Synology

Sergej Brazdeikis — Thu, 27 Oct 2022 14:29:42 +0000

Recently I wanted to set up and explore a logging tool for Docker, so all the logs would go into one location. After a quick research, I decided to try out Loki. It natively integrates with Grafana and has Prometheus vibes :)

Loki and Docker setup

Let's install the Loki and Grafana combination via Docker compose I used this template from the official Loki GitHub repo:

version: "3"

networks:
  loki:

services:
  loki:
    image: grafana/loki:2.6.0
    ports:
      - "3100:3100"
    command: -config.file=/etc/loki/local-config.yaml
    networks:
      - loki

  promtail:
    image: grafana/promtail:2.6.0
    volumes:
      - /var/log:/var/log
    command: -config.file=/etc/promtail/config.yml
    networks:
      - loki

  grafana:
    image: grafana/grafana:latest
    ports:
      - "3000:3000"
    networks:
      - loki

It also comes with promtail, which pipes the system logs into Loki.

For the first login into Grafana - use admin for username and password.

Configure Docker to route logs to Loki

Install Docker Loki plugin

Grafana Loki supports Docker plugin and integrates natively with it.

There is Docker plugin which simplifies log routing. To install it:

docker plugin install grafana/loki-docker-driver:latest --alias loki --grant-all-permissions

Configure Docker to route logs to Loki

For this, we need to locate dockerd.json. It may be in different locations:

Linux default location: /etc/docker/daemon.json
Synology: /var/packages/Docker/etc/dockerd.json

Create or alter dockerd.json to include the following:

{
    "log-driver": "loki",
    "log-opts": {
        "loki-url": "http://localhost:3100/loki/api/v1/push"
    }
}

Restart Docker Daemon

It may vary depending on your system:

Linux:

sudo systemctl start docker

Synology:

sudo synopkgctl stop Docker
sudo synopkgctl start Docker

Test Loki Docker configuration

All newly started containers will pick up new configuration. You can quickly check the current configuration of the running container:

> docker inspect -f '{{.HostConfig.LogConfig}}' $YOUR_CONTAINER_ID
{loki map[loki-url:http://localhost:3100/loki/api/v1/push]}

P.S. docker ps to get the list of running containers. Just in case ;)

Confuguring Grafana to query logs

For this part of the docs, I think it is best to use the official page since it is clearly described here - Configure Loki as a source in Grafana

Reference Links

TIL: Fix for MariaDb docker "incorrect definition of table" errors

Sergej Brazdeikis — Thu, 27 Oct 2022 07:29:46 +0000

I recently upgraded MariaDB Docker image, and everything went well because everything worked. It was until I checked the logs:

...
2022-10-26 19:30:12 17 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2022-10-26 19:30:12 17 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
...

I was confused, and then I found the fix I needed. Add this ENV variable to fix it:

MARIADB_AUTO_UPGRADE: '1'

for me it did fix the issue. Note:** Backup your data first!**
Read further if you want to learn what it does :)

I used mariadb:latest image from official DockerHub

MARIADB_AUTO_UPGRADE explained

After quick checking, I found the explanation on GitHub repo here:

Set MARIADB_AUTO_UPGRADE to a non-empty value to have the entrypoint check whether mysql_upgrade/mariadb-upgrade needs to run, and if so, run the upgrade before starting the MariaDB server.

Before the upgrade, a backup of the system database is created in the top of the datadir with the name system_mysql_backup_*.sql.zst. This backup process can be disabled with by setting MARIADB_DISABLE_UPGRADE_BACKUP to a non-empty value.

What MARIADB_AUTO_UPGRADE actually does

The repository for the docker image is here

And here is a snippet of what it does (permalink to Maria DB 10.10)

    mysql_note "Starting temporary server"
    docker_temp_server_start "$@" --skip-grant-tables \
        --loose-innodb_buffer_pool_dump_at_shutdown=0 \
        --skip-slave-start
    mysql_note "Temporary server started."

    docker_mariadb_backup_system

    mysql_note "Starting mariadb-upgrade"
    mariadb-upgrade --upgrade-system-tables
    mysql_note "Finished mariadb-upgrade"

    mysql_note "Stopping temporary server"
    docker_temp_server_stop
    mysql_note "Temporary server stopped"

So as simple as it is, it does upgrade :)

Keep your data safe, do backup and test your backups! #peace

Software Engineer to Engineering Manager. Should I?

Sergej Brazdeikis — Sun, 06 Feb 2022 11:28:37 +0000

You think about the manager path, and you see there is an opportunity, And you just can't put the dot on the decision. Let's explore it together.

Assess the motivation

Before doing it, first, explore why do want to do it? Writes down some notes for you:

Why do I think about it?
Do I imagine what the engineering manager role is?
Do I like it?
Am I curious to try it?
What benefits and disadvantages do you see?
There will be a lot to learn. Do I want to commit?
How do I see myself in two years? Five years?

Engineering manager. Should I switch from the engineer?

If you are curious to try it - do it.

When switched from an engineer to a more managerial role, I had to think on a team level. The very definition of success changed - now it is team success. System thinking on the team level is a daily exercise.

I also had to improve my soft skill tools. What I tell, how I speak, approach problems, react, behave, etc. All this suddenly had a way bigger impact. I learned many essential things about it for me as a manager. This experience had a very positive effect on my personality.

There are many other minor benefits from only getting this experience as an engineer and a person. This experience to look at teamwork from different expertise is invaluable.

It is not a promotion

Be aware of this! It is not a promotion - it is a track change!
When you switch from engineer to manager, you are a Junior Engineering manager. And it is fair. Folks who support your growth see you in that role.

Amount of coding

Hands-on is a very tricky topic. I can tell it is all in your hands, and it is tough to balance coding and accelerated growth as starting manager. I'm sure you will have to find the best balance for you while on the job, which approach works for you the best.

One thing is vital to say, as your responsibility will not be purely code related, we will have to delegate coding tasks on critical implementation path to full-time software engineers. This way, your other tasks as manager will not negatively impact the whole team's performance. I'm not saying you cannot do this, be aware of it.

What is Success for an Engineering Manager?

All managers are on a different path—some in first years, fifth or maybe tenth year. Our views will change during the trip, how we do stuff changes. Today I personally, the successful engineering manager is the one who facilitates the team to be successful. And we are accountable for that.

Today how I see the success for a team:

Software engineering team delivers measurable impact to end-users and business.
The team is motivated, engaged and happy. Team members are open and tell that it is a team, not just a group of people.
The team is excellent from the DORA metrics perspective
The team grows skills, and as the team as each personally.

If interested, I suggest reading about The Five Dysfunctions of a Team: A Leadership Fable

The window of opportunity for about two years

Even if you find out later that it is not for you - you can switch back to a single contributor.

Two years is a window opportunity because your coding skills will not have a chance to disappear; you, in any case, are still as an engineer as today. And the switchback will be smooth.

Worth mentioning that you will never really fully come back to a single contributor mindset after switching back. You will think on a team level. It is a permanent upgrade :)

The only possible downside is that you can potentially get not up to speed with the latest technologies. The reason is that you will have a lot to learn within your new role. When you switch back to engineer - you quickly catch up, and if you continue being a manager - you find a way to keep up.

Engineering manager salary

It is a natural motivator to scale. I want to clear one thing right away - if you think or find out that this is not your thing, there is no point living in pain. As a single contributor, you can grow further on the Software expert track. Staff engineers, Senior Staff Engineers, Architect roles are parallel growth tracks, and salary matches or, in some cases, surpasses the manager's path.

It is all about how good you are, your passion, and your role impact. Great Software engineers in the right company grow and significantly positively impact engineering quality.

Consult with a good manager

Talk to respected by you the manager, who you think it could be a good reference. A good manager will tell you how it is in the first person. What are the joys, the challenges, and what are the downsides? The experience can vary a lot. Ask a few managers to form a better understanding.

Types of the Engineering manager role

Here is a good dedicated blog post about it 5 Engineering Manager Archetypes from Pat Kua

If you choose to try it out, you will learn more techniques, like situational leadership and different ways to depend on the team state.

It is a rabbit hole on its own, and it is a beautiful path.

It is your decision

Whatever you decide is your choice, what path to go forward or when to do it. Do not get others to do it for you.

Enjoy the path you choose! It is most probably the right one :)

Install Docker on Mac M1 without Docker Desktop

Sergej Brazdeikis — Sat, 29 Jan 2022 20:20:54 +0000

Docker Desktop is the default way to get docker on your machine, and when you search, it suggests installing it. To find the documentation to do it differently - is a hassle.

It remains free for small businesses (fewer than 250 employees AND less than $10 million in annual revenue), personal use, education, and non-commercial open-source projects.

Here I'm sharing the simplest way to go around it.

Docker on Mac with no Docker Desktop



brew install docker docker-compose

Docker Compose on Mac without Docker Desktop

After installing the Docker, you will see the message.



Compose is now a Docker plugin. For Docker to find this plugin, symlink it:
  mkdir -p ~/.docker/cli-plugins
  ln -sfn /opt/homebrew/opt/docker-compose/bin/docker-compose ~/.docker/cli-plugins/docker-compose

Just execute the above, and we did not finish yet!

Enjoy!

Free Replacement Docker Desktop

Docker Desktop installs Docker Engine, which can run only on Linux. So far, we have installed only CLI tools. For images to run, we need an environment.

Rancher Desktop replaces Docker Desktop! Happily to Rancher Desktop v1.0.0 was just released with M1 chip compatibility and solved this problem for us!

If you are on the Mac with an M1 chip, download the file with ...mac.aarch64.zip from
releases page on GitHub

Rancher Desktop No access error

In case you are a missing directory or existing directory permissions will not compatible, run these:



sudo mkdir /usr/local/bin

sudo chown $USER /usr/local/bin

Other Docker Desktop alternatives

Docker + Hyperkit + Minikube: does not work on M1 Macs
Podman - lacks of ability to mount volumes from the host OS
Lima + nerdctl - It is not really a docker

Thanks for reading!

Enjoy!

Strapi v4 setup with Docker Compose and MySQL

Sergej Brazdeikis — Wed, 22 Dec 2021 15:31:53 +0000

I have to say it was confusing - today existing docker image strapi/strapi has v3 under the hood and there is no simple way or docs to install Strapiv4 via Docker Compose.

Today the only way to create new project Strapi v4 is to use:

npx create-strapi-app@latest strapi-project --quickstart

and use it natively with node with

npm run develop

Today, I'm sharing my solution with Docker-compose.

Create Strapi project via Docker

Install Strapi project files:

set node version. Here is version 16.
set project name. I use strapi-project

docker run -it \
    -v $PWD:/data\
    -w=/data\
    node:16-alpine sh -c "yes | npx create-strapi-app@latest strapi-project --no-run --dbclient=mysql --dbhost=mariadb --dbport=3306 --dbname=strapi --dbusername=strapi --dbpassword=strapi"

All those CLI options are listed here on the Strapi Github repo - packages/cli/create-strapi-app/create-strapi-app.js With these options the configuration is setup correctly, together with the right database driver in package.json:

...
"dependencies": {
    ...
    "mysql": "2.18.1"
  }
...

Launch with Docker compose with Mysql together

Then lets launch the docker compose. Here as an option I use MariaDb as a choise.

For reference I used this folder structure:

Use following docker-compose.yml

version: '3'
services:
  strapi:
    image: node:16-alpine
    command: npm run develop
    labels:
      com.example.description: "Strapi project"
    environment:
      DATABASE_CLIENT: mysql
      DATABASE_HOST: mariadb
      DATABASE_PORT: 3306
      DATABASE_NAME: strapi
      DATABASE_USERNAME: strapi
      DATABASE_PASSWORD: strapi
      DATABASE_SSL: 'false'
    volumes:
      - ./strapi-project:/data
    working_dir: /data
    ports:
      - '1337:1337'
    depends_on:
      - mariadb

  mariadb:
    image: mariadb
    volumes:
      - ./mariadb:/var/lib/mysql
    environment:
      MYSQL_ROOT_PASSWORD: strapi
      MYSQL_DATABASE: strapi
      MYSQL_USER: strapi
      MYSQL_PASSWORD: strapi

Deploy with Docker compose to the production environment

For production deployment - please change the passwords above.

Also, it seems Strapi suggests using strapi start instead as launch command.

Happy content creation!