DEV Community: Adriano Galello

Curso ABC.Docker

Adriano Galello — Wed, 27 Dec 2023 09:54:01 +0000

Curso introductorio a Docker. Diseñado para todas las personas que tengan un conocimiento básico de desarrollo y quieran comenzar a utilizar Docker.
Obtendrás los conocimientos necesarios para comenzar a utilizarlo en tu día a día.

https://gdi3d.github.io/abcdocker/docs/#/

Build Safer Docker Images

Adriano Galello — Tue, 20 Jun 2023 06:25:14 +0000

When you're building your container image there are at least two effortless practices that you can follow:

Prefer minimal base images
Run process with Least privileged user

Prefer minimal base images

Base images not only help speed up your CI/CD pipelines, reduce storage space and network traffic. They also help you reduce the attack surface of your container.

Your container should only have the dependencies that it needs to run and nothing more.

I know, we all love to ssh into the container and use vim, ping, or some other tool to debug it. But being lazy can get us into trouble very quickly 😊.

Explore the following base image options:

Scratch If you are going to run a binary that doesn't need any other libs (GO programs for example).
Alpine The image is only 5 MB in size and has access to a package repository to install what you need.
If you want/need more, you can also try images with -slim version or even with -alpine.

Run process with Least Privileged User

Your process inside the container shouldn't run as root unless is mandatory.

Having a process running as root means that an attacker could gain root access and perform an attack on the host.

Imagine that someone mounts a volume to the container and an attacker was able to gain access to the container by the process running as root. They would be able to manipulate that external volume with no issues at all.

What if that container has been launched using --privileged flag?

The --privileged flag gives all capabilities to the container. When the operator executes docker run --privileged, Docker will enable access to all devices on the host as well as set some configuration in AppArmor or SELinux to allow the container nearly all the same access to the host as processes running outside containers on the host. Additional information about running with --privileged is available on the Docker Blog.
https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities

As you can see, potential issues begin to emerge very quickly.

How do you run a process as non-root?

In your Dockerfile make sure you create a new regular user and then activate it.

Here's an example of how it can be done

...
# crate a non-root user
ENV NONROOT_USER myappser
ENV NONROOT_UID 1000
ENV HOME /app

RUN adduser --disabled-password \
    --gecos "Default user" \
    --uid ${NONROOT_UID} \
    ${NONROOT_USER}

USER root
# all files in the home folder will
# be owned by the non-root user
RUN chown -R ${NONROOT_USER}:${NONROOT_USER} ${HOME}
USER ${NONROOT_USER}

# You can add your CMD after this if you want to
...

Photo credit: Francesco Ungaro

Every Day Docker Commands You Need To Learn

Adriano Galello — Tue, 10 Jan 2023 08:43:35 +0000

Interacting with containers

How to list all your running containers

docker ps

How to list all running and stopped containers

docker ps -a

How to run a command inside a container

docker exec CONTAINER_NAME_OR_ID command

Cool tip: You can use the first 4 characters of the container id instead of the full name or full id. It works with all commands 😎

How to access a Linux container

docker exec -it CONTAINER_NAME_OR_ID bash

# in case of no bash, like alpine versions for ex., you can try
docker exec -it CONTAINER_NAME_OR_ID sh

How to display logs live

docker logs -f CONTAINER_NAME_OR_ID

How to launch and run a container

docker run CONTAINER_NAME:TAG

# use -d and will be launched in the background
docker run -d nginx:latest

How to stop a container

docker stop CONTAINER_NAME_OR_ID

How to restart a container

docker restart CONTAINER_NAME_OR_ID

Managament Commands

How to list all images

docker image ls

How to delete an image

docker rmi IMAGE_NAME:TAG

# or
docker image rm IMAGE_NAME:TAG

How to delete all images

docker rmi -f $(docker images -qa)

How to remove all stopped containers (-f to force it)

docker container prune -f

How to remove all unused local volumes (-f to force it)

docker volume prune -f

How to remove all unused networks (-f to force it)

docker network prune -f

How to remove unused images (-f to force it)

docker image prune -f

How to stop all containers (-f to force it)

docker stop -f $(docker ps -qa)

How to delete all running and stopped containers (-f to force it)

docker rm -f $(docker ps -qa)

Clean up your environment by running multiple commands

docker stop -f $(docker ps -qa) && docker container prune -f && docker volume prune -f && docker image prune -f && docker network prune -f

Extras

How to build a docker image

docker build -t IMAGE_NAME:TAG location/to/Dockerfile

# Ex.: Running the command inside the of directory the Dockerfile
docker build -t mydockerimg:3.11 .

How to display all containers stats (% CPU, Mem, I/O, PIDS..)

docker stats

Image credit: https://www.pexels.com/@dana-tentis-118658/

Become a FullStack DEV

Adriano Galello — Sat, 07 Jan 2023 15:51:45 +0000

Are you trying to become a fullstack dev?

I've created a step by step tutorial to build your own web app from scratch using python, javascript, docker and more!

It's not a TO-DO app, it's something real, with real problems and using multiple technologies to achieve it.

https://gdi3d.github.io/learning-to-build/

Feel free to contact me if you need support with it 😉

Creating Distributed Tasks using Python+Celery+Docker

Adriano Galello — Thu, 25 Aug 2022 13:59:00 +0000

I have written a small and fun project for anyone who wants to learn how to build webapps from the ground up.

Today I want to share the infrastructure documentation about the project.

With this project, you will learn how to build a webapp to convert YouTube videos to MP3.

https://gdi3d.github.io/learning-to-build/#/docker/architecture

How to use Python Celery+Docker to convert YT Videos to MP3

Adriano Galello — Mon, 22 Aug 2022 07:13:18 +0000

This project was created as a tutorial for people that are trying to improve their coding skills.

You can check the full documentation to learn more, or just read this section

https://gdi3d.github.io/learning-to-build/#/app/converting-youtube-video-to-mp3-celery

And then build your own.

Your Own YouTube to MP3 Web App (Tutorial)

Adriano Galello — Mon, 08 Aug 2022 11:20:00 +0000

This project was created as a tutorial for people that are trying to improve their coding skills.

To-Do apps are ok for a quick look, but the best way to learn is by trying to do something a little bit complex and being able to change parts of the code and see what happens.

https://gdi3d.github.io/learning-to-build

Mac Battery Fully Charged Notification

Adriano Galello — Sat, 16 Jul 2022 16:15:43 +0000

I kept on forgetting my mac was charging and most of the time the battery would be at 100% for a few hours before I realize it.

So created a small script that will notify you when your mac is fully charged.

https://gdi3d.github.io/osx-mac-notify-battery-fully-charge/

Google Authenticator does not backup up your keys in your Google account

Adriano Galello — Mon, 03 Jan 2022 15:01:28 +0000

On Sunday noon I decided to install Lineage OS on my phone.

I knew I had everything backed up and the only thing I was worried about was Google Authenticator App. Then I remember that when I installed the app, google gave me recovery codes that I had safely stored away.

After an hour the whole installation was over and it was time to restore my apps and settings in the new OS.

Once the Google Authenticator app was installed I opened the app and notice it was empty. That's ok, I thought, pretty sure I've to enter some of those codes to recovery my keys.

But I didn't see any options to do that.

https://www.youtube.com/watch?v=pl5gIYq3PUo

Play me for dramatic effect

It turns out that Google Authenticator does not save your accounts in your Google Account. They exist only on your phone. (You can search about this if you don't trust me)

After having 5 heart attacks in a row, I started to think: Well... how many accounts did I had set up in there?? and for what sites? and what happened to those codes, what were they for?.

And so it began the task to recover my 8 2FA secured accounts without the 2FA tokens available. Oh, and those codes were just for recovering the Google account and nothing else (big mistake on my part).

Luckily I had a lot of them open on my computer and was able to quickly disable and re-enable to add them back again on the Google Authenticator app.

Some websites know that stupid people like me do stupid things and give you a way to disable your 2FA, with a code that they provide when you enable the 2FA, or offer a recovery mode sending an SMS to your phone.

But others just give you the finger and you get locked out.

One of those sites, not going to disclose which one, didn't offer me any alternative method like sending an SMS when you don't have the 2FA token. And this site was EXTREMELY IMPORTANT to me.

After panicking a little more I started to look at how to solve it.

I searched for any hacks, tricks, or anything to do, but didn't find anything. So I decided to try the support number.

No one answered.... I waited for about an hour or so and called again. This time someone did answer and after a 15 minutes call, they were able to set the SMS method as the primary one. Luckily I had that enabled and set up before when I enabled the 2FA.

This whole mess started at around 13:00 and ended at 18:30.

How to backup your Google Authenticator then?

There's no tool or option to do it.

The only thing you can do is use the option to export your keys to another device.

This will show you a QR that you can scan with your new device to transfer the accounts. But you can't screenshot it, save it, or anything like that (for security reasons).

My solution is to start looking for a new MFA App, there aren't many out there and most of them use their cloud to store your keys.

Fun Fact: If I had searched 'How to backup google authenticator' I would have found out all of this on my first click 🥲

A Tiny Microservices for Emails and Calendar Invitations

Adriano Galello — Tue, 23 Nov 2021 15:05:29 +0000

I've created a tiny Microservice in Go that can be used by anyone building an app with the need of sending emails and calendar invitations.

https://gdi3d.github.io/calenvite/

A simple microservice designed in GO using Echo Microframework for sending emails and/or calendar invitations to users.

Features

Send emails using your Mailgun API credentials.
Send using a standar SMTP server.
Support for HTML and Plain Text emails.
Calendar invitation with RSVP.
Docker image is built using multistage and alpine image to keep it as small and secure as possible.

How to Use

Build the Docker image

Download the repo and build the docker image:

  $ git clone https://github.com/gdi3d/calenvite
  $ docker build -t calenvite_svc:latest .

and use provided docker-compose files for more info.

Or Build the binary

$ git clone https://github.com/gdi3d/calenvite
$ go get -d -v
$ go mod download
$ go mod verify
$ go build -a -o calenvite

# run de service
$ ./calenvite

Set the Env vars

There's a few env vars that you need to set when you launch the container in order to work:

# If you want use Mailgun API:
CALENVITE_SVC_MAILGUN_DOMAIN: The domain from which the email are going to be sent
CALENVITE_SVC_MAILGUN_KEY: The Mailgun API secret key

# If you want to use SMTP:
CALENVITE_SVC_SMTP_HOST: The host/ip of the SMTP server
CALENVITE_SVC_SMTP_PORT: The port of the SMTP server
CALENVITE_SVC_SMTP_USER: The username to authenticate to the SMTP server
CALENVITE_SVC_SMTP_PASSWORD: The password to authenticate to the SMTP server

# common to both options
CALENVITE_SVC_EMAIL_SENDER_ADDRESS: The email address that would be used to send the email (this value will be used in the FROM part of the email)
CALENVITE_SVC_SEND_USING: MAILGUN or SMTP
CALENVITE_SVC_PORT: Port to expose (optional, default: 8000)

Sample docker-compose files included

# mailgun-docker-compose.yml

version: "3.9"
services:
  app_backend:
    image: calenvite_svc:latest
    ports:
      - "8080:8000"
    environment:
      - CALENVITE_SVC_MAILGUN_DOMAIN=mycooldomain.com
      - CALENVITE_SVC_MAILGUN_KEY=abcd1234
      - CALENVITE_SVC_EMAIL_SENDER_ADDRESS=no-reply@mycooldomain.com
      - CALENVITE_SVC_SEND_USING=MAILGUN

# smtp-docker-compose.yml

version: "3.9"
services:
  app_backend:
    image: calenvite_svc:latest
    ports:
      - "8080:8000"
    environment:
      - CALENVITE_SVC_SMTP_HOST=smtp.mailprovider.com
      - CALENVITE_SVC_SMTP_PORT=587
      - CALENVITE_SVC_SMTP_USER=mysmtpuser
      - CALENVITE_SVC_SMTP_PASSWORD=shhhh.is.secret
      - CALENVITE_SVC_EMAIL_SENDER_ADDRESS=no-reply@mycooldomain.com
      - CALENVITE_SVC_SEND_USING=SMTP

API Docs

Healtcheck Endpoint

A healthcheck endpoint to test if the service is up and running and a valid configuration is present.

Note: This healtcheck is only going to check that the environment vars are defined. It will not check if the credentials are valid or not

URL

/healthcheck/

Request Example

curl --location --request GET 'http://127.0.0.1:8080/healthcheck'

Responses

Service ok

HTTP/1.1 200 
Content-Type: application/json; charset=UTF-8
Vary: Accept-Encoding

null

Service not working

HTTP/1.1 500 Internal Server Error 
Content-Type: application/json; charset=UTF-8
Vary: Accept-Encoding

{
    "message": "Internal Server Error"
}

Invite Endpoint

Send email, and optionally, an calendar invitation with RSVP to the users.

URL

/invite/

Payload

{
    "users": [
        {
            "full_name": "Eric Cartman",
            "email": "ihateyouguys@southpark.cc"
        },
        {
            "full_name": "Tina belcher",
            "email": "aaaooo@bobsburger.com"
        }
    ],
    "invitation": {
        "start_at": "2030-10-12T07:20:50.52Z",
        "end_at": "2030-10-12T08:20:50.52Z",
        "organizer_email": "meetingorganizer@meeting.com",
        "organizer_full_name": "Mr. Mojo Rising",
        "summary": "This meeting will be about...",
        "location": "https://zoom.us/332324342",
        "description": "Voluptatum ut quis ut. Voluptas qui pariatur quo. Omnis enim rerum dolorum. Qui aut est sed qui voluptatem harum. Consequuntur et accusantium culpa est fuga molestiae in ut. Numquam harum"
    },
    "email_subject": "You've just been invited!",
    "email_body": "<html><body><h1>email body about the invitation/event</h1></body></html>",
    "email_is_html": true
}

Notes about fields:

If you don't need to send a calendar invitation you can omit the field invitation
If you want to send plain text messages set the key email_is_html to false

Request example

curl --location --request POST 'http://127.0.0.1:8080/invite/' \
--header 'Content-Type: application/json' \
--data-raw '{"users":[{"full_name":"Eric Cartman","email":"ihateyouguys@southpark.cc"},{"full_name":"Tina belcher","email":"aaaooo@bobsburger.com"}],"invitation":{"start_at":"2030-10-12T07:20:50.52Z","end_at":"2030-10-12T08:20:50.52Z","organizer_email":"meetingorganizer@meeting.com","organizer_full_name":"Mr. Mojo Rising","summary":"This meeting will be about...","location":"https://zoom.us/332324342","description":"Voluptatum ut quis ut. Voluptas qui pariatur quo. Omnis enim rerum dolorum. Qui aut est sed qui voluptatem harum. Consequuntur et accusantium culpa est fuga molestiae in ut. Numquam harum"},"email_subject":"You'\''ve just been invited!","email_body":"<html><body><h1>email body about the invitation/event</h1></body></html>","email_is_html":true}'

Responses

Successful

HTTP/1.1 200 
Content-Type: application/json; charset=UTF-8
Vary: Accept-Encoding

{
    "message": "SENT_OK",
    "status_code": 200,
    "error_fields": null
}

Field missing/invalid

HTTP/1.1 400 BAD REQUEST
Content-Type: application/json; charset=UTF-8
Vary: Accept-Encoding

{
    "message": "INVALID_PAYLOAD",
    "status_code": 400,
    "error_fields": [
        {
            "field": "email_body",
            "message": "",
            "code": "required"
        }
    ]
}

Error

HTTP/1.1 500 Internal Server Error
Content-Type: application/json; charset=UTF-8
Vary: Accept-Encoding

{
    "message": "ERROR",
    "status_code": 500,
    "error_fields": null
}

Questions, complains, death threats?

You can Contact me 🙋🏻‍♂️ on LinkedIn if you have any questions. Otherwise you can open a ticket 😉

Docker environment variables naming

Adriano Galello — Mon, 02 Aug 2021 09:16:12 +0000

When you start to build an application you will definitively want to use environment variables for sensitive values like passwords, or configurations.

These values should never be hardcoded since it's a security issue. It will also help you make your application easier to maintain.

Typically you will also have one configuration file per environment:
Development, Preproduction, and Production.

Take a look at the following docker-compose for our next unicorn app:

version: "3.9"
services:
  app_backend:
    image: killerapp:latest
    ports:
      - "8080:8000"
    environment:
      - DB_HOST=${DB_HOST}
      - DB_USER=${DB_USER}
      - DB_PASSWORD=${DB_PASSWORD}
      - DB_SCHEMA=${DB_SCHEMA}

This looks about right, and you're probably familiar with this kind of configuration. But this presents a problem that could be potentially dangerous.

There's a chance that someone accidentally set the env vars for one environment into another one.

Imagine what can happen if you end up using the production values in your preproduction or development environment.

One best practice is to have different variables for your environments.

There are other ways to prevent these issues like having your environments in different servers, VPC's, accounts, etc.

In our example, the docker-compose.yml will look like this:

version: "3.9"
services:
  app_backend:
    image: killerapp:latest
    ports:
      - "8080:8000"
    environment:
      - DEV_DB_HOST=${DEV_DB_HOST}
      - DEV_DB_USER=${DEV_DB_USER}
      - DEV_DB_PASSWORD=${DEV_DB_PASSWORD}
      - DEV_DB_SCHEMA=${DEV_DB_SCHEMA}
      - PRE_DB_HOST=${PRE_DB_HOST}
      - PRE_DB_USER=${PRE_DB_USER}
      - PRE_DB_PASSWORD=${PRE_DB_PASSWORD}
      - PRE_DB_SCHEMA=${PRE_DB_SCHEMA}
      - PRO_DB_HOST=${PRO_DB_HOST}
      - PRO_DB_USER=${PRO_DB_USER}
      - PRO_DB_PASSWORD=${PRO_DB_PASSWORD}
      - PRO_DB_SCHEMA=${PRO_DB_SCHEMA}

This way is much harder for a developer/DevOps to accidentally use the production credentials in another environment.

The takeaway is that if you should have an environment-sensitive variable naming approach to make them crystal clear. This way, anyone that it's not familiar with the app config will be less prone to make mistakes.

How to connect docker containers

Adriano Galello — Thu, 29 Jul 2021 08:20:59 +0000

Did you know that you can use the service name of a container or id to connect to it on Docker?

If you have used the --link option when launching a container or docker compose, you can call the container by its name or id from other containers. But how does this work?

The answer is simple: DNS.

Whenever you launch a container, Docker provides a network for you and uses the DNS service provided by the Docker daemon.

You can override the default docker network and DNS settings Check the docs here

Scenario

Let's assume that you have 3 tier application: Frontend, Backend, and a Database.

Your backend needs to be able to connect to the database. The safest way to do so is to keep your database in a private network to connect to it.

With docker you can use the service name or container ID as the server name in your configuration file.

Ex.:

cn = mysql.connect(user='produser',
                 password='secretvalue',
                 host='svc_database', # this is the service name in the incoming example
                 database='mysuperapp')

Launching a few containers

Clone the following repository using the command below:

git clone git@github.com:gdi3d/docker-dns.git .

Now take a look at the file docker-compose.yml in the repository you just cloned.

version: "3.9"  # optional since v1.27.0
services:
  svc_frontend:
    image: busybox:latest
    command: top
  svc_database:
    image: busybox:latest
    command: top
  svc_backend:
    image: busybox:latest
    command: top

You can see three services defined:

svc_frontend: App frontend
svc_database: App database
svc_backend: App backend

To make things easier we're using busybox image to simulate the services.

Launch the containers using docker-compose up -d.

You can check that the containers are running using docker ps

The Test

We're going to test the following:

Where's the Docker DNS service running at (the ip address)
Try to ping a container by its service name and ID.
Are the requests being resolved by Docker DNS service?.

These are our containers with their service name and IDs:

Service Name	ID
svc_frontend	1ee9c76938ee
svc_database	fe0228bd99a5
svc_backend	a96e0cb74351

Where's the Docker DNS service at?

Let's access svc_backend container and type cat /etc/resolv.conf

Here you can see that nameserver has been pointed to 127.0.0.11. That's the Docker DNS service provided by the Docker daemon.

Pro tip: You don't need to type the whole ID of the container to perform an operation, just use the first two or three characters

Try to ping a container by it's service name and ID

You can check the connectivity against the svc_database using the ping command:

As you can see, we obtain the same response when calling it by its service name svc_database or docker id fe0228bd99a5.

They both resolve to the same IP 172.19.0.3 as expected.

Are the requests being resolved by Docker DNS service?

Let's use nslookup to have a detailed report on what's going on when we try to call a container by its service name or ID.

Running these two commands will give us a better understanding of what's going on: nslookup svc_database and nslookup fe0228bd99a5

Server:     127.0.0.11
Address:    127.0.0.11:53

Non-authoritative answer:
Name:   svc_database
Address: 172.23.0.2

--------------------------

Server:     127.0.0.11
Address:    127.0.0.11:53

Non-authoritative answer:
Name:   fe0228bd99a5
Address: 172.19.0.3

From the results of nslookups we can see that the request is made using the docker DNS service at 127.0.0.11 and that, again, using the service name svc_database or the container id fe0228bd99a5 the destination is the same ip 172.19.0.3

DEV Community: Adriano Galello

Curso ABC.Docker

Build Safer Docker Images

Prefer minimal base images

Run process with Least Privileged User

How do you run a process as non-root?

Every Day Docker Commands You Need To Learn

Interacting with containers

Managament Commands

Extras

Become a FullStack DEV

Creating Distributed Tasks using Python+Celery+Docker

How to use Python Celery+Docker to convert YT Videos to MP3

Your Own YouTube to MP3 Web App (Tutorial)

Mac Battery Fully Charged Notification

Google Authenticator does not backup up your keys in your Google account

How to backup your Google Authenticator then?

A Tiny Microservices for Emails and Calendar Invitations

Features

How to Use

Build the Docker image

Or Build the binary

Set the Env vars

Sample docker-compose files included

API Docs

Healtcheck Endpoint

URL

Request Example

Responses

Invite Endpoint

URL

Payload

Request example

Responses

Questions, complains, death threats?

Docker environment variables naming

How to connect docker containers

Scenario

Launching a few containers

The Test

Where's the Docker DNS service at?

Try to ping a container by it's service name and ID

Are the requests being resolved by Docker DNS service?

Further Reading