Move an already running process to screen

Ilhan Ates — Mon, 02 Oct 2023 21:01:00 +0000

You've realized some command you ran on your remote server is taking long, and want to move it to a Screen session. Tough situation, easy solution!

Use top or htop to grab the PID of the process.
Open screen in another ssh session / terminal tab.
Use reptyr (reptyr PID) to attach it back!

Building Multi-Arch Docker Images on Github Actions

Ilhan Ates — Mon, 02 Oct 2023 17:13:22 +0000

I recently needed to create multi-arch images so that I could spin up containers on my arm-based laptop. Github Actions only runs on x64-86 VMs, which can make building images for other architectures tricky.

Regular old Docker Build

name: Docker
on:
  push:
    branches: ['main']
jobs:
  docker:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
      - name: Login to Docker Hub
        uses: docker/login-action@v2
        with:
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_TOKEN }}
      - name: Build and push
        uses: docker/build-push-action@v3
        with:
          push: true
          tags: user/app:latest

With the above Github Actions file, we could build and push to Docker Hub easily. Now let's check out how we can use buildx to build for multiple architectures.

QEMU and Buildx

QEMU is a generic and open source machine emulator and virtualizer. When used as a machine emulator, QEMU can run OSes and programs made for one machine (e.g. an ARM board) on a different machine (e.g. your own PC). By using dynamic translation, it achieves very good performance.

Docker buildx is a Docker plugin that extends the building ability of images by using the BuildKit builder. It allows us to build images for different platforms and architectures.

With a combination of QEMU and buildx, which are conveniently also available as Github Actions, we can easily get our images built on emulated OSes and push onto the docker registry with multi-platform support!

Add the following after the registry checkout on our Github Actions workflow file:

- name: Set up QEMU
  uses: docker/setup-qemu-action@v2
  with:
    platforms: 'arm64,arm'
- name: Set up Docker buildx
  uses: docker/setup-buildx-action@v2

Note: The QEMU setup step has to be put before the buildx step!

By default, setup-qemu-action will create many VMs, most of which you might not care about. Make sure to put in the platforms you actually want to build for in the platforms: field to limit this to only ones you actually need.

Arch-specific Dockerfile instructions

The docker-build-push-action will provide the --platform flag to docker buildx. This will set the following build ARGs automatically:

TARGETPLATFORM - platform of the build result. Eg linux/amd64, linux/arm/v7, windows/amd64.
TARGETOS - OS component of TARGETPLATFORM
TARGETARCH - architecture component of TARGETPLATFORM
TARGETVARIANT - variant component of TARGETPLATFORM
BUILDPLATFORM - platform of the node performing the build.
BUILDOS - OS component of BUILDPLATFORM
BUILDARCH - architecture component of BUILDPLATFORM
BUILDVARIANT - variant component of BUILDPLATFORM

To use these ARGs in your Dockerfiles, you need to add them like the following example grabbed from Docker docs:

FROM alpine
ARG TARGETPLATFORM
RUN echo "I'm building for $TARGETPLATFORM"

End result

As a result, we can build and push multi-arch images by emulating other architectures on Github's x86_64 VMs using QEMU and building with docker buildx. We can even do arch-specific actions by using the ARGs automatically set by buildx.

Closing notes

Although this is convenient, no land without stones, or meat without bones.

There is one big and one relatively small problem this might cause for you.

1. Super long build times

Your builds might be taking very long after enabling multi-arch builds. This seems to be a known problem with emulating, but you could use some tricks to make less emulated code run while building.

In my case, I was building a Go application inside a build image, then copying that onto a target image in a two-step Dockerfile like so:

FROM golang:1.20-alpine as builder

WORKDIR /

RUN go env -w GOPRIVATE=github.com/myorganization

ARG GITHUB_TOKEN
RUN echo "machine github.com login github password $GITHUB_TOKEN" >> /root/.netrc

COPY . .
RUN go build "-ldflags=-s -w" -o /my-application cmd/main.go


FROM alpine:3.8

COPY --from=builder /my-application /app/my-app

WORKDIR /app
CMD /app/my-app

Instead of running the go build inside the Dockerfile, I switched to building directly inside the Github Actions using GOOS and GOARCH variables to specify building for a specific arch:

- name: Set up Go 1.20
  uses: actions/setup-go@v1
  with:
    go-version: 1.20

- name: Setup netrc
  run: |
    echo "machine github.com login github password ${{ secrets.ACCESS_TOKEN }}" > ~/.netrc
- name: Build application
  run: |
    go env -w GOPRIVATE=github.com/myorganization
    GOOS=linux GOARCH=amd64 go build "-ldflags=-s -w" -o build/my-app-amd64 cmd/main.go
    GOOS=linux GOARCH=arm64 go build "-ldflags=-s -w" -o build/my-app-arm64 cmd/main.go

And then copying those builds into the container using the buildx ARGs:

FROM alpine:3.8

ARG TARGETARCH
COPY build/my-app-${TARGETARCH} /app/my-app

WORKDIR /app
CMD /app/my-app

In the end, this lowered the whole execution time from ~6m to ~1,5m. I've heard people lower theirs down from ~1h to ~15m as well.

Pushing to AWS ECR creates multiple images

If you're pushing to AWS ECR, you might realize one push is creating many untagged images of type Image and one image of type Image Index containing all the tags, which looks like this:

This is actually normal, since pushing a multi-arch image really is just pushing untagged images for each arch and using an Image Index to redirect the user to the correct architecture image when pulling. The arch-specific images are untagged so that they cannot be easily discovered and pulled, and instead the Image Index that handles this is reached.

It looks like AWS might update the UI to make this look less cluttered in the future.

References

Send emails using utopia-php/messaging and MailSlurp

Ilhan Ates — Mon, 02 Oct 2023 17:11:33 +0000

If you've ever used a web application, chances are quite high that you've received an email from it.

If you're building a web application, chances are you might also want to send an email from it.

Well, using MailSlurp and utopia-php/messaging, you can do that with just a couple lines!

Getting Started

1. Getting the MailSlurp API key

First we will go ahead and sign up for a free personal MailSlurp account.

Go to mailslurp.com and click on Sign Up.
Sign up using your email or one of the OAuth providers.
Go to settings and copy your API key.
Optionally setup a domain under Inboxes > Domains and create an inbox under the Inbox tab. We will skip for the simplicity of this tutorial.

Note that unfortunately with a free plan, you can't send emails to big providers as stated here, so you might also want to upgrade your account.

2. Using utopia-php/messaging

Now that we got the API key, you will need to install utopia-php/messaging in your PHP project, following their README:

composer require utopia-php/messaging

<?php

use \Utopia\Messaging\Messages\Email;
use \Utopia\Messaging\Adapters\Email\MailSlurp;

$message = new Email(
    to: ['you@email.com'],
    subject: 'Hello World',
    content: '<h1>Hello World</h1>',
    from: 'your-inbox-identifier@mailslurp.com'
);

$messaging = new MailSlurp('YOUR_API_KEY');
$messaging->send($message);

And simple as that, the email should be sent!

DEV Community: Ilhan Ates