DEV Community: MrViK

Bringing the power of Go templates to service descriptions

MrViK — Mon, 11 Jan 2021 19:07:09 +0000

Hey, on previous articles from this series I mentioned some missing features on the Go-PID1 and its service launcher.

To recap:

We have no templates - It's a big deal
Support for reloading services
Better handling of mounts
Other things like setting up the keymap for console
(Added now) Init is very fragile, a single fail on service-launcher halts the system.

And I fixed all of them and the fixes have been merged into master by this time.

But we're here to talk about templates. Let's see how systemd does it:

From systemd.unit(5) on section SPECIFIERS:

Many settings resolve specifiers which may be used to write generic unit files referring to runtime or unit parameters that are replaced when the files are loaded. Specifiers must be known and resolvable for the setting to be valid. The following specifiers are understood:

"%i" Instance name For instantiated units this is the string between the first "@" character and the type suffix. Empty for non-instantiated units.

There are a lot more of specifiers on that list, but let's focus on this one (and it's unescaped variant %I).

If you have systemd on the PID1 you can do systemctl cat getty@tty1.service to see the code under those templates.

systemd templates always come with an @ symbol and the text between the '@' and the end of the name (w/o .service or whatever) is the "argument".

Here is the service description for getty@tty1, note that is a template linked to getty@.service.

# /usr/lib/systemd/system/getty@.service
#  SPDX-License-Identifier: LGPL-2.1-or-later
#
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.

[Unit]
Description=Getty on %I
Documentation=man:agetty(8) man:systemd-getty-generator(8)
Documentation=http://0pointer.de/blog/projects/serial-console.html
After=systemd-user-sessions.service plymouth-quit-wait.service getty-pre.target

# If additional gettys are spawned during boot then we should make
# sure that this is synchronized before getty.target, even though
# getty.target didn't actually pull it in.
Before=getty.target
IgnoreOnIsolate=yes

# IgnoreOnIsolate causes issues with sulogin, if someone isolates
# rescue.target or starts rescue.service from multi-user.target or
# graphical.target.
Conflicts=rescue.service
Before=rescue.service

# On systems without virtual consoles, don't start any getty. Note
# that serial gettys are covered by serial-getty@.service, not this
# unit.
ConditionPathExists=/dev/tty0

[Service]
# the VT is cleared by TTYVTDisallocate
# The '-o' option value tells agetty to replace 'login' arguments with an
# option to preserve environment (-p), followed by '--' for safety, and then
# the entered username.
ExecStart=-/sbin/agetty -o '-p -- \\u' --noclear %I $TERM
Type=idle
Restart=always
RestartSec=0
UtmpIdentifier=%I
TTYPath=/dev/%I
TTYReset=yes
TTYVHangup=yes
TTYVTDisallocate=yes
IgnoreSIGPIPE=no
SendSIGHUP=yes

# Unset locale for the console getty since the console has problems
# displaying some internationalized messages.
UnsetEnvironment=LANG LANGUAGE LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT LC_IDENTIFICATION

[Install]
WantedBy=getty.target
DefaultInstance=tty1

On this unit all occurrences of "%I" get replaced by the argument.

Ways of implementing this

I initially thought about using Regex replace. And it should work but we have text/template, a powerful way of creating data driven templates.

What data? Well, copying the good things from systemd, when a service has an @ symbol on it's name, we pass the file to the templates interpreter with the filename and the argument.

Environment exposed to templates

Data -> The Argument variable based on what follows the '@' (before .yml or .yaml).
Functions -> os.Getenv is exposed as Getenv so you can retrieve environment variables.

Currently there are no plans on adding more things, but the data struct and funcmap are easily extensible. Feel free to add functions adapted to your needs.

Trying templates

So how do they look.

Let's see the template for getty. We took a look to the systemd approach (which includes a lot of things). How was the world before templates on Go-PID1?

---
name: agetty@tty1
description: Getty on tty1
exec: setsid
arguments:
  - --wait
  - /sbin/agetty
  - --noclear
  - tty1  # Hardcoded values
  - linux
restart: always

Previously each file was only allowed to define a single unit. Now each file can define a slice of services so templates are more powerful yet.

---
# vim: filetype=yaml
{{with .Argument}}  # If no argument, no services are defined here
- name: agetty@{{.}}  # Dynamic name!
  description: Getty on {{.}}. We have templates. yay!
  exec: setsid
  arguments:
    - --wait
    - /sbin/agetty
    {{- if eq . "tty1"}}
    - --noclear  # Conditionals inside templates. Take this!
    {{- end}}
    - {{.}}  # This argument takes the value from .Argument
    - {{Getenv "TERM"}}  # We call the Getenv function from template
  restart: always
{{end}}

And how do they look on the FS?

And another new feature is the hot reload. Add, remove or update services and do slc reload to apply changes.
So if you link another service to a template and do a reload, the service will be read and started.

So cool, isn't it?

Following slippery processes

MrViK — Thu, 07 Jan 2021 13:17:32 +0000

Hey, if you followed the previous parts about writing a PID 1 (and a service launcher) with Go you may expect this.
The approach of the created service-launcher is to follow a single process, so if it forks and main process exits, the service fails/succeeds with code from main process instead of following the forked one.

To follow those processes (that can spawn several processes) we should tweak the service-launcher to follow a variable number of them.

Instead of doing this and thus, making a monolith around service-launcher I have decided to create a helper to launch and follow processes that fork.

Important idea: What do we call "main" process?
We call main process to the process spawned by service-launcher. That process will be monitored and will decide if a service is running or exited (failed/succeeded).

Default behavior

From fork(2):

fork() creates a new process by duplicating the calling process. The new process is referred to as the child process. The calling process is referred to as the parent process.

So if we call a fork-ing process and do pstree, there will be 2 identical processes, one as parent and the other as child.

Take the following C code:

# include <unistd.h>
# include <sys/types.h>
# include <sys/wait.h>

int main() {
    pid_t pid=fork();
    switch(pid){
        case -1:
            return -1;
        case 0:
            // Forked process. Sleep and exit
            sleep(20);
            return 0;
    }

    wait(NULL); // Wait for its child

    return 0;
}

If we run htop or pstree while running that code we get the following tree:

Ok, that's right, but, what if the parent process exits before child? To achieve this, remove the wait call from C code. In sake of brevity I won't show it, but that "forked" process bubbles trying to find the next CHILD_SUBREAPER. The PID 1 acts as child reaper implicitly, so if none of the processes on tree has that role, PID 1 will become the parent of our orphaned process. We lost it.

This is the default behavior, and the process that launched ./fork cannot follow the fork(2)-ed process.

Having control over them

So we want a program able to follow all its descendent processes.

Thanks Linux for cgroups(7). We'll be using the version 2 (unified hierarchy).

How do `cgroups2` work

First, you need cgroup2 mounted on /sys/fs/cgroup. service-launcher does mount this filesystem by default on init.

Now, simply cd to /sys/fs/cgroup and mkdir. cd into that dir and surprise! All the enabled controllers are present just after you did mkdir. Add your current shell PID to the cgroup echo $$ >> cgroup.procs. You can also cat that file to see all PIDs inside.

Some rules that apply to "adding processes to cgroups":

If the added process already has child processes, child processes are not moved inside the cgroup, only the specified PID.
Processes and their threads are added into the cgroup.
Child processes created by a process inside the cgroup, are also added automatically to the cgroup (and their PID to cgroup.procs)

If we take this rules we can make a program that:

Creates a new cgroup with an specified name
Executes another process inheriting an fd ( with options O_APPEND|O_WRONLY) to the cgroup.procs file. Arguments are to be passed to the real executable process. This child process does:
- Append it's PID to the fd 3 (first fd inherited)
- Closes the fd
- Calls execve(2) (golang.org/x/sys/unix.Exec in Go) with arguments from cmdline.
Child process is followed until it fork(2)s and exits.
Now their children (from cgroup.procs) are watched.
Exits with status from last process in cgroup
Removes cgroup before exiting

How do we "watch". First thing I thought is to create a ticker and unix.Wait4 with WNOHANG but then came across a section on cgroups(7) talking about the cgroup.events file. So we could use inotify(7) to watch them for changes and react to process changes inside the cgroup.

Ok, this works, but what about the process tree? It still holds the same default behavior explained previously.

Changing the default behavior

We simply call prctl(2) with PR_SET_CHILD_SUBREAPER to become a subreaper on the current tree so orphaned child processes are attached to us.

But with great power comes great responsibility. If this process becomes a subreaper, it must wait(2) child processes when appended. To achieve this we listen to our favourite signal, SIGCHLD, wait processes and check if cgroup still contains at least 1 process. If cgroup is empty, our exit code will be the code from last process inside the cgroup.

Code & Run

Main process (supervisor). cmd/run-in-cgroup
Subprocess (executor). cmd/exec-in-cgroup

Let's run dhclient on eth0. This process forks and previously the service succeeded when main process exited, leaving the forked process untracked.

We changed the service file as follows:

---
name: dhclient@eth0
description: DHCP client for eth0
exec: run-in-cgroup
arguments:
  - -name=dhclient-eth0
  - --
  - dhclient
  - eth0

The result looks like this:

So service-launcher detects dhclient as running because the monitor process is alive and trackable. Also dhclient is a child of its monitor process so the tree is clean and there are no lost things floating around.

Do you have any thoughts here? Feedback? Can we make this better? Write a comment or feel free to create an issue.

Help is needed with the name, I don't feel good calling this thing go-pid1 (and my lack of creativity is notable, I thought gp1 😅).

On next article I'll be making it work in a complete system with Void + SDDM + KDE (or something simpler, but KDE should work).

See you next time.

Writing an init with Go (part 3, running it!)

MrViK — Tue, 05 Jan 2021 16:36:59 +0000

Hey there. Sorry for the delay after parts 1 and 2. I've been busy doing some changes, will make a 4th article about those.

As said before, we'll be running this init on a qemu VM.

While running it, a thing I have noticed (running getty only) is that we're used to udev where all things Just Work™️ when plugged. systemd has involved into all those things contributing to create mysticism around the PID 1 and what it does.

No more theory, let's just dive inside this thing.
For the test I used a VM with root on 9p (could be NFS, but was just another test) so the dev and test env are shared and I can push changes and reboot to get them (at first I did poweroff, mount, copy, unmount and run it again).

I did test on 2 chroots, one with Arch Linux and another one with Void (it's init-agnostic so is easier to get things done). Also compiled a kernel to have all virtio modules builtin (Arch doesn't have a working udev and didn't bother to get one so this is the bruteforce solution).

Compiling

Source is here: https://gitlab.com/mrvik/go-pid1

Go has one of the fastest compilers, so you can't slack off.
We're using make to manage all the targets so you need:

make (tested with GNU Make)
go toolchain (package is called go or golang depending on the distro)

Then make all or simply make and done, you have an out dir with all the utils statically linked (as they are in pure Go).

You should copy them to your VM under /usr/local/bin to avoid conflicts and link init and service-launcher to root (service launcher path is hardcoded, this should be changed).

There is a folder on project called examples, copy it as /etc/yml-services so we have something to talk about.

Void w/ go on PID 1

Here is my qemu cmdline:

qemu-system-x86_64 -accel kvm \
  -kernel ../linux-5.10.4/arch/x86/boot/bzImage \
  -initrd root/boot/initramfs-linux-fallback.img \
  -append "root=192.168.1.10 rootfstype=9p rootflags=aname=$PWD/void,msize=16000,version=9p2000.L,uname=root,access=user rw init=/init ip=dhcp loglevel=3" \
  -m 1024 -nic user,model=virtio-net-pci -vga qxl

kernel is from the compilation dir (latest stable release at the time of preparing this). Did also test with mine (vanilla linux package from Arch) and worked, so Void's should also work.
initrd comes from my system (yes, kernel version doesn't match, but it doesn't matter as all modules are already built in). It was created using mkinitcpio with the net hook to enable networking on the early userspace. Maybe this could be done with dracut but, whatever, it works.
append has a lot of joy. I'm using v9fs as root so all those arguments are for that purpose and ip=dhcp does what you already know. The most important thing here is init=/init as it's the location where I did copy the init target. Setting loglevel is important to avoid being flooded with kernel logs.
Other boring things are the memory, a virtio net device (needed for 9p on root) and the VGA model.

And here we are! All lines printed by go-pid1 have the same format (from log package). We can login and see what's going on here:

htop shows the current hierarchy, every process is child of service-launcher (who has started everything present on /etc/yml-services ending with .yml or .yaml. dhclient is forking, so he's now child of PID 1 (see next article on how do we manage this now).

The control socket

On Part 2 we mentioned service-launcher exposes a socket so we can control daemons and a utility called slc to connect to the socket w/o openbsd-netcat installed (-U option support).

List contains currently loaded services and their state. It's worth noting that dhclient shows as not started (not running) as this process has forked and main process exited. We cannot track it's descendents (see next article to track progress on this).

We can use the restart command (also the start and stop) with a service to make the needed actions.
Also, the journal command shows logs for the specified service (we face a lot of "operation not permitted" errors w/ root on 9p).

And it works. The socket can potentially run or expose any function from the service-launcher.

Run init systems for fun and... fun

As we're not dropping capabilities nor doing any type of confining, we can virtually execute all processes.
The only limitation is the PID. Init must be PID 1, but it's already coped.

pid_namespaces(7) to the rescue! Look at cmd/run-on-pid-ns. It will create a new pid_namespace(7), time_namespace(7) and mount_namespace(7). The time namespace is just to isolate uptime inside the namespace. Mount namespace allows us to have a different procfs and whatever mounts does the other init system.

Let's try it. The examples/systemd.yml.disabled service starts /sbin/init using run-on-pid-ns. It's disabled by default, you may want to disable all other services (at least agetty to avoid clashes).

We're now on Void so let's try runit first.

Here we can see the full process hierarchy and /sbin/init is not PID 1. That's because htop uses /proc which didn't get remounted so we see the full hierarchy (but cannot kill them because the PID is not accessible heh!).

So if we mount procfs again, we get processes on the current namespace and htop (and others) work properly.

Another caveat is the reboot/poweroff logic. We cannot do it from the init system as it won't be able to do the reboot(2) syscall (blocked because it's working on a pid namespace). We can anyway do slc reboot or slc poweroff.

Let's try systemd now. systemd is more complex and makes use of a lot of things, but works surprisingly well.
I'm using Arch Linux to test as it uses systemd by default.

As you can see, it works but we can see all processes until we mount procfs.

This init is isolated on a mount namespace so, if we launch htop from inside systemd and outside, we see the effect.

Outside of the namespace

Inside of the namespace

The /run dir is not remounted, so the socket exposed by service-launcher is there and we can connect to it.

What's next

We're missing some useful features systemd has (ok, we're missing a lot).

Templates: so useful for things like agetty@<tty> or dhclient@<interface>. It's queued but not planned.
Follow forking processes. Some processes do fork(2) and we can't follow them. We're currently working here, stay tuned on the 4th article to follow the process.
Better handle of mounts. Also, working on this, but w/o priority.
Set machine hostname at start from /etc/hostname.
Document commands on the socket.
You may have missed the reload command (something like daemon-reload on systemd. It's not currently implemented, but its planned.
Fix race conditions on service-launcher. More likely, the process state. This may be looked with the reload feature.

There are some other features on systemd that we're not going to support like:

Confining all processes on cgroups
Derived from previous point, do firewalling on cgroups
Securing folders against those processes (RO home directory or private tmp).

Next article will address "How to follow slippery processes". Stay tuned for more.

Writing an init with Go (part 2)

MrViK — Fri, 01 Jan 2021 13:49:40 +0000

Welcome to the second article on this series "Writing an init with Go".

As mentioned on first article, we're making an init with a decoupled service-launcher. This way we keep the PID 1 small on features (just added a handler for SIGUSR2 to reboot with kexec today).

Something being argued against decoupling service management form PID 1 (systemd has this coupled) is the service manager won't be able to track processes if they fork(2) away. That's almost true and is a caveat on the current code.

Hands on

The first thing that comes to our head is "we have to initialize all services" and that's true, but there is a prerequisite on that: filesystems are not mounted at this moment (or we don't expect anything but / to be mounted).
Buut, first we need to watch signals so we exit properly when killed.

Signals

A very basic thing here: we watch for SIGINT (PID 1 will kill us with that signal when a reboot(2) is needed) in order to do the teardown. And SIGCHLD is needed to notify process watchers to check if their process is alive (we call this the deadChildrenChannel and will take over this on services section)

Filesystems

So here we start the real work, we mount some needed tmpfs filesystems (/dev and others are already mounted at the time init is run). Our current approach is to exec mount -a and done, everything in fstab is mounted now. This is too lazy, to be fair filesystems don't get too much love here. This is truly a point to improve.
Call to unmount is deferred so it gets executed on main() exit (maybe a panic).

Services

Here is where the fun begins. All this work is done on the services package, in case you want to give it a look first.

Each service is a yaml file because, who doesn't like yaml?
Description of units is very basic and near of the underlying
cmd/exec.Cmd struct (e.g. arguments are an array of strings and are separated from the exec line, which will be looked on $PATH).

A service looks like this

After all units are read, their names are mapped so it's easier to lookup dependencies by their name. (We'll talk about them on the Dependencies section).

After units are read they are checked as they must have a valid config (no empty name, no empty exec line and valid restart value). Dependencies whose yaml cannot be parsed or config is invalid are skipped (not included on the dependencies slice) and reported to stderr.

Then we move on...

Launching services

Each service may have dependencies, so we start first the services w/o dependencies. Those are launched in parallel with a max goroutine count of runtime.NumCPU().

Then the rest of services which are not started yet are launched. Their dependencies are looked on the services map and cached on a field of the Service struct. If there are dependencies not found or a basic cyclic dependency chain is detected, dependency will be skipped with an error.

Each Service struct has an Start function that gets called. Note that we call all this functions in parallel w/o any concurrency control (I mean, the number of goroutines being launched) because most of them will be sleeping while their dependencies get started.

We're starting to talk about dependencies, so let's address them.

Dependency management

This is the most complex thing on this service launcher.

Dependencies have a lot of theory but copying from the systemd devs (given a service A that depends on B):

Service A has to be started, we ensure B is running (This is working)
When service B is stopped, all services depending on it have to be stopped (including A). This is not working (you just shot your feet IMO). Anyway it would be cool but challenging to do this type of traceback finding dependents (we only look for dependencies ATM).

So basic dependency management is done. I came across 3 ways of handling dependencies before settling with the "launch every service with dependencies concurrently and wait for dependencies to be started (or errored) before we really start (or error on dependency failed) each service".

After all dependencies are settled the service is started (call exec) but if one of them has an error (bad config or failed to start), service will exit with error w/o trying to start it.

Tracking children

Why do we need tracking?
Well, you may not like agetty being killed after you log out. You may want it to be restarted so you can login again. We need to watch the agetty process to achieve this.

After a service is started, we have it's main process PID on the os.Process type so we can track it. Tracking processes isn't free, the current approach is to have a goroutine for each service.
This goroutine is almost always sleeping and waked up on demand, meaning that unless a process attached to service-launcher is dead (we get SIGCHLD), no one will move a finger.

Also tracker goroutines have a context. If a context exits, goroutine will return immediately.

This goroutine has a given design due to some limitations on the exec.Process type. That struct has a Wait function, but it doesn't allow context cancellation and as such, we can't use it. Instead we use the syscall.Wait4 with WNOHANG on flags. This way we can probe our process aliveness w/o locking until it exits and wait for another dead child to be reported from channel to check it again.
It isn't the best approach but it seems simple.

Who notifies of a "dead child"? The signal watcher receives from kernel a SIGCHLD (but don't say the exact PID hah!) so we notify all Services so they can see if the dead children is theirs. Note that kernel won't be repareting dead children to our PID so one of the services must match. Then the watcher can start doing the needed actions.

Needed actions are basic, but I think it's enough. You can specify a policy to restart the process, "always" (main process exists and we start it again. You may not like to do this on forking processes), "failed" (only restarted if main process exit!=0) or "never" (process is waited and it does nothing even on errors).

As said, when a process fork(2)s we're not notified and can't follow it. See the article by Lennart Poettering "Rethinking PID 1", it has a lot of interesting things. He talked about using cgroups so we're able to control child processes of our current service.
This project has not implemented cgroups still, but may do it.

Killing processes

The time has arrived. On teardown (or stop command received from socket, we'll be over the socket in a minute) we have to kill all services processes.

Again, the current approach is to kill the main process. The expected behavior of a process when it starts another is to kill all of them and don't leave those processes floating as they won't be notified of the shutdown and may lose information or block an unmount operation.
Cgroups may help (and systemd has a good approach at killing all processes in the cgroup).

So we're done with services. We parsed files, started them and also stopped them. Restarting services is simple. We stop the watcher (so we don't have to think on what will that goroutine do), call Stop function and then the Start function again.

The socket

We're not using dbus nor anything fancy. Just an unix socket.
Socket is exposed with mode 600 (owner is root OFC) on /run/service-launcher.sock (see socket/socket.go)

You can connect to the socket using openbsd-netcat and writing things or using the slc included tool (<30 lines of Go) to send commands like slc reboot.

Commands are very basic:

Power management: Kill PID 1 with needed signal to achieve this.
- reboot: SIGINT
- poweroff: SIGUSR1
- kexec: SIGUSR2
Service management
- stop, start, restart: With service name as argument, do that action over a service.
- list: list all services and their brief status
- status: get current status report for a given service (as argument).
Journal (we'll be over this on a sec)
- journal: with the service as argument, get journal for that service (ordered by date asc)

Those are the available commands at the moment. On a near future we need more control over journal and some other things (see the planned steps on the next article).

Journal

All service managers need a journal to store the stdout/err of the started services, try debugging without logs heh!.

Journal here is in-memory, so it's volatile and won't retain anything between reboots (it does between service restarts).
It's implementation is very basic and can be found at journal/journal.go.

What did we (at least I did) learn?

Making a service-launcher is fun and full of challenges. We may think at first it only launches processes, but the logic under it is huge and requires a lot of things.
To be honest, side logic didn't receive too much love, so you may notice some bugs and sure, limitations.

What's next?

Talk is cheap. Show me the code
~ Linus Torvalds

That's true, we left a lot of theory above. The repository is on mrvik/go-pid1 so you may want to take a look (and maybe try) this thing on a VM.

On the next article we'll be trying it and seeing how it works (could we start systemd as just another service?) we'll be over it.

I hope you liked this travel over what should we expect from a PID 1. Please, don't get me wrong, I'm not aiming to replace anything nor being just another competing standard but a proof of concept of something new, decoupled from PID 1.

See you next time!

Writing an init with Go (part 1)

MrViK — Wed, 30 Dec 2020 13:57:27 +0000

Welcome to the first article of the series about making an init process with Go.

So recently I was making some systems programming stuff with Go (a Podman service manager in case you want to check it out).

Then thought, let's write an init, how hard can it be?

Hotting up

So, what is the PID 1 supposed to be on a Linux system?

Wait zombie processes
Start other needed services (not really attached to PID 1)
Mount filesystems (also not really attached to PID 1)
Do the teardown, sync filesystems and good night.

For sure you have been thinking how to do those things on your favourite language, at first sight it isn't so hard.

Tracing the first lines

From points before:

PID 1 does point 1 and 4 (partly). Then it launches a thing called "service launcher".
Service Launcher (name is so original!). It
- mounts filesystems (calls mount -av) (BUG here, / is not remounted according to /etc/fstab settings)
- listens on a unix socket (control socket)
- launches all services, watches them (only the main process, see last links for further info).
- watches signals
- does teardown (stop services and unmount fs)

Heavy load was put under service-manager.

Hands on PID 1

Check out the code at cmd/init

Ok, this one is simple. If you had a glance at the code, you may notice it's heavily based on sinit, the suckless init

First, we start the service daemon. This call may fail, but who's afraid of an early panic on the morning?
Then we make sure ctrl+alt+del sends SIGINT to PID 1 instead of calling reboot routine in kernel.
Catch signals. Which signals?
- SIGUSR1 - poweroff
- SIGINT, SIGTERM - reboot
- SIGCHLD - Wait zombie processes.
After signal catcher exits (or service-launcher exits) it kills service-launcher, then waits for it, does syscall.Sync() and syscall.Reboot() with selected option (from signal watcher).

Only 3 explicit syscalls here (others are done from stdlib code).

syscall.Sync will block until disks are synchronized.
syscall.Reboot requests changes in reboot logic from kernel. E.g. it changes ctrl+alt+del behavior, asks for a reboot, poweroff, halt, and so on.
syscall.Wait4. Go allows to wait for processes from stdlib, so why do we do this from syscall package?. That call allows us to pass WNOHANG avoiding a blocking call, we only check if is anybody dead and ready to be removed from process table.

Nothing too complicated here, overall experience was pretty straight-forward and didn't hit any limitation in the stdlib nor the language.

Also, this can be compiled as an 1.6 MiB static binary (as it has no cgo calls).
Binary size can be reduced by removing the "log" package but I didn't see any problem with that size.

So, what about the service launcher? It looks promising

Yes, it was hard and has a lot of concepts, so we'll get around it on the next article (part 2). Stay tuned.

Third (and last) article we'll be running this on qemu and contain some final thoughts about the future of this thing.

See you next time!

Writing a new screen autolocker, with Go

MrViK — Mon, 05 Aug 2019 21:11:26 +0000

Initial thoughts

Well, I've been using a tiling window manager for some weeks, but still there's something I'm not comfortable with; the autolocker. I'm using xautolock(1), but it haven't implemented signal handling (USR1 to lock screen now, etc), so I'm thinking about creating my own.

Initial planning

So I'll write my own replacement for xautolock(1). What should the new software do:

Watch X11 idle time (should have an interface)
Listen USR{1,2} signals
Send desktop notifications
Parse arguments from command line
Low start times
Have as few dependencies as possible

I started looking on the xautolock(1) source code to see what did it do to check the idle time, so I found this line. I know it's not the official source, but this is enough. Now it's time to look on the man(1) pages for XScreenSaver(3). Ok, there's a library with the things I need, so I should use a language that can call C functions with a reasonable overhead.

Here comes the inevitable choose the right language. My possible choices were:

Javascript -> My first programming language. I thought it first, but as the last resort.
C -> The best language to call C functions from 😄. But I'm not so proficient on that lang. (Now I'm more confident writing things with it). Also, what about threads, I should learn how to use them, and it looks so complicated. Maybe another time.
Java -> I've written tons of (shitty) programs with it, but I don't and won't use them. Java was discarded from the beginning.
Rust -> A beautiful project to get started with Rust, but maybe it would not be a production-grade software.
Golang -> I've learning it but I haven't used it on a formal project. On the tests, some concepts caught my attention like simple concurrency with goroutines and channels. Also has a simple calling between C and Go

So golang is the chosen one. Now it's time to design it.

Let's take the above steps from initial planning.

X11 libraries docs are available under their man page, and functions are callable with cgo.
Signal catching is easy with os/signal and syscall.
Desktop notifications can be achieved with go-notify.
Parsing arguments from the command line is possible, but the standard library package is too basic. I ended up with github.com/akamensky/argparse.
Golang has a runtime statically linked to the final binary, but the startup times are very low and doesn't have a hard impact on the performance.
Well, the dependencies. LibX11 for XOpenDisplay(3) and xscrnsaver extension to get idle time info. Also depends on libnotify. Some distributions split the libraries and headers. Headers are needed then compiling.

Now, let's start. take ... hmm, how do I name this thing? Sorry for the low effort name, but it's called goautolock(1). Ok, take goautolock&&git init. (take is a function defined on oh-my-zsh that runs mkdir with all parameters and does cd on the last one).

Writing it

With the initial version I faced my first race conditions. Those weren't causing unexpected behavior, but go expects memory accesses to be synced, so I improved some features to convert some flags to channels. This change improved the performance (e.g. waiting 2 seconds at max to lock screen after SIGUSR1 vs locking immediately after getting a message on a channel).
That didn't fix all of the race conditions caused by the initial design, so a recent version implemented the mutexes provided by the go standard library.
It suffered many changes from the first time written. Take a look at the functions calling C libraries.
First time written:

Now looks like this:

Note the calls to C.free to fix memory leaks.
When I got into signal handling, I thought: "There are 2 user defined signals without any special meaning, one of them will call the locker immediately, but ¿what about the other one?" So, I decided to stole a concept from LineageOS (formerly Cyanogen MOD) where you could add caffeine to prevent looking the screen for a time. The implementation on goautolock(1) is simple:

Each SIGUSR2 adds X seconds to the lock delay. Take X like the value of --caffeine option (defaulting to 10).
After time is elapsed and screen is locked, this extra delay is set to 0.

Writing experience with golang

The first impression I had is that golang has a very limited set of options (coming from JavaScript that allows you to do whatever you want). And the error handling (split on panic and error returns) was chaotic at the first glance, but when used to it, the experience is the most consistent I had.
Also, it has some syntactic sugar like prepending defer to a expression to make it run after the actual function has been finished. This allows you to free C variables not managed by the GC just after the allocation, so you don't have to find the original declaration when you see a call to C.free.

Documentation

godoc is a very powerful utility for generating docs from source code, but this is a user utility, so the docs should be on the standard man(1). This was the most difficult part, using groff(7) to create a man page. There are a lot of macros to format the output. Now the result is available at the /docs folder on project (links on the bottom).

Packaging

This is an utility for Linux, each distro has a different packaging system (I won't use flatpak or ~~spam~~ snap for this).
The first thing I did was to build a release binary on the GitLab pipeline.
On the other hand, I want goautolock to be updated on my laptop when I do pakku -Syu, so I uploaded it to the AUR. It was so easy using ldd and namcap and testing with makepkg on a clean chroot.

The other distro I use is Fedora. This gave me some headaches.

Package names aren't the same nor similar to ArchLinux.
Devel packages are separated from the main ones.
Which utility should I use to create a RPM following a spec file? rpkg? a Makefile? fedpkg? tito?
The AUR counterpart (saving the distances) on Fedora is the COPR. Let's start a repo!
Wait, I want a RPM, what the hell is a SRPM?

After figuring out I should make another repo in Fedora Pagure to upload the spec and source tarball, I started using rpkg to build the srpm. And after a lot of failed builds it's finally uploaded to Fedora COPR. Then I started to pack some other projects, so everybody can easily use them.