Linux from the user's perspective - Part 3: Graphical Interface

This blog is part of a series. More here

Before reading this blog, I recommend watching this talk. I am very much influenced by it, and I don't want to rip it off.

Graphics are pretty cool, right? Along with having multiple applications running, that you can interact with at the same time. But all applications need to be able to interact with the graphics system in a consistent way - which requires a common standard. Let's see what the industry came up with.

The X11 protocol, and it's open-source implementation X.Org have a rich history. Originally designed for a network-transparent mainframe-centric model, along with the vast zoo of incompatible displays, keyboards, mice, and the like, they had to be repurposed for the modern age. Over the course of that journey, it went through three stages of evolution, and how has a successor - Wayland.

I won't touch on history of the protocol's development - the talk linked above does an excellent job already. Let's discuss the technical side. In the original incarnation, the vision was as follows.

You have a powerful mainframe, and a graphics-capable terminal. First you run the X server on the terminal - it takes control of the screen, the keyboard and the mouse. It loads from the disk fonts, bitmaps, themes and styles, configs. It does so by the virtue of the startx script that runs the commands from ~/.xinitrc, which typically had the following contents:

xrdb -merge ~/.Xresources &    # load x resources
xsetroot -solid grey &         # background color
xclock -geometry 50x50+40+40 & # simple app
xterm -geometry 80x50+20+150 & # terminal
exec twm                       # window manager; 'exec' makes it keep X running until WM exits

But this is only a part of the configuration. Configuration of devices goes into xorg.conf or xorg.conf.d/ file/directory, located at /etc/X11/xorg.conf.d/, /usr/share/X11/xorg.conf.d/, ~/.xorg.conf.d/ - they are merged together.

Then the X server is started - and it would query for available XDMCP remotes. XDMCP daemon would run on a remote host, a mainframe, and connect to the X server via TCP. The X server would present a login screen. After login, a remote desktop environment would be started, and you could start apps - all of them would run remotely, on the mainframe.

When a graphical app would start, it would:

• set up a TCP socket to the X server, via Xlib
• use themes, fonts, bitmaps, that were loaded by the X server
• get a window, which is a framebuffer to be rendered-to in immediate mode
• send drawing commands - put pixel, draw line, draw square, draw text

As for arranging windows on a screen, a special privileged client connected to the X server - the window manager. Just as regular apps would subscribe to events, including keyboard events, the window manager would subscribe to window events - window created, destroyed, moved, resized. It would then tell the X server, where to place windows. This allowed for clean separation of concerns, and allowed the X server to stay flexible, and allow for different window management strategies, by adhering to the "mechanism, not policy" philosophy.

Let's configure a server, for exactly that experience. We will use the VM as the XDMCP server, so the X server should already be running on your host. Make sure it allows for TCP connection, by editing /etc/X11/xinit/xserverrc. Also authorize the remote server with the command

xhost +<IP>

Configuring the network and the unprivileged user

To prepare the host, let's configure the network with NetworkManager, and disable dhcpcd.

[root@archlinux ~]# pacman -S networkmanager
[root@archlinux ~]# systemctl enable networkmanager
[root@archlinux ~]# nmtui # configure the interfaces here
[root@archlinux ~]# systemctl disable dhcpcd@enp1s0

Let's now create an unprivileged user.

[root@archlinux ~]# useradd -m -G adm,wheel,tty,sys -s /usr/bin/bash user

Allow root without a password

[root@archlinux ~]# pacman -S sudo
[root@archlinux ~]# nano /etc/sudoers # input the text below
root ALL=(ALL) ALL
Defaults targetpw
ALL ALL=(ALL) NOPASSWD: ALL

Set up password, switch to the user

[root@archlinux ~]# passwd user
[root@archlinux ~]# sudo su user

Now install the xfce4 desktop environment, xorg, and gdm - it will be our XDMCP daemon

[user@archlinux ~]$ sudo pacman -S xfce4 xorg gdm
[user@archlinux ~]$ sudo systemctl enable gdm
[user@archlinux ~]$ sudo systemctl start gdm
[user@archlinux ~]$ nano ~/.xsession # set contents to
startxfce4

Edit /etc/gdm/custom.conf, set contents to:

[daemon]
WaylandEnable=false

[security]
DisallowTCP=false

[xdmcp]
Enable=true

[chooser]

[debug]
Enable=true

And now, on your local host, run from TTY:

[user@archlinux ~]$ sudo X :2 -query <ip>

Or, from an emulated terminal:

[user@archlinux ~]$ Xephyr :2 -query 192.168.100.2

(this is actually running Gnome, idk, whatever)

And while this currently works (even if over plain TCP - use SSH or TLS tunneling), the modern approach is remote desktop - RDP or VNC. The difference of what's being transmitted is vector graphics vs raster graphics. And X11 does vector graphics poorly, at least from what I've surmised. I'd love to see a modern immediate-mode vector graphics rendering technology that works remotely, but alas, we are stuck with raster graphics for now. Speaking of remote desktop, let's set it up!

First, install yay
Then run

[user@archlinux ~]$ yay -S xrdp
[user@archlinux ~]$ nano ~/.xsession # set to text below
xfce4-session
[user@archlinux ~]$ nano /etc/X11/Wrapper.config # set to text below
allowed_users=anybody
needs_root_rights=no
[user@archlinux ~]$ sudo systemctl start xrdp.service

Now connect to it, for example, with Remmina

This is with xrdp - modern desktop enviroments, such as KDE and GNOME provide their own built-in RDP servers, which you can use.

Since we are discussing RDP, it would be useful to know, how it actually works under the hood. On X, any client can read the contents of the entire window, with XGetImage - and send them over. With a modern alternative, like Wayland, there is a special protocol - you can only read the contents of the windows that the user authorizes you to read.

The next step of evolution came with supporting hardware acceleration, through shared memory and graphics APIs - like OpenGL, with the xgl extension. Of course - none of this is possible over a network. So, local X was the new paradigm.

We can also forward any single application. Simply run

[user@archlinux ~]$ export DISPLAY=<hostIP>:0
[user@archlinux ~]$ xterm

The correct display number can be seen in env on the host
Result:

The final step of evolution came with compositors, which broke an important architectural decision in the X server. Namely - because every application drew to its window in immediate mode, you couldn't have some windows cast shadows on other windows, or have them be transparent. That would require baking such interactions into the protocol, and making every application support them - which would be impossible. So, a hack was found - applications would draw not to the screen, but to an off-screen buffer. After a buffer was drawn, the compositor, which is the more powerful window manager, would be notified - it would then create it's own buffer, into which the buffers of all active applications would be overlayed in correct order, and with effects. Then only this buffer would be given to X for rendering.

The trajectory of the X11's development is as follows - it performed less and less functions. More and more drawing was done either by the app, or the compositor X was just passing buffers back and forth. So, a more modern protocol was developed - Wayland. The goal of it is to be minimal, and allow apps and the compositor to do their job, and stay out of their way otherwise. It does raster graphics, and only on the current machine. Remote solutions exist, such as waypipe - but it sends over raster graphics for a window, not vector graphics.

Let's do some reflection. Which is better - raster graphics or vector graphics? I say vector graphics, but you have to do them well, and they are more complicated. Next would be running applications remotely - how do you do that well? Ideally, you would run applications on a cluster - and not care about which individual server that is. But that requires that all resources an application may use are also available on the cluster - which is not a trivial problem. For now, seems that local apps + RDP is what the industry settled for.

That's it, when it comes to the Graphical Interface on Linux - next time, we will discuss system administration.