Writing D-Bus Services in Python and Twisted

I mentioned in a previous post that while writing korbenware, my Linux desktop management software, that I ended up writing a D-Bus service framework. Today I wanted to write about what exactly that means and how you can interact with it using Python, Twisted and txdbus.

What is D-Bus?

D-Bus is a system that allows for locally-running desktop-oriented processes to communicate with each other and to make remote procedure calls between them. It's part of the freedesktop stack, systemd integrates with it, and pretty much every Linux desktop has it installed. D-Bus enables one program to trigger actions or query for data from another program.

One use case for this is system notifications. For instance, if you go to your Linux machine (which has notify-send installed) and run:

$ notify-send "hello world!"

your machine should show a desktop notification:

The way that this actually works is this: my notifications server program, which happens to be dunst, connects to the D-Bus server running on my machine and exposes a notifications interface that conforms to the standard that Linux desktops use for notifications. Then, when I want to make a notification, notify-send goes to D-Bus looking for this expected notifications service and calls the remote method, supplying my humble message.

This is pretty cool. Because D-Bus exists, any desktop services that have a use case for actions triggered by outside programs can use this common system to expose functionality to them. Because the notifications interface is standardized, I can plug different implementations of the same idea into that spot. In my case, I'm using dunst, but GNOME and KDE all have their own implementations of notifications that are deeply integrated with their desktops.

The D-Bus stack has a bunch of slick features besides basic RPC. It can expose properties, advertise reflectable interfaces and, with systemd integration, even allow for services to be woken up lazily on first use.

Making RPC Calls over D-Bus

Any programming environment can connect and interact with D-Bus. My project is written in Python and Twisted, so I use a library called txdbus.

To send a notification with txdbus, first we set up a Twisted reactor and create a connection to D-Bus:

from twisted.internet import reactor

from txdbus import client

conn = await client.connect(reactor)

Then, we make a call to D-Bus to fetch the RPC interface:

remote_obj = await conn.getRemoteObject(
    'org.freedesktop.Notifications',
    '/org/freedesktop/Notifications'
)

Services on D-Bus expose remote objects, which are on a bus at an object path. Each service has its own bus name, and each of multiple objects exposed by a service has its own object path. In this case, the notifications server uses the 'org.freedesktop.Notifications' bus and exposes a Notifications object on '/org/freedesktop/Notifications'.

D-Bus will send back a payload that describes the interface for that object, meaning that txdbus can construct a working RPC object by reflecting off it without any configuration on the client's end.

Once we have the object, actually creating the notification looks like this:

await remote_obj.callRemote(
    'Notify',  # The name of the method
    'my-test-app',  # The name of my app
    0,  # An optional ID of a previous notification to replace
    '', # An optional path to an icon
    'This is a test message.', # A summary
    'Hello World!', # A message
    [],  # Actions that the user can take
    dict(),  # Hints - extra parameters
    5000  # How long the notification sticks around
)  # Returns the ID of the notification

The Notifications object exposes a method called 'Notify' that takes a big list of arguments. D-Bus arguments don't have a true concept of a null value, so any given optional argument takes a "falsey" argument. For instance, a notification can display an icon, but if you don't have an icon to share you send an empty string instead of a null value.

This API is a little clumsy, but for clients like this we can wrap our RPC objects in a simple facade. Here's the one I wrote for korbenware:

import attr


@attr.s
class Notifier:
    connection = attr.ib()
    remote_obj = attr.ib()

    async def create(connection):
        remote_obj = await connection.getRemoteObject(
            'org.freedesktop.Notifications',
            '/org/freedesktop/Notifications'
        )

        return Notifier(connection, remote_obj)

    async def notify(
        self,
        appname='',
        replaces=0,
        icon='',
        summary='',
        message='',
        actions=None,
        hints=None,
        timeout=1000
    ):
        actions = actions or []
        hints = hints or dict()

        return await self.remote_obj.callRemote(
            'Notify',
            appname,
            replaces,
            icon,
            summary,
            message,
            actions,
            hints,
            timeout
        )

Given this facade, making notifications from korbenware looks like this:

from korbenware.notifications import Notifier

notifier = await Notifier.create(conn)

await notifier.notify(
    appname='my-test-app',
    message='Hello World!',
    summary='This is a test message.'
)

That's not bad at all - I'm pretty happy with it.

Exposing Services over D-Bus

Calling client objects is cool and all, but I want to be able to control korbenware's session manager through D-Bus. In other words, I want to expose a service myself - so what does that look like?

To accomplish this, I'll need a few more imports:

from twisted.internet import defer
from txdbus import objects
from txdbus.interface import DBusInterface, Method

Now I can define a D-Bus object class which has a D-Bus interface and associated methods:

class MyObj(objects.DBusObject):
    iface = DBusInterface(
        'org.example.MyIFace',
        Method('Ping', arguments='s', returns='s')
    )

    dbusInterfaces = [iface]

    def __init__(self, objectPath):
        super().__init__(objectPath)

    def dbus_Ping(self, arg):
        print(f'Received: {arg}')
        return f'You sent: {arg}'

conn.exportObject(MyObj('/MyObjPath'))

await conn.requestBusName('org.example')

In this snippet, I define a class that inherits from txdbus.objects.DBusObject, which has a property on it called dbusInterfaces, a list containing an instance of txdbus.interfaces.DBusInterface. This interface defines its DBus path and any methods it has - in this case a method called "Ping" that takes and returns a string. The object also defines the method's logic with a Python method called dbus_Ping. Finally, I export the object to D-Bus with an object instantiated under the "/MyObjPath" object path and request the 'org.example' bus name.

Now my service is online, and I can call it like I did the notification object:

myObj = await conn.getRemoteObject(
    'org.example',
    '/MyObjPath'
)

await myObj.callRemote('Ping', 'Hello world!')

This will print "Received: Hello world!" on the server side of the connection, and return "You sent: Hello world!" on the client. I'm running these snippets in a Jupyter notebook, so my client connection is also my server.

D-Bus Types and Signatures

You probably have questions about how the arguments and return value are specified with "s" and "s", respectively. D-Bus has a mini-language for describing payloads sent and received using its own binary serialization format. This format has a number of data types, and each data type has its own syntax for describing what it looks like. Any collection of these is called a "signature". Here are some examples:

• 's' - a string
• 'b' - a boolean
• 'i' - a 32 bit integer
• 'u' - an unsigned 32 bit integer
• 'v' - a variant type - D-Bus will send the type of its value alongside the value inside the payload
• '(si)' - a struct containing a string and an integer
• 'ai' - an array of integers
• '{sv}' - a dict with string keys and variant values

The full documentation can be found online.

These can be put together to form a full payload. Here's what the arguments signature for the notify call looks like:

susssasa{sv}i

These correspond to the arguments we saw earlier: a string (the app name), an unsigned integer (the notification ID), three more strings (the icon, summary and message respectively), an array of strings (our actions), a dict with string keys and variant values (extra hints), and an integer (the notification timeout).

What Else Can It Do?

This is scratching the surface of the capabilities of D-Bus - it also supports remote properties, pubsub via a mechanism called signals, error propagation and more. txdbus has a full tutorial online that goes over some of these advanced features.

What's Next?

This post covers the basics of D-Bus, what it does and how to interact with it. However, while a facade worked well for reflected clients, I found writing wrappers for services a little clumsy. Instead of settling, I decided to shave a yak and write a custom framework around it using attrs and marshmallow. I think the work I did here is very cool and I'm excited to share it with people - but in a future blog post. Stay tuned!