DEV Community

Cover image for Stop Guessing systemd Service APIs: Practical varlinkctl on Linux
Lyra
Lyra

Posted on

Stop Guessing systemd Service APIs: Practical varlinkctl on Linux

#linux #systemd #devops #opensource

You need one clean answer from a local service: resolve a name, describe the host, extend a PCR, or talk to a custom tool that already speaks JSON on stdin/stdout. The usual options are brittle: scrape systemctl text, write a one-off D-Bus client, or invent yet another Unix-socket protocol.

Modern systemd components expose Varlink interfaces instead. varlinkctl is the operator-facing client for those interfaces: discover what a service implements, print the interface IDL, call methods with JSON arguments, stream multi-reply methods, tunnel calls over SSH, and even wrap a sandboxed command as a Varlink service.

This guide is operational. Commands and behavior come from varlinkctl(1), the UAPI.20 Varlink IPC specification, and the Debian/unstable systemd man pages for the services used in the labs (notably systemd-resolved).

What Varlink is (and is not)

Piece Job
Varlink JSON method calls over a stream (typically AF_UNIX), with a typed interface definition language
UAPI.20 The UAPI Group spec consolidating protocol + transport bindings
varlinkctl CLI to introspect and invoke Varlink services (systemd 255+)
busctl / D-Bus Parallel IPC stack; still widely used, different wire format and tooling
systemctl text Human UI — fine for shells, awkward as a stable machine API

Varlink messages are plain JSON objects. On stream transports each message is NUL-terminated. Services describe themselves at runtime, so clients can list interfaces and fetch the IDL without a separate schema package.

You do not need to replace every D-Bus workflow. Use varlinkctl when the component already speaks Varlink (resolved, hostnamed, many systemd helpers) or when you want a JSON-friendly, socket-activated service of your own.

Prerequisites 

command -v varlinkctl
varlinkctl --version
# Added in systemd 255; several commands below need 257–262 features.

# Labs that call resolved need the stub/service running:
systemctl is-active systemd-resolved.service
ls -l /run/systemd/resolve/io.systemd.Resolve
Enter fullscreen mode Exit fullscreen mode

Privileged examples (PCR extend, some hostnamed paths) need root or an authorized policy. DNS resolve calls as a normal user usually work when resolved is active and the socket is accessible.

Address forms you will actually use

varlinkctl accepts several service address syntaxes (from varlinkctl(1)):

Form Meaning
unix:/run/path.sock or /run/path.sock Connect to an AF_UNIX stream socket
@name after unix: Abstract-namespace socket
exec:/usr/lib/systemd/tool Fork the binary and speak Varlink on the passed socket
ssh-unix:host:/run/... OpenSSH ≥ 9.4 path to a remote AF_UNIX socket
ssh-exec:host:cmdline Run a remote command and speak Varlink on its stdio
Relative ./socket or ./binary Local path forms (must start with / or ./)

Convenience: a bare absolute socket path or executable path is enough when the target is local.

Lab 1 — Inventory a live service (resolved) 

# General metadata
varlinkctl info /run/systemd/resolve/io.systemd.Resolve

# Interface names only
varlinkctl list-interfaces /run/systemd/resolve/io.systemd.Resolve

# Method names (list-methods added in 257)
varlinkctl list-methods /run/systemd/resolve/io.systemd.Resolve

# Full IDL for the primary interface
varlinkctl introspect /run/systemd/resolve/io.systemd.Resolve io.systemd.Resolve
Enter fullscreen mode Exit fullscreen mode

Typical info fields include vendor, product, version, URL, and the interface list (io.systemd, io.systemd.Resolve, org.varlink.service, …).

Pretty JSON for scripts:

varlinkctl info /run/systemd/resolve/io.systemd.Resolve -j
varlinkctl list-methods /run/systemd/resolve/io.systemd.Resolve --json=short
Enter fullscreen mode Exit fullscreen mode

-j is “pretty when interactive, short when piped”; --json=pretty|short forces the mode.

Lab 2 — Call a method with JSON arguments

Resolve a hostname through resolved’s ResolveHostname method (example shape from varlinkctl(1)):

varlinkctl call \
  /run/systemd/resolve/io.systemd.Resolve \
  io.systemd.Resolve.ResolveHostname \
  '{"name":"systemd.io","family":2}' \
  -j
Enter fullscreen mode Exit fullscreen mode

Notes that prevent foot-guns:

  • Method names are fully qualified: interface.Method.
  • Parameters are a JSON object. Use {} for empty input.
  • If you omit the arguments parameter, varlinkctl reads JSON from STDIN.
  • Replies are JSON objects on STDOUT — pipe to jq when you want fields only. 
varlinkctl call \
  /run/systemd/resolve/io.systemd.Resolve \
  io.systemd.Resolve.ResolveHostname \
  '{"name":"systemd.io","family":2}' \
  --json=short \
| jq -r '.addresses[]? | "\(.family) \(.address)"'
Enter fullscreen mode Exit fullscreen mode

family: 2 is AF_INET in the usual Linux numbering; adjust if you want IPv6 (10 / AF_INET6) or leave the field out when the interface allows defaults (check the IDL from introspect).

Lab 3 — Multi-reply methods, collection, and oneway

Some methods stream updates or enumerate objects. Flags from varlinkctl(1):

# Expect a sequence of replies (JSON-SEQ). Default call timeout is still 45s.
varlinkctl call --more ADDRESS INTERFACE.Method '{"...":"..."}'

# Same idea, but keep listening (timeout disabled) — shortcut -E
varlinkctl call -E ADDRESS INTERFACE.Method '{}'

# Gather every reply into one JSON array
varlinkctl call --collect ADDRESS INTERFACE.Method '{}'

# Fire-and-forget (no reply expected)
varlinkctl call --oneway ADDRESS INTERFACE.Method '{"...":"..."}'
Enter fullscreen mode Exit fullscreen mode

Timeout control:

# Default 45s; disable for long subscriptions
varlinkctl call --more --timeout=infinity ADDRESS INTERFACE.Method '{}'

# Treat a specific Varlink error as success (257+)
varlinkctl call \
  --graceful=org.varlink.service.InvalidParameter \
  ADDRESS INTERFACE.Method '{"experimental":true}'
Enter fullscreen mode Exit fullscreen mode

Use --more with a sane timeout for finite enumerations; use -E / --timeout=infinity only for true subscriptions so a stuck peer cannot hang a cron job forever.

Lab 4 — Exec targets and helper binaries

Not every Varlink peer is a long-running daemon socket. Some tools speak Varlink when executed. The man page demonstrates systemd-pcrextend:

# Inspect the executable as a Varlink service (requires privileges for real PCR ops)
sudo varlinkctl info /usr/lib/systemd/systemd-pcrextend
sudo varlinkctl introspect /usr/lib/systemd/systemd-pcrextend io.systemd.PCRExtend

# Example method shape from the man page — only on systems where PCR extend is appropriate
# sudo varlinkctl call /usr/lib/systemd/systemd-pcrextend \
#   io.systemd.PCRExtend.Extend '{"pcr":15,"text":"foobar"}'
Enter fullscreen mode Exit fullscreen mode

exec: form is equivalent when you want to be explicit:

sudo varlinkctl info exec:/usr/lib/systemd/systemd-pcrextend
Enter fullscreen mode Exit fullscreen mode

Treat PCR/measurement labs as maintenance-window work on hosts where measured boot policy is intentional. Do not extend PCRs on production machines as a casual test.

Lab 5 — Remote calls over SSH

When the remote side has OpenSSH 9.4+ (for ssh-unix:) and the socket path exists:

# Talk to hostnamed on a remote machine via its AF_UNIX socket
# varlinkctl call ssh-unix:somehost:/run/systemd/io.systemd.Hostname \
#   io.systemd.Hostname.Describe '{}' -j

# Or run a Varlink-capable binary on the remote stdio path
# varlinkctl call ssh-exec:somehost:systemd-creds \
#   org.varlink.service.GetInfo '{}' -j
Enter fullscreen mode Exit fullscreen mode

This is useful for fleet introspection without installing a custom agent: SSH provides transport and auth; Varlink provides a typed method call. Abstract-namespace sockets are not supported over ssh-unix: (filesystem path sockets only).

Lab 6 — Registry and socket discovery (260+)

Newer systemd builds keep well-known entrypoints under /run/varlink/registry/:

# System registry (default)
varlinkctl list-registry

# Per-user registry when applicable
varlinkctl list-registry --user

ls -l /run/varlink/registry/ 2>/dev/null
Enter fullscreen mode Exit fullscreen mode

list-sockets (262+) enumerates listening AF_UNIX stream sockets marked as Varlink entrypoints via the user.varlink=entrypoint xattr (needs kernel support for xattrs on socket inodes; documented as Linux 7.0+ in the man page). Prefer list-registry on typical current distro kernels if list-sockets is unavailable.

Lab 7 — Serve a sandboxed stdio tool as Varlink (261+)

varlinkctl serve turns a command that speaks a protocol on stdio into a socket-activated Varlink service. On upgrade, the client’s connection is handed to the command. The man page’s decompressor example:

/etc/systemd/system/varlink-decompress-xz.socket

[Socket]
ListenStream=/run/varlink/registry/com.example.Decompress.XZ

[Install]
WantedBy=sockets.target
Enter fullscreen mode Exit fullscreen mode

/etc/systemd/system/varlink-decompress-xz.service

[Service]
ExecStart=varlinkctl serve com.example.Decompress.XZ xz -d
DynamicUser=yes
PrivateNetwork=yes
ProtectSystem=strict
ProtectHome=yes
NoNewPrivileges=yes
SystemCallFilter=~@privileged @resources
MemoryMax=256M
Enter fullscreen mode Exit fullscreen mode

Enable and call:

sudo systemctl daemon-reload
sudo systemctl enable --now varlink-decompress-xz.socket

echo "hello" | xz | varlinkctl call --upgrade \
  unix:/run/varlink/registry/com.example.Decompress.XZ \
  com.example.Decompress.XZ '{}'
# expected stdout: hello
Enter fullscreen mode Exit fullscreen mode

Quick test without unit files (ephemeral listen socket):

systemd-socket-activate -l /tmp/decompress.sock -- \
  varlinkctl serve com.example.Decompress.XZ xz -d &

echo "hello" | xz | varlinkctl call --upgrade \
  unix:/tmp/decompress.sock \
  com.example.Decompress.XZ '{}'
Enter fullscreen mode Exit fullscreen mode

Why this pattern matters:

  • The heavy lifting stays in a forked child with systemd sandboxing (ProtectSystem=, MemoryMax=, …).
  • Clients discover a stable method name instead of ad-hoc FIFO paths.
  • --upgrade moves from JSON control plane to raw stream payload once the call succeeds.
  • Pair with --exec on the client when the reply includes file descriptors ($LISTEN_FDS hand-off; 258+).

Lab 8 — Validate IDL before you ship an interface 

cat > /tmp/com.example.Echo.varlink <<'EOF'
# Minimal echo interface for validation practice
interface com.example.Echo

method Ping(message: string) -> (message: string)

error EmptyMessage ()
EOF

varlinkctl validate-idl /tmp/com.example.Echo.varlink
# prints the definition with syntax highlighting on success
Enter fullscreen mode Exit fullscreen mode

Catch naming and structure mistakes early; the wire protocol expects the same IDL shape services return from org.varlink.service introspection.

Operator cheat sheet

Goal Command pattern
Who am I talking to? varlinkctl info ADDRESS
What can it do? list-interfaces / list-methods / introspect
One shot call call ADDRESS Fully.Qualified.Method '{"k":"v"}' -j
Stream / subscribe call -E … or call --more --timeout=…
Bundle replies call --collect …
No reply call --oneway …
Remote socket ssh-unix:host:/path
Remote binary ssh-exec:host:command
Wrap stdio tool varlinkctl serve METHOD cmdline… + socket unit
Check IDL file validate-idl FILE
Registry list-registry (260+)

Common failure modes

Symptom Likely cause Fix
No such file or directory on socket Service down or path wrong systemctl status …; confirm path under /run/systemd or /run/varlink/registry
Connection refused / permission denied Socket mode or user Check unit SocketUser=/SocketGroup=; use root only when required
Method not found Typo or old package list-methods / introspect; upgrade systemd
Hang then timeout Multi-reply method without --more, or stuck peer Match flags to IDL; set --timeout= deliberately
SSH unix path fails OpenSSH < 9.4 or abstract socket Upgrade SSH; use filesystem-bound sockets only
serve never accepts Missing socket activation Use .socket unit or systemd-socket-activate
JSON parse errors Shell quoting Single-quote the JSON object; or pass args via STDIN
Pretty JSON breaks scripts -j on a TTY Use --json=short in pipelines 
journalctl -u systemd-resolved.service -u 'varlink-*.service' -b --no-pager
Enter fullscreen mode Exit fullscreen mode

How this fits nearby tooling

  • busctl / D-Bus — still the right tool for classic desktop and many system APIs. Varlink is an additional, JSON-native path used heavily by newer systemd components.
  • resolvectl / hostnamectl — polished UX on top of the same daemons; varlinkctl is the generic wrench when you need raw methods or automation without scraping.
  • systemd-ssh-proxy / generators — solve VM/container SSH transport; orthogonal to Varlink method calls (though ssh-unix: reuses SSH as a Varlink tunnel).
  • Custom REST agents — often unnecessary on localhost when a sandboxed varlinkctl serve unit already gives you discovery, socket activation, and cgroup limits.

References

If you have been parsing semi-stable command output or writing miniature D-Bus clients for one method call, point varlinkctl at the socket, read the IDL, and call the method with JSON. Same skill scales from a quick resolved query to a socket-activated, sandboxed helper you can ship as a pair of unit files.

Top comments (0)