Raghu Bharadwaj

Posted on Aug 26 Originally published at techveda.live

systemd vs BusyBox init: Which Init System Fits Your Device?

#linux #embedded #architecture #yocto

For most embedded products with a fixed set of services and no user-installable software, BusyBox init is the right default: small, no dependency chain, and its behaviour fits in one readable file. Choose systemd when the device genuinely needs supervised, interdependent services — readiness ordering, watchdog-backed liveness detection, resource limits — and accept in return a much larger image, a kernel floor of 5.10, a dependency set you maintain for the life of the product, and (if you build with Buildroot) a glibc toolchain for the whole system.

The choice of systemd vs BusyBox init is one of the first structural decisions in an embedded Linux product, and one of the hardest to reverse. PID 1 defines how services declare dependencies, how failures are detected, and which libraries stay in the image forever. Many teams inherit the choice from the vendor BSP without recording why, then work around it for two years. The problem is the unrecorded decision, not the inherited default.

When this decision arises

The decision arises once you move past a single application binary. A device that boots, runs one process and reboots on failure needs nothing beyond a supervising parent. The forces appear when the product gains a second and third long-running service with an ordering relationship, and when support asks why a unit stopped responding without rebooting. Four forces decide it: the image budget from the bill of materials, doubled in an A/B layout; the service topology, meaning whether any service can fail without exiting; the platform floor, meaning the C library and kernel version you are committed to; and the maintenance horizon, meaning how long you must ship security updates.

One clarification first, because it causes real design errors. Init and device management are two decisions, not one. Yocto's INIT_MANAGER values bundle them, but the device manager has its own variable, VIRTUAL-RUNTIME_dev_manager, with documented values udev, busybox-mdev and systemd. Buildroot keeps them in separate menus, and its /dev management choice is hidden only when BR2_INIT_SYSTEMD is selected. Either build system will run eudev under BusyBox init, so a USB modem or a camera that needs real udev rules does not oblige you to take systemd.

# Yocto: distro config or local.conf
INIT_MANAGER = "sysvinit"       # Poky default, SysVinit + udev
INIT_MANAGER = "mdev-busybox"   # BusyBox init + BusyBox mdev
INIT_MANAGER = "systemd"        # systemd + udev

Buildroot's System configuration → Init system menu is longer than most write-ups suggest: four general-purpose options — BR2_INIT_BUSYBOX (the default), BR2_INIT_SYSV, BR2_INIT_OPENRC and BR2_INIT_SYSTEMD — plus four special-purpose ones that are mostly container reapers.

The options: systemd vs BusyBox init

BusyBox init

BusyBox init lives inside the BusyBox multi-call binary you almost certainly already ship. It reads /etc/inittab at startup; entries take the form <id>:<runlevels>:<action>:<process>, with the action one of sysinit, wait, once, respawn, askfirst, restart, ctrlaltdel or shutdown. The runlevel field is unused. Buildroot's default inittab mounts a few filesystems, runs /etc/init.d/rcS and starts a getty.

Myth: "respawn gives me service supervision."

An entry marked respawn is restarted when it exits, and that is the whole of the guarantee. There is no back-off and no start limit, so a service that crashes at startup restarts about once a second, paced only by the one-second sleep in init's main loop, until someone power-cycles the board. BusyBox's own documentation in init/init.c states: "Unlike sysvinit, BusyBox init does not stop processes from respawning out of control." A process that is alive but no longer working never exits, so it is never restarted at all.

BusyBox also ships pieces people assume it lacks. The watchdog applet takes -T N (reboot after N seconds if not reset, default 60), -t N (reset every N seconds) and -F (foreground), so you can run it from inittab as a respawn entry. The syslogd applet accepts -C[size_kb] to log to a shared-memory circular buffer that logread reads — on a flash-based device, often exactly the logging policy you want.

In its favour: negligible size on top of BusyBox; no new libraries; a boot sequence one engineer can read in full before changing it; no C library or kernel constraint beyond BusyBox's own; and failure behaviour that is exactly what you wrote.

Against it: ordering is whatever order you wrote the script in, so a race between two services can remain undetected until a unit fails at a customer site; a hung service stays hung; there is no restart back-off; and supervision beyond respawn is yours to write, test and document.

systemd

systemd replaces init with a service manager. Services are declarative unit files rather than shell scripts, they start in parallel, and the manager tracks them using control groups. Three mechanisms carry most of the value on a device, and each of the three is commonly misread.

Myth: "systemd makes service ordering reliable."

Ordering is not readiness. After= orders unit start-up; it does not wait for the dependency to be usable. Type=notify does: the service sends READY=1 through sd_notify, and systemd "will proceed with starting follow-up units after this notification message has been sent." That is what removes sleep loops from your boot path, and it is also the real migration cost, because it means changing the application. Teams that convert rcS into units without adopting Type=notify keep their race and conclude systemd did not help.

Myth: "systemd automatically detects hung services."

Liveness detection is two independent settings, both off by default. A unit's WatchdogSec= makes the service ping PID 1: it calls sd_notify with WATCHDOG=1, and if the gap between two calls exceeds the configured time, systemd marks the service failed and terminates it with SIGABRT (or WatchdogSignal=), restarting it under a suitable Restart= value. It "defaults to 0, which disables this feature." Separately, RuntimeWatchdogSec= in systemd-system.conf makes PID 1 ping the hardware watchdog — WatchdogDevice=, default /dev/watchdog0 — so the board resets if PID 1 itself stops. It also defaults to 0. Escalating a hung service into a board reset therefore needs three things: the unit setting, the manager setting, and an application that actually calls sd_notify.

The start limit will stop restarting your service. DefaultStartLimitIntervalSec= is 10 seconds and DefaultStartLimitBurst= is 5, so the sixth start inside that window is refused. Units "which are configured for Restart=, and which reach the start limit are not attempted to be restarted anymore."

Myth: "Restart=always means the service always comes back."

The consequence is invisible on a desktop, where an administrator sees the failed state and runs systemctl reset-failed. On a headless device in the field nobody sees it, and the product stops working while PID 1 continues to report a normal system state. StartLimitAction= exists for this and defaults to none; on a product, setting it to reboot and raising RestartSec= above its 100 ms default is usually correct.

[Unit]
Description=Sensor gateway
After=sensor-bus.service

[Service]
Type=notify              # follow-up units wait for READY=1
NotifyAccess=main
WatchdogSec=30s          # also needs RuntimeWatchdogSec= in system.conf
Restart=on-failure
RestartSec=5s
StartLimitBurst=5
StartLimitAction=reboot  # do not give up silently on a headless device

In its favour: readiness ordering rather than implied ordering; detection of a service that has stopped responding, in addition to one that has exited; per-service resource limits and sandboxing, auditable with systemd-analyze security (it reports an exposure level from 0.0 to 10.0, where a high value means little sandboxing); integrated logging; boot analysis through systemd-analyze critical-chain; and one operational interface new engineers already know.

Against it: size and dependencies. The Yocto documentation states that using systemd increases the core-image-minimal image size by 160% for qemux86-64 on Mickledore (4.2), compared to SysVinit. Note what that compares: systemd against SysVinit, not against BusyBox init. A BusyBox-init image is smaller than a SysVinit one, so treat 160% as a lower bound on the gap this decision creates.

Myth: "systemd is too big for embedded."

That lower bound is a percentage of a reference image with almost nothing in it, which is why it travels badly. On a product carrying a graphics stack or a Python runtime, the same absolute delta is a small fraction of the total. The size objection is decisive on a 64 MB flash part and close to irrelevant on a 4 GB one, which is why the number has to come from your own image rather than from a published comparison.

The dependency set is the more durable cost. Upstream requires glibc 2.34 or newer (musl 1.2.6 only when built with -Dlibc=musl), sets a minimum kernel baseline of 5.10 and recommends 5.14, below which it sets an old-kernel taint flag. Buildroot's package/systemd/Config.in is stricter: among other gates it depends on BR2_TOOLCHAIN_USES_GLIBC, an MMU, kernel headers of at least 5.4 and GCC 8, and selects merged /usr, D-Bus (or dbus-broker), libcap, util-linux, kmod and timezone data.

The options this decision does not take

A decision record is judged partly on the alternatives it rejects. SysVinit sits between the two: shell-script services, runlevels, and udev rather than mdev. It suits an inherited body of /etc/init.d/ scripts and is a poor choice starting fresh, costing more than BusyBox init without the supervision that justifies systemd. OpenRC is the genuine middle option; Buildroot ships it as BR2_INIT_OPENRC, depending only on an MMU, dynamic libraries and a non-uClibc toolchain, so it is available on musl where systemd is not. s6, runit and finit offer stronger supervision than respawn on similar terms. All are rejected here for one reason: they add a framework your team must learn and integrate, without the ecosystem, vendor BSP alignment or hiring pool that make systemd worth its cost. If your team already runs one, that reasoning does not apply.

A trimmed systemd deserves naming too, because a reviewer will ask what a minimal build costs. Read the option list for your own release rather than a figure from a blog post: in Yocto, bitbake -e systemd and look at PACKAGECONFIG; in Buildroot, the BR2_PACKAGE_SYSTEMD_* sub-options. Then apply the sentence that matters — trimming moves the size number, not the kernel floor, the C library constraint, or the count of upstream projects you track.

Measure four things before you decide

An argument about systemd vs BusyBox init that contains no numbers from your own product will not survive a design review. Four measurements take an afternoon.

Size delta on your image, not a reference image. In Buildroot, run make clean all then make graph-size with each init selected; it writes output/graphs/graph-size.pdf, package-size-stats.csv and file-size-stats.csv, and size-stats-compare diffs two of those CSVs. In Yocto, set INHERIT += "buildhistory" and BUILDHISTORY_COMMIT = "1", build with each INIT_MANAGER value, then read installed-package-sizes.txt or run buildhistory-diff. Report it twice: absolute bytes on the image you ship, and a percentage of one A/B slot. On a 400 MB image with a graphics stack, a few megabytes is not an argument.

Resident memory, usually the binding constraint below 256 MB. Flash is what teams budget; RAM is what runs out. Under systemd, PID 1, the journal daemon, the D-Bus daemon and udevd are four resident processes BusyBox init does not have. Sum Pss from /proc/<pid>/smaps_rollup, or use systemd-cgtop -m.

Your real service topology. Count long-running processes that must survive a crash; under five, BusyBox init is almost certainly right. For each, write down what must be ready before it starts and count the pairs. Count how many can hang without exiting — anything with a blocking I/O loop or a third-party SDK. Count how many you did not write. Then count the sleep calls and lock files in your current rcS: three or more means you have already built a service manager, badly.

Boot time, before assuming either option helps. Under systemd, systemd-analyze time splits firmware, loader, kernel and userspace, and critical-chain shows the path that gates boot. Under either init, initcall_debug printk.time=1 covers the kernel side, and instrumenting rcS covers user space.

raghu@techveda.org:~$ systemd-analyze critical-chain
raghu@techveda.org:~$ read -r up _ < /proc/uptime; echo "rcS-done $up" > /dev/kmsg

The only number a product review should accept is a GPIO your application toggles, measured from power-on.

The decision

In systemd vs BusyBox init, the default for a fixed-function device — a sensor gateway, a controller, a camera node, a kiosk — with a known set of services is BusyBox init. It matches the product's complexity and leaves the C library and kernel version open. This is also the Buildroot developers' recommendation: BusyBox init is sufficient for most embedded systems.

Choose systemd when at least one of these is true, and be honest about whether it is:

The platform is already systemd-based and has sufficient flash. Diverging from the vendor BSP is not a one-time task; it re-opens at every BSP bump, and it should be budgeted as recurring engineering. Most silicon-vendor reference distributions ship systemd, so verify what yours does before assuming a free choice.
Services depend on each other in ways a linear script cannot express, or you already order them with sleep loops and lock files.
A service can hang without exiting and the product must notice. WatchdogSec= plus RuntimeWatchdogSec= answers this; respawn does not.
You need per-service resource limits or sandboxing, so one service cannot starve another. On a device hosting a customer application this is the strongest argument in systemd's favour.
The device runs software your team does not control: customer applications, containers, third-party agents.

Two conditions rule out systemd regardless of everything above. If your toolchain is musl or uClibc-ng and you build with Buildroot, systemd is unavailable without changing the C library for the whole product, and OpenRC is the middle option there. That is Buildroot's packaging constraint rather than an upstream one, since upstream supports musl when systemd is built with -Dlibc=musl. If your kernel is older than 5.10, upstream systemd does not support it at all, which is a reason to update the kernel before adopting systemd.

Consequences

Boot time does not improve automatically. Parallel startup helps a given service reach ready state sooner, but the Yocto documentation is direct about the whole-system effect: systemd also provides more services by default, therefore increasing the total system boot time.

Myth: "systemd makes the device boot faster."

The honest argument for systemd on boot time is that it provides the measurement tools, not that the device boots sooner. Total boot time is set by what you start and in what order, which is a property of your image rather than of PID 1. Our series on boot time phases and measuring the kernel and user space covers the method under either init.

Where the logs go is decided once, for the life of the product. systemd's integrated logging is a real benefit, and on raw NAND or eMMC a persistent journal is also a wear mechanism. Set Storage=volatile and the journal lives in /run/log/journal, touching no flash. The documented default has changed across releases — older systemd documents auto, current systemd documents persistent — so run systemd-analyze cat-config systemd/journald.conf on your target rather than trusting one figure. If the journal stays in RAM, bound it: RuntimeMaxUse= defaults to 10% of the filesystem it sits on, capped at 4G, which on a tmpfs means RAM. Under BusyBox init you reach the same place by default, with syslogd -C and logread.

The dependency set becomes your security surface. D-Bus, udev, util-linux, kmod and libcap are five more upstream projects to track for the supported life of the device. Produce the number rather than asserting it: build with each init selected and diff the manifests. make legal-info in Buildroot writes output/legal-info/manifest.csv; a Yocto build with CVE checking and an SBOM class gives the equivalent. That difference is the maintenance cost, as a component count you can put in the record.

Layout constraints propagate, including to a read-only root. systemd requires a build prefix of /usr and does not support split-usr systems; a separate /usr must be mounted from the initrd. On a read-only root — standard on A/B devices, and enabled in Yocto with IMAGE_FEATURES += "read-only-rootfs" — the handling of /var becomes an explicit choice, which Buildroot exposes as BR2_INIT_SYSTEMD_VAR_FACTORY (default), _OVERLAYFS and _NONE. This qualifies one argument for BusyBox init: on a read-only root you cannot edit rcS on the target, so the benefit is a boot sequence short enough to reason about, not one editable in the field. On a PREEMPT_RT product, upstream additionally recommends CONFIG_RT_GROUP_SCHED=n under systemd, because real-time group scheduling requires an explicit budget per unit.

Choosing BusyBox init means owning the supervision logic. The unthrottled crash loop described earlier is yours to contain. Set panic= on the kernel command line so a dead PID 1 still resets the board, feed the hardware watchdog from a small dedicated process rather than the application, and write the supervision design down as a document rather than leaving it in rcS. The staffing risk is not that BusyBox init is hard to learn; it is that only its author knows what it does.

When this recommendation is wrong

The recommendation fails when "fixed-function" is doing more work than it can bear. An instrument cluster or a medical monitor is fixed-function in the product sense and still runs a dozen interdependent services with real failure modes. The useful test is not what the product does, it is the topology count from the measurement section. It also fails for a team whose background is server or desktop Linux: the argument that BusyBox init is simpler assumes an engineer who reads shell more fluently than unit files, and that assumption is worth checking rather than presuming.

Key takeaways

In systemd vs BusyBox init, default to BusyBox init for fixed-function devices, as Buildroot's developers recommend — and record why, so the next team does not repeat the argument.
Deviate when the platform is already systemd-based, when a service can hang without exiting, or when the device runs software you do not control.
Neither option supervises by default. BusyBox init does not throttle respawns; systemd's WatchdogSec=, RuntimeWatchdogSec= and StartLimitAction= are all off or inert until you set them.
systemd sets a platform floor: kernel 5.10 minimum, 5.14 recommended, merged /usr, glibc 2.34 — and in Buildroot, a glibc toolchain for the whole product.
Bring four numbers to the review: size delta on your own image, resident memory, service topology count, and boot time measured to application readiness.

Frequently asked questions

Can I use systemd with a musl toolchain?
Upstream systemd lists musl 1.2.6 or newer as a requirement when built with -Dlibc=musl, so upstream support exists. Buildroot's systemd package still declares depends on BR2_TOOLCHAIN_USES_GLIBC, so choosing systemd in Buildroot means choosing a glibc toolchain for the whole product. On musl in Buildroot, OpenRC is the dependency-ordered alternative.

What is the minimum kernel version for systemd?
The upstream systemd README sets 5.10 as the minimum baseline and states that kernels below it are not supported at all. Version 5.14 is the recommended baseline; below it, systemd sets an old-kernel taint flag and upstream support is limited.

Does BusyBox init detect a service that has stopped responding?
No. An inittab entry with the respawn action restarts a process when it exits, but a process still running and no longer doing useful work will not be restarted, and BusyBox's own documentation states that it does not stop processes from respawning out of control. Detecting a hung service needs a watchdog protocol you implement yourself, or systemd's WatchdogSec=.

Does the licence difference affect the decision?
Not materially. systemd is LGPL-2.1-or-later with exceptions noted in its tree, and BusyBox is distributed under version 2 of the GPL only. Both run as separate processes from your application, so the compliance obligation is broadly the same. The one case worth noting is linking libsystemd into a proprietary daemon to obtain sd_notify; the readiness protocol is a documented datagram to $NOTIFY_SOCKET with reference implementations published upstream, so you can implement it without linking anything.

DEV Community