Systemd Services: Writing Your First Custom Unit File

A small worker I’d expected to run all weekend stopped the moment my SSH session ended. The fix wasn’t a clever shell trick; it was letting systemd own the process. In this walkthrough, you’ll create a custom unit, start and enable it, inspect its logs, and learn what to check when it fails.

What we’re building

We’ll run a tiny script that prints a timestamp every 30 seconds. It won’t do useful work, but it demonstrates the important parts of a service: systemd starts it, restarts it after a failure, and collects its output.

The commands below assume a systemd-based Linux distribution and a user with sudo access. Paths and available hardening options can vary slightly between distributions and systemd versions.

Create a dedicated service user and script

A long-running service usually shouldn’t run as root. Create a system account and a directory for the script:

sudo useradd --system --home-dir /opt/demo-worker --shell /usr/sbin/nologin demo-worker
sudo install -d -o root -g root -m 755 /opt/demo-worker

Create the script:

sudo tee /opt/demo-worker/worker.sh > /dev/null <<'EOF'
#!/bin/sh

while true; do
    printf 'Worker heartbeat: %s\n' "$(date --iso-8601=seconds)"
    sleep 30
done
EOF

Make it executable:

sudo chmod 755 /opt/demo-worker/worker.sh

The script is owned by root and readable and executable by the service account. That’s intentional: the service can run it, but cannot modify its own code.

Write the unit file

System-wide custom unit files normally live in /etc/systemd/system/. Create one there:

sudo tee /etc/systemd/system/demo-worker.service > /dev/null <<'EOF'
[Unit]
Description=Demo background worker
After=network.target

[Service]
Type=simple
User=demo-worker
WorkingDirectory=/opt/demo-worker
ExecStart=/opt/demo-worker/worker.sh
Restart=on-failure
RestartSec=5s
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true

[Install]
WantedBy=multi-user.target
EOF

Here’s what the main directives do:

• Type=simple tells systemd that the process started by ExecStart is the service. It’s the usual choice for a foreground process.
• User runs the process as the dedicated account.
• WorkingDirectory sets the process’s current directory.
• Restart=on-failure restarts the service when it exits unsuccessfully, but not after a normal exit.
• RestartSec=5s waits five seconds before attempting a restart.
• ProtectSystem=strict makes most of the filesystem read-only inside the service’s view. Add an explicit writable path if your application needs to write files.
• WantedBy=multi-user.target makes the service eligible to start during a normal boot when enabled.

Don’t add & to ExecStart. systemd expects the application to stay in the foreground so it can track the right process. If your program daemonizes itself, look for a foreground mode or a suitable service type rather than backgrounding it from the unit.

Load, start, and enable the service

Tell systemd to reload its unit definitions, then start the service now and enable it for future boots:

sudo systemctl daemon-reload
sudo systemctl enable --now demo-worker.service

daemon-reload is needed after creating or changing a unit file. It does not restart a service that is already running. If you edit the unit later, run daemon-reload again and then restart it:

sudo systemctl restart demo-worker.service

Check its status:

systemctl status demo-worker.service

You should see active (running) and recent output. You can also check whether it’s configured to start at boot:

systemctl is-enabled demo-worker.service

Read logs and troubleshoot failures

The script writes to standard output, which systemd sends to the journal. Follow new log entries with:

sudo journalctl -u demo-worker.service -f

To inspect recent entries after a failure:

sudo journalctl -u demo-worker.service -n 50 --no-pager

If the service won’t start, check the unit and script paths first. A misspelled ExecStart, a missing execute bit, or a script interpreter that isn’t installed are common causes. You can ask systemd to check the unit file’s syntax:

sudo systemd-analyze verify /etc/systemd/system/demo-worker.service

After editing, remember the distinction: daemon-reload loads the changed unit definition; restart applies that definition to the running service. For a script-only change, a restart is normally enough.

Two details that save time later

A restart policy is not a repair policy. Restart=on-failure can bring a process back after a crash, but it can’t fix a bad configuration or unavailable dependency. If a service keeps failing, inspect the journal rather than assuming repeated restarts will solve it. systemd also limits rapid restart attempts.

Use systemctl edit for local overrides. If you need to adjust a vendor-provided unit, avoid editing its file under /usr/lib/systemd/system/ or /lib/systemd/system/; package updates can replace it. Instead, run sudo systemctl edit name.service to create an override under /etc/systemd/system/. For your own unit in /etc/systemd/system/, edit that file directly.

Where this fits in a deployment

A custom unit is useful for a web worker, a scheduled-independent background process, or an application you want started on boot. It’s process supervision, not a replacement for backups, monitoring, or deployment automation.

If you need a VPS to host a conventional Linux service, PowerVPS is one option to consider. For GPU-backed workloads, Vast.ai offers GPU rental; check the instance’s lifecycle and storage details before relying on it for a process that must survive restarts. A rented instance may be stopped or replaced independently of systemd. For comparing hosting choices, Server Rental Guide is a useful resource.

Summary

A working systemd service needs a foreground process, a unit file in the right location, and a reload before systemd can use a new or changed definition. Run it as a dedicated user, check its status with systemctl, and use journalctl to see what it’s doing. Once those basics are familiar, most custom units are just a few carefully chosen directives around the command you already know how to run.