Tips for Running Stable Background ML Inference on macOS

📝 Originally published (in Japanese) at forge.workstyle.tech.

Running an inference service as a background process on macOS, with a Linux server mindset, can lead to subtle issues. Things like "a one-liner that works on Linux doesn't work on Mac" or "grepping logs results in garbled text errors and crashes" — these are minor but time-consuming problems.

This article compiles a collection of short tips gathered from running a Seed-VC based voice conversion service (FastAPI + uvicorn, local 127.0.0.1:8770) as a background process on macOS. It focuses on macOS-specific pitfalls not covered in Linux-centric articles.

TIP 1: setsid / timeout are not available on macOS

First, it's important to note that macOS (BSD-based) does not include GNU coreutils' setsid or timeout by default. If you use these commands, which are used for backgrounding and timed execution on Linux, directly in a Mac script, you'll get a command not found error.

There are two solutions:

• Install coreutils via Homebrew and use gsetsid/gtimeout
• Use standard tools as alternatives (next TIP)

For background scripts that avoid external dependencies, using standard tools as alternatives is a safer option.

TIP 2: Use nohup + disown for background processes

In environments without setsid, the combination of nohup and disown is reliable for keeping processes alive even after closing the shell.

# Run the inference service as a background process
nohup bash scripts/start-backend.sh > backend.log 2>&1 &
disown

• nohup ... Ignores hangup signals (SIGHUP), keeping the process alive even after the terminal is closed
• > backend.log 2>&1 ... Redirects standard output and standard error to a log file
• & ... Runs the process in the background
• disown ... Removes the job from the shell's job table, preventing it from being terminated when the shell is closed

While nohup alone usually keeps the process alive, adding disown ensures that closing the terminal won't accidentally terminate the process.

TIP 3: tr / grep fail with binary data in logs → use LC_ALL=C

This was the most problematic issue on Mac. Inference logs may contain progress bar control characters or, occasionally, garbled multibyte sequences. When processed by macOS's tr or grep, you'll see:

tr: Illegal byte sequence

This happens because the locale is set to UTF-8, causing invalid byte sequences to be treated as "invalid characters" and throwing an exception.

The solution is to set the locale to C (pass-through as byte sequences) for those commands.

# Remove unwanted control characters from logs (avoids Illegal byte sequence)
LC_ALL=C tr -d '\r' < backend.log > backend.clean.log
LC_ALL=C grep "ERROR" backend.log

Setting LC_ALL=C treats text as "bytes" rather than "characters," preventing crashes due to invalid sequences. This is safer for pipelines that process or search logs programmatically.

TIP 4: Wait for startup completion using the health endpoint

Loading models takes time, so sending requests immediately after starting with nohup will fail because the service isn't ready. Using sleep 10 as a workaround is unreliable—too short for slow machines and too long for fast ones.

The proper approach is to poll the service's health endpoint until it returns a 200 status. In this setup, the health endpoint is http://127.0.0.1:8770/health, so we poll it.

# Wait for the health endpoint to be ready before proceeding
until curl -sf http://127.0.0.1:8770/health >/dev/null; do
  sleep 1
done
echo "backend ready"

curl -sf exits with a non-zero status on failure, so combining it with until allows you to wait until the service is ready. Waiting based on state, not a fixed delay, significantly improves the reliability of startup scripts.

TIP 5: Stop processes with pkill using pattern matching

For background processes without a saved PID, pkill with a command-line pattern is convenient for stopping them.

# Stop the inference service started with uvicorn
pkill -f "uvicorn server:app"

The -f option matches the entire command line, so including specific details like the port or app name in the pattern prevents unrelated processes from being affected. For more precision, save the PID during startup and target it directly.

Summary

• macOS does not have setsid/timeout. Either install Homebrew's coreutils (gsetsid/gtimeout) or use standard tool alternatives
• Use nohup ... & disown for background processes. Ignore SIGHUP and remove the job from the job table to prevent termination when the terminal is closed
• Binary data in logs causes tr/grep to fail with Illegal byte sequence → Temporarily set LC_ALL=C to treat data as byte sequences
• Instead of fixed sleep delays, poll the health endpoint with until curl -sf to wait based on state
• Stop processes with pkill -f "specific pattern". Use a detailed pattern to avoid affecting unrelated processes