Go Daemon for MCP Servers: Shared Background Processes Without systemd or Other Supervisors

My team builds GoGPU — a pure-Go GPU ecosystem (1.25M+ LOC, 21 repositories) gogpu/ui — a GPU-accelerated UI framework, and Born ML — a machine learning framework on top of it. Working on codebases this size with AI agents requires serious tooling: you can't just grep and hope. So we built GLIDE — a headless Go IDE written in Go, with 65 MCP tools, a knowledge graph, and full gopls integration, designed to give AI coding agents — Claude Code, Cursor, Copilot — semantic understanding of Go code. It's an internal tool we're preparing for open source, still in progress.

Here's where it gets expensive. GLIDE runs as an MCP server, and each AI agent spawns its own instance. Each instance starts a gopls language server. gopls eats about 1–2 GB of RAM per workspace. Run six agents on GoGPU — congrats, you just burned 12 GB on identical type-checking of the same modules.

The obvious fix: share one gopls across all agents. Run GLIDE as a background daemon, let agents connect on demand.

The non-obvious part: making that daemon reliable on three operating systems, under concurrent access from processes that die without warning, without adding a single dependency to the binary.

This is the story of grpmsoft/daemon — a pure-Go library for on-demand background process lifecycle. We extracted it from GLIDE because the problem is universal: any CLI tool that needs a shared background server faces the same dance of PID files, health checks, and process identity. No CGO, no vendored C, no x/sys. Stdlib only.

The Problem Is Not "How to Fork"

Go can't fork(). The runtime has threads, and forking a multi-threaded process is asking for deadlocks. The typical workaround is re-executing the binary with an environment variable marker. sevlyar/go-daemon does exactly this. kardianos/service takes a different approach — it wraps your binary to run under a system service manager (systemd, launchd, Windows SCM) and can install unit files via an explicit Install() call.

Neither solves our problem. We don't want a system service that lives forever. We want a process that:

1. Starts when the first agent needs it.
2. Shares itself with every subsequent agent.
3. Dies when the last agent leaves.

The owner of the process lifetime is demand, not an init system.

EnsureRunning: The Core Abstraction

The entire library exists so you can write this:

port, err := daemon.EnsureRunning(ctx, daemon.Config{
    Name:    "myapp",
    DataDir: ".myapp",
    Args:    []string{"serve"},
})

If the daemon is already running, you get the port. If not, the library spawns it, waits for the health check, and gives you the port. If two agents call EnsureRunning at the exact same millisecond, one wins the startup race and the other gets the same port. No duplicate daemons. Ever.

The trick is a startup lock file — flock on Unix, LockFileEx on Windows — that serializes all startup attempts across processes. The winner spawns the child, passes the PID file lock via ExtraFiles (so the child inherits ownership with zero gap), and the loser polls until the PID file contains a port.

This required more thought than I expected.

The PID File Is the Lock

Most daemon libraries use the PID file as a data file: write the PID, read it later, check if the process is alive with kill(pid, 0). This has a classic race: process dies, PID gets recycled to an unrelated process, your library kills someone's database.

We took a different approach. The PID file is opened with an exclusive flock (or Windows share-mode lock), and the lock is held for the entire lifetime of the daemon. When the daemon dies — cleanly, via SIGKILL, via OOM — the kernel releases the lock. No timers, no polling, no PID reuse bugs. Identity is a kernel lock, not a number.

lock, err := pidlock.TryLock(pidPath)  // flock(LOCK_EX|LOCK_NB)
// ... write PID data through the lock (Seek+Write+Truncate) ...
// ... pass lock.File() to child via ExtraFiles ...
// child inherits the lock — zero window where it's released

On Unix, ExtraFiles passes the locked fd to the child — the lock is never released between parent and child. On Windows, ExtraFiles is not supported (Go issue #26182), so the parent releases the lock under the startup lock, and the child re-acquires it in Serve with a retry loop. The startup lock serializes this — no window for a third process to steal it.

The lock file is never deleted. Deleting a flock'd file while another process waits on flock() causes the inode-reuse race — both processes end up locking different inodes at the same path. We learned this the hard way.

Counting Connections Was Wrong

v0.1.0 had connect/disconnect endpoints. Proxy starts, POSTs /daemon/connect, counter goes up. Proxy stops, POSTs /daemon/disconnect, counter goes down. When the counter hits zero and IdleTimeout fires, the daemon exits.

Sounds clean. Here's what actually happens: a proxy gets SIGKILL'd. A terminal closes. An OOM killer strikes. The disconnect POST never fires. The counter stays at 1 forever. A daemon with IdleTimeout: 5 * time.Minute sits there consuming RAM until someone manually kills it.

We patched the counter twice — clamping to prevent negative counts from stray disconnects, and CAS loops for concurrent disconnect races. Both fixed real bugs, but neither touches the actual problem: counting requires both endpoints to fire, and one of them is called by a dead process. The positive count leak is inherent to the model.

Leases: Let the Kernel Count

The fix landed in v0.3.1. Instead of POST/POST, the proxy opens a GET /daemon/attach request and holds the TCP connection open for its entire lifetime:

Proxy               Daemon
  │                    │
  ├── GET /daemon/attach ──► ct.Connect()
  │         │          │
  │    TCP alive       │  count > 0, idle timer blocked
  │         │          │
  ╳ SIGKILL            │
  │                    │
kernel closes socket ──► ct.Disconnect()
                        │
                   count drops to 0 → idle shutdown

The TCP connection is the lease. When the proxy dies — for any reason — the kernel closes the socket, the HTTP handler's r.Context() gets cancelled, the deferred ct.Disconnect() fires, and the count drops. No heartbeats. No watchdogs. No stale counts.

A v0.3.1+ proxy against an older daemon without /daemon/attach detects 404 and falls back to the old connect/disconnect pair. The reverse (old proxy, new daemon) gets 401 because the token is missing — upgrade the proxy.

Bearer Token from the PID File

The daemon listens on 127.0.0.1 with a random port, so you might think it's safe. DNS rebinding is a real threat — a malicious page can resolve a domain to 127.0.0.1 and attempt requests. The Host header check (loopbackGuard, added in v0.1.1) stops this: browsers cannot forge the Host header, so the rebinding attack fails at the middleware.

The token protects against a different threat: other local processes that can already reach the port, or any non-browser client. The daemon generates a random token at startup (crypto/rand) and writes it to the PID file, which has 0600 permissions on Unix (on Windows, the DataDir ACL is the boundary). Every /daemon/* control-plane endpoint requires Authorization: Bearer <token>. The proxy reads the token from the PID file and sends it automatically. Your application handler (/mcp, custom routes) is open by default for curl debugging — set Config.RequireToken = true to lock it down too.

// In Serve():
token := rand.Text()  // crypto/rand, Go 1.24+
pidData.Token = token
lock.WriteData(marshal(pidData))

// Middleware:
func authGuard(next http.Handler, token string) http.Handler {
    // /daemon/* → require Bearer, constant-time compare
    // /health → pass through (external probes)
}

What We Didn't Build

No process supervision. If you need your daemon to restart after a crash, use systemd with Restart=on-failure. Our library complements kardianos/service — it doesn't replace it. Mixing Restart=always with idle auto-shutdown creates two supervisors that fight each other.

No remote access. Localhost only. If you need remote dev, that's your application's job, not the daemon library's.

No log rotation. We write to a log file. When it gets big, you rotate it. If this bothers enough people, we'll add it.

No heartbeats. The lease model eliminates the need. Heartbeats are the wrong abstraction when the kernel already tells you the connection is dead.

Numbers

• 2,651 lines of library code. ~5.4K lines of tests. 160+ test functions. 2:1 test-to-code ratio.
• Zero dependencies. go.mod has no require block.
• Three platforms. CI runs build + test + lint + race detector on Linux, macOS, and Windows.
• 80%+ code coverage (85% root package). Every lifecycle feature has an integration test using the helper-process pattern (no external binaries).
• No marginal binary cost if your binary already imports net/http (most MCP/CLI tools do). Otherwise you pay for net/http, not for this library — there are no transitive dependencies.

Try It

go get github.com/grpmsoft/daemon

The README has complete examples for all three modes — client, server, proxy. The library is at v0.3.3, pre-1.0. The API evolves fast — we break things when the design demands it, not on a schedule.

If you're building a CLI tool that needs a shared background server — an MCP server, a language server proxy, a build daemon, a local API cache — this might save you from reinventing the same PID-file-plus-health-check dance that every Go CLI eventually writes.

---

grpmsoft/daemon — Pure-Go on-demand process lifecycle for shared background services. MIT license.