DEV Community

Cover image for Your Go Binary Won't Run on macOS? syspolicyd Is the Culprit
Lei Tang
Lei Tang

Posted on

Your Go Binary Won't Run on macOS? syspolicyd Is the Culprit

#go #claude #ai

A bizarre Go project debugging saga — full investigation assisted by Claude Code

The Strange Bug

It was an ordinary afternoon. I was developing a Go project on my Mac as usual. Code written, go build succeeded, everything looked normal — then ./server started and nothing happened.

No log output. No port listening. No error messages. The process was created, but it was like frozen in place — RSS stuck at 32 bytes.

Even stranger, air (my hot-reload tool) ran perfectly fine — and it's also a Go-compiled binary. Only newly compiled binaries refused to run.

If you've ever run into something similar, you know the drill: first you suspect your code, then the Go version, then you start questioning your life choices.

Fortunately, I used Claude Code (Anthropic's AI coding assistant) throughout this investigation. It was like having an experienced SRE sitting next to me, helping narrow things down step by step. Here's the complete AI-assisted debugging process.

The Investigation: One Elimination at a Time

Step 1: Verify the Build

First, I confirmed it wasn't a code issue:

go build ./...    # Passed, zero errors
go run ./cmd/server/  # Process created, but… no output
Enter fullscreen mode Exit fullscreen mode

Checking the process state:

PID   STAT  RSS  COMMAND
92672 SN      32  server
Enter fullscreen mode Exit fullscreen mode

RSS of 32 bytes basically means the process was created but never actually executed any code.

Step 2: Check Logs (Claude Code's Auto-Discovery)

I asked Claude Code to check the project startup. It automatically read logs/debug.log and noticed the log file timestamps hadn't been updated — meaning InitLogger() was never reached.

The code was stuck somewhere, very early in initialization. Claude Code quickly narrowed the problem to the process initialization phase by analyzing log timestamps and process memory.

Step 3: Confirm Binary Integrity (Claude Code's Automatic Check)

Claude Code ran file and otool -l to analyze the binary:

The binary was compiled correctly, the Mach-O structure was intact — not a corrupted file.

Step 4: Rule Out CGO

I suspected a CGO dynamic linking issue:

CGO_ENABLED=0 go build -o /tmp/testhello ./cmd/server/
Enter fullscreen mode Exit fullscreen mode

Result: Same failure. CGO ruled out.

Step 5: Rule Out Code Signing

I tried signing the binary manually:

codesign -s - --force /tmp/testhello
Enter fullscreen mode Exit fullscreen mode

Result: Same failure. Signing ruled out.

Step 6: Rule Out Quarantine Attributes

macOS tags files downloaded from the internet with com.apple.quarantine or com.apple.provenance attributes. I checked and the newly compiled binary did have com.apple.provenance, but the working air binary had the same attribute — so that wasn't the cause.

Step 7: The Breakthrough — syspolicyd (Claude Code's Key Finding)

After six steps ruled out all the usual suspects, I started suspecting a system-level issue. Claude Code suggested checking system processes and ran:

ps aux | grep syspolicyd
Enter fullscreen mode Exit fullscreen mode

The result was shocking:

root 478  98.7%  /usr/libexec/syspolicyd
Enter fullscreen mode Exit fullscreen mode

CPU usage at 98.7%. This process had been running for 6 days and 16 hours, essentially stuck.

Step 8: Connecting the Dots (Claude Code's Root Cause Analysis)

After discovering the syspolicyd anomaly, Claude Code immediately laid out the complete causal chain:

syspolicyd is the System Policy Daemon. One of its responsibilities is verifying code signatures for new binaries. When the system launches a new process, the kernel calls amfid (Apple Mobile File Integrity), which in turn calls syspolicyd for security validation. If syspolicyd is stuck, the verification request never completes, and the process remains in a loading state, never executing any code.

This also explained why RSS was only 32 bytes — the binary was mmap'd into memory (accounting for those 32 bytes), but the code was never executed.

This also explained why air worked fine — it was already loaded and didn't need re-verification.

Root Cause Analysis 

Frequent go build of new binaries
    ↓
amfid frequently calls syspolicyd for verification
    ↓
Verification requests pile up, queue gets blocked
    ↓
syspolicyd CPU at 100%, enters a deadlock
    ↓
New binaries get stuck at the verification stage
    ↓
Process RSS=32, code never executes
Enter fullscreen mode Exit fullscreen mode

Trigger conditions:

  1. Long uptime without reboot — my Mac had been running for 6 days 16 hours
  2. Frequent compilation of new binaries — repeated go build during development generates many new Mach-O files
  3. Possible compounding factor — macOS notarization checks timing out and retrying, further exacerbating the issue

Solutions

Method 1: Reboot (Simple and Effective) 

sudo shutdown -r now
Enter fullscreen mode Exit fullscreen mode

After rebooting, syspolicyd resets naturally and everything returns to normal.

Method 2: Just Restart syspolicyd (No Reboot Needed) 

sudo killall syspolicyd
Enter fullscreen mode Exit fullscreen mode

The system automatically restarts it, and CPU usage drops back to normal levels. This is the fastest fix.

Method 3: Verify the Issue Is Resolved 

# Check syspolicyd CPU usage
ps aux | grep syspolicyd

# Normal operation should be 0-1% CPU
Enter fullscreen mode Exit fullscreen mode

How to Prevent It

  1. Reboot your Mac weekly — macOS syspolicyd tends to misbehave after long uptime. Regular reboots are the simplest prevention.
  2. Watch your CPU during development — glance at Activity Monitor occasionally. If syspolicyd spikes, deal with it immediately.
  3. Keep macOS updated — Apple has fixed similar issues in later releases.
  4. Don't hesitate to kill it — if you notice abnormal CPU usage, run sudo killall syspolicyd right away. Don't wait for it to lock up.

Reflections

AI-Assisted Debugging Experience

This investigation was conducted entirely with Claude Code. The experience was like pair-programming with a seasoned SRE:

  • I just said "the project won't start," and it automatically ran go build, go run, and analyzed logs
  • When it found the binary process with RSS of 32 bytes, I didn't even need to do the math — it told me directly "the process was created but never executed any code"
  • After systematically ruling out CGO, code signing, and quarantine attributes, it proactively suggested checking system process status and eventually pinpointed syspolicyd
  • From problem discovery to solution (sudo killall syspolicyd), it took less than 20 minutes

This really drove home the point: AI-assisted debugging doesn't replace human experience — it amplifies it. An experienced engineer might figure this out faster, but having AI handle all the repetitive checks, cross-references, and log analysis dramatically accelerates the process.

Back to the Tech

What this investigation taught me: sometimes the problem isn't in your code — it's in your operating system.

When developing Go projects, if compilation succeeds but execution fails, most people's first instinct is to check the code, configuration, and dependencies. But if none of those pan out, consider the system level — a stuck system daemon is all it takes to keep your binary frozen at the starting line.

macOS is generally stable, but syspolicyd seems to be a weak link. Especially for Go developers, frequent compilation triggers its verification mechanism more often than you'd think, making it more prone to breaking down.

If you ever encounter a Go binary that compiles but won't run, with RSS stuck at 32 bytes — don't rush to reinstall your system. Check syspolicyd first. Chances are, that's your culprit.

Have you encountered similar macOS development environment issues? Share your debugging stories in the comments. And if you've used AI-assisted debugging tools, I'd love to hear about your experience too.

Top comments (0)