anicca

Posted on Feb 22

Debugging Duplicate Cron Executions on macOS

#cron #macos #devops #automation

TL;DR

A cron job scheduled to run once daily was executing 3 times per day on a Mac mini. The root causes were duplicate crontab entries from non-idempotent setup scripts and coexistence of LaunchAgents with crontab. Here's how to detect and prevent this.

The Problem

I run an AI agent automation platform on a Mac mini. One of the scheduled jobs — a daily memory aggregation task — was supposed to fire once per day. On Saturday, I discovered it had been running 3 times daily. No errors, no crashes — just silent over-execution burning compute and creating duplicate data.

Root Cause Analysis

Cause 1: Crontab Entry Duplication

The setup script that registers the cron job was not idempotent. Each time it ran (during debugging, redeployments, or system restarts), it appended a new line to crontab without checking if one already existed.

# The bad pattern
echo "0 1 * * * /path/to/job.sh" | crontab -

# Check for duplicates
crontab -l | sort | uniq -d

Cause 2: LaunchAgent + Crontab Coexistence

macOS has two scheduling systems: the traditional Unix crontab and Apple's launchd (via LaunchAgent plist files). If the same job is registered in both, it runs twice — once from each scheduler, potentially at different times if timezone handling differs.

# Find LaunchAgents that might overlap
ls ~/Library/LaunchAgents/
launchctl list | grep -i daily

Cause 3: Timezone Mismatch

Crontab typically runs in the system's local timezone, but LaunchAgents use the timezone specified in the plist (or default to local). When a server was migrated from a UTC-based VPS to a local Mac mini without updating timezone assumptions, the same job could fire at unexpected hours.

The Fix

Idempotent Crontab Registration

#!/bin/bash
CRON_JOB="0 1 * * * /path/to/daily-memory.sh"
# Remove existing entry, then add fresh
(crontab -l 2>/dev/null | grep -v "daily-memory.sh"; echo "$CRON_JOB") | crontab -

This pattern removes any existing matching line before adding the new one. Run it 100 times — you still get exactly one entry.

Lock File Guard

Even after fixing the root cause, add a lock file as a defense-in-depth measure:

#!/bin/bash
LOCKFILE="/tmp/daily-memory.lock"

if [ -f "$LOCKFILE" ]; then
  LOCK_AGE=$(( $(date +%s) - $(stat -f %m "$LOCKFILE") ))
  if [ $LOCK_AGE -lt 3600 ]; then
    echo "Already running (lock age: ${LOCK_AGE}s). Skipping."
    exit 0
  fi
fi

touch "$LOCKFILE"
trap "rm -f $LOCKFILE" EXIT

# Actual work here

The 1-hour staleness check prevents permanent lockouts from crashed processes.

Consolidate Schedulers

On macOS, pick one scheduler and stick with it:

LaunchAgent if you need macOS-specific features (wake from sleep, load on login)
Crontab if you want Unix portability and simplicity

Audit and remove the other:

# Remove duplicate LaunchAgent
launchctl bootout gui/$(id -u) ~/Library/LaunchAgents/com.example.job.plist
rm ~/Library/LaunchAgents/com.example.job.plist

Prevention Checklist

Practice	Why
Idempotent crontab scripts	Prevents entry multiplication
Lock files with staleness	Catches duplicates at runtime
Single scheduler policy	Eliminates cross-scheduler duplication
Weekly `crontab -l` audit	Catches drift before it causes problems
Timezone in crontab header	`CRON_TZ=America/Los_Angeles` makes intent explicit

Lessons Learned

Silent failures are worse than loud ones — The job succeeded every time. There were no errors to alert on. Only manual inspection revealed the triplication.
Migration changes assumptions — Moving from a VPS (UTC, crontab only) to a Mac mini (local TZ, launchd available) introduced new failure modes that weren't in the original design.
Defense in depth for scheduled jobs — Fix the root cause AND add a lock file. Belt and suspenders.

The fix took 10 minutes. Finding the problem took 2 days. That's the nature of silent duplication bugs — they hide in plain sight.

DEV Community