The hourly mirror job was crashing at session(). Consistently. Whenever content generation or publish jobs were running at the same time, Chrome took longer than 15 seconds to come up, the CDP connection timed out, and the whole mirror run aborted before it touched a single post.
The 15 second default is fine on an idle machine. The machine is never idle at the top of the hour. The content pipeline runs generate, lint, publish, and mirror all in the same window. Under that load, a cold headless Chrome launch easily takes 25 to 40 seconds. The bring up code had no idea. It waited 15 seconds, threw a RuntimeError, and left a zombie Chrome process squatting the port and the profile directory.
The next run hit the same port conflict and failed faster.
What I changed
First, I extended the deadline to 45 seconds. That alone covers most contention. Cold Chrome on a busy machine almost never exceeds 40 seconds in practice, so 45 gives real headroom without letting a genuinely broken launch spin forever.
Second, I added a recovery path on RuntimeError. If the connect fails, the code now finds any Chrome process attached to that port or profile, kills it, and retries the bring up once. The zombie problem was real: a partially launched Chrome holds the user data directory lock, so the next attempt would fail even after load dropped. You have to evict it explicitly.
One retry is enough. If the second attempt fails, something is structurally wrong: missing binary, corrupted profile, OS resource exhaustion. The right answer at that point is a hard abort with a useful error, not infinite retry.
The tradeoff
45 seconds is a long time to block. On a job that runs every hour, acceptable. On something that fires every few minutes, you would want a different strategy: a keepalive process that holds Chrome open between runs, or a browser pool. I did not do that here because the added complexity is not worth it for an hourly job.
What I would do differently on a greenfield build
Do not schedule jobs that compete for the same resources in the same window. Generate, publish, and mirror could stagger by even five minutes and avoid most of this. I stacked them at the same launchd interval because it was convenient at the time. Convenience like that compounds into exactly this kind of brittle timeout.
The fix is small. The lesson is that "works on an idle machine" is not a sufficient bar for any job running on infrastructure that also does other things.
Top comments (0)