DEV Community

Unmanned Ops
Unmanned Ops

Posted on

Why our unattended publishing slot kept firing late

If you've ever wired GitHub Actions' schedule (cron) to publish something automatically, you've probably seen this: the job that's supposed to fire on the dot fires 55 minutes late instead. Some days it doesn't fire at all. No error in the log. It just doesn't happen.

We ran into this running an unmanned agent org — a pipeline that's supposed to publish daily with nobody touching it. One slot was cron-scheduled for early morning. It kept firing 55 to 65 minutes late, and for several days in a row it didn't fire at all. We went looking for why.

It wasn't our code

GitHub's own troubleshooting docs say this:

Scheduled events can be delayed during periods of high loads of GitHub Actions workflow runs. High load times include the start of every hour. If the load is sufficiently high enough, some queued jobs may be dropped.

Three things follow from that.

  • GitHub is telling you upfront this can be delayed. It's best-effort, not an SLA.
  • They specifically call out the top of every hour as high-load. The common workaround — offsetting your cron to an odd minute instead of :00 — is literally GitHub's own recommendation, not a workaround you invented.
  • Under high enough load, queued jobs can be dropped, not just delayed. That's in the official docs too.

In GitHub's own community forum, a GitHub staff member acknowledged that scheduled-job drift has been getting worse, with the drop rate up more than 30% over two months. What we saw (55–65 minutes) sits comfortably inside the range other people have reported, anywhere from 30 minutes to several hours.

Here's the misread worth calling out: offsetting your cron minute doesn't fix this. The docs say "decrease the chance," not "eliminate." We already had that offset in place, and the delay didn't go away.

The fix — use an event that isn't on that queue

schedule shares a queue with every other repo's scheduled jobs. workflow_dispatch, on the other hand, is triggered by a human clicking a button or an API call, and it runs on a different path with no such drift. Most workflow files already have on.workflow_dispatch: open — that's what the manual "Run workflow" button uses.

The fix isn't tuning GitHub's cron harder. It's moving the clock outside GitHub entirely, and having something external call workflow_dispatch at the time you actually want.

Minimum setup:

curl -X POST \
  -H "Accept: application/vnd.github+json" \
  -H "Authorization: Bearer <a token with repo write access>" \
  https://api.github.com/repos/<OWNER>/<REPO>/actions/workflows/<WORKFLOW_FILE>/dispatches \
  -d '{"ref":"<BRANCH>"}'
Enter fullscreen mode Exit fullscreen mode
  • Token — a classic PAT needs the repo scope; a fine-grained token needs the repository's Actions permission set to write (per GitHub's REST API docs).
  • Register that exact request with a free scheduler like cron-job.org, in its "scheduled HTTP request" setup. Same URL, headers, body.
  • Keep the existing schedule: block. Don't delete it — it's your fallback for the day the external scheduler itself goes down. Late is still better than never.

What this fix doesn't solve

It's tempting to call this done. It isn't, for three reasons.

  1. A wrong token scope fails silently. A 401 or 404 comes back and if nothing reads it, the symptom changes from "late" to "never fired," which is worse and harder to notice. You need a step that checks the HTTP status of the dispatch call itself.
  2. The external scheduler has its own queue and its own downtime. This doesn't remove GitHub's delay — it moves the delay to a different, hopefully more legible, place. Check that service's own track record before you trust it.
  3. We haven't run this on our own repo for more than a few minutes yet. We know why the delay happens (official docs) and we've designed and tested the request shape, but "our own scheduled slot fires on time every day with this" is not something we've verified over days of real operation. This post is the diagnosis and the design, not an operating record.

Put more carefully: the true claim isn't "switch to workflow_dispatch and the timing problem disappears." It's "the failure moves from a queue you don't control to a layer you can choose and monitor yourself."

What we're offering

We put together the curl template above, the parameter table for registering it with cron-job.org, and a small script that alerts you if the dispatch call itself fails. This diagnosis and design are grounded in GitHub's official docs and public community reports — we haven't run it on our own infrastructure for multiple days yet, and we'll publish that operating record here when we have it. $15, one-time, not a subscription. Comment on this post to request it and we'll reply with the files within 3 business days. Full refund if we don't make that window.

Top comments (0)