Drain Payload’s Schedule Queue on Serverless, End to End

Payload CMS scheduled publishing begins with a single job in a queue. You set a publishSpecificVersion task via the admin UI, Payload enqueues it with a waitUntil timestamp, and then the job sits there. On a long-running server this drains invisibly. On serverless infrastructure—Vercel, Netlify, AWS Lambda—nothing happens. The database holds a perfect, never-executed intent.

If you deploy Payload on Vercel, you hit this wall immediately. The rest of this guide shows the production pattern we use at techpotions to drain that queue reliably with an external scheduler—and the three failure modes that break it silently.

Draining the Queue: jobs.autoRun vs the Manual Route

The Payload docs point you to two paths: automatic cron tickers and manual queue invocation. Only one is real on serverless.

Option	How it works	Works on serverless?
jobs.autoRun with '* * * * *'	Polls the jobs table every minute from the running Node.js process	No. The process sleeps between requests.
Manual payload.jobs.run()	An HTTP route calls the job runner directly	Yes, if an external scheduler pings it.

Most Payload projects set scheduler: 'cron' in their Payload config and move on. That works on a VPS where the process is alive 24/7. On Vercel, Hobby plan crons fire at most once a day—useless for minute-level publish windows. Even on Pro, a function that sleeps between invocations loses the event loop that would normally drain jobs automatically. The autoRun tick never fires because there is no persistent Node.js event loop.

The manual route is the answer. You expose a route that kicks the queue, then ask something external to call it on a schedule.

The Authenticated Drain Route (What We Actually Ship)

Here is the route we add to every production Payload deploy. It accepts two auth styles because real infrastructure is mixed.

// app/api/run-jobs/route.ts
import { getPayload } from 'payload'
import config from '@payload-config'
import { NextRequest, NextResponse } from 'next/server'

export async function GET(req: NextRequest) {
  // 1 — Vercel Cron sends this automatically
  const authHeader = req.headers.get('authorization')
  // 2 — n8n and other schedulers prefer a query param
  const urlToken = req.nextUrl.searchParams.get('token')
  const valid = process.env.CRON_SECRET

  if (
    (authHeader && authHeader === `Bearer ${valid}`) ||
    (urlToken && urlToken === valid)
  ) {
    const payload = await getPayload({ config })
    await payload.jobs.run({ allQueues: true })
    return NextResponse.json({ ok: true })
  }

  return NextResponse.json({ ok: false }, { status: 401 })
}

// Vercel edge needs this config
export const runtime = 'nodejs'
export const maxDuration = 60 // seconds

What this route does

• It uses allQueues: true to drain every pending job in Payload’s jobs table—not just a single named queue.
• It validates requests against a shared secret, the same pattern Vercel Cron uses natively.
• It bumps maxDuration to 60 seconds. A queue with many stalled jobs can blow past the default 10-second Vercel function timeout.

How we schedule it

We currently use n8n, an open-source workflow automation engine, to ping /api/run-jobs?token=$CRON_SECRET every minute. Why n8n? Because it already orchestrates other business logic in our stack—sending email digests, warming caches, syncing to analytics. Adding the Payload drain there centralises observability without spreading cron configs across platforms.

If you live entirely inside Vercel, Vercel Cron on a Pro plan works identically. Point it at the same route with the Authorization header set to Bearer $CRON_SECRET. Just don’t run it on Hobby—it will only trigger once per day and your posts will go live unpredictably.

Why This Matters: The Stale Queue Problem

We’ve seen teams ship scheduled publishing into production, test it by waiting 30 seconds, see the post flip, and assume it works. Then the Monday morning blog post goes live at 4 PM because the queue drained only when someone triggered an unrelated admin action.

The only thing that drains the queue is payload.jobs.run(). On serverless, it is never called automatically. Not by the payload server, not by Next.js middleware, not by onInit hooks. The job sits in the mongodb or postgres jobs table, status pending, waitUntil set to the chosen publish time, until something external wakes the runner.

Three gotchas that break the drain

1. Vercel Hobby cron limit: The docs say “cron jobs,” the pricing page says “once per day.” If your publish schedule needs minute-level precision, you cannot run the drain on Hobby. Either upgrade to Pro, or offload scheduling to a service like n8n that you control.
2. Timeout without maxDuration: The default function timeout kills the runner mid-loop. If you have 50 jobs to drain, the route returns 504 and half the jobs stay pending. Set maxDuration high enough to handle your worst-case queue depth.
3. Cold starts and retries: External schedulers retry by design, but Payload’s job runner is idempotent by job ID—a duplicate ping won’t publish the same doc twice. No harm in over-scheduling; the real risk is under-scheduling.

Managing Scheduled Work Beyond Publishing

Once the manual drain is running, it benefits more than the publishSpecificVersion job. Any task you enqueue with a waitUntil timestamp—data syncs, bulk operations, revalidation—rides the same route. The pattern is identical: enqueue the work via payload.jobs.create(), set the delay, and let the scheduler drain it.

We use this for AI embedding generation and batch processing where the UI shouldn’t hang. That’s covered in our web delivery services, where we build Payload backends that handle background work without degrading the admin panel.

Testing the Drain Locally

Before shipping, test the route in isolation.

1. Start Payload dev and set a post to publish 2 minutes in the future.
2. Wait 1 minute. Hit http://localhost:3000/api/run-jobs?token=dev-secret. The post stays draft—the waitUntil hasn’t passed.
3. Wait another minute. Hit the route again. The post flips to published.

This confirms that timing respects waitUntil and that the route itself is the triggering mechanism. If it doesn’t flip, check that the jobs collection in your database contains the pending job with the correct waitUntil timestamp.

FAQ

How do I enable Payload CMS scheduled publishing on Vercel?

You configure schedule publishing in Payload’s admin UI, but the jobs won’t drain automatically. You need an authenticated route that calls payload.jobs.run({ allQueues: true }) and an external scheduler (Vercel Cron on Pro, or a service like n8n) to ping it every minute.

Why aren’t my Payload scheduled posts publishing on time?

Nearly always because the jobs queue isn’t being drained. On serverless hosts, the Node.js process doesn’t stay alive to run internal cron ticks. You must manually invoke the job runner via a route that an external scheduler calls.

Does the manual drain route handle future-dated jobs safely?

Yes. Payload’s job runner checks each job’s waitUntil timestamp. Jobs scheduled in the future are skipped until their time arrives, so it’s safe to ping the drain every minute regardless of what is in the queue.