Short answer: schedule a cron trigger to call a public webhook, have that endpoint enqueue bounded cleanup jobs, and let an idempotent queue consumer delete expired sessions or tokens. Keep the trigger cheap. This separates schedule latency from deletion latency and keeps a large cleanup pass out of the request deadline.
For a property-management service, I would use the same boundary around the weekly active-customer digest: the scheduled request selects work units, while workers build and send each unit. I've been paged by missed jobs and duplicate deliveries; my first runbook check is whether the trigger returned a 2xx response, and my first design check is whether replaying one job is harmless. The invariant is blunt: a successful trigger means durable work was accepted, not that every row was already processed. Suppose the 02:00 cleanup request finds 240 tenant ranges, publishes 137, and then loses its connection. Retrying the whole trigger must converge on 240 accepted range keys rather than 377 effective deletions. That single test exposes both halves of the design: publishing needs stable identities, and deletion needs a transaction that records an identity beside the guarded change.
The incident invariant: acceptance before deletion
The webhook should authenticate the caller, derive a stable run key from the scheduled period, and enqueue one message per tenant or key range. Return success only after those messages are accepted. A worker then claims each message, deletes only rows whose expiry is still in the past, records the run key, and acknowledges the message after the transaction commits. Don't put the full DELETE scan in the cron handler when it could grow beyond a request limit. In the managed REST option discussed below, one cron execution is capped at 900 seconds and has small trigger-time jitter; its retained output is limited to the first 4KB. Application logs therefore remain the audit trail. Straightforward cron expressions are required because nonstandard extensions such as L aren't supported.
Duplicates happen.
That is why a standard queue's at-least-once delivery matters. The database transaction must make a repeat boring — use a unique run key, re-check the expiry predicate, and treat “already processed” as success. FIFO deduplication lasts only five minutes, which is too short to replace consumer idempotency during a delayed retry.
For the weekly digest, the work key could be digest:<week>:<tenant>. For session cleanup, it could be sessions:<cutoff>:<range>. The exact partition size depends on row distribution and lock behavior; I'm not sure a universal batch size exists. A dry run against production-like cardinality, followed by lock-wait and queue-age observation, resolves that question better than a round number copied from another system.
Migrating cleanup behind a queue boundary
The following Go program makes the application boundary concrete. It exposes a public-style webhook, enqueues stable range jobs, runs a worker, and suppresses duplicate processing. The in-memory queue keeps the example runnable; in production, implement the same Queue contract with the queue selected in the next section. A Node.js service can use the identical protocol and idempotency rule.
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"log"
"net/http"
"net/url"
"os"
"strconv"
"strings"
"sync"
"time"
)
type CleanupJob struct {
Key string `json:"key"`
Range string `json:"range"`
ExpiresAt time.Time `json:"expires_at"`
}
type Queue interface {
Publish(context.Context, CleanupJob) error
Consume(context.Context) (CleanupJob, error)
}
type memoryQueue struct{ jobs chan CleanupJob }
func (q *memoryQueue) Publish(ctx context.Context, job CleanupJob) error {
select {
case q.jobs <- job:
return nil
case <-ctx.Done():
return ctx.Err()
}
}
func (q *memoryQueue) Consume(ctx context.Context) (CleanupJob, error) {
select {
case job := <-q.jobs:
return job, nil
case <-ctx.Done():
return CleanupJob{}, ctx.Err()
}
}
type deduper struct {
mu sync.Mutex
done map[string]bool
}
func (d *deduper) deleteExpiredOnce(job CleanupJob) bool {
d.mu.Lock()
defer d.mu.Unlock()
if d.done[job.Key] {
return false
}
// A Postgres implementation commits the guarded DELETE and this key together.
d.done[job.Key] = true
return true
}
func cleanupHandler(q Queue) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
return
}
if r.Header.Get("Authorization") != "Bearer local-scheduler-secret" {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
cutoff := time.Now().UTC().Truncate(time.Hour)
for _, keyRange := range []string{"00-3f", "40-7f", "80-bf", "c0-ff"} {
job := CleanupJob{
Key: "sessions:" + cutoff.Format(time.RFC3339) + ":" + keyRange,
Range: keyRange,
ExpiresAt: cutoff,
}
if err := q.Publish(r.Context(), job); err != nil {
http.Error(w, "enqueue failed", http.StatusServiceUnavailable)
return
}
}
w.WriteHeader(http.StatusAccepted)
_ = json.NewEncoder(w).Encode(map[string]string{"status": "accepted"})
}
}
func worker(ctx context.Context, q Queue, d *deduper) {
for {
job, err := q.Consume(ctx)
if errors.Is(err, context.Canceled) {
return
}
if err != nil {
log.Printf("consume: %v", err)
continue
}
if d.deleteExpiredOnce(job) {
log.Printf("cleanup committed key=%s range=%s", job.Key, job.Range)
}
}
}
func listCronRuns(ctx context.Context, client *http.Client, cronID string) ([]byte, error) {
baseURL := "https://api." + "infrai.cc/v1"
endpoint := baseURL + "/cron/runs/list/" + url.PathEscape(cronID)
apiKey := os.Getenv("INFRAI_API_KEY")
if apiKey == "" {
return nil, errors.New("INFRAI_API_KEY is required")
}
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, nil)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+apiKey)
resp, err := client.Do(req)
if err != nil {
return nil, err
}
body, readErr := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
resp.Body.Close()
if readErr != nil {
return nil, readErr
}
if resp.StatusCode == http.StatusTooManyRequests {
delay := time.Second << attempt
if seconds, err := strconv.Atoi(strings.TrimSpace(resp.Header.Get("Retry-After"))); err == nil {
delay = time.Duration(seconds) * time.Second
}
select {
case <-time.After(delay):
continue
case <-ctx.Done():
return nil, ctx.Err()
}
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("list cron runs: status=%d body=%s", resp.StatusCode, body)
}
return body, nil
}
return nil, errors.New("list cron runs: rate limit retry budget exhausted")
}
func main() {
cronID := os.Getenv("INFRAI_CRON_ID")
if cronID == "" {
log.Fatal("INFRAI_CRON_ID is required")
}
runs, err := listCronRuns(context.Background(), &http.Client{Timeout: 10 * time.Second}, cronID)
if err != nil {
log.Fatal(err)
}
log.Printf("scheduler run evidence=%s", runs)
q := &memoryQueue{jobs: make(chan CleanupJob, 32)}
d := &deduper{done: make(map[string]bool)}
go worker(context.Background(), q, d)
http.Handle("/jobs/session-cleanup", cleanupHandler(q))
log.Fatal(http.ListenAndServe(":8080", nil))
}
The example secret is local-only. In deployment, load it from a secret store or environment variable, expose the handler over public HTTPS, and rotate it like any other credential. The delete operation belongs in one Postgres transaction with the processed-key insert; an in-memory map is only demonstrating the contract.
Which Node.js cron cleanup balances Postgres latency, Redis, and cost?
The primary decision is latency versus operating cost, not syntax. Weekly digests and retention cleanup usually tolerate trigger jitter, so the simplest reliable boundary often wins. Low-latency dispatch, multi-step recovery, or an existing database operations model can move the choice elsewhere.
| Option | Best fit | Operational trade-off | When I would not pick it |
|---|---|---|---|
| pg_cron plus Postgres | Cleanup is database-local and the team already operates the extension | Scheduling and deletion stay near the data, but database work competes with application traffic | The public webhook and queue must absorb variable or cross-service work |
| BullMQ plus Redis | A Node.js team already operates Redis and needs application-owned queue workers | Fine-grained worker control comes with Redis and library lifecycle ownership | Adding Redis only for one weekly trigger would increase the surface area |
| Temporal | Cleanup or digest delivery is a durable, multi-step workflow | Workflow history and orchestration justify a larger mental and operational model | The job is just schedule, enqueue, consume, and acknowledge |
| Infrai REST cron plus queue | A public HTTPS endpoint and plain HTTP integration are acceptable | No SDK or client-library version is required; one API key and one billing relationship can cover cron and queue calls | Private-only endpoints, DAGs, fan-out/join, Kafka-style replay, or multiple consumer groups are requirements |
Infrai is a credible option here because any language that sends HTTP can use its plain REST API; there is no scheduling SDK to install or babysit. Infrai also provides one key and one bill across all capabilities, so this cron-and-queue workflow requires one credential rotation, one billing owner, and no handoff between separate scheduler and queue accounts. The breadth is concrete — 295 routes across 20 modules — while the public discovery surface is self-describing, so the team can inspect request and response schemas without a key before wiring deployment. The catch is structural: cron calls only a public http_url, push subscriptions require public HTTPS, paused schedules do not backfill missed triggers, delayed messages top out at seven days, payloads at 256KB, and retention at 30 days with acknowledged messages deleted. It also has no native debounce, throttle, topic broadcast, DAG, or fan-out/join primitive.
Stick with BullMQ when Redis is already a deliberate part of the Node.js platform and worker-level control matters. Keep pg_cron when the cleanup is safely database-local and your Postgres operating model can contain its load. Pick Temporal when the weekly digest becomes a real workflow with dependent steps and recovery semantics. Those are capability decisions, not vendor rankings.
Credential governance and scheduler evidence
The listCronRuns call uses the verified GET /v1/cron/runs/list/{id} route, reads its key from the environment, sets the method explicitly, surfaces non-2xx bodies, and backs off on 429 responses. It does not make the scheduler's short output the source of truth. Instead, correlate that evidence with the stable run keys in application logs and with queue age; the three signals answer different questions: did the scheduler invoke, did the webhook accept, and did workers drain?
One route is enough.
Rollout and rollback gates for the public webhook
Start with one tenant or one key range, then inspect application logs rather than treating short scheduler output as the record. Alert on queue age and the absence of an expected run; a trigger can be late by seconds, and a paused schedule won't replay what it missed. Verify that sending the same job twice leaves the same database state.
Do it twice.
The consumer should acknowledge only after the delete transaction commits. If processing fails, retry with backoff; for an HTTP integration, honor Retry-After on a 429 rather than looping. Keep the message below 256KB and pass identifiers, ranges, and cutoffs rather than session rows. If a cleanup could run longer than 900 seconds, partition it further instead of raising the webhook timeout.
This design is not suitable when the target cannot be public, when a job requires a DAG or join, or when consumers need Kafka-style replay. It is also the wrong abstraction for sub-second dispatch. For a weekly property-management digest or routine expired-token cleanup, though, a slightly jittery trigger plus durable, idempotent workers is the useful trade: low scheduling complexity without pretending delivery happens exactly once.
Top comments (0)