Bottom line: use an external scheduler to request one daily cleanup run, but make the cleanup itself a resumable, idempotent worker that deletes bounded batches by an immutable retention cutoff. A cron expression answers when to try. It does not prove that old uploads, logs, or database records were deleted.
That distinction is my selection criterion. For one Express instance on a machine that rarely restarts, an in-process timer can be adequate. Once there are multiple replicas, deployments, or enough data that deletion can outlive a request timeout, I want scheduling outside the web process and completion state inside durable storage. Keep 0 3 * * * in configuration, use UTC, and treat every invocation as a duplicate until the worker proves otherwise.
I've been paged for both halves of this mistake: a job that never ran after a deployment and a job that ran twice. The runbook now starts with evidence, not the comforting sight of a timer callback.
How should an Express service run a daily cleanup job from a cron expression?
Start by separating four responsibilities: the scheduler emits an intent, a dispatcher gives that intent a stable run key, a worker claims and processes bounded batches, and storage records progress. The Express service may expose an authenticated internal trigger or enqueue the intent, but it should not hold the whole cleanup loop open in an ordinary public request. If the process exits after accepting the request, another worker must be able to continue from persisted state.
For a daily policy, derive the run key from the policy name and the UTC date, such as uploads-retention:2026-08-06. Put a unique constraint on that key. Two scheduler deliveries then converge on one logical run instead of producing two independent deletion sweeps. Calculate the cutoff once when the run is created; don't keep evaluating “older than 30 days” during pagination, because the target set moves while the worker is walking it.
Use an immutable ordering key for each dataset. Upload objects might use (created_at, object_id), while audit records use (created_at, id). Fetch a fixed-size page strictly after the saved cursor and at or before the cutoff. Delete, save the new cursor, and repeat. For blobs plus database metadata, define which side is authoritative and make each operation safe to repeat. A missing object can mean the desired state is already true; a missing row can mean the same. Record that outcome rather than turning it into a mystery retry.
The simplest architecture is the one with the fewest failure states you cannot observe. For small workloads, that may be a host cron calling a private endpoint. For larger or slower ones, it may be a scheduler feeding a durable queue. The expression stays simple either way. The execution contract is where the real engineering lives.
A fired timer is not completed work
One cleanup I owned looked harmless in review: select expired rows, delete associated files, then remove the rows. The first monthly bill arrived at $812 against my $180 estimate. I had scheduled a full retention scan every minute during testing and the production configuration kept that cadence, so the database repeatedly read the same large cold range even when there was nothing to delete. I'm not sure why none of us challenged the interval in review; we were staring at deletion correctness and missed the read amplification. We changed the schedule to daily, added a covering index beginning with the cutoff column, bounded each page, and put scanned rows beside deleted rows on the dashboard. That incident taught me the invariant I now use: a run is healthy only when its intended scope, progress, terminal state, and work amplification are visible.
Tiny detail. Big bill.
“Triggered” is therefore a scheduling metric, not a success metric. I track trigger lag, claim lag, run age, pages completed, rows or objects examined, items deleted, retry count, and the oldest eligible item left behind. The last gauge catches a green job that advances but never catches up. I also alert on a missing terminal record after the expected completion window, rather than alerting merely because no process is currently running.
Retries need ownership rules. A worker lease should expire so another worker can recover abandoned work, but the durable cursor must only move after a page reaches its defined postcondition. If a queue sits between the trigger and worker, acknowledge only after that durable transition. RabbitMQ documents consumer acknowledgements as the mechanism by which a consumer confirms delivery processing; Google Cloud Pub/Sub likewise describes acknowledged and unacknowledged message handling in its subscriber model. Those products differ, but the portable lesson is plain — delivery and completion are separate events.
This is also why I test duplicate delivery, process exit between delete and checkpoint, a daylight-saving boundary, an empty page, and a page containing an already-missing object. Happy-path tests don't earn much confidence here.
How can a worker make each deletion batch safe to repeat?
The following Go sketch is the shape I expect behind an Express trigger. It uses a repository interface rather than a vendor SDK. The important parts are a fixed cutoff, a stable run ID, a bounded page, and a checkpoint written only after every item in that page reaches the deleted state.
package cleanup
import (
"context"
"errors"
"time"
)
var ErrNotFound = errors.New("not found")
type Item struct {
ID string
CreatedAt time.Time
}
type Run struct {
ID string
Cutoff time.Time
Cursor string
}
type Repository interface {
ClaimRun(ctx context.Context, runID string, cutoff time.Time) (Run, error)
NextPage(ctx context.Context, cutoff time.Time, cursor string, limit int) ([]Item, error)
DeleteObject(ctx context.Context, id string) error
DeleteRecord(ctx context.Context, id string) error
SaveCursor(ctx context.Context, runID, cursor string) error
Complete(ctx context.Context, runID string) error
}
func Execute(ctx context.Context, repo Repository, runID string, cutoff time.Time) error {
run, err := repo.ClaimRun(ctx, runID, cutoff)
if err != nil {
return err
}
for {
items, err := repo.NextPage(ctx, run.Cutoff, run.Cursor, 500)
if err != nil {
return err
}
if len(items) == 0 {
return repo.Complete(ctx, run.ID)
}
for _, item := range items {
if err := repo.DeleteObject(ctx, item.ID); err != nil && !errors.Is(err, ErrNotFound) {
return err
}
if err := repo.DeleteRecord(ctx, item.ID); err != nil && !errors.Is(err, ErrNotFound) {
return err
}
run.Cursor = item.ID
}
if err := repo.SaveCursor(ctx, run.ID, run.Cursor); err != nil {
return err
}
}
}
In a real implementation, the cursor should encode every ordering column, not just the ID shown here, and the repository must enforce the claim and checkpoint atomically. I also place a database retention index under load test before enabling deletion. Explain plans, rows scanned per page, lock duration, and replica lag matter more than how elegant the cron wrapper looks.
Don't delete an unbounded result set in one transaction. Short batches reduce lock duration and give the run a useful cancellation boundary — but smaller isn't automatically better, since excessively small pages increase round trips. Start with a conservative batch, measure production-like data, and adjust. Your mileage may vary.
Choose the smallest control plane that preserves the evidence
I make the service selection after writing down the failure model. Replica count, maximum run time, retry ownership, audit requirements, and operational staffing narrow the field faster than feature checklists do.
| Option | Good fit | Main trade-off |
|---|---|---|
| In-process timer | One long-lived instance, tiny cleanup, missed run is tolerable | Restarts can erase timing state; multiple replicas can duplicate work |
| Host or container scheduler | A controlled host, short trigger, simple operations | Host lifecycle and schedule ownership must be explicit |
| External scheduler plus private trigger | Multiple web replicas, centralized schedule and run history | Authentication, deduplication, and trigger availability become part of the design |
| Scheduler plus durable queue | Long work, backpressure, independent worker scaling | More components, acknowledgement policy, dead-letter handling, and queue observability |
My default for a replicated Express deployment is an external scheduler that sends a short authenticated trigger, plus a worker with a durable run ledger. The catch is that this is not suitable when the cleanup must participate in a single database transaction with the business write that creates the data; use a database-native mechanism or transactional outbox pattern there. Stick with an in-process timer when the application is genuinely single-instance, the dataset is small, and a missed day has little impact. A queue is also unnecessary overhead when one bounded run consistently finishes in seconds and can be retried safely from its ledger.
Deployment deserves a line in the runbook. Pause creation of new runs if a schema migration changes the cleanup query, allow a compatible worker to drain claimed work, then resume. Keep the previous worker contract available until no old-format runs remain. For emergency disablement, stop new triggers without deleting the ledger; the cursor is evidence and the restart point.
Finally, set a retention objective rather than promising an exact wall-clock deletion instant. “Eligible data is removed within 26 hours” leaves room for a daily run and a bounded recovery window. Measure the oldest eligible survivor against that objective. A cron dashboard tells me a clock fired. The survivor age tells me the cleanup worked.
Top comments (1)
"A cron expression answers when to try. It does not prove that old uploads, logs, or database records were deleted."—that sentence should be printed and pinned on every backend engineer's wall!
The distinction between delivery vs completion and using an immutable retention cutoff instead of re-evaluating timestamps mid-pagination is such a crucial lesson. The read-amplification incident ($812 bill due to repeated cold scans) is a textbook example of why observability matters just as much for background jobs as it does for public APIs. Outstanding write-up! 👏