Short answer: schedule the marketplace cleanup after nightly payment reconciliation, but let durable settlement evidence — not the cron expression — decide which old uploads, logs, and records may be deleted. Use the simplest external scheduler that can identify a missed business date, and make every retry present the same dated run key to an idempotent worker.
This order matters because age is a poor proxy for business completion. A seller document can be old while its payout is disputed. A provider-response log can look disposable while it remains the only correlation evidence for an unmatched transaction. If a second trigger arrives, it must rediscover the same decisions rather than create a second version of the night.
Retention rules precede scheduling
Start with a deletion contract. For each candidate, require a retention policy version, a cutoff derived from an explicit business date and timezone, a settled reconciliation reference, and the absence of a legal or dispute hold. Record why an item qualified. A scheduler may request an evaluation; it shouldn't grant deletion authority.
That distinction changes the service-selection question. An in-process timer is suitable only when one process is guaranteed and a missed run is harmless. Host cron can be enough when a team owns a stable machine and monitors the expected run date. An external scheduler is the cleaner default for replicated or ephemeral Express deployments because application restarts no longer own the clock. A durable queue or workflow engine becomes useful when a nightly run fans out, needs bounded concurrency, or must resume many independent items.
The cron expression can stay boring. 0 2 * * * requests a run at 02:00 in the scheduler's configured timezone; it does not prove that reconciliation finished, that the worker received the request, or that deletion was authorized. Put business_date, timezone, policy_version, and a stable run key in durable state. Derive the cutoff from those values, never from whatever time the worker happens to observe after a retry.
One missed date is more important than one failed invocation.
I bring the same bias to notification pipelines. I've learned to treat an HTTP 429 as a delivery-state transition, not permission to invent a fresh message; otherwise retries can multiply email, SMS, or OTP attempts. Cleanup deserves that discipline too. The idempotency boundary belongs around the business effect, while the transport remains free to redeliver.
What should a daily Node.js Express cleanup job record before deleting old uploads and logs?
A marketplace purge predicate usually crosses several domains. The upload row has an age and an object key. The payment record has a provider reference and settlement state. A dispute table can suspend an otherwise valid policy. Operational logs may contain the correlation value needed to explain why local and provider totals differ. Running DELETE WHERE created_at < cutoff collapses those domains into a timestamp and silently assumes they finish together.
They don't.
Instead, make reconciliation produce evidence that cleanup can consume. A candidate is eligible only when its canonical payment record is matched for the requested business date, every relevant hold is clear, and the applicable policy says the artifact has aged out. Keep the policy version beside the decision so an auditor can distinguish “the rule changed” from “the worker behaved differently.” For object storage, record a tombstone keyed by the artifact and policy decision. If a retry finds the object already absent and the matching tombstone exists, that is the same completed effect, not a new failure.
This is also a compliance boundary. Cleanup logs should contain identifiers needed to prove the decision, but copying raw customer data into those logs creates another retention surface. Prefer internal record IDs, provider correlation references where necessary, policy versions, and structured exclusion reasons. Redaction is part of the design, not a later logging task.
There is a catch: this evidence model costs schema work and operational attention. It is not suitable when the data is disposable cache material with no settlement, dispute, or audit dependency; a bounded age query and one external trigger may be enough there. Conversely, stick with a workflow engine when a single run spans enough independent sellers or provider pages that leases, fan-out limits, and item-level recovery would otherwise become a homegrown control plane. Your mileage may vary at that boundary, and load tests plus an interruption drill are better evidence than a feature checklist.
Test idempotency under interruption
Use a unique key such as marketplace-retention:2026-08-20 for the run. Claim it before reading provider data. Store item decisions independently, because a process can stop after reconciling item 200 while item 201 has not started. Don't hold one database transaction open across provider calls and object deletion; those systems do not share the transaction, and long remote waits would extend local locks.
The core can be expressed as a small Python contract even when Express owns the HTTP boundary:
from dataclasses import dataclass
from datetime import date
@dataclass(frozen=True)
class RetentionRun:
business_date: date
policy_version: str
@property
def key(self) -> str:
return f"marketplace-retention:{self.business_date.isoformat()}"
def evaluate_and_purge(store, payments, objects, run: RetentionRun) -> None:
if not store.claim_once(run.key, run.policy_version):
return
for candidate in store.retention_candidates(run.business_date):
settlement = payments.lookup(candidate.provider_reference)
decision = store.record_decision_once(
run_key=run.key,
record_id=candidate.id,
policy_version=run.policy_version,
settled=settlement.is_matched,
hold_clear=store.hold_is_clear(candidate.id),
)
if not decision.delete_allowed:
continue
objects.delete_if_present(candidate.object_key)
store.record_tombstone_once(candidate.id, run.key)
store.complete_once(run.key)
The names carry the contract: claim_once, record_decision_once, delete_if_present, and record_tombstone_once must converge under repetition. Their implementations need uniqueness constraints at the database boundary, not a check-then-insert sequence in application memory. The payment lookup may happen again. The deletion decision and tombstone may be presented again. Neither repetition may create a new logical outcome.
Notifications sit outside the final database update unless an outbox joins them. Commit the completed run and an outbox row together, then let a separate sender deliver the operator email or SMS with a stable message key. This avoids choosing between a duplicate alert after a crash and a silent gap when the process stops between “complete” and “send.” Rate limits and spam filtering are why I care about this detail — operational correctness includes what the on-call engineer actually receives.
Failure alarms expose missed runs
Compare mechanisms using evidence from a forced retry and a missed-date test, not by counting configuration lines.
| Constraint | Simplest acceptable mechanism | Evidence to collect |
|---|---|---|
| One stable process; missed run is harmless | In-process timer | Restart test shows the accepted gap |
| Stable host with clear ownership | Host cron plus durable run table | Expected business dates are monitored |
| Replicas or ephemeral deployment | External scheduler plus idempotent worker | Duplicate delivery resolves to one run key |
| Large fan-out or multi-step recovery | Durable queue or workflow engine | Interrupted items resume independently |
| Regulated deletion | Any trigger behind a decision ledger | Policy, settlement evidence, and tombstone are queryable |
RabbitMQ consumer acknowledgements make successful processing explicit, and rejected deliveries can be requeued. Google Cloud Pub/Sub likewise treats message delivery and subscriber processing as separate concerns. Neither mechanism can make a payment-provider read, a local commit, and an object-store deletion one atomic event. That is why the worker still owns idempotency even when the transport offers useful delivery controls.
Monitor absence as well as errors. Track the latest completed business date, the oldest unfinished run, candidate and exclusion counts, retry counts, and tombstones without matching decisions. A trigger that never arrived emits no worker exception, so an error-only dashboard can remain green through a missed night. Alerting on business-date age exposes the gap.
I'm not sure a generic comparison can settle the queue-versus-workflow threshold for every marketplace. The decision depends on item count, provider rate limits, acceptable recovery time, and who will operate the extra control plane. A trial should kill the worker mid-page, redeliver the same run, and backfill an earlier date through the same entry point. Those tests reveal ownership far better than a polished scheduler UI.
Rollout grants deletion authority gradually
Deploy the decision ledger and unique constraints before enabling deletion. For several business dates, run in report-only mode: persist candidates, inclusion decisions, and exclusion reasons, but leave objects and records intact. Finance and compliance can then inspect samples involving unmatched payments, open disputes, and policy exceptions.
Next, permit deletion for one low-risk class of derived records. Keep uploads and reconciliation evidence protected until completed-date monitoring, retry tests, and tombstone queries show the expected behavior. Backfills must use the same dated entry point; an untracked maintenance script creates a second policy engine at exactly the moment controls matter most.
Rollback is deliberately small: stop granting new deletion decisions. Recovery is replaying the same run key after the underlying reconciliation or hold state is corrected. The scheduler remains replaceable because correctness lives in durable evidence, idempotent effects, and explicit business dates.
Choose the clock last.
Top comments (0)