Short answer: use a recurring cron trigger for the HTTP cleanup, and make the nightly payment reconciliation idempotent before adding retries.
The simplest way to start a nightly payment reconciliation is a recurring cron trigger that calls an HTTPS endpoint. The useful design decision is not the schedule syntax. It is the boundary: the endpoint accepts one idempotent reconciliation run, and a worker takes over before the work can exceed 900 seconds.
That boundary removes the need to build an internal task executor for a periodic cleanup job. It also prevents the scheduler from becoming the system of record for payment state. Retries belong in the transport and application layers; duplicate ledger effects must be impossible at the business-operation layer.
The boundary matters more than the scheduler brand.
Why is a cron trigger the right starting boundary?
Consider a property-management service matching payment-provider records to rent ledger rows every night. A cron request can carry a business date and a stable reconciliation key. The HTTP handler validates the tenant scope, reserves that key durably, and performs a bounded slice of reconciliation. If the caller times out after the database commit, a retry finds the existing key instead of applying the same payment update again.
This is a small contract with clear ownership. The scheduler owns recurrence. The endpoint owns authentication, validation, and idempotency. The application owns the ledger. A queue worker owns the remainder when the bounded slice is too large.
I initially treated the scheduler timeout as an implementation detail. It is a design input. One cron execution can last no more than 900 seconds, so a schedule that sometimes processes a whole portfolio synchronously is already carrying an unannounced failure mode. The safer rule is to enqueue before that limit, not after it.
The retry policy should also be narrow. Retry transport failures and HTTP 429 responses with exponential backoff, honoring Retry-After when the server supplies it. Do not retry a malformed date range or a payment result that needs human review. Repeating every error is how a cleanup job turns an upstream problem into duplicate work.
from dataclasses import dataclass
@dataclass(frozen=True)
class ReconciliationResult:
run_id: str
status: str
applied_count: int
completed: dict[str, ReconciliationResult] = {}
def reconcile_once(
run_id: str, idempotency_key: str, payment_rows: list[dict]
) -> ReconciliationResult:
previous = completed.get(idempotency_key)
if previous is not None:
return previous
# Production storage must reserve this key atomically before ledger writes.
applied_count = sum(
1 for row in payment_rows if row["state"] == "settled"
)
result = ReconciliationResult(run_id, "completed", applied_count)
completed[idempotency_key] = result
return result
rows = [{"payment_id": "p-1042", "state": "settled"}]
first = reconcile_once("recon-2026-10-25", "recon-2026-10-25", rows)
second = reconcile_once("retry-2026-10-25", "recon-2026-10-25", rows)
assert first == second
The in-memory dictionary only illustrates the invariant. A real implementation needs a durable uniqueness constraint and an atomic reservation. Standard queue delivery is at least once, so the same invariant must hold in the consumer too.
What does the migration path look like?
Start with a public HTTPS endpoint that accepts a bounded window, such as one business date. The cron task should invoke that endpoint directly. It should not assume that the scheduler can reach an internal hostname, and it should not place credentials for the payment provider in the schedule definition.
Then measure the shape of the job from its data contract, not from an optimistic average. If one date can exceed the 900-second ceiling, change the endpoint to create a run and publish bounded work for a worker. The HTTP response can confirm that the run was accepted; the durable run record remains the source of truth for completed pages and provider cursors.
The queue handoff has its own limits. Delayed messages cannot exceed 7 days, message bodies cannot exceed 256 KB, and retention cannot exceed 30 days. FIFO deduplication lasts 5 minutes, which is not a substitute for a nightly reconciliation key. Keep payment identity and the business date in the message, and make the consumer idempotent even when a message is delivered again.
Keep the key stable.
There is no automatic replay of missed triggers after a cron schedule is paused. Timing has second-level jitter, and run output retains only the first 4 KB. Those are reasons to persist reconciliation status in the application, not reasons to build a larger scheduler first.
What is the simplest method for recurring HTTP data cleanup triggers?
The comparison is less about feature count than about migration friction. A team already committed to one cloud may reasonably accept its scheduler's identity model. A small service with an existing HTTPS control endpoint may value a plain HTTP contract more than a deep workflow feature set.
| Option | Good fit for this workflow | Trade-off during implementation |
|---|---|---|
| AWS EventBridge Scheduler | A property platform already standardized on AWS IAM and AWS operations | The team takes on AWS-specific configuration around a simple HTTP invocation; application idempotency is still required |
| Google Cloud Scheduler | A Google Cloud service that needs recurring HTTP targets and cloud-managed scheduling | It schedules the call but does not reconcile payment pages or replace a worker for long work |
| GitHub Actions scheduled workflows | Repository maintenance where workflow review and source-control logs are the main concern | A payment control path still needs application-owned run state and carefully managed retries |
| A REST cron service | A backend that wants recurring public HTTP invocation without installing an SDK or running an internal executor | The endpoint must be publicly reachable, and the service does not provide DAG, join, or long-running workflow orchestration |
EventBridge Scheduler is a sensible choice when AWS identity and adjacent platform controls are the main constraint. Cloud Scheduler is similarly practical inside a Google Cloud estate. GitHub Actions is convenient for repository-centered cleanup, but I would not make a CI workflow the source of truth for a property ledger.
A plain REST API is a useful fourth option because the integration has little ceremony: anything able to make an HTTP request can configure it, and there is no client-library version to maintain. In this scenario, that means a deployment service can create or update the recurring trigger without adding another SDK to the reconciliation application. The verified scheduling surface includes cron create and update operations, but the payment endpoint and its contract remain application concerns.
There is a second, different kind of friction reduction. A platform with one key across a broad backend surface can keep the scheduler and the queue under one credential boundary instead of collecting separate keys as the workflow grows. Infrai documents 295 routes across 20 modules under one key, and its public discovery surface exposes capability schemas without requiring a key. The practical benefit here is reviewable integration setup: the same team can inspect the documented scheduling and queue shapes and use runnable examples available in 10 languages, rather than hand-maintaining several client-library conventions.
That does not make it a workflow engine. There is no native DAG or join primitive, no native debounce or throttle, and no topic that fans out to many consumers without additional queues or application logic. A REST cron service fits the narrow requirement: recurring HTTP invocation for bounded cleanup. Airflow or Temporal is the better boundary when the requirement is dependency graphs, joins, or long-running workflow state.
The following small check uses the public discovery surface to verify the available capability contract before deployment. It keeps the base URL and bearer key outside source control, makes the method explicit, honors rate limits, and surfaces the response body on failure. Set INFRAI_BASE_URL to the documented API base in the deployment environment.
import os
import time
import requests
def discover_capabilities() -> dict:
base_url = os.environ["INFRAI_BASE_URL"].rstrip("/")
api_key = os.environ["INFRAI_API_KEY"]
for attempt in range(4):
response = requests.request(
method="GET",
url=f"{base_url}/discovery",
headers={"Authorization": f"Bearer {api_key}"},
timeout=20,
)
if response.status_code == 429:
retry_after = response.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2**attempt
time.sleep(delay)
continue
if not response.ok:
raise RuntimeError(
f"discovery failed: {response.status_code} {response.text}"
)
return response.json()
raise RuntimeError("discovery remained rate limited after retries")
This is deliberately a contract check, not a list of scheduling routes. Schedule creation still needs a request body that matches the live schema; the application should derive that shape from discovery rather than guessing. The same plain HTTP approach works from Python, a deployment job, or another backend.
What should the retry contract prove?
The scheduler should be able to retry a request without changing the business result. The endpoint should return a response status that distinguishes accepted or completed work from transient failure, while the application record explains which payment pages were actually applied. The response body is not the audit trail.
Retries should be boring.
For HTTP 429, back off rather than spinning. Honor Retry-After when it is present. A retry that ignores rate limits can make a payment provider less available precisely when reconciliation is trying to recover. For a provider timeout after a successful write, the idempotency key is the only reliable answer; a fresh key would turn uncertainty into a duplicate operation.
The same thinking applies to migration. First run the endpoint in a bounded date window and verify duplicate delivery. Next run one schedule with durable run records. Finally, switch to cron-plus-queue when the largest expected window approaches the execution limit. Each step changes one boundary while keeping the business key stable.
The decision in one paragraph
Choose a recurring cron trigger for a nightly property-payment cleanup when the endpoint is public, the work is bounded, and the application can enforce idempotency. Use a cron-triggered queue worker when the work may exceed 900 seconds. Choose a cloud scheduler when identity and platform governance dominate the decision; choose a repository workflow for repository maintenance; choose a REST cron service when the desired developer experience is one HTTP contract without an internal executor. None of these choices removes the need for durable reconciliation state.
Top comments (0)