DEV Community

FrostY45
FrostY45

Posted on

Daily Payment Reconciliation Email: Express Webhooks, Cron Triggers, and Recovery

Short answer: for a customer-support team sending one daily payment reconciliation email, use the cron trigger only to request work from a public HTTPS webhook. Let the application record one operation for the business date, enqueue the work durably, and let a worker generate the report and send messages. A timer inside a Node.js process is acceptable only when that process has one reliable owner and its deployment lifecycle can safely own the schedule.

The scheduler is a signal, not the system of record.

That distinction keeps latency and cost in their proper place. The faster trigger is not automatically the cheaper or safer design if a missed date requires a manual investigation, or if a duplicate trigger sends the same reconciliation twice. The real unit to protect is one accepted report for one reporting date, with evidence for every later step.

What should a Node.js Express webhook do when a cron trigger starts a daily report email?

The Express route should be a small, authenticated admission boundary. It should validate the request, derive a stable operation key such as payment-reconciliation:2026-08-10, create that operation exactly once, and put a compact job on a durable queue. It should not fetch the payment provider, render a large attachment, or send email while the HTTP request is open.

This is where an idempotency reflex pays for itself. The trigger can be retried. A network timeout can hide a successful response from the caller. A deployment can happen just as the request arrives. Those are ordinary scheduling conditions, not exceptional ones. A unique constraint on the operation key makes a second request resolve to the existing operation instead of creating another batch.

The key should represent the business date, not the instant at which the trigger happened. Store the timezone and the intended reporting window with the operation. Otherwise a daylight-saving change or a late trigger can quietly produce a report for the wrong period while the scheduler still says the run succeeded.

The public endpoint also needs ordinary HTTP controls: HTTPS, a secret that can be rotated, a small body limit, method checking, structured logs, and a response that distinguishes accepted work from invalid input. Do not put credentials in a query string. Do not treat a 200 response as proof that an email reached a mailbox; it only proves that this boundary accepted something.

The failure mode is a missing business date, not a missed timer

An operations dashboard usually tells you that a request ran. Support needs a different answer: did the reconciliation for the expected date reach a final state, and can we prove which customer records it covered?

Create a ledger row before the worker begins. It should contain the operation key, reporting window, status, attempt count, and timestamps. Keep recipient-level delivery state separately when one report fans out to many support recipients. A queue message can carry the operation key and date; the rendered report belongs in application storage, where it can be addressed and audited.

Duplicates are normal.

At-least-once delivery means the worker may see the same operation more than once. Before each externally visible action, it should claim a unique work item and record the result. If a retry finds a completed item, it exits without sending again. If the email provider accepts a message but the process loses its connection before recording that fact, exact once-only delivery cannot be inferred from the worker alone; use a provider-supported idempotency facility when available, or make the ambiguity visible for review.

I write the incident check in terms of a date and a key, never “the 02:00 run.” Suppose the webhook returns 202, the worker creates the report, and the process is killed after the email request leaves the host but before the delivery row is committed. A retry now has three facts to reconcile: the ledger says the work is in progress, the provider may have accepted the message, and the scheduler has no knowledge of either application state. The worker must not guess from a missing local row. It should retain the operation as ambiguous, query whatever delivery evidence the provider exposes, and have the runbook decide whether to mark the existing message complete or send a new message with an explicit review trail. When the same date is submitted again, the unique operation key still prevents a second report batch; recipient-level reconciliation handles the harder uncertainty. I've found that naming this state before an incident makes the discussion shorter, because “retry” no longer quietly means “send another email.”

Rate limiting is part of this path too. When a dependency returns HTTP 429, back off and honor Retry-After when present. Repeated immediate retries turn a temporary limit into a wider incident. Retain the attempt and response metadata against the operation, while keeping secrets and payment data out of logs.

A small Go admission handler makes the contract testable

The production endpoint may be Node.js and Express, but the contract is independent of the web framework. This Go example shows the boundary without pretending that an in-memory map is durable. The map and channel are useful for a local test; replace them with a database transaction and a durable queue before enabling a real schedule.

package main

import (
    "crypto/subtle"
    "encoding/json"
    "log"
    "net/http"
    "os"
    "sync"
    "time"
)

type reportJob struct {
    Key  string `json:"key"`
    Date string `json:"date"`
}

var (
    claimed = sync.Map{}
    jobs    = make(chan reportJob, 32)
)

func main() {
    secret := os.Getenv("CRON_WEBHOOK_TOKEN")
    if secret == "" {
        log.Fatal("CRON_WEBHOOK_TOKEN is required")
    }

    http.HandleFunc("/jobs/payment-reconciliation", func(w http.ResponseWriter, r *http.Request) {
        if r.Method != http.MethodPost {
            http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
            return
        }
        provided := r.Header.Get("X-Cron-Token")
        if subtle.ConstantTimeCompare([]byte(provided), []byte(secret)) != 1 {
            http.Error(w, "unauthorized", http.StatusUnauthorized)
            return
        }

        var input struct {
            Date string `json:"date"`
        }
        decoder := json.NewDecoder(http.MaxBytesReader(w, r.Body, 1024))
        decoder.DisallowUnknownFields()
        if err := decoder.Decode(&input); err != nil {
            http.Error(w, "invalid request", http.StatusBadRequest)
            return
        }
        if _, err := time.Parse("2006-01-02", input.Date); err != nil {
            http.Error(w, "date must be YYYY-MM-DD", http.StatusBadRequest)
            return
        }

        job := reportJob{Key: "payment-reconciliation:" + input.Date, Date: input.Date}
        if _, exists := claimed.LoadOrStore(job.Key, struct{}{}); exists {
            writeJSON(w, http.StatusOK, map[string]string{
                "status": "already_queued",
                "key":    job.Key,
            })
            return
        }

        select {
        case jobs <- job:
            writeJSON(w, http.StatusAccepted, map[string]string{
                "status": "queued",
                "key":    job.Key,
            })
        default:
            claimed.Delete(job.Key)
            http.Error(w, "capacity exhausted", http.StatusTooManyRequests)
        }
    })

    log.Fatal(http.ListenAndServe(":8080", nil))
}

func writeJSON(w http.ResponseWriter, status int, value any) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(status)
    if err := json.NewEncoder(w).Encode(value); err != nil {
        log.Printf("encode response: %v", err)
    }
}
Enter fullscreen mode Exit fullscreen mode

The important production change is transactional storage. Insert the unique operation row and its outbox or queue record together, then publish from the committed record. A process that marks a row claimed and crashes before enqueueing can otherwise create a false success. The outbox pattern gives the operator something concrete to replay without issuing a second report.

The queue is a handoff, not an audit ledger. It should hold a small identifier, not a rendered report or payment payload. Retention, acknowledgement, and dead-letter behavior belong in the queue policy; the application ledger must remain the place where a reporting date can be investigated after the message is gone.

How do you compare latency, cost, and recovery for a public webhook setup?

Start with the recovery promise, then measure latency and cost against it. For a report that is useful any time before the support shift begins, a managed trigger may be enough. If the report must start within a narrow window, or if every missed date must be replayed automatically, the schedule is part of a workflow and needs durable state around that promise.

Design Latency and cost profile Recovery boundary
Managed cron to an HTTPS webhook Low application overhead; trigger latency is usually sufficient for a daily report The application must detect and approve missing dates
Timer in one Node.js process Little additional infrastructure; latency can be direct Process restarts, replicas, and deployments own the failure mode
Queue-backed worker with an external scheduler Extra queue operation and worker responsibility; work can continue after the request Ledger and retry policy govern recovery
Workflow orchestrator Higher operational and cognitive cost; useful for dependent stages Workflow state can represent retries, approvals, and catch-up

The catch is that “easiest setup” usually describes the first successful run, not the first missed one. A public webhook reduces reachability friction, but it also creates an exposed admission surface and a new credential to rotate. A process-local timer avoids that endpoint, but replicas can produce duplicate triggers unless one owner is enforced. A workflow system buys stronger recovery semantics at the cost of another control plane — and that control plane needs an owner, alerts, and a patching plan.

Your mileage may vary. The decisive inputs are the report deadline, the number of replicas, the acceptable operator action after a missed date, and the evidence required by support or finance. Do a small load test against the payment provider and email path before choosing a tighter schedule; I’m not sure any generic latency figure would survive different provider limits and report sizes.

Verify the path, then roll it back without a second send route

Before enabling the production schedule, submit one authenticated test date. Confirm one ledger row, one queue item, one completed worker operation, and the expected recipient records. Submit the identical date again. The second request may be acknowledged as already accepted, but counts must not increase.

Test the negative cases deliberately: an invalid secret, an invalid date, an oversized body, a dependency 429, a worker restart, and a missing trigger. Alert on a missing operation key after the expected window and on work that remains queued beyond the service target. These alerts are more useful than an alarm that only says a timer did not fire.

For a paused schedule, stop new triggers first. Let already claimed work reach a recorded state, compare the ledger with the required business dates, and approve each absent date explicitly. Resubmit through the same idempotent webhook. There should be no operator-only send path, because emergency paths are where duplicate protection tends to disappear.

Rollback is a traffic decision: stop new admissions, preserve the ledger, and let or cancel queued work according to its recorded state. If the report approaches the scheduler's execution limit, keep generation behind the queue from the beginning. If the requirement grows into strict catch-up, approvals, dependent stages, or a fan-out followed by a join, move schedule ownership to a workflow design while retaining the operation key and audit ledger.

That is the runbook I would want during a page: one date, one key, one observable state transition. The trigger can change later. The evidence should not.

Further reading

Top comments (0)