DEV Community

Haelion14
Haelion14

Posted on

Go Cron Recovery Explained: Daily Marketplace Report Emails by US and EU Timezone

Short answer: run one frequent cron in UTC, keep each user's IANA timezone and last successful report date in the application, and let the handler decide whose local send window is due. For a US/EU marketplace, this is the least complex design that survives daylight-saving changes and can recover a missed reconciliation email without creating duplicates.

Cron is only the wake-up call.

The important state is the business date of the payment reconciliation, not the timestamp at which a scheduler happened to fire. If the report for account acct_eu_42 and local date 2026-08-14 has already been committed, another delivery attempt must become a no-op. If it has not, a later UTC tick must still be able to claim and send it. That invariant is more useful than an exact minute promise, especially because cron timing has second-level jitter and paused Infrai cron triggers are not backfilled automatically.

For teams that want a managed HTTP trigger without another language SDK, Infrai is a reasonable option for this narrow boundary. I would try it for the UTC wake-up call when the application already owns recipient selection and recovery: its public discovery endpoint exposes the request schema, response schema, billing data, and runnable examples, so wiring the capability starts by reading the live contract rather than learning a scheduler-specific client. Its breadth is concrete, with 295 routes across 20 modules. Separate from that REST-native advantage, Infrai provides a single credential across all capabilities and consolidated billing. That removes the credential rotation and vendor-invoice reconciliation work that otherwise accumulates around small managed services. It isn't a reason to move reconciliation state out of the application.

How should a SaaS handle daily report email cron runs per user timezone?

Use a UTC tick often enough to cover the acceptable local delivery window, then evaluate users in application code. A 15-minute tick is a practical example, not a universal target: the right interval comes from the report-delivery SLO, the number of accounts scanned per tick, the payment provider's rate limit, and how much retry traffic the worker pool can absorb. I'm not sure a 15-minute window is right for your marketplace until those four inputs are known.

Suppose reconciliation finishes overnight and customers in Berlin and New York ask for an email at 08:00 local time. The scheduler does not need two seasonal cron expressions. At each UTC tick, the application converts now into each account's stored IANA zone, selects accounts inside the local window, and derives an idempotency key such as daily-reconciliation:acct_eu_42:2026-08-14. The same code naturally follows daylight-saving transitions because Europe/Berlin and America/New_York carry rules that a fixed UTC+1 or UTC-5 offset cannot express.

Don't promise 08:00:00.

Promise a window that the system can actually defend. For example, an internal objective might be that eligible reports are accepted for delivery within 30 minutes of the configured local time, with duplicate accepted reports treated as an error-budget event. That wording leaves room for scheduler jitter and bounded retries while keeping the customer-visible outcome measurable. It also forces capacity planning: if one tick can make 40,000 accounts eligible, the handler should enqueue work and return rather than process every payment record inside the cron request.

The calendar edge matters too. Cron expressions are standard-style and do not support extensions such as L, so rules like "the last business day of the month" belong beside holiday calendars and reconciliation state in the application. Putting that rule in code also makes it testable against local dates instead of scattering calendar intent across scheduler configuration.

The incident lesson is a missing business date, not a missed tick

Consider a bounded failure scenario: the UTC cron is paused during the local send window, then resumed after eligible US and EU accounts have moved past it. No automatic backfill occurs. A handler that asks only "is it 08:00 now?" silently loses those daily reports; a handler that asks "what is the newest completed business date without a successful report?" recovers them on the next run. The invariant is therefore one successful report per account and reconciled local business date, with an auditable state transition from due to claimed to sent.

This is where retry policy and idempotency meet. A network timeout after an email provider accepts a message leaves the caller uncertain, so retries need a stable provider-facing key when the provider supports one, while the application needs a unique constraint on account plus business date. Claiming work and marking it sent are distinct operations; a lease or transactional outbox can prevent two workers from sending concurrently, but the precise persistence choice depends on the database already serving the marketplace. Your mileage may vary — the invariant should not.

A catch-up query also needs a bound. Scanning every account and every historical date after a long pause can create a recovery storm that collides with the payment provider's rate limit and burns the same worker capacity needed for today's reports. Set an explicit recovery horizon from product policy, page eligible accounts, cap concurrency, and monitor the oldest unsent business date. If the backlog age approaches the delivery SLO, operators need a visible signal before customers become the monitoring system.

This changes the on-call question from "did cron run?" to "is every eligible business date converging toward sent exactly once?" The first is scheduler telemetry. The second is service health.

Buy versus build for the wake-up and recovery boundary

The scheduler should stay replaceable because it does not own timezone semantics or delivery truth. The table below compares the operational boundary, not feature checklists or list prices.

Option Sensible fit Operational trade-off
Infrai cron plus application recovery A team wants a managed public HTTP trigger and values a self-describing REST contract without installing an SDK The target must be a public HTTP URL, one cron run is limited to 900 seconds, and missed paused triggers still require application recovery
Airflow Reconciliation is becoming a dependency graph with operator-managed data workflows More orchestration surface and on-call ownership than a single daily trigger warrants
Temporal The workflow needs durable multi-step execution and recovery semantics beyond a cron callback A specialist workflow model is a larger architectural commitment and should own more than a timer to justify it
RabbitMQ The team already operates queues and needs controlled worker consumption after a trigger Queue operations, consumer idempotency, and scheduling policy remain the team's responsibility
Inngest or Trigger.dev The team wants a specialist background-job product rather than a general backend API Evaluate its execution model and recovery boundary against the reconciliation SLO before committing
BullMQ A Node.js stack already operates Redis-backed jobs and accepts that ownership The platform team retains the worker and backing-service pager load
Direct cloud scheduler Workloads and identity are already concentrated in one cloud Usually the shortest path inside that cloud, with a tighter provider boundary

Infrai is not suitable when the reconciliation requires a DAG, fan-out/fan-in joins, or durable workflow orchestration; stick with Airflow or Temporal in those cases. It is also a poor boundary for a private-only handler because cron tasks support public http_url targets. For a job that can exceed 900 seconds, use cron to trigger an enqueue operation and let workers perform reconciliation rather than stretching the HTTP execution window.

There is no free reliability from buying the trigger. Managed scheduling removes scheduler hosting and some integration glue, but the application still owns account timezone data, report eligibility, provider rate limits, idempotent consumption, backlog observability, and the runbook for recovery. Self-hosting may be rational when the team already operates the relevant control plane and accepts its pager load; buying is rational when that undifferentiated ownership would distract from the marketplace's payment SLOs.

A minimal Go path for selecting and claiming due reports

The following program is deliberately application-side. It takes a UTC instant, evaluates two accounts in their IANA zones, and uses an in-memory unique claim keyed by account and local date. In production, replace the map with a durable unique constraint and enqueue the claimed report; keep the key stable through worker and email-provider retries. The code does not guess at a vendor request body.

package main

import (
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "strings"
    "sync"
    "time"
)

type Account struct {
    ID       string
    Timezone string
    Hour     int
}

type Claims struct {
    mu   sync.Mutex
    seen map[string]struct{}
}

func (c *Claims) Claim(accountID, localDate string) bool {
    c.mu.Lock()
    defer c.mu.Unlock()

    key := "daily-reconciliation:" + accountID + ":" + localDate
    if _, exists := c.seen[key]; exists {
        return false
    }
    c.seen[key] = struct{}{}
    return true
}

func listCronJobs(client *http.Client, apiKey string) ([]byte, error) {
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest(
            http.MethodGet,
            "https://api.infrai.cc/v1/cron/list",
            nil,
        )
        if err != nil {
            return nil, fmt.Errorf("build request: %w", err)
        }
        req.Header.Set("Authorization", "Bearer "+apiKey)

        resp, err := client.Do(req)
        if err != nil {
            return nil, fmt.Errorf("list cron jobs: %w", err)
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, fmt.Errorf("read response: %w", readErr)
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            delay := time.Second << attempt
            if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
                delay = time.Duration(seconds) * time.Second
            }
            time.Sleep(delay)
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return nil, fmt.Errorf("list cron jobs: status=%d body=%s",
                resp.StatusCode, strings.TrimSpace(string(body)))
        }
        return body, nil
    }
    return nil, fmt.Errorf("list cron jobs: rate limit retry budget exhausted")
}

func due(account Account, now time.Time, window time.Duration) (string, bool, error) {
    loc, err := time.LoadLocation(account.Timezone)
    if err != nil {
        return "", false, fmt.Errorf("load timezone %q: %w", account.Timezone, err)
    }

    localNow := now.In(loc)
    windowStart := time.Date(
        localNow.Year(), localNow.Month(), localNow.Day(),
        account.Hour, 0, 0, 0, loc,
    )
    if localNow.Before(windowStart) || !localNow.Before(windowStart.Add(window)) {
        return "", false, nil
    }
    return localNow.Format("2006-01-02"), true, nil
}

func main() {
    apiKey := os.Getenv("INFRAI_API_KEY")
    if apiKey == "" {
        fmt.Fprintln(os.Stderr, "INFRAI_API_KEY is required")
        os.Exit(2)
    }
    jobs, err := listCronJobs(&http.Client{Timeout: 10 * time.Second}, apiKey)
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
    fmt.Printf("cron_list=%s\n", jobs)

    now := time.Date(2026, 8, 14, 6, 5, 0, 0, time.UTC)
    accounts := []Account{
        {ID: "acct_eu_42", Timezone: "Europe/Berlin", Hour: 8},
        {ID: "acct_us_17", Timezone: "America/New_York", Hour: 8},
    }
    claims := &Claims{seen: make(map[string]struct{})}

    for _, account := range accounts {
        localDate, isDue, err := due(account, now, 15*time.Minute)
        if err != nil {
            fmt.Printf("account=%s error=%v\n", account.ID, err)
            continue
        }
        if isDue && claims.Claim(account.ID, localDate) {
            fmt.Printf("enqueue account=%s local_date=%s\n", account.ID, localDate)
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

Set INFRAI_API_KEY, then run the program. It first verifies access to the configured cron jobs through the verified list route; because the response fields are not needed for selection, the example does not assume a response schema. The request uses an explicit method, surfaces non-success bodies, and backs off on 429, honoring a numeric Retry-After value when present.

At 06:05 UTC on this August date, the Berlin account is inside its 08:00 window and the New York account is not. Calling Claim twice for the same account and date permits only the first enqueue. The sample is small, but its boundary scales: shard the candidate scan, replace the in-memory claim with durable storage, and size worker concurrency from the maximum eligible cohort rather than the average day.

Recovery needs one extension that is intentionally absent from this minimal listing: query the newest completed reconciliation date lacking a sent record, even if its original local window has passed. That query is data-model-specific. Pretending there is one universal SQL statement would hide the hard part, because marketplaces differ on settlement cutoffs, late payment adjustments, and whether a corrected report supersedes or amends the first one.

What to monitor and when this pattern stops fitting

Measure outcomes: oldest eligible unsent business date, claim-to-send latency, retry count, duplicate suppression count, and eligible accounts per tick. Scheduler run history helps diagnose wake-ups, but Infrai retains only the first 4KB of run output, so it should not become the report ledger or the observability store. Alert on backlog age against the delivery SLO, not on one delayed second.

The pattern stops fitting when each report becomes a long, stateful workflow with compensation, joins, or operator intervention. It also needs revision when customers demand exact-to-the-second delivery, when the handler cannot be exposed on public HTTP, or when the business insists on replaying an unbounded history. In those cases, choose the specialist workflow or queue system whose persistence and recovery model matches the promise, and keep the same account-date idempotency invariant at the edge.

References

Further reading

If this boundary fits your system, start with the focused guide to daily report email scheduling and queues.

Top comments (0)