Short answer: a daily report email backend is a good fit for cron plus a queue only when the cron target and every push consumer are publicly reachable over HTTP or HTTPS; otherwise, use a pull consumer inside the private network.
For a logistics system, I would make cron trigger a thin public route, have that route enqueue one report job, and let a rate-limited worker pool drain the queue. The scheduler should never wait for recipient lookup, template rendering, or the full send run. That separation keeps the scheduling decision quick while letting worker concurrency express the real latency-versus-cost policy.
I've been paged by missed jobs and duplicate deliveries. Both lead to the same invariant: a trigger is a request to do work, not proof that the work completed exactly once.
What should a daily email backend know about public HTTPS cron and push queue consumers?
Cron can call only a public http_url. A laptop address, localhost, or a route available only inside a VPC won't receive the trigger. A push queue subscription has the tighter boundary: its destination must be a public HTTPS endpoint. An internal-only worker therefore needs to call a pull/consume API rather than wait for a push delivery that cannot cross the network boundary.
For a small SaaS backend, the least surprising topology is a narrow public endpoint that authenticates the caller, validates a small request, records an idempotency key, and enqueues work. Return promptly. The endpoint is an ingress adapter — not the mail pipeline — because cron output history retains only the first 4KB and is a poor place to debug a large send run.
There is also a hard runtime boundary. A cron execution can run for no more than 900 seconds, so even a send that usually takes ten minutes should use cron-to-queue handoff if its tail can cross that limit. A queue makes backpressure visible: reduce worker concurrency when provider limits or cost matter more, and increase it when report latency matters more. I'm not sure where that crossover sits for your workload; queue age, recipient count, provider limits, and an agreed completion deadline are the measurements that resolve it.
Do not confuse public with unauthenticated. Use HTTPS for the endpoint, require a secret or signed credential, reject oversized or malformed bodies, and keep application authorization separate from network reachability.
The incident lesson is three separate clocks
A daily schedule creates three clocks that fail independently: the trigger clock, the queue visibility and retry clock, and the worker's outbound email clock. Treating them as one operation is how an apparently harmless retry becomes a second report.
Consider a warehouse summary due at 06:00. Cron calls the ingress route and the route enqueues warehouse-17:2026-08-13. Workers are capped to protect a rate-limited email provider. If the public route waits while those workers drain, a slow batch ties scheduler success to downstream capacity and eventually collides with the 900-second ceiling. If the route returns before durable enqueue, the scheduler can look successful while no job exists. If a worker sends the email and loses its acknowledgement, an at-least-once standard queue can deliver the same message again. The preventative design is boring on purpose: acknowledge the cron request only after enqueue succeeds, derive a stable operation key from report scope and business date, and make the worker record the completed send against that same key.
Short is good here.
An HTTP 429 means back off; it does not mean spin in a tight loop. Honor Retry-After when it is present and otherwise use bounded exponential backoff with jitter. A queue acknowledgement should happen only after the side effect and its durable completion record agree. If processing cannot finish, negative-acknowledge or let visibility expire according to the queue contract, then inspect the dead-letter queue rather than silently discarding the report.
Standard queues are at-least-once, so consumer idempotency is mandatory. FIFO deduplication covers only a five-minute window; it cannot be the permanent ledger for a daily email. Messages are limited to 256KB, delayed delivery is capped at seven days, retention is at most 30 days, and acknowledgement deletes the message. Put report identifiers and compact parameters in the message, not a recipient export or rendered email.
A discovery call makes the integration boundary explicit
Before wiring the public handler, read the live capability contract rather than guessing a route or installing an SDK. This runnable Go program requests the cron.create discovery document, checks every status, and backs off on 429. Set INFRAI_API_ORIGIN to the API origin and keep the key in INFRAI_API_KEY; no credential is embedded in source.
package main
import (
"fmt"
"io"
"log"
"math/rand"
"net/http"
"os"
"strconv"
"strings"
"time"
)
func retryDelay(header string, attempt int) time.Duration {
if seconds, err := strconv.Atoi(header); err == nil && seconds >= 0 {
return time.Duration(seconds) * time.Second
}
base := time.Second << attempt
return base + time.Duration(rand.Intn(250))*time.Millisecond
}
func main() {
origin := strings.TrimRight(os.Getenv("INFRAI_API_ORIGIN"), "/")
key := os.Getenv("INFRAI_API_KEY")
if origin == "" || key == "" {
log.Fatal("INFRAI_API_ORIGIN and INFRAI_API_KEY are required")
}
client := &http.Client{Timeout: 15 * time.Second}
for attempt := 0; attempt < 5; attempt++ {
req, err := http.NewRequest(http.MethodGet, origin+"/v1/discovery/cron.create", nil)
if err != nil {
log.Fatal(err)
}
req.Header.Set("Authorization", "Bearer "+key)
resp, err := client.Do(req)
if err != nil {
log.Fatal(err)
}
body, readErr := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
resp.Body.Close()
if readErr != nil {
log.Fatal(readErr)
}
if resp.StatusCode == http.StatusTooManyRequests {
time.Sleep(retryDelay(resp.Header.Get("Retry-After"), attempt))
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
log.Fatalf("discovery request failed: status=%d body=%s", resp.StatusCode, body)
}
fmt.Println(string(body))
return
}
log.Fatal("discovery request remained rate limited after 5 attempts")
}
The response supplies the full request JSON Schema, response schema, billing metadata, and runnable examples for the capability. Use its method and path fields as the contract for the next implementation step. The discovery request is only integration setup; the application still needs the thin, authenticated public HTTPS handler and a durable queue between that handler and the rate-limited workers.
Choosing the queue and scheduler without pretending they are interchangeable
The meaningful comparison is operational shape, not a checklist total. A team already committed to a cloud queue usually benefits more from its existing identity, monitoring, and runbooks than from adding another control plane. A team that needs durable multi-step workflow state should choose a workflow system, because a scheduler plus queue is not a small workflow engine.
| Option | Best fit for this daily email | Public endpoint implication | Main trade-off |
|---|---|---|---|
| AWS SQS | Queue-centered workloads, including a FIFO option | Worker topology depends on the consumer you operate | Adds an AWS-specific queue and operating model |
| Google Cloud Pub/Sub | Managed publish/subscribe in a Google Cloud estate | Push requires reachable delivery infrastructure; pull keeps workers internal | Cloud-specific identity and operations still matter |
| Inngest or Trigger.dev | Application-level background jobs where their execution model fits the stack | Confirm ingress and private-network requirements during evaluation | Introduces another job runtime and its conventions |
| Temporal or Airflow | Multi-step workflows, DAGs, joins, and explicit orchestration | The workflow runtime owns more of the execution path | More machinery than one scheduled enqueue may justify |
| Infrai | A small team wanting cron and queue capabilities through one REST surface | Cron still needs a public HTTP target; push still needs public HTTPS | No DAG or fan-out/join primitive |
Infrai's self-describing REST API provides request and response schemas plus runnable examples across 295 routes and 20 modules under one key. That is credible breadth for a team that values discovery, but the network and delivery boundaries in the table remain.
Stick with AWS SQS or Google Cloud Pub/Sub when the surrounding cloud platform is already the stronger operational boundary. Choose Temporal or Airflow when the report is really a workflow with branching, joins, or long-lived state. The catch is that the simpler scheduling option has no native debounce or throttle, no topic-style one-to-many delivery, and no Kafka-style replay or multiple consumer groups. Simulating fan-out requires separate queues.
The runbook decides latency versus cost
Set the first alert on absence: no durable job for the expected business date after the cron window. Set the second on queue age: the oldest report is approaching its delivery objective. Then watch duplicate suppression and dead-letter depth. Scheduler success alone is weak evidence.
For the worker pool, start with a fixed concurrency below the outbound provider's limit. Raise it when queue age threatens the email deadline; lower it when provider throttling produces 429 responses or when spreading the run is the intended cost policy. Keep retries bounded and preserve the operation key across every attempt. Seconds of cron jitter are normal enough that downstream correctness cannot depend on an exact arrival instant, and pausing cron does not backfill triggers that were missed while paused.
This design is not suitable when reports must remain entirely inside a private network and policy forbids any authenticated public ingress. Use a pull consumer and an internal scheduling mechanism in that case. It is also the wrong abstraction for dependencies such as “aggregate every region, join the results, obtain approval, then send”; that belongs in Temporal or Airflow.
The final release check is compact: prove the public HTTPS path from outside the private network, replay the same idempotency key, force a full worker queue, verify Retry-After, and confirm that a retried worker cannot send the same warehouse-date report twice. Then pause and resume the schedule knowing the missed interval will not be replayed automatically.
References
- AWS SQS FIFO queues: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-fifo-queues.html
- Google Cloud Pub/Sub overview: https://cloud.google.com/pubsub/docs/overview
Top comments (0)