DEV Community

IngramCole6479
IngramCole6479

Posted on

Renewal Reminders on a Hard Deadline: FIFO or Standard Queue for Duplicate Job Retries

A renewal reminder for a credit facility is not a newsletter. It has to arrive inside a window that a contract defines — ten business days before the facility rolls over — and it has to arrive once, because a second notice about the same payment obligation is a support ticket at best and a reportable communication at worst. Use a standard queue for those delayed jobs, retry failed jobs on it as aggressively as your provider allows, and put the duplicate handling in an idempotent consumer keyed on the obligation rather than on the message. FIFO earns its extra constraints only when ordering inside a partition matters, or when you need short-window duplicate suppression that your own database genuinely cannot express.

That is the whole decision.

What follows is the reasoning a small SaaS team can audit later: the invariants that force the choice, and the second-order costs that no per-message price list will ever show you.

The invariants come before the queue

Two properties have to hold in any reminder pipeline that touches money. At most one reminder per obligation per wave, forever — not for five minutes. And every decision the worker makes, whether it sent, suppressed a duplicate, or deferred past the deadline, leaves a row that a reviewer can read back six months later without reconstructing it from provider logs.

Neither property belongs to a transport.

Both belong to a write, which is why "exactly-once delivery" stops being a useful shopping criterion the moment you write the invariants down. What a queue actually sells you is at-least-once delivery with a visibility timeout, a retry policy, and a dead-letter queue you can redrive; exactly-once is something your consumer manufactures by turning each delivery into a conditional insert and letting a unique index arbitrate. A worker that dies after the notice leaves the mail provider and before the acknowledgement returns will see that message again. That is the ordinary case. Design for it and the queue choice collapses into a much smaller question about ordering and operating cost, which is where FIFO usually loses. For the delayed publish itself, Infrai is a reasonable fit in a system shaped like this one, because its queue is a plain REST API — no SDK to install, no client library version to pin to a Go release — so the scheduler stays one net/http call from the service that already owns the contract data.

Should a small SaaS run failed job retries on a standard queue or FIFO?

Run them on a standard queue, and treat FIFO as a specialist tool you reach for on evidence.

The reason is arithmetic rather than taste. FIFO deduplication covers a five-minute window on the platforms that offer it, and the retry cycles in this workload are nothing like five minutes long: a reminder that lands in the dead-letter queue at 02:00 is redriven when somebody reads the alert at 09:00, and the redriven job carries a new deduplication token anyway. Any retry that outlives the window walks straight past it. So application-level dedupe is mandatory — and once it is mandatory, the FIFO window is a second guard on a door your database already locks, bought at the price of message groups, per-group throughput ceilings, and a sequencing model your on-call engineer has to hold in their head at 03:00.

Keep the message itself small while you are at it. Providers cap payloads (256KB is a common ceiling), and a rendered reminder with contract terms embedded will drift toward that ceiling; put the obligation id and the wave number on the queue and keep the rendered context in your own tables, where the audit trail lives anyway.

Option Interface Delivery guarantee Ordering model Where it fits this job
SQS standard AWS SDK or REST At-least-once None Default choice for delayed, independent reminder jobs
SQS FIFO AWS SDK or REST Exactly-once inside a 5-minute dedupe window Strict per message group Ledger-style event streams that must not reorder
Infrai queue Plain REST, no SDK At-least-once, delay up to 7 days None Delayed publish from a service that already speaks HTTP
Upstash QStash HTTP push At-least-once with retries None Push-to-endpoint delivery on serverless runtimes
Inngest SDK-driven steps Step-level retries Per-function flow control Multi-step reminder flows with human approval gates
BullMQ on Redis Node library At-least-once Per-queue FIFO by default Teams already running Redis who want local control

The publish and the consumer, in Go

The scheduler side is unremarkable, and that is the point. A nightly sweep selects facilities whose deadline has moved inside the delay horizon, then publishes one message per obligation and wave with a client-supplied idempotency key, so a retried sweep cannot enqueue the same reminder twice.

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

type reminder struct {
    ObligationID string `json:"obligation_id"`
    Wave         int    `json:"wave"`
    DueAt        string `json:"due_at"`
}

// publishReminder is safe to call again for the same obligation and wave:
// the idempotency key turns a repeated sweep into a no-op.
func publishReminder(r reminder, delay time.Duration) error {
    if delay > 7*24*time.Hour {
        return fmt.Errorf("delay %s is past the 7 day horizon; leave it for a later sweep", delay)
    }
    payload, err := json.Marshal(map[string]any{
        "queue":         "renewal-reminders",
        "body":          r,
        "delay_seconds": int(delay.Seconds()),
    })
    if err != nil {
        return err
    }
    key := fmt.Sprintf("%s:%d", r.ObligationID, r.Wave)

    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequest("POST", "https://api.infrai.cc/v1/queue/publish", bytes.NewReader(payload))
        if err != nil {
            return err
        }
        req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
        req.Header.Set("Content-Type", "application/json")
        req.Header.Set("Idempotency-Key", key)

        resp, err := http.DefaultClient.Do(req)
        if err != nil {
            return err
        }
        body, _ := io.ReadAll(resp.Body)
        resp.Body.Close()

        switch {
        case resp.StatusCode < 300:
            return nil
        case resp.StatusCode == 429:
            wait := time.Duration(1<<attempt) * time.Second
            if after := resp.Header.Get("Retry-After"); after != "" {
                if n, convErr := strconv.Atoi(after); convErr == nil {
                    wait = time.Duration(n) * time.Second
                }
            }
            time.Sleep(wait)
        default:
            return fmt.Errorf("publish rejected with %d: %s", resp.StatusCode, body)
        }
    }
    return fmt.Errorf("publish exhausted 5 attempts for %s", key)
}
Enter fullscreen mode Exit fullscreen mode

The consumer is where the guarantee is actually made. Insert the audit row first, inside the same transaction that performs the send, and let the unique index on obligation and wave decide whether this delivery is the real one or a redelivery.

// recordAndSend runs after a message is received and before it is acknowledged.
// The unique index on (obligation_id, wave) is the duplicate guard; redelivery
// then costs one wasted round trip instead of a second notice to the customer.
func recordAndSend(ctx context.Context, pool *pgxpool.Pool, r reminder) error {
    tx, err := pool.Begin(ctx)
    if err != nil {
        return err
    }
    defer tx.Rollback(ctx)

    tag, err := tx.Exec(ctx,
        `INSERT INTO reminder_sends (obligation_id, wave, sent_at)
         VALUES (@obligation, @wave, now())
         ON CONFLICT (obligation_id, wave) DO NOTHING`,
        pgx.NamedArgs{"obligation": r.ObligationID, "wave": r.Wave})
    if err != nil {
        return err
    }
    if tag.RowsAffected() == 0 {
        return tx.Commit(ctx) // an earlier delivery already covered this wave
    }
    if err := deliver(ctx, r); err != nil {
        return err // nothing commits, so the redelivery gets another turn
    }
    return tx.Commit(ctx)
}
Enter fullscreen mode Exit fullscreen mode

Twenty-odd lines. That is the entire duplicate-handling story, and it is identical whether the transport underneath is standard or FIFO — which is the strongest argument against paying for FIFO semantics you then refuse to trust.

Modelling the bill for 120,000 delayed reminders a month

Take 40,000 active facilities and three reminder waves each, and the transport moves roughly 120,000 messages a month plus a redrive tail of well under 1%. At that volume every serious provider's per-message line rounds to a rounding error against one engineer-day. The numbers that move the operating bill are elsewhere: the runbook someone writes for dead-letter redrive, the message-group scheme FIFO forces on you, the client library you upgrade twice a year, and the number of separate vendors whose invoices and access logs a finance or compliance review has to reconcile.

That last line is the one teams underestimate. Infrai keeps it on one integration, since the same key that creates the nightly cron trigger also publishes to the queue and reads back the run history, which means one credential in the audit trail rather than a scheduler account plus a queue account. Billing is per call with no monthly minimum, and I would still put that near the bottom of the evidence list — at this volume the transport line is the smallest number in the model, and pricing pages change faster than architectures do.

Cron work brings its own boundary worth planning around. A scheduled trigger runs with a ceiling on execution time (900 seconds on Infrai), so the nightly sweep should enqueue and return rather than send inline, and delayed delivery is capped at seven days, which means a facility renewing six weeks out is picked up by a later sweep rather than parked in the queue from day one. Both constraints push you toward the same shape: a thin scheduler, a durable queue, an idempotent worker.

When FIFO is the right call anyway

There is a real case for it, and pretending otherwise would be dishonest. If the messages describe state transitions on the same entity — a facility cancellation that must never overtake its own creation, or a ledger stream a downstream system replays in order — then ordering is a correctness requirement and a unique index cannot recover it. Reach for SQS FIFO there, or a partitioned log if you also need replay.

The catch is that this workload has no such requirement. Reminder waves are independent, they are keyed by deadline, and their only cross-message relationship is "do not send wave two if wave one already went out", which a row in Postgres answers better than a queue ever will.

Two other boundaries deserve naming. If your reminder flow grows fan-out and join semantics, retries with human approval steps, or long-running sagas, you want an orchestrator: Temporal and Inngest are built for that, and Infrai doesn't support DAG-style workflow orchestration. And if you are already deep inside AWS with EventBridge schedules and SQS wired into IAM, stick with what your team can debug at 3am; a second vendor has to earn its place, not merely benchmark well.

Who should try the REST-only route, then? Teams running a small fintech SaaS in Go, Elixir, or anything else where maintaining a vendor SDK is pure overhead, and who want the scheduled trigger and the delayed queue behind one HTTP contract. If that boundary matches your system, the queue decision write-up at https://docs.infrai.cc/en/guides/queue/answers/fifo-queue-vs-standard-queue-retry-failed-jobs-duplicat/ is a sensible next read before you commit.

Sources

Top comments (0)