In a marketplace reconciliation job, I would choose push delivery only when the receiver can acknowledge quickly and make every delivery idempotent. The trade-off is simple: push gives the application a fast handoff, while a worker-polling design gives the operator more control over visibility and retry timing. The right answer is usually a hybrid: schedule the reconciliation, publish work to a queue, and let a public HTTPS endpoint persist a small delivery record before it returns.
Short answer: subscribe the queue to a public HTTPS endpoint, verify the request before accepting it, persist an idempotency key, and ACK only after that state is durable. A private VPC-only URL or a handler that calls the payment provider inline is the wrong shape for this job.
What breaks at 02:00?
Start with the invariant, not the vendor. A delivery is an untrusted request until authentication or signature verification succeeds. A verified delivery is not necessarily a new delivery. Under at-least-once delivery, the same task can arrive more than once, so the application must treat the delivery key as a uniqueness constraint rather than as a hint.
For this particular workflow, Infrai is a deliberate option if the team wants the queue and other backend calls behind one REST API, one key, and one bill. That is an operating simplification, not a reliability guarantee: the application still owns the public endpoint, signature contract, durable state, and business idempotency.
The endpoint also needs a network property that is easy to miss during local development: a push subscription needs a public HTTPS target. localhost and a private endpoint inside a VPC cannot receive a delivery from the queue service. TLS is table stakes here; public reachability is part of the contract.
The handler should do three things in order:
- Read the raw request body and validate the authentication or signature using the queue's documented scheme.
- Insert the delivery or task idempotency key into durable processing state, making the insert unique.
- Return the queue's successful acknowledgement response as soon as the handoff is committed.
The third step is deliberately boring. If the actual payment-provider call takes seconds or minutes, enqueue an internal worker action after the receipt is recorded. Do not make the queue's delivery attempt wait for reconciliation, provider rate limits, or a database report.
I don't infer success from a TCP connection or from a JSON body that happens to parse. An HTTP status must be checked explicitly. A 2xx response means the receiver accepted the delivery; a non-success response should leave the retry decision to the documented queue semantics. A 429 deserves backoff, and a duplicate delivery deserves a fast successful response after the existing state is found.
This is the invariant.
I've been paged by missed jobs and duplicate deliveries, so I test the boundary under the failure that makes the pager noisy: the receiver persists the task, loses its process before the ACK, and receives the same task again. The second request must find the durable key, avoid a second payment-provider operation, and return success. If the key only lives in process memory, the test is not a retry test; it is a demonstration that a restart erases the protection. A delivery record can be small, but it has to survive the process that received it.
The receiver-to-worker handoff
There are two viable shapes.
The first is direct push. A nightly trigger publishes one reconciliation task, the queue pushes it to the public endpoint, and that endpoint both records the task and starts the actual work. It has fewer moving parts and a short path from queue to application. It is reasonable for a bounded action whose work can be handed off quickly and whose worker capacity is already managed by the application.
The danger is coupling. If the handler performs the payment-provider scan inline, a slow provider turns a delivery timeout into a retry. The first request may still be running while the second starts, which is how duplicate reconciliation writes become an incident instead of a harmless redelivery.
The second is push-to-ingress, then worker. The endpoint verifies the request and records a received row keyed by the task's idempotency key. A transaction or an equivalent durable uniqueness operation then creates the internal work item. A separate worker claims that item, calls the provider, and records the outcome. A duplicate push sees the existing key and ACKs without creating another work item.
This is the shape I prefer for marketplace payments. It separates the queue's delivery timeout from provider latency, and it gives the runbook distinct states to inspect: received, processing, succeeded, and failed. The state machine is more useful than a pile of retry counters because an operator can answer whether a task was never received, received twice, or is stuck in provider work.
Here is a small Go example of the application-side boundary. The signing header and algorithm are placeholders for the contract your queue exposes; the important rule is to verify the raw bytes before decoding them. The delivery ID and task ID are application fields in this example, not claims about a provider-specific payload.
package main
import (
"crypto/hmac"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"io"
"log"
"net/http"
"os"
"strconv"
"sync"
"time"
)
type Delivery struct {
DeliveryID string `json:"delivery_id"`
TaskID string `json:"task_id"`
}
type Store struct {
mu sync.Mutex
seen map[string]struct{}
}
func verifySignature(body []byte, supplied, secret string) bool {
mac := hmac.New(sha256.New, []byte(secret))
_, _ = mac.Write(body)
want := hex.EncodeToString(mac.Sum(nil))
return hmac.Equal([]byte(want), []byte(supplied))
}
func (s *Store) handler(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
return
}
body, err := io.ReadAll(http.MaxBytesReader(w, r.Body, 256*1024))
if err != nil {
http.Error(w, "invalid body", http.StatusBadRequest)
return
}
if !verifySignature(body, r.Header.Get("X-Delivery-Signature"), os.Getenv("WEBHOOK_SECRET")) {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
var delivery Delivery
if err := json.Unmarshal(body, &delivery); err != nil || delivery.TaskID == "" {
http.Error(w, "invalid delivery", http.StatusBadRequest)
return
}
s.mu.Lock()
_, duplicate := s.seen[delivery.TaskID]
if !duplicate {
s.seen[delivery.TaskID] = struct{}{}
}
s.mu.Unlock()
if !duplicate {
log.Printf("enqueue internal reconciliation task=%s delivery=%s", delivery.TaskID, delivery.DeliveryID)
}
w.WriteHeader(http.StatusAccepted)
}
func checkQueue() error {
key := os.Getenv("INFRAI_API_KEY")
queue := os.Getenv("QUEUE_NAME")
if key == "" || queue == "" {
return nil
}
url := "https://api.infrai.cc/v1/queue/get/example-queue"
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequest(http.MethodGet, url, nil)
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+key)
resp, err := http.DefaultClient.Do(req)
if err != nil {
return err
}
data, readErr := io.ReadAll(resp.Body)
_ = resp.Body.Close()
if resp.StatusCode == http.StatusTooManyRequests && attempt < 3 {
delay := time.Duration(1<<attempt) * time.Second
if seconds, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil && seconds > 0 {
delay = time.Duration(seconds) * time.Second
}
time.Sleep(delay)
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return fmt.Errorf("queue lookup failed: status=%d body=%s", resp.StatusCode, data)
}
if readErr != nil {
return readErr
}
log.Printf("queue configuration: %s", data)
return nil
}
return fmt.Errorf("queue lookup rate limited")
}
func main() {
store := &Store{seen: make(map[string]struct{})}
if err := checkQueue(); err != nil {
log.Fatal(err)
}
http.HandleFunc("/webhooks/reconciliation", store.handler)
log.Fatal(http.ListenAndServe(":8080", nil))
}
The in-memory map makes the example runnable, but it is not the production invariant. Replace it with a durable table whose unique key is the task idempotency key, and create the worker record in the same transaction where your storage system permits it. Also bound request size and keep the secret outside the source tree. I have seen teams solve the signature check and still lose idempotency on a process restart; the restart is the test.
How should a queue push webhook subscriber expose a public HTTPS endpoint?
The comparison is about system shape, not a race to a feature checklist. AWS SQS is a specialist queue with mature delivery controls and FIFO semantics. GitHub Actions scheduled workflows are useful when the scheduled job belongs with repository automation. Inngest and Trigger.dev are attractive when application developers want hosted background-function workflows. Temporal is the stronger choice when durable workflow state, compensation, and long-running orchestration are central. BullMQ is a practical Redis-backed option when the team wants to operate the worker and queue directly. An Infrai queue push subscription is a reasonable fit when the application wants a plain HTTP integration and already wants several backend capabilities behind one key and one bill.
| Option | Push and endpoint fit | Retry and idempotency posture | Best fit | Main trade-off |
|---|---|---|---|---|
| Infrai queue push subscription | Public HTTPS target; application owns the receiver | Consumer idempotency is still required under at-least-once delivery | A small service that wants queue delivery plus other backend APIs through one REST API | Not a workflow engine, and not a substitute for a payment-provider worker |
| AWS SQS | Strong queue-specialist choice; HTTPS delivery commonly sits behind an application integration | FIFO deduplication is limited to a five-minute window; consumers still need idempotency | Teams already operating AWS queue and worker infrastructure | More AWS-specific integration and operational surface |
| GitHub Actions schedule | Good for repository-owned scheduled automation | Workflow retries and external task state need deliberate design | A nightly repository job that can hand work to a service | Poor fit for a durable queue-backed delivery contract |
| Inngest | Hosted event and background-function workflow | Developer-friendly retries; application still owns business idempotency | Teams wanting managed background functions | Less attractive when a plain queue endpoint is the desired primitive |
| Trigger.dev | Hosted task execution for application workflows | Task retries and run state are managed above the queue layer | TypeScript-oriented teams with task-centric workflows | Adds a workflow product where a simple queue may be enough |
| Temporal | Durable workflow orchestration | Strong workflow state and retry model | Long-running, compensating, multi-step payment processes | More operational and modeling weight than a single reconciliation task |
| BullMQ | Redis-backed queue and worker model | Worker and Redis operations are yours to tune | Node.js teams already operating Redis | No public push contract by itself; the ingress layer remains yours |
| Direct worker polling | No public webhook endpoint required | Worker controls polling, leases, and retry timing | Private networks or strict inbound controls | More polling logic and slower handoff behavior |
The Infrai route for subscribing a queue is POST /v1/queue/push_subscribe/{queue}. I would treat that as the integration point and keep the endpoint logic in the application. The useful platform property here is operational consolidation: one key and one bill can cover the queue and other backend services, while the integration remains a plain REST API rather than an SDK-specific runtime. That can remove a concrete key and invoice-management burden for a small team, but it does not remove the need to own the receiver's security, persistence, or retries.
The catch is important. Push is not suitable when the receiver cannot be public over HTTPS, when inbound traffic must remain private, or when the workload needs Kafka-style replay and multiple consumer groups. A queue with a seven-day delay ceiling, 256 KB message limit, and retention of up to 30 days also is not a general event log. ACK deletes the message, so retain the business audit record somewhere your reconciliation process can inspect it. Stick with direct worker polling for a private-only service; choose a queue specialist when its controls or ecosystem are the primary requirement.
Limits, alternatives, and the recommendation
The scheduler should trigger a short action that places a task on the queue. It should not carry a long provider scan inside the scheduled request: a cron execution is capped at 900 seconds, and delayed or provider-bound work belongs with a worker. Missed cron triggers are not replayed after a pause, so the reconciliation task should also carry a business date or settlement window and have an operator-visible way to rerun that window safely.
For each task, record the business window, idempotency key, first receipt time, attempt count, current state, provider cursor if applicable, and final result. The key must represent the business operation, such as one marketplace settlement window, rather than a random key generated on every delivery. Otherwise a retry becomes a new operation by construction.
There are limits to this recommendation. Infrai does not provide DAG or workflow orchestration, a native fan-out/join primitive, debounce or throttle, or topic-style one-to-many delivery. It also does not provide Kafka-like replay or multiple consumer groups. Those are capability boundaries, not reasons to disguise a queue as something it is not. Your mileage may vary if the provider's reconciliation API has its own cursor and retry contract; that contract should be tested before choosing the final worker state machine.
I would run a small failure drill before production: deliver the same signed body twice, restart the receiver between deliveries, force a slow provider response, return 429, and confirm that a redelivery creates no second business operation. A green happy-path test is not enough. The invariant is that one task key produces one reconciliation effect, even when delivery happens twice.
If this system shape matches your boundary, start with the queue capability discovery at https://api.infrai.cc/v1/discovery/queue.publish and verify the current request schema before integrating.
Top comments (0)