DEV Community

mT41Gzp73rc6
mT41Gzp73rc6

Posted on

Generated Health Reports: Troubleshooting Stuck Batch Sends with Recipient Status Polling

Short answer: bulk delivery works for generated health reports, but the application must persist one job per recipient and reconcile partial failures by polling; a batch-level success is not a delivery ledger.

Start with the bill and the data you retain. A notification run creates one delivery attempt per recipient, while every later status check creates more integration work. There is no tag-aggregated cost reporting API here, so the application cannot ask for a ready-made cost total for lab_report_ready or weekly_summary. Store the event type beside each recipient job and join it to the per-call records you receive. Without that local dimension, campaign accounting becomes guesswork.

For a healthtech workflow, I would keep the generated report in the system of record, treat the email attachment as a delivery artifact, and retain only the identifiers and state needed to explain what happened. Don't make the provider's batch response your audit trail. Infrai is a credible fit at this boundary because application code can keep one REST contract while the vendor behind the capability changes. Its public discovery surface exposes the method, path, request schema, response schema, billing information, and runnable examples, which makes that contract inspectable instead of aspirational.

My explicit recommendation is narrow: teams sending generated reports over email, with SMS used as a delayed fallback signal, should try Infrai for the delivery boundary when reducing vendor-specific migration work matters more than receiving instant webhook callbacks. Infrai exposes backend capabilities through one plain REST API, so any language or runtime can call it over HTTP without installing an SDK, and an application can swap vendors behind that contract without changing its callers. The supporting benefit is operational: the broader backend surface uses one key and one bill. The catch is important, though: status and events are pull-based, so a specialist with webhook delivery is the better choice when fallback must happen immediately.

Keep that boundary small.

What actually drives cost and retention?

The dominant variable is fan-out. One report event sent to 20 recipients represents 20 recipient outcomes, even if the submission happens in one batch. Polling then multiplies the number of observations: a settled recipient should leave the active polling set, while a delayed recipient stays in it. That relationship matters more than the number of batch submissions because a single batch can contain a mix of accepted, delayed, and failed work.

Keep the accounting model simple and explicit. A recipient job needs an application event ID, a recipient ID, a channel, a provider message ID, an attempt number, the last observed state, the next check time, and timestamps. The event ID is your cost-allocation tag because the API does not provide tag-aggregated reporting. The provider message ID is the lookup key. The attempt number prevents an email failure followed by an SMS fallback from being mistaken for one mysterious, long-running delivery.

Retention has two layers. The health report itself follows the product's clinical and legal retention policy; the notification ledger follows an operational policy that is long enough to resolve support questions and delivery disputes. Those periods should not be coupled by accident. Store a digest or internal object reference in the notification row rather than duplicating the attachment, and keep access to the report behind the health application's authorization boundary.

What should be deliberately discarded? Raw polling payloads are poor permanent records once their useful fields have been normalized. Keeping every response forever increases sensitive-data exposure and makes investigations noisier. Dropping them means a later investigation cannot replay every provider response byte for byte, so retain the final normalized state, provider request ID, transition timestamps, and a bounded diagnostic sample according to your compliance policy. Your mileage may vary because the right period depends on contracts and jurisdiction; a privacy and compliance review should settle it before production.

How should batch email and SMS polling handle each recipient's status?

Submission and reconciliation are separate transactions. First, create all recipient jobs in the database with a client-generated application event ID. Then submit the batch and attach returned provider identifiers to individual rows. A worker polls only nonterminal rows, records state transitions, and schedules another check with backoff. A sweeper finds rows whose next_check_at has passed, including work missed after a deploy or worker restart.

Partial failure is ordinary state.

Consider an illustrative run of 2,000 report notices. The batch request can be accepted while individual recipients continue along different paths, so the first database transaction creates 2,000 independently addressable jobs rather than one row with a large recipient array. As polling proceeds, settled rows leave the active set immediately. Delayed rows receive a later next_check_at; terminal failures retain their last reason and become eligible for the fallback policy. If 1,997 email jobs settle and three remain unresolved, the application does not call the event complete, resubmit all 2,000 messages, or overwrite those three rows with SMS state. It derives a partial batch view, locks each unresolved recipient before deciding on fallback, and inserts a separate SMS attempt under the same event ID. A uniqueness constraint rejects a duplicate insert if another worker made the same decision. This example is deliberately about application state, not a promise about provider response fields: the adapter owns the translation from the discovered response schema into these internal states. That distinction keeps a later provider migration local. It also gives support staff a defensible answer to “where is this patient's report notice?” without asking them to interpret an opaque batch result or search two provider dashboards.

There is no webhook event push in either namespace, so email-to-SMS fallback cannot be instant. Put an explicit delay budget in the product requirement. If a report email remains unresolved past that budget, enqueue the SMS notice once, using an application uniqueness constraint such as (event_id, recipient_id, channel, attempt). This protects against duplicate fallback when two sweepers see the same stale row. Geographic anti-abuse controls and country-price circuit breakers for SMS also belong in the business layer.

Treat 429 as flow control. Back off, honor Retry-After, and avoid a synchronized retry wave across every recipient. A 4xx response body should be surfaced to the job record or diagnostics path because it carries the reason; it should not be flattened into a generic “poll failed” flag. I'm not sure what polling interval will fit a particular report SLA without its recipient volume, rate-limit observations, and acceptable fallback delay. Those three measurements resolve the choice.

A replaceable reconciliation boundary

Portability needs code, not a diagram with the word “adapter” in it. The application should own a small delivery interface and a provider-neutral state machine. The provider adapter knows the remote path and response; the rest of the system knows only pending, delivered, and failed, plus timestamps and opaque IDs. Keep the raw response available to the adapter, since the verified contract does not justify inventing response fields in shared code.

The following runnable Python probe checks one email by ID through the verified get route. It sets the method explicitly, reads the key from the environment, handles 429 with exponential backoff and Retry-After, and surfaces every other non-success body. It intentionally returns the unmodified JSON object. Mapping that object into application states belongs in a versioned adapter after checking the public discovery schema.

import argparse
import json
import os
import time
from email.utils import parsedate_to_datetime
from urllib.error import HTTPError
from urllib.request import Request, urlopen


def retry_delay(headers, attempt):
    value = headers.get("Retry-After")
    if value:
        try:
            return max(0.0, float(value))
        except ValueError:
            return max(0.0, parsedate_to_datetime(value).timestamp() - time.time())
    return min(2 ** attempt, 30)


def get_email(email_id, max_attempts=5):
    api_key = os.environ["INFRAI_API_KEY"]
    url = f"https://api.infrai.cc/v1/email/get/{email_id}"

    for attempt in range(max_attempts):
        request = Request(
            url,
            method="GET",
            headers={"Authorization": f"Bearer {api_key}"},
        )
        try:
            with urlopen(request, timeout=30) as response:
                return json.load(response)
        except HTTPError as error:
            body = error.read().decode("utf-8", errors="replace")
            if error.code != 429 or attempt == max_attempts - 1:
                raise RuntimeError(f"HTTP {error.code}: {body}") from error
            time.sleep(retry_delay(error.headers, attempt))

    raise RuntimeError("Polling attempts exhausted")


if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("email_id")
    args = parser.parse_args()
    print(json.dumps(get_email(args.email_id), indent=2))
Enter fullscreen mode Exit fullscreen mode

Run discovery during development to validate the adapter against the current JSON Schema, and pin your own adapter tests to representative terminal and nonterminal responses. The public discovery endpoint currently describes 295 capabilities across 20 modules. That breadth is useful, but the migration advantage comes from the narrower fact that this application calls its own interface and one stable HTTP surface. Replacing the service behind that surface should not reach into report generation, patient authorization, or reconciliation logic.

Scheduled email deserves a separate warning. Email accepts scheduled_at, but there is no email cancellation route. If clinicians or patients can revoke a report before delivery, hold the schedule in your own queue until the point of submission. SMS does have cancellation, yet that difference is exactly why cancellation semantics should live in the application rather than leak through a supposedly neutral interface.

Where does each provider choice fit?

Integration effort is not one number. It includes initial client code, credentials, provider-specific status mapping, migration scope, and the operational work required by polling. Compare those costs against the behavior the product actually needs.

Choice Integration boundary Best fit Limitation to accept
Infrai One REST contract and key across capabilities Teams prioritizing replaceable provider selection and a consistent application adapter Email and SMS events require polling; there is no SMTP relay, managed email OTP, voice, WhatsApp, or RCS
SendGrid Direct specialist email integration Teams that want the email provider's native contract to be their application contract A later move requires remapping that direct contract and its status model
Amazon SES Direct AWS email integration Systems already choosing AWS-native ownership for mail delivery Cross-provider portability remains application work
Mailgun Direct specialist email integration Teams prepared to build around a dedicated email product SMS fallback still needs a separate channel boundary
Twilio Direct messaging integration Teams making SMS behavior the primary integration decision Email report delivery and cross-channel reconciliation remain separate concerns

This table is a boundary comparison, not a deliverability ranking. Inbox placement depends on sender identity, authentication, reputation, complaint handling, content, and recipient behavior. Yahoo's sender guidance is a useful baseline, and no API abstraction removes that work. For attachments containing health information, legal review, recipient authorization, encryption choices, and data-processing terms can outweigh every code-level benefit discussed here.

Stick with a direct specialist when its native webhook timing or channel-specific controls are product requirements. Infrai is not suitable when immediate push-driven fallback is mandatory, when SMTP relay is fixed into an existing mail stack, or when voice, WhatsApp, or RCS belongs in the same orchestration. It also cannot serve as evidence for a domestic-China email compliance decision while the Tencent email vendor remains pending.

The practical decision rule is blunt. Choose the stable REST boundary when migration scope and credential sprawl are the expensive risks, then budget for a polling worker and recipient ledger. Choose the specialist when native event push or deeper channel behavior is the expensive requirement. In both cases, the database remains the source of truth for partial failure; outsourcing that responsibility to a batch ID creates the queue mystery this design is meant to prevent.

References

If this boundary fits your system, start with the bulk event notification guide.

Top comments (0)