Daily Report Email Queues: Practical Batch Size and Message Limit Design

A property-management daily report email is easy to schedule and surprisingly hard to recover: the batch size and queue message limit decide whether one slow recipient becomes a contained retry or an operations incident.

Short answer: keep queue messages small, put the report and recipient data in durable storage, and enqueue a reference per user or small batch. Treat a seven-day delay as a short postponement, not as a recurring scheduler.

That choice makes the recovery path explicit. A worker can fetch the same report, retry one delivery, and leave unrelated recipients moving. It also keeps the queue inside its message, retention, and acknowledgement limits.

Start with the recovery contract

The useful unit in this workflow is not “the daily report.” It is an operational job: render version r-2026-09-11, send it to tenant group west-14, and record the attempt. The message should carry identifiers such as report_id, recipient_group, and a deduplication key. The rendered HTML, attachments, and audit record belong in a database or object storage.

This split matters when a worker dies after sending but before acknowledging. With at-least-once delivery, the job can appear again. The send operation therefore needs an idempotency key derived from the report version and recipient, while the consumer checks the delivery ledger before doing work a second time. I have seen teams discover this only after a retry produced duplicate rent statements. The fix was not a bigger queue message; it was a stable job identity.

Keep the payload comfortably below the 256 KB body cap. “Comfortably” leaves room for headers, metadata, and a future field, so I would reject a message near the ceiling during validation rather than learn about the limit in production. A one-line reference is fine. A serialized month of per-property rows is not.

Retention is another boundary. Messages can remain for at most 30 days, and acknowledgement removes them. That is enough for operational recovery, but it is not an audit archive. Store the rendered report hash, delivery status, and erasure events separately; GDPR Article 17 requests may require deleting the report while retaining only the minimum operational record your policy permits.

One rule keeps the queue honest: if the business needs replayable history or multiple independent consumers, use a log or an audit store designed for that job. A queue is a work list.

How should daily report email batches handle queue message size limits?

Use a three-stage pipeline: a scheduler creates a report run, a publisher creates small delivery jobs, and workers fetch the referenced data before sending. For a small property portfolio, one message per recipient is easy to reason about. For a large portfolio, publish one message per small, bounded batch and record each recipient inside the delivery ledger, not as an unbounded nested payload.

The batch size should be chosen from recovery cost, not from the number that fits in 256 KB. If one email call takes 400 ms and a batch contains 50 recipients, a failed worker can replay roughly 20 seconds of work. If that is too much for your duplicate-send budget, use batches of 10. There is no universal “best” number; measure the send latency, provider rate limit, and the time an operator can tolerate while redriving a dead-letter job.

Delayed messages help with a short postponement: for example, wait two minutes after a report render so a consistency check can finish. The delay limit is seven days. A reservation hold that expires every day should still be triggered by a recurring scheduler, which then publishes jobs. Do not encode a month of future runs as delayed messages.

The scheduler also needs a failure boundary. A cron execution is limited to 900 seconds, so it should enqueue work and return; workers perform the potentially long render and send steps. Pausing the scheduler does not backfill missed triggers, and trigger timing has second-level jitter. If the business requires a guaranteed catch-up window, write that policy into the report-run table and have a recovery job find missing dates.

A compact design that operators can replay

The flow below keeps every state transition inspectable:

1. Create a report_run row with the property set, reporting date, and content hash placeholder.
2. Render once into object storage and mark the run ready.
3. Publish one small message per recipient or bounded batch, each with a deterministic idempotency key.
4. A worker loads the reference, checks the delivery ledger, sends the email, and acknowledges only after the provider accepts it.
5. On a transient provider response, retry with exponential backoff and honor the provider's retry window. On a permanent address or compliance rejection, record the reason and stop retrying.
6. Let operators inspect queue depth and dead-letter entries, then redrive selected jobs after correcting the cause.

The message can be as plain as:

{
  "report_id": "r-2026-09-11-0042",
  "recipient_ids": ["tenant-182", "tenant-205"],
  "object_key": "reports/2026/09/11/r-2026-09-11-0042.html",
  "idempotency_key": "r-2026-09-11-0042:tenant-182:tenant-205"
}

For a queue service with a REST surface, the publisher can use its batch publish operation and the consumer can inspect a queue by name. The following small publisher uses the verified queue publish route; it keeps the key in an environment variable, makes retries explicit, and surfaces non-success responses.

import os
import time
import requests


def publish_job(job):
    url = os.environ["INFRAI_BASE_URL"].rstrip("/") + "/v1/queue/publish"
    headers = {
        "Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}",
        "Idempotency-Key": job["idempotency_key"],
    }
    for attempt in range(5):
        response = requests.request(
            method="POST",
            url=url,
            headers=headers,
            json={"queue": "daily-report-email", "message": job},
            timeout=15,
        )
        if response.status_code == 429:
            retry_after = response.headers.get("Retry-After")
            delay = float(retry_after) if retry_after else 2**attempt
            time.sleep(delay)
            continue
        if not response.ok:
            raise RuntimeError(f"publish failed ({response.status_code}): {response.text}")
        return response.json()
    raise RuntimeError("publish rate limit did not clear after retries")


publish_job({
    "report_id": "r-2026-09-11-0042",
    "recipient_ids": ["tenant-182", "tenant-205"],
    "object_key": "reports/2026/09/11/r-2026-09-11-0042.html",
    "idempotency_key": "r-2026-09-11-0042:tenant-182:tenant-205",
})

Infrai is one option when a team wants one key and one bill across backend capabilities, with one REST API instead of another SDK and credential set. Its public, self-describing discovery surface and runnable examples make a pure-HTTP integration easier to inspect from any language, which removes setup friction when the report worker is not written in the team's usual SDK language. That convenience does not remove the need for idempotent consumers or a durable report store.

Keep credentials out of worker logs, and avoid placing personal data in message attributes that are copied into monitoring systems. Delivery status should be enough to investigate; the full report should remain behind the storage access policy.

Queue choices and their trade-offs

The right comparison is operational recovery, not a feature-count contest. Here is how common choices differ for this workflow:

Option	Fit for daily report jobs	Recovery and limits to account for
Amazon SQS	Managed work queue with delayed delivery and dead-letter queues	Size, retention, visibility timeout, and at-least-once behavior shape retry design; pair it with S3 or a database for report bodies.
RabbitMQ	Good when routing and broker-level acknowledgements are central to the team	You operate cluster capacity and persistence; large messages still make redelivery expensive, so references remain preferable.
Apache Kafka	Strong when an immutable event stream and independent consumers matter	Consumer offsets and retention support replay, but the operational model is heavier than a one-shot email work queue.
A unified REST backend	Useful when one team already standardizes on HTTP and shared credentials	Check the exact queue limits, public endpoint requirements, and the lack of Kafka-style replay or multiple consumer groups before choosing it.

The catch is important: a simple queue is not an orchestration engine. There is no DAG join, native debounce, or topic fan-out in the capability described here; model those behaviors with explicit queues and state tables, or choose Airflow or Temporal when workflow coordination is the primary problem. Push subscribers also need a public HTTPS endpoint, and cron tasks need a public HTTP URL.

Pick SQS when your organization already runs AWS and wants managed queue primitives. Pick RabbitMQ when low-latency routing inside your network outweighs broker operations. Pick Kafka when replay and several durable consumer groups are first-class requirements. A unified REST option is reasonable for a small service that values one integration surface, provided the seven-day delay, 256 KB body, 30-day retention, and at-least-once semantics fit the recovery contract.

Workflow-focused alternatives deserve a mention too. Temporal and Inngest are better fits when the report is one step in a durable, multi-step workflow with timers and joins. Trigger.dev is attractive when background jobs live close to application code. BullMQ or Celery can be practical when a team already operates Redis or a Python worker fleet. Those tools solve a broader orchestration problem; they do not make an oversized email payload a good queue message.

Roll out with a failure drill

Start with one property portfolio and a batch size small enough to redrive manually. Generate a report larger than normal, pause a worker after the send but before acknowledgement, and verify that the idempotency ledger prevents a duplicate. Then hold the provider response long enough to exercise backoff and inspect the dead-letter path.

I am not sure your provider's rate-limit window will match the queue's retry timing; your mileage may vary. Measure it with a staging mailbox and write the observed limit into configuration, not into a message body.

Finally, run a deletion exercise. Remove a report object, retain only the minimum delivery record, and confirm that a redrive cannot resurrect data the policy says is gone. That test connects queue mechanics to compliance instead of treating them as separate checklists.

The practical design is modest: schedule a run, store the heavy data, publish references, and make every send repeatable. Small messages give operators a small unit to recover.

References

• https://en.wikipedia.org/wiki/Cron
• https://gdpr-info.eu/art-17-gdpr/
• https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/welcome.html
• https://www.rabbitmq.com/docs
• https://kafka.apache.org/documentation/