Background Jobs in Go with Asynq: Emails, Notifications, and Resilience

When a customer places an order on a merchant's store, the checkout path does not call an SMTP server. After saving the order, it asks a mailer interface to enqueue a task in Redis.

Why Separate the Worker

Our worker binary is a dedicated process that pulls tasks from Redis queues using Asynq. It runs independently from the three HTTP binaries (site, shops, admin).

The reasoning is simple: SMTP latency is variable, and transient failures need retries. Neither concern should block the HTTP response. The worker can retry independently if the mail server is temporarily unavailable.

Task Architecture

Email delivery uses one generic task type and a template discriminator:

const TypeEmailDelivery = "email:deliver"

type EmailPayload struct {
    To       string         `json:"to"`
    Lang     string         `json:"lang"`
    Template string         `json:"template"`
    Data     map[string]any `json:"data"`
}

The producer marshals the payload and checks both operations:

payload, err := json.Marshal(EmailPayload{
    To:       recipient,
    Lang:     lang,
    Template: "new_shop_order",
    Data:     map[string]any{"OrderID": orderID},
})
if err != nil {
    return err
}
task := asynq.NewTask(TypeEmailDelivery, payload)
if _, err := client.EnqueueContext(ctx, task); err != nil {
    return err
}

The worker handles it:

func (h *EmailHandler) ProcessTask(ctx context.Context, t *asynq.Task) error {
    var p EmailPayload
    if err := json.Unmarshal(t.Payload(), &p); err != nil {
        return fmt.Errorf("decode email payload: %v: %w", err, asynq.SkipRetry)
    }
    return h.sender.Send(ctx, p)
}

Malformed JSON is permanent, so SkipRetry is appropriate. SMTP/network errors are returned normally and remain retryable.

What Lives in the Worker

Beyond order emails, the worker handles:

• Email delivery — sign-up codes, password resets, support mail, orders, and inquiries
• Telegram notifications — queued reports and alerts
• Marketplace imports — longer, low-priority ingestion tasks

Each task type is registered with its own handler. The worker also uses weighted critical, default, and low queues. Those weights bias which queued task is selected next; they do not reserve workers, so long-running imports can still occupy the whole pool.

The Dev Gotcha

During local development, a queued email will not be sent unless both Redis and the worker are running. Use the project's documented service and log commands rather than copying production hostnames or credentials into an article.

docker compose up -d worker
docker compose logs worker

If the worker is down while Redis remains available, pending tasks can be processed after the worker restarts. That is not the same as guaranteed durability: data loss still depends on Redis persistence, replication, eviction policy, and backup configuration.

Retry Policy

Asynq retries failed tasks according to its retry policy. Short notification tasks use the standard policy, while imports run in a lower-priority queue with bounded retries and execution time. After retries are exhausted, Asynq archives the task. Monitor queue latency, retry counts, and archived tasks with tooling such as Asynqmon or your own metrics.

Asynq provides at-least-once execution, not exactly-once execution. A worker can complete a side effect and fail before acknowledging the task, so handlers must be idempotent. For email, a provider idempotency key or a database send ledger can prevent unwanted duplicates.

Why Not Just goroutines?

A bare go func() { sendEmail() }() has three structural problems:

1. Lost tasks on restart. A deploy kills the process and all in-flight goroutines die.
2. No retry. If SMTP is down for 5 minutes, all emails from that window are lost.
3. No visibility. A bare goroutine provides no queue, failure, or processing state to inspect.

Redis + Asynq gives us durable-enough queuing, retries, scheduling, and observability with little application code. The trade-offs are another stateful dependency and at-least-once semantics.

One final boundary matters: a database commit and a queue publication are separate operations unless the design explicitly bridges them. For business-critical delivery, use a transactional outbox in the same database transaction and publish it asynchronously.

Task payloads may contain email addresses or order details. Keep Redis on a private network with authentication and encryption as appropriate, minimize payloads and retention, and never log full payloads.

---

This article is based on lessons from building Towami, a multi-country marketplace and white-label storefront platform. Follow for more practical notes on Go, HTMX, infrastructure, and SaaS.