DEV Community: Greg Pabijan-Morawski

The infrastructure I run alone for a transactional email API

Greg Pabijan-Morawski — Mon, 07 Sep 2026 08:56:25 +0000

I build and run Pulsenote by myself. It's a multi-tenant transactional email API — you POST a payload, it gets delivered, you get told what happened to it. Behind that sentence sits a NestJS monorepo, a Kubernetes cluster, ArgoCD, Vault, Terragrunt and two client SDKs. It sat on two git hosts until recently, which is one of the reductions described at the end.

That's a lot of surface for one person, and this isn't a post that presents it as a triumph. Some of it is load-bearing. Some of it is me enjoying infrastructure more than I enjoy marketing. This is my attempt to separate the two. If you're deciding how much platform to build before you have customers, the useful part is probably the section near the end where I list what I'd cut.

What the product actually has to do

Every decision below traces back to five requirements. Writing them down is the only way to judge whether the stack is justified or indulgent.

Accept a send request fast and never lose it, even when the provider is slow or throttling. Accept path and send path must be separate.
Send through a real provider with real reputation management. Domains, DKIM, SPF, bounces, suppression — the product, not plumbing.
Track what happened after the send. Webhooks arrive on the outside world's schedule and have to be ingested independently of everything else.
Be multi-tenant from day one. API keys, per-tenant domains and templates, data isolation. Retrofitting tenancy is a rewrite; building it in is a schema decision.
Have a dashboard, an admin surface, and a public site.

Requirements 1–4 are why there is a queue, a worker, a tracker and a domain model. Requirement 5 is why there are three frontends. None of that is Kubernetes yet — hold that thought.

The app layer: one monorepo, several deployables

Pulsenote is a NestJS native monorepo: one repo, one toolchain, several independently deployable apps sharing two libraries.

Apps: api-gateway, email-worker, delivery-tracker, dashboard-api, admin-api, web (Next.js 16), backoffice (Next.js 14), landing.
Libs: common (shared DTOs, guards, config) and database (TypeORM entities and migrations against Postgres).

This is deliberately not microservices: no service mesh, no per-service database, no independent versioning, no internal contract tests. Nor is it one monolith handling both a burst of inbound API calls and a long tail of send retries. The split follows failure and scaling boundaries, nothing else:

   client
     │  POST /v1/notifications   (X-API-Key)
     ▼
┌──────────────┐   validate, authenticate tenant,
│ api-gateway  │   persist, enqueue, return 202
└──────┬───────┘
       │ publish
       ▼
┌──────────────┐
│   LavinMQ    │  durable queue — the buffer between
└──────┬───────┘  "accepted" and "actually sent"
       │ consume
       ▼
┌──────────────┐   render template, apply tenant
│ email-worker │   domain/identity, hand off to provider
└──────┬───────┘
       │ send
       ▼
┌──────────────┐
│   AWS SES    │ ──────► recipient
└──────┬───────┘
       │ delivery / bounce / complaint events
       ▼
┌──────────────────┐   ingest webhooks, reconcile
│ delivery-tracker │   status back onto the message
└──────┬───────────┘
       │
       ▼
┌──────────────┐        ┌───────────────┐   ┌─────────────┐
│  Postgres    │◄───────│ dashboard-api │   │  admin-api  │
└──────────────┘        └───────┬───────┘   └──────┬──────┘
                                │                  │
                             web (N16)        backoffice (N14)

The monorepo's value is that common and database are one source of truth. Add a column, and one migration plus one entity change propagate to every app at build time, with the type checker telling me what broke. In a polyrepo I'd be publishing an internal npm package to myself at 11pm. Cross-repo coordination cost is what kills a solo maintainer.

Separate deployables matter because api-gateway and email-worker have different profiles. The gateway is latency-sensitive and bursty; the worker is throughput-oriented and must back off, retry and stall without ever making an API call slow. As one process, a single SES throttling episode would degrade signup, dashboard and API traffic simultaneously.

The two Next.js versions (16 for web, 14 for backoffice) are not a design decision. That's just what an unmigrated internal tool looks like. Leaving it in rather than tidying it up for the article.

The platform layer: DOKS, ArgoCD and why GitOps as one person

The cluster is DigitalOcean Kubernetes in AMS3, with managed Postgres and LavinMQ alongside it. It ran in LON1 until recently; moving it was a day of work and is a story of its own. AWS provides the organization structure and SES; Cloudflare does DNS; Sentry does errors.

On top sits an ArgoCD app-of-apps setup: a bootstrap argocd chart, an argo-config chart holding ApplicationSets, and a core-apps umbrella chart bundling external-secrets, Traefik, cert-manager, Vault, external-dns and a CI runner. The application deploys through its own ApplicationSet using a published nestjs Helm chart.

Why GitOps when there is no team to coordinate with? It's usually sold as a collaboration mechanism, and I have nobody to collaborate with. The honest answer: I'm not defending against a colleague's mistake, I'm defending against my own memory. Six weeks after touching something I am effectively a new team member with no context. Declarative cluster state in git means "what is running and why" is a file, not an archaeology session with kubectl, and recovery is a rebuild rather than a recall exercise — which matters far more when nobody else can rebuild it for you.

Secrets go through external-secrets backed by Vault, with the app reading paths like pulsenote/{db,mq,app,aws}. Two simpler options lost out. Plain env vars in the manifests: no, manifests live in git and I want them boring. Sealed secrets: tempting, one fewer component to run, but a sealed secret is ciphertext committed to a repo, so rotating means re-sealing and re-committing every consumer. With provider credentials and per-tenant sending identities in play, I wanted rotation to be a Vault operation, not a git commit.

That reasoning holds. Whether it justifies operating a Vault instance solo is a different question, and I'll come back to it.

Terragrunt: what it buys, and the bug I should confess

Infrastructure is a Terragrunt monorepo over about fifteen reusable Terraform modules — AWS organization, DOKS, SES pieces, Vault, LavinMQ, Cloudflare DNS. State lives in S3, provider config derives from the directory path, and each cloud gets its own tree of units.

What Terragrunt buys over plain Terraform modules is DRY-ness of config, not of code. Backend blocks, provider blocks and shared inputs are declared once and inherited down the tree, so every new unit is a small terragrunt.hcl naming a module and its inputs — no copy-pasted backend stanza that drifts. With one AWS org and a second account for production, that inheritance made "add prod later" a plausible sentence rather than a rewrite — which is what it turned out to be when production went live.

The cost is a second layer of indirection and a second tool's failure modes. And here's the concrete embarrassment, because a post like this is worthless without one.

Some of my DigitalOcean units point their source at a local absolute path on my old laptop:

terraform {
  source = "/Users/<me>/<some-other-project>/infra/tf_modules/..."
}

A machine-specific path, in an otherwise portable repo, sitting next to a perfectly good in-repo tf_modules/ directory. Those units will not plan on any other machine — a CI runner, a fresh laptop, or mine after a reinstall. It's what happens when you move fast during a migration and it works locally, so the feedback loop that would catch it never fires.

Two lessons. First: if your infra only plans on one machine, it isn't infrastructure-as-code, it's a very elaborate shell history. Second: nobody caught it because nobody else was ever going to run it. That's the tax of solo work — no second machine means no second opinion. A trivial CI job running terragrunt plan on every unit from a clean checkout would have caught it in a day.

The same migration left ArgoCD manifests and DO units still pointing at an old GitLab home for the infra repo. Which brings me to the git hosts.

Two git hosts, and whether the split is worth it

The application lived on GitLab; the infrastructure and both SDKs on GitHub, provisioned by Terraform — the repos themselves are Terraform resources, which I like: a new SDK repo is a module instantiation, not a click-through.

How did this happen? Historically. The app came from a GitLab-centric consulting context with GitLab CI already wired up and a runner in the cluster. That pipeline is migration-first — database migrations run as their own stage before any app image is rolled out, so a deploy can never land code that expects a column the database doesn't have yet. It's the single CI rule I'd port to any stack. The SDKs are public developer artifacts, and GitHub is where that audience searches, stars and files issues, and where the npm and Packagist links point. Infra landed on GitHub as part of a migration that isn't finished.

The part I'd defend: public developer artifacts belong where developers are. Nobody discovers a PHP SDK in a private GitLab group.

The part I wouldn't: everything else is friction. Two CI systems, two permission models, two places to look, and cross-references silently pointing at a repo location that moved. Starting today I'd put the app and infra on the same host as the SDKs and eat a one-time migration. The split only pays rent on the SDK repos.

Update: I took my own advice. The application repository and its CI have since moved to GitHub alongside everything else. One thing did not move — container images still push to the GitLab registry, because that part was working and the migration had a scope. So the split survives in exactly one place, at the layer where it costs nothing, which is roughly where it should have been all along.

The SDK strategy: generated TypeScript, hand-written PHP, and a drift test

This is the part of the stack I'm happiest with, and the most reusable idea here.

Both SDKs cover the same surface: the data plane, the X-API-Key endpoints for notifications, templates and domains — 17 operations. JWT account-management endpoints are deliberately out of scope; they belong to the dashboard, not to a customer's integration. Deciding that explicitly stopped a lot of scope creep.

The Node SDK is generated. openapi-typescript-codegen runs against the OpenAPI spec via npm run generate. Everything under the client is machine output; only src/client.ts and src/main.ts are hand-written wrappers. When the spec moves I regenerate and the types change. Cost: ergonomics are whatever the generator gives you, and the diffs are large and unreadable.

The PHP SDK is hand-written. PSR-18 transport, named arguments in, typed models out, PHP 8.1+, plus an auto-discovered Laravel integration — service provider, facade, and a pulsenote notification channel so Laravel users can do the idiomatic thing instead of learning my client. Generated PHP wouldn't have given me any of that. In an ecosystem where most of your users are on one framework, the framework integration is the SDK.

The risk with a hand-written client is drift: the API grows an endpoint and the SDK quietly doesn't have it. So the PHP repo carries a spec-coverage test. Each public operation is annotated:

#[Operation('POST', '/v1/notifications')]
public function send(
    string $to,
    string $subject,
    ?string $templateId = null,
    // ...
): Notification {
    // ...
}

and the test reflects over the SDK, collects every #[Operation], loads the committed openapi/pulsenote-api.json, and diffs the two sets in both directions:

public function testSdkCoversEverySpecOperation(): void
{
    $spec = $this->operationsFromSpec();   // method+path from openapi JSON
    $sdk  = $this->operationsFromAttributes(); // method+path from #[Operation]

    $this->assertSame([], array_diff($spec, $sdk), 'Endpoints in the spec but missing from the SDK');
    $this->assertSame([], array_diff($sdk, $spec), 'SDK methods pointing at endpoints the API no longer has');
}

The second assertion matters as much as the first: it catches methods that outlived the endpoint they call. make spec refreshes the committed JSON, so updating the spec is a deliberate act that shows up in a diff and turns the test red until the SDK catches up.

The pattern is cheap — one test file and one attribute — and it converts "I hope the SDK is current" into a build failure. If you maintain a hand-written client against your own API, this is the highest-leverage thing in this post. You don't need codegen to get codegen's main guarantee; you need a test that fails when the two drift.

(The one plug in this post: all of the above serves Pulsenote, which has a free tier if you'd rather see the output than the diagram.)

The self-critical part: is this too much platform for one person?

Yes. Parts of it plainly are, and I want to be precise about which parts rather than doing a performative "maybe I over-engineered it" and moving on.

The free tier is 1,000 emails a month, and the product is newly launched with no traction to speak of. Against that, I operate a Kubernetes cluster, ArgoCD with an app-of-apps hierarchy, Vault plus external-secrets, cert-manager, Traefik, external-dns, a self-hosted CI runner, and a Terragrunt monorepo over a multi-account AWS organization. It was one component longer until I deleted the self-hosted auth server and moved authentication into the application — the single biggest reduction in this list, and the one I should have made sooner. Every one has upgrades, CVEs, breaking chart changes and a 2am failure mode. There is no rotation. I am the rotation.

Here's the distinction I've landed on, and it's the one thing from this post I'd keep: complexity that is the product pays for itself; complexity that is around the product usually doesn't.

For an email API, the things that look like infrastructure to an outsider are the product:

Multi-tenancy — API keys, per-tenant domains and templates, isolation — is a feature customers buy, not a deployment detail.
Deliverability plumbing — SES identities, DKIM/SPF, bounce and complaint ingestion, suppression — is exactly what someone pays to not build. delivery-tracker exists because "did it arrive" is the question the product answers.
The durable queue between accept and send is a correctness requirement. Without it, a provider hiccup becomes lost mail.

None of that is over-engineering. Strip it out and I have a thin wrapper over SES with no reason to exist.

The hosting of it is another matter. Kubernetes, ArgoCD, Vault and Terragrunt are not the product, they're the substrate — and they're where most of my operational hours have gone. The absolute-path bug above cost me nothing in customer value and a real evening of my life. Labelled clearly as an impression rather than measured data: the platform layer has taken meaningfully more of my time than the app layer, and it's not the part customers can see. The recurring bill is dominated by the cluster's node pool and managed Postgres, both of which exist whether or not anyone sends mail — my costs are provisioned, not usage-driven, which is exactly backwards pre-traction.

The counter-argument, which I do believe: I knew this stack cold before I started. My background is platform work, so standing up DOKS and ArgoCD was faster for me than learning a PaaS's opinions. The trap in build-in-public posts is that the author's prior expertise silently subsidises the recommendation. So: don't copy this stack because it worked for me — copy it only if you already have these scars.

What I'd cut if I started over

Concretely, unsentimentally, in the order I'd cut them:

Kubernetes, initially. For a pre-traction product with a handful of long-running processes, managed containers (App Platform, Fly, ECS, whatever) would run the same eight deployables with a fraction of the operational surface. Move to Kubernetes when a real constraint demands it, not on day one.
Self-hosted Vault. Rotation is a genuine requirement and I'd keep external-secrets as the interface, but point it at a managed secrets store. Same guarantee, no upgrade path to own.
Self-hosted auth (FusionAuth). For a B2B email API, auth is table stakes, not a differentiator. Running my own identity provider is the definition of undifferentiated heavy lifting.
The second git host. Consolidate app and infra onto the SDKs' host. Keep the SDK repos where developers find them; stop paying for two CI systems.
The self-hosted CI runner. It exists because of the cluster. Remove the cluster and it removes itself.
One of the two admin surfaces, for now. admin-api plus backoffice on an older Next.js is real maintenance for an audience of exactly one person: me. Database access and a couple of scripts would have covered it early on.

Two of those have since happened: the self-hosted identity provider is gone, replaced by auth inside the application, and the app repository has moved to sit with the SDKs. Neither was hard once written down — which is an argument for writing the list rather than for having better judgement.

What I'd keep without hesitation: the NestJS monorepo with separate deployables, the accept/queue/send/track split, Terraform for anything cloud-shaped, and the SDK drift test.

And the least glamorous line here: I built the platform before I had users to justify it, because the platform was the part I already knew how to do and shipping to strangers is the part I didn't. That's not an architecture mistake. It's a procrastination pattern that happens to compile.

—

Why your transactional email needs a queue, not a try/catch

Greg Pabijan-Morawski — Mon, 31 Aug 2026 06:04:25 +0000

Almost every codebase I've inherited sends email the same way: somewhere inside a POST handler, between the database write and the response, there's an await on the mail provider's SDK. It works, for months. Then one afternoon your signup endpoint starts timing out, and it takes an hour to work out that the cause is your email provider having a bad day three thousand kilometres away.

I build Pulsenote, a transactional email API, so I've spent an unreasonable amount of time in the space between "your API call returned 200" and "the message is in the inbox". This post is what lives in that gap, why a try/catch doesn't cover it, and where the line sits between "you need a pipeline" and "you're overengineering a side project".

The naive version

Here's the code. You've written this.

// users.controller.ts
@Post('signup')
async signup(@Body() dto: SignupDto) {
  const user = await this.users.create(dto);

  await this.mailer.send({
    to: user.email,
    subject: 'Confirm your email',
    html: renderConfirmation(user),
  });

  return { id: user.id };
}

Nine lines, obvious intent, no infrastructure. For a lot of applications this is genuinely the right answer, and I'll come back to that at the end. But let's be precise about what it costs, because "it's fine" and "I haven't measured it" are different statements.

It puts a third party in your request path. Your p99 for POST /signup is now your p99 plus the provider's p99. Not their median — their tail.

A slow provider becomes a slow endpoint, then no endpoint. This is the failure mode that actually takes services down. If the provider degrades to five seconds per call, every signup request holds a connection and an event-loop continuation for five seconds. Your connection pool fills, your load balancer queues, health checks fail, the pod gets restarted, and now you're down — because of email. The blast radius of a non-critical dependency became the whole endpoint.

A provider 5xx loses the mail entirely. What does your catch do? Realistically one of two things. It rethrows, so the user sees a 500 for a signup that already succeeded in the database — now you have a user row with no confirmation email and a client that will retry and hit a unique constraint. Or it swallows the error, returns 200, and the email is simply gone. No record, no retry, no way to answer "did we ever send that?"

There is no retry. SES will return a ThrottlingException with Maximum sending rate exceeded when you exceed your account's send rate (AWS docs) — AWS's own guidance is to wait and retry the send request. Inline, inside an HTTP handler, you have nowhere to wait. Your only options are to block the user or drop the message.

There is no idempotency. The client's HTTP retry — after a timeout your code caused — sends the email twice, or the user twice, or both.

There is no audit trail. When support asks "did the password reset go out to this customer at 14:12?", the honest answer is "there's a log line if the log retention hasn't rolled over".

The fix people reach for, which is worse

The instinct, once the latency problem shows up, is to stop awaiting.

@Post('signup')
async signup(@Body() dto: SignupDto) {
  const user = await this.users.create(dto);

  // don't block the response
  this.mailer.send({ to: user.email, /* ... */ })
    .catch(console.error);   // <- this line is a landmine

  return { id: user.id };
}

The latency problem does go away. Everything else gets strictly worse.

.catch(console.error) is silent loss with extra steps. The failure is now a log line nobody reads instead of a stack trace someone would have seen. You've converted a loud bug into a quiet one, which is the wrong direction.

There's no backpressure. Awaiting at least served as an accidental rate limiter — one in-flight send per request. Fire-and-forget lets a traffic spike launch ten thousand concurrent sends at a provider that will throttle you, and the throttling errors land in console.error.

And it's lost on restart. A floating promise lives in one process's heap. On Kubernetes — which is where Pulsenote runs, DOKS in LON1 — pods restart constantly: deploys, node drains, evictions, OOM kills, scale-downs. Every rollout silently drops whatever was in flight, and "we rolled out at 14:10" is never the first hypothesis when a customer reports a missing email.

Fire-and-forget doesn't solve the problem. It moves it somewhere you can't see.

Enqueue, then process

The actual fix is to split the operation at the point where you make a promise to the caller.

An HTTP handler should do exactly the work needed to accept responsibility for a message, then return. Delivering it happens elsewhere, on its own schedule, with its own failure handling. That boundary — accept vs. deliver — is the whole idea. Everything else is implementation.

   client
     │  POST /v1/notifications
     ▼
┌──────────────┐  1. authenticate tenant
│ api-gateway  │  2. INSERT message (status=queued)  ◄── source of truth
└──────┬───────┘  3. publish {messageId}
       │          4. return 202 Accepted
       ▼
┌──────────────┐
│   LavinMQ    │  durable queue — a pointer, not the payload
└──────┬───────┘
       │ consume
       ▼
┌──────────────┐  load row, check status, render,
│ email-worker │  send, record attempt, ack
└──────┬───────┘
       │            ┌──────────────┐
       │  on give-up │ dead letter │
       │ ───────────►└──────────────┘
       ▼
┌──────────────┐
│   AWS SES    │ ──────► recipient's mail server
└──────┬───────┘
       │ Delivery / Bounce / Complaint  (SNS)
       ▼
┌──────────────────┐  reconcile status onto the row,
│ delivery-tracker │  auto-suppress bad recipients
└──────────────────┘

In Pulsenote that's api-gateway → LavinMQ → email-worker, with delivery-tracker as a separate ingest path for provider callbacks. The accept side:

// notifications.controller.ts
@Post()
@UseGuards(ApiKeyGuard)
async enqueue(@Tenant() tenant: TenantContext, @Body() dto: SendEmailDto) {
  const message = await this.messages.accept(tenant, dto);
  return { id: message.id, status: message.status };  // 202
}

// messages.service.ts
async accept(tenant: TenantContext, dto: SendEmailDto): Promise<Message> {
  const message = await this.dataSource.transaction(async (em) => {
    const existing = await em.findOne(Message, {
      where: { tenantId: tenant.id, idempotencyKey: dto.idempotencyKey },
    });
    if (existing) return existing;              // replay, not a new send

    return em.save(em.create(Message, {
      tenantId:       tenant.id,
      to:             dto.to,
      templateId:     dto.templateId,
      payload:        dto.variables,
      idempotencyKey: dto.idempotencyKey,
      status:         MessageStatus.QUEUED,
      attempts:       0,
    }));
  });

  // publish AFTER the row is committed
  await this.broker.publish('email.send', { messageId: message.id });
  return message;
}

Two details in there matter more than the rest.

The row is committed before the message is published. If you publish first and the transaction rolls back, the worker consumes a message ID that doesn't exist. Commit first: worst case the publish fails and you have a queued row nobody picked up, which a sweeper query finds in seconds. An orphaned row is recoverable; a phantom job is not. (To close that window entirely, the transactional outbox pattern is the next step up — worth it eventually, not on day one.)

The queue carries a pointer, not the payload. The database row is the source of truth. The queue is a transport — no schema evolution, no query interface, no history. If the broker loses a message you can requeue from the table; if the payload only existed in the message, it's gone. And when support asks what happened to a specific email, you need SELECT, not a queue browser.

The caller gets a 202 and an ID. That's an honest response — "I have durably accepted this and I will tell you what happens to it" — and a much stronger promise than a 200 that meant "an SDK call didn't throw".

Idempotency, and why "exactly once" is a lie

Every real broker gives you at-least-once delivery. LavinMQ, RabbitMQ, SQS, Kafka — the guarantee is the same, because the alternative requires a distributed transaction between your broker and your side effect, and the side effect here is an HTTP call to Amazon.

Your worker crashes after SES accepts the message but before the ack. The broker sees an unacked message and redelivers. That's not a bug, that's the design. Your worker will see duplicates. Plan for it. So dedupe in two places.

At the edge, an idempotency key from the client — the accept() above, with a unique index on (tenant_id, idempotency_key). A client retrying a timed-out POST gets the original message back instead of a second send. Make the key required for anything expensive, or derive one and document it.

In the worker, a status check inside a row lock:

// email.worker.ts
@RabbitSubscribe({ exchange: 'pulsenote', routingKey: 'email.send' })
async handle(msg: { messageId: string }) {
  const claimed = await this.dataSource.transaction(async (em) => {
    const row = await em.findOne(Message, {
      where: { id: msg.messageId },
      lock: { mode: 'pessimistic_write' },
    });
    if (!row) return null;
    if (row.status !== MessageStatus.QUEUED) return null;  // already handled

    row.status = MessageStatus.SENDING;
    row.attempts += 1;
    return em.save(row);
  });

  if (!claimed) return;   // duplicate delivery — ack and move on
  await this.deliver(claimed);
}

The lock is what makes this safe when two consumers get the same message concurrently, which happens whenever you scale the worker past one replica. Without it you have a check-then-act race and you'll ship the occasional double email.

"Exactly-once delivery" as a product claim generally means exactly-once processing — at-least-once transport plus deduplication at the consumer. Which is what you just built. There's no version of this where the network stops being able to lose an ack.

Retries: not all failures are equal

The single most valuable thing the worker does is classify errors. Retrying a hard bounce is worse than useless — it inflates the bounce rate that your provider judges you on.

function classify(err: unknown): 'retry' | 'terminal' {
  const name = (err as { name?: string })?.name ?? '';
  const status = (err as { $metadata?: { httpStatusCode?: number } })
    ?.$metadata?.httpStatusCode ?? 0;

  // retryable: throttling, provider 5xx, transport failures
  if (name === 'ThrottlingException') return 'retry';
  if (name === 'TooManyRequestsException') return 'retry';
  if (status === 429 || status >= 500) return 'retry';
  if (['ETIMEDOUT', 'ECONNRESET', 'EAI_AGAIN'].includes((err as any)?.code)) {
    return 'retry';
  }

  // terminal: the request itself is wrong, or the recipient is unreachable
  if (name === 'MessageRejected') return 'terminal';
  if (name === 'AccountSuppressionListException') return 'terminal';
  if (status >= 400 && status < 500) return 'terminal';

  return 'retry';   // unknown → retry, and alert on it
}

Retryable means the same request might succeed later: throttling, 5xx, connection resets, DNS blips. Terminal means it will never succeed: malformed address, unverified sending identity, a recipient on the suppression list. Terminal failures go straight to failed — one attempt, no backoff, immediate status the customer can see.

For retryable failures, exponential backoff with jitter:

function nextDelayMs(attempt: number): number {
  const base = Math.min(1_000 * 2 ** attempt, 15 * 60_000);  // cap at 15 min
  return Math.round(base * (0.5 + Math.random() * 0.5));     // full-ish jitter
}

The jitter is not decoration. When a provider throttles you it throttles everything at once, so every failed message becomes due for retry at the same instant. Without jitter your retries arrive as a synchronised thundering herd and get throttled again, in lockstep, forever. Spreading them out is the difference between draining a backlog and oscillating.

The cap matters too. AWS's guidance for a throttling error is to wait — their docs suggest an interval of up to 10 minutes before retrying (SES quota errors). Backing off for hours on a transactional email is pointless; a password reset that arrives 90 minutes late has already failed at its job. Pick a max attempt count (I use 5) and a delay ceiling in the low tens of minutes, then stop.

When attempts are exhausted, the message goes to a dead-letter queue and the row goes to failed. The DLQ is not a graveyard — it's an inbox for a human. Something is broken if it's non-empty, and the message body is enough to replay once you've fixed it.

The 250 is not delivery

Here is the part that a queue alone doesn't solve, and the reason email is genuinely harder than most async work.

SES accepting your message — a MessageId on the API, a 250 on SMTP — means only that SES will attempt delivery. AWS states it plainly: the Send event means "the send request was successful and Amazon SES will attempt to deliver the message to the recipient's mail server" (monitoring sending activity). Actual Delivery, Bounce and Complaint are separate events that arrive later, over SNS or an event destination, on the receiving world's schedule.

So your message has a state machine, not a boolean:

queued ──► sending ──► sent ──┬──► delivered
   │           │              ├──► bounced      (hard → suppress)
   │           └──► failed    ├──► complained   (→ suppress)
   └──► failed                └──► delayed ──► delivered | bounced

sent and delivered are different columns and different truths. A system that only models "did the API call succeed" will report 100% success while a domain silently rejects every message you send it.

This is what delivery-tracker exists for: a separate ingest path that verifies SNS signatures, writes the event, and reconciles status onto the message row. Separate because it's driven by an external system with its own retry behaviour — you do not want provider webhook traffic sharing a deployment with your customer-facing API.

And suppression has to be automatic. On a hard bounce or a complaint, the recipient goes on a suppression list and future sends to that address are rejected at accept time, before they ever reach the provider. The reason is commercial, not aesthetic. SES will place your account under review if your bounce rate reaches 5%, and may pause your sending entirely at 10%; for complaints those numbers are 0.1% and 0.5% (SES reputation metrics). Those are small numbers. Manual suppression is not a control that operates at that resolution — by the time a human notices, the rate is already set. (SES maintains its own account-level suppression list, but you want your own too: yours is per-tenant, queryable, and lets you reject at the API boundary instead of burning a send.)

Multi-tenancy: one noisy tenant, everyone's problem

If you're building this for one application you can skip ahead. If you're building a platform, fairness is a first-class concern, because your provider quota is a shared, finite resource. A new SES account starts in the sandbox at 200 messages per 24 hours and 1 message per second (production access docs); production limits are higher but still an account-wide ceiling you can hit.

A single FIFO queue means one tenant dumping 50,000 messages puts every other tenant's password reset behind 50,000 of them. The queue is fair in ordering and grossly unfair in outcome. Two mechanisms fix most of it:

Admit at the edge. Enforce a per-tenant rate limit and plan quota at accept time, in the gateway — a token bucket in Redis keyed by tenant. Rejecting with a 429 the client can back off from is far better than accepting work you'll deliver hours late. Backpressure the caller can see is a feature.
Don't let one tenant own the consumers. Prefetch of 1 per consumer plus a bounded per-tenant in-flight count stops a single tenant occupying every worker. Separate queues by priority class if you have a real distinction between transactional and bulk — and if you're a transactional API, you should.

The global send rate also needs a limiter in front of the provider, shared across worker replicas, set below your actual SES rate. Retrying throttles works; not being throttled works better.

What to log, what to alert on

Structured, on every attempt: messageId, tenantId, templateId, attempt, status, provider MessageId, error class, latency. Recipient address hashed or redacted depending on where the logs go. The provider's message ID is the join key for every "what happened to this email" investigation — without it you cannot correlate your row with their events.

Alert on a much shorter list:

DLQ depth > 0. Not a threshold. Any dead-lettered message means something needs a human.
Bounce rate approaching 5%, complaint rate approaching 0.1% — per tenant and account-wide. Alert well below the provider's line, because by the time you cross it the damage is a trailing average you can't undo quickly.
Queue depth trending up over N minutes. Depth is meaningless as an instant value and diagnostic as a derivative: consumers are slower than producers.
Oldest message age in queued. The one metric that maps directly to what a user experiences. Queue depth can look healthy while one message sits stuck for an hour.

What I would not alert on: individual send failures. That's what retries are for, and alerting on transient noise trains you to ignore the channel — which is how you miss the DLQ page.

When you genuinely don't need any of this

I'd rather be useful than sell you architecture, so: most applications don't need this pipeline.

If you're sending ten emails a day from a side project, await mailer.send() in the handler is correct. Fewer moving parts, no broker to operate, no worker to deploy, and the failure mode — you notice an email didn't arrive and click resend — costs you a minute. Building a durable pipeline for that volume is a way of avoiding the harder work of getting users. Roughly where I'd draw the lines:

Inline await — low volume, email isn't load-bearing, you'd notice a failure yourself. Add a messages table anyway, purely for the audit trail. That's a one-hour change with permanent value.
Queue + worker — the email is part of a flow a user is waiting on (signup, reset, receipt), or a provider outage would mean silent loss, or you have enough volume to hit rate limits. Note that this doesn't require Kafka. A single durable queue and one worker process is most of the benefit.
Full pipeline (idempotency keys, classified retries, DLQ, webhook ingest, automatic suppression, per-tenant fairness) — you're sending on behalf of other people, or the mail is regulated, or you're big enough that your bounce rate is your provider relationship.

The honest test is a question: if your provider returned 503 for the next thirty minutes, what would happen? If the answer is "some emails wouldn't arrive and I'd resend them" — you're fine, stop reading. If it's "I don't know" or "we'd lose them and never find out" — that's a queue-shaped hole, and no amount of try/catch fills it.

One product note, clearly marked: everything above is the architecture behind Pulsenote, which is what I do instead of asking you to build it — a transactional email API with the queue, retries, suppression and delivery tracking already wired up, free tier at 100 emails/month. If you'd rather own the pipeline yourself, the post above is the whole design, and I'd genuinely rather you build it well than not build it at all.

—

SPF, DKIM and DMARC, explained by someone who had to make it work

Greg Pabijan-Morawski — Fri, 21 Aug 2026 14:06:30 +0000

I build a transactional email API, which means I have spent an unreasonable amount of my life reading DMARC aggregate reports and explaining why password reset emails land in spam even though "nothing changed."

Most email authentication documentation is either a vendor wizard telling you to paste a record, or an RFC. This is the middle thing I wanted when I started: what each mechanism does, what breaks, and the order to do it in.

Why any of this exists

SMTP was designed in 1982 with no concept of authentication. A mail server accepts a connection and believes what it is told. Concretely, a single message carries two different sender addresses:

MAIL FROM:<bounces@sendinginfra.example>   <-- the envelope sender (SMTP-level)
RCPT TO:<you@gmail.com>
DATA
From: Support <support@yourbank.com>        <-- the header From (what the user sees)
Subject: Please confirm your account

The envelope sender is a routing detail: it is where bounces go. The From: header is just a line of text inside the message. Nothing in SMTP connects them. Your mail client displays the second and ignores the first.

So the whole attack is: connect to Gmail from any VPS, put From: support@yourbank.com in the headers, done. Every mechanism below closes part of that gap.

SPF authorises the envelope sender's domain against the connecting IP.
DKIM cryptographically signs the message, independent of the connection.
DMARC ties either of those back to the visible From: domain and tells receivers what to do when neither matches.

You need all three. SPF and DKIM on their own authenticate things the user never sees.

SPF: which IPs may send for this domain

SPF is one TXT record at the domain apex, listing authorised senders.

example.com.  IN  TXT  "v=spf1 include:amazonses.com include:_spf.google.com ip4:203.0.113.7 -all"

Read left to right; the first mechanism that matches wins.

include: — pull in another domain's SPF record (your ESP, your CRM, Google Workspace).
ip4: / ip6: — literal addresses or CIDR blocks.
a, mx — the domain's own A/MX records.
all — matches everything; the qualifier on it is your default.

The qualifier on all is the only interesting policy decision:

	meaning	receiver behaviour
`-all`	hard fail	reject or heavily penalise
`~all`	soft fail	accept but mark suspicious
`?all`	neutral	no opinion, i.e. pointless

Use ~all while you are still discovering which systems send as you, then move to -all. Do not stop at ~all forever: once DMARC is at enforcement, -all is what makes an unauthorised relay actually fail rather than merely look odd.

The 10-lookup limit, and how everyone blows it

RFC 7208 §4.6.4 caps SPF evaluation at 10 DNS-querying mechanisms. include, a, mx, ptr, exists and redirect each count — and they count recursively. Exceed it and the result is permerror, which most receivers treat as "SPF did not pass."

ip4: and ip6: cost nothing. That is the escape hatch.

This record looks harmless and is already broken:

"v=spf1 include:_spf.google.com include:amazonses.com include:servers.mcsv.net include:_spf.salesforce.com include:spf.protection.outlook.com ~all"

_spf.google.com alone expands to three further includes. Five vendor includes routinely resolve to 12–15 lookups. The failure is silent: nothing bounces, deliverability just degrades, and it degrades for every message from the domain, not just the vendor that pushed you over.

Fixes, in order of preference:

Delete includes for services you no longer use. Most SPF records are archaeology.
Move a vendor onto its own subdomain (see the subdomain section below) so it gets its own SPF record and its own budget of 10.
Replace an include with the ip4: ranges it expands to — only for vendors with stable IPs, and accept that you now own keeping it current. Automated "flattening" services do this for you at the cost of a dependency that breaks when a vendor renumbers quietly.

Check the count before you ship a change. dig +short TXT example.com shows the record; a validator shows the expansion.

Why SPF alone is not enough

SPF authenticates the connecting IP against the envelope domain. Two consequences:

It says nothing about what the user sees. An attacker publishes perfect SPF for evil.example, uses it as the envelope sender, and still puts From: you@yourbank.com in the headers. SPF passes. That is exactly why DMARC's alignment check exists.
It breaks on forwarding. A user forwards your mail from a university address to Gmail. That server connects from an IP your record does not list while keeping the original envelope sender, so SPF fails — correctly and uselessly. Some forwarders rewrite the envelope (SRS); many do not. This is the practical reason DKIM matters.

DKIM: a signature that travels with the message

DKIM is asymmetric crypto. You hold a private key; the public key lives in DNS. Your outbound server hashes a canonicalised set of headers plus the body, signs it, and adds a DKIM-Signature header:

DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=s1;
    h=from:to:subject:date:message-id:mime-version:content-type;
    bh=2jUSOH9NhtVGCQWNr9BrIAPreKQjO6Sn7XIkfJVOzv8=;
    b=dzdVyOfAKCdLXdJOc9G2q8LoXSlEniSbav+yuU4zGeeruD00lszZVoG4ZHRNiYzR

d= — the signing domain. This is what DMARC will compare against From:.
s= — the selector, which locates the public key. Multiple selectors per domain is normal and is how rotation works.
h= — the list of headers covered by the signature. from is mandatory.
bh= — hash of the body; b= — the signature itself.

The public key is a TXT record at <selector>._domainkey.<domain>:

s1._domainkey.example.com.  IN  TXT  "v=DKIM1; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAv9tZ0hR2m..."

Practical notes that cost me time:

DNS TXT strings max out at 255 characters. A 2048-bit key exceeds that and must be split into multiple quoted strings the resolver concatenates. Most providers handle it; some paste it verbatim and silently produce an invalid key. If DKIM "just doesn't verify," check this first with dig +short TXT s1._domainkey.example.com.
1024 vs 2048. RFC 8301 sets 1024 as the floor and 2048 as what signers should use. Use 2048; 1024 is a downgrade with no upside. Ed25519 (k=ed25519) produces tiny records but verifier support is still uneven — dual-sign if you want it.
Rotation is the whole reason selectors exist. Publish s2._domainkey with the new key, wait for propagation, switch signing to s2, wait past the longest TTL plus anything in retry queues (I give it a week), then delete s1. There is never a window where valid mail cannot be verified. Rotate annually, or immediately if a key may have leaked.

Why DKIM survives forwarding

The signature covers headers and body, not the connection. A forwarder that relays the bytes unchanged relays a still-valid signature. That is why DKIM is the load-bearing mechanism in practice and SPF is the one that keeps failing on mailing lists.

It is not bulletproof. Mailing lists that append a footer or prefix [list] to the subject invalidate the body hash or the signed subject. That is what ARC (RFC 8617) papers over: intermediaries record the authentication result they saw so the final receiver can choose to trust it. You do not implement ARC as a sender — you just need to know it is why some list traffic still gets through.

DMARC: alignment, policy, and reports

DMARC connects the previous two to what the user actually sees. It defines alignment, publishes a policy for failures, and gets you reports.

The spec is now on the IETF Standards Track: RFC 9989, 9990 and 9991 were published on 20 May 2026, obsoleting the 2015 informational RFC 7489. I flag below where that changed the tags you write.

Alignment is the entire point

A message passes DMARC if at least one of the following holds:

SPF passes and the envelope MAIL FROM domain aligns with the From: domain, or
DKIM passes and the signature's d= aligns with the From: domain.

"Aligns" has two modes, set per-mechanism:

Relaxed (adkim=r, aspf=r — the default): the organizational domains must match. mail.example.com aligns with example.com.
Strict (adkim=s, aspf=s): the FQDNs must match exactly. mail.example.com does not align with example.com.

Here is the failure that catches almost everyone on a first integration. You send through an ESP: your From: is billing@example.com, but the envelope sender is the ESP's own bounce domain.

MAIL FROM:<0100018e-bounces@eu-west-1.amazonses.com>
From: billing@example.com

SPF passes — for amazonses.com. It does not align with example.com, so it contributes nothing to DMARC. If DKIM also signs with d=amazonses.com, DMARC fails outright and you never see an SMTP error, because p=none means "deliver anyway and tell me." Fix it with a custom MAIL FROM subdomain (bounce.example.com, delegated to the ESP) or — easier and sufficient — DKIM signing with d=example.com.

The record

_dmarc.example.com.  IN  TXT  "v=DMARC1; p=none; rua=mailto:dmarc@example.com; adkim=r; aspf=r; fo=1"

p= — policy for the organizational domain: none, quarantine, reject.
sp= — policy for subdomains. If absent, p= applies to them too.
np= — new in RFC 9989: policy for non-existent subdomains. Set np=reject early; nothing legitimate sends from a subdomain with no DNS records, so it is free protection against invoices.example.com-style spoofing.
rua= — where aggregate reports go. fo=1 — request a failure report when any mechanism fails, not only when all do.
pct= — deprecated in RFC 9989, replaced by t=y (testing) / t=n (default, enforce). Deployed receivers still honour pct=, so it works during a transition, but write new records with t=.

Rolling out without breaking payroll

The order matters, and the whole point of starting at none is that it is observation-only.

p=none. Nothing changes for recipients; you just start receiving reports. Leave it two to four weeks — long enough to catch the monthly invoice run and whatever marketing does.
Fix what the reports show. Every legitimate source must reach alignment on SPF or DKIM. This is where you discover the CRM nobody told you about.
p=quarantine, initially sampled (t=y, or the older pct=25), then full.
p=reject, once reports show near-100% alignment for a couple of weeks.

Do not jump straight to reject. I have watched a company do that and take down their own recruiting pipeline for two days.

Actually reading an aggregate report

rua reports arrive daily as gzipped XML, one file per reporting receiver. The useful part is the per-source-IP rows:

<record>
  <row>
    <source_ip>203.0.113.7</source_ip>
    <count>412</count>
    <policy_evaluated>
      <disposition>none</disposition>
      <dkim>pass</dkim>
      <spf>fail</spf>
    </policy_evaluated>
  </row>
  <identifiers>
    <header_from>example.com</header_from>
  </identifiers>
  <auth_results>
    <dkim><domain>example.com</domain><result>pass</result></dkim>
    <spf><domain>bounces.esp.example</domain><result>pass</result></spf>
  </auth_results>
</record>

Read it in this order:

policy_evaluated is the aligned result; auth_results is the raw one. When auth_results/spf says pass but policy_evaluated/spf says fail, that is an alignment problem, not an SPF problem — exactly the ESP case above.
count is messages, not recipients. Sort descending; two or three IPs will be 95% of your volume.
Rows with dkim=fail and spf=fail and a meaningful count are either a source you forgot or someone spoofing you. Look up the IP before assuming which.

Do not parse these by hand past week one — point rua at an aggregator and read a dashboard. Understand the XML once so you know what the dashboard is summarising.

The bulk-sender rules you now have to meet

Since 2024 the large mailbox providers have made this mandatory rather than advisory, and enforcement has hardened considerably.

Google and Yahoo apply their requirements to senders of 5,000+ messages per day to their users. Per Google's sender guidelines FAQ, messages from the same primary domain — subdomains included — count toward that 5,000, and once you are classified as a bulk sender the classification does not expire. Requirements: SPF and DKIM both set up, a DMARC record at minimum p=none, and the From: organizational domain aligned with either the SPF or the DKIM organizational domain. Spam complaint rate should stay below 0.1%, and at 0.3% or above you become ineligible for mitigation until you have been back under 0.3% for seven consecutive days. Since November 2025 Gmail has escalated from temporary deferrals to permanent rejections for non-compliant traffic (Proofpoint's write-up).

Microsoft joined on 5 May 2025 with the same 5,000/day threshold for consumer Outlook/Hotmail/Live addresses, requiring SPF, DKIM and DMARC with alignment on at least one (dmarcian's summary). Non-compliant mail is rejected with 550 5.7.515 Access denied, sending domain [domain] does not meet the required authentication level (uriports) — a hard bounce, not a spam-folder problem.

One-click unsubscribe (RFC 8058) applies to marketing and promotional mail, not to transactional mail like password resets or receipts. Both headers are required:

List-Unsubscribe: <https://example.com/u/9f3a2c>, <mailto:unsub@example.com?subject=unsub>
List-Unsubscribe-Post: List-Unsubscribe=One-Click

The URL must accept an unauthenticated POST with body List-Unsubscribe=One-Click and unsubscribe on that single request — no confirmation page, no login. Requests must be honoured within two days. A mailto:-only or plain-link unsubscribe does not satisfy it.

Also expected across all three: TLS on delivery, valid forward-confirmed reverse DNS on sending IPs, and RFC 5322-conformant messages.

These are floors, not targets — meeting them gets you considered, not delivered. And do not wait until 5,000/day: the threshold is where enforcement begins, but the reputation you build below it is what you arrive with.

Split transactional and marketing onto different subdomains

Reputation is tracked per domain (and per IP), and it is dominated by complaint rate. Marketing email gets complaints structurally — people who wanted a discount code six months ago now hit "spam" instead of unsubscribe. Password resets get essentially none. Send both from example.com and the newsletter's complaints degrade the login flow. That is how you end up with users unable to receive a verification code.

mail.example.com   -> transactional: receipts, resets, alerts
news.example.com   -> marketing: newsletters, campaigns
example.com        -> corporate mail (Google Workspace / Microsoft 365)

Each gets its own SPF record — and therefore its own 10-lookup budget — and its own DKIM keys. DMARC still works: with default relaxed alignment, From: noreply@mail.example.com aligns against the policy at _dmarc.example.com, and you can override per-subdomain with a dedicated _dmarc.mail.example.com record. Publish sp= and np= on the parent to cover subdomains you have not delegated.

The trade-off: a fresh subdomain has no reputation and needs warming — low volume, ramped over two to four weeks. Do this before you need it, not during a migration.

Failure modes, with symptoms

Mail delivers, but DMARC reports show spf=fail while raw SPF passes.
Cause: the envelope sender is your ESP's bounce domain, so SPF authenticates the wrong domain. Fix: DKIM-sign with d=yourdomain.com, or set a custom MAIL FROM subdomain.

Deliverability degraded gradually, nothing obviously changed.
Cause: SPF exceeded 10 lookups after someone added a vendor include, yielding permerror. Fix: count the expansion, prune dead includes, move vendors to subdomains.

DKIM never verifies for a newly added selector.
Cause: the 2048-bit key was pasted as a single TXT string over 255 characters, the provider added surrounding quotes, or the p= value carries line breaks. Fix: dig +short TXT s1._domainkey.example.com, reassemble, compare byte-for-byte with the signer's key.

Direct mail is fine, mailing-list traffic fails DMARC.
Cause: the list modified the subject or body, breaking the DKIM signature; SPF was already broken by the relay. Fix: nothing on your side. Expected, and why ARC exists. Do not weaken your policy over it.

550 5.7.515 from Outlook.
Cause: you crossed 5,000/day to consumer Microsoft addresses without full SPF + DKIM + DMARC alignment. Fix: publish DMARC (even p=none) and confirm at least one aligned mechanism.

DMARC passes but mail still goes to spam.
Cause: authentication is identity, not reputation — complaint rate, list hygiene and sending history decide placement. Fix: suppress hard bounces and complaints, honour unsubscribes fast, stop mailing people who have not opened anything in a year.

The afternoon checklist

Inventory every system that sends as your domain. Billing, CRM, monitoring, the app, HR. This step is longer than you expect.
Publish one SPF record per sending domain, ending in ~all. Verify the expansion is under 10 lookups.
Enable DKIM on every source. 2048-bit, unique selector per source. Verify each with dig.
Publish _dmarc with p=none; rua=mailto:...; fo=1; np=reject. Point rua at an aggregator.
Wait two to four weeks. Read reports. Fix alignment until every legitimate source passes.
Move SPF to -all. Move DMARC to p=quarantine, then p=reject.
Add List-Unsubscribe + List-Unsubscribe-Post to marketing mail and make the POST endpoint work unauthenticated.
Split transactional and marketing subdomains; warm the new one.
Set a calendar reminder to rotate DKIM keys in 12 months. You will not remember otherwise.

Steps 1–4 genuinely fit in an afternoon. Step 5 is the one people skip, and it is the only one that tells you whether the rest worked.

Disclosure, since it is relevant: I build Pulsenote, a transactional email API that handles DKIM signing, bounce/complaint suppression and delivery webhooks so the above is mostly configuration rather than code — free tier is 100 emails/month, no card. Everything in this article applies identically whether you use it, a competitor, or your own Postfix box; the DNS is yours either way.