Muhammad Ali Mughal

Posted on Nov 10

Email That Never Landed: SPF/DKIM/DMARC Drift & Bounce Pipelines

#email #devops #laravel #dns

The support ticket said: "I never got the password reset email." Then three more like it. Our mail provider's dashboard showed every message as "delivered". The application logs showed every send as successful. What had actually happened was a marketing team adding a new newsletter tool to the SPF record two weeks earlier. The record now needed eleven DNS lookups. The limit is ten. SPF started returning a permanent error, DMARC failed for messages that did not also pass DKIM alignment, and a large mailbox provider began putting our resets in spam. Nothing in our stack raised an alert.

I wrote this with help from AI tools and checked every DNS record, command and code sample myself. I am a Principal Software Engineer in Karachi with 9+ years across PHP/Laravel, Vue, Node and .NET microservices, and at Loop Origin we set up and maintain servers and SaaS backends where transactional email is part of the critical path. This is Part 3 of the series Silent Failures in Production SaaS. Part 1 covered cron jobs that never ran and Part 2 covered webhooks that stopped arriving. Email is the outbound version of the same problem: "sent" is not the same as "received".

📚 Series Roadmap — Silent Failures in Production SaaS

Part 1: Cron Jobs That Never Ran: Heartbeats & Dead-Man Switches for SaaS

Part 2: Webhooks That Stopped Arriving: Gap Detection & Reconciliation

Part 3 (This Post): Email That Never Landed: SPF/DKIM/DMARC Drift & Bounce Pipelines

Part 4: Config & Feature Flags That Quietly Diverged Across Environments

Part 5: Backups That Never Restored: Automated Restore Drills

Part 6: Synthetic Checks & SLOs for the Paths Nobody Clicks

Related earlier posts: Production Observability on a Budget · Subdomain Takeover & Dangling DNS Reconnaissance · Observability for Async Work: Correlation IDs & Job Traces

How do you catch transactional email that silently stops landing?

Direct answer: Monitor DNS authentication records on a schedule and alert when SPF, DKIM or DMARC change or fail validation. Read DMARC aggregate reports to see real pass rates by sending source. Feed provider bounce and complaint events into a suppression table, and run a synthetic send to a seed inbox that checks the message arrived and authenticated.

Why email failures do not look like failures

Your application hands a message to an SMTP server or an API, gets a 200 or a 250 OK, and moves on. From that point everything that can go wrong happens outside your logs:

SPF lookup limit exceeded. Each include, a, mx, ptr, exists and redirect costs a DNS lookup, including nested ones. More than ten gives permerror.
Two SPF records. Someone adds a second v=spf1 TXT record instead of editing the first. That is also a permerror.
DKIM key missing after a DNS migration. Moving zones between providers often drops the long selector TXT or CNAME records.
DKIM selector rotated at the provider while your delegated CNAME still points at the old key.
Alignment broken. A new service sends as billing@yourapp.com but signs with its own domain, so DMARC fails even though SPF and DKIM each "pass".
Bounces ignored. Hard-bouncing addresses keep getting mail, your reputation drops, and good messages start landing in spam.

Since early 2024, Gmail and Yahoo require bulk senders to authenticate with SPF and DKIM, publish DMARC, support one-click unsubscribe on marketing mail and keep spam complaint rates low, and Microsoft has introduced similar rules for Outlook consumer domains. Authentication drift that used to cost a bit of inbox placement now gets mail rejected.

Step 1: know what good looks like

Start with a quick manual audit:

DOMAIN=yourapp.com
dig +short TXT "$DOMAIN" | grep -i 'v=spf1'
dig +short TXT "_dmarc.$DOMAIN"
# One query per DKIM selector your providers use
for s in s1 s2 resend google; do
  printf '%-8s ' "$s"; dig +short TXT "$s._domainkey.$DOMAIN" | head -c 80; echo
done

A healthy baseline looks something like:

yourapp.com.          TXT "v=spf1 include:amazonses.com include:_spf.google.com -all"
_dmarc.yourapp.com.   TXT "v=DMARC1; p=quarantine; rua=mailto:dmarc@yourapp.com; adkim=r; aspf=r"
s1._domainkey.yourapp.com. CNAME s1.domainkey.u123.wl.sendgrid.net.

Then count SPF lookups recursively. This small script walks includes and redirects:

# spf_lookups.py  -- pip install dnspython
import sys, dns.resolver

def spf(domain):
    for r in dns.resolver.resolve(domain, 'TXT'):
        txt = b''.join(r.strings).decode()
        if txt.lower().startswith('v=spf1'):
            return txt
    return ''

def count(domain, seen=None):
    seen = seen or set()
    n = 0
    for term in spf(domain).split()[1:]:
        t = term.lstrip('+-~?').lower()
        if t.startswith(('include:', 'redirect=')):
            target = t.split(':', 1)[-1].split('=', 1)[-1]
            n += 1
            if target not in seen:
                seen.add(target)
                n += count(target, seen)
        elif t.startswith(('a', 'mx', 'ptr', 'exists:')) and not t.startswith('all'):
            n += 1
    return n

d = sys.argv[1]
total = count(d)
print(f"{d}: {total} DNS lookups", "OK" if total <= 10 else "PERMERROR RISK")

Step 2: monitor records for drift

Run the audit on a schedule and compare against a stored baseline. Any change should notify a human, because a legitimate change still needs a review of alignment and lookup counts. In Laravel:

// app/Console/Commands/CheckMailDns.php
public function handle(): int
{
    $domain = config('mail.auth_domain');
    $checks = [
        'spf'   => $this->txt($domain, 'v=spf1'),
        'dmarc' => $this->txt("_dmarc.$domain", 'v=DMARC1'),
        'dkim'  => collect(config('mail.dkim_selectors'))
            ->mapWithKeys(fn ($s) => [$s => (bool) dns_get_record("$s._domainkey.$domain", DNS_TXT | DNS_CNAME)])
            ->all(),
    ];

    $hash = hash('sha256', json_encode($checks));
    $previous = Cache::get('mail_dns_hash');
    $missing = collect($checks['dkim'])->filter(fn ($ok) => ! $ok)->keys();

    if (! $checks['spf'] || ! $checks['dmarc'] || $missing->isNotEmpty() || ($previous && $previous !== $hash)) {
        Log::critical('mail_dns_drift', ['checks' => $checks, 'missing_dkim' => $missing]);
        // notify on-call channel here
    }
    Cache::forever('mail_dns_hash', $hash);
    Http::get(config('services.heartbeats.mail_dns')); // dead-man switch from Part 1
    return self::SUCCESS;
}

private function txt(string $host, string $prefix): ?string
{
    foreach (dns_get_record($host, DNS_TXT) ?: [] as $r) {
        if (str_starts_with(strtolower($r['txt']), strtolower($prefix))) return $r['txt'];
    }
    return null;
}

Schedule it hourly. The heartbeat ping at the end means you also hear about it when the check itself stops running.

Step 3: read DMARC aggregate reports

The rua address in your DMARC record receives daily XML reports from large mailbox providers. Each row says which IP sent mail as your domain, how many messages, and whether SPF and DKIM passed and aligned. This is the only place you see what receivers actually observed.

You can parse them yourself or use a DMARC reporting service. Either way, alert on:

Pass rate for a known sending source dropping below its normal level.
A new source IP sending volume as your domain.
dkim=fail for a provider that previously passed, which usually means a missing or rotated key.

Move from p=none to p=quarantine and then p=reject only after reports show every legitimate source aligned.

Step 4: build a bounce and complaint pipeline

Every serious provider can send bounce and complaint events by webhook or a notification topic. Treat them like the webhook receipts from Part 2: verify, store, process.

Schema::create('email_suppressions', function (Blueprint $table) {
    $table->id();
    $table->string('email')->unique();
    $table->string('reason', 32);        // hard_bounce | complaint | manual
    $table->string('provider', 32);
    $table->timestampTz('suppressed_at')->useCurrent();
});

Hard bounces and complaints go straight into the table. Soft bounces only after repeated failures over several days. Then block sends at the framework level, so every mailable respects it:

// app/Listeners/BlockSuppressedRecipients.php
use Illuminate\Mail\Events\MessageSending;

public function handle(MessageSending $event): bool
{
    $to = collect($event->message->getTo())->map->getAddress();
    if (DB::table('email_suppressions')->whereIn('email', $to)->exists()) {
        Log::info('mail_suppressed', ['to' => $to->all()]);
        return false; // returning false cancels the send
    }
    return true;
}

Track bounce and complaint rates per day as metrics. A sudden jump in hard bounces is often a signup form being abused, and a jump in complaints is often one template going to the wrong list.

Step 5: synthetic sends prove delivery

Finally, close the loop. A scheduled job sends a real password reset to seed inboxes at major providers. A second job reads those inboxes over IMAP or an API, checks the message arrived within a few minutes, and parses the Authentication-Results header for spf=pass, dkim=pass and dmarc=pass. Missing or failing messages page someone.

Failure signals compared

Signal	Catches	Delay	Effort	Blind spots
Provider "delivered" status	Server-to-server handoff	Seconds	None	Spam folder placement
Scheduled DNS drift check	SPF/DKIM/DMARC record changes	Up to 1 hour	Low	Receiver-side policy changes
DMARC aggregate reports	Real pass/fail by source	~24 hours	Medium	Small receivers that do not report
Bounce and complaint events	Bad addresses, reputation risk	Minutes	Medium	Silent spam filtering
Seed inbox synthetic send	End-to-end inbox delivery	Minutes	Medium	Per-user filters
User support tickets	Everything, eventually	Days	None	Customer trust

When we add this to a client system at Loop Origin, the order is: audit and baseline the DNS records, wire the drift check with a heartbeat, route DMARC reports somewhere readable, then add suppressions and seed tests for the templates that matter most. Our server setup and maintenance work covers DNS, mail authentication and monitoring, and you can contact us for a deliverability audit.

Diagnostic checklist

[ ] Exactly one SPF record, with ten or fewer DNS lookups including nested includes.
[ ] Every sending service signs DKIM with a key published under your domain.
[ ] DMARC is published with an rua address someone actually reads.
[ ] From domains align with SPF or DKIM domains for every source.
[ ] A scheduled job checks DNS records and alerts on any change.
[ ] The DNS check itself sends a heartbeat.
[ ] Bounce and complaint events are verified, stored and processed.
[ ] A suppression check runs before every send.
[ ] Bounce and complaint rates are tracked per day.
[ ] Seed inbox tests confirm arrival and dmarc=pass for critical templates.
[ ] DNS migrations have a mail-records step in their runbook.

FAQ

Should SPF end with -all or ~all?

With DMARC enforcement, receivers mostly act on the DMARC result, so either works. Use -all once you are confident every sender is listed.

How often should DKIM keys rotate?

Many teams rotate every six to twelve months. Use 2048-bit keys and keep the old selector published until mail signed with it has stopped arriving.

Do I need DMARC for transactional mail only?

Yes. Major mailbox providers expect it, and it is the only feedback channel that shows how receivers judge your domain.

Should marketing and transactional mail share a domain?

Use a subdomain for marketing, such as news.yourapp.com. Its reputation problems then hurt password resets less.

What about MTA-STS and TLS-RPT?

They protect and report on inbound TLS for mail sent to you. Worth adding, but they will not fix outbound authentication drift.

Can I just retry failed sends?

Retry transient failures, never hard bounces. Retrying bad addresses damages reputation for everyone on your domain.

Next in this series, Part 4 covers config and feature flags that quietly diverged across environments.

DEV Community