DEV Community: Email Campaign LLC

Email deliverability in CI: what you can actually automate, and what you can't

Email Campaign LLC — Sat, 19 Sep 2026 20:20:30 +0000

"Did that email arrive?" is two questions wearing one coat. One of them automates cleanly. The other does not automate at all, and most of the frustration around deliverability testing comes from not separating them.

Did we send a well-formed, authenticated message? Deterministic. Test it in CI.
Did a receiver put it in the inbox? A judgement made by someone else's classifier, using your history. Not a test. A measurement, and a lagging one.

Automating the first is worth doing and is mostly ignored. Trying to automate the second produces a suite that fails for reasons nobody can action.

The part that belongs in CI

DNS invariants

These are pure functions of your DNS, so they can assert:

def test_exactly_one_spf_record(resolver, domain):
    txt = resolver.resolve(domain, "TXT")
    spf = [r for r in txt if r.to_text().strip('"').startswith("v=spf1")]
    assert len(spf) == 1, f"{domain} publishes {len(spf)} SPF records; must be 1"

def test_spf_lookup_budget(resolver, domain):
    used = count_lookups(domain, resolver)   # walks includes recursively
    assert used <= 10, f"{domain} uses {used} of 10 DNS lookups"

The second one is the test that earns its keep. The lookup limit counts the whole include tree, so a record that passes today breaks when a vendor adds an include inside their record — a change you will never be notified about. That is exactly the class of failure a scheduled job should catch and a human never will.

Run these nightly against production DNS, not just on deploy. The failure arrives on a day you did not ship anything.

Message structure

Assert on your own output before it goes anywhere:

A List-Unsubscribe header, and List-Unsubscribe-Post for one-click
A Message-ID with your sending domain in it
A plain-text alternative that is not empty and not just stripped HTML
From: aligning with the domain you intend to sign with
No unsubscribe link pointing at localhost — this ships more often than anyone admits

Authentication, end to end

The strongest automated check sends one real message to a mailbox you control, then reads the Authentication-Results header the receiver stamped on it:

Authentication-Results: mx.google.com;
  spf=pass smtp.mailfrom=yourdomain.com;
  dkim=pass header.i=@yourdomain.com;
  dmarc=pass header.from=yourdomain.com

Assert all three pass — and assert on the d= / header.i= value, not just the verdict. dkim=pass with your ESP's domain there is a pass that aligns with nothing, and DMARC fails anyway. That assertion catches a misconfiguration that every "is DKIM working?" check reports as fine.

The part that does not belong in CI

Inbox placement. Whether Gmail files you under Primary, Promotions or Spam depends on your domain's history, the recipient's own behaviour, and a classifier that changes without notice. Seed-list tools sample it, and the sample is drawn from mailboxes with no real engagement history — which is precisely the input the classifier cares most about. Useful as a trend. Not a pass/fail gate.

Reputation. A lagging aggregate. There is no assertion to write.

Spam-word scoring. Content is real but it is the smallest factor, and the scores are not the receiver's scores. A well-written message from an unauthenticated domain still lands in spam, which is why rewriting subject lines is the first thing people try and the last thing that helps.

Gate your build on none of these. Put them on a dashboard with a date on it.

Dev and staging, without becoming a spammer

The cheapest mistake in this whole area is a staging environment that can reach the open internet.

Point non-production at a catching SMTP server — MailHog, Mailpit, or your provider's sandbox. Make it the default, so reaching real recipients requires an explicit override rather than the reverse.
If production credentials exist anywhere in a non-production environment, one seeded fixture with real addresses is all it takes. Do not rely on the seed data being fake.
Never warm a domain with traffic generated by tests. Warm-up is a claim about human sending behaviour; synthetic volume against seed mailboxes teaches the receiver something, and it is not what you want it to learn.

A shape that works

Stage	Frequency	Gates the build?
DNS invariants — one SPF record, ≤10 lookups, DKIM resolves, DMARC present	Nightly + deploy	Yes
Message structure assertions	Every commit	Yes
Live send → assert `Authentication-Results` incl. `d=` alignment	Nightly	Yes
Bounce-code classification tests (fixtures)	Every commit	Yes
Inbox placement sample	Weekly	No — trend only
Complaint rate vs the 0.1% threshold	Continuous	Alert, not a gate

Everything in the top four is a property of your own system and fails for reasons you can fix. Everything below is somebody else's verdict on your history, and belongs on a graph rather than in a red build.

If you want the DNS half checked right now without wiring anything up, this runs the same lookups against live DNS, and spf-audit is the CI-shaped version — it walks the full include tree and exits non-zero over the limit.

One domain per job: why password resets should not share a reputation with outreach

Email Campaign LLC — Sat, 19 Sep 2026 20:17:36 +0000

Most teams send every kind of mail from one domain, because that is the obvious thing to do and nothing appears to go wrong. Then a marketing send does badly, and two weeks later password reset emails start landing in spam — and nobody connects the two, because those systems share no code, no queue, and no vendor.

They share the only thing that mattered: the domain in the From: header.

What reputation is attached to

Receivers score several identities independently: the sending IP, the From: domain, the DKIM signing domain (d=), and the envelope sender's domain. The one you control most cheaply, and the one that follows you across every provider, is the domain.

That domain accumulates one history. Your password resets — near-100% open rate, effectively zero complaints, universally wanted — contribute to it. So does the cold campaign that went to a list somebody bought in 2023. The receiver does not know which system emitted which message. It sees one domain with one blended history.

This is why the failure is always asymmetric. Transactional mail is the best-behaved traffic you have, and it is the thing that gets hurt, because it is the thing that could least afford the hit.

The split

Separate by risk profile, not by team:

Domain	Traffic	Risk
`yourcompany.com`	Nothing. Corporate mail only.	Protect at all costs
`mail.yourcompany.com`	Transactional: resets, receipts, alerts	Must never fail
`news.yourcompany.com`	Marketing to opted-in lists	Moderate
`a-separate-domain.com`	Cold outreach	Expendable

The first three are subdomains, and that is deliberate — subdomains inherit some of the organizational domain's standing, which is an advantage when the parent is healthy.

Cold outreach is the exception. It gets a separate registered domain, not a subdomain, precisely because you do not want inheritance running in the other direction. Cold outreach is the highest-variance traffic you will ever send, and the whole point is that when a domain burns, it burns alone.

The tag that decides whether this works

Splitting into subdomains does almost nothing unless you publish sp= on the organizational domain.

v=DMARC1; p=quarantine; sp=quarantine; rua=mailto:dmarc@yourcompany.com; pct=100

p= applies to the organizational domain. sp= applies to subdomains, and when it is absent, subdomains inherit p=. Two consequences people discover the hard way:

If you enforce p=reject without thinking about subdomains, every subdomain inherits reject — including ones you forgot you had, and including any that are not fully authenticated yet.
Conversely, sp=none is a deliberate escape hatch. It lets you enforce the brand domain while leaving one lagging subdomain at monitoring, so a single unfinished migration does not hold up enforcement everywhere else.

Both are useful. Neither happens by accident.

Each domain authenticates on its own

Subdomains do not inherit SPF or DKIM. mail.yourcompany.com needs its own v=spf1 TXT record and its own DKIM keys — inheritance applies to DMARC policy, not to authentication records.

Some common ways this goes wrong:

A second SPF record. One domain may publish exactly one v=spf1 record. Two are a permanent error, not a merge, and both look valid inspected separately. This happens most often when a website host and a mail provider each add one, months apart.
The ten-lookup ceiling. Every include, a, mx, ptr, exists and redirect costs a DNS lookup, including the ones nested inside your includes. include:_spf.google.com alone is four. Four vendors will break a record that still looks short.
DKIM signing with the wrong domain. If d= is your ESP's domain rather than yours, the signature verifies and DMARC still fails, because nothing aligned.

Worth checking the whole tree rather than the first record: SPF, DKIM and DMARC checker, or spf-audit if you would rather run it in CI.

New domains start with no history, which is not the same as good history

A freshly registered outreach domain has no reputation at all, and receivers treat absence of history with suspicion rather than neutrality. It needs two to three weeks of gradually increasing volume before it carries real traffic.

The part that catches people: warm-up traffic and real traffic come out of the same daily budget. A mailbox warming at twenty a day that also sends thirty real messages is sending fifty from a mailbox with no history, and it is not warming — it is being pushed past what its history supports while a dashboard reports progress.

Migrating without a gap

If you are on one domain today, do not cut over:

Stand up the new subdomain with its own SPF, DKIM and DMARC, and confirm it aligns by reading the Authentication-Results header on a real test message.
Move transactional first — it is your most valuable and best-behaved traffic, so it establishes the new subdomain's history quickly and honestly.
Move marketing next, and watch complaint rate as a leading indicator.
Move cold outreach to its own registered domain last, after warm-up.

Keeping the old domain sending throughout is fine. What you are building is history on the new names, and that takes weeks whatever order you do it in.

The reason to start now rather than when something breaks: reputation recovers slowly, and separation only helps traffic you moved before the damage.

Your bounce handler is probably suppressing the wrong address

Email Campaign LLC — Sat, 19 Sep 2026 20:14:55 +0000

Here is a bug I have now seen in four separate codebases, including one I wrote.

Mail goes out. A bounce comes back. The handler parses the status code, sees a 5.x.x, concludes the address is bad, and adds the recipient to the suppression list. Clean, obvious, and wrong for an entire class of bounce.

Because not every 5.x.x is about the recipient. Some of them are about you.

Two families, one code space

550 5.1.1 The email account that you tried to reach does not exist
550 5.7.1 Our system has detected that this message is likely unsolicited

Both are permanent. Both are 550. Suppressing on the first is correct — that mailbox is not there and will not be there tomorrow.

Suppressing on the second is a mistake that compounds. 5.7.x is a policy rejection: the receiver evaluated your IP, your domain, your authentication or your reputation, and declined. The recipient is fine. Their mailbox exists. They might be your best customer.

What you have actually learned is that you cannot currently deliver to that provider. Recording it against the recipient throws away the real signal and permanently burns an address you never had a problem with.

The sub-code is the part that matters

The three-digit code tells you almost nothing. The enhanced status code after it — RFC 3463 — tells you who the problem belongs to.

Class	Meaning	Suppress the recipient?
`5.1.x`	Addressing. No such user, bad syntax.	Yes
`5.2.1`	Mailbox disabled	Yes
`5.2.2`	Mailbox full	No — this is temporary in practice, retry later
`5.4.x`	Routing / network	No
`5.7.x`	Policy, authentication, reputation	No — this is about you

5.2.2 deserves a note of its own. It is formatted as permanent and behaves as temporary, because mailboxes get emptied. Treating a full mailbox as a dead address is a slow, invisible leak from your list.

What to do with a 5.7.x instead

The useful unit is not the recipient, it is the (sending identity, receiving provider) pair.

Keep a separate ledger of policy refusals recording the sending domain or IP, the receiving provider, the code, and the raw text. Then act on aggregates rather than individual events:

Rising 5.7.x from one provider against one sending domain means stop sending to that provider from that domain and fix the cause.
The same code across every provider means the problem is the domain or the IP, not any single relationship.
A provider-specific code — Google's 5.7.26, for instance, which means unauthenticated — tells you precisely what is broken. 5.7.26 is an authentication failure, not a reputation verdict, and it is fixable in DNS this afternoon.

The practical shape is an automatic hold: when policy refusals from one provider cross a threshold, pause sending to that provider from that sender, and keep the recipients untouched. Delivery resumes when the cause is fixed. Nothing permanent was recorded about anyone.

Parse the enhanced code properly

The enhanced status code is not always where you expect it, and plenty of servers put a different code in the text than in the status field.

import re

ENHANCED = re.compile(r"\b([245])\.(\d{1,3})\.(\d{1,3})\b")

def classify(smtp_response: str):
    m = ENHANCED.search(smtp_response)
    if not m:
        # fall back to the basic code; 5xx without an enhanced code
        # is genuinely ambiguous — do not guess
        return "unknown"
    cls, subject, _detail = m.group(1), m.group(2), m.group(3)
    if cls != "5":
        return "transient"
    if subject == "1":
        return "bad_recipient"     # safe to suppress
    if subject == "7":
        return "policy"            # about the SENDER — never suppress
    return "other"

Two things that save you later: search the whole response, not just the first line, because multi-line replies routinely carry the enhanced code on a continuation line. And when there is no enhanced code at all, record unknown rather than defaulting to "bad recipient" — a bare 550 from an unfamiliar server is not enough to destroy an address over.

Why this stays invisible

Nothing about this bug looks like a bug. Your bounce rate goes down after you start suppressing policy refusals, because you stop retrying addresses that were refusing you. The list gets smaller and cleaner-looking. Meanwhile the actual problem — an authentication failure, a reputation issue, a provider that has decided something about your domain — is being quietly converted into permanent suppressions and never surfaced to anyone.

If you want to know whether the cause is authentication rather than reputation, that part is checkable in about thirty seconds: SPF, DKIM and DMARC against live DNS. Authentication problems are the cheap ones. Reputation is the expensive one, and no DNS change fixes it.

DMARC aggregate reports are XML, in a gzip, in an email. Nobody reads them.

Email Campaign LLC — Sat, 19 Sep 2026 20:08:59 +0000

Publishing a DMARC record with rua= is the easy part. You add the TXT record, reports start arriving, and then you discover what a report actually is: a gzipped XML attachment on an email, sent daily by every receiver that got mail claiming to be from your domain.

So the reports pile up in a mailbox nobody opens, the domain stays at p=none forever, and the record that was supposed to stop people spoofing you protects nothing at all. Monitoring mode is not a policy. It is the absence of one.

What is actually in the file

Under the gzip it is a <feedback> document. The parts that matter:

<policy_published>
  <domain>example.com</domain>
  <p>none</p>
  <adkim>r</adkim>
  <aspf>r</aspf>
</policy_published>
<record>
  <row>
    <source_ip>203.0.113.10</source_ip>
    <count>12</count>
    <policy_evaluated>
      <dkim>pass</dkim>
      <spf>pass</spf>
    </policy_evaluated>
  </row>
  <identifiers>
    <header_from>example.com</header_from>
  </identifiers>
  <auth_results>
    <dkim><domain>example.com</domain><result>pass</result></dkim>
    <spf><domain>example.com</domain><result>pass</result></spf>
  </auth_results>
</record>

Two things about this structure trip people up.

policy_evaluated is not auth_results. The first is alignment, the second is raw authentication. You can have auth_results showing dkim=pass and policy_evaluated showing dkim=fail, and that combination is common: the message was validly signed, just by somebody else's domain. Your ESP signs with d=esp.example.net, the signature verifies perfectly, and it aligns with nothing. DMARC fails.

If you only read auth_results, every one of those looks fine.

count is not 1. Each <record> is an aggregate over a source IP, so a single record can represent thousands of messages. Summing records instead of counts will quietly understate the traffic.

The question you are actually asking

Nobody reads these reports for fun. There is exactly one question: can I move off p=none without quarantining my own mail?

That reduces to: is every source sending as my domain aligned, and if not, which ones are not, and are they mine?

So I wrote the smallest thing that answers it:

$ dmarc_report.py ~/Maildir/new ~/Maildir/cur

Domain        : example.com
Published p=  : none
Reporters     : google.com
Reports read  : 3  (13 messages)

SOURCE IP        ALIGNED  FAILING  HEADER FROM
203.0.113.199          0        1  mail-1.example.com
                 why: SPF none; no DKIM signature
203.0.113.10          12        0  example.com

Aligned: 12 of 13 (92.3%)

VERDICT: 1 message(s) did not align. Identify each source above
         before enforcing, or you will quarantine your own mail.
         A subdomain in HEADER FROM can be spared with sp=none.

It reads .xml, .gz, .zip, a raw .eml, or a whole Maildir, and it is Python standard library only — no dependencies, no network calls, nothing uploaded anywhere. That last point matters more than it sounds: DMARC reports name every IP that sends as you, which is an infrastructure map you probably do not want to paste into a web form.

github.com/coldemailmarketing/dmarc-report-parser — MIT.

Two details worth stealing even if you write your own

Deduplicate by report_id. The same report routinely arrives twice. If you are counting messages to decide whether enforcement is safe, duplicates inflate your confidence in exactly the wrong direction.

Treat a subdomain failure differently from an organizational-domain failure. Subdomains inherit the organizational policy unless you publish an sp= tag. So a stray subdomain failing alignment does not have to block enforcement on your brand domain — you enforce p=quarantine and set sp=none, then deal with the subdomain separately.

That one distinction is the difference between enforcing this month and enforcing next year.

The limit of the whole exercise

A verdict of "safe to enforce" is only as good as the sample. Aggregate reports cover mail that receivers chose to report on. If your invoicing runs through a vendor whose mail happened not to reach a reporting receiver that week, it will not appear, and it will not be in the verdict.

Before you enforce, write down every system that puts your domain in a From: header — app, billing, helpdesk, newsletter, anything a vendor sends on your behalf — and confirm each one appears in the reports as aligned. The parser tells you what the reports say. It cannot tell you what they left out.

If you want the records themselves checked against live DNS rather than after the fact, this does SPF, DKIM and DMARC in one pass — it counts every v=spf1 record on the host rather than the first one, which is its own separate trap.

SPF permerror: the two failures that look exactly like a working record

Email Campaign LLC — Sat, 19 Sep 2026 15:14:46 +0000

Your SPF record can be syntactically perfect, published on the right host, and still fail every check a receiving mail server runs. I have watched it take down three sending domains at once. The record looked correct in the registrar dashboard, it looked correct in three online validators, and mail was being refused anyway.

Here are the two failures behind almost every "but the record is right there" case I have investigated, and how to actually detect them.

1. The ten-lookup limit, which nothing warns you about

RFC 7208 caps SPF evaluation at ten DNS lookups. Every include, a, mx, ptr, exists and redirect mechanism costs one, and includes nest, so the cost of your record is the cost of the whole tree it pulls in.

Cross the limit and the result is permerror. That is not a soft failure or a partial pass. It means the receiving system could not evaluate SPF at all, and most receivers treat that the same way they treat an outright fail.

This is a nasty bug class for three reasons:

It is invisible in the record. Your TXT record is 120 characters and looks fine. The eleventh lookup is three levels down inside somebody else's include.
It appears without you touching anything. A provider adds an include to their own record, your tree grows, and a record that worked for a year starts failing on a Tuesday.
It gets worse as you add vendors. One provider is usually fine. The third one is where teams land.

Counting properly means walking the tree, not counting the includes you can see:

import dns.resolver

MECHANISMS = ("include:", "a:", "mx:", "ptr:", "exists:", "redirect=")

def spf_of(domain):
    for r in dns.resolver.resolve(domain, "TXT"):
        txt = b"".join(r.strings).decode()
        if txt.lower().startswith("v=spf1"):
            return txt
    return None

def count(domain, seen=None, depth=0):
    """Return the number of DNS lookups an SPF evaluation costs."""
    if seen is None:
        seen = set()
    if domain in seen or depth > 10:
        return 0
    seen.add(domain)
    record = spf_of(domain)
    if not record:
        return 0
    total = 0
    for term in record.split():
        low = term.lower()
        for m in MECHANISMS:
            if low.startswith(m) or low in ("a", "mx", "ptr"):
                total += 1                      # this mechanism costs one
                target = term.split(":", 1)[-1].split("=")[-1]
                if low.startswith(("include:", "redirect=")):
                    total += count(target, seen, depth + 1)
                break
    return total

print(count("example.com"))   # 11 or more means permerror

Two things to note. ptr is one lookup and you should not be using it at all. And the seen set matters, because providers do occasionally publish records that point at each other.

The fix is flattening or consolidation, not "add one more include". If you must flatten, automate the re-flattening, because you have now taken on the job of noticing when your provider changes their IP ranges.

2. Two SPF records, which is an instant fail

This one is simpler and even more common. Publishing two TXT records that both start v=spf1 on the same host is a permerror by definition. RFC 7208 is explicit: more than one record means the check terminates.

It happens because each record arrives from a different place and each is individually correct. The web host gives you one during setup. The mail provider gives you another. Neither knows about the other, and the registrar interface happily shows both.

Detection is one query, and the trap is that you have to count records rather than read the first one:

dig +short TXT example.com | grep -ci "^\"v=spf1"
# anything other than 1 is a problem

Note the -c. A checker that reads the first matching record and validates it will tell you everything is fine. We had exactly that bug in our own tooling, which is a large part of why I am writing this: our checker took the first v=spf1 it found and never counted the rest, and never counted lookups either. Both are now counted, and a sweep of 274 domains after the fix came back clean.

The fix is a merge, not a delete. One record, one v=spf1, every include inside it, ending in ~all:

v=spf1 include:_spf.google.com include:spf.protection.outlook.com ~all

Use ~all rather than -all on a sending domain. A soft fail leaves room for forwarding paths that legitimately break SPF, and receivers weight DMARC alignment more heavily anyway.

A third one, if you are on Microsoft 365

Slightly different shape, same class of problem: the record exists, looks right, and does nothing.

Microsoft 365 DKIM uses two CNAMEs, on selector1._domainkey and selector2._domainkey, pointing at a host inside your tenant's onmicrosoft.com domain. Almost every guide prints that target as a fixed pattern. It is not fixed. The host contains a per-tenant value, so a CNAME copied from a blog post, or from a tenant you configured last month, resolves to nothing at all.

The symptom is the worst kind: no error anywhere. The records are published, the portal shows them, and DKIM simply never signs.

Read the target from the tenant you are actually configuring, then confirm the provider reports signing rather than record present. Those are different states, and only one of them puts a d= header on your mail.

Verify against a resolver, not a dashboard

Everything above shares one root cause: the registrar dashboard shows what you saved, and the mail provider sees what a public resolver serves. Those are different things, and the gap between them is where silent failures live. A zone that did not reload. A record saved on the wrong host. A CNAME flattened by a proxy. An apex record the registrar quietly rewrote.

So treat a record as unverified until a resolver outside your own account returns it:

dig TXT example.com @1.1.1.1
dig TXT selector1._domainkey.example.com @1.1.1.1
dig TXT _dmarc.example.com @1.1.1.1

Then send one real message to a mailbox at a different provider and read the Authentication-Results header. A test tool proves the record. A real message proves the path.

I write about this because it is the layer underneath cold email, and it is the layer that decides whether anything else you do matters. If you want the longer version, with DMARC alignment and the order to write the records in, it is here: SPF, DKIM and DMARC for cold email. There is also a free checker that counts lookups and duplicate records rather than reading the first one it finds: emailcampaign.ai/tools/dns-checker.

Disclosure: I build emailcampaign.ai, which provisions and authenticates sending infrastructure. The bug in our own checker described above was real and is fixed.