You built a SaaS app. Users sign up. Your app sends a welcome email. It goes to spam.
You google "email deliverability" and every result says "set up SPF and DKIM." Great. But none of them tell you how this actually works under the hood, what your code needs to do, or why your perfectly authenticated emails still end up in the junk folder.
This is the guide I wish I had when I first started building email sending systems. We're going deep on the implementation side — DNS record construction, SMTP handshake mechanics, bounce processing pipelines, and the code that ties it all together.
How Email Authentication Actually Works (Under the Hood)
Most developers know they need SPF, DKIM, and DMARC. Fewer understand the actual protocol-level flow. Here's what happens when your server sends an email to Gmail:
Your Server (MTA) Gmail MX Server
│ │
│── EHLO mail.yourdomain.com ───────────────>│
│<─ 250-mx.google.com at your service ───────│
│── MAIL FROM:<bounce@yourdomain.com> ──────>│
│<─ 250 OK ──────────────────────────────────│
│── RCPT TO:<user@gmail.com> ────────────────>│
│<─ 250 OK ──────────────────────────────────│
│── DATA ────────────────────────────────────>│
│── [headers + body with DKIM signature] ───>│
│── . ────────────────────────────────────────>│
│ │
│ Gmail now checks: │
│ 1. SPF: Is sending IP in DNS record? │
│ 2. DKIM: Does signature verify? │
│ 3. DMARC: Do SPF/DKIM align with From? │
│ 4. IP reputation lookup │
│ 5. Content analysis │
│ │
│<─ 250 2.0.0 OK (queued) ───────────────────│
The critical thing: authentication checks happen at the receiving end. Your job as the sender is to make sure the DNS records and cryptographic signatures are there when the receiving server goes looking for them.
SPF: The 10-Lookup Trap
SPF seems simple until you hit the 10 DNS lookup limit. Every include:, a:, mx:, and redirect= counts as a lookup. And include: chains recursively — if _spf.google.com includes two more records, those count against your limit too.
Here's how to audit your SPF lookup count programmatically:
import dns.resolver
def count_spf_lookups(domain, depth=0):
if depth > 10:
return 0, ["ERROR: Exceeded 10-lookup limit"]
lookups = 0
issues = []
try:
answers = dns.resolver.resolve(domain, 'TXT')
for rdata in answers:
txt = rdata.to_text().strip('"')
if not txt.startswith('v=spf1'):
continue
mechanisms = txt.split()
for mech in mechanisms:
if mech.startswith(('include:', 'a:', 'mx:', 'redirect=')):
lookups += 1
target = mech.split(':', 1)[-1].split('=', 1)[-1]
sub_lookups, sub_issues = count_spf_lookups(
target, depth + 1
)
lookups += sub_lookups
issues.extend(sub_issues)
elif mech == 'a' or mech == 'mx':
lookups += 1
except Exception as e:
issues.append(f"DNS error for {domain}: {e}")
return lookups, issues
lookups, issues = count_spf_lookups('yourdomain.com')
print(f"Total SPF lookups: {lookups}")
if lookups > 10:
print("WARNING: Exceeds 10-lookup limit — SPF will permerror")
for issue in issues:
print(f" - {issue}")
When you exceed 10 lookups, SPF returns a permerror — which most receivers treat as a fail. The fix is SPF flattening: resolve the include: chains to their underlying IP ranges and hardcode them.
# Before flattening (12 lookups):
v=spf1 include:_spf.google.com include:sendgrid.net include:mailgun.org
include:amazonses.com include:spf.protection.outlook.com ~all
# After flattening (0 lookups, but static IPs):
v=spf1 ip4:209.85.128.0/17 ip4:74.125.0.0/16 ip4:167.89.0.0/17
ip4:168.245.0.0/17 ip4:198.2.128.0/18 ~all
The downside: flattened records break when providers change their IP ranges (and they do). You need a cron job or service to re-flatten periodically.
DKIM: Signing Emails in Code
DKIM is an RSA (or Ed25519) signature over selected email headers and the body. If you're building your own sending infrastructure, here's what the signing process looks like:
const crypto = require('crypto');
function signEmail(headers, body, privateKey, selector, domain) {
// Canonicalize body (simple or relaxed)
const canonBody = body.replace(/\r?\n/g, '\r\n').trimEnd() + '\r\n';
const bodyHash = crypto
.createHash('sha256')
.update(canonBody)
.digest('base64');
// Build the DKIM-Signature header (without b= value)
const dkimFields = [
`v=1`,
`a=rsa-sha256`,
`c=relaxed/relaxed`,
`d=${domain}`,
`s=${selector}`,
`h=from:to:subject:date:message-id`,
`bh=${bodyHash}`,
`b=`
];
const dkimHeader = `DKIM-Signature: ${dkimFields.join('; ')}`;
// Canonicalize headers for signing
const signedHeaders = ['from', 'to', 'subject', 'date', 'message-id'];
let headerBlock = signedHeaders
.map(h => {
const value = headers[h];
return `${h}:${value.trim()}`;
})
.join('\r\n');
headerBlock += '\r\n' + `dkim-signature:${dkimFields.join('; ')}`;
// Sign with RSA private key
const signer = crypto.createSign('RSA-SHA256');
signer.update(headerBlock);
const signature = signer.sign(privateKey, 'base64');
return dkimHeader.replace('b=', `b=${signature}`);
}
In practice, you won't hand-roll DKIM signing — libraries like nodemailer or mailcomposer handle it. But understanding the mechanism matters when you're debugging signature failures.
Common DKIM failures:
-
Body modified in transit. If any relay server modifies the body (adding a footer, rewriting URLs), the body hash won't match. Use
l=(body length tag) to limit the signed body region, or use relaxed canonicalization. -
Header mismatch. The
h=tag specifies which headers are signed. If a relay adds or modifies a signed header, verification fails. - Key rotation. When you rotate DKIM keys, keep the old public key in DNS for at least 7 days. Emails in transit may still carry the old signature.
DMARC: Alignment Is the Hard Part
DMARC doesn't add new authentication — it validates that SPF and DKIM align with the From: header domain. This is where things get tricky.
From: noreply@yourdomain.com <- The "From" domain
Return-Path: bounce@mail.yourdomain.com <- The "envelope from" (SPF domain)
DKIM-Signature: d=yourdomain.com <- The DKIM signing domain
For DMARC to pass, at least one of these must be true:
-
SPF alignment: The
Return-Pathdomain matches (or is a subdomain of) theFrom:domain -
DKIM alignment: The
d=domain in the DKIM signature matches (or is a subdomain of) theFrom:domain
In relaxed mode (adkim=r), mail.yourdomain.com aligns with yourdomain.com. In strict mode (adkim=s), it doesn't.
This matters when you use third-party email services. If you send through an ESP but they sign with their own domain (d=esp-domain.com), DKIM alignment fails against your From: domain. The fix: configure the ESP to sign with your domain (custom DKIM), not theirs.
Parsing DMARC Reports
DMARC aggregate reports arrive as XML attachments. Here's how to parse them:
import xml.etree.ElementTree as ET
from collections import defaultdict
def parse_dmarc_report(xml_file):
tree = ET.parse(xml_file)
root = tree.getroot()
results = defaultdict(lambda: {'pass': 0, 'fail': 0})
for record in root.findall('.//record'):
source_ip = record.find('.//source_ip').text
count = int(record.find('.//count').text)
spf_result = record.find('.//auth_results/spf/result').text
dkim_result = record.find('.//auth_results/dkim/result').text
dmarc_disposition = record.find('.//policy_evaluated/disposition').text
print(f"IP: {source_ip} | Count: {count} | "
f"SPF: {spf_result} | DKIM: {dkim_result} | "
f"Action: {dmarc_disposition}")
if dmarc_disposition == 'none':
results[source_ip]['pass'] += count
else:
results[source_ip]['fail'] += count
return results
Run this on your rua reports and you'll quickly find unauthorized senders (spoofing your domain) and legitimate services you forgot to authorize.
Building a Bounce Processing Pipeline
Bounce handling is where most DIY email infrastructure falls apart. You need to process bounces in real-time and act on them, or your sender reputation degrades fast.
There are two types of bounces:
- Hard bounces (5xx SMTP codes): The address doesn't exist. Remove it immediately and never send again.
- Soft bounces (4xx SMTP codes): Temporary issue (mailbox full, server down). Retry with exponential backoff, but suppress after 3-5 consecutive soft bounces.
interface BounceEvent {
email: string;
type: 'hard' | 'soft';
code: string;
timestamp: Date;
diagnosticCode: string;
}
async function processBounce(event: BounceEvent) {
const { email, type, code, diagnosticCode } = event;
if (type === 'hard') {
// Immediate suppression — never send to this address again
await db.suppressionList.upsert({
where: { email },
create: {
email,
reason: 'hard_bounce',
code,
diagnosticCode,
suppressedAt: new Date(),
},
update: {
reason: 'hard_bounce',
code,
diagnosticCode,
suppressedAt: new Date(),
},
});
// Remove from all active lists
await db.subscriber.updateMany({
where: { email },
data: { status: 'bounced' },
});
return;
}
// Soft bounce: track consecutive failures
const existing = await db.bounceLog.findMany({
where: {
email,
type: 'soft',
createdAt: { gte: subtractDays(new Date(), 30) },
},
orderBy: { createdAt: 'desc' },
});
await db.bounceLog.create({
data: { email, type, code, diagnosticCode },
});
// Suppress after 5 soft bounces in 30 days
if (existing.length >= 4) {
await db.suppressionList.create({
data: {
email,
reason: 'repeated_soft_bounce',
code,
suppressedAt: new Date(),
},
});
}
}
Before every send, check the suppression list. This is a hard requirement — sending to known-bad addresses is the fastest way to tank your reputation.
async function canSendTo(email: string): Promise<boolean> {
const suppressed = await db.suppressionList.findUnique({
where: { email },
});
return !suppressed;
}
IP Warming: The Automated Approach
New IPs start with zero reputation. If you send 50,000 emails on day one from a fresh IP, you'll get throttled or blocked. IP warming is the process of gradually increasing volume to build trust.
Here's an automated warming schedule implementation:
const WARMING_SCHEDULE = [
{ day: 1, limit: 50 },
{ day: 2, limit: 100 },
{ day: 3, limit: 250 },
{ day: 5, limit: 500 },
{ day: 7, limit: 1000 },
{ day: 10, limit: 2500 },
{ day: 14, limit: 5000 },
{ day: 21, limit: 10000 },
{ day: 28, limit: 25000 },
{ day: 35, limit: 50000 },
{ day: 42, limit: -1 }, // unlimited
];
function getDailyLimit(warmingStartDate: Date): number {
const daysSinceStart = Math.floor(
(Date.now() - warmingStartDate.getTime()) / (1000 * 60 * 60 * 24)
);
// Find the applicable schedule entry
for (let i = WARMING_SCHEDULE.length - 1; i >= 0; i--) {
if (daysSinceStart >= WARMING_SCHEDULE[i].day) {
return WARMING_SCHEDULE[i].limit;
}
}
return WARMING_SCHEDULE[0].limit;
}
async function sendWithWarmingLimit(
ipAddress: string,
emails: QueuedEmail[]
): Promise<{ sent: number; deferred: number }> {
const ip = await db.sendingIp.findUnique({
where: { address: ipAddress },
});
const limit = getDailyLimit(ip.warmingStartedAt);
const sentToday = await db.sendLog.count({
where: {
ipAddress,
sentAt: { gte: startOfDay(new Date()) },
},
});
const remaining = limit === -1 ? emails.length : limit - sentToday;
const toSend = emails.slice(0, Math.max(0, remaining));
const toDefer = emails.slice(toSend.length);
for (const email of toSend) {
await sendEmail(email, ipAddress);
}
// Deferred emails go back to the queue for the next day
// or route to a different (warmed) IP
for (const email of toDefer) {
await requeueEmail(email);
}
return { sent: toSend.length, deferred: toDefer.length };
}
During warming, prioritize sending to your most engaged recipients. Gmail and other providers weigh early engagement heavily — if your first few hundred emails all get opened and clicked, your reputation ramps up faster.
Feedback Loops and Complaint Processing
Major inbox providers offer Feedback Loop (FBL) programs. When a recipient marks your email as spam, the provider sends a notification to your registered abuse address.
Gmail handles this differently — they use the List-Unsubscribe header and require a spam complaint rate below 0.1%. Google Postmaster Tools is the only way to monitor this.
For Microsoft, Yahoo, and others, you register for their FBL programs and process complaint reports in ARF (Abuse Reporting Format):
// Webhook handler for processing FBL complaints
async function handleFeedbackLoop(req: Request) {
const complaint = await parseARF(req.body);
// Immediately unsubscribe the complaining user
await db.subscriber.update({
where: { email: complaint.reportedEmail },
data: {
status: 'complained',
unsubscribedAt: new Date(),
},
});
// Add to suppression list
await db.suppressionList.create({
data: {
email: complaint.reportedEmail,
reason: 'spam_complaint',
source: complaint.reportingProvider,
suppressedAt: new Date(),
},
});
// Track complaint rate for monitoring
await db.complaintMetric.create({
data: {
email: complaint.reportedEmail,
provider: complaint.reportingProvider,
campaignId: complaint.campaignId,
},
});
}
Your spam complaint rate must stay below 0.1% (that's 1 complaint per 1,000 emails). Above 0.3%, Gmail will start bulk-rejecting your emails. This is non-negotiable.
Monitoring: What to Track
Build a monitoring dashboard that tracks these metrics per sending IP and domain:
-- Daily deliverability metrics query
SELECT
DATE(sent_at) AS send_date,
sending_ip,
COUNT(*) AS total_sent,
COUNT(*) FILTER (WHERE status = 'delivered') AS delivered,
COUNT(*) FILTER (WHERE status = 'bounced' AND bounce_type = 'hard') AS hard_bounces,
COUNT(*) FILTER (WHERE status = 'bounced' AND bounce_type = 'soft') AS soft_bounces,
COUNT(*) FILTER (WHERE status = 'complained') AS complaints,
ROUND(
COUNT(*) FILTER (WHERE status = 'bounced' AND bounce_type = 'hard')::numeric
/ NULLIF(COUNT(*), 0) * 100, 2
) AS bounce_rate,
ROUND(
COUNT(*) FILTER (WHERE status = 'complained')::numeric
/ NULLIF(COUNT(*) FILTER (WHERE status = 'delivered'), 0) * 100, 3
) AS complaint_rate
FROM email_sends
WHERE sent_at >= CURRENT_DATE - INTERVAL '30 days'
GROUP BY DATE(sent_at), sending_ip
ORDER BY send_date DESC;
Set alerts for:
- Bounce rate above 2%
- Complaint rate above 0.05% (catch it before you hit 0.1%)
- Delivery rate drop of more than 10% day-over-day
- Any sending IP appearing on a blacklist (poll Spamhaus, Barracuda, and Sorbs via DNS lookups)
The List-Unsubscribe Header
As of June 2024, Gmail and Yahoo require List-Unsubscribe with one-click support for bulk senders (5,000+ messages/day). This isn't optional.
List-Unsubscribe: <https://yourdomain.com/unsubscribe?token=abc123>,
<mailto:unsub@yourdomain.com?subject=unsubscribe>
List-Unsubscribe-Post: List-Unsubscribe=One-Click
The List-Unsubscribe-Post header tells the mail client it can unsubscribe the user with a single POST request, without opening a browser. Gmail surfaces this as a prominent "Unsubscribe" link next to the sender name.
Your endpoint must handle POST requests with the body List-Unsubscribe=One-Click and process the unsubscribe within 2 days (Google's requirement). In practice, process them immediately.
The Reality: Build vs. Buy
Everything I've described above — DKIM signing, bounce processing, IP warming, FBL integration, suppression list management, reputation monitoring — is a full-time job. At minimum, you're looking at:
- SMTP server management (Postfix, Haraka, or custom)
- Queue management for sending throttling
- DNS record management and monitoring
- Bounce and complaint processing pipelines
- IP pool management and rotation
- Deliverability monitoring and alerting
- Blacklist monitoring and delisting
- Keeping up with provider policy changes (Gmail/Yahoo update requirements regularly)
For most engineering teams, this is a distraction from building your actual product. You can spend weeks building email infrastructure, or you can use a platform that handles the operational complexity and exposes a clean API.
A platform like GetMailer gives you the API and SMTP relay with all the deliverability infrastructure managed — authentication, IP warming, bounce handling, reputation monitoring, and compliance. You integrate with a few lines of code:
// Send via API
const response = await fetch('https://api.getmailer.co/v1/send', {
method: 'POST',
headers: {
'Authorization': `Bearer ${GETMAILER_API_KEY}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({
from: 'noreply@yourdomain.com',
to: 'user@example.com',
subject: 'Welcome aboard',
html: '<h1>Welcome!</h1><p>Your account is ready.</p>',
}),
});
Or via SMTP relay if you prefer dropping it into an existing codebase:
Host: smtp.getmailer.co
Port: 587
Username: your-api-key
Password: your-api-key
Encryption: STARTTLS
The point isn't that you can't build this yourself — it's that every hour spent on email infrastructure is an hour not spent on your product. The engineering effort to do it right is substantial, and the cost of doing it wrong (blacklisted IP, trashed domain reputation) is high and slow to recover from.
If you're looking for an email platform built for developers — with API-first design, managed deliverability infrastructure, and real-time analytics — GetMailer handles SPF/DKIM/DMARC setup, IP warming, bounce processing, and reputation monitoring so you can ship emails instead of managing mail servers.
Top comments (0)