DEV Community

WindwhisperBoren33
WindwhisperBoren33

Posted on

TTL, Caching, and Failover Limits at the DNS Layer, Demystified (and Where Mail Differs)

#dns #email #architecture #reliability

A mail cutover carries a constraint that web traffic does not: the client on the other end is another mail server, and it owns a queue. That single property settles the layer question before any vendor comparison starts. Use DNS to declare which provider owns the flow, then use MX preference and the sender's retry queue for failover — geographic routing at the DNS layer, with its TTL caching and its hard failover limits, buys a newsroom close to nothing on mail.

The system behind this piece is unremarkable. A media company, one apex domain, six newsrooms, editorial mail plus transactional alerts, moving company mail off a self-run cluster onto a hosted provider. The decision axis was not latency and it was not a dashboard; it was deliverability evidence. What can you actually prove, in writing, a week after the switch?

What TTL actually promises, and what resolvers do instead

A TTL is an upper bound on how long a cached answer may be reused. RFC 2181 pins the field to an unsigned 31-bit range and tells implementations to treat values with the top bit set as zero, and that's close to the only hard rule in the neighbourhood. The rest is discretionary. A resolver may cap your 86400 down to whatever its operator prefers, and under RFC 8767 it may keep answering from an expired entry when the authoritative servers are unreachable — that document suggests a stale limit in the range of one to three days. So a 300-second TTL is a request, not a contract.

Plan against the ceiling, not against the number you published.

Negative answers cache too, and that is the half people forget. Under RFC 2308 a NODATA or NXDOMAIN response is cached according to the SOA MINIMUM field, with a recommended ceiling of three hours. Create the hostname before anything asks for it. If a monitoring probe queries mx1.example.com an hour before you publish the record, that cached absence can outlive the record you add, and you will spend the afternoon blaming the registrar for a cache you populated yourself.

There is a bill attached to this knob, which is the part I tend to be the loudest about. Dropping an MX record from 3600 to 300 can multiply authoritative queries for that name by twelve. On a hosted zone that is query volume on an invoice; on your own anycast pair it is CPU and log lines. Paying it for a migration week is fine. Paying it for a year because nobody put the TTL back is the kind of quiet waste that never shows up in a postmortem, because nothing broke.

Should geographic routing live at the DNS layer when TTL caching sets the failover limits?

Geographic answers are computed from the source address of the resolver, not the client. That is the entire mechanism, and it explains why accuracy is uneven. EDNS Client Subnet (RFC 7871) forwards a truncated prefix of the client network so the authoritative side can answer better, and the RFC is blunt that this leaks user information and should be limited to the cases that genuinely need it. Plenty of resolvers decline to send it.

For a web origin that trade is often worth taking, because a wrong answer costs tens of milliseconds and nothing else. For mail the premise collapses. There is no user behind the lookup at all: the client is some other organisation's MTA, sitting wherever its operator placed it, resolving through whichever resolver it trusts. Geo-steering an MX name shards your inbound path by the sending network's resolver placement — a property you have no reason to optimise, and one that makes every deliverability question harder to answer, because from then on the answer depends on who asked. Managed DNS platforms expose steering as a product feature with health checks attached, and Route 53, Cloudflare and NS1 each document their own variant of it; every one of those still sits behind a cache it does not control.

Layer Convergence bound Precision Evidence you get back
DNS record swap TTL, plus resolver caps and stale serving Resolver network, not client Authoritative query logs, delayed
Anycast withdrawal Routing convergence, seconds to a minute Network path BGP state, probe results
Application proxy Immediate, per request Real client address Request logs, per user
MX preference plus sender queue One connection attempt None, and it does not need any SMTP logs on both sides

So here is the rule I would defend in review: DNS is the right layer when the thing you steer changes slowly, when a stale answer is merely suboptimal rather than wrong, and when you can live with a convergence window equal to the TTL plus whatever the slowest resolver adds on top. Anything faster belongs at the IP layer or inside the application. Stick with a single, boring, globally identical MX set when the workload is mail.

Mail is the counterexample: MX preference, retry windows, and a four-day give-up clock

SMTP has had failover since long before traffic management became a product category. RFC 5321 has the sending client look up the MX RRset, sort it by preference, try the lowest-numbered host first, and distribute load across hosts that share a preference value. If that host refuses the connection, the client walks to the next one. If every host fails, the message returns to the queue instead of dying: the RFC calls 30 minutes a reasonable retry interval and states that the give-up time generally needs to be at least 4–5 days.

Four days. That's your failover budget, and it's already paid for.

Set that against what a DNS change can offer. Publish a replacement MX and the honest convergence estimate is the TTL, plus whatever cap or stale window the far-side resolver applies, plus the fact that a busy sender may hold its own answer for the length of a queue run. A second MX host at preference 20 converges in one failed connection attempt, with no cache in the path. That is not a clever trick, it is just the protocol doing what it was specified to do — and it is why an inbound outage of twenty minutes costs you latency rather than mail.

Two details keep this from being free. The target of an MX record must be a hostname with address records, never an alias, which RFC 2181 states directly, so the CNAME shortcut that works elsewhere in your zone is unavailable here. And if you publish an MTA-STS policy under RFC 8461, that policy enumerates the MX patterns a sender will accept; answers that vary by geography force you to enumerate every regional variant in the policy file or watch strict-mode senders refuse the connection. As far as I can tell, that alone disqualifies geo-steered MX for any domain that has adopted strict transport security, and it is the cheapest argument to make to a room that has already fallen in love with a steering dashboard.

The evidence layer: what aggregate reports cost you to keep

Deliverability evidence for a cutover comes from DMARC aggregate reporting, defined in RFC 7489. You publish a _dmarc TXT record with a rua address, participating receivers mail you XML on an interval whose default is 86400 seconds, and each report enumerates source IP, message counts, SPF and DKIM results with their alignment, and the policy disposition applied.

One day of latency. That rules the feed out as a failover signal and rules it straight into the cutover checklist.

Now the part I get paid to think about. Six sending addresses and roughly forty reporting organisations produce on the order of 240 report rows a day, which is about 95,000 rows across thirteen months — as compressed XML in object storage, that is a rounding error on any bill. As a metrics series it is not. Expand the same tuple into labels — source IP by reporter by policy by disposition by header-from — and you have manufactured a high-cardinality time series for data that updates once per day, which is how a $0 problem becomes a line item in an observability contract. Keep the raw reports for thirteen months so you have one full seasonal comparison, count three aggregates on top of them, and resist every request to graph the rest.

The catch is what aggregate reports don't contain. They describe authentication and policy disposition, not inbox placement, so a spotless rua feed and a spam folder are entirely compatible states. Failure reports carry message-level detail, a minority of receivers send them at all, and they pull real recipient data into your retention policy — I would rather not collect them than collect them badly. If you need placement evidence, that means seed lists or feedback loops, and it is a different budget line with a different privacy review.

A migration that actually converges

Order matters more than tooling here. Roughly two old-TTL intervals before the switch, lower the MX TTL to 300 and confirm the zone's SOA MINIMUM is not parked at three hours, because a negative answer cached for that long will outlive your change. Publish the new provider's MX hosts at a higher preference number first, so they accept mail without taking it, and verify a real message lands. Then invert the preferences and watch the SMTP logs move.

Verification is the step people skip, and it is one curl away over DNS-over-HTTPS (RFC 8484). Query the same name through resolvers in each region you care about and compare what they hold, including the remaining TTL:

curl -s -H 'accept: application/dns-json' \
  'https://doh.example.net/dns-query?name=example.com&type=MX'
Enter fullscreen mode Exit fullscreen mode 
{"Status":0,"Answer":[{"name":"example.com","type":15,"TTL":287,"data":"10 mx1.provider.example."}]}
Enter fullscreen mode Exit fullscreen mode

That TTL: 287 is the countdown on a cached copy, not your published value, which is exactly the number you want during a cutover. Run it as a loop across several resolvers and you have a convergence check that costs nothing and, unlike a steering dashboard, tells you what senders will really see:

for r in doh1.example.net doh2.example.net doh3.example.net; do
  curl -s -H 'accept: application/dns-json' \
    "https://$r/dns-query?name=example.com&type=MX" \
    | tr ',' '\n' | grep -E '"TTL"|"data"'
done
Enter fullscreen mode Exit fullscreen mode

Hold the low TTL for the first week of aggregate reports, because that is the window where you might need to move quickly, and then put it back to something in the hours range. The cost of leaving it low is not dramatic, it is just permanent, which in my experience is worse.

If your answer to geographic routing is that inbound mail should land closer to European recipients, the mechanism you want is anycast in front of one MX set, or a provider that already runs one — not different answers for different askers. The DNS layer is where you name the destination. The queue is where you survive it.

Sources

Top comments (0)