DEV Community

HarrisonFord3572
HarrisonFord3572

Posted on

Signing-Key Rotation on a Schedule vs After an Incident: Budget the DNS TTL First

#dkim #dns #email #security

Use a fixed rotation calendar for your DKIM signing keys, and size that calendar around the slowest resolver cache in the path rather than around your incident channel. A rotation you schedule costs you a planned wait. A rotation you're forced into costs you the same wait, except mail is already moving and a merchant's customers are waiting on statements.

That's the whole argument. The rest is arithmetic — and the arithmetic is the part people skip.

The system I'm describing here is a fintech onboarding flow: a merchant can't finish signup until we've proven they control the domain their statements will be sent from. Two DNS artifacts gate that step — a verification TXT record at the apex, and a DKIM selector record at <selector>._domainkey.<domain> carrying v=DKIM1; k=rsa; p=<base64>. A verifier pulls that public key out of DNS at the moment it checks a message (RFC 6376), which means the record that decides your outcome is the one their resolver can see, not the one sitting in your zone file.

Should I schedule DKIM key rotation or wait for a mail incident?

Schedule it, on a period you pick deliberately, because scheduling is what buys you two live selectors at the same time.

Dual publication is the entire trick. You publish stmt-b next to the currently signing stmt-a, let it propagate, flip the signer to stmt-b, and leave stmt-a published long enough that anything signed with it can still be verified. Every step except the flip is slow, and every slow step happens while nothing is broken.

Now do the same thing under pressure. You cannot compress a cache you don't own: resolvers may serve the old selector data until its TTL expires, and if a resolver already asked for the new selector name and got NXDOMAIN, that negative answer is cached too — bounded by the SOA MINIMUM field, per RFC 2308, and applied to the whole name because NXDOMAIN means nothing exists underneath it (RFC 8020). So the emergency path hands you the worst-case propagation delay at the exact moment you wanted the fastest cutover, plus a second problem: pulling a key does not un-sign mail that already left. Messages parked in a forwarder's queue lose their DKIM result, and if SPF doesn't align either, DMARC policy decides what happens to them (RFC 7489). For transactional statements, that blast radius is the real cost, not the DNS edit.

Pick a cadence longer than your longest legitimate in-transit window and short enough that a compromise stays bounded. Quarterly is a defensible default for 2048-bit RSA keys, which is what RFC 8301 recommends as key size while ruling out SHA-1 signing; if you want shorter records and faster propagation, Ed25519 selectors are standardized in RFC 8463, though you'll still need an RSA selector alongside them for verifiers that don't implement it.

From the onboarding TXT check to a signed statement

One row per tenant domain holds three fields: the apex verification token, the selector name we expect to be signing, and the selector name we are currently publishing ahead of the flip. Onboarding writes the first, the signer reads the second, and a probe reconciles the third against what public resolvers actually return. The TTLs are design choices, not discoveries — 300 seconds on the apex verification TXT so onboarding retries feel responsive, 3600 on selector records so verifiers aren't hammering the zone, and a 900-second SOA MINIMUM so a stale NXDOMAIN can't outlive a coffee break.

Three moving parts, one source of truth.

A propagation probe to run before you flip the signer

This is the piece I'd write first, before any scheduler. It answers one question per domain: is the new selector visible from resolvers we don't control, and what does the old record's TTL say about when it can be retired? Python 3.11, dnspython 2.x, no other dependencies.

import dns.exception
import dns.resolver
from dataclasses import dataclass, field

RESOLVERS = ["9.9.9.9", "1.1.1.1", "8.8.8.8"]
DNS_ERRORS = (dns.resolver.NXDOMAIN, dns.resolver.NoNameservers, dns.exception.Timeout)


def txt_records(name: str, nameserver: str, timeout: float = 5.0):
    """Return (values, ttl) for a TXT name as seen by one specific resolver."""
    r = dns.resolver.Resolver(configure=False)
    r.nameservers = [nameserver]
    r.lifetime = timeout
    try:
        answer = r.resolve(name, "TXT", raise_on_no_answer=False)
    except DNS_ERRORS:
        return [], None
    if answer.rrset is None:
        return [], None
    # A 2048-bit key does not fit in one 255-byte character-string, so the
    # record arrives as several strings that have to be joined back together.
    values = [b"".join(rd.strings).decode("ascii", "replace") for rd in answer.rrset]
    return values, answer.rrset.ttl


def dkim_key(values: list[str]) -> str | None:
    for value in values:
        tags = dict(
            part.split("=", 1) for part in
            (chunk.strip() for chunk in value.split(";")) if "=" in part
        )
        if tags.get("v", "DKIM1") == "DKIM1" and tags.get("p"):
            return tags["p"].replace(" ", "")
    return None


@dataclass
class Cutover:
    domain: str
    visible_on: list[str] = field(default_factory=list)
    missing_on: list[str] = field(default_factory=list)
    retire_old_after: int | None = None

    @property
    def safe_to_flip(self) -> bool:
        return not self.missing_on


def probe(domain: str, new_selector: str, old_selector: str, expected_p: str) -> Cutover:
    result = Cutover(domain=domain)
    for ns in RESOLVERS:
        values, _ = txt_records(f"{new_selector}._domainkey.{domain}", ns)
        target = result.visible_on if dkim_key(values) == expected_p else result.missing_on
        target.append(ns)
    _, old_ttl = txt_records(f"{old_selector}._domainkey.{domain}", RESOLVERS[0])
    result.retire_old_after = old_ttl
    return result


if __name__ == "__main__":
    state = probe("merchant-example.test", "stmt-b", "stmt-a", expected_p="MIIBIjANBg...")
    print(state.domain, "flip:", state.safe_to_flip, "old ttl:", state.retire_old_after)
Enter fullscreen mode Exit fullscreen mode

Two details earn their keep here. configure=False plus an explicit nameservers list stops the probe from reading /etc/resolv.conf and quietly measuring your own recursive cache instead of the internet's view. And b"".join(rd.strings) is not cosmetic: a parser that reads rd.strings[0] passes every staging test built on a 1024-bit key, then truncates the first real 2048-bit key it sees, because RFC 1035 caps each character-string at 255 bytes.

The flip itself is one field update, and retire_old_after is deliberately not a boolean. Visibility of the new record is a fact you can measure; safety of deleting the old one is a policy you have to state, because it depends on how long a message may sit in someone else's queue before verification.

What the propagation budget costs, and when scheduling is the wrong call

Approach Lead time you control Cutover speed under pressure Main exposure
Scheduled dual-selector rotation Full TTL window, planned One field update Automation rot between rotations
Incident-driven rotation None Bounded by caches you don't own Unverifiable mail in flight
No rotation until something forces it None Same, plus untested tooling Unbounded key lifetime

The catch is that scheduled rotation only pays off if the boring path runs often enough to stay honest. A quarterly job that nobody watched for nine months is worse than a manual runbook, because it produces the confident feeling of coverage without the coverage.

It's also not a good fit when you don't own the zone. In the onboarding case the merchant frequently controls their own DNS, so the schedule isn't an automation pipeline at all — it's a notification pipeline with a deadline, an inbox nudge, and a probe that tells support whether the tenant did the thing. Different failure mode, different tooling, same calendar.

For a single domain you send from yourself, stick with a documented manual rotation and a reminder. And I'm honestly not sure any universal cadence generalizes: the right number depends on how many tenant domains you sign for, who owns those zones, and whether your mail path includes forwarders that delay verification by hours.

Running it for real

Test the parser, not the network. I keep a fixture set of raw TXT payloads — a chunked 2048-bit key, a record with v=DKIM1 missing, a selector carrying two TXT records because someone appended instead of replacing, a name that returns NXDOMAIN — and assert the parser's output on each, which is the part that breaks silently. The network portion gets one integration test against a staging zone whose TTLs are set to 60 seconds, so a full publish-propagate-flip-retire cycle runs inside a test window instead of a quarter.

Watch the result, not the record. DMARC aggregate reports (RFC 7489) are the only feedback channel that tells you how real verifiers judged your signatures, broken down by source; a rotation that went wrong shows up there as a DKIM pass rate that drops for one selector and never recovers. Wire that into whatever you already page on, keep the probe's per-domain output in the same store as your onboarding state, and give the rotation calendar one named owner plus a zone diff that goes through code review like any other change. The queries themselves are cheap. Waking a human at 2 a.m. to guess at cache state is not.

References

Top comments (0)