DEV Community

Rasika Dangamuwa
Rasika Dangamuwa

Posted on

Why DNSSEC Breaks in Production: 5 Chain-of-Trust Traps That Cause Silent SERVFAIL

#devops #networking #security #webdev

You deploy an innocent DNS migration or SSL key rotation, and suddenly 35% of your global users report your domain is down. Yet when you test curl https://example.com or query your local ISP DNS, it works completely fine. When you query Google (8.8.8.8) or Cloudflare (1.1.1.1), you get an abrupt, unhelpful response:

$ dig @1.1.1.1 example.com +dnssec

;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 48219
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
Enter fullscreen mode Exit fullscreen mode

Welcome to DNSSEC failure. Unlike standard DNS errors that return NXDOMAIN (non-existent domain) or stale cached records, DNSSEC validation is binary: if any signature in the cryptographic chain fails, validating recursive resolvers drop the answer and return SERVFAIL.

Here is a breakdown of how the DNSSEC chain of trust works, the 5 subtle traps that trigger catastrophic validation failures, and how to debug them.

Understanding the Chain of Trust

DNSSEC establishes authenticity via cryptographic hierarchy:

  1. Root Trust Anchor: Validating resolvers have the ICANN Root KSK pre-configured.
  2. TLD DS Record: The .com or .org registry publishes a Delegation Signer (DS) record containing the cryptographic hash of your domain's Key Signing Key (KSK).
  3. Domain DNSKEY: Your authoritative nameserver serves your public KSK (Flags 257) and Zone Signing Key (ZSK) (Flags 256).
  4. RRSIG (Resource Record Signature): Every DNS record set (A, CNAME, MX) is accompanied by an RRSIG signature generated by your ZSK. 
[ Root Zone (.) ] -> DS -> [ TLD (.com) ] -> DS -> [ example.com DNSKEY ] -> RRSIG -> [ A Record ]
Enter fullscreen mode Exit fullscreen mode

If any link in this chain breaks, validating resolvers cannot verify authenticity.

Trap 1: The Orphaned DS Record During DNS Migrations

The single most common DNSSEC disaster happens when changing nameservers (e.g., migrating from AWS Route53 to Cloudflare, or moving to a managed hosting provider).

When you update your domain's NS records at your registrar, the parent TLD registry immediately directs queries to the new nameservers. However, if the old provider had DNSSEC enabled, the parent registry still holds the old DS record matching the old provider's KSK.

The new nameservers do not possess the old private keys to sign records. Validating resolvers see a DS record at the parent, query the new nameservers, find mismatched or missing DNSKEY/RRSIG data, and immediately return SERVFAIL.

The Fix: Always remove the DS record at your domain registrar at least 24 to 48 hours before switching NS records. Once the DS TTL expires worldwide, migrate nameservers, and only re-enable DNSSEC once the new zone is stable.

Trap 2: Expired RRSIG Signatures & Clock Skew

Unlike static DNS records with caching TTLs, every RRSIG record includes two explicit UTC timestamps:

  • Signature Inception: When the signature becomes valid.
  • Signature Expiration: When the signature expires (typically 7 to 30 days). 
$ dig +dnssec example.com A

example.com.  300  IN  RRSIG  A 13 2 300 20260905120000 20260828100000 12345 example.com. ...
Enter fullscreen mode Exit fullscreen mode

If an automated zone-signing cron daemon crashes, or if an authoritative server suffers from system clock drift, the signatures will expire. Even though your A records are valid, resolvers will refuse to serve expired signatures.

To quickly verify whether an outage is caused by DNSSEC validation rather than a dead nameserver, test with the Checking Disabled (CD) flag:

# Bypasses DNSSEC validation on the resolver
$ dig @8.8.8.8 example.com +cdflag
Enter fullscreen mode Exit fullscreen mode

If +cdflag returns a valid A record while regular +dnssec returns SERVFAIL, your DNSSEC chain is definitively broken. You can also test your full zone delegation and DS digest match in a browser using a free DNSSEC Checker to inspect individual key tags and signature expiration windows.

Trap 3: Algorithm Rollover Without Double-Signing

When transitioning between cryptographic algorithms (for example, upgrading from RSA/SHA-256 (Algorithm 8) to ECDSA P-256 (Algorithm 13)), RFC 6781 requires strict multi-step rollovers.

If you simply replace your DNSKEY records and DS records simultaneously, cached data across global recursive resolvers will mismatch:

  1. Resolvers caching the old DS record will receive new RRSIGs they cannot verify.
  2. Resolvers caching the new DS record will query authoritative servers that may still serve cached old DNSKEYs.

The Solution: Follow a Double-DS or Double-Sign transition:

  1. Sign the zone with both old and new algorithms simultaneously.
  2. Publish both new and old DNSKEY records.
  3. Update the DS record at the parent registrar to include both hashes.
  4. Wait for TTLs to expire before removing the legacy algorithm.

Trap 4: EDNS0 Buffer Size Truncation and UDP Dropping

DNSSEC responses are significantly larger than standard DNS packets. A typical DNS query response is under 512 bytes, but a DNSKEY response with multiple RSA-2048 keys and RRSIGs frequently exceeds 1,400 bytes.

DNSSEC relies on EDNS0 (Extension Mechanisms for DNS) to negotiate larger UDP payload sizes (typically 1232 or 4096 bytes). However:

  • Strict enterprise firewalls or middleboxes may drop UDP packets exceeding standard MTUs (1500 bytes) due to IP fragmentation.
  • If UDP is blocked or truncated (TC bit set), resolvers fallback to TCP port 53. If your firewall blocks incoming TCP 53, resolution fails.

Ensure all authoritative nameservers permit TCP port 53 traffic alongside UDP.

Trap 5: NSEC3 Iteration Traps (RFC 9276)

DNSSEC provides authenticated denial of existence for subdomains using NSEC or NSEC3 records. To prevent zone-walking (harvesting all subdomains in a zone), NSEC3 hashes domain names with multiple iterations and a salt.

Historically, administrators set high NSEC3 iteration counts (e.g., 500+ iterations). However, RFC 9276 demonstrated that high iteration counts create significant CPU denial-of-service vulnerabilities on validating resolvers. Modern resolvers (BIND, Unbound, PowerDNS) now treat zones with iteration counts greater than 100 as bogus or unvalidated.

Use modern standards: NSEC3 with 0 to 1 iterations and no salt, or modern compact denial-of-existence implementations.

Summary Checklist for DNSSEC Resilience

  1. Test with delv: Run delv @8.8.8.8 example.com to inspect the full validation path from root to host.
  2. Check the CD Flag: If a query returns SERVFAIL, verify with dig +cd to pinpoint whether DNSSEC is the culprit.
  3. Monitor Expiration Dates: Set automated alerts for RRSIG expiration at least 5 days prior to expiry.
  4. Validate Before Migrating: Before modifying NS records, verify your DS records with a DNSSEC Checker to ensure no lingering signatures cause unexpected downtime.

Top comments (0)