How the DNS root rotated its master key on October 11 without breaking the internet
On October 11, 2026, the most trusted key on the internet changed hands. No deploy went out. No status page turned red. Nobody got paged. The DNS root zone simply stopped signing its list of keys with KSK-2017 and started signing with KSK-2024.
It was only the second time this has ever happened. The first was in 2018.
In Malaysia that weekend, some websites stopped loading for users on Maxis DNS. Local press tied the reports to the rollover. Maxis has not confirmed a network-wide cause, so treat that link as reported, not proven. But the failure shape fits exactly, and the shape is the interesting part: when a resolver misses this kind of change, nothing looks broken except everything.
The one key you cannot look up
DNSSEC lets a resolver check digital signatures on DNS answers, so it knows the answer is authentic and unmodified. Trust forms a chain. To believe example.com's keys, your resolver checks the signature from .com. To believe .com's keys, it checks the signature from the root. Each parent vouches for its child with a DS record, which is just a fingerprint of the child's key.
The root has no parent. So every validating resolver ships with one key it believes without proof: the root key-signing key, kept as a trust anchor in its software or configuration.
The division of labor is neat. The KSK signs only one thing: the root zone's DNSKEY record set, which is the published list of the root's public keys. That list contains the zone-signing key, the ZSK. The ZSK signs everything else, including the DS records for top-level domains like .com. So the KSK is the anchor for the anchor list. Lose it, and the chain has no starting point. Not one domain fails. Every domain fails.
Changing a lock while the building is full
You cannot email every resolver operator on earth and ask them to update a config on the same Sunday. Firewalls, home routers, containers, and appliances run resolvers whose owners never chose their DNS behavior and rarely check it. So ICANN ran the rollover in slow motion. The full schedule, from IANA's trust anchor page:
- October 27, 2016: KSK-2017 generated.
- October 11, 2018: KSK-2017 takes over signing the root. First rollover ever done.
- April 27, 2023: KSK-2023 generated. Then abandoned, superseded before it ever signed anything. Someone made a master key for the internet and threw it away.
- April 26, 2024: KSK-2024 generated.
- January 11, 2025: KSK-2024 published in the root DNSKEY set, alongside KSK-2017. Nothing trusts it yet. It just sits there, its presence signed by the old key.
- February 10, 2025: resolvers following RFC 5011 can finish adopting it.
- October 11, 2026: the switch. KSK-2024 signs the DNSKEY set. KSK-2017 stops.
- January 11, 2027: KSK-2017 published with the REVOKE bit set.
- March 22, 2027: KSK-2017 removed from the root zone.
The quiet hero here is RFC 5011, automated trust-anchor updates. Your resolver watches the root DNSKEY set. One day it notices a new key in there, and the new key's presence is signed by the old key it already trusts. It does not trust it immediately. It waits. For 30 days it must keep seeing the new key, consistently, in normally signed responses. Only then does it promote the key to a trust anchor on its own. No config push, no restart, no human.
The 30-day hold-down is the whole security model in one number. It exists so a brief spoof or glitch cannot plant a fake anchor. But read the assumption hiding inside it: the resolver must be up and querying regularly through the transition. A validator powered off for a month, a VM snapshot resumed after the window, a container image that restores stale state on every deploy, a trust-anchor file on a read-only filesystem where updates fail silently. Any of them can still trust only KSK-2017 on the morning of the switch. Automation is not magic. It is a scheduled job with preconditions.
The question you are not allowed to ask
This is my favorite part of the story. Suppose you run a fleet of resolvers and want to know which ones trust the new key. There is no DNS query for "list your trust anchors." So RFC 8509 encodes the question inside a domain name.
Try these two lookups:
dig root-key-sentinel-is-ta-38696.dnstest.dev
dig root-key-sentinel-not-ta-38696.dnstest.dev
The authoritative server answers both names. But a validating resolver that trusts key tag 38696 will fail validation on the second name and return SERVFAIL to you. The resolver's own trust state leaks out through which answers survive. It is an interrogation disguised as a lookup.
How to read your result:
-
is-taanswers,not-tareturns SERVFAIL: your resolver trusts KSK-2024. You are fine. - Both names answer: inconclusive. Your resolver either does not validate DNSSEC or does not speak the sentinel protocol. Most home setups land here.
-
is-tareturns SERVFAIL: your resolver validates but does not trust the new key. Fix it before January, when the old key gets revoked.
What "not ready" looks like
It does not look like an error page. A validating resolver that only trusts KSK-2017 can no longer verify the root's key list, which means it cannot verify anything below it. Every lookup returns SERVFAIL. The sites are up. Your connection is fine. The names just stop resolving, all of them, at once.
That is why the scattered reports after October 11 are worth noting even though none of them is a confirmed smoking gun. Users in Malaysia reported .my sites failing to load on Maxis DNS between October 11 and 13. Router vendor forums had users describing lookups failing with DNSSEC enabled and recovering the moment they switched it off. No broad outage materialized, which is the good news. The bad news is the shape: this failure is silent and total, and it lands on exactly the devices nobody inventories.
Check the root yourself
I pulled the live root DNSKEY set today, October 16, and computed the key tags from it. Five days after the rollover, three keys are published:
- 8763, the zone-signing key, doing the daily signing work.
- 20326, KSK-2017: still published, no longer signing, revocation scheduled for January 2027.
- 38696, KSK-2024: the new signer.
You can do the same check from any machine with Python. DNS-over-HTTPS avoids depending on your local resolver's mood, and the key tag follows RFC 4034's recipe. One detail that bit me: the tag is computed over the full DNSKEY RDATA in wire format, not just the key bytes. My first attempt hashed only the public key and produced nonsense tags. Adding the four header bytes (flags, protocol, algorithm) fixed it.
import base64, json, struct, urllib.request
req = urllib.request.Request(
"https://cloudflare-dns.com/dns-query?name=.&type=DNSKEY",
headers={"Accept": "application/dns-json"},
)
data = json.load(urllib.request.urlopen(req, timeout=20))
def key_tag(rdata):
flags, proto, algo, b64 = rdata.split()
wire = struct.pack("!HBB", int(flags), int(proto), int(algo))
wire += base64.b64decode(b64 + "=" * (-len(b64) % 4))
total = 0
for i, byte in enumerate(wire):
total += byte << (8 if i % 2 == 0 else 0)
total += total >> 16
return total & 0xFFFF
for ans in data["Answer"]:
flags = ans["data"].split()[0]
role = "ZSK" if flags == "256" else "KSK"
print(f"{role:>3} key tag {key_tag(ans['data'])}")
Real output, October 16, 2026:
ZSK key tag 8763
KSK key tag 20326
KSK key tag 38696
The numbers going in
The headline telemetry looked strong. ICANN reported that more than 95 percent of resolvers signaling their trust anchors had adopted KSK-2024. Verisign's measurements in March 2025 put adoption at 91.3 percent of monitored resolvers.
Two numbers complicate the picture. About 3.5 percent of reporting resolvers still carried KSK-2010, a key revoked back in 2019. Seven years after its funeral, a dead master key is still sitting in trust stores on real machines. That is how long stale trust state survives.
And APNIC's Geoff Huston measured from the user side rather than the root side. In April 2026, his sentinel-based tests found under 20 percent of users behind DNSSEC-validating resolvers showing KSK-2024 as trusted, which he called, quote, "certainly not the result that we had anticipated." Different populations, different methods, noisy data. But it is a reminder that the view from the root and the reality at the edge can disagree, and the edge is where your users live.
The actual lesson
Strip away the cryptography and the root KSK rollover is a distributed state problem. One organization publishes a key. Millions of independently operated resolvers have to observe it, keep that observation across restarts, upgrades, snapshots, and config management, and then the entire world switches which key it depends on. There is no central dashboard that can say everyone is ready, because DNS does not work that way. The 21-month runway, the 30-day hold-down, the sentinel interrogation protocol, the staged revocation: every piece of the design is an admission that trust, at internet scale, is a coordination problem wearing a cryptography costume.
Run the two sentinel queries against your own resolver and post what you got. I am genuinely curious about the ratio of clean passes to inconclusives out there.
Top comments (0)