DEV Community

Cover image for Debugging DNS Propagation When Records Just Don't Show Up
Schiff Heimlich
Schiff Heimlich

Posted on

Debugging DNS Propagation When Records Just Don't Show Up

We've all been there. You update a DNS record, wait for it to propagate, and... nothing. The old record still shows up. You check the TTL, you check the authoritative server, everything looks right on paper, but your application still can't resolve the new value.

Here's the thing: DNS propagation isn't just about TTL expiry. There's a whole async world of background tasks, caching layers, and stuck processes that can delay or completely block propagation, and standard tools won't show you what's actually happening.

The Real Problem

Most DNS debugging starts with dig or nslookup, checking TTLs, querying different resolvers. That's fine for basic issues. But when you're dealing with propagation failures in production—especially in complex infrastructure with multiple DNS layers—those tools hit a wall.

The issue isn't usually the DNS servers themselves. It's the stuff between them. Background tasks that haven't finished. Stale caches in unexpected places. Async propagation jobs that got stuck or failed silently.

I spent weeks tracking down a propagation issue that turned out to be a background sync task between our primary and secondary DNS infrastructure. The task was running, but not actually completing the updates. Standard DNS queries showed everything was "correct" at each layer, but the actual propagation wasn't happening.

Building Internal Debug Tools

The solution wasn't to add more monitoring or alerts. It was to build a small internal debug tool that could trace the actual propagation path, step by step.

Here's what that looked like:

  1. Trace propagation state at each layer - Not just querying DNS, but checking the actual state of background sync jobs, cache invalidation status, and async task queues
  2. Identify stuck tasks - Background jobs that were running but not making progress, or failing silently
  3. Compare expected vs actual state - What the DNS config said vs what was actually deployed at each point in the chain

The tool was simple—just a few shell scripts that pulled state from our DNS management system, our background job queue, and our caching layer. But it gave us visibility into where propagation was actually breaking down.

What to Look For

When you're debugging DNS propagation that's not behaving:

  • Check your background sync jobs - Are they actually completing, or just running without doing work?
  • Look for async delays - Some systems queue DNS updates and process them asynchronously. If that queue is backed up or stuck, propagation stalls.
  • Verify cache invalidation - It's not enough to update the record. The cache needs to know to drop the old value. Sometimes invalidation fails silently.
  • Check the actual deployment state - What's in your DNS config vs what's actually deployed at each nameserver can be different if deployment is multi-stage.

Keep It Simple

You don't need a fancy observability platform for this. A few scripts that can query your DNS management system's internal state, check your job queues, and compare config vs deployed state will get you 90% of the way there.

The key is understanding that DNS propagation is not just a TTL thing. It's a distributed system with async tasks, caches, and multiple layers. When propagation fails, the problem is usually in those layers, not in the DNS protocol itself.

Build a tool that shows you what's actually happening in those layers. You'll save yourself hours of staring at dig output wondering why the record "should" have propagated but didn't.

Top comments (0)