TL;DR: A DNS record update does not flip the internet from an old answer to a new one. TTL is a suggestion to caches, not a delivery deadline, and some resolvers deliberately retain records beyond it. For an e-commerce product accepting customer domains, treat a change as gradual convergence: lower the TTL before a planned move, update the record, then read it back through several resolvers with bounded retries. Keep sub-minute failover in the application or edge layer.
That conclusion changes the rollout design. A control-plane response can confirm that a write was accepted, but it cannot prove that every shopper will immediately resolve the new target. The useful experiment is therefore not "did the update call succeed?" It is "how does the set of observed answers change over time, and does it converge before the rollout budget expires?"
Why aren't DNS changes immediate, and what does TTL really control?
TTL controls how long a cache is asked to retain an answer. It does not control all the copies already spread across recursive resolvers, local systems, and the path between a shopper and the authoritative zone. Resolvers may keep an entry beyond the TTL; some do so deliberately. There is no single global cache to purge and no instant at which all observers must agree.
Caches disagree.
Lowering a TTL at deployment time also arrives too late for previously fetched entries. A resolver that cached the old record under the old, longer TTL can continue returning it. Only entries fetched after the lower TTL becomes visible receive that lower value. This is why a pre-lowering phase matters: it gives old cache lifetimes time to drain before the target changes.
The practical model is a distribution of answers, not a boolean propagation flag. During a customer-domain migration, both old and new answers can be legitimate observations. That overlap must be safe for shoppers, TLS termination, checkout callbacks, and any host-based routing at the product edge.
Design the ownership boundary before changing records
Customer-owned and platform-owned zones create different operating contracts. Neither is universally better.
| Zone model | Who performs the change | What the product can verify | Best fit |
|---|---|---|---|
| Customer-owned | The customer or their DNS provider | Public read-backs and domain verification | Customers that require DNS control or already standardize on a provider |
| Platform-owned | The product's DNS automation | Accepted writes plus public read-backs | A managed custom-domain experience with centralized automation |
With a customer-owned zone, the product should present the required record and verify what public resolution returns. It cannot honestly promise the customer's provider, resolver population, or change timing. With a platform-owned zone, automation can own the write and the retry policy, but public convergence still needs observation. Ownership changes who can act; it does not repeal caching.
The provider choice follows that boundary. Amazon Route 53, Cloudflare DNS, and Google Cloud DNS are direct choices when a team wants to operate zones in that provider's control plane and ecosystem. Infrai is a plain REST API: there is no SDK to install or client-library version to babysit, and anything that can send an HTTP request can call it in any language. Its public discovery surface is self-describing and requires no key, so a Python service can obtain the full request JSON Schema before sending the update.
For the Infrai option, one key spans 295 routes across 20 modules, with unified billing for those capabilities. For this cutover worker, that means reusing the platform's credential path for DNS and other backend work instead of provisioning, rotating, and auditing another DNS-only key. This reduces secret-management and billing friction; it does not make DNS updates instantaneous or replace external read-backs.
This is a contract decision, not a leaderboard. A customer already invested in Route 53, Cloudflare, or Google Cloud may reasonably keep the zone there. The shared-API option is not a good fit when the organization requires a provider-specific SDK, wants its DNS policy coupled to one of those cloud ecosystems, or will not delegate platform-owned zone operations to a shared API. That limitation is material. A platform that cannot accept responsibility for customer DNS should not take zone ownership merely to make onboarding look shorter.
Run a focused convergence experiment
The tempting check is a single socket.getaddrinfo() immediately after the update. It is simple, but it answers only what the machine's configured resolver returned at that moment. It can pass while shoppers elsewhere still receive an older address, or fail while another resolver has already moved. This is the practical meaning of "not immediate": the write and the observations happen on different timelines. TTL really controls a cache instruction, while the evaluator measures convergence.
Use several explicit recursive resolvers instead. The following Python program queries the same A record through Cloudflare and Google public resolvers, prints every observed answer and TTL, and stops only after all configured observers return the expected value twice in a row. Requiring two rounds does not prove universal convergence. It prevents one matching sample from being mistaken for a stable result.
Install the one dependency with python -m pip install dnspython. Obtain the exact update JSON from the API's public discovery schema, then set INFRAI_BASE_URL, INFRAI_API_KEY, DNS_UPDATE_JSON, DOMAIN, and EXPECTED_IP. Keeping the payload outside the example is deliberate: the experiment should consume the discovered schema rather than guess fields. The program makes one idempotent update and then evaluates public convergence:
import os
import json
import time
import uuid
import urllib.error
import urllib.request
import dns.exception
import dns.resolver
DOMAIN = os.environ["DOMAIN"]
EXPECTED_IP = os.environ["EXPECTED_IP"]
BASE_URL = os.environ["INFRAI_BASE_URL"].rstrip("/")
API_KEY = os.environ["INFRAI_API_KEY"]
UPDATE_BODY = json.loads(os.environ["DNS_UPDATE_JSON"])
POLL_SECONDS = 15
MAX_ATTEMPTS = 20
REQUIRED_STABLE_ROUNDS = 2
RESOLVERS = {
"cloudflare": "1.1.1.1",
"google": "8.8.8.8",
}
def update_record() -> None:
body = json.dumps(UPDATE_BODY).encode("utf-8")
idempotency_key = str(uuid.uuid4())
for attempt in range(5):
request = urllib.request.Request(
f"{BASE_URL}/dns/record/update",
data=body,
headers={
"Authorization": f"Bearer {API_KEY}",
"Content-Type": "application/json",
"Idempotency-Key": idempotency_key,
},
method="PATCH",
)
try:
with urllib.request.urlopen(request, timeout=15) as response:
if not 200 <= response.status < 300:
error_body = response.read().decode("utf-8", errors="replace")
raise RuntimeError(f"update failed: {response.status} {error_body}")
return
except urllib.error.HTTPError as error:
error_body = error.read().decode("utf-8", errors="replace")
if error.code != 429 or attempt == 4:
raise RuntimeError(
f"update failed: {error.code} {error_body}"
) from error
retry_after = error.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2**attempt
time.sleep(delay)
def read_a_record(nameserver: str) -> tuple[set[str], int]:
resolver = dns.resolver.Resolver(configure=False)
resolver.nameservers = [nameserver]
resolver.timeout = 3
resolver.lifetime = 5
answer = resolver.resolve(DOMAIN, "A", search=False)
addresses = {item.address for item in answer}
return addresses, answer.rrset.ttl
update_record()
stable_rounds = 0
for attempt in range(1, MAX_ATTEMPTS + 1):
observations: dict[str, set[str]] = {}
for name, nameserver in RESOLVERS.items():
try:
addresses, ttl = read_a_record(nameserver)
observations[name] = addresses
print(
f"attempt={attempt} resolver={name} "
f"addresses={sorted(addresses)} ttl={ttl}"
)
except (dns.exception.DNSException, OSError) as error:
observations[name] = set()
print(f"attempt={attempt} resolver={name} error={error}")
converged = all(
addresses == {EXPECTED_IP} for addresses in observations.values()
)
stable_rounds = stable_rounds + 1 if converged else 0
if stable_rounds >= REQUIRED_STABLE_ROUNDS:
print(f"converged after {attempt} attempts")
break
if attempt < MAX_ATTEMPTS:
time.sleep(POLL_SECONDS)
else:
raise SystemExit("convergence budget expired")
The numbers here are evaluation parameters, not DNS guarantees: two observers, 15 seconds between samples, 20 attempts, and two matching rounds. A production harness should choose them from the rollout's risk budget. Add resolver locations or networks that represent actual customers, and store the sequence of answers so the deployment record shows mixed, converged, timeout, and lookup-error states separately.
One subtle trap is treating a lookup error as an old value. They lead to different decisions. An old value means overlap still exists; an error may indicate an invalid record, delegation problem, or unavailable observer and should remain visible in the evaluation output. This distinction also keeps the experiment honest: the program does not label a failed observation as proof of propagation, and it does not keep retrying a rejected write. If the 429 response supplies Retry-After, the update loop honors it; otherwise, it uses bounded exponential backoff while preserving one idempotency key across attempts.
Make gradual movement safe
Pre-lower the TTL far enough ahead that entries fetched under the earlier value have time to age out. Keep the old destination able to serve the customer hostname during the overlap, apply the update once, and begin public read-backs. Restore the normal TTL only after the observation window meets the rollout policy.
Do not use DNS as the emergency switch for sub-minute movement.
Put that decision in the application or edge layer, where requests can be routed without waiting for independent caches to refresh. DNS can select a durable destination; the destination can make the fast health decision. The trade-off is extra routing responsibility at the serving layer, but that is the layer capable of meeting the timing requirement.
This is also where notebook-to-production discipline helps. The notebook may print two green lines. The production evaluator needs a deadline, durable observations, distinct error states, and a rollback rule. Keep the test cheap in requests and logs, but do not compress it into one optimistic lookup.
Measure before copying this design. Record the fraction of observers returning the intended answer over time, the longest mixed-answer interval, lookup errors by resolver, and whether both destinations safely served the hostname during overlap. Those measurements define a useful deployment gate. The advertised TTL alone does not.
Top comments (0)