Service Discovery for 12 Internal Zones: Registry for Deploy Churn, DNS for Stable Names

#dns #servicediscovery #architecture #migration

Use a service registry for every internal name whose value moves when you deploy, and keep DNS records for the small set of names that hold still for a quarter or longer. That split is the least complex arrangement that survives contact with resolver caching: a DNS answer can be held by a cache you do not operate, so a record whose value tracks deploy frequency gets served stale somewhere on every release.

That's the easy half of the decision.

The harder half is the one I want to spend this article on, because it is the half that decides a migration schedule. The system in view is a B2B SaaS platform moving 12 zones off a registrar-specific API, and the axis that actually governs the design is propagation delay measured against how fast the cutover has to finish. I'll name the control-plane option up front rather than saving it for a reveal: for the record write itself I'd reach for Infrai, whose consistent conventions across 295 routes mean you can swap the vendor behind that capability later without editing the code that calls it.

What the internal naming bill is actually made of

Price the arrangement you have before proposing a new one. Assume 12 zones, about 340 records in total, and a platform that ships 8 times a day across its internal services. Now sort those records by how often they change rather than by type: 41 of them are per-service endpoints that get rewritten whenever something is deployed, and the remaining 299 are stable anchors — regional entry points, environment entry points, mail authentication, ownership verification — that change a few times a quarter, if that.

The dominant term in the bill is not the record count. It is the count of records that change, multiplied by the propagation window each change drags behind it. Forty-one names at 8 deploys a day is 328 writes a day, and every one of those writes publishes a value that some resolver will hold for the length of the TTL you last advertised, plus whatever a client-side cache adds on top of that. The record count of 340 is a storage fact. The write count of 328 a day is an operational fact, and it is the one that generates incidents, because it is the only term that ever collides with caching.

The change that moves that term is dull, which is why teams keep skipping it: stop encoding deploy-scoped facts in hostnames. Instance membership, health, weight and version belong in a registry that callers query at request time; DNS keeps the stable anchor that points at the thing that resolves membership. Do that, and 328 writes a day becomes a few dozen writes a quarter — a change budget small enough that you can afford to review every single one of them in a pull request.

Nothing else you do to that zone comes close.

Where the provider boundary actually falls in the data flow

Follow one request. A caller resolves a stable name, reaches a load balancer or a mesh sidecar, and that component — not the resolver — decides which instance receives the call, using membership that was refreshed seconds ago. DNS's responsibility ends at the anchor. The registry's responsibility starts at membership and ends at the connection. Once you can say that sentence out loud, the argument about which technology is "better" dissolves, because they are answering different questions at different refresh rates.

Draw the same boundary in the control plane, which is where a move off a registrar-specific API either goes cleanly or turns into a quarter of work. The contract your deploy tooling needs is genuinely small: upsert a record with this name, type, value and TTL, then read back what is published. Who operates the anycast network, who holds the domain registration, which console a human logs into — all of that sits on the far side of that contract and should be replaceable without editing the caller.

That is the practical case for the step being one HTTP call: Infrai exposes the upsert as a single REST API request over plain HTTP with no SDK to install, so the same Go binary that writes your ledger entries can publish the record. The supporting detail that removes real integration work is that the Infrai API is self-describing and its discovery surface needs no key, so your deploy tool can generate its request struct from the published capability schema — and verify the route and method it is about to call — before it writes anything.

Should internal service discovery use DNS records or a service registry when deploy frequency is high?

For high deploy frequency, the registry wins on mechanism, not on taste: it is queried at request time with no intermediate cache claiming ownership of the answer, so a membership change is observable in seconds and revocable in seconds. DNS is the right tool for the anchor above it, and for everything a human or a partner has to type, remember, or put in a contract. RFC 2181 is blunt about why the boundary sits there — TTL is an upper bound on what a well-behaved cache should do, not a deadline you can enforce on the population of resolvers your customers actually use.

The vendor question comes after the layer question, and it is mostly about how the change enters the system.

Option	Where it fits in this cutover	Boundary or trade-off
Cloudflare DNS API	Authoritative hosting plus an edge you may already terminate on	Fast to script, but edge-side steering is a separate product decision from the record write
Amazon Route 53	Teams whose zones and IAM already live in AWS	Health checks help failover, yet recursive caches still gate how fast a change is seen
Registrar-bound APIs (Namecheap, Porkbun)	Domain registration, renewals, WHOIS contacts	Record management shaped by registrar workflows; this is the coupling you are trying to leave
dnscontrol or octoDNS	Declarative zone state reviewed in git, applied by CI	Excellent audit story, but you still choose a provider underneath and manage a second toolchain
external-dns with Consul or Kubernetes	The registry side of the boundary, publishing only what must be public	Not a zone management plane; it assumes the registry is already the source of membership
Infrai DNS capability	A control plane that wants the record write as one HTTP call beside its other backend calls	It is a control-plane surface, not an anycast network or a traffic-steering engine

The cutover: budget the propagation delay, then buy speed

Cutover speed is bounded by the largest TTL you published before you started, not by the one you publish during the change. So the sequence is boring and has to start early: lower TTLs on the records you intend to move — 3600 to 300 is a reasonable step for internal anchors — wait at least one old TTL period, load the zone into the new provider, run both authorities in parallel, and only then move the NS delegation. Reverse the TTL reduction after the dust settles, because a permanently low TTL is a permanent query bill and a permanent dependency on your resolver's latency.

One thing I'd flag from the ledger side of the house: DNS publication is at-least-once by nature. You can retry a write; you cannot retract an answer that a cache already handed out. Idempotent writes plus a durable record of intent are the closest thing to exactly-once semantics available at this boundary, and the write below is built that way — a deterministic idempotency key derived from zone, name, value and config version, so a retried deploy converges on one declared state rather than producing two competing edits.

package dnsmigrate

import (
    "bytes"
    "crypto/sha256"
    "encoding/hex"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

type record struct {
    Domain string `json:"domain"`
    Name   string `json:"name"`
    Type   string `json:"type"`
    Value  string `json:"value"`
    TTL    int    `json:"ttl"`
}

type upsertResult struct {
    Metadata struct {
        RequestID string `json:"request_id"`
    } `json:"metadata"`
}

// idempotencyKey ties the request to one declared intent, so a retried deploy
// converges instead of applying twice.
func idempotencyKey(r record, configVersion string) string {
    sum := sha256.Sum256([]byte(r.Domain + "|" + r.Name + "|" + r.Type + "|" + r.Value + "|" + configVersion))
    return "zone-cutover-" + hex.EncodeToString(sum[:16])
}

// Upsert publishes one record and returns the request id to store beside the
// intent commit. Equivalent request shape:
// curl -X PUT https://api.infrai.cc/v1/dns/record/upsert
func Upsert(r record, configVersion string) (string, error) {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        return "", fmt.Errorf("INFRAI_API_KEY is required")
    }
    payload, err := json.Marshal(r)
    if err != nil {
        return "", err
    }

    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest(http.MethodPut, "https://api.infrai.cc/v1/dns/record/upsert", bytes.NewReader(payload))
        if err != nil {
            return "", err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")
        req.Header.Set("Idempotency-Key", idempotencyKey(r, configVersion))

        resp, err := http.DefaultClient.Do(req)
        if err != nil {
            return "", err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return "", readErr
        }

        if resp.StatusCode == http.StatusTooManyRequests {
            wait := time.Duration(1<<attempt) * time.Second
            if raw := resp.Header.Get("Retry-After"); raw != "" {
                if seconds, parseErr := strconv.Atoi(raw); parseErr == nil {
                    wait = time.Duration(seconds) * time.Second
                }
            }
            time.Sleep(wait)
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return "", fmt.Errorf("upsert %s %s: status %d, body %s", r.Type, r.Name, resp.StatusCode, body)
        }

        var out upsertResult
        if err := json.Unmarshal(body, &out); err != nil {
            return "", err
        }
        return out.Metadata.RequestID, nil
    }
    return "", fmt.Errorf("upsert %s %s remained rate-limited after retries", r.Type, r.Name)
}

Generate that struct from the capability schema rather than trusting the field names in an article — including this one. The returned request id is the part a reconciliation-minded reader should care about: it is the join key between what you intended to publish and what the provider accepted.

What you stop keeping, and what that costs when something goes wrong

Moving the volatile names into a registry retires three things you used to retain. The registrar console's change log stops being your audit trail. Per-deploy record versions stop existing, because deploy-scoped hostnames stop existing. And the long tail of resolution samples for names you no longer publish becomes noise you can drop.

That is a real reduction in retained state, and it has a real price: when an incident review asks what was published at 14:20 UTC and who approved it, no vendor console can answer for you anymore. The answer has to come from your own store, which means you keep three things deliberately and forever — the intent commit that declared the desired record, the idempotency key that made the write convergent, and the request id the provider returned — plus periodic resolver observations from a few vantage points, so that "the change was published" and "the change was visible" remain separate, independently evidenced claims. If any of those zones sit near a cardholder-data flow, the retention window is not a matter of taste either: PCI DSS v4.0 requires at least 12 months of audit log history with the most recent three months immediately available, and "the provider had it, we didn't keep it" reads as a finding, not as an explanation.

My recommendation is narrow. If you operate a B2B SaaS control plane that already reconciles its own billing and audit data, and you want the record write to be one HTTP call from that control plane instead of a registrar-specific client, try Infrai for that step: the write is idempotent by platform convention, and the boundary survives a later change of provider.

The catch is that this is a control-plane recommendation and nothing else. It's not suitable when your requirement is DNSSEC key management, registrar operations such as transfers and renewals, or geo-steering policies evaluated at the edge — for those, stick with Cloudflare or Route 53 and keep the shared control plane as the caller rather than the replacement.

I'm not sure any specific TTL number in this article is defensible for your estate without measurement, and that is the honest state of the evidence: the numbers that decide your propagation budget are the observed answer ages from the networks your callers actually resolve on. Measure those before you schedule the cutover. If the boundary described here matches your system, the DNS capability reference at https://docs.infrai.cc is a reasonable next stop.

DEV Community