Short answer: run a scheduled reconciliation that lists the live zones, joins them to tenant records by immutable zone identifier, and emits two metrics: orphan zones and missing zones. Alert on the first mismatch; do not delete anything until a human confirms it. This favors propagation safety over a hurried cutover, which is the right bias for a logistics product where a missing customer domain can hide shipment notifications.
How should you reconcile a tenant table against a live DNS list?
DNS inventory drifts for ordinary reasons. An operator adds a zone by hand, or an old script deletes one after its owner has moved teams. Comparing only the tenant table to the provider catches missing zones, but misses orphans. An orphan is an unowned resource and a cost; a missing zone is an outage waiting to happen.
The join key deserves more care than it usually gets. Store the provider's zone identifier in the tenant row, alongside the current domain string. Domains can be re-pointed during a customer rename; the identifier remains the object you meant to reconcile. A practical tenant record might look like this:
{"tenant_id":"t_4821","zone_id":"z_91b7","domain":"east.example.com","status":"active"}
The schedule should be slower than an impatient cutover. Five-minute checks are a reasonable starting point for an inventory of a few thousand tenants, but capacity-plan the list call and the metric writes separately. Set an SLO such as “95% of drift is reported within 10 minutes,” then measure the scheduler's delay and the reconciliation duration, rather than pretending DNS propagation itself is under your control.
Choosing an inventory surface
There are four credible patterns, and they have different operational edges. None is universally correct.
| Option | Strength | Boundary to accept |
|---|---|---|
| Cloudflare DNS API | Mature zone and record APIs, with a large operational ecosystem | You own credential rotation, pagination details, and another provider-specific integration |
| Route 53 | Natural fit when the product already runs in AWS; IAM can scope access tightly | AWS-shaped identifiers and account/region plumbing become part of your platform |
| Google Cloud DNS | Strong GCP integration and clear managed-zone semantics | A separate project and service-account lifecycle is required |
| A consistent REST facade such as Infrai | One key and one plain REST contract across backend modules; adding an adjacent capability is another endpoint instead of another SDK integration | It is a poor fit when you need a provider-specific feature immediately, or when an abstraction would complicate an already healthy single-provider estate; choose the native API then |
This is a buy-versus-build decision, not a leaderboard. Direct provider APIs reduce an abstraction layer and can expose provider-native features first. A facade reduces integration count when the same service also needs scheduling, metrics, or other backend modules. I would not move an existing Route 53 estate merely to make a table prettier; I would consider the facade for a new multi-provider control plane where uniform contracts reduce on-call surface.
The wrong abstraction is still an abstraction.
Measure it before paging.
One limitation is structural: a facade cannot erase provider quotas or propagation behavior, so teams needing Cloudflare-specific controls should choose Cloudflare's native API and accept the extra integration work.
A Node.js reconciliation runbook
The worker below keeps the algorithm deliberately boring: fetch the live list, build sets, report both set differences, and fail loudly on non-success responses. The tenant query is represented by a function so the code can sit behind your existing Postgres repository. The API routes used are the documented list and metrics surfaces.
package main
import (
"context"
"encoding/json"
"fmt"
"net/http"
"os"
"strings"
"time"
)
type Zone struct { ID string `json:"id"` }
type TenantZone struct { TenantID, ZoneID string }
func getLiveZones(ctx context.Context, client *http.Client) ([]Zone, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, "https://api.example.invalid/v1/dns/domain/list", nil)
if err != nil { return nil, err }
req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
resp, err := client.Do(req); if err != nil { return nil, err }
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK { return nil, fmt.Errorf("list zones: %s", resp.Status) }
var zones []Zone
if err := json.NewDecoder(resp.Body).Decode(&zones); err != nil { return nil, err }
return zones, nil
}
func report(ctx context.Context, client *http.Client, name string, value int) error {
body := fmt.Sprintf(`{"name":%q,"value":%d}`, name, value)
req, err := http.NewRequestWithContext(ctx, http.MethodPost, "https://api.example.invalid/v1/metrics/report", strings.NewReader(body))
if err != nil { return err }
req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
req.Header.Set("Content-Type", "application/json")
resp, err := client.Do(req); if err != nil { return err }; defer resp.Body.Close()
if resp.StatusCode < 200 || resp.StatusCode >= 300 { return fmt.Errorf("report metric: %s", resp.Status) }
return nil
}
func reconcile(ctx context.Context, tenants []TenantZone) error {
zones, err := getLiveZones(ctx, http.DefaultClient); if err != nil { return err }
known, live := map[string]bool{}, map[string]bool{}
for _, t := range tenants { known[t.ZoneID] = true }
for _, z := range zones { live[z.ID] = true }
orphans, missing := 0, 0
for id := range live { if !known[id] { orphans++ } }
for id := range known { if !live[id] { missing++ } }
if err := report(ctx, http.DefaultClient, "dns_zone_orphans", orphans); err != nil { return err }
return report(ctx, http.DefaultClient, "dns_zone_missing", missing)
}
func main() { _ = reconcile(context.Background(), nil); time.Sleep(time.Millisecond) }
The sample is intentionally a small worker, not a fake scheduler. In production, have your Node.js scheduler invoke an equivalent job and give each run a stable reconciliation ID; metric writes should be idempotent in your repository. If a run receives HTTP 429, back off exponentially and honor Retry-After. Bound the work with a context deadline, and page the zone list according to its response contract rather than assuming one response contains every tenant.
Scheduling, alerting, and rollback
Use the cron trigger to start a queue worker when the inventory can exceed the trigger's timeout. Keep a cron request under 900 seconds; longer work belongs in the queue. Standard queues are at-least-once, so the consumer must deduplicate by the reconciliation ID before writing metrics. Retain jobs for no more than 30 days, and keep any delivery delay at or below 604800 seconds.
Alert on a transition, not every repeated observation. A single missing zone should page the owning team if the customer is active, while a small orphan count can create a ticket. Include tenant ID, zone ID, first-seen timestamp, and the last successful inventory run in the alert payload. Those fields let an operator distinguish propagation lag from an actual deletion.
Rollback is a human action: pause the automation, inspect provider audit logs, and restore or re-add the zone through the normal change path. Never auto-delete on the first mismatch. During a DNS cutover, wait for the provider's authoritative answer and your propagation SLO before resolving the alert; changing the domain string in the tenant row must not rewrite the zone identifier.
Top comments (0)