DEV Community

OneLifeVibe
OneLifeVibe

Posted on

The Map Is Not the Latency: Measuring Real Paths Between Cloud Regions Before Committing Workloads

Placement decisions in a multi-cloud estate usually start with a map. A workload sits in one cloud's region, its data sits in another cloud's region a few hundred kilometres away, and the two points look close enough that nobody questions the pairing. Then the application goes live and every request takes three times longer than the design assumed, not because the distance was wrong, but because packets do not travel the way maps suggest.

Geographic distance sets a floor for latency. It never sets the actual number. What a workload experiences is the path: which networks carry the traffic, where those networks hand off to each other, how far the traffic detours before it turns toward its destination, and how loaded each segment is at the hour the workload actually runs. Two region pairs at similar distances can differ by a factor of two or three in round trip time, and the same pair can behave like two different networks at different times of day.

Why the obvious region pairing is often the wrong one

Cloud regions are not points on a road network. A provider's region in one city may route traffic out through a different city entirely before it heads toward the other cloud, because that is where the interconnect points, the backbone entry, or the peering exchange happens to be. Traffic between two clouds in the same metro can leave the metro, visit a hub hundreds of kilometres away, and come back.

Three patterns catch teams out most often.

First, the hairpin: both clouds are present in the same city, but their networks only meet somewhere else, so a request that looks local travels a long loop. Second, the asymmetric return: the outbound path is direct and the return path is not, so a measurement that only looks one way, or a tool that reports a single averaged number, hides which direction is slow. Third, the peak hour divergence: the path is fast at the time it was tested, usually mid morning during a quiet design week, and noticeably slower during the estate's real peak, when every other tenant of the same shared segments is also busy.

None of these show up in a region list, a pricing page, or a single ping run from a laptop. All of them show up in production, attached to a workload that is now expensive to move.

Measure the path the workload will actually take

A useful latency measurement has four properties, and most quick tests have none of them.

Measure from inside the estate. Run the probes from real instances or containers in the source region, in the same network configuration the workload will use, to real endpoints in the destination region. A test from an office connection measures the office connection. A test over the public internet says little about the private interconnect path, and a test over the private path says little about the internet path. If the workload could use either, measure both and record which is which.

Measure the whole exchange, not just the echo. A ping round trip is a floor, not a forecast. Application latency also pays for connection setup, TLS negotiation, and the number of round trips the protocol makes before useful data moves. A chatty protocol can turn a modest 30 millisecond path into a second of waiting across dozens of sequential calls, while a protocol that pipelines or reuses connections barely notices the same path. Time a realistic request pattern: connection setup separately, first byte, and a complete representative transaction.

Measure over days, and at the workload's hours. A single afternoon of clean results proves the path can be fast. It does not prove it usually is. Run lightweight probes continuously for at least several days, keep the results as percentiles rather than averages, and read the numbers for the hours the workload is busiest. The 95th percentile at peak is the number a design should be built on. The average is the number that makes a design look fine in a review and slow in production.

Measure in both directions, with the real packet sizes. Request and response are rarely the same size, and large transfers behave differently from small probes on congested or shaped paths. If the workload ships large payloads one way and small answers back, test exactly that shape. Record loss and jitter alongside latency: for interactive traffic, variable delay is often worse than a slightly higher but steady delay, because it defeats timeouts, retries, and the user's patience in less predictable ways.

Turn measurements into a placement decision

Measurement only matters if it changes a decision, so decide the thresholds before measuring. For each workload, write down what the path has to deliver: the round trip budget the application design assumes, the percentile that budget applies at, and what happens if the best available pairing misses it.

That last part is where the real design work lives, because a missed budget is not a dead end. It is a fork with several honest options. Move the workload closer to the data. Move a copy of the data closer to the workload, accepting the replication and consistency work that creates. Reduce the round trips, by batching calls, caching what is safe to cache, or replacing a chatty integration with an asynchronous one. Or accept the latency deliberately, with the number written into the design and the timeouts, retry policies, and user experience built around the measured value rather than an assumed one.

What does not work is committing first and measuring later. Region choices harden quickly: data accumulates, integrations multiply, other teams build on the assumption that the pairing is permanent. Re-measuring after a major change, a new interconnect, a provider backbone change, a workload moving regions, is equally part of the discipline. Paths change underneath estates that do not move at all, and a pairing that was right last year deserves a fresh measurement, not inherited confidence.

A short checklist for region pairing decisions

  1. Never pair regions on distance or same city naming alone.
  2. Probe from inside the real network configuration, on the real path, private and public measured separately.
  3. Time connection setup, first byte, and a complete representative transaction, not just ping.
  4. Collect days of data and design to the peak hour 95th percentile, not the average.
  5. Test both directions with realistic payload sizes, and record jitter and loss.
  6. Set the latency budget and the fallback options before measuring, so results drive a decision.
  7. Re-measure after interconnect, provider, or placement changes, and on a schedule even when nothing changed.

The map will always be the easiest input to a placement decision, and the least reliable one. The estates that place workloads well are the ones that treat latency as something measured on their own paths, at their own hours, with their own traffic shape, before the commitment is made. That measurement discipline, applied to private connectivity between clouds, is part of the multi-cloud network design work described at cloudverg.com, where VergMesh and VergRail are used to build and run those measured paths.

Distance is a guess. A measured path is a design.

Top comments (0)