DEV Community

OneLifeVibe
OneLifeVibe

Posted on

The Third Cloud Breaks Your Peering Model: Transitive Routing and the Hub Decision

Two clouds are easy to connect. A pair of private links, a handful of route tables, and traffic flows both ways. The design fits on a whiteboard and it works — until a third cloud arrives, usually through an acquisition, a GPU region that only exists on one provider, or a SaaS workload that had to live somewhere specific.

That is the moment most multi-cloud networks quietly break. Not with an outage, but with a diagram that stops matching reality.

Peering is not transitive, and that is the whole problem

In most cloud networks, peering is deliberately non-transitive. If network A is peered with network B, and B is peered with network C, A cannot reach C through B. Each provider enforces this for good reasons — isolation, billing boundaries, and failure containment — but it means a full mesh is the only way to keep direct connectivity as the estate grows.

The maths turns against the design quickly. Two networks need one connection. Three need three. Four need six. Five need ten. Every new connection is not just a link: it is route tables on both sides, firewall rules on both sides, health checks, and a new place where a prefix can be advertised incorrectly.

Worse, the mesh hides an assumption. Teams start treating "connected" as a property of the estate, when it is actually a property of each pair. A service that works from cloud one to cloud two may have no path at all from cloud one to cloud three, and the failure looks like an application bug — timeouts, retries, partial results — rather than what it is: there was never a route.

What a hub actually changes

A hub-and-spoke design replaces the mesh with a transit point. Each cloud network connects once, to the hub, and the hub forwards between spokes under explicit policy. Adding a fourth or fifth cloud becomes one new spoke instead of a new connection to every existing network.

That simplification is real, but it is not free. Three trade-offs arrive with it, and a useful design names all three up front.

First, latency. Traffic between two spokes now crosses the hub, so placement matters. A hub in the wrong region turns a short cross-cloud hop into a long detour, and for chatty service-to-service calls or GPU checkpoint traffic that detour is paid on every request.

Second, blast radius. The hub is now on the path for everything. If it fails, fails over badly, or receives a bad route advertisement, spokes lose each other at the same time. The mesh failed one pair at a time; the hub can fail all pairs at once. That argues for a hub that is itself redundant across zones, with route advertisements validated before they are accepted, not after traffic has already shifted.

Third, cost shape. Mesh costs grow with the number of pairs. Hub costs grow with the volume that crosses the hub, including traffic that previously went direct. Neither shape is always cheaper — the right comparison is measured traffic between specific spokes, not the total estate size.

Three ways teams build the transit layer

Cloud-native transit services attach directly to one provider's networking model. They are the fastest to adopt inside that provider and handle route propagation natively, but each provider's version stops at its own boundary, so a three-cloud estate often ends up with three hubs that then need connecting to each other — the mesh problem, one level up.

Self-managed routing, usually virtual appliances or open-source routers in a dedicated transit network, works the same way in every cloud and gives full control over route policy and filtering. The cost is operational: patching, high availability, and performance testing become the team's job, and throughput limits are discovered in production unless they are load-tested deliberately.

Managed overlay fabrics build a private backbone between clouds and present one routing and policy model across all of them. They trade some provider-native visibility for consistency: the same route policy, the same segmentation, and the same troubleshooting view in every cloud. For estates where different platform teams own different clouds, that consistency is often the deciding factor, because most cross-cloud incidents are really disagreements between three locally-correct configurations.

None of these is the correct answer in general. The useful question is narrower: where should route policy live so that exactly one place can be checked when two clouds cannot talk?

Route policy is the design, the links are not

Whether the transit is native, self-managed, or an overlay, the failures that matter are almost always policy failures: a spoke advertising a default route it should never advertise, overlapping prefixes arriving at the hub from two spokes at once, or summarisation that hides a more specific route a workload depends on.

A hub design that survives growth therefore keeps a few disciplines from the start:

  1. One source of truth for which prefixes each spoke owns, validated at the hub before advertisement is accepted.
  2. No spoke may advertise a default route or another spoke's prefixes. Reject, alert, and log it when one tries.
  3. Summarise only where the underlying addressing plan guarantees no workload needs a hidden, more specific route.
  4. Test spoke isolation deliberately: a new spoke should reach only what policy grants it on day one, not everything the hub can route to.
  5. Record measured latency and loss per spoke pair through the hub, so the detour cost is known from data rather than discovered from application complaints.

Addressing discipline decided early pays for all of this. If each cloud and each region receives a distinct, non-overlapping block from a single plan, hub routing stays simple. If prefixes were picked independently per cloud, the hub inherits that collision problem as well, and no transit product fixes addressing after the fact.

When to stay with the mesh

A hub is not mandatory at three networks. If traffic between the third cloud and the other two is small, one-directional, or served well by a single private API endpoint, a full transit layer may be more machinery than the estate needs. The signal to move is not the cloud count alone. It is when route tables are being edited in several places to add one service, when nobody can answer which path traffic between two clouds actually takes, or when a new region takes longer to connect than the workload took to build.

At that point the peering model has already broken — the hub decision is just acknowledging it and choosing where the complexity will live instead of letting it spread across every pair.

Teams that operate these estates day to day — including the work described at CloudVerg around VergMesh private multi-cloud connectivity and VergForge network design — tend to treat the transit decision as an addressing and policy decision first and a product decision second. However the estate is built, the principle holds: make routing transitive on purpose, in one governed place, or accept that every new cloud multiplies the work and the failure modes of all the clouds already there.

Top comments (0)