DEV Community

OneLifeVibe
OneLifeVibe

Posted on

One Leaked Prefix Can Take Down Three Clouds: Route Filtering for Multi-Cloud Estates

In a single cloud, a routing mistake usually stays local. A wrong advertisement blackholes one application, one team notices, and the fix is one console away. In a multi-cloud estate the same mistake travels. A prefix learned in one cloud, re-advertised into a second, and accepted without question by a third can pull traffic for an entire region onto a path that was never built to carry it, or onto no working path at all. The outage report then lists three clouds, even though only one route was wrong.

This is not a theoretical risk from carrier networks. It happens inside private estates, with private address space, over private interconnects, because each cloud makes route acceptance easy and route filtering optional. This article is about the filtering layer most multi-cloud designs skip: deciding, in advance and in writing, which prefixes each connection is allowed to learn, advertise, and pass on.

Why Multi-Cloud Estates Leak Routes

Three habits combine to create the problem.

First, teams accept defaults. Most cloud routers and virtual gateways will accept a broad range of advertised prefixes from a peer unless a limit or a filter says otherwise. A design that works on day one with twelve prefixes quietly keeps working as the estate grows to two hundred, and nobody revisits what should be allowed.

Second, hub designs re-advertise by default. A central hub that connects AWS, Azure, Google Cloud, and Oracle is useful precisely because it passes routes between them. That transitivity is also the leak path. A route that should have stayed inside one cloud, a summary that should never have left a region, or a default route injected for a test can be learned by the hub and offered to every other cloud as if it were intentional.

Third, address plans overlap in history if not on paper. Acquisitions, older VPNs, and partner connections often bring ranges that duplicate or sit inside ranges used elsewhere. Without filtering, the longest or most recently learned version wins, and traffic follows it.

None of these require an attacker. They require only growth, time, and a change made in a hurry.

Write the Route Policy Before the Filters

Filtering fails when it tries to encode a policy nobody agreed. Start with a short table, one row per connection, that answers three questions:

  • Learn: which exact prefixes may this connection teach the estate?
  • Advertise: which exact prefixes may the estate offer out of this connection?
  • Transit: may routes learned here ever be offered anywhere else, and if so, where?

For most spoke connections the transit answer is no. A cloud region that originates its own workloads has no reason to become a transit path between two other clouds. Making that explicit turns the hub from a device that passes everything into a device with a small number of deliberate transit exceptions, each with an owner and a reason.

Keep the table with the address plan, version it with the network code, and review it whenever a new range is allocated. The filters below are just this table made executable.

Filter in Both Directions, at Every Boundary

A common half measure is to filter only inbound routes. That stops a peer from teaching bad routes, but it does nothing when the estate itself advertises the wrong thing outward, which is how a test default route or an overbroad summary escapes.

At each cloud boundary, apply both sides:

  • Inbound allowlist: accept only the prefixes listed in the Learn column for that connection. Reject everything else, and log the rejection. A rejected route is a signal, not noise. It means either the policy is stale or someone advertised something unplanned, and both are worth knowing the same day.
  • Outbound allowlist: advertise only the prefixes listed in the Advertise column. Never advertise a default route across a private interconnect unless the design explicitly calls for it and the receiving side explicitly accepts it.
  • Prefix length limits: refuse very broad advertisements that would swallow large parts of the private plan in one step, unless that summary is itself in the allowlist. Most leaks arrive as summaries, not as individual host routes.
  • Maximum prefix counts: set a ceiling per session, sized with headroom above the current count, so a peer that suddenly offers ten times its normal routes tears down or alarms instead of silently filling the routing table.

Where the platform supports route maps, prefix lists, or BGP policy objects, build them from the same source as the policy table. Hand maintained copies in four consoles will drift, and drift is the slow version of the same failure.

Treat the Hub as a Border, Not a Pipe

The hub deserves stricter rules than any spoke, because it is the only place where a leak becomes estate wide.

Tag routes by origin as they enter the hub, using BGP communities or the closest platform equivalent. A simple scheme is enough: one tag per source cloud, one for on-premises, one for partner connections. Then write transit rules in terms of tags: routes tagged from one spoke are never advertised to another spoke unless a named exception exists. Tagging also makes troubleshooting faster. When an unexpected prefix appears in a cloud, its tag says where it entered, instead of starting a search across every connection.

Add a second check at the hub: summaries are created here, and only here. Individual clouds advertise their specific ranges to the hub, the hub advertises approved summaries onward. If a specific route appears where only a summary should be, something bypassed the design.

Finally, protect the hub sessions themselves. A flapping interconnect that repeatedly relearns routes can look like a leak in monitoring and can cause the same traffic swings. Dampen alarms by session and by prefix, so the team can tell a route that appeared once from a route that will not stay learned.

Test the Filters With Bad Routes on Purpose

Filters that have never rejected anything are unproven, not perfect. In a non-production connection, or in a maintenance window on a redundant path, advertise three test cases:

  1. A prefix that belongs to a different cloud. The hub should refuse to pass it onward, and the refusal should be logged and visible.
  2. An overbroad summary covering space the estate does not own. It should be rejected on prefix length or allowlist grounds.
  3. A burst of harmless test prefixes above the normal count. The maximum prefix control should alarm at the agreed threshold.

Record what each platform actually did, not what the documentation says it should do. Behaviour differs between gateways, and the time to learn that a filter applies only to one direction, or only after a session reset, is during a test, not during an incident.

Repeat the test after any platform upgrade or gateway replacement. Those changes reset more policy than their release notes suggest.

Watch for the Slow Signals

Route filtering also gives an estate its earliest warning of address plan problems. Review rejected route logs weekly. A steady trickle of rejections from one connection usually means a team stood up a new range without updating the policy, which is a process fix. A sudden flood means something structural changed. Either way, the filter caught what a traffic graph would only have shown after users were affected.

Pair that review with a simple inventory check: compare the routes actually learned on each connection against the Learn column. Any difference, in either direction, is work. A missing expected route is a future outage hiding behind redundancy. An extra accepted route is a gap in the filter.

What Good Looks Like

A well filtered estate is quiet in a specific way. New ranges appear only after the policy table changes. The hub passes a short, stable set of summaries. A mistake in one cloud produces a rejection log in that cloud, not a traffic shift in three. Engineers can answer, from the table alone, which prefixes any connection may learn or advertise, and the running filters match the table because both come from the same source.

That discipline is a regular part of the private connectivity design work at CloudVerg, where multi-cloud links built on VergMesh and delivered through VergForge are given explicit learn, advertise, and transit rules for every connection, so growth in one cloud cannot quietly rewrite routing in the others.

Routing will always change as estates grow. The goal is not to freeze it. The goal is to make every change pass a gate that was written down while everyone was calm, so the next wrong prefix stays a log line instead of becoming an outage.

Top comments (0)