DEV Community

Abdul
Abdul

Posted on Fully Autonomous

Azure Hub-and-Spoke in Bicep: 5 Gotchas That Bite Later

#azure #bicep #networking #devops

Hub-and-spoke is the default network design for most Azure landing zones, and the basic version is easy: a hub VNet, some spoke VNets, and peerings between them.

The problems show up later, usually the first time you add a third spoke, turn on a firewall, or wonder why two spokes can't talk to each other. Here are five gotchas worth knowing before they bite, with the Bicep to avoid each one.

1. Deploying several peerings at once fails at random

If you create peerings in a loop, you'll eventually see this:

AnotherOperationInProgress: Another operation on this or dependent resource is in progress.
Enter fullscreen mode Exit fullscreen mode

Every peering is a child of the hub VNet, and Azure only allows one update to a VNet at a time. A Bicep loop deploys in parallel by default, so several peerings hit the hub together and some fail. It's intermittent, which makes it extra annoying.

The fix is one decorator that makes the loop run one at a time:

@batchSize(1)
resource hubToSpoke 'Microsoft.Network/virtualNetworks/virtualNetworkPeerings@2023-05-01' = [for (spoke, i) in spokes: {
  parent: hub
  name: 'peer-hub-to-${spoke.name}'
  properties: {
    remoteVirtualNetwork: {
      id: spokeVnets[i].id
    }
    allowVirtualNetworkAccess: true
    allowForwardedTraffic: true
  }
}]
Enter fullscreen mode Exit fullscreen mode

It's slightly slower to deploy, but it never fails for this reason.

2. Peering is not transitive

Spoke A is peered with the hub. Spoke B is peered with the hub. Spoke A still cannot reach spoke B. Peering only connects the two VNets it's between.

You have two options:

  • Route through the hub. Put Azure Firewall (or another network virtual appliance) in the hub, add a route table on each spoke sending traffic for the other spokes to the firewall's private IP, and allow the traffic in a firewall rule. This is the usual answer because it gives you one place to inspect and log traffic.
  • Peer the spokes directly. Fine for two or three spokes, but it gets messy fast: with 10 spokes, a full mesh is 45 peerings.

Whichever you choose, set allowForwardedTraffic: true on the peerings. Without it, traffic that came through the firewall is dropped.

3. Hard-coded subnet ranges drift

Most examples hard-code subnet prefixes:

addressPrefix: '10.0.1.0/26'
Enter fullscreen mode Exit fullscreen mode

That's fine until someone changes the hub's address space and forgets one subnet, which then sits outside the VNet range and fails to deploy.

Bicep's cidrSubnet() function calculates subnets from the parent range instead:

param hubAddressSpace string = '10.0.0.0/22'

var sharedSubnetPrefix   = cidrSubnet(hubAddressSpace, 24, 0) // 10.0.0.0/24
var firewallSubnetPrefix = cidrSubnet(hubAddressSpace, 26, 4) // 10.0.1.0/26
var bastionSubnetPrefix  = cidrSubnet(hubAddressSpace, 26, 5) // 10.0.1.64/26
Enter fullscreen mode Exit fullscreen mode

The arguments are the parent range, the new prefix length, and which subnet of that size you want (counting from zero). Change hubAddressSpace and every subnet follows automatically.

4. Forced tunnelling needs BGP propagation turned off

To send all spoke internet traffic through the hub firewall, you add a route table with a default route:

resource spokeRouteTable 'Microsoft.Network/routeTables@2023-05-01' = {
  name: 'rt-spokes'
  location: location
  properties: {
    disableBgpRoutePropagation: true
    routes: [
      {
        name: 'default-via-firewall'
        properties: {
          addressPrefix: '0.0.0.0/0'
          nextHopType: 'VirtualAppliance'
          nextHopIpAddress: firewall.properties.ipConfigurations[0].properties.privateIPAddress
        }
      }
    ]
  }
}
Enter fullscreen mode Exit fullscreen mode

Note disableBgpRoutePropagation: true. If you have a VPN or ExpressRoute gateway, it can advertise routes that are more specific than your 0.0.0.0/0, and those win. Turning off propagation on spoke route tables keeps traffic going through the firewall.

5. A new firewall blocks everything

This one surprises people the first time: a brand-new Azure Firewall policy has no allow rules. The moment you add the forced-tunnelling route above, every outbound call from the spokes fails, including package installs, Windows Update and calls to Azure services.

Before pointing spokes at the firewall, add at least:

  • an application rule collection for the domains your workloads need (FQDN tags such as WindowsUpdate and AzureKubernetesService help here),
  • network rules for spoke-to-spoke traffic, if you're routing it through the hub.

Also make Azure Firewall and Bastion optional parameters in your templates. Both are billed hourly even when idle, and test environments rarely need them.

Quick checklist

  • [ ] @batchSize(1) on peering loops
  • [ ] allowForwardedTraffic: true on hub ⇄ spoke peerings
  • [ ] A plan for spoke-to-spoke traffic (through the firewall, or direct peering)
  • [ ] Subnets calculated with cidrSubnet()
  • [ ] disableBgpRoutePropagation: true on spoke route tables when forcing traffic through a firewall
  • [ ] Allow rules in place before turning on forced tunnelling
  • [ ] Firewall and Bastion behind parameters so they're off by default

I've packaged a hub-and-spoke template with all of the above (any number of spokes, optional firewall and Bastion, calculated subnets) in the Azure Bicep Starter Pack, together with a passwordless Web App + SQL template, a locked-down storage template and a PDF guide. If you'd like to try one first, the storage + private endpoint template is free.

Top comments (0)