DEV Community

Cover image for Build multi-cloud & hybrid networks
aakhtar3
aakhtar3

Posted on

Build multi-cloud & hybrid networks

Goals

The purpose of this guide is to create four distributed networks and combine them into one highly available logically connected network.

  • Build with the top three cloud vendors (aws.amazon.com, azure.microsoft.com, cloud.google.com) and one On-Premise (pfsense.org) network
  • Scale Mesh network topology to allow additional point-to-point connections
  • Dynamic routing between Autonomous Systems (AS) using Border Gateway Protocol (BGP)
  • Encrypt network traversal over Virtual Private Network (VPN) tunnels using Internet Protocol Security (IPSec)

Network

4 Network

  • Final mesh network topology architecture

AWS

AWS

Network

VPC

  • Create a Virtual Private Cloud Network in AWS

VPC config-a

VPC config-b

Resource to Create VPC and more
Name vpc-aws
IPv4 CIDR block 172.16.11.0/24
Num of AZs 2
Public 0
Private 2
NAT None
Endpoint None

Gateway

Gateway

  • Identify the IP address of the ISP
  • Point to Point Identification and traffic passthrough

Customer Gateway

Customer Gateway

Name pfsense
BGP ASN 65000
IP address 4.4.4.4

Transit Gateway

Transit Gateway config-a

Transit Gateway config-b

Name tg-aws
Description tg-aws
ASN 64512

Route Table

Update Route Table

  • Update routes to TGW

DNS

DNS

  • AWS will dedicate a reserved IP address x.x.x.2 for a VPC resolver
  • Outbound Endpoints will allow you to forward DNS requests for resolvers on other networks
  • Inbound Endpoints will allow resolvers on other networks to forward requests to AWS

Outbound Endpoint

Outbound Endpoint config-a

Outbound Endpoint config-b

Outbound Endpoint config-c

Outbound Endpoint config-d

Endpoint Name oe-aws
VPC vpc-aws-vpc
Security Group Default
Endpoint Type IPv4
IP Address #1 AZ us-east-1, subnet 1, IPv4
IP Address #2 AZ us-east-2, subnet 2, IPv4
Rule Name onpremise
Rule Rule Type Forward
Domain Name firewall.lan
VPC Rule vpc-aws-vpc
Target IP #1 10.0.1.2:53
Target IP #2 10.0.4.2:53

Inbound Endpoint

Inbound Endpoint config-a

Inbound Endpoint config-b

Endpoint Name ie-aws
VPC vpc-aws-vpc
Security Group Default
Endpoint Type IPv4
IP Address #1 AZ us-east-1, subnet 1, IPv4
IP Address #2 AZ us-east-2, subnet 2, IPv4

Site to Site

Site to Site

  • Use IPsec tunnels to connect AWS to another datacenter
  • Have a failover connection for High availability

Site to Site config-a

Site to Site config-b

Site to Site config-c

Site to Site config-d

AWS s2s-aws-pfsense
Target gateway type Transit Gateway
Transit Gateway TGW
Customer Gateway CGW
Routing Options Dynamic
Tunnel inside IP IPv4
Inside IPv4 CIDR for tunnel 1 169.254.11.0/30
Pre-shared key for tunnel 1 strong password
Inside IPv4 CIDR for tunnel 2 169.254.12.0/30
Pre-shared key for tunnel 2 strong password

Status

Status

  • When BGP session is established, the status will go from down to up

SSM

SSM

  • Using AWS System Manager will allow remote access without opening any ssh ports
  • Use to keep your network private
  • Use to debug any connectivity issues

IAM Role Policy

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "ssm:DescribeAssociation",
                "ssm:GetDeployablePatchSnapshotForInstance",
                "ssm:GetDocument",
                "ssm:DescribeDocument",
                "ssm:GetManifest",
                "ssm:GetParameter",
                "ssm:GetParameters",
                "ssm:ListAssociations",
                "ssm:ListInstanceAssociations",
                "ssm:PutInventory",
                "ssm:PutComplianceItems",
                "ssm:PutConfigurePackageResult",
                "ssm:UpdateAssociationStatus",
                "ssm:UpdateInstanceAssociationStatus",
                "ssm:UpdateInstanceInformation"
            ],
            "Resource": "*",
            "Effect": "Allow"
        },
        {
            "Action": [
                "ssmmessages:CreateControlChannel",
                "ssmmessages:CreateDataChannel",
                "ssmmessages:OpenControlChannel",
                "ssmmessages:OpenDataChannel"
            ],
            "Resource": "*",
            "Effect": "Allow"
        },
        {
            "Action": [
                "ec2messages:AcknowledgeMessage",
                "ec2messages:DeleteMessage",
                "ec2messages:FailMessage",
                "ec2messages:GetEndpoint",
                "ec2messages:GetMessages",
                "ec2messages:SendReply"
            ],
            "Resource": "*",
            "Effect": "Allow"
        }
    ]
}
Enter fullscreen mode Exit fullscreen mode

Endpoint

SSM Endpoint

ssm Endpoint config-a

ssm Endpoint config-b

ssm Endpoint config-c

Name ssm-endpoint
Service Category AWS Service
Service SSM
VPC vpc-aws-vpc
Subnets us-east-1, us-east-2
Security Group Default
Policy Full Access
SSMMessage Endpoint

Image description

Name ssmmessage-endpoint
Service Category AWS Service
Service SSMmessages
VPC vpc-aws-vpc
Subnets us-east-1, us-east-2
Security Group Default
Policy Full Access
EC2Message Endpoint

Image description

Name ec2message-endpoint
Service Category AWS Service
Service ec2messages
VPC vpc-aws-vpc
Subnets us-east-1, us-east-2
Security Group Default
Policy Full Access

Azure

Azure

VNET

VNET

  • Create a Virtual Network on Azure

Resource Group

RG Review

  • Resource Group provides a single detailed view of all resources in a groups stack
Resource group rg-aws-azure
Region East US

Virtual Network

Vnet Review

  • This Iaas will build a virtual network similar to a VPC
  • Create 1 network /24 CIDR to create 4 subnets with /26 CIDR
Resource group rg-aws-azure
Name vnet-aws-azure
Region East US
Bastion Disabled
Firewall Disabled
DDoS Disabled
Adress Space 172.16.12.0/24
Subnet 172.16.12.0/26

VWAN

VWAN

  • TODO: Azure VWAN
  • Have a AWS site-to-site connection config to populate data

Local Network Gateway

LNG Review

  • IP of the customer/data center Gateway
Resource group rg-aws-azure
Region East US
Endpoint IP Address
IP Address 1.1.1.1
Address Space(s) None
ASN 64512
BGP 169.254.21.1
Reserved APIPA
AWS 169.254.0.0/16
Azure 169.254.21.0/24 - 169.254.22.0/24

Virtual Network Gateway

VNG Review

Resource group rg-aws-azure
Region East US
SKU VPNGw2AZ
Generation 2
VNET vnet-aws-azure
Subnet 172.16.12.64/27
Gateway Type VPN
VPN Type Route Based
Active-active Disabled
BGB Enabled
ASN 65000
Custom APIPA 169.254.21.2, 169.254.22.2
Public IP adress vng-aws-azure-pip

Private Resolver

Private Resolver

Connection

Connection

  • Use to create an IPsec connection using BGP
  • Create a second connection for failover

Conn Review

Resource group rg-aws-azure
Region East US
Connection type Site-to-site(IPsec)
Connection name conn-1-aws-azure
Virtual Network Gateway vng-aws-azure
Local Network Gateway lng-aws-azure
IKE Protocol IKEv2
IpSec / IKE policy Default
Use Policy based traffic selector Disable
DPD timeout 45
Connection Mode Default
BGP 169.254.21.2

BGP

VNG Sidebar

Connection Status

  • Verify Connection is enabled
  • Create second connection for failover

BGB Status

  • Verify Route propagation from BGP

Azure VM

Ping

  • Azure -> AWS

GCP

GCP

VPC

VPC

  • Create a Virtual Private Cloud on Google

VPC config-a

VPC config-b

VPC config-c

Name vpc-gcp-aws
Description VPC
IPv6 Disabled
Subnet Custom
Subnet Name Private
Subnet Region us-east-1
IP stack IPv4
IP range 172.16.13.0/24
Private Google Access off
Flow Logs off
IPv4 Firewall Rule Ingress Apply to all 0.0.0.0/0 ICMP Allow
Dynamic Routing Regional

Network Connectivity Center

Network Connectivity

Cloud Router

Cloud Router

Name cr-gcp
Description route
Network vpc-gcp-aws
Region us-east-1
ASN 65000
Interval 20
Routes Advertise all subnets to CR

VPN Gateway

VPN Gateway

Name vpn-gcp-aws
Network vpc-gcp-aws
Region us-east-1
IP stack IPv4

Cloud DNS

Cloud DNS

Peer VPN

Peer VPN

  • Set up the infrastructure for GCP VPN

Peer VPN config-a

Peer VPN config-b

Peer VPN config-c

  • Repeats these steps on interface 1 (failover)
Name vpng-gcp-aws
Interfaces two interfaces
Interface 0 3.3.3.3
Interface 1 3.3.3.2
Peer VPN Gateway On-Prem or Non Google
Peer VPNG Name vpng-gcp-aws
High Availability Create a pair of VPN tunnles
Cloud Router cr-gcp
Associated Peer VPNG interface 0: 1.1.1.1
Name conn1-gcp-aws
pre-shared key strong password
Peer ASN 64512

BGP

BGP

Name conn1
Peer ASN 64512
BGB IPv4 address Manually
Cloud Router BGP 169.254.250.138
BGP Peer Address 169.254.250.137

BGP Status

  • Verify Dynamic Route update

GCP vm

Image description

  • GCP -> AWS

PFSense

Pfsense

VLAN

Check out this write-up on how to configure VLANs with pfsense

ISP

TODO: Check out this write-up on how to configure a VPN Server with pfsense

PiHole

TODO: Check out this write-up on how to configure a DNS server with PiHole

IPSec

Phase 1

Edit Tunnel 1

  • Start by creating a primary tunnel and repeat the below steps for the failover connection tunnel 2

Tunnel1 Config-a

Tunnel1 Config-b

Description conn1-aws-pfsense
Key Exchange version IKEv2
Remote Gateway 1.1.1.1
Pre-Shared Key strong password key token
Algorithm AES
Key Length 128 bits
Hash SHA256
DH Group 14 (2048 bit)
Max failures 3

Phase 2

Edit Tunnel 2

  • Start by creating a primary tunnel and repeat the below steps for the failover connection tunnel 2

Tunnel1 Config-a

Tunnel1 Config-b

Tunnel1 Config-b

Description conn1-aws-pfsense
Mode Routed (VTI)
Local Network address: 169.254.11.12
Remote Network address: 169.254.11.11
Encryption Algorithm AES256-CGM 128bits
Ping Host 172.16.11.11
Keep Alive Enabled

Status

Image description

  • Both primary and failover tunnels connected with IPSec

BGP

FRR Global Settings

Global Settings

Enabled true
Master Password strong password
FRR Route Maps

Route Maps

Route Maps config

Name Allow-all
Action Permit
Sequence 100
FRR BGB

FRR BGB

BGB-config-a

BGB-config-a

Enabled true
Local AS 65000
Router ID 10.0.1.1
Networks to distrbute 10.0.1.0/28, 10.0.2.0/29, 10.0.4.0/28
FRR Neighbors

FRR Neighbors

  • Start with the primary tunnel and repeat the steps for the failover tunnel

FRR Neighbors config-a

FRR Neighbors config-b

FRR Neighbors config-c

Name/Address 169.254.11.12
Description conn1-aws-pfsense
Remote AS 64512
Inbound Route Map Filters Allow-all
Outbound Route Map Filters Allow-all
FRR Status

Status Routes

  • Verify Dynamic Routes have been updated

Status Summary

  • View the BGP Summary

Mac

Mac

  • Mac -> AWS

Do your career a big favor. Join DEV. (The website you're on right now)

It takes one minute, it's free, and is worth it for your career.

Get started

Community matters

Top comments (0)

AWS Security LIVE!

Tune in for AWS Security LIVE!

Join AWS Security LIVE! for expert insights and actionable tips to protect your organization and keep security teams prepared.

Learn More

👋 Kindness is contagious

Dive into an ocean of knowledge with this thought-provoking post, revered deeply within the supportive DEV Community. Developers of all levels are welcome to join and enhance our collective intelligence.

Saying a simple "thank you" can brighten someone's day. Share your gratitude in the comments below!

On DEV, sharing ideas eases our path and fortifies our community connections. Found this helpful? Sending a quick thanks to the author can be profoundly valued.

Okay