DEV Community

Cover image for Designing High-Availability Cloud Infrastructure & Multi-AZ Failover
Amir Mehmood
Amir Mehmood

Posted on Originally published at hostingpodium.com

Designing High-Availability Cloud Infrastructure & Multi-AZ Failover

#linux #database #sre #devops

Originally published on the HostingPodium Enterprise Engineering Blog.

Free SRE & Developer Diagnostics: Test your stack live with our Global DNS Propagation Checker & SSL Health Auditor, Free Website Speed & Core Web Vitals Checker, or explore all 9 Free Webmaster & Cloud Engineering Tools.

This guide defines the systems architecture, kernel parameters, and storage topologies required to operate enterprise-grade managed cloud infrastructure. It provides production-tested configurations, eBPF networking designs, and concrete disaster recovery runbooks for platform engineers.

Architectural Foundations: Hardware to Kernel Abstraction

Operating enterprise managed cloud infrastructure requires strict isolation, predictable I/O scheduling, and deterministic memory distribution across dense multi-tenant topologies. At the foundation, modern compute clusters decouple compute resources from state via Type-1 hypervisors (KVM) or bare-metal container runtimes orchestrated over high-density physical nodes. Achieving deterministic throughput starts at the NUMA (Non-Uniform Memory Access) topology boundary. When designing hypervisor nodes populated with dual-socket AMD EPYC or Intel Xeon processors, crossing NUMA nodes to fulfill single-guest memory accesses introduces memory latency penalties between 30% and 40%.

Hypervisor instances must strictly pin vCPUs to physical cores residing within the identical NUMA node where guest RAM is provisioned. Using libvirt domain XML specifications, memory allocation must leverage pre-allocated Static HugePages (1GB pages) rather than Transparent Huge Pages (THP) to eradicate page-table walks and runtime compaction stalls:
Enter fullscreen mode Exit fullscreen mode

By isolating host OS interrupts to dedicated physical cores (e.g, cores 0 and 1) through the Linux boot parameter isolcpus=2-63,66-127 nohz_full=2-63,66-127 rcu_nocbs=2-63,66-127, compute engines running tenant workloads avoid kernel context switches and scheduler latency spikes. The hypervisor achieves raw hardware execution performance while preserving multi-tenant safety guarantees.

Linux Kernel & Network Stack Optimization

Default Linux kernel profiles favor balanced desktop or standard server workloads, creating severe bottlenecks when subjected to hundreds of thousands of concurrent TCP sessions and line-rate network transfers. Managed cloud infrastructure nodes require explicit tuning across memory subsystem buffers, socket queues, and connection tracking tables.

Deploy the following kernel tuning parameters via /etc/sysctl.d/99-cloud-infrastructure.conf to prevent TCP packet drops, socket starvation, and unmanaged memory paging:

# Socket Queue and TCP Memory Allocations
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535
net.core.netdev_max_backlog = 32768
net.core.rmem_max = 33554432
net.core.wmem_max = 33554432
net.ipv4.tcp_rmem = 4096 87380 33554432
net.ipv4.tcp_wmem = 4096 65536 33554432

# Connection Management and Ephemeral Port Expansion
net.ipv4.ip_local_port_range = 10240 65535
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_slow_start_after_idle = 0
net.ipv4.tcp_congestion_control = bbr

# Conntrack Table Sizing for High Density
net.netfilter.nf_conntrack_max = 2097152
net.netfilter.nf_conntrack_tcp_timeout_established = 600

# Virtual Memory and Dirty Page Flushing
vm.swappiness = 10
vm.dirty_ratio = 15
vm.dirty_background_ratio = 5
vm.vfs_cache_pressure = 50
Enter fullscreen mode Exit fullscreen mode

In addition to kernel memory adjustments, physical Network Interface Cards (NICs) operating at 25GbE, 40GbE, or 100GbE require manual ring buffer resizing and IRQ balance pinning. Hardware packet drops at the driver layer can be mitigated by maxing out the descriptor rings using ethtool:

# Check and set ring buffer limits
ethtool -g eth0
ethtool -G eth0 rx 4096 tx 4096

# Enable adaptive interrupt moderation
ethtool -C eth0 adaptive-rx on adaptive-tx on
Enter fullscreen mode Exit fullscreen mode

Switching TCP congestion control from Reno or Cubic to BBR (Bottleneck Bandwidth and RTT) eliminates packet-loss penalties over long-haul multi-region cloud backbones, sustaining line-rate bandwidth utilization across variable latency environments.

Distributed Storage Topology: NVMe-oF and Ceph BlueStore

Reliable managed cloud infrastructure mandates decoupling compute from raw disk state without compromising I/O performance. Traditional SAN and NAS systems create monolithic failure points and vertical scaling ceilings. Production-grade platforms implement a horizontally scalable, software-defined storage fabric based on Ceph or NVMe-over-Fabrics (NVMe-oF) using RDMA over Converged Ethernet (RoCEv2).

For high-density, multi-tenant persistence, a Ceph cluster running the BlueStore engine writes object payloads directly to raw block devices without an underlying POSIX filesystem layer. This eliminates filesystem journaling overhead and double-write penalties. The underlying architecture utilizes an optimized CRUSH map that distributes placement groups (PGs) across distinct failure domains: host, rack, and power distribution unit (PDU).

Ceph Production Configuration Snippet 

[global]
fsid = 4a8b7923-d922-4bb3-96b4-5cc02c81da59
mon_initial_members = mon-01, mon-02, mon-03
mon_host = 10.240.0.11, 10.240.0.12, 10.240.0.13
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx

[osd]
osd_memory_target = 8589934592
osd_op_threads = 4
osd_disk_threads = 2
bluestore_cache_autotune = true
bluestore_min_alloc_size = 4096
bluestore_prefer_deferred_size = 0
ms_bind_msgr2 = true
ms_bind_msgr1 = false
Enter fullscreen mode Exit fullscreen mode

Setting bluestore_min_alloc_size = 4096 on all-flash NVMe clusters prevents space amplification for small 4KB random write workloads while reducing fragmentation. In environments requiring raw sub-100 microsecond latency, NVMe-oF bypasses TCP processing overhead by mapping remote NVMe submission and completion queues directly into host memory via RDMA, achieving wire-speed block access comparable to bare-metal local drives.

Network Fabric: EVPN-VXLAN and eBPF Dataplanes

Traditional VLAN segmentation fails at scale due to the 4,094 network limit and Spanning Tree Protocol (STP) link blocking. Managed cloud architectures rely on a non-blocking Clos Leaf-Spine network running BGP EVPN (Ethernet Virtual Private Network) with VXLAN (Virtual Extensible LAN) encapsulation. This constructs an arbitrary Layer 2 overlay across an unconstrained Layer 3 IP underlay.

Within the hypervisor or container host, replace legacy iptables routing with an eBPF (Extended Berkeley Packet Filter) dataplane, implemented via Cilium or native Linux kernel drivers. While iptables scales at $O(N)$ complexity—degrading throughput as thousands of firewall and NAT rules accumulate—eBPF leverages direct-lookup BPF maps executed inside the kernel socket buffer layer, ensuring deterministic $O(1)$ routing performance.

Cilium eBPF Host Routing Deployment 

# Deploy Cilium with kube-proxy replacement and native routing mode
helm install cilium cilium/cilium --version 1.15.0 \
 --namespace kube-system \
 --set kubeProxyReplacement=true \
 --set routingMode=native \
 --set autoDirectNodeRoutes=true \
 --set bpf.masquerade=true \
 --set ipam.mode=kubernetes \
 --set bpf.tproxy=true \
 --set bpf.clockSource=ktime \
 --set loadBalancer.mode=dsr
Enter fullscreen mode Exit fullscreen mode

Deploying Direct Server Return (DSR) allows reply packets from backend instances to bypass the load balancer node and stream directly back to clients, halving transit hop counts and eliminating balancer egress interface saturation.

Architecture Trade-Off Analysis

Selecting the operational model for cloud infrastructure dictates long-term cost structures, security liability, and performance limits. The following matrix details trade-offs across deployment methodologies:

Metric / Capability Hyperscaler Multi-Tenant (Public Cloud) Bare-Metal DIY Infrastructure Managed Cloud Infrastructure
Storage Performance (p99 Latency) Variable (1-5ms due to multi-tenant noisy neighbors) Deterministic (

Free Webmaster, SRE & Cloud Diagnostic Tools (No Signup Required)

Verify and benchmark your production infrastructure directly from your browser using the HostingPodium Free Webmaster & Developer Suite:

Free Engineering Utility Primary Diagnostic Use Case
Global DNS Propagation & SSL Checker Verify live A, AAAA, CNAME, MX, TXT, NS & CAA records across 21+ global resolvers + TLS certificate audits
Website Speed Test & Core Web Vitals Auditor Measure server TTFB, LCP, INP, CLS, and HTTP/2 compression latency in real time
Rich Snippets & Schema.org JSON-LD Generator Generate & validate structured JSON-LD markup for TechArticles, FAQs, Products & Organizations
HTTP Redirect Chain & Status Code Tracer Trace 301/302 redirect hops, inspect response headers, and verify load balancer health checks
DMARC, SPF & DKIM Email Security Validator Audit domain email authentication policies and generate compliant DMARC records
Google SERP Simulator & Meta Tag Previewer Preview desktop/mobile Google snippets, OpenGraph cards, and pixel-width truncation
Free WebP Image Converter & Optimizer Client-side PNG/JPG to WebP compression for faster Core Web Vitals LCP

Related Enterprise Architecture & Benchmarks

Top comments (0)