DEV Community: Korakrit Chariyasathian

On‑Premises Kubernetes Networking Architecture: A Comprehensive Architectural Synthesis

Korakrit Chariyasathian — Mon, 05 Jan 2026 08:17:01 +0000

1. Context, Scope, and Design Objectives

This document presents a rigorous architectural synthesis of an on‑premises Kubernetes networking design implemented atop a Hyper‑V virtualization substrate. The objective is to construct a networking architecture that compensates for the absence of a cloud provider while remaining conceptually and operationally aligned with cloud‑native paradigms.

The design explicitly targets the following goals:

Operation in a fully on‑premises environment without reliance on a managed cloud provider
Functional support for Service objects of type LoadBalancer
Clear and enforceable separation of concerns among compute, networking, and ingress responsibilities
Immediate operational viability coupled with architectural compatibility for future autoscaling at both the pod and node levels
Conceptual portability, such that the underlying mental model remains valid upon eventual migration to a public cloud environment

The core infrastructural components considered herein are:

FRR Virtual Machine: An Ubuntu‑based virtual appliance running FRRouting, responsible for fabric‑level routing and control‑plane signaling
Kubernetes Worker Nodes: Linux virtual machines hosting workloads and participating in routing advertisement
MetalLB: Deployed in BGP mode to provide load‑balancer semantics in the absence of a cloud‑native implementation

2. Rationale for FRR and MetalLB in On‑Premises Kubernetes

2.1 Absence of a Cloud Provider and Its Consequences

In contrast to managed Kubernetes offerings, an on‑premises Kubernetes deployment lacks the following capabilities by default:

A managed Layer‑4/Layer‑7 load balancer
A managed routing plane capable of advertising service reachability
Automatic integration between Kubernetes service abstractions and the surrounding network fabric

Consequently, the Service abstraction of type LoadBalancer is inert unless explicitly backed by external infrastructure. The burden of implementing these capabilities therefore shifts to the platform architect.

2.2 MetalLB as a Functional Analog to Cloud Load Balancers

MetalLB is introduced to fill this infrastructural void by providing two essential capabilities:

Allocation of externally reachable virtual IP addresses for Kubernetes services
Advertisement of reachability for those IPs to the upstream network

MetalLB supports two operational modes:

Layer‑2 (L2) Mode, based on ARP/NDP announcements
Border Gateway Protocol (BGP) Mode, based on explicit route advertisement

This architecture deliberately adopts BGP mode, for reasons grounded in scalability, determinism, and alignment with Kubernetes’ distributed systems model.

3. Justification for BGP Mode over L2 Mode (Engineering Perspective)

3.1 Architectural Implications of L2 Mode

In L2 mode, MetalLB assigns ownership of a LoadBalancer IP to a single node. That node responds to ARP requests on behalf of the service, and failover is achieved indirectly via ARP cache invalidation.

From an architectural standpoint, this entails:

Tight coupling between a network identity (the service IP) and a specific node
Implicit, non‑contractual ownership semantics
Dependence on shared mutable state in the form of ARP caches

Such properties undermine predictability, complicate failure analysis, and conflict with Kubernetes’ expectation that nodes are ephemeral and interchangeable.

3.2 Architectural Advantages of BGP Mode

Under BGP mode, service reachability is expressed through explicit route advertisements. Nodes hosting service endpoints announce routes, and the routing fabric (FRR) selects viable forwarding paths. When a node becomes unavailable, its routes are withdrawn in a deterministic and protocol‑defined manner.

This yields several architectural advantages:

Explicit contracts governing reachability and ownership
Well‑defined state machines for convergence and failure handling
Decoupling of node lifecycle events from ingress semantics

The conceptual correspondence between Kubernetes primitives and BGP behavior is direct:

Kubernetes Event	BGP Effect
Pod scheduled	Route advertised
Pod terminated	Route withdrawn
Node added	BGP peer established
Node removed	BGP session torn down

This symmetry renders BGP mode a natural fit for Kubernetes’ distributed control model.

4. Network Roles and Separation of Responsibilities

4.1 FRR Virtual Machine as a Fabric Router

The FRR virtual machine functions as a stable fabric‑level routing component. Its responsibilities are intentionally constrained and explicitly defined.

It does not act as:

An Internet gateway
A NAT device
A default gateway for Kubernetes nodes

Instead, FRR serves exclusively as a fabric router, providing deterministic Layer‑3 reachability for Kubernetes Service and Pod traffic via BGP.

Network Interface Partitioning

The FRR VM is provisioned with two network interfaces, each mapped to a distinct routing domain:

eth0 (External – 192.168.1.0/24)
- Management access
- Outbound Internet connectivity (package installation, updates)
- Connectivity to the broader on‑premises LAN
eth1 (Internal – 10.10.0.0/24)
- Kubernetes fabric
- BGP peering with Kubernetes nodes
- Transport for Pod and LoadBalancer traffic

Critically, only internal fabric routes are advertised into BGP, preserving strict separation between management traffic and cluster data paths.

Netplan Configuration (FRR VM)

The following netplan configuration illustrates the canonical interface setup on the FRR VM:

# /etc/netplan/50-cloud-init.yaml
network:
  version: 2
  ethernets:
    eth0:
      dhcp4: false
      addresses:
        - 192.168.1.13/24
      routes:
        - to: default
          via: 192.168.1.1
      nameservers:
        addresses:
          - 1.1.1.1
          - 8.8.8.8

    eth1:
      dhcp4: false
      addresses:
        - 10.10.0.10/24

This configuration enforces a single default route via the external interface, ensuring that internal fabric traffic is never misrouted toward the management plane.

FRR (FRRouting) Configuration Overview

Within FRR, the routing policy is deliberately minimalistic and explicit:

router bgp 65001
 bgp router-id 10.10.0.10
 no bgp default ipv4-unicast

 address-family ipv4 unicast
  redistribute connected route-map INTERNAL_ONLY
 exit-address-family

Supporting policy objects restrict route advertisement to the Kubernetes fabric only:

ip prefix-list INTERNAL_NET seq 10 permit 10.10.0.0/24

route-map INTERNAL_ONLY permit 10
 match ip address prefix-list INTERNAL_NET

This ensures that:

External subnets (e.g., 192.168.1.0/24) are never advertised to Kubernetes nodes
FRR cannot be accidentally interpreted as a default or Internet gateway

4.2 Kubernetes Nodes as Disposable Compute Elements

Kubernetes nodes are treated strictly as ephemeral compute resources. They host workloads and participate in routing advertisements via MetalLB, but they do not retain long‑term ownership of ingress IPs.

Each node:

Establishes an explicit BGP peering relationship with FRR
Advertises LoadBalancer IPs only while hosting active service endpoints

This design reinforces the principle that nodes remain stateless with respect to ingress, enabling both horizontal pod autoscaling and future node‑level autoscaling without architectural refactoring.

5. FRR Configuration: Conceptual Overview

5.1 Governing Principles

The FRR configuration adheres to three guiding principles:

The routing fabric must be stable and long‑lived
Compute nodes must remain disposable
External and internal routing domains must be strictly segregated

5.2 Functional Responsibilities of FRR

Operate a BGP process under a dedicated autonomous system (AS 65001)
Accept peering relationships from Kubernetes nodes (AS 65002)
Advertise only internal fabric connectivity

5.3 On Explicit Peer Configuration

The requirement to explicitly configure BGP neighbors when adding new nodes is intrinsic to BGP’s design. This characteristic reflects protocol intentionality rather than architectural deficiency.

Crucially, this constraint does not impede future autoscaling; it merely indicates that automation of peer management has not yet been introduced.

6. MetalLB Configuration: Conceptual Overview

In BGP mode, MetalLB:

Allocates LoadBalancer IPs from a predefined pool
Advertises those IPs to FRR via BGP
Withdraws advertisements automatically upon failure or topology changes

MetalLB deliberately refrains from managing FRR configuration or dynamically creating BGP neighbors. This boundary enforces a clean separation between Kubernetes‑level intent and infrastructure‑level policy.

7. Autoscaling Readiness and Evolutionary Path

7.1 Pod‑Level Autoscaling

Horizontal Pod Autoscaling operates entirely within the Kubernetes control plane and is unaffected by the external routing architecture. The design fully supports HPA without modification.

7.2 Node‑Level Scaling: Present and Future

At present:

Nodes are provisioned manually
BGP peers are added manually
System behavior remains stable and predictable

In a future automated environment:

Node provisioning becomes orchestrated
BGP peering is automated or abstracted

Importantly, the network architecture itself remains unchanged; only the automation layer evolves.

8. Architectural Continuity Across Cloud Migration

On‑Premises Component	Cloud Analog
FRR VM	Managed cloud router / VPC router
MetalLB	Managed cloud LoadBalancer
Internal fabric	VPC subnet
BGP semantics	Provider‑managed routing

The architectural mental model therefore persists intact across deployment environments.

9. Architecture Diagram (Textual Representation)

        External LAN (192.168.1.0/24)
                 |
              [ eth0 ]
            +----------------+
            |     FRR VM     |
            |    AS 65001    |
            +----------------+
              [ eth1 ]
                 |
        Kubernetes Fabric (10.10.0.0/24)
        ---------------------------------
        |               |               |
    [ Node‑1 ]       [ Node‑2 ]     [ Node‑N ]
     AS 65002         AS 65002        AS 65002
        |               |               |
      Pods            Pods            Pods

10. Sequence Diagram: LoadBalancer Traffic Flow

@startuml
top to bottom direction

component Client
component "FRR\nBGP Speaker" as FRR
component "Kubernetes\nNode" as Node
component "Pod\nApplication" as Pod

Client --> FRR : TCP SYN to LoadBalancer IP
FRR --> Node : BGP-selected\nnext hop
Node --> Pod : Service routing\n(kube-proxy / dataplane)
Pod --> Node : Response payload
Node --> FRR : Return traffic
FRR --> Client : Forward response
@enduml

11. Concluding Observations

BGP mode is selected on the basis of architectural rigor rather than convenience
FRR serves as a stable routing substrate, not an application gateway
Nodes remain ephemeral and autoscaling‑compatible
LoadBalancer semantics closely mirror those of managed cloud environments
The design is simultaneously operationally sound today and structurally extensible for the future

This architecture prioritizes clarity, determinism, and long‑term evolutionary capacity—attributes essential to robust Kubernetes infrastructure at scale.

PlantUML Diagram

@startuml
skinparam backgroundColor #FFFFFF
skinparam shadowing false
skinparam componentStyle rectangle
skinparam defaultFontName Monospace

title Kubernetes on-prem with MetalLB (BGP) and FRR Fabric Router

'========================
' External Network
'========================
package "External LAN\n192.168.1.0/24\n(Management / Internet)" {

  node "Client\n192.168.1.2" as client

  node "is-kube-01\nControl Plane\neth0: 192.168.1.11" as kube01_ext
  node "is-kube-02\nWorker Node\neth0: 192.168.1.12" as kube02_ext

  node "FRR-VM\neth0: 192.168.1.13\n(No BGP here)" as frr_ext
}

'========================
' Internal Fabric
'========================
package "Kubernetes Internal Fabric\n10.10.0.0/24" {

  node "FRR-VM\nFabric Router\neth1: 10.10.0.1\nAS 65001" as frr_int

  node "is-kube-01\neth1: 10.10.0.11\nkubelet --node-ip\nAS 65002" as kube01_int
  node "is-kube-02\neth1: 10.10.0.12\nkubelet --node-ip\nAS 65002" as kube02_int

  component "MetalLB Speaker\n(on each node)" as metallb
}

'========================
' Kubernetes Objects
'========================
package "Kubernetes Cluster" {

  component "kube-apiserver\n(6443)" as apiserver
  component "kube-proxy\niptables / IPVS" as kubeproxy
  component "Pods\n(Containers)" as pods
}

'========================
' Management / Admin Path
'========================
client --> kube01_ext : SSH / HTTPS / kubectl
client --> kube02_ext : (optional admin)

kube01_ext --> apiserver : control-plane
apiserver --> kube01_ext
apiserver --> kube02_ext

'========================
' Routing Control Plane (BGP)
'========================
kube01_int --> frr_int : BGP (TCP 179)\nAdvertise LB IPs
kube02_int --> frr_int : BGP (TCP 179)\nAdvertise LB IPs

metallb --> kube01_int : Speaker binds\nInternalIP
metallb --> kube02_int

note right of frr_int
FRR role:
- BGP fabric router
- Learns LoadBalancer IPs
- No NAT
- No data forwarding
end note

'========================
' Data Plane (Service Traffic)
'========================
frr_int ..> kube01_int : Routing info only\n(NO packets)
frr_int ..> kube02_int : Routing info only\n(NO packets)

kube01_int --> kubeproxy
kube02_int --> kubeproxy
kubeproxy --> pods

note bottom of kubeproxy
Actual data path:
Client -> Node holding LB IP
kube-proxy forwards to Pod
FRR NOT in packet path
end note

'========================
' Separation of Concerns
'========================
note bottom
eth0 (192.168.1.x):
- Internet
- Admin
- OS routing / apt

eth1 (10.10.x.x):
- Kubernetes fabric
- BGP
- MetalLB
end note

@enduml

Author's Note

This document was prepared as a general guide and reference note only. The content was drafted with the assistance of an AI system and subsequently reviewed, refined, and curated by the author.

Hyper-V Configuration Guide for Kubernetes and Supporting Network Services

Korakrit Chariyasathian — Sun, 04 Jan 2026 13:22:13 +0000

This document describes a baseline, production-ready approach for configuring Hyper-V Virtual Machines used with Kubernetes (Control Plane / Worker Nodes) and an FRR (Free Range Routing) VM. The focus is on correct network adapter design, virtual switch usage, and security-related settings to ensure long-term stability and predictable behavior.

Architecture Overview

The Hyper-V host provides at least two virtual switches:
- vEthernet-External: Used for Internet access
- K8S-Internal: Used for intra-cluster (node-to-node) communication
Every Kubernetes node (Control Plane and Worker):
- Has two NICs
- External NIC → Internet access
- Internal NIC → Cluster communication
The FRR VM acts strictly as an L3 router

Baseline: Create a New Virtual Machine (Applies to All VMs)

1. Create New Virtual Machine

Open Hyper-V Manager → New → Virtual Machine
Specify Name:
- Control Plane: is-kube-control
- Worker: is-kube-worker-XX
- FRR: is-frrouting-vm
Specify Generation:
- Generation 2 (Required)
  - UEFI-based firmware
  - Secure Boot support
  - Required for modern Linux kernels (Kubernetes / FRR)

2. Assign Memory

Disable Dynamic Memory (recommended)
Startup Memory:
- Control Plane / Worker: ≥ 4 GB
- FRR VM: ≥ 2 GB

Rationale:

Kubernetes does not behave well with Dynamic Memory
Avoids latency, eviction, and scheduling edge cases

3. Configure Networking (Initial Wizard Step)

Assign the first NIC to:
- vEthernet-External
The second NIC will be added after VM creation

4. Connect Virtual Hard Disk

Create a new virtual hard disk
Format: VHDX (Dynamic)
Recommended size:
- Control Plane / Worker: ≥ 40 GB
- FRR VM: ≥ 20 GB

5. Installation Options

Select Install an operating system later
Mount the ISO after VM creation

Operating System Baseline

OS: Ubuntu 24.04 LTS (Server)
Installation Mode: Minimal Install
Initial State:
- No additional packages installed
- Docker / containerd / Kubernetes components not installed yet
Enabled services:
- SSH only (for initial access and automation)

This approach minimizes the attack surface and ensures a clean baseline before installing Kubernetes or FRR components.

Netplan Baseline Configuration

Static network configuration is used to ensure deterministic routing and traffic flow.

Example: /etc/netplan/50-cloud-init.yaml

network:
  version: 2
  ethernets:
    eth0:
      dhcp4: false
      addresses:
        - 192.168.1.10/24
      routes:
        - to: default
          via: 192.168.1.1
      nameservers:
        addresses:
          - 1.1.1.1
          - 8.8.8.8

    eth1:
      dhcp4: false
      addresses:
        - 10.10.0.10/24

eth0:
- External NIC (vEthernet-External)
- Default route and Internet access
eth1:
- Internal NIC (K8S-Internal)
- Node-to-node / Pod network traffic

Note: No default route is configured on the internal NIC to avoid asymmetric routing issues.

Post-Creation VM Configuration (Before First Boot)

1. Firmware / Secure Boot

VM Settings → Firmware
Secure Boot:
- Enabled
- Template: Microsoft UEFI Certificate Authority

Compatible with Ubuntu, Debian, RHEL, Rocky Linux, and FRR-based systems.

2. Processor / Hardware Acceleration

Virtual processors: ≥ 2 vCPU
Nested Virtualization:
- Disable unless explicitly required (e.g., KinD, KVM-in-VM)

Default Hyper-V processor settings are sufficient. No additional hardware acceleration tuning is required.

3. Memory Settings

Dynamic Memory: Disabled
Leave advanced memory mapping options at default values

4. Network Adapters – Advanced Features

External NIC (`vEthernet-External`)

Enable:
- MAC Address Spoofing (Kubernetes nodes only)
- VRSS
- RSC
Disable:
- DHCP Guard
- Router Guard

Internal NIC (`K8S-Internal`)

Add a second Network Adapter
Switch: K8S-Internal
MAC Address Spoofing:
- Kubernetes Nodes: Recommended: On
- FRR VM: Off

5. VLAN Configuration

VLAN Mode: Untagged (Default)

6. Integration Services

Keep the default integration services enabled:

Heartbeat
Key-Value Pair Exchange
Shutdown
Time Synchronization

Optional:

Guest Service Interface (only if file copy from host is required)

No integration services need to be disabled for Kubernetes or FRR.

Part 1: Control Plane Node Configuration

Network Adapter Layout

NIC	Hyper-V Switch	Purpose
NIC-1	vEthernet-External	Internet, image pulls, external APIs
NIC-2	K8S-Internal	Pod-to-Pod and Node-to-Node traffic

MAC Address Spoofing

External NIC: On (Required)
Internal NIC:
- Overlay CNI (Flannel VXLAN, Calico VXLAN): Optional
- Non-overlay / L2 CNI: On

Best practice: Enable MAC Address Spoofing on both NICs.

Resource Recommendation

CPU: ≥ 2 vCPU
Memory: ≥ 4 GB
Disk: ≥ 40 GB

Part 2: Worker Node Configuration

Network Adapter Layout

NIC	Hyper-V Switch	Purpose
NIC-1	vEthernet-External	Internet and image pulls
NIC-2	K8S-Internal	East–West cluster traffic

MAC Address Spoofing

External NIC: On (Mandatory)
Internal NIC: Depends on CNI

Recommended: Enable on both NICs to avoid future edge cases.

Resource Recommendation

CPU: ≥ 2 vCPU (scale with workload)
Memory: 4–8 GB
Disk: ≥ 40 GB

Part 3: FRR VM Configuration

Network Adapter Layout

NIC	Hyper-V Switch	Purpose
NIC-1	vEthernet-External	Upstream / Internet routing
NIC-2	K8S-Internal	Internal routing / peering

MAC Address Spoofing

All NICs: Off

Rationale:

FRR operates strictly at Layer 3
Always uses its own interface MAC addresses
No container, bridge, or L2 forwarding traffic

Resource Recommendation

CPU: ≥ 2 vCPU
Memory: ≥ 2 GB
Disk: ≥ 20 GB

Summary of Best Practices

Kubernetes Nodes:
- Enable MAC Address Spoofing at least on the External NIC
- Prefer enabling it on both NICs
FRR VM:
- MAC Address Spoofing is not required
Clearly separate External and Internal NIC roles
Overlay CNIs reduce Layer-2 dependencies but do not eliminate the need for spoofing on egress NICs

Additional Notes

Re-evaluate MAC Address Spoofing if the CNI or network design changes
Enabling MAC Address Spoofing has no meaningful performance impact on Hyper-V

Author's Note

This guide was written primarily as a set of personal design and implementation notes. Its purpose is to document not only what was configured, but why specific decisions were made, in order to preserve architectural intent and reduce ambiguity during future maintenance, scaling, or redesign efforts.

The content of this document was drafted with the assistance of an AI system and subsequently reviewed and curated by the author.

Installing Kubernetes and Worker Node Setup on Ubuntu 24.04

Korakrit Chariyasathian — Thu, 25 Dec 2025 07:32:08 +0000

Kubernetes Setup on Ubuntu 24.04 (Kubernetes v1.33)

1. Prepare the Ubuntu VM

Update system packages:

sudo apt update && sudo apt upgrade -y

Disable swap (required by Kubernetes):

sudo swapoff -a
sudo sed -i '/ swap / s/^/#/' /etc/fstab
sudo sed -i '/swap.img/ s/^/#/' /etc/fstab

2. Enable Kernel Modules and Sysctl Settings

Load required kernel modules:

sudo modprobe overlay
sudo modprobe br_netfilter

Ensure they load on boot:

cat <<EOF | sudo tee /etc/modules-load.d/kubernetes.conf
overlay
br_netfilter
EOF

Set required sysctl parameters:

sudo tee /etc/sysctl.d/kubernetes.conf <<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF

Apply sysctl changes:

sudo sysctl --system

3. Install and Configure Containerd (Container Runtime)

Kubernetes v1.33 requires CRI v1. Docker (dockershim) is not supported.

Install containerd:

sudo apt install -y containerd

Generate the default containerd configuration:

sudo mkdir -p /etc/containerd
sudo containerd config default | sudo tee /etc/containerd/config.toml > /dev/null

Enable systemd cgroup:

sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml

Restart and enable containerd:

sudo systemctl daemon-reload
sudo systemctl enable --now containerd

Note: The CRI plugin is enabled by default in a standard containerd package config. You generally do not need to manually add a CRI block. If you choose to verify, ensure the CRI plugin section is present and not disabled. The pause image you referenced is reasonable.
Ensure the CRI plugin is enabled:

[plugins."io.containerd.grpc.v1.cri"]
  sandbox_image = "registry.k8s.io/pause:3.10"

Restart and enable containerd:

sudo systemctl restart containerd
sudo systemctl enable containerd

4. Add the Kubernetes APT Repository (v1.33)

sudo apt install -y apt-transport-https ca-certificates curl gpg
sudo mkdir -p /etc/apt/keyrings

curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.33/deb/Release.key | \
sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] \
https://pkgs.k8s.io/core:/stable:/v1.33/deb/ /" | \
sudo tee /etc/apt/sources.list.d/kubernetes.list

sudo apt update

5. Install Kubernetes Tools

sudo apt install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

(Optional but common) Ensure kubelet is enabled:

sudo systemctl enable --now kubelet

6. Install Helm

curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

Verify Helm and add repositories:

helm version
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

7. Initialize the Kubernetes Control Plane

sudo kubeadm init --pod-network-cidr=10.0.0.0/16

8. Set Up kubeconfig

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

9. Verify Node Status

kubectl get nodes

The node will show NotReady until a CNI plugin is installed.
Output: is-kube shows as NotReady (because Pod Network is not yet installed)

Install Cilium CNI

Install cilium-cli

Download the Cilium CLI:

curl -L --remote-name https://github.com/cilium/cilium-cli/releases/latest/download/cilium-linux-amd64.tar.gz
sudo tar xzvf cilium-linux-amd64.tar.gz -C /usr/local/bin
rm cilium-linux-amd64.tar.gz

Install Cilium

Install Cilium into the cluster:

cilium install

Verify Cluster Status

kubectl get nodes
cilium status

Worker Node Setup

1. Prepare the Ubuntu VM

Update system packages:

sudo apt update && sudo apt upgrade -y

sudo swapoff -a
sudo sed -i '/ swap / s/^/#/' /etc/fstab

2. Setup kernel modules and sysctl

sudo modprobe overlay
sudo modprobe br_netfilter

cat <<EOF | sudo tee /etc/modules-load.d/kubernetes.conf
overlay
br_netfilter
EOF

cat <<EOF | sudo tee /etc/sysctl.d/kubernetes.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables  = 1
net.ipv4.ip_forward                 = 1
EOF

sudo sysctl --system

3. Install and configure containerd

sudo apt install -y containerd

sudo mkdir -p /etc/containerd
sudo containerd config default | sudo tee /etc/containerd/config.toml > /dev/null

sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml

sudo systemctl daemon-reload
sudo systemctl enable --now containerd

4. Add Kubernetes apt repo (v1.33) and install kubelet/kubeadm

sudo apt install -y apt-transport-https ca-certificates curl gpg
sudo mkdir -p /etc/apt/keyrings

curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.33/deb/Release.key | \
sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] \
https://pkgs.k8s.io/core:/stable:/v1.33/deb/ /" | \
sudo tee /etc/apt/sources.list.d/kubernetes.list > /dev/null

sudo apt update
sudo apt install -y kubelet kubeadm
sudo apt-mark hold kubelet kubeadm

sudo systemctl enable --now kubelet

Join the worker to the cluster

1. Create Kubernetes Cluster's Token

Run this on the control-plane node:

kubeadm token create --print-join-command

It prints something like:

sudo kubeadm join <CONTROL_PLANE_IP>:6443 --token <TOKEN> --discovery-token-ca-cert-hash sha256:<HASH>

2. Run that join command on the worker

Paste and run it on the worker node (with sudo).

sudo kubeadm join <CONTROL_PLANE_IP>:6443 --token <TOKEN> --discovery-token-ca-cert-hash sha256:<HASH>

3. Verify from the Control Plane

kubectl get nodes -o wide

Installing Kubernetes Single-Node Setup on Ubuntu 24.04

Korakrit Chariyasathian — Wed, 21 May 2025 15:47:17 +0000

1. Prepare the Ubuntu VM

Update system packages:

sudo apt update && sudo apt upgrade -y

Disable swap (Kubernetes requires this):

sudo swapoff -a
sudo sed -i '/ swap / s/^/#/' /etc/fstab

2. Enable Kernel Modules and Sysctl Settings

sudo modprobe overlay
sudo modprobe br_netfilter

sudo tee /etc/sysctl.d/kubernetes.conf <<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF

sudo sysctl --system

3. Install Docker or Containerd (Container Runtime)

sudo apt install -y docker.io
sudo systemctl enable docker
sudo systemctl start docker

or Install Containerd

sudo apt install -y containerd

sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml

# Enable systemd cgroup
sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml

sudo nano /etc/containerd/config.toml

[plugins."io.containerd.grpc.v1.cri"]
and then,

sandbox_image = "registry.k8s.io/pause:3.10"

sudo systemctl restart containerd
sudo systemctl enable containerd

4. Add the Kubernetes APT Repository

sudo apt install -y apt-transport-https ca-certificates curl gpg

sudo mkdir -p /etc/apt/keyrings

curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.33/deb/Release.key | \
sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] \
https://pkgs.k8s.io/core:/stable:/v1.33/deb/ /" | \
sudo tee /etc/apt/sources.list.d/kubernetes.list

sudo apt update

5. Install Kubernetes Tools: kubeadm, kubelet, kubectl

sudo apt install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

6. Install Helm

curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

curl -LO https://get.helm.sh/helm-v3.17.3-linux-amd64.tar.gz
tar -zxvf helm-v3.17.3-linux-amd64.tar.gz
sudo mv linux-amd64/helm /usr/local/bin/helm
rm -rf helm-v3.17.3-linux-amd64.tar.gz

helm version

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

7. Initialize the Kubernetes Control Plane

sudo kubeadm init --pod-network-cidr=10.0.0.0/16

8. Set up kubeconfig:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

9. Check Node Status

kubectl get nodes

Output: is-kube shows as NotReady (because Pod Network is not yet installed)

Install Cilium CNI

1. Install cilium-cli

curl -L --remote-name https://github.com/cilium/cilium-cli/releases/latest/download/cilium-linux-amd64.tar.gz
sudo tar xzvf cilium-linux-amd64.tar.gz -C /usr/local/bin
rm cilium-linux-amd64.tar.gz

2. Install Cilium

cilium install

3. Check Node Status and Cilium Status

kubectl get nodes
cilium status

Grafana (MySQL) + Nginx reverse proxy

Korakrit Chariyasathian — Thu, 21 May 2020 17:57:49 +0000

Git clone from korakrit-c/grafana-with-nginx-on-docker.git

git clone https://github.com/korakrit-c/grafana-with-nginx-on-docker.git

Make sure you can run docker-compose then starting it
```
docker-compose up -d
```
Create a new config file on "nginx-revers-storage/conf.d/"
```
cd nginx-revers-storage/conf.d/
nano grafana.conf
```

Add this config

upstream grafana_dashboard {
  server    grafana_dashboard:3000;
}
server {
    listen 80;
    listen [::]:80;
    server_name grafana.local;
    location / {
        proxy_pass http://grafana_dashboard;
    }
}

Restart nginx

docker exec nginx_reverse nginx -s reload

Installing Docker on Raspberry Pi 4

Korakrit Chariyasathian — Tue, 19 May 2020 17:18:27 +0000

Update repository and upgrade packages

sudo apt-get update
sudo apt-get upgrade

Download installer from get.docker and install Docker

curl -fsSL get.docker.com -o get-docker.sh && sh get-docker.sh

Add "pi" user to Docker Group
```
sudo usermod -aG docker pi
```

Setup the Docker Repo

nano /etc/apt/sources.list.d/docker.list

Add this

deb https://download.docker.com/linux/raspbian/ stretch stable

Update repository and upgrade packages again
```
sudo apt-get update
sudo apt-get upgrade
```

DEV Community: Korakrit Chariyasathian

On‑Premises Kubernetes Networking Architecture: A Comprehensive Architectural Synthesis

1. Context, Scope, and Design Objectives

2. Rationale for FRR and MetalLB in On‑Premises Kubernetes

2.1 Absence of a Cloud Provider and Its Consequences

2.2 MetalLB as a Functional Analog to Cloud Load Balancers

3. Justification for BGP Mode over L2 Mode (Engineering Perspective)

3.1 Architectural Implications of L2 Mode

3.2 Architectural Advantages of BGP Mode

4. Network Roles and Separation of Responsibilities

4.1 FRR Virtual Machine as a Fabric Router

Network Interface Partitioning

Netplan Configuration (FRR VM)

FRR (FRRouting) Configuration Overview

4.2 Kubernetes Nodes as Disposable Compute Elements

5. FRR Configuration: Conceptual Overview

5.1 Governing Principles

5.2 Functional Responsibilities of FRR

5.3 On Explicit Peer Configuration

6. MetalLB Configuration: Conceptual Overview

7. Autoscaling Readiness and Evolutionary Path

7.1 Pod‑Level Autoscaling

7.2 Node‑Level Scaling: Present and Future

8. Architectural Continuity Across Cloud Migration

9. Architecture Diagram (Textual Representation)

10. Sequence Diagram: LoadBalancer Traffic Flow

11. Concluding Observations

PlantUML Diagram

Author's Note

Hyper-V Configuration Guide for Kubernetes and Supporting Network Services

Architecture Overview

Baseline: Create a New Virtual Machine (Applies to All VMs)

1. Create New Virtual Machine

2. Assign Memory

3. Configure Networking (Initial Wizard Step)

4. Connect Virtual Hard Disk

5. Installation Options

Operating System Baseline

Netplan Baseline Configuration

Post-Creation VM Configuration (Before First Boot)

1. Firmware / Secure Boot

2. Processor / Hardware Acceleration

3. Memory Settings

4. Network Adapters – Advanced Features

External NIC (vEthernet-External)

Internal NIC (K8S-Internal)

5. VLAN Configuration

6. Integration Services

Part 1: Control Plane Node Configuration

Network Adapter Layout

MAC Address Spoofing

Resource Recommendation

Part 2: Worker Node Configuration

Network Adapter Layout

MAC Address Spoofing

Resource Recommendation

Part 3: FRR VM Configuration

Network Adapter Layout

MAC Address Spoofing

Resource Recommendation

Summary of Best Practices

Additional Notes

Author's Note

Installing Kubernetes and Worker Node Setup on Ubuntu 24.04

Kubernetes Setup on Ubuntu 24.04 (Kubernetes v1.33)

1. Prepare the Ubuntu VM

2. Enable Kernel Modules and Sysctl Settings

3. Install and Configure Containerd (Container Runtime)

4. Add the Kubernetes APT Repository (v1.33)

5. Install Kubernetes Tools

6. Install Helm

7. Initialize the Kubernetes Control Plane

8. Set Up kubeconfig

9. Verify Node Status

Install Cilium CNI

Install cilium-cli

Install Cilium

Verify Cluster Status

Worker Node Setup

1. Prepare the Ubuntu VM

External NIC (`vEthernet-External`)

Internal NIC (`K8S-Internal`)