In part 1 of this series, I have discussed Ingress controllers and Gateway APIs as a way to control ingress traffic into applications deployed on top of a Kubernetes cluster.
In the second of this series, I will discuss intra-cluster (East-West) traffic passed through inside a Kubernetes cluster (i.e., between the Pods) and egress traffic outside the Kubernetes cluster.
Terminology
Before we deep dive into the article, let's review some important concepts:
- Container Network Interface (CNI): A Cloud Native Computing Foundation project that provides a standardized specification and set of plugins for configuring network interfaces and connectivity for Linux containers (Reference: CNI - the Container Network Interface).
- Network Policies: Rules that control the allowed inbound and outbound network traffic to and from Pods, providing fine-grained segmentation and security within the cluster (Reference: Network Policies).
- Service Mesh: An infrastructure layer that manages service-to-service communication in a microservices architecture by providing reliability, observability, and security features uniformly across all services without requiring code changes (Reference: Service Mesh).
- eBPF (extended Berkeley Packet Filter): A Linux kernel technology that lets small, safe programs run inside the kernel without changing its code. These programs can monitor, filter, and control system events like network traffic or system calls, improving observability and security with low overhead (Reference: eBPF).
- Calico: An open-source networking and security solution for Kubernetes that provides scalable, high-performance Layer 3 networking and enforces fine-grained network policies (Reference: What is Project Calico).
- Cilium: An open-source, cloud-native networking, security, and observability solution for Kubernetes that leverages eBPF technology to provide high-performance, identity-aware network policies and scalable pod connectivity (Reference: Cilium).
- Istio: An open-source service mesh that provides a transparent layer for managing, securing, and observing service-to-service communications in microservices architectures, using Envoy proxies for traffic control, security features like mutual TLS, and observability without requiring changes to application code (Reference: Istio).
- Envoy Proxy: A high-performance, open-source edge and service proxy designed for cloud-native applications that facilitates service-to-service communication with advanced load balancing, traffic routing, and security features in distributed microservices architectures (Reference: Envoy).
- Pod: The smallest deployable unit in Kubernetes that consists of one or more tightly coupled containers sharing storage, network, and a specification for how to run them in a shared context (Reference: Pods).
Container Network Interface and Network Policies
A CNI (Container Network Interface) in Kubernetes is needed to provide a standardized, pluggable way to set up networking for pods. It handles assigning IP addresses, configuring network interfaces, and routing so pods can communicate inside the cluster and with external systems. Without CNI, Kubernetes would lack the consistent networking required for container communication, service discovery, and networking scalability.
Network Policies in Kubernetes are needed to control how pods communicate with each other and external endpoints. They provide fine-grained traffic rules to isolate workloads, improve security, and reduce the attack surface. By defining ingress and egress rules, Network Policies help prevent unauthorized access, contain potential attacks, and enforce compliance requirements within the cluster.
Amazon
AWS implements network policies in EKS by integrating Kubernetes NetworkPolicy resources with the Amazon VPC CNI plugin. This integration provides native support for Kubernetes network security in EKS clusters, allowing operators to create fine-grained rules that control ingress and egress traffic to and from pods. By default, pods in Kubernetes accept traffic from any source, which can pose security risks. AWS addresses this by enabling network policies that act as a whitelist, allowing you to define which pods can communicate with each other based on labels and namespaces.
Azure
Azure implements network policies by integrating Kubernetes NetworkPolicy resources with Azure's native networking features through the Azure Container Networking Interface (CNI). AKS supports two implementations of network policies: Azure Network Policies and Calico Network Policies. The Azure Network Policy engine runs as a DaemonSet on every node and watches for Kubernetes NetworkPolicy objects to enforce traffic rules.
Google Cloud Platform
Google implements network policies in Google Kubernetes Engine (GKE) by enabling Kubernetes NetworkPolicy resources that act as pod-level firewall rules to control ingress and egress traffic between pods and services within the cluster. Network policy enforcement is integrated into GKE and built on top of the GKE Dataplane V2, providing native, scalable enforcement without additional setup on Autopilot clusters. For Standard clusters, network policy enforcement can be enabled via the Google Cloud Console, CLI, or API, which re-creates node pools with the necessary configurations.
Here is a comparison table of the various capabilities of Network Policies offered by AWS, Azure, and GCP:
Open-Source Alternatives for CNI and Network Policies
Other than the built-in CNI and Network Policies offered by AWS, Azure, and GCP, there are open-source solutions that can be integrated into a Kubernetes cluster when building a cloud-agnostic architecture. Below are some examples:
Cilium
Cilium is an open-source, cloud-native networking and security project that uses eBPF technology to provide high-performance networking, observability, and security for Kubernetes clusters. It implements network policies by enforcing identity-aware, layer 3, 4, and 7 rules using eBPF-based programs attached to network interfaces.
Calico
Calico is an open-source networking and security solution for Kubernetes that provides scalable, high-performance Layer 3 networking and enforces fine-grained network policies. It uses standard IP routing and protocols like BGP to connect workloads securely across cloud and on-premises environments.
Egress Traffic
In Kubernetes, egress traffic control is managed by creating NetworkPolicy resources with egress rules that explicitly allow or deny outbound connections from pods to specific destinations such as IP ranges, namespaces, or ports; by default, pods can send traffic outbound freely, so applying a default deny egress policy ensures isolation until specific egress permissions are granted. This mechanism provides fine-grained control over which external services or destination pods can access, enhancing cluster security and compliance by minimizing unintended data exposure or communication. Service meshes like Istio can also complement this by providing egress gateways for centralized policy-driven control and observability of outbound traffic. Overall, Kubernetes network policies enable secure, controlled, and auditable management of egress traffic at the pod level.
AWS, Azure, and GCP all control egress traffic using similar fundamental building blocks like Kubernetes NetworkPolicies and cloud-native networking features, but each has unique implementations and integrations with its networking stacks.
- AWS uses the Amazon VPC CNI plugin integrated with Elastic Network Interfaces (ENIs) and Security Groups, providing pod-level network isolation with egress filtering enforced via eBPF programs on nodes, plus cloud-level firewall controls and Transit Gateway for centralized traffic management.
- Azure uses Azure CNI with network policies enforced via iptables (Linux) or HNS ACLs (Windows), supporting both overlay and flat networks for different egress use cases, integrated with Azure Virtual Network and Network Security Groups for fine-grained outbound traffic control.
- GCP uses the native GKE Dataplane V2 with Calico and eBPF-based enforcement, integrated deeply with Google Cloud VPC and Cloud Router for dynamic routing and policy enforcement, enabling scalable and automated egress control combined with Google's global backbone network.
Service Mesh
A service mesh in Kubernetes is a dedicated infrastructure layer that manages and controls service-to-service communication within a cluster by transparently injecting sidecar proxies alongside application containers. It provides features like traffic management, security (including mutual TLS encryption and authorization), observability, load balancing, and service discovery without requiring changes to application code, allowing teams to securely and reliably connect, monitor, and control microservices at scale.
Service mesh in Kubernetes provides capabilities such as advanced traffic management (including load balancing, traffic routing, retries, and circuit breaking), security features like mutual TLS authentication and authorization policies, deep observability through metrics, distributed tracing, and logging, and resilience enhancements like request timeouts and rate limiting. It abstracts service-to-service communication from application code, enabling easier management, security, and monitoring of microservices at scale. Service meshes also facilitate canary deployments and traffic splitting for smooth application updates.
Amazon
Amazon implements service mesh in Kubernetes through AWS App Mesh, a managed service mesh that provides application-level networking to control service-to-service communication.
On September 30, 2026, AWS will discontinue support for AWS App Mesh, so for service mesh in this blog post, I will recommend integrating Amazon EKS with Istio.
Istio uses a two-component architecture of a data plane, which consists of Envoy proxy sidecars injected alongside each microservice pod to intercept and manage all network traffic, and a control plane that dynamically configures the Envoy proxies based on policies and rules.
Azure
Azure implements service mesh in Kubernetes primarily through the Istio-based service mesh add-on for Azure Kubernetes Service (AKS). The Istio-based add-on is fully supported and tested by Microsoft, providing a managed Istio control plane integrated with AKS. It enables features like mutual TLS encryption, advanced traffic routing, access control, observability through integrations with Azure Monitor and Grafana, and lifecycle management of Istio components.
Google Cloud Platform
Google Kubernetes Engine (GKE) implements Cloud Service Mesh as a fully managed service mesh based on Istio, known as Managed Cloud Service Mesh. This service mesh uses a managed control plane hosted by Google Cloud and Envoy sidecar proxies injected automatically into Kubernetes pods to manage and secure service-to-service communication without changing application code. The mesh supports traffic management features such as service discovery, load balancing, canary and blue-green deployments, and circuit breaking at the L7 application layer.
Here is a comparison table of the various capabilities of Service Mesh offered/supported by AWS, Azure, and GCP:
Open-Source Alternatives for Service Mesh
Other than the service mesh support offered by AWS, Azure, and GCP, there are open-source solutions that can be integrated into a Kubernetes cluster when building a cloud-agnostic architecture. Below are some examples:
Istio
Istio is an open-source service mesh platform that manages, secures, and observes service-to-service communication in Kubernetes. It works by injecting Envoy proxies as sidecars into each pod to intercept network traffic, while a control plane dynamically configures these proxies based on policies. Istio provides features like advanced traffic routing, mutual TLS security, and detailed observability, allowing centralized control over microservices networking without code changes.
Linkerd
Linkerd is a lightweight, simple service mesh for Kubernetes that adds security, reliability, and observability features by injecting a small proxy sidecar into each pod. It automatically handles traffic routing, mutual TLS encryption, retries, and load balancing to secure and manage service communication without code changes.
Kuma
Kuma is a universal open-source service mesh that supports Kubernetes, VMs, and bare metal. It uses a control plane and Envoy sidecar proxies injected into pods to provide traffic routing, security with automatic mTLS, observability, and multi-zone/multi-cluster support. Kuma is designed for ease of use, supporting multiple meshes on a single control plane and hybrid infrastructure environments.
Network Policies vs. Service Mesh
Choosing between Network Policies and Service Mesh depends on your specific needs and environment complexity:
Summary
In the second part of this series, I have explained the different alternatives for controlling traffic inside a Kubernetes cluster (i.e., between Pods), and the various managed services offered by AWS, Azure, and GCP.
Kubernetes Network Policies control pod traffic at the IP and port level for basic network isolation, while Service Mesh manages service-to-service communication with advanced features like secure mutual TLS encryption, fine-grained traffic routing, and observability. Use Network Policies for simple boundary enforcement and Service Mesh when requiring enhanced security, traffic control, and monitoring. Often, combining both provides layered, robust control in Kubernetes environments.
Following this two-part series, I encourage the readers to invest time in learning about Kubernetes capabilities, knowing which one to choose for which scenario and business requirements.
Disclaimer: AI tools were used to research and edit this article. Graphics are created using AI.
About the author
Eyal Estrin is a seasoned cloud and information security architect, AWS Community Builder, and author of Cloud Security Handbook and Security for Cloud Native Applications. With over 25 years of experience in the IT industry, he brings deep expertise to his work.
Connect with Eyal on social media: https://linktr.ee/eyalestrin.
The opinions expressed here are his own and do not reflect those of his employer.
Top comments (0)