We had a use case to expose Kubernetes Gateway API from a multi-tenant AKS cluster through Azure Application Gateway(With WAF). The integration wasn't straightforward, health probes failed, backends showed unhealthy, and traffic wouldn't flow. This post details a solution that addresses these integration challenges.
Since this solution and POC is based on K8s Gateway API on AKS, as a bonus this approach aligns well with the broader Kubernetes ecosystem direction. With Ingress-NGINX being retired in March 2026 and Ahmed Sabbour, Principal PM Lead for Azure Kubernetes Service, confirming that AKS Application Routing will be powered by Istio and Gateway API in H1 2026, adopting Gateway API now positions you well for the future.
I have developed a POC to demonstrate how to set up K8s Gateway API on AKS and solve the integration challenges when exposing it via Azure Application Gateway in a multi-tenant environment. The complete solution is available at Github
The Challenge
Integrating Kubernetes Gateway API with Azure Application Gateway in a multi-tenant AKS cluster isn't as straightforward as it should be. When I attempted this integration, I encountered several issues:
- Health probes failing - Application Gateway couldn't verify backend health
- Backends marked unhealthy - Despite pods running correctly
- Connection timeouts - Traffic wouldn't flow through the expected path
- No clear documentation - Limited guidance for this specific architecture
This POC documents the solution to these challenges, providing a working reference architecture for teams facing similar requirements.
The POC Solution
The POC deploys two sample applications (app1 and app2) in an AKS cluster, exposed via Kubernetes Gateway API and accessed through Azure Application Gateway.
| Component | Description |
|---|---|
| sample-app-1 | Nginx pod returning "Hello from App 1" on path /app1 |
| sample-app-2 | Nginx pod returning "Hello from App 2" on path /app2 |
| health-responder | Dedicated pod for Application Gateway health probes on /healthz |
| Istio Gateway | Shared Gateway resource implementing K8s Gateway API |
| HTTPRoutes | Per-path routing rules in application namespaces |
Each application has its own HTTPRoute in its namespace, demonstrating the self-service routing model where application teams manage their own routes while sharing a common gateway infrastructure.
Note: To keep this POC simple and focused on the Azure integration challenges, app1 and app2 simulate applications running in two separate tenants inside the AKS cluster. I am planning a separate blog post on how to set up multi-tenancy on Kubernetes with Capsule and K8s Gateway API.
Architecture Overview
| Component | Purpose |
|---|---|
| Azure Application Gateway | Public entry point with WAF and TLS termination |
| Internal Load Balancer | Connects App Gateway to AKS cluster |
| Istio Gateway | Kubernetes Gateway API implementation |
| HTTPRoutes | Per-namespace routing rules for tenants |
| Istio Ambient Mesh | mTLS between services without sidecars |
End-to-End TLS Flow
Kubernetes Gateway API Components
Network Architecture
Critical Fixes for Azure Integration
During implementation, I discovered two critical issues specific to Azure. These are the core fixes that make this integration work. They are expected to be resolved when AKS releases their managed Gateway API solution.
Fix 1: HTTPRoute for Health Probes
Problem: App Gateway health probes to /healthz/ready returned 404 because Istio Gateway (Envoy) didn't know how to route health check requests.
Solution: Create a dedicated HTTPRoute that routes /healthz/* to a health-responder service.
HTTPRoute Configuration:
# From kubernetes/05-httproutes.yaml
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: health-route
namespace: gateway-health
spec:
parentRefs:
- name: mtkc-gateway
namespace: istio-ingress
rules:
- matches:
- path:
type: PathPrefix
value: /healthz
backendRefs:
- name: health-responder
port: 8080
Fix 2: externalTrafficPolicy for Azure DSR
Problem: Azure Internal Load Balancer uses Direct Server Return (DSR) mode. With DSR, inbound requests flow through the load balancer, but responses bypass it and return directly to the client (Application Gateway). The default externalTrafficPolicy: Cluster causes kube-proxy to apply SNAT when forwarding traffic across nodes, which breaks this asymmetric routing pattern and causes connection timeouts.
Solution: Patch the Gateway service to use externalTrafficPolicy: Local. Ideally include in the automation scripts to patch after the K8s Gateway API is created.
kubectl patch svc <gateway-service> -n istio-ingress \
-p '{"spec":{"externalTrafficPolicy":"Local"}}'
How DSR works with the fix
Inbound: Request travels through App Gateway → Internal Load Balancer → directly to node with pod (no cross-node SNAT)
Outbound: Response bypasses the Internal Load Balancer entirely and returns directly to App Gateway
This asymmetric routing (different paths for request and response) is the key characteristic of DSR that improves performance but requires externalTrafficPolicy: Local to work correctly with Kubernetes. This fix ensures proper traffic flow between Azure Application Gateway and the Kubernetes Gateway API.
Why Kubernetes Gateway API?
While the retirement of Ingress-NGINX is a catalyst for change, the real reason to adopt Kubernetes Gateway API is its superior design for building advanced platforms like multi-tenant Kubernetes clusters.
In nutshell, Kubernetes Gateway API provides a superior approach to multi-tenancy compared to legacy Ingress controllers. With Gateway API, a single shared Gateway serves as the centralised entry point, while individual HTTPRoutes in each tenant's namespace define their own routing rules. This eliminates the need to deploy separate Ingress controllers per tenant, reducing infrastructure overhead, simplifying operations, and enabling consistent traffic policies across all tenants through a unified control plane. Stay tuned, I'll explore this further in my next post.
Why Istio Ambient Mesh?
This solution uses Istio Ambient Mesh, which provides a Gateway API implementation. We chose Ambient mode over the traditional sidecar mode for the reasons outlined below.
| Aspect | Sidecar Mode | Ambient Mode |
|---|---|---|
| Containers per Pod | 2 (app + proxy) | 1 (app only) |
| Resource Overhead | Higher | Lower |
| Startup Latency | Higher | Lower |
| Mesh Updates | Pod restart required | Node-level update |
| Complexity | Sidecar injection management | Simplified operations |
| Security | Larger attack surface (full Envoy per pod, proxy in app pod) | Smaller attack surface (lightweight Rust-based ztunnel, isolated from app pod) |
We chose Ambient mode for this POC because of its lower resource overhead, simplified operations, and improved security posture—no sidecar injection to manage, no pod restarts required for mesh updates, and a reduced attack surface.
Note on AKS Istio Add-on: AKS fully supports custom Istio Ambient Mesh deployments, which is the approach used in this POC. The AKS Istio service mesh add-on is currently based on sidecar mode, which is a mature and well-supported option. It would be great to see AKS introduce Ambient mode support in their managed add-on in the future, giving users the choice between both approaches.
Try It Yourself
The complete solution is available as a POC repository with:
Terraform infrastructure as code
Kubernetes manifests for Gateway API
Automated deployment scripts
Validation test suite
Troubleshooting guide
Repository: github.com/shanaka-versent/shanaka-aks-poc
The README contains detailed prerequisites, step-by-step deployment instructions, and solutions to Azure-specific challenges.
Conclusion
We had a use case to expose Kubernetes Gateway API from a multi-tenant AKS cluster through Azure Application Gateway, and the integration wasn't straightforward. This POC provides a working solution that addresses the critical issues, specifically the HTTPRoute configuration for health probes and the externalTrafficPolicy: Local setting to ensure proper traffic flow with Azure's DSR.
Key takeaways
- Kubernetes Gateway API is the right choice for building advanced platforms like multi-tenant clusters, offering role-oriented design, standardised extensibility, and self-service routing that legacy Ingress controllers cannot match
- Istio Ambient Mesh provides a Gateway API implementation with operational and security advantages over sidecar mode, including lower resource overhead, simplified operations, and a reduced attack surface
- Azure-specific fixes (HTTPRoute for health probes and externalTrafficPolicy for DSR) are required for this integration today, and are expected to be resolved when AKS releases their managed Gateway API solution
This approach also aligns with the broader ecosystem direction. With Ingress-NGINX being retired and AKS planning to adopt Istio-powered Gateway API in H1 2026, the patterns documented here position you well for the future.
Resources
About the Author
Shanaka Jayasundera is a Solutions Architect specialised in Application Modernisation, Application Integration and cloud-native technologies. Connect on GitHub or via email at shanakaj@gmail.com.
Top comments (0)