webp)Envoy Proxy is a modern, high-performance open-source edge and service proxy originally developed by Lyft. It's designed for cloud-native applications and is widely used in service meshes like Istio. Envoy excels at handling traffic management, observability, and security in microservices architectures. It supports advanced protocols like HTTP/2 and gRPC natively, making it ideal for modern APIs, including gRPC services exposed to web clients via gRPC-Web.
In this article, we'll cover Envoy's core architecture, key concepts like upstream/downstream traffic and connection pooling, and then dive into a practical example: configuring Envoy to handle both gRPC-Web requests from browsers and native gRPC to your backend, while serving static frontend assets.
What is Envoy Proxy?
Envoy is an L7 proxy (with strong L3/L4 capabilities) that acts as a communication bus for distributed systems. Key features include:
.Dynamic configuration via xDS (discovery services)
.Rich observability (stats, logging, tracing)
.Advanced load balancing, circuit breaking, and retries
.First-class support for HTTP/2 and gRPC (including trailers)
.Extensibility through filters
It's often deployed as a sidecar in Kubernetes or as an edge proxy.
Envoy Architecture Overview
Envoy's architecture is modular and thread-based:
Listeners: Bind to downstream ports/IPs and accept connections from clients (downstream hosts).
Filter Chains: Process incoming data (e.g., TCP proxy, HTTP connection manager).
Clusters: Groups of upstream hosts (services Envoy forwards to).
Cluster Manager: Handles load balancing, health checking, and connection pooling to upstreams.
A typical request flow:
1.Downstream client connects to a listener.
2.Data passes through listener filters and network filters (e.g., 3.HTTP Connection Manager).
For HTTP, it goes through HTTP filters (routing, gRPC-Web translation, etc.).
4.The router filter selects a cluster based on route rules.
5.Envoy picks an upstream host, acquires a connection from the pool, and forwards the request.
Key Concepts: Downstream vs. Upstream
.Downstream: Refers to clients connecting to Envoy (e.g., browsers, other services). Envoy receives requests here via listeners.
.Upstream: Refers to backend services Envoy connects to (defined in clusters). Envoy acts as a client to these hosts.
Connection Pooling in Envoy
Connection pooling is a critical performance feature, especially for HTTP/2 and gRPC.
Envoy maintains per-cluster, per-worker-thread connection pools to upstream hosts. For gRPC (which uses HTTP/2):
.Connections are long-lived and multiplexed (multiple streams over one TCP connection).
. New connections are created on-demand when no idle
. connection/stream is available (up to circuit breaker limits).
. HTTP/2 features like GOAWAY drain connections gracefully.
. Pooling happens automatically in clusters configured with http2_protocol_options.
You can tune it via circuit breakers (e.g., max_connections, max_requests) in the cluster config.
In your gRPC setups, enabling HTTP/2 ensures efficient multiplexing—perfect for high-concurrency gRPC calls.
Where is the connection pool?
It's managed internally by the Cluster Manager for each upstream cluster. No separate "pool" resource—it's part of the cluster's runtime state.
Hands-On:
Configuring Envoy for gRPC-Web and Static Frontend
A common scenario: Your backend is a gRPC service (e.g., on port 50051), but browsers can't speak native gRPC (HTTP/2 with trailers). Use gRPC-Web from the browser, and let Envoy translate it to native gRPC.
Additionally, serve static frontend files from another server.
Here's a complete static Envoy config that does exactly that:
static_resources:
listeners:
- name: listener_http
address:
socket_address:
address: 0.0.0.0
port_value: 8080
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
codec_type: AUTO
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains: ["*"]
routes:
# Route gRPC-Web requests to the backend gRPC cluster
- match:
prefix: "/sales.CsvProcessor"
route:
cluster: backend_grpc
# Also handle /fileUpload if frontend uses it
- match:
prefix: "/fileUpload"
route:
cluster: backend_grpc
# All other requests go to the frontend static server
- match:
prefix: "/"
route:
cluster: frontend_server
http_filters:
- name: envoy.filters.http.grpc_web
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.grpc_web.v3.GrpcWeb
- name: envoy.filters.http.cors
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.cors.v3.Cors
- name: envoy.filters.http.router
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
clusters:
- name: backend_grpc
connect_timeout: 30s
type: STRICT_DNS
lb_policy: ROUND_ROBIN
http2_protocol_options: {}
typed_extension_protocol_options:
envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
"@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
explicit_http_config:
http2_protocol_options: {}
load_assignment:
cluster_name: backend_grpc
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: backend
port_value: 50051
- name: frontend_server
connect_timeout: 5s
type: STRICT_DNS
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: frontend_server
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: frontend
port_value: 80
admin:
access_log_path: "/tmp/admin_access.log"
address:
socket_address:
address: 0.0.0.0
port_value: 9901
Breaking It Down 🚀
Listener (Port 8080):
Accepts HTTP/1.1 (from browsers) or HTTP/2.
gRPC-Web Filter:
Converts gRPC-Web (HTTP/1.1) requests into native gRPC (HTTP/2) for your backend.
Routing:
/sales.CsvProcessor/* → backend_grpc cluster (gRPC service)
/fileUpload → same backend_grpc cluster
Everything else (/) → static frontend server
backend_grpc Cluster:
Uses HTTP/2 for efficient gRPC multiplexing and connection pooling.
Includes a CORS filter for seamless browser access.
Pro Tips:
Use DNS names like backend and frontend (ideal for Docker Compose or Kubernetes).
For local development without Docker, replace with host.docker.internal.
Run It:
envoy -c envoy.yaml
This setup leverages Envoy's connection pooling for high-performance gRPC traffic.
Next Steps:
Add TLS for production security
Enable stats and tracing
Switch to dynamic xDS config for zero-downtime updates
Envoy is powerful yet approachable—start with this config, monitor via the admin port (:9901), and scale effortlessly!
Happy proxying! 🚀
Top comments (0)