How I Built a Custom Kubernetes Ingress in Rust/Go That Outperforms Nginx by 11%
Silex: Re-engineering K8s Edge Routing for the Cloud-Native Era
The Pain Point: The Heavy Burden of Legacy Ingress
In modern cloud-native architectures, we often accept unnecessary complexity as a default. When deploying standard enterprise ingress controllers like Nginx into a lightweight cluster, several architectural inefficiencies immediately become apparent:
- Image Bloat and Cold Starts: The container footprint for standard controllers often exceeds 280MB. In our testing on a bare-metal environment, this resulted in cold start pull times stretching to over 4.5 minutes. For dynamic, auto-scaling environments, this latency is unacceptable.
- Cluster Pollution: Traditional ingress setups pollute the global cluster scope with validating webhook configurations. If a namespace is forcefully deleted, these webhooks leave behind orphaned states that block the creation of future network resources.
- Resource Overhead: Legacy architectures carry legacy codebases. Running a multi-process, configuration-reloading proxy inside a resource-constrained node wastes CPU cycles and memory that should belong to the actual backend applications.
To solve this, I designed Silex: a dedicated Kubernetes Ingress Controller built from scratch with a strict separation of concerns, utilizing Go for configuration orchestration and Rust for raw data plane execution.
The Architecture: Separating the Brain from the Muscle
Silex splits the traditional controller architecture into two distinct, isolated layers: the Control Plane and the Data Plane.
The Control Plane (Go Operator)
The control plane is written in Go, leveraging the official Kubernetes client-go libraries. Its sole responsibility is to act as the cluster's "brain." It watches the Kubernetes API for Ingress resources and corresponding EndpointSlices. By utilizing Go's robust concurrency patterns and native integration with the Kubernetes ecosystem, it monitors cluster state changes with minimal footprint, completely decoupled from the data path.
The Data Plane (Rust Ingress Engine)
The data plane is written in pure Rust using Tokio for asynchronous network I/O. It acts as the "muscle," binding directly to host ports 80 and 443. Instead of relying on heavy configuration reloads or complex parsing libraries, the engine keeps routing rules inside an ultra-fast, concurrent hash map using the dashmap crate. This allows the proxy to achieve thread-safe, non-blocking lookups and route client traffic in fractions of a millisecond.
The Debugging Story: Hunting a Silent 1.8 Million Request Drop
Building infrastructure from scratch means encountering unique engineering hurdles. During our initial benchmark, Silex registered a staggering 1.8 million read errors with zero successful responses. The engine was accepting connections at a rate of 55,000 per second without crashing, but dropping the sockets instantly.
The investigation revealed a subtle architectural friction between the control plane's payload structure and the data plane's raw stream matching. To maximize routing speeds, the Rust data plane avoids heavy reflection or generic JSON deserialization. Instead, it scans the raw byte buffers using direct string pattern matching:
if let (Some(h_idx), Some(i_idx)) = (req.find("\"host\":\""), req.find("\"ip\":\""))
The bug was purely syntax-driven: the testing payload contained a single white space ("host": "silex.local"), which failed the strict, space-less match condition inside the Rust binary. Because the error handling logic was designed to exit the task silently rather than panicking, the thread dropped the TCP connection immediately without an HTTP response status. Correcting the payload format to align perfectly with the low-level byte-matching logic resolved the issue instantly, unlocking the engine's true performance.
The Benchmark: Head-to-Head Against Nginx
The benchmark was conducted on an identical K3s cluster, routing traffic to a standard HTTP echo-server backend under a concurrent load of 400 connections spread across 4 processing threads for 30 seconds using wrk.
| Metric | Nginx Ingress Controller | Silex (Rust/Go Architecture) | Performance Delta |
|---|---|---|---|
| Requests per Second | 2,274.13 req/sec | 2,536.39 req/sec | Silex outperformed by 11.5% |
| Total Processed Requests | 68,293 requests | 76,161 requests | Silex served 7,868 more requests |
| Maximum Latency | 1.97 seconds | 1.98 seconds | Identical (Backend saturated) |
| Socket Errors (Timeouts) | 286 | 302 | Saturation transferred to backend |
| Binary Size / Boot Time | ~280MB / ~4.5 minutes | 1.7MB / Instantaneous | Silex eliminates cold starts |
During the Nginx test, the controller experienced 286 socket timeouts as the architecture struggled under the connection pool limit. Silex handled the connection limits with ease; the 302 timeouts observed during its run were verified to be backend drops, where the single-instance Node.js echo-server's event loop became completely saturated by the sheer speed of the Rust data plane.
Future Roadmap: Moving to Production-Ready
While Silex successfully demonstrated superior raw routing capabilities and an almost non-existent resource footprint compared to Nginx, transforming it into an enterprise-grade product requires implementing production safeguards:
-
Native TLS/HTTPS Termination: Integrating native SSL/TLS decryption into the Rust data plane using
rustls, mapped dynamically to cert-manager secrets synced via the Go operator. - Active Health Checking: Implementing a continuous loop where the Go operator validates backend endpoint availability, dynamically flushing dead IPs from Rust's memory map without traffic interruption.
-
Prometheus Metrics Integration: Exposing a low-overhead
/metricsendpoint from the Tokio worker threads to track active connection states, latency percentiles, and error tracking natively.
Open Source & Contributions
If you want to try Silex, contribute to the Go Operator, or optimize the Rust Data Plane, check out the repository on GitHub.
Contributions, bug reports, and performance feedback are welcome.
- GitHub Repository: github.com/AbdullrahmanEissa/Silex
- Licensing: Open-Source (MIT License)
Join the project to help build a lighter, faster, and cloud-native alternative to legacy ingress architectures.
Top comments (0)