🎯 Core Architectural Philosophy
"Keep data moving through memory, delegate persistence to the center; the edge does the heavy lifting, the center does the thinking."
The fundamental rule of this architecture is the absolute segregation of the Data Plane and the Control Plane. The edge gateway pursues an absolute $O(1)$ in-memory computing model with zero local disk I/O. All complex states, business logic syncing, and heavy data persistence are delegated to the central management engine asynchronously or via on-demand up-streaming.
🏗️ Architectural Overview
[ Client HTTPS Request ]
│
▼ (with SNI Domain)
┌─────────────────────────────────────────────────────────────┐
│ 1. TLS Termination Layer (In-Memory O(1) Cert Lookup) │
│ ➔ Misses are synced via global network stream; │
│ NEVER read from a local disk. │
└─────────────────────────────────────────────────────────────┘
│
▼ (Decrypted Cleartext HTTP Request)
┌─────────────────────────────────────────────────────────────┐
│ 2. Risk & Billing Gatekeeper (DashMap Segmented Locks) │
│ ➔ Local atomic accumulation (fetch_add), │
│ zero blocking remote RPCs. │
└─────────────────────────────────────────────────────────────┘
│
▼ (Security Interception Passed)
┌─────────────────────────────────────────────────────────────┐
│ 3. Sandbox Execution Layer (Wasmtime Engine) │
│ ➔ Deserializes central AOT pre-compiled machine-code │
│ snapshots with zero cold-start latency. │
└─────────────────────────────────────────────────────────────┘
│
┌──────────────────────┴──────────────────────┐
▼ (Asynchronous Data Stream) ▼ (Synchronous Response)
┌───────────────────────────────────┐ ┌───────────────────────────────────┐
│ 4. Bypass Logging Layer │ │ 5. Response Output Layer │
│ (Lock-Free Ring Buffer) │ │ │
│ ➔ Flushed directly to Kafka │ │ ➔ Blazing fast turnaround │
│ via network streams. │ │ back to the client. │
└───────────────────────────────────┘ └───────────────────────────────────┘
🛠️ Deep Dive: The 4 Core Solutions
- Atomic Billing Accumulation: Why we aren't afraid of data loss
The Core Logic: Metrics and billing data are flushed to the central infrastructure in batches every 10 seconds via network streams. If an edge node experiences a catastrophic hardware crash, the theoretical worst-case scenario is losing just a few cents worth of transaction metrics for a handful of tenants within that 10-second window. We trade a highly acceptable, microscopic precision loss for 99.999% gateway throughput and rock-bottom P99 latency.
Advanced Resilience (mmap / Shared Memory): The memory counters are instantiated within the operating system's shared memory (shm/mmap). If the gateway process panics or crashes, a system daemon spins up a new instance in milliseconds. The new process instantly re-attaches to the existing shared memory space, resulting in zero data loss (only a total bare-metal power failure would wipe it).
- TLS Certificates: Fitting millions of certs into edge nodes
Lazy Loading in Memory: When the edge gateway boots up, it loads zero SSL/TLS certificates into its configuration. Upon receiving a client's TLS handshake containing the SNI, it performs an $O(1)$ memory lookup. On a cache miss, it asynchronously fetches the cert from a high-speed, distributed data-center KV store and caches it in memory for subsequent requests.
Keyless SSL: For enterprise clients where private keys cannot leave on-premise infrastructure, the edge nodes store only the public certificates. During the TLS handshake, cryptographic signatures are calculated by sending brief, secure RPC queries to the client’s private key server, finalizing the handshake securely.
- Sandbox Isolation: Executing custom tenant code in < 1ms
AOT Pre-compiled Snapshots: Runtime calls to functions like compile_wasm() are strictly prohibited on the data plane. When a user uploads custom edge code, the control plane immediately pre-compiles it via AOT (Ahead-of-Time) into machine code optimized for target architectures (AMD64/ARM64). The edge gateway simply streams and deserializes this snapshot (deserialize()), eliminating compilation overhead entirely.
Two-Tier Caching (L1 Memory + L2 Local Area KV): The gateway utilizes concurrent structures (like Rust’s DashMap) to manage hot code modules. It leverages an LRU cache pool to recycle sanitized sandbox instances, compressing cold-start latencies down to microseconds.
- High-Throughput Logging: Writing logs at hundreds of thousands of RPS
Lock-Free Memory Queues + Network Streams (Logpush): The gateway’s execution context pushes raw logs into a lock-free ring buffer and terminates the client request immediately. Dedicated background threads periodically flush these buffers over UDP or gRPC to remote telemetry pipelines like Kafka or ClickHouse. The gateway process never touches local persistent disk storage.
Cloud-Native stdout Redirection: By using asynchronous, non-blocking configurations in tracing frameworks, logs are written straight to standard output (stdout), where they are immediately drained away by external daemon utilities like FluentBit, Vector, or Logstash.
⚠️ High-Risk Architectural Boundaries (Bug Mitigation)
🚨 Risk 1: Cold Domain Flood Attacks leading to Out-Of-Memory (OOM)
The Scenario: Malicious actors forge millions of non-existent domain requests targeting the edge gateway. These domain misses bypass the internal cache, overwhelm the central KV lookup clusters, and clutter the edge node's memory with useless lookup states.
Mitigation A (Negative Caching): For domains unrecognized by the central directory, the gateway records a temporary (Domain, NotFound) placeholder in its cache, proactively rejecting duplicate lookups for a sliding 5-second interval.
Mitigation B (Pre-Distributed Bloom Filters): The control plane periodically generates a highly compressed Bloom Filter representing all legitimate customer domains and broadcasts it to all edge nodes. Incoming requests are filtered against this lightweight, in-memory array. Non-existent domains are dropped immediately at the edge.
🚨 Risk 2: The Infamous Rust DashMap Deadlock Trap
The Scenario: Within the exact same execution scope/thread, a read lock guard (Ref) on a DashMap remains active while the code simultaneously attempts a write or modification operation (insert/remove) on the very same map shard.
Mitigation: Enforce strict lexical block scoping {} to precisely control the lifecycle of map.get() return handles, or explicitly trigger an immediate drop(read_guard) before calling mutating operations.
📝 Architectural Takeaways
"Data consistency isn't a game of 'the stronger the better'. It's a game of 'the weaker the better', as long as it satisfies business constraints."
"Never force an edge gateway to act like a storage engine. Its destiny is to forward traffic at maximum velocity, and to produce telemetry at the lowest possible cost."
This architectural blueprint mirrors the low-level philosophies utilized by modern Content Delivery Networks and Edge Compute infrastructures like Cloudflare and Fastly. What are your thoughts on building completely diskless data planes? Let's discuss in the comments below! 🦀⚡
Top comments (0)