In this post, I analyze a distributed architecture built with PHP, designed to handle data entirely in-memory. The model, titled DIM-DRRA (Distributed In-Memory Delta-Replicated Reactor Architecture), moves away from the typical PHP request-response lifecycle to implement a stateful and persistent system.
The primary goal was to achieve high throughput without relying on external PHP frameworks, managing the core logic through a custom event-loop server.
1. Full Architectural Flow (CQRS)
The system is built on a clear separation of concerns between write and read operations (CQRS). This allows the Master node to focus on consistency while the Slaves handle horizontal scaling.
General Mechanism:
The entry point is managed by NGINX, which routes requests based on the operation type. Write requests are directed to a single Master node to prevent race conditions and ensure sequential consistency. Read requests are distributed across a pool of Slave nodes. Data propagation from Master to Slaves happens via asynchronous delta-replication.
2. Write Path & Persistence Logic
The write path is designed for high consistency and durability. Even though the system is "In-Memory", it implements a protection layer to prevent data loss.
Internal Process:
- Event-Loop Master: The Master Kernel runs a non-blocking TCP server. When a write request arrives, it is processed within the event-loop.
- Journaling (WAL): The system follows a Write-Ahead Logging strategy. Every change is first serialized and logged to a persistent Journal (SQLCipher on SSD).
- State Commit: The In-Memory Master DB is updated only after the journal entry is successfully secured on disk. This ensures that the state in RAM is always consistent with the persistent record.
- Snapshotting: To facilitate rapid restarts, the Master periodically generates binary snapshots. During a "warm-up" phase, the system loads these snapshots directly into memory, bypassing standard SQL parsing overhead.
3. Read Path & Horizontal Scaling
The read path is optimized for low-latency by eliminating disk I/O and distributing the load across multiple Docker instances.
Internal Process:
- Delta-Replication: Instead of sending the full state, the Master pushes only the specific changes (deltas) to the Slave nodes. This keeps the network overhead low and synchronization fast.
- Slave Kernels: Each Slave runs a dedicated kernel based on an event-loop. Since the entire dataset is resident in the Slave's local RAM, the response time is limited only by network latency and memory access.
- Scaling: Because the Slaves are read-only and decoupled from the write persistence logic, they can be scaled horizontally by adding more containers. NGINX manages the load balancing, ensuring that the Master node is never burdened by read traffic.
Technical Results
During validation with k6, the architecture maintained the following metrics:
- Throughput: 4,300+ req/sec.
- Latency: 10ms median over 60,000 requests.
- Error Rate: 0% during the saturation phase.
While this PHP prototype verifies the efficiency of these patterns, for real-world production, the same architecture could be implemented in native languages built for event-loops.
The complete architectural breakdown and benchmark session are available in this video:
DevArchOverclock - YouTube
This channel is a lab dedicated to exploring the architectural limits of PHP and high-performance systems. The goal is to deconstruct and rebuild web software to understand how to truly scale, starting from the foundations to handle massive loads. The videos focus on: PHP Scalability: Developing custom architectures and building frameworks from scratch for high-traffic scenarios. Low-level & Networking: Optimizing HTTP flow, socket management, and analyzing low-level stack behavior. Bottleneck Analysis: Benchmarking and stress testing to identify infrastructure limits and go beyond standard implementations. High-Performance Ecosystem & Research. A space for those who want to understand how data moves between the network and the code, optimizing every single packet to maximize throughput. Overclock your Dev.
youtube.com
Top comments (0)