When ingestion pipelines approach tens of millions of events per second on commodity hardware, traditional logging architectures fail at three specific bottlenecks: heap allocation overhead, thread lock contention, and CPU cache thrashing.
Most structured logging implementations rely on dynamically allocated string buffers, lock-based thread synchronization, or complex LRU (Least Recently Used) cache evictions. At line-rate telemetry scale, these patterns introduce non-deterministic latencies and context switching that degrade throughput.
This article details the architectural design of PULP, a native Windows C11 telemetry engine designed to process, precompress, and write high-volume logs to disk with bounded memory usage and zero dynamic memory allocations on the ingestion hot path.
1. The Zero-Allocation Hot Path
The primary rule of the ingestion hot path is zero malloc calls after initialization. Dynamic memory allocations introduce non-deterministic execution times and allocator lock contention across threads.
Instead of stringifying telemetry fields on the fly, incoming structured events are transformed into fixed 32-byte cache-aligned binary records.
Fixed 32-Byte Serialized Records
Here is the exact memory layout used to guarantee that each event fits perfectly into half a standard 64-byte L1 CPU cache line:
|--------------------------------------------------------------------|
| sequence (8 Bytes) |
|--------------------------------------------------------------------|
| timestamp (8 Bytes) |
|---------------------------------|----------------------------------|
| url_idx (4B) | ip_idx (4B) |
|----------------|----------------|----------------|--------|--------|
| http_code (2B) | duration_ms(2B)|resp_size (2B) |verb(1B)|flag(1B)|
|----------------|----------------|----------------|--------|--------|
typedef struct __declspec(align(32)) {
uint64_t sequence;
uint64_t timestamp;
uint32_t url_idx;
uint32_t ip_idx;
uint16_t http_code;
uint16_t duration_ms; // 0-65s
uint16_t response_size; // 0-65KB
uint8_t http_verb;
uint8_t flags; // Bits for SSL, cache, etc.
} SerializedEntry;
(Code snippet representing the core logging struct)
- Pre-allocation: All memory pools (thread-local queues, compression buffers, staging blocks) are allocated up front during runtime initialization using _aligned_malloc.
- Cache Alignment: The struct is strictly forced to a 32-byte boundary via __declspec(align(32)). This eliminates false sharing and maximizes memory bus efficiency during SIMD register loads.
2. Lock-Free, Evictionless Thread-Local Dictionaries
Sharing a global symbol or dictionary table across multiple worker threads introduces lock contention or synchronization overhead (atomics, CAS loops) that severely caps multi-core scaling.
Multi-Hash Cascade Design
Each worker thread maintains its own isolated dictionary table. To handle collisions without costly dynamic resizing, the cache uses a fully unrolled scalar design with a Multi-Hash Cascade:
- L1 Probe (CityHash64): The primary bucket is hit first.
- L2 Probe (XXH3_64): If the primary probe range is saturated, it breaks primary clustering using a completely different hashing algorithm.
- L3 Probe (CityHash ^ XXH3): A combined hash fallback.
Evictionless Architecture (No LRU)
Implementing eviction algorithms like LRU requires maintaining access order pointers or counters, which degrades throughput.
Instead, the dictionary uses a fixed capacity per block lifecycle.
When a dictionary block reaches capacity, the thread flushes the block to disk and soft-resets the local table index, preserving the data for a future entry, thus extending the buffering effect.
The cost of resetting an integer index is O(1).
3. Vectorized (AVX2) In-Line Anonymization
When handling network telemetry (e.g., IPv4/IPv6 logs), regulatory frameworks often require masking IP addresses before writing to persistent storage. Performing ASCII string parsing and manipulation per packet introduces significant CPU overhead.
The engine uses an AVX2-accelerated inline pipeline for IP masking:
Zero Parsing Strategy: By collecting hextet offsets directly during a single-pass scan, the engine performs IPv6 :: expansion and applies configurable bitmasks (up to 8 hextets) without intermediate string allocations.
Vectorized Loading: It operates directly on the binary representation prior to serialization, guaranteeing that IP masking remains a constant-time operation regardless of the ingestion volume.
4. Semantic Precompression Prior to LZ4
LZ4 is extremely fast at byte-level repetition matching, but it is unaware of structure semantics. Applying semantic transformation before handing data to LZ4 significantly improves the compression ratio without sacrificing processing speed.
Dictionary Tokenization: Highly repetitive string values (URIs, IPs) are converted to 32-bit integer handles (url_idx, ip_idx)
Entropy Reduction: By converting messy ASCII strings into dense, structured, binary arrays, the entropy of the byte stream drops dramatically, allowing LZ4 to find longer match sequences in its sliding window
Under adversarial conditions (1.3 million+ unique file paths and URLs crossed with 5,000+ unique IPv4 addresses), this pipeline sustains over 5.23M logs/sec written to physical disk, achieving a 2.6× compression ratio. On structured production workloads, throughput exceeds 20M logs/sec with 3.6×–5.0× compression on a standard development laptop.
Summary
Achieving tens of millions of events per second does not require specialized hardware or complex distributed clusters. By designing around modern CPU cache lines, eliminating dynamic allocations, removing inter-thread locks via thread-local state, and applying vectorization to data pre-processing, C11 on Windows can deliver deterministic, line-rate performance for high-throughput telemetry.
Source code, decoder, and reproducible benchmarks: github.com/superwired-labs/Pulp
Previous article (full benchmark methodology — 250M logs in 11.5s):
Benchmark
Top comments (0)