Building a Lock-Free Single-Producer Single-Consumer (SPSC) Ring Buffer in C++20: Memory Orders and False Sharing Elimination

In high-performance concurrent systems, real-time audio processing pipelines, and low-latency financial order routers, passing messages between threads using standard mutex-protected queues (std::queue wrapped in std::mutex) introduces severe performance penalties. Mutex locks force kernel context switches, thread blocking, and unpredictable latency spikes.

To achieve lock-free inter-thread communication with nanosecond-level determinism, we must engineer a Single-Producer Single-Consumer (SPSC) Ring Buffer using C++20 atomic memory order semantics and strict cache-line alignment.

1. The Limitations of Mutexes in Concurrent Queues When a producer thread and a consumer thread contend for a mutex-protected queue:

Kernel Context Switches: If a thread fails to acquire a lock, the operating system suspends it, triggering an expensive transition between user space and kernel space.

Cache Coherency Invalidation: Mutex locks and condition variables constantly invalidate CPU cache lines across different cores, causing high interconnect traffic.

Unpredictable Latencies: P99/P999 tail latencies blow out because of scheduling jitter and lock contention.

Lock-Free SPSC Solution: By guaranteeing that exactly one thread produces items and exactly one thread consumes them, we can eliminate locks entirely, replacing them with atomic indices and memory barriers.

1. Atomic Memory Orders (acquire-release Semantics) In modern multi-core processors, CPU out-of-order execution and compiler reordering can cause memory writes to become visible in unpredictable sequences. We enforce strict visibility rules using C++20 atomic memory orders:

std::memory_order_relaxed: No synchronization or ordering constraints; only atomic atomicity.

std::memory_order_acquire: Ensures no reads or writes in the current thread can be reordered before this load.

std::memory_order_release: Ensures no reads or writes in the current thread can be reordered after this store.

1. Production-Ready C++20 SPSC Ring Buffer Core Here is a complete, production-ready C++20 implementation of a lock-free SPSC ring buffer featuring cache-line padding to prevent false sharing and power-of-two bitmask indexing:

C++

include

include

include

include

include

include

template
requires (Capacity > 0 && (Capacity & (Capacity - 1)) == 0) // Capacity must be power of 2
class SPSCRingBuffer {
private:
// Pad cache lines independently to eliminate false sharing between producer and consumer cores
alignas(64) std::atomic head_{0};
alignas(64) std::atomic tail_{0};

alignas(64) T buffer_[Capacity];

public:
SPSCRingBuffer() = default;
~SPSCRingBuffer() = default;

// Non-copyable, non-movable
SPSCRingBuffer(const SPSCRingBuffer&) = delete;
SPSCRingBuffer& operator=(const SPSCRingBuffer&) = delete;

bool push(const T& item) noexcept {
    const size_t current_head = head_.load(std::memory_order_relaxed);
    const size_t current_tail = tail_.load(std::memory_order_acquire);

    // Check if ring buffer is full (Capacity mask optimization)
    if ((current_head - current_tail) >= Capacity) {
        return false;
    }

    buffer_[current_head & (Capacity - 1)] = item;

    // Release memory barrier: ensures item is written before head index is updated
    head_.store(current_head + 1, std::memory_order_release);
    return true;
}

bool pop(T& item) noexcept {
    const size_t current_tail = tail_.load(std::memory_order_relaxed);
    const size_t current_head = head_.load(std::memory_order_acquire);

    // Check if ring buffer is empty
    if (current_tail == current_head) {
        return false;
    }

    item = buffer_[current_tail & (Capacity - 1)];

    // Release memory barrier: ensures item is read before tail index is updated
    tail_.store(current_tail + 1, std::memory_order_release);
    return true;
}

bool empty() const noexcept {
    return head_.load(std::memory_order_relaxed) == tail_.load(std::memory_order_relaxed);
}

size_t size() const noexcept {
    size_t head = head_.load(std::memory_order_relaxed);
    size_t tail = tail_.load(std::memory_order_relaxed);
    return (head >= tail) ? (head - tail) : (Capacity - (tail - head));
}

};

// =========================================================
// Example Execution Driver
// =========================================================
int main() {
constexpr size_t QueueCapacity = 1024; // Must be power of 2
SPSCRingBuffer ring_buffer;

std::atomic<bool> producer_done{false};
uint64_t total_items_to_send = 100'000;

std::cout << "Starting Lock-Free SPSC Ring Buffer Benchmark...\n";

// Producer Thread
std::thread producer([&]() {
    for (uint64_t i = 1; i <= total_items_to_send; ++i) {
        while (!ring_buffer.push(i)) {
            std::this_thread::yield(); // Backoff if buffer is full
        }
    }
    producer_done.store(true, std::memory_order_release);
});

// Consumer Thread
std::thread consumer([&]() {
    uint64_t received_count = 0;
    uint64_t val = 0;
    while (!producer_done.load(std::memory_order_acquire) || !ring_buffer.empty()) {
        if (ring_buffer.pop(val)) {
            received_count++;
        } else {
            std::this_thread::yield();
        }
    }
    std::cout << "Consumer successfully received " << received_count << " items with zero locks.\n";
});

producer.join();
consumer.join();

std::cout << "Lock-Free SPSC Queue concurrency test completed successfully.\n";
return 0;

}

1. Conclusion & Performance Impact By migrating from mutex-protected queues to a Lock-Free SPSC Ring Buffer in C++20:

Zero Mutex Contention: Elimination of kernel locks drops push/pop latencies down to single-digit nanoseconds.

False Sharing Prevention: alignas(64) padding on atomic indices prevents cache-line bouncing between producer and consumer CPU cores.

Deterministic Throughput: Powers high-speed inter-thread messaging engines where predictable, low-latency execution is non-negotiable.(https://www.blogger.com/blog/post/edit/preview/4750431103251998125/6904450898359926599)