Your Struct Layout Is Part of Your Wire Protocol

The problem

A message-processing system that comfortably handled 50,000 messages/second fell off a cliff one night: 8,000 msg/sec and collapsing. The confusing part was the dashboards. Network: green. Database: barely working. CPU utilization: 23%.

Nothing looked busy, and yet nothing was moving. Three weeks of profiling later, the culprit turned out not to live in any code path. It lived in a struct definition:

struct PaymentMessage {
    struct MessageHeader header;   // 28 bytes of mixed-size fields
    char     sender_id[16];
    char     recipient_id[16];
    uint64_t amount;
    char     currency[4];
    char     reference[32];
    uint8_t  flags;
} __attribute__((packed));         // 101 bytes. Practically always misaligned.

101 packed bytes means nearly every message straddled two cache lines — sometimes three.

Why it happens

CPUs don't read bytes. They read cache lines: 64-byte blocks. Ask for one byte and the CPU pulls in that byte's 63 neighbors whether you need them or not.

Most of us design binary protocols as if the struct exists in isolation — fields ordered by meaning, packed to save bytes on the wire. But in memory, that struct lands wherever the allocator put it, and the fast path (validate → route → count) reads fields scattered across the whole layout: magic at the front, checksum at the back, timestamp in the middle.

At tens of millions of field accesses per second, every extra cache line touched is an extra trip to memory. An L1 hit costs about a nanosecond; a main-memory fetch costs about a hundred. Multiply "2–3 lines per message instead of 1" by 50K messages/sec and the pipeline spends its life stalled on the memory bus while the cores look eerily idle.

What to do about it

1. Split hot from cold. Everything the fast path reads goes in the first 64 bytes, aligned to a line boundary:

struct CacheOptimizedMessage {
    // Hot: validation + routing — first 32 bytes
    uint32_t magic;
    uint32_t message_type;     // widened for alignment
    uint64_t timestamp;
    uint64_t amount;
    uint64_t sequence_id;

    // Warm: next 32 bytes
    char     sender_id[12];    // compacted
    char     recipient_id[12];
    uint32_t checksum;
    uint32_t payload_offset;   // variable data lives elsewhere
} __attribute__((aligned(64)));   // exactly one cache line

The fast path now costs exactly one cache fetch: validate, route, and update without ever reaching past the first line.

2. Move variable-length data behind an offset. Flexible fields get their own 64-byte-aligned section with a small offset table. One extra indirection when you need them — zero cost when you don't (which is most of the time).

3. Batch and prefetch. Once messages are line-aligned, batches become predictable, and the prefetcher becomes your friend:

for (size_t i = 0; i < count; i++) {
    if (i + 1 < count)
        __builtin_prefetch(&messages[i + 1], 0, 3);
    process_message_fast_path(&messages[i]);
}

The rework took the system from 14,700 to 50,100 msg/sec — 3.4× — on the same hardware, same algorithms. P50 latency went from 89ms to 12ms; P99 from 456ms to 34ms. Cache misses dropped 78%.

When to bother

Above ~10K messages/sec with CPU-bound processing: this is one of the highest-leverage changes available to you. Between 1K and 10K: measure first. Below 1K, or if you're IO-bound, or your messages vary wildly in size: skip it — you're trading real development velocity for latency you won't notice.

Key takeaways

• CPUs fetch 64-byte cache lines, not fields. A struct that straddles lines multiplies memory traffic on every access.
• Put every field your fast path reads into the first cache line; push cold metadata to the next.
• aligned(64) guarantees your hot line starts at a line boundary — packed-and-misaligned undoes everything.
• Variable-length data goes behind an offset, fetched only when needed.
• Verify with perf stat (cache-misses, instructions-per-cycle) before and after. The numbers don't lie; intuition does.

---

This is a condensed version of a longer war story I published on Medium — the full piece walks through the L1/L2 layering, platform portability, and the batch buffer design: Binary Protocols: Designing Messages For Cache Lines