DEV Community

Cover image for Breaking the Memory Wall in .NET: The Buffer–Model Architecture
Lucifer
Lucifer

Posted on

Breaking the Memory Wall in .NET: The Buffer–Model Architecture

#dotnet #csharp #architecture #performance

The Invisible Performance Killer: The "Memory Wall"

Modern hardware has a secret: Your CPU is a Ferrari, but your RAM is a crowded city street. The gap between CPU cycles and Memory latency is the "Memory Wall". In high-throughput systems, the fastest code isn't the one with the best algorithm—it's the one that never lets the CPU wait for RAM.

Traditional Object-Oriented Programming (OOP) hides this reality behind layers of abstraction, leading to fragmented memory, constant Cache Misses, and exhausting Garbage Collection (GC). It’s time to move toward Data-Oriented Design (DOD).

1. The "Buffer": A Physical Masterpiece

The Buffer in LuciferCore isn't just a wrapper; it's a high-performance virtualization of raw byte[]. It is designed to be the "Matter" that stays contiguous and "hot" in the CPU Cache.

Core Technical Pillars:

  • Integrated Hybrid Pooling: Instead of renting and returning raw byte[] thousands of times—which incurs significant overhead—the Buffer stays "attached" to its memory. It leverages a custom Object Pool for the Buffer itself and the System.Buffers.ArrayPool for the underlying bytes. This double-pooling strategy eliminates the heavy cost of frequent array allocations.
  • Spatial Locality & SIMD: By keeping data in a flat array, we ensure Spatial Locality. We use SIMD-accelerated copies (Vectorized Span.CopyTo, FastCopy, and Unsafe memory manipulation) to move data at the theoretical limit of the hardware.
  • Static Cache & Thresholds: Each Thread maintains a Static Local Cache (32 Buffers / ~8MB). This specific size is tuned to fit within the L3 Cache, ensuring that "renting" a Buffer happens in nanoseconds without any lock contention.
  • Smart Trimming: Buffers exceeding the MaxRetainedCapacity (e.g., 256KB) are automatically evicted to prevent Cache Pollution, keeping only the most efficient "hot" data for the CPU.

2. The "Model": Stateless Rule Engines

The Model is the "Spirit" or the "Intelligence". Unlike traditional POCOs or Classes, the LuciferCore Model is Stateless.

How it Reinvents Logic:

  • Zero-Field Architecture: A Model has zero fields. It only contains Methods (Rules). This means 100,000 requests can technically be processed by a single "Model" instance because the machine code stays permanently in the Instruction Cache (I-Cache).
  • Non-Blocking Attachment: A Model is rented from its own Object Pool, "attaches" to a Buffer to interpret its bytes, and immediately releases the Buffer once the rule is applied. This ensures that Buffers are recycled instantly across different layers of the system.
  • Zero-Copy Views: By using Span<byte> and ReadOnlySpan<byte>, the Model provides a "view" into the Buffer. It never copies data. If you need a String or an Integer, the Model reads it directly from the raw bytes using Bit-manipulation and MemoryMarshal.

3. Why This Is the "Final Tier" of Optimization

When you combine these two, you get a system where Data is a Stream and Logic is a Lens.

  • Zero GC Pressure: Since everything—the Buffer and the Model—is pooled and recycled, the Garbage Collector has nothing to do. Your RAM usage stays as a flat line, even under millions of requests per second.
  • Mechanical Sympathy: You aren't just writing code; you are choreographing electrons. The Buffer stays in the Data Cache, the Model stays in the Instruction Cache. The CPU execution pipeline never stalls.

Implementation Blueprint (LuciferCore Snippets)

The Lock-Free Pool Mechanism

We use [ThreadStatic] to achieve zero-lock contention for the most frequent operations.

using System.Collections.Concurrent;
using System.Runtime.CompilerServices;

namespace LuciferCore.Pool;

public static class Pool<T> where T : PooledObject, new()
{
    [ThreadStatic]
    private static Stack<T>? _localStack;

    private static readonly ConcurrentStack<T> _globalStack = new();

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public static T Rent()
    {
        _localStack ??= new Stack<T>(32);

        if (_localStack.TryPop(out var obj))
        {
            obj.SetRefCount(1);
            return obj;
        }

        if (_globalStack.TryPop(out var globalObj))
        {
            globalObj.SetRefCount(1);
            return globalObj;
        }

        var newObj = new T();
        newObj.SetRefCount(1);
        return newObj;
    }

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public static void Return(T obj)
    {
        if (obj == null) return;
        if (obj.DecrementRef() != 0) return;

        obj.Reset();

        _localStack ??= new Stack<T>(32);

        if (_localStack.Count < 32)
        {
            _localStack.Push(obj);
        }
        else
        {
            _globalStack.Push(obj);
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

The High-Performance Buffer

Notice the use of MethodImplOptions.AggressiveInlining and SIMD optimization.

using System.Buffers.Text;
using System.Diagnostics;
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
using System.Text;
using LuciferCore.Core;
using LuciferCore.Main;
using LuciferCore.Pool;

namespace LuciferCore.Storage;

public class Buffer : PooledObject, IDisposable
{
    #region Constructors
    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public Buffer() => Attach();

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public Buffer(long capacity) => Attach(capacity);

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public Buffer(byte[] data) => Attach(data);

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public Buffer(byte[] data, long offset) => Attach(data, offset);

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public Buffer(byte[] data, long size, long offset) => Attach(data, size, offset);

    #endregion

    #region Private fields and constants

    private byte[] _data = Array.Empty<byte>();
    private long _size;
    private long _offset;

    public const long MaxBufferCapacity = int.MaxValue; // 2,147,483,647 bytes

    private const int DefaultCapacity = 256;
    private const int MaxRetainedCapacity = 1024 * 256;

    #endregion

    #region Public properties

    public bool IsEmpty => _data == null || _size == 0;
    public bool IsValid => _data != null && _size >= 0 && _offset >= 0 && _offset <= _size && _size <= _data.Length;

    public byte[] Data => _data;
    public long Capacity => _data.Length;
    public long Size => _size;
    public long Offset => _offset;
    public static long MaxSupportedCapacity => MaxBufferCapacity;

    public byte this[long index]
    {
        get
        {
            ValidateIndex(index, nameof(index));
            return _data[index];
        }
    }

    public byte[] this[Range range]
    {
        get
        {
            var (offset, length) = range.GetOffsetAndLength(_data.Length);

            ValidateRange(offset, length);
            return _data.AsSpan(offset, length).ToArray();
        }
    }

    #endregion

    #region Memory buffer methods
    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public Span<byte> AsSpan()
    {
        Debug.Assert(_size - _offset <= int.MaxValue, "Span size exceeds int.MaxValue");
        return new Span<byte>(_data, SafeToInt32(_offset), SafeToInt32(_size - _offset));
    }

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public Span<byte> AsSpan(long start, long length)
    {
        Debug.Assert(start >= 0 && length >= 0, "Invalid slice parameters");
        Debug.Assert(start + length <= _size, "Invalid slice range!");

        if (start < 0 || length < 0)
            throw new ArgumentOutOfRangeException("Start and length must be non-negative");

        if (start + length > _size)
            throw new ArgumentException("Invalid slice range!", nameof(start));

        return new Span<byte>(_data, SafeToInt32(start), SafeToInt32(length));
    }

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public Memory<byte> AsMemory()
    {
        var start = SafeToInt32(_offset);
        var length = SafeToInt32(_size - _offset);

        return new Memory<byte>(_data, start, length);
    }

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public Memory<byte> AsMemory(long start, long length)
    {
        return new Memory<byte>(_data, SafeToInt32(start), SafeToInt32(length));
    }

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public ArraySegment<byte> AsSegment()
    {
        return new ArraySegment<byte>(_data, SafeToInt32(_offset), SafeToInt32(_size - _offset));
    }

    public ReadOnlySpan<byte> Slice(long start, long length)
    {
        Debug.Assert(start >= 0 && length >= 0, "Invalid slice parameters");
        Debug.Assert(start + length <= _size, "Invalid slice range!");

        if (start < 0 || length < 0)
            throw new ArgumentOutOfRangeException("Start and length must be non-negative");

        if (start + length > _size)
            throw new ArgumentException("Invalid slice range!", nameof(start));

        return new ReadOnlySpan<byte>(_data, SafeToInt32(start), SafeToInt32(length));
    }

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public override string ToString() => ExtractString(0, SafeToInt32(_size));

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public string ExtractString(long offset, long size)
    {
        Debug.Assert(offset >= 0 && size >= 0, "Invalid parameters");
        Debug.Assert(offset + size <= _size, "Invalid offset & size!");

        if (offset < 0 || size < 0)
            throw new ArgumentOutOfRangeException("Offset and size must be non-negative");

        if (offset + size > _size)
            throw new ArgumentException("Invalid offset & size!", nameof(offset));

        return Encoding.UTF8.GetString(_data, SafeToInt32(offset), SafeToInt32(size));
    }

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public void Remove(long offset, long size)
    {
        Debug.Assert(offset >= 0 && size >= 0, "Invalid parameters");
        Debug.Assert(offset + size <= _size, "Invalid offset & size!");

        if (offset < 0 || size < 0)
            throw new ArgumentOutOfRangeException("Offset and size must be non-negative");

        if (offset + size > _size)
            throw new ArgumentException("Invalid offset & size!", nameof(offset));

        var remaining = _size - offset - size;
        if (remaining > 0)
        {
            // Same array, overlapping → use safe Copy()
            FastCopy.Copy(
                _data.AsSpan(SafeToInt32(offset + size), SafeToInt32(remaining)),
                _data.AsSpan(SafeToInt32(offset))
            );
        }

        _size -= size;

        if (_offset >= offset + size)
            _offset -= size;
        else if (_offset >= offset)
            _offset = offset;
    }


    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public void Reserve(long capacity)
    {
        ValidateCapacity(capacity, nameof(capacity));

        if (capacity > Capacity)
        {
            var newCapacity = Math.Max(capacity, Math.Min(2L * Capacity, MaxBufferCapacity));
            var newData = Lucifer.Rent(SafeToInt32(newCapacity));

            // Copy all used data [0, _size)
            if (_size > 0)
            {
                // Different arrays → no overlap
                FastCopy.CopySimd(
                    _data.AsSpan(0, SafeToInt32(_size)),
                    newData.AsSpan(0, SafeToInt32(_size))
                );
            }

            if (_data != null && _data.Length != 0)
                Lucifer.Return(_data);

            _data = newData;
        }
    }

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public void Resize(long size)
    {
        ValidateCapacity(size, nameof(size));

        Reserve(size);
        _size = size;

        if (_offset > _size)
            _offset = _size;
    }

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public void Shift(long offset)
    {
        var newOffset = _offset + offset;

        Debug.Assert(newOffset >= 0 && newOffset <= _size, "Invalid shift");

        if (newOffset < 0 || newOffset > _size)
            throw new ArgumentOutOfRangeException(nameof(offset),
                $"Shift would move offset to {newOffset}, valid range is [0, {_size}]");

        _offset = newOffset;
    }

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public void Unshift(long offset)
    {
        var newOffset = _offset - offset;

        Debug.Assert(newOffset >= 0, "Unshift would create negative offset");

        if (newOffset < 0)
            throw new ArgumentOutOfRangeException(nameof(offset),
                "Unshift would create negative offset");

        _offset = newOffset;
    }

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    protected internal override void Reset()
    {
        if (_data != null && _data.Length > MaxRetainedCapacity)
        {
            Detach();
            _data = Lucifer.Rent(DefaultCapacity);
        }

        _size = 0;
        _offset = 0;
    }

    public void Dispose()
    {
        Lucifer.Return(this);
    }

    #endregion

    #region Attach/Detach methods

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    private void Detach()
    {
        if (_data != null && _data.Length != 0)
            Lucifer.Return(_data);

        _data = Array.Empty<byte>();
        _size = 0;
        _offset = 0;
    }


    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public void Attach() { Detach(); }

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public void Attach(long capacity)
    {
        ValidateCapacity(capacity, nameof(capacity));
        Detach();
        _data = Lucifer.Rent(SafeToInt32(capacity));
        _size = 0;
        _offset = 0;
    }

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public void Attach(byte[] buffer)
    {
        if (buffer == null)
            throw new ArgumentNullException(nameof(buffer));

        Detach();
        _data = buffer;
        _size = buffer.Length;
        _offset = 0;
    }

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public void Attach(byte[] buffer, long offset)
    {
        if (buffer == null)
            throw new ArgumentNullException(nameof(buffer));

        ValidateOffset(offset, buffer.Length, nameof(offset));

        Detach();
        _data = buffer;
        _size = buffer.Length;
        _offset = offset;
    }

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public void Attach(byte[] buffer, long size, long offset)
    {
        if (buffer == null)
            throw new ArgumentNullException(nameof(buffer));

        ValidateSize(size, buffer.Length, nameof(size));
        ValidateOffset(offset, size, nameof(offset));

        Detach();
        _data = buffer;
        _size = size;
        _offset = offset;
    }

    #endregion

    #region Buffer I/O methods

    #region Append binary basic overloads
    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public long Append(byte[] buffer) => Append(buffer.AsSpan(0, buffer.Length));

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public long Append(byte[] buffer, long offset, long size) => Append(buffer.AsSpan(SafeToInt32(offset), SafeToInt32(size)));

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public long Append(Buffer buffer) => Append(buffer.AsSpan());

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public long Append(string text) => Append(text.AsSpan());

    #endregion

    #region Append binary number overloads
    ...
    #endregion

    #region Read binary number overloads
    ...
    #endregion

    #endregion    

    #region Validation helpers
    ...
    #endregion
}
Enter fullscreen mode Exit fullscreen mode

The Stateless Packet Model

The rule engine that attaches to the buffer without overhead.

using System.Buffers.Binary;
using System.Runtime.CompilerServices;
using System.Text.Json;
using LuciferCore.Main;
using LuciferCore.Pool;
using LuciferCore.Utf8;

namespace LuciferCore.Model;

/// <summary>Zero-copy packet wrapper with URL and body.</summary>
public class PacketModel : PooledObject
{
    /// <summary>Attached buffer for packet data.</summary>
    public Buffer? Buffer { get; private set; }

    /// <summary>Packet magic constant.</summary>
    public const int MAGIC = 0x4643554C;
    /// <summary>Default URL when invalid.</summary>
    public static ByteString UrlDefault = ByteString.CopyFromAscii("/v1/wss/Default"u8);
    /// <summary>Magic header length.</summary>
    public const int MagicLength = 4;
    /// <summary>URL length field size.</summary>
    public const int UrlLengthFieldSize = 4;
    /// <summary>URL offset in packet.</summary>
    public const int UrlOffset = MagicLength + UrlLengthFieldSize; // = 8

    /// <summary>Checks whether packet header is valid.</summary>
    public bool IsValid
    {
        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        get
        {
            if (Buffer == null || Buffer.Size < UrlOffset) return false;

            var span = Buffer.AsSpan();

            if (BinaryPrimitives.ReadInt32LittleEndian(span) != MAGIC) return false;

            var urlLen = BinaryPrimitives.ReadInt32LittleEndian(span.Slice(4));

            return urlLen >= 0 && (UrlOffset + urlLen <= Buffer.Size);
        }
    }

    /// <summary>Gets raw URL length from header.</summary>
    private int UrlLengthRaw
    {
        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        get
        {
            if (Buffer == null || Buffer.Size < UrlOffset) return 0;

            return BinaryPrimitives.ReadInt32LittleEndian(Buffer.Data.AsSpan((int)(Buffer.Offset + 4), 4));
        }
    }

    /// <summary>Gets URL as ByteString view.</summary>
    public ByteString UrlView
    {
        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        get
        {
            if (Buffer == null || !IsValid) return PacketModel.UrlDefault;

            var len = UrlLengthRaw;
            return len <= 0
                ? ByteString.Empty
                : new ByteString(Buffer.Data, (int)(Buffer.Offset + UrlOffset), len);
        }
    }

    /// <summary>Gets body span.</summary>
    public ReadOnlySpan<byte> Body
    {
        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        get
        {
            if (Buffer == null) return ReadOnlySpan<byte>.Empty;

            if (!IsValid) return Buffer.AsSpan();

            var urlLen = UrlLengthRaw;
            var bodyOffset = UrlOffset + urlLen;

            return Buffer.Data.AsSpan((int)(Buffer.Offset + bodyOffset), (int)(Buffer.Size - bodyOffset));
        }
    }

    /// <summary>Cached deserialized body.</summary>
    private object? _cachedBody;
    /// <summary>Whether body cache is set.</summary>
    private bool _bodyCached;

    /// <summary>Deserializes body as type T.</summary>
    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public T? BodyAs<T>() where T : class
    {
        if (_bodyCached && _cachedBody is T cached) return cached;

        var bodySpan = Body;
        if (bodySpan.IsEmpty) return null;

        var result = Lucifer.DeserializeFromBuffer<T>(bodySpan);
        _cachedBody = result!;
        _bodyCached = true;
        return result;
    }

    public static Buffer ToBufferWithMagic(byte[] buffer, long offset, long size, ByteString url)
        => ToBufferWithMagic(buffer.AsSpan((int)offset, (int)size), url);

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public static Buffer ToBufferWithMagic(ReadOnlySpan<char> buffer, ByteString url)
    {
        var buf = Lucifer.Rent<Buffer>();
        buf.Reset();

        // 1. MAGIC
        buf.Append(PacketModel.MAGIC);
        // 2. UrlLength
        buf.Append(url.Length);
        // 3. Url bytes
        buf.Append(url.AsSpan());
        // 4. Body bytes
        buf.Append(buffer);
        return buf;
    }

    /// <summary>Builds packet buffer from byte span.</summary>
    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public static Buffer ToBufferWithMagic(ReadOnlySpan<byte> buffer, ByteString url)
    {
        var buf = Lucifer.Rent<Buffer>();
        buf.Reset();

        // 1. MAGIC
        buf.Append(PacketModel.MAGIC);
        // 2. UrlLength
        buf.Append(url.Length);
        // 3. Url bytes
        buf.Append(url.AsSpan());
        // 4. Body bytes
        buf.Append(buffer);
        return buf;
    }

    /// <summary>Converts object to byte array.</summary>
    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public static byte[] GetBytes(object data)
        => data switch
        {
            byte[] bytes => bytes,
            null => Array.Empty<byte>(),
            _ => JsonSerializer.SerializeToUtf8Bytes(data)
        };

    /// <summary>Attaches buffer and clears cache.</summary>
    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public void Attach(Buffer newBuffer)
    {
        if (Buffer != null && Buffer != newBuffer) Lucifer.Return(Buffer);

        Buffer = newBuffer;
        _cachedBody = null;
        _bodyCached = false;
    }

    /// <summary>Returns buffer to pool and clears cache.</summary>
    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    protected internal override void Reset()
    {
        if (Buffer != null)
        {
            Lucifer.Return(Buffer);
            Buffer = null;
        }
        _cachedBody = null;
        _bodyCached = false;
    }
}
Enter fullscreen mode Exit fullscreen mode

Final Thoughts & Contribution

The Buffer-Model architecture is the core philosophy of LuciferCore. It bridges the gap between high-level C# productivity and low-level C++ performance. By moving from "Object-thinking" to "Data-thinking," we can build backends that are prepared for the next decade of hardware evolution.

Author & Copyright

This architecture and the LuciferCore framework are authored by Nguyễn Minh Thuận (thuangf45).

License Notice

The LuciferCore framework and all code snippets in this article are licensed under the MIT License (see GitHub repository for full text).

Copyright © 2025 Nguyễn Minh Thuận (thuangf45).

Feedback & Collaboration

This is my biggest passion project. I believe this DOD framework can become a new standard for high-performance .NET development. I am eager to hear your technical critiques, contributions, or suggestions to push this even further.

Thank you for reading and for your interest in extreme performance!

Top comments (3)

Collapse
 
thuangf45 profile image
Lucifer •

If you find this architecture interesting, you can explore LuciferCore directly on NuGet. It’s the framework I’m actively developing and optimizing based on the principles described here.
NuGet: nuget.org/packages/LuciferCore

Collapse
 
thuangf45 profile image
Lucifer •

If this article gains enough interest, I’m considering making the full LuciferCore repository public in the near future. I’d love to share the entire architecture once I know there’s a community that truly wants to explore it.

Collapse
 
thuangf45 profile image
Lucifer •

Thank you for taking the time to read this article. I genuinely appreciate every technical insight, critique, or perspective you share in the comments. I’m always eager to engage in thoughtful, academic-level discussions to refine and push this architecture even further.