ioxide, a .NET io_uring runtime

ioxide is a foundation or engine for building network servers, in C#. The closest existing class in dotnet is the System.Net.Socket, ioxide covers more area though, it provides protocol abstractions to build HTTP/2 and HTTP/3 servers by vendoring ngtcp2, nghttp2 and nghttp3 libraries. It doesn't provide HTTP/1.1 protocol implementation because this is typically fully implemented by the frameworks that use ioxide, currently SimpleW and GenHTTP, it also provides an extension to replace Kestrel transport, but it is being dropped/deprecated as the work stealing threadpool model is a bad fit for io_uring.

What is io_uring

A Linux kernel interface for asynchronous I/O via shared ring buffers, released in 2019 to fix AIO in high-throughput work.

Core Idea

Two lock-free ring buffers in memory shared between the process and the kernel, in ioxide these are memory mapped. Basically we write submissions(requests to read or write to a socket or a file for example) to the submission ring buffer, and collect the completed "requests" from the completion ring buffer, pushed by the kernel.
io_uring is completion based, epoll tells you "you can read now" and you still do the read yourself, io_uring tells you "the read is done, bytes are already available here", this is a core concept that is largely mentioned when introducing io_uring, it significantly reduces the syscall count when compared with epoll.

Why does ioxide exist, Can't we directly use io_uring in dotnet?

Yes, io_uring_setup, mmap, io_uring_enter are three syscalls, you can have a working ring in an afternoon. Then.. you hit the problem, a completion is not a continuation, ioxide bridges io_uring into the .NET async/await model seamlessly, providing productivity out of the box, hiding all the nitty gritty interop with this kernel interface.

How ioxide works

Thread per core model, one core, one worker, one queue.
A CPU has several cores, ioxides gives each one its own worker thread, ring and set of connections. They don't share anything, they don't talk to each other. When a request arrives one core takes it from start to finish, ideally never leaving the worker thread.

Why is this fast?
Sharing is caring.. and expensive, if two cores can touch the same connection, they have to take turns at it. Modern CPUs keep their own local copy of recently used data, this means that everytime work moves between cores, data has to be shuffled across.

io_uring's fastest mode comes with a promise, only one thread will ever submit to this ring. The kernel takes that promise and drops its internal locking in exchange.

That promise is the sticking point in .NET. The normal model is work stealing, when your code suspends, it resumes on whatever thread is free next, not the one it started on. So a read loop that finishes a read and immediately starts the next one would be submitting to the ring from the wrong thread. Promise broken.

So what happens when a continuation does get scheduled somewhere else?

Ideally, it never does. ioxide ships its own HTTP client, Postgres and Redis drivers, so ordinary I/O work rides the worker's own ring and the request never leaves the core it arrived on.

But nothing stops you from calling an existing .NET API that hands your work to the thread pool.. and that's fine. ioxide notices when a handler has drifted off its worker thread and routes anything touching the ring back to it. Correct, but not free, it costs a hop. Worth avoiding where you can.

One thing that isn't just slow, though, never block a worker thread. A .Result or a .Wait() inside a handler doesn't make that core slower, it stops it. The thread is sitting still waiting for work that only it can process, and it's the one thing standing in the way. That core is finished until you restart the process.

Building a basic TCP server with ioxide

Typicallt ioxide is used through other high level frameworks as mentioned at the start of this article, but.. it is very easy to wire a simple TCP or QUIC server with it. ioxide provides a very friendly API similar to PipeReader and PipeWriter, these already take care of issues like TCP fragmentation and buffer overflows. There are a lot of working examples at the repository playground.

using System.IO.Pipelines;
using ioxide;

var config = new ServerConfig
{
    ReactorCount = Environment.ProcessorCount,   // one ring + one thread per core
    Tcp = new TcpOptions { Port = 8080 },
    Udp = null,                                  // TCP only
};

byte[] response = "HTTP/1.1 200 OK\r\nContent-Length: 2\r\n\r\nok"u8.ToArray();

var threads = new Thread[config.ReactorCount];

for (int i = 0; i < threads.Length; i++)
{
    var reactor = new Reactor(i, config);

    reactor.TcpHandle = async (r, conn) =>
    {
        var reader = new TcpConnectionPipeReader(conn);
        var writer = new TcpConnectionPipeWriter(conn);

        try
        {
            while (true)
            {
                ReadResult result = await reader.ReadAsync();

                reader.AdvanceTo(result.Buffer.End);   // no parsing: consume everything

                response.CopyTo(writer.GetSpan(response.Length));
                writer.Advance(response.Length);
                await writer.FlushAsync();

                if (result.IsCompleted) return;
            }
        }
        finally
        {
            reader.Complete();
            conn.DecRef();
        }
    };

    threads[i] = new Thread(reactor.Run) { Name = $"reactor-{i}" };
    threads[i].Start();
}

foreach (Thread t in threads) t.Join();

The write part can be simplified, TcpConnectionPipeWriter does not override Write, so it inherits PipeWriter's base implementation, which does exactly what we show above.

We can replace

response.CopyTo(writer.GetSpan(response.Length));
writer.Advance(response.Length);

with

writer.Write(response);

or even further, replace

response.CopyTo(writer.GetSpan(response.Length));
writer.Advance(response.Length);
await writer.FlushAsync();

with

await writer.WriteAsync(response);

TLS is widely supported across ioxide, while QUIC transport already has it, it can be added to a TCP connection.

ioxide supports SslStream or directly using OpenSsl with Kernel TLS both TX and RX as opt ins for the highest performance possible.

Performance and Benchmarks

So, let's see the numbers.

Let's compare ioxide with Rust's most famous counterpart, tokio.

The results are taken from HttpArena engine section, filtering ioxide and tokio entries.

There is no entry for Json-TLS test on tokio so the composite score favours ioxide, but let's head to head compare the results on each test that both engines cover.

The tests

Baseline - 4096 Keep-Alive connections saturation test to achieve as many requests per second as possible.

Short-Lived - Connection churn test, each TCP connection closes after 10 requests.

Latency-10k - Fixed throughput at 10K requests per second, measures CPU usage and tail latencies at close to idle state.

Latency-1M - Fixed throughput at 1 Million requests per second, measures CPU usage and tail latencies at decent load.

Async Delay - 32k connections with a 10ms delay on each request, tests the asynchronous model.

Baseline

Short Lived

Latency-10k

Latency-1M

Async Delay

Overall ioxide and tokio performances are very close with somewhat adavantage for ioxide on P99.9 tail latencies. This measurement is arguable since the benchmark duration might not have been long enough to give us comparable values.

Another interesting comparison is for GenHTTP using ioxide vs GenHTTP using System.Net.Socket and kestrel.

By switching the udnerlying engine and keeping the framework exactly the same, GenHTTP performance numbers improve over 50% extra performance. Other relevant metric is the fact that with ioxide, GenHTTP consumes 1/5 of the CPU when close to idle load.