DEV Community

Cover image for System.Threading.Channels in .NET 10, Measured: The Write Said True and the Item Was Gone
Juan Gómez
Juan Gómez

Posted on

System.Threading.Channels in .NET 10, Measured: The Write Said True and the Item Was Gone

System.Threading.Channels in .NET 10, Measured: The Write Said True and the Item Was Gone

When a producer is faster than its consumer, something has to give. The queue between them can grow without limit, the producer can be made to wait, or items can be thrown away. System.Threading.Channels makes you pick one of those, which is exactly what it should do. What surprised me is how two of the choices report it: a bounded channel in a drop mode answers true to the write of an item it has just thrown away.

I measured an unbounded channel and all four modes of a bounded one under the same load on .NET 10, then broke a three-stage pipeline in the middle to see who notices. Short answer: nobody, unless every stage is wired to tell the others.

Every example below is a .NET 10 file-based app: save the block as a .cs file and run it with dotnet run -c Release <file>.cs, with no project file. The numbers come from .NET 10.0.15 on an 8-core laptop. All of them, plus the pipeline variants the text describes without printing, are in dotnet-10-channels on GitHub, with a script that checks each behaviour.

A robotic arm places a cube into a three-slot tray that is already full, under a green checkmark, while a cube falls off the far end of the tray into a bin

A channel in one paragraph

A channel is a thread-safe queue with two ends: a ChannelWriter<T> that producers write to and a ChannelReader<T> that consumers read from. Both ends are async. A reader on an empty channel awaits without holding a thread, and so does a writer on a full one. Channel.CreateUnbounded<T>() never fills up. Channel.CreateBounded<T>(capacity) does, and its FullMode decides what happens then. When the producer is done it calls Writer.Complete(), and ReadAllAsync() ends once the channel is complete and empty. It has shipped with .NET since .NET Core 3.0, so there is no package to install.

Five writes into a channel that holds three

The quickest way to see the four full modes is to write five items into a channel of three and read back what is left. itemDropped is an optional callback, added in .NET 6, that the channel calls with every item it throws away:

using System.Threading.Channels;

// Five writes into a channel that holds three, once per full mode.
foreach (BoundedChannelFullMode mode in Enum.GetValues<BoundedChannelFullMode>())
{
    List<int> dropped = [];
    Channel<int> channel = Channel.CreateBounded<int>(
        new BoundedChannelOptions(capacity: 3) { FullMode = mode },
        itemDropped: dropped.Add);

    List<bool> accepted = [];
    for (int item = 1; item <= 5; item++)
        accepted.Add(channel.Writer.TryWrite(item));
    channel.Writer.Complete();

    List<int> read = [];
    await foreach (int item in channel.Reader.ReadAllAsync())
        read.Add(item);

    Console.WriteLine($"{mode,-10}  TryWrite: {string.Join(" ", accepted),-26}  read: [{string.Join(", ", read)}]  itemDropped: [{string.Join(", ", dropped)}]");
}
Enter fullscreen mode Exit fullscreen mode
Wait        TryWrite: True True True False False  read: [1, 2, 3]  itemDropped: []
DropNewest  TryWrite: True True True True True    read: [1, 2, 5]  itemDropped: [3, 4]
DropOldest  TryWrite: True True True True True    read: [3, 4, 5]  itemDropped: [1, 2]
DropWrite   TryWrite: True True True True True    read: [1, 2, 3]  itemDropped: [4, 5]
Enter fullscreen mode Exit fullscreen mode

Three things in those four lines are worth a second look.

In the three drop modes, every write returned True. Two items out of five are gone in each, and the writer was told it succeeded. WriteAsync behaves the same way: in the load test below, all 20,000 calls completed normally, including the thousands whose item was dropped. The itemDropped callback is the only place the channel tells you, and it is easy to miss, because it is an optional parameter on a factory method. It also runs on the writer's thread, during the write and outside the channel's lock, so with several producers it can run on several threads at once. Count in it with something like the Interlocked.Increment in the load test below; the List in this example is safe only because one thread does all the writing. If that sounds familiar, it is the same lesson as channel.publish returning true in RabbitMQ from Fundamentals to Production, Part 2: Building a Scalable Service.

DropNewest does not drop the item you are writing. It drops the newest item already in the channel to make room for it: writing 4 evicted 3, and writing 5 evicted 4. The channel keeps the oldest items plus the very last one. DropWrite is the mode that refuses the incoming item.

In Wait mode, TryWrite does not wait. It returns False when the channel is full, and only WriteAsync waits for space. A producer that calls TryWrite and ignores the result loses items in the one mode whose name promises it never will.

The same choice under load

Five items show what each mode does, not what it costs. For that I simulated the sensor gateway of a refrigerated truck: 20,000 temperature readings, each received in 20 µs and stored in 200 µs, so the producer is ten times faster than the consumer. The capacity is 1,000. Both costs are a busy-wait, a stand-in for real work with a steady, known cost, so that the channel is the only thing that changes between rows:

using System.Diagnostics;
using System.Threading.Channels;

const int Total = 20_000;
const int Capacity = 1_000;
TimeSpan receiveCost = TimeSpan.FromMicroseconds(20);   // reading one message off the socket
TimeSpan storeCost = TimeSpan.FromMicroseconds(200);    // inserting it into the database

Console.WriteLine($"{Total:N0} readings, received in {receiveCost.TotalMicroseconds} µs and stored in {storeCost.TotalMicroseconds} µs each, capacity {Capacity:N0}");
Console.WriteLine($"{"mode",-10} {"producer",8} {"total",7} {"stored",8} {"dropped",8} {"median wait",12} {"max wait",9}");

await MeasureAsync("Unbounded",
    Channel.CreateUnbounded<Reading>(new UnboundedChannelOptions { SingleReader = true }),
    dropped: () => 0);

foreach (BoundedChannelFullMode mode in Enum.GetValues<BoundedChannelFullMode>())
{
    long dropped = 0;
    Channel<Reading> channel = Channel.CreateBounded<Reading>(
        new BoundedChannelOptions(Capacity) { FullMode = mode },
        itemDropped: _ => Interlocked.Increment(ref dropped));
    await MeasureAsync(mode.ToString(), channel, dropped: () => Interlocked.Read(ref dropped));
}

async Task MeasureAsync(string mode, Channel<Reading> channel, Func<long> dropped)
{
    int stored = 0;
    TimeSpan[] waits = new TimeSpan[Total];   // from receiving each stored reading to storing it
    Stopwatch clock = Stopwatch.StartNew();

    Task consumer = Task.Run(async () =>
    {
        await foreach (Reading reading in channel.Reader.ReadAllAsync())
        {
            Work(storeCost);
            waits[stored++] = Stopwatch.GetElapsedTime(reading.ReceivedAt);
        }
    });

    for (long sequence = 0; sequence < Total; sequence++)
    {
        Work(receiveCost);
        await channel.Writer.WriteAsync(new Reading(sequence, "truck-17", 4.2, Stopwatch.GetTimestamp()));
    }
    TimeSpan producer = clock.Elapsed;
    channel.Writer.Complete();
    await consumer;   // after this, the counters the consumer wrote are safe to read here
    TimeSpan total = clock.Elapsed;

    Array.Sort(waits, 0, stored);
    Console.WriteLine($"{mode,-10} {producer.TotalSeconds,7:F2}s {total.TotalSeconds,6:F2}s {stored,8:N0} {dropped(),8:N0} {waits[stored / 2].TotalMilliseconds,9:N1} ms {waits[stored - 1].TotalMilliseconds,6:N0} ms");
}

// Spins for a fixed time: a stand-in for real work with a steady, known cost.
static void Work(TimeSpan duration)
{
    long until = Stopwatch.GetTimestamp() + (long)(duration.TotalSeconds * Stopwatch.Frequency);
    while (Stopwatch.GetTimestamp() < until) { }
}

record Reading(long Sequence, string SensorId, double Celsius, long ReceivedAt);
Enter fullscreen mode Exit fullscreen mode

SingleReader = true tells the unbounded channel that only one read will ever be in progress at a time, which a single await foreach loop guarantees even when its awaits resume on different threads. It lets CreateUnbounded pick a faster implementation, and nothing checks that it is true, so set it only when it is. The bounded channel has no such variant in .NET 10: CreateBounded ignores both SingleReader and SingleWriter, so I left them off.

20,000 readings, received in 20 µs and stored in 200 µs each, capacity 1,000
mode       producer   total   stored  dropped  median wait  max wait
Unbounded     0.42s   4.04s   20,000        0   1,813.4 ms  3,621 ms
Wait          3.86s   4.06s   20,000        0     203.3 ms    204 ms
DropNewest    0.41s   0.61s    3,018   16,982     200.4 ms    202 ms
DropOldest    0.40s   0.60s    3,012   16,988      20.4 ms    201 ms
DropWrite     0.40s   0.60s    3,012   16,988     200.6 ms    201 ms
Enter fullscreen mode Exit fullscreen mode

That is one run of three. Another matched it on every row, to within 0.02 seconds and 25 stored readings. In the third, something else on the laptop slowed the consumer during the drop modes: they stored between 1,673 and 2,720 readings instead of about 3,000, and their median waits went up, though DropOldest's was still the lowest (53 ms against 236 and 508). The times move between runs; what each mode does with the readings does not.

  • Unbounded lost nothing and never slowed the producer. It moved the problem into memory instead. When the burst ended after 0.42 seconds, the consumer had stored fewer than 2,500 of the 20,000 readings, and the rest sat in the queue: a median wait of 1.8 seconds each. Under a burst that does not end, that queue grows until the process runs out of memory.
  • Wait lost nothing and made the producer pay. It took about 4 seconds instead of 0.4, the pace of the consumer. Each reading waited a median of 203 ms, which is the capacity times the cost of storing one reading: 1,000 × 200 µs. That is the latency you choose when you choose the capacity.
  • The three drop modes kept the producer at full speed and threw away 85% of the readings. Each stored about 3,000: the 2,000 or so the consumer could store in 0.4 seconds at 200 µs each, plus the 1,000 left in the buffer.
  • Only DropOldest kept the consumer on fresh data. Its median wait was 20 ms, against about 200 ms for the other two. Its buffer always holds the last 1,000 readings to arrive, and one arrives every 20 µs, so the oldest of them, the one the consumer takes next, arrived 1,000 × 20 µs = 20 ms earlier. In DropNewest and DropWrite, a reading gets in only when the consumer frees a slot, and then waits for the 1,000 ahead of it to be stored: 1,000 × 200 µs, 200 ms. The maximum is about 200 ms in all three because it measures the buffer draining after the producer stopped.

So the right mode depends on the data, not on the speed:

  • Wait for anything you cannot lose: orders, payments, messages you will acknowledge to a broker. The producer slows down, and whatever feeds it has to slow down too.
  • DropOldest for state where the latest value replaces the earlier ones, such as a temperature or a position, as long as the channel carries a single source. A channel has no keys: with a whole fleet in one channel, a quiet truck's only reading is often the oldest one, and it is the one that goes. For many sources, keep a channel per source, or a dictionary with the latest value for each.
  • DropWrite for best-effort data where any sample will do, such as analytics events. It keeps what is already queued and refuses what is new. DropNewest behaved almost the same under load; the only difference is which item goes, the incoming one or the last one queued.
  • In any drop mode, count itemDropped and export it as a metric. A reading that was dropped and never counted looks exactly like one that was never sent.

Four trays already holding three numbered cubes, each with a new cube arriving: the first shut by a gate with an hourglass, one dropping its oldest cube out of the far end, and one knocking the incoming cube away

A pipeline where one stage throws

Channels are at their best connecting stages: each stage reads from one channel and writes to the next. Here is the gateway as three stages, receive → parse → store, with a bounded channel between each pair. At reading 5,000, the probe on truck 17 disconnects and the gateway sends N/A instead of a temperature:

using System.Globalization;
using System.Threading.Channels;

// receive → parse → store, with a bounded channel between each pair of stages.
Channel<string> lines = Channel.CreateBounded<string>(100);
Channel<Reading> readings = Channel.CreateBounded<Reading>(100);
int received = 0, parsed = 0, stored = 0;

Task receive = Task.Run(async () =>
{
    for (int sequence = 0; sequence < 10_000; sequence++)
    {
        // Truck 17's probe disconnects once, and the gateway sends N/A instead of a temperature.
        string line = sequence == 5_000 ? $"{sequence};truck-17;N/A" : $"{sequence};truck-17;4.2";
        await lines.Writer.WriteAsync(line);
        Interlocked.Increment(ref received);
    }
    lines.Writer.Complete();
});

Task parse = Task.Run(async () =>
{
    await foreach (string line in lines.Reader.ReadAllAsync())
    {
        string[] fields = line.Split(';');
        Reading reading = new(long.Parse(fields[0], CultureInfo.InvariantCulture), fields[1], double.Parse(fields[2], NumberStyles.Float, CultureInfo.InvariantCulture));
        await readings.Writer.WriteAsync(reading);
        Interlocked.Increment(ref parsed);
    }
    readings.Writer.Complete();
});

Task store = Task.Run(async () =>
{
    await foreach (Reading reading in readings.Reader.ReadAllAsync())
        Interlocked.Increment(ref stored);
});

Task pipeline = Task.WhenAll(receive, parse, store);
if (await Task.WhenAny(pipeline, Task.Delay(TimeSpan.FromSeconds(10))) != pipeline)
    Console.WriteLine($"Still running after 10 s. receive: {receive.Status}, parse: {parse.Status}, store: {store.Status}");
Console.WriteLine($"received {Volatile.Read(ref received):N0}, parsed {Volatile.Read(ref parsed):N0}, stored {Volatile.Read(ref stored):N0}, waiting in lines: {lines.Reader.Count}");

record Reading(long Sequence, string SensorId, double Celsius);
Enter fullscreen mode Exit fullscreen mode
Still running after 10 s. receive: WaitingForActivation, parse: Faulted, store: WaitingForActivation
received 5,101, parsed 5,000, stored 5,000, waiting in lines: 100
Enter fullscreen mode Exit fullscreen mode

parse threw a FormatException on reading 5,000, and its task is Faulted. Nothing else knows:

  • receive is blocked. It filled lines with the next 100 lines and is waiting in WriteAsync for a slot that will never free up, because nothing reads lines anymore.
  • store is waiting for readings that will never come. readings was never completed, so its ReadAllAsync cannot tell "no data yet" from "no more data".
  • Task.WhenAll cannot help. It waits for all three tasks, two of them never finish, and the exception it would rethrow stays inside a faulted task.

No crash, no log line, no CPU: as failures go, a very calm one. Without the 10-second watchdog at the end, the process would sit there until someone restarts it, and in a service it would look like a consumer that stalled for no reason.

The usual advice is to complete the writer in a finally, so the next stage is never left waiting. I measured that too, with each stage's loop wrapped in try { … } finally { writer.Complete(); } and nothing else changed:

Still running after 10 s. receive: WaitingForActivation, parse: Faulted, store: RanToCompletion
received 5,101, parsed 5,000, stored 5,000, waiting in lines: 100
Enter fullscreen mode Exit fullscreen mode

store now ends as RanToCompletion, that is, successfully, having stored 5,000 of the 10,000 readings as if the stream had ended there. And receive still hangs, because a finally that completes the channel after a stage does nothing for the stage before it. The hang is still there, and now the stage after it reports success on half the data.

A stage that fails has to tell both sides:

  • Downstream, complete its output with the exception: writer.Complete(exception). The next stage's ReadAllAsync reads what is left in the channel, then throws.
  • Upstream, cancel the stage before it. Whatever that stage is waiting on, a full channel, an empty one or its own source, throws OperationCanceledException instead of waiting forever, and that stage, stopping in turn, cancels the one before it.

The pipeline, fixed

Every stage below runs through one helper that does both. Two other things changed. A malformed line is now counted and skipped, because bad data is not a reason to stop a pipeline. And to trigger the failure path, store throws when it reaches reading 7,000, the way a database connection eventually drops:

using System.Diagnostics.CodeAnalysis;
using System.Globalization;
using System.Threading.Channels;

// One token per stage, for everything that stage waits on. Only the stage after it cancels it,
// when that stage fails, so the stop travels back to the first stage, and the stages after a
// failure are never cancelled: they read what is left, then get the exception. In an app, link
// all three to the shutdown token too.
using CancellationTokenSource stopReceive = new(), stopParse = new(), stopStore = new();
Channel<string> lines = Channel.CreateBounded<string>(100);
Channel<Reading> readings = Channel.CreateBounded<Reading>(100);
int received = 0, rejected = 0, parsed = 0, stored = 0;

Task receive = RunStageAsync(stopReceive, stopPrevious: null, complete: error => lines.Writer.TryComplete(error), body: async cancellation =>
{
    for (int sequence = 0; sequence < 10_000; sequence++)
    {
        string line = sequence == 5_000 ? $"{sequence};truck-17;N/A" : $"{sequence};truck-17;4.2";
        await lines.Writer.WriteAsync(line, cancellation);
        Interlocked.Increment(ref received);
    }
});

Task parse = RunStageAsync(stopParse, stopPrevious: stopReceive, complete: error => readings.Writer.TryComplete(error), body: async cancellation =>
{
    await foreach (string line in lines.Reader.ReadAllAsync(cancellation))
    {
        // A malformed line is bad data, not a broken pipeline: count it and move on.
        if (!TryParse(line, out Reading? reading))
        {
            Interlocked.Increment(ref rejected);
            continue;
        }
        await readings.Writer.WriteAsync(reading, cancellation);
        Interlocked.Increment(ref parsed);
    }
});

Task store = RunStageAsync(stopStore, stopPrevious: stopParse, complete: null, body: async cancellation =>
{
    await foreach (Reading reading in readings.Reader.ReadAllAsync(cancellation))
    {
        // The failure nobody planned for.
        if (reading.Sequence == 7_000) throw new IOException("Database connection lost.");
        Interlocked.Increment(ref stored);
    }
});

try
{
    await Task.WhenAll(receive, parse, store);
    Console.WriteLine("Pipeline finished.");
}
catch (Exception ex)
{
    Console.WriteLine($"Pipeline stopped: {ex.GetType().Name}: {ex.Message}");
}
Console.WriteLine($"receive: {receive.Status}, parse: {parse.Status}, store: {store.Status}");
Console.WriteLine($"received {received:N0}, rejected {rejected}, parsed {parsed:N0}, stored {stored:N0}");

// Runs one stage on the thread pool. When it ends, it completes the stage's output channel,
// passing on the exception if it failed, and a failure also cancels the stage before it.
static async Task RunStageAsync(CancellationTokenSource stop, CancellationTokenSource? stopPrevious,
    Action<Exception?>? complete, Func<CancellationToken, Task> body)
{
    try
    {
        await Task.Run(() => body(stop.Token));
        complete?.Invoke(null);
    }
    catch (OperationCanceledException ex) when (!stop.IsCancellationRequested)
    {
        // Cancelled by something other than this pipeline, such as an HTTP timeout. That is a
        // failure: wrapped, it ends the task as Faulted and cannot be mistaken for a stop.
        StageFailedException failure = new(ex);
        await FailAsync(failure);
        throw failure;
    }
    catch (Exception ex)
    {
        await FailAsync(ex);
        throw;
    }

    async Task FailAsync(Exception failure)
    {
        complete?.Invoke(failure);              // the next stage reads what is left, then gets the failure
        if (stopPrevious is not null)
            await stopPrevious.CancelAsync();   // the stage before stops, wherever it is waiting
    }
}

static bool TryParse(string line, [NotNullWhen(true)] out Reading? reading)
{
    string[] fields = line.Split(';');
    // NumberStyles.Float, not the default, which also accepts a thousands separator and would
    // read "4,2" as 42. IsFinite rejects "NaN" and "Infinity".
    if (fields.Length == 3
        && long.TryParse(fields[0], CultureInfo.InvariantCulture, out long sequence)
        && double.TryParse(fields[2], NumberStyles.Float, CultureInfo.InvariantCulture, out double celsius)
        && double.IsFinite(celsius))
    {
        reading = new Reading(sequence, fields[1], celsius);
        return true;
    }
    reading = null;
    return false;
}

// Stands in for an OperationCanceledException the pipeline did not cause, so the stage's task
// ends as Faulted rather than Canceled.
sealed class StageFailedException(Exception inner) : Exception(inner.Message, inner);

record Reading(long Sequence, string SensorId, double Celsius);
Enter fullscreen mode Exit fullscreen mode
Pipeline stopped: IOException: Database connection lost.
receive: Canceled, parse: Canceled, store: Faulted
received 7,202, rejected 1, parsed 7,100, stored 6,999
Enter fullscreen mode Exit fullscreen mode

The N/A line was rejected and the pipeline carried on. When the database "connection" dropped, all three stages stopped, and Task.WhenAll rethrew the IOException instead of hanging. A few details in the helper are easy to miss:

  • Each stage has its own token and uses it for everything it waits on: its reads, its writes and, in a real service, its own I/O. Only the stage after it cancels it. When store failed, it cancelled stopParse; parse threw wherever it was waiting and cancelled stopReceive on its way out, and so on back to the start.
  • The stages after a failure are never cancelled. They read what is left in their channel, and then ReadAllAsync throws the original exception.
  • An OperationCanceledException the pipeline did not cause is a failure. An HTTP timeout inside a stage throws one, and left alone it would end the task as Canceled, so a caller that treats OperationCanceledException as a normal shutdown would swallow an outage. The when filter wraps it in a StageFailedException instead.
  • A stage the pipeline stopped ends as Canceled, not Faulted, so it adds nothing to what Task.WhenAll reports, and the exception you get is the cause.
  • store passes complete: null because it is the last stage and has no output channel. And Task.Run starts each stage on the thread pool, so the three run side by side from the start, instead of the first one running on the caller's thread until its first real wait.

Getting there took me three tries. My first version cancelled one shared token on any failure, so the stages after the failure were cancelled too: with a slow store, it ended Canceled before it could see the real exception, because WaitToReadAsync checks the token before it looks at the channel. My second passed each token only to the writes, and a stage waiting for input never heard about the failure. With a source that sent 20 lines and then went quiet, and store failing at the 10th, the second version was still running after 10 seconds; this one stopped in 36 ms.

The downstream half of the helper did not run in that capture, because store is the last stage. So I ran two more variants, both with store slowed to 200 µs per reading, as in the load test. When parse threw an InvalidOperationException at reading 7,000, store saved all 6,999 readings parse had passed on, and then its ReadAllAsync threw that same exception, not a ChannelClosedException wrapping it; receive ended as Canceled. When receive itself failed at reading 7,000, all three stages ended Faulted with its exception, and every reading it had sent was parsed and stored. And when store threw a TaskCanceledException, the way an insert that times out does, the pipeline stopped with StageFailedException: The insert timed out. and store ended Faulted.

Now count what was lost in the capture above: 7,202 lines received, 1 rejected, 6,999 stored. The other 202 never reached the database. They were in the two channels, 100 in each, or in the hands of a stage when it stopped: the reading store failed on, and the one parse could not write. (Run it yourself and you may get 201: it depends on whether receive gets one more line in before the cancellation reaches it.) That is not a bug in the fix. It is what a channel is: it lives in memory, and on a failure or a restart, whatever is in it is gone. When every reading has to survive, the buffer has to be durable and the consumer has to acknowledge each item after storing it, which is a message broker's job.

Three stations joined by conveyors: station 1 holds a cube with the belt after it packed full, station 2 is marked stopped with a warning sign and a broken gear, and station 3 stands empty under an hourglass

The baseline: BlockingCollection

Before channels, the standard producer/consumer queue in .NET was BlockingCollection<T>, and it still works. The difference is in how its consumers wait. GetConsumingEnumerable() blocks the thread it runs on until an item arrives, while a channel's ReadAllAsync awaits and gives the thread back. To measure what that costs, I started 64 consumers on an empty queue, waited a second, then queued one more work item the way Kestrel, ASP.NET Core's web server, queues an incoming request, and timed how long it waited for a thread:

using System.Collections.Concurrent;
using System.Diagnostics;
using System.Threading.Channels;

const int Consumers = 64;
int threadsAtStart = Process.GetCurrentProcess().Threads.Count;
Console.WriteLine($"{Environment.ProcessorCount} cores, {Consumers} consumers waiting for work that has not arrived yet");
Console.WriteLine($"{"consumers",-32} {"new threads",11} {"pool threads",12} {"a queued request waited",24}");

// A consumer waiting on a channel holds no thread, only a callback to run when an item arrives.
Channel<int> channel = Channel.CreateBounded<int>(1_000);
Task[] readers = Enumerable.Range(0, Consumers)
    .Select(_ => Task.Run(async () => { await foreach (int job in channel.Reader.ReadAllAsync()) { } }))
    .ToArray();
await ReportAsync("Channel, await foreach");
channel.Writer.Complete();
await Task.WhenAll(readers);

// BlockingCollection on dedicated threads: each consumer blocks a thread of its own.
await MeasureBlockingCollectionAsync("BlockingCollection, LongRunning", TaskCreationOptions.LongRunning);

// BlockingCollection on the thread pool, the way Task.Run starts it in the first example of its docs.
await MeasureBlockingCollectionAsync("BlockingCollection, thread pool", TaskCreationOptions.None);

async Task MeasureBlockingCollectionAsync(string label, TaskCreationOptions options)
{
    using BlockingCollection<int> queue = new(boundedCapacity: 1_000);
    Task[] consumers = Enumerable.Range(0, Consumers)
        .Select(_ => Task.Factory.StartNew(
            () => { foreach (int job in queue.GetConsumingEnumerable()) { } },
            CancellationToken.None, options, TaskScheduler.Default))
        .ToArray();
    await ReportAsync(label);
    queue.CompleteAdding();
    await Task.WhenAll(consumers);
}

async Task ReportAsync(string label)
{
    await Task.Delay(TimeSpan.FromSeconds(1));

    // QueueUserWorkItem uses the pool's global queue, which is where Kestrel puts an incoming request.
    TaskCompletionSource started = new(TaskCreationOptions.RunContinuationsAsynchronously);
    Stopwatch waited = Stopwatch.StartNew();
    ThreadPool.QueueUserWorkItem(_ => started.SetResult());
    await started.Task;

    int newThreads = Process.GetCurrentProcess().Threads.Count - threadsAtStart;
    Console.WriteLine($"{label,-32} {newThreads,11} {ThreadPool.ThreadCount,12} {waited.Elapsed.TotalMilliseconds,21:N1} ms");
}
Enter fullscreen mode Exit fullscreen mode
8 cores, 64 consumers waiting for work that has not arrived yet
consumers                        new threads pool threads  a queued request waited
Channel, await foreach                    10            8                   8.8 ms
BlockingCollection, LongRunning           75            9                   4.8 ms
BlockingCollection, thread pool           65           65              43,093.5 ms
Enter fullscreen mode Exit fullscreen mode
  • 64 consumers awaiting a channel cost no threads. The thread pool stayed at 8 threads, one per core, and the queued work item started in under 10 ms.
  • BlockingCollection on dedicated threads, with TaskCreationOptions.LongRunning, costs one thread per consumer: 64 threads doing nothing, each with its own stack. They are not the pool's threads, so the queued work item still started in under 10 ms.
  • BlockingCollection on the thread pool is where Task.Run puts it, and Task.Run is how the first example in its own docs starts the consumer. Each waiting consumer holds a pool thread, and the pool cannot tell a blocked thread from a busy one, so it adds threads slowly. Microsoft's guide to thread pool starvation describes it: past a first quick jump, "further threads are added 1-2 per second". The queued work item waited 43.1 seconds in this run, and between 37.1 and 40.0 in three others. In ASP.NET Core, that work item is a request.

BlockingCollection is not broken; it is synchronous. In a console tool or a service that runs on its own threads, it is fine. In async code, a consumer waiting on it blocks a thread the rest of the app needs, and a channel does the same job without one.

On the left, thread spools tied to sleeping boxes while a long row of envelopes waits in front of them; on the right, tickets pinned on a board and free spools, one of them already taking an envelope

When to use a channel, and when not

Use one for:

  • Producer and consumer in the same process, such as a BackgroundService in ASP.NET Core reading jobs that endpoints write to a bounded channel.
  • Pipelines, where each stage runs at its own pace and a bounded channel between two stages keeps a fast one from flooding a slow one.
  • Absorbing bursts, with a capacity sized to the latency you can accept: capacity × time per item.

Use something else when:

  • Items must survive a crash or a deploy. A channel is memory. Use a durable queue, as in RabbitMQ from Fundamentals to Production, Part 2: Building a Scalable Service, or a database table, and acknowledge each item after the work is done.
  • The producer and the consumer are different processes. Same answer.
  • You need retries, schedules or a dashboard of jobs. That is a job scheduler such as Hangfire or Quartz.NET, not a queue.
  • You only need to process a known list in parallel. Parallel.ForEachAsync with MaxDegreeOfParallelism is shorter and leaves nothing to complete or drain.

Key Takeaways

  • In a drop mode, a write that loses its item still returns true. itemDropped is the only signal: count it and export it as a metric.
  • DropNewest drops the newest item already in the channel, not the one being written. Of the three drop modes, only DropOldest kept the consumer on fresh data, with a median wait of 20 ms against about 200.
  • Wait never loses an item and makes the producer pay: capacity × time per item is the latency you choose. And TryWrite never waits, even in Wait mode.
  • A stage that throws stalls the whole pipeline without a word. Complete its output with the exception and cancel the stage before it, wherever it is waiting; never cancel the stages after it. finally { Complete(); } alone leaves the hang and makes the next stage report success on half the data.
  • BlockingCollection holds a thread for every consumer that waits. On the thread pool, 64 of them kept an unrelated work item waiting for between 37 and 43 seconds.

Top comments (0)