Asynchronous streams in C# 8.0

Also in this series

1. C# 8.0 Nullable Reference types are here!
2. Pattern matching in C#
3. Asynchronous streams
4. Indices and ranges
5. Default interface methods
6. 5 tips to improve your productivity in C# 8.0

All code samples are available on github

Asynchronous streams: what are they?

Asynchronous streams are a new way to iterate over a stream of data. It was previously possible to do this synchronously in C#. Now you can do it even if the stream producer is asynchronous! Woot, woot!

What you need to get started

To enable asynchronous streams you need 3 things:

• Your method must be async
• The return type of your method must be IAsyncEnumerable<T>
• The method body must contain at least one yield return

How is this useful?

To show the value of this feature, nothing is better than a real use case.

Use case

As a user, I want to count all the users' names that contain the letter 'n'.

Context

All the users' names are in an SQL database. That database contains millions of users.

The problem

The problem is that this dataset is really big and to calculate the final value we'll need to load it all. If you try to load that many records in memory, you'll probably run out of memory and get an OutOfMemoryException.

Yes, yes... I know, you can compute this value directly with a SQL query. Remember, this is just a simple example to show how the feature works.

Let's look at how it's done with async streams

Producer

public static async IAsyncEnumerable<string> GetAllNames()
{
    var pageIndex = 0;
    const int pageSize = 100;
    var hasMore = false;
    do
    {
        // **** IMPORTANT ****
        // Never EVER do this in a real application. The following code block is
        // not secure and could be subject to an attack known as SQL Injection.
        await using var conn = new SqlConnection("ConnectionString here");
        await using var cmd = new SqlCommand(
            @$"
            SELECT Name
            FROM Users
            ORDER BY Name
            OFFSET {pageIndex * pageSize} ROWS
            FETCH NEXT {pageSize} ROWS ONLY",
            conn);
        await using var reader = await cmd.ExecuteReaderAsync();
        while (reader.Read())
        {
            yield return reader.GetString(0); // This is the "Name" column
        }

        pageIndex++;
        hasMore = reader.HasRows;
    } while (hasMore);
}

Consumer

public async Task<int> CountNamesWithN()
{
    var namesContainingN = 0;
    await foreach (var name in GetAllNames())
    {
        if (name.Contains("n"))
        {
            namesContainingN++;
        }
    }

    return namesContainingN;
}

This is fairly straightforward. From the producer's side, simply use
yield return to stream some data on demand. On the consumer's side, notice the use of await right before the foreach keyword, this is where the magic happens. The rest behaves just like a plain old foreach over a synchronous
IEnumerable, but now it's asynchronous.

At first, this may look like magic, but under the hood, while iterating over the stream of data on the consumer's side, new database queries are issued seamlessly when it needs to load more data. This allows us to nicely stream our data without blowing up the memory, and all the complexity is managed by
IAsyncEnumerable.

Conclusion

With async and await that are now part of our daily development tools, this new feature will simplify a lot of code that you had to do manually. Let me know in the comments how you think this could help you in your day to day work.

Also in this series

1. C# 8.0 Nullable Reference types are here!
2. Pattern matching in C#
3. Asynchronous streams
4. Indices and ranges
5. Default interface methods
6. 5 tips to improve your productivity in C# 8.0

All code samples are available on github