Evaluating “ReadLine using System.IO.Pipelines” Performance in C#

Read string line by line using System.IO.Pipelines API in C

A typical approach for reading a string line by line from a file would probably be using ReadLine APIs. I have heard all good things about System.IO.Pipelines, which was born from the work of the .NET Core team to make Kestrel one of the fastest web servers.

Ben Adams #BlackLivesMatter

@ben_a_adams

Round 19 of @TFBenchmarks is out and ASP​.NET Core on Linux racing to the top!

Who would have thought .NET would have been fully OSS and top performer *on Linux* a several years ago? #aspnetcore #dotnet

techempower.com/benchmarks/#se…

23:04 PM - 28 May 2020

168 379

Although it is probably better suited for network socket IO, I was wondering how easy and how does it perform compared to the traditional ReadLine APIs? So I decided to give it a spin by creating simple benchmarks using BenchmarkDotNet as part of my learning journey.

But, for this post, instead of using a file as the source of the stream, I decided to use in-memory, presumably more-stable-for-benchmark MemoryStream. Feel free to modify it to use a file as the stream source and try it yourself.

	private Stream _stream;
	[Params(300_000)]
	public int LineNumber { get; set; }
	[ParamsSource(nameof(LineCharMultiplierValues))]
	public int LineCharMultiplier { get; set; }
	public IEnumerable<int> LineCharMultiplierValues => Enumerable.Range(1, 15).Concat(new[] { 20, 30, 50, 80, 100 });
	[IterationSetup]
	public void IterationSetup()
	{
	_stream = PrepareStream();
	}
	[IterationCleanup]
	public void IterationCleanup()
	{
	_stream.Dispose();
	}
	public Stream PrepareStream()
	{
	var stream = new MemoryStream();
	using var sw = new StreamWriter(stream, Encoding.UTF8, leaveOpen: true);
	foreach (var no in Enumerable.Range(1, LineNumber))
	{
	foreach (var _ in Enumerable.Range(1, LineCharMultiplier))
	{
	sw.Write($"ABC{no:D7}");
	}
	sw.WriteLine();
	}
	sw.Flush();
	stream.Seek(0, SeekOrigin.Begin);
	return stream;
	}

view raw ReadLineBenchmarksInit.cs hosted with ❤ by GitHub

The code for ReadLine is so dead simple; nothing fancy here.

	public async Task<string> ReadLineUsingStringReaderAsync()
	{
	using var sr = new StreamReader(_stream, Encoding.UTF8);
	string str;
	while ((str = await sr.ReadLineAsync()) is not null)
	{
	// simulate string processing
	str = str.AsSpan().Slice(0, 5).ToString();
	}
	return str;
	}

view raw ReadLineUsingStringReaderAsync.cs hosted with ❤ by GitHub

(My apologies for the possibly misleading method name ReadLineUsingStringReaderAsync; it doesn’t use StringReader class at all).

You will probably spot that unfamiliar C# 9.0 Pattern Combinators is not null; Yes, I’m using the latest (at the time of writing) .NET 5 Preview 7 (5.0.0-preview.7) and the preview version of Visual Studio (Version 16.8.0 Preview 1.0). Don’t blame me, I love bleeding edge stuff!😎

See how is not null is transformed here.

To simplify the processing of a ReadOnlySequence, here I used SequenceReader; taken from the official sample here, modified it slightly to make it follow a similar pattern while (... is not null) as ReadLineUsingStringReaderAsync does.

	public async Task<string> ReadLineUsingPipelineAsync()
	{
	var reader = PipeReader.Create(_stream);
	string str;
	while (true)
	{
	ReadResult result = await reader.ReadAsync();
	ReadOnlySequence<byte> buffer = result.Buffer;
	while ((str = ReadLine(ref buffer)) is not null)
	{
	// simulate string processing
	str = str.AsSpan().Slice(0, 5).ToString();
	}
	reader.AdvanceTo(buffer.Start, buffer.End);
	if (result.IsCompleted) break;
	}
	await reader.CompleteAsync();
	return str;
	}
	[MethodImpl(MethodImplOptions.AggressiveInlining)]
	private static string ReadLine(ref ReadOnlySequence<byte> buffer)
	{
	var reader = new SequenceReader<byte>(buffer);
	if (reader.TryReadTo(out var line, NewLine))
	{
	buffer = buffer.Slice(reader.Position);
	return Encoding.UTF8.GetString(line);
	}
	return default;
	}

view raw ReadLineUsingPipelineAsync.cs hosted with ❤ by GitHub

And finally, here is the result: 40 benchmarks in 15 minutes on my machine.

Here is the gist version:

BenchmarkDotNet=v0.12.1, OS=Windows 10.0.20190
Intel Core i5-9400F CPU 2.90GHz (Coffee Lake), 1 CPU, 6 logical and 6 physical cores
.NET Core SDK=5.0.100-preview.4.20258.7
  [Host]     : .NET Core 5.0.0 (CoreCLR 5.0.20.25106, CoreFX 5.0.20.25106), X64 RyuJIT
  Job-GWLPZR : .NET Core 5.0.0 (CoreCLR 5.0.20.25106, CoreFX 5.0.20.25106), X64 RyuJIT

InvocationCount=1  UnrollFactor=1  

Method	LineNumber	LineCharMultiplier	Mean	Error	StdDev	Ratio	RatioSD	Gen 0	Gen 1	Gen 2	Allocated	Code Size
ReadLineUsingStringReaderAsync	300000	1	19.92 ms	0.281 ms	0.263 ms	1.00	0.00	9000.0000	-	-	44.32 MB	0 MB
ReadLineUsingPipelineAsync	300000	1	42.67 ms	0.684 ms	0.639 ms	2.14	0.03	5000.0000	-	-	23.04 MB	0 MB
ReadLineUsingStringReaderAsync	300000	2	23.93 ms	0.114 ms	0.101 ms	1.00	0.00	11000.0000	-	-	49.5 MB	0 MB
ReadLineUsingPipelineAsync	300000	2	45.17 ms	0.096 ms	0.075 ms	1.89	0.01	6000.0000	-	-	27.71 MB	0 MB
ReadLineUsingStringReaderAsync	300000	3	27.88 ms	0.076 ms	0.071 ms	1.00	0.00	12000.0000	-	-	57.58 MB	0 MB
ReadLineUsingPipelineAsync	300000	3	41.74 ms	0.256 ms	0.240 ms	1.50	0.01	7000.0000	-	-	34.73 MB	0 MB
ReadLineUsingStringReaderAsync	300000	4	32.32 ms	0.119 ms	0.111 ms	1.00	0.00	13000.0000	-	-	62.6 MB	0 MB
ReadLineUsingPipelineAsync	300000	4	46.22 ms	0.163 ms	0.136 ms	1.43	0.01	8000.0000	-	-	39.4 MB	0 MB
ReadLineUsingStringReaderAsync	300000	5	36.50 ms	0.329 ms	0.292 ms	1.00	0.00	15000.0000	-	-	70.56 MB	0 MB
ReadLineUsingPipelineAsync	300000	5	47.11 ms	0.288 ms	0.255 ms	1.29	0.01	10000.0000	-	-	46.42 MB	0 MB
ReadLineUsingStringReaderAsync	300000	6	40.39 ms	0.495 ms	0.463 ms	1.00	0.00	16000.0000	-	-	75.92 MB	0 MB
ReadLineUsingPipelineAsync	300000	6	48.40 ms	0.553 ms	0.490 ms	1.20	0.02	11000.0000	-	-	51.09 MB	0 MB
ReadLineUsingStringReaderAsync	300000	7	45.12 ms	0.471 ms	0.441 ms	1.00	0.00	18000.0000	-	-	84.1 MB	0 MB
ReadLineUsingPipelineAsync	300000	7	48.48 ms	0.447 ms	0.396 ms	1.08	0.01	12000.0000	-	-	58.11 MB	0 MB
ReadLineUsingStringReaderAsync	300000	8	49.05 ms	0.240 ms	0.224 ms	1.00	0.00	19000.0000	-	-	89.46 MB	0 MB
ReadLineUsingPipelineAsync	300000	8	53.48 ms	0.627 ms	0.587 ms	1.09	0.01	13000.0000	-	-	62.78 MB	0 MB
ReadLineUsingStringReaderAsync	300000	9	53.54 ms	0.629 ms	0.589 ms	1.00	0.00	21000.0000	-	-	98.53 MB	0 MB
ReadLineUsingPipelineAsync	300000	9	51.14 ms	0.539 ms	0.505 ms	0.96	0.01	15000.0000	-	-	69.8 MB	0 MB
ReadLineUsingStringReaderAsync	300000	10	59.32 ms	0.657 ms	0.614 ms	1.00	0.00	23000.0000	-	-	104.66 MB	0 MB
ReadLineUsingPipelineAsync	300000	10	53.99 ms	0.445 ms	0.417 ms	0.91	0.01	16000.0000	-	-	74.47 MB	0 MB
ReadLineUsingStringReaderAsync	300000	11	63.84 ms	0.467 ms	0.437 ms	1.00	0.00	25000.0000	-	-	114.48 MB	0 MB
ReadLineUsingPipelineAsync	300000	11	52.76 ms	0.627 ms	0.586 ms	0.83	0.01	18000.0000	-	-	81.49 MB	0 MB
ReadLineUsingStringReaderAsync	300000	12	68.20 ms	0.803 ms	0.711 ms	1.00	0.00	26000.0000	-	-	120.31 MB	0 MB
ReadLineUsingPipelineAsync	300000	12	57.81 ms	0.597 ms	0.529 ms	0.85	0.01	19000.0000	-	-	86.16 MB	0 MB
ReadLineUsingStringReaderAsync	300000	13	72.07 ms	0.642 ms	0.601 ms	1.00	0.00	28000.0000	-	-	129.91 MB	0 MB
ReadLineUsingPipelineAsync	300000	13	57.15 ms	0.707 ms	0.662 ms	0.79	0.01	20000.0000	-	-	93.18 MB	0 MB
ReadLineUsingStringReaderAsync	300000	14	77.42 ms	0.628 ms	0.556 ms	1.00	0.00	30000.0000	-	-	136.4 MB	0 MB
ReadLineUsingPipelineAsync	300000	14	60.68 ms	0.526 ms	0.492 ms	0.78	0.01	21000.0000	-	-	97.85 MB	0 MB
ReadLineUsingStringReaderAsync	300000	15	81.45 ms	0.793 ms	0.703 ms	1.00	0.00	32000.0000	-	-	146.52 MB	0 MB
ReadLineUsingPipelineAsync	300000	15	59.10 ms	0.702 ms	0.657 ms	0.73	0.01	23000.0000	-	-	104.87 MB	0 MB
ReadLineUsingStringReaderAsync	300000	20	103.58 ms	1.378 ms	1.289 ms	1.00	0.00	41000.0000	-	-	187.59 MB	0 MB
ReadLineUsingPipelineAsync	300000	20	67.48 ms	0.953 ms	0.892 ms	0.65	0.01	29000.0000	-	-	132.92 MB	0 MB
ReadLineUsingStringReaderAsync	300000	30	149.09 ms	1.873 ms	1.564 ms	1.00	0.00	63000.0000	-	-	283.98 MB	0 MB
ReadLineUsingPipelineAsync	300000	30	81.96 ms	0.973 ms	0.910 ms	0.55	0.01	42000.0000	-	-	191.37 MB	0 MB
ReadLineUsingStringReaderAsync	300000	50	243.59 ms	1.986 ms	1.858 ms	1.00	0.00	115000.0000	-	-	518.67 MB	0 MB
ReadLineUsingPipelineAsync	300000	50	108.48 ms	0.446 ms	0.372 ms	0.45	0.00	68000.0000	-	-	308.27 MB	0 MB
ReadLineUsingStringReaderAsync	300000	80	389.95 ms	2.720 ms	2.544 ms	1.00	0.00	217000.0000	2000.0000	-	975.49 MB	0 MB
ReadLineUsingPipelineAsync	300000	80	148.69 ms	1.200 ms	1.064 ms	0.38	0.00	107000.0000	-	-	483.62 MB	0 MB
ReadLineUsingStringReaderAsync	300000	100	489.55 ms	3.036 ms	2.840 ms	1.00	0.00	302000.0000	5000.0000	-	1349.87 MB	0 MB
ReadLineUsingPipelineAsync	300000	100	178.96 ms	3.555 ms	3.492 ms	0.37	0.01	133000.0000	1000.0000	-	600.52 MB	0 MB

view raw ReadLinesBenchmarks-report-github.md hosted with ❤ by GitHub

Legends:

LineNumber : Value of the 'LineNumber' parameter
LineCharMultiplier : Value of the 'LineCharMultiplier' parameter
Mean : Arithmetic mean of all measurements
Error : Half of 99.9% confidence interval
StdDev : Standard deviation of all measurements
Ratio : Mean of the ratio distribution ([Current]/[Baseline])
RatioSD : Standard deviation of the ratio distribution ([Current]/[Baseline])
Gen 0 : GC Generation 0 collects per 1000 operations
Gen 1 : GC Generation 1 collects per 1000 operations
Gen 2 : GC Generation 2 collects per 1000 operations
Allocated : Allocated memory per single operation (managed only, inclusive, 1KB = 1024B)
Code Size : Native code size of the disassembled method(s)
1 ms : 1 Millisecond (0.001 sec)

You can find the source code in my GitHub repository.

Conclusion

• Pipelines versions are better in terms of memory usage (using less memory).
• In terms of speed, it is surprisingly slower than the ordinary ReadLine version given the string length ≤ 80 (perhaps I am doing it wrong? Let me know! I am still learning!). It is starting to shine, getting faster and faster if the string length ≥ 90. (270% 🚀 faster for string length = 1000).
• Less GC pressure (a good thing) for Pipelines versions (Gen 0, Gen 1).
• The amount of code to write for the Pipelines version is longer.

[Update] See part 2 of this series for a better version! (Spoiler: faster on every test!)

DISCLAIMER: Your mileage may vary. As with all performance work, each of the scenarios chosen for your application should be measured, measured and measured. There is no silver bullet.

Resources:

• System.IO.Pipelines: High performance IO in .NET
• System.IO.Pipelines in .NET
• Work with Buffers in .NET
• An Introduction to SequenceReader by Steve Gordon

Comments

[Vojtech Machacek]

What if the line is longer than buffer?