Traditional async/await
For a long time, async/await has been the foundation of asynchronous programming in .NET. It lets us write asynchronous code that looks much like synchronous code, making an otherwise tricky problem considerably easier to manage.
Under the hood, async/await is implemented using a CPS (Continuation-Passing Style) transformation.
First, the async keyword tells the compiler that a method is asynchronous. This marks the entry point for the CPS transformation. Strictly speaking, the async/await model itself does not require an async keyword. C# requires it so the compiler knows that await is a keyword marking a possible suspension point rather than an ordinary identifier. C++ uses a similar async/await model without requiring async.
The await keyword tells the compiler, "execution may stop here." The compiler splits the method around each await, then runs the remaining code after the awaited operation completes.
Here is a simple example:
public async Task<int> GetDataAsync()
{
// Stand in for an asynchronous operation by waiting for one second.
await Task.Delay(1000);
return 42;
}
If Task.Delay(1000) has not completed yet, await suspends GetDataAsync. One second later, execution resumes and the method returns 42. Roughly speaking, the compiler divides an asynchronous method at each await and arranges for the next piece to run when the operation finishes.
class StateMachine
{
private int state = 0;
// Create a Task<int> to hold the result.
// It completes when the current asynchronous method completes.
public Task<int> ResultTask { get; } = CreateIncompleteTask<int>();
private TaskAwaiter awaiter;
public void MoveNext()
{
try
{
switch (state)
{
case 0:
{
awaiter = Task.Delay(1000).GetAwaiter();
if (!awaiter.IsCompleted)
{
// Record where execution should resume.
state = 1;
// Register the continuation.
// When Task.Delay completes, it invokes the registered continuation,
// causing the state machine to call MoveNext again.
// Where the continuation eventually runs depends on the awaiter,
// the current SynchronizationContext, the TaskScheduler, and so on.
awaiter.OnCompleted(MoveNext);
return;
}
goto case 1;
}
case 1:
{
state = -1;
// Confirm that the awaited operation completed successfully.
// If it failed, GetResult throws the exception here.
awaiter.GetResult();
// Complete the Task<int> with a result of 42.
CompleteTask(ResultTask, 42);
return;
}
}
}
catch (Exception ex)
{
// If MoveNext throws, complete the Task<int> in a faulted state.
// The caller then receives the exception from await GetDataAsync().
FailTask(ResultTask, ex);
}
}
}
The asynchronous method becomes a state machine, with a state corresponding to each await. When the awaited operation completes, the state machine runs the next piece of code. Conceptually, GetDataAsync is compiled into something like this:
public Task<int> GetDataAsync()
{
var stateMachine = new StateMachine();
stateMachine.MoveNext();
return stateMachine.ResultTask;
}
CreateIncompleteTask and CompleteTask are only pseudocode used to illustrate the mechanism. The code generated by the C# compiler does not manipulate Task directly. It uses AsyncTaskMethodBuilder<int> to create and complete the Task<int> representing the asynchronous method.
The limits of traditional async
async/await is extremely convenient. That convenience is not free, though.
When the C# compiler transforms an asynchronous method, it cannot know whether a given call will actually suspend. Many asynchronous methods complete synchronously without suspending at all:
public async Task<int> GetDataAsync()
{
return await GetValueAsync();
}
public async Task<int> GetValueAsync()
{
return 42;
}
The C# compiler generates a state machine and a Task<int> for both methods. It performs this transformation one method at a time, so it cannot see through GetDataAsync into the implementation of GetValueAsync.
At this point, you might reasonably ask: if the C# compiler cannot figure it out, surely the JIT can sort it out at runtime?
Unfortunately, it cannot. The C# compiler has already rewritten the asynchronous method into a state machine driven by MoveNext, an awaiter, and a method builder. By the time the code reaches the JIT, the original A -> await B -> await C call structure has been pulled apart. The JIT sees a complicated conversation among state machines, Task objects, awaiters, and continuations. Optimizations that could have crossed method boundaries are now much harder to perform.
If the entire asynchronous call chain completes without suspending, it could, in principle, run like an ordinary chain of synchronous calls. But the C# compiler has already dismantled the original control flow, and asking the JIT to reconstruct it afterward is a tall order.
To make matters worse, MoveNext contains the logic for the entire asynchronous method and can grow quite large. The JIT often declines to inline it because of its size, which hides even more of the call chain. Inlining several asynchronous calls into one another is difficult for the same reason.
Allocation is another problem. An asynchronous method returns a Task or Task<T>, which usually means creating a new Task object on every call. That cost matters in performance-sensitive code. This is why ValueTask exists: it reduces allocations through a value-type result and reuse via IValueTaskSource.
If the operation genuinely suspends, none of this is especially troubling. Even with full visibility into the call chain, the JIT cannot optimize the suspension away. The runtime must resume the code later, and it needs something such as a Task object to hold the result and completion state.
The real problem is that asynchronous methods complete synchronously more often than one might expect. This is especially common in deep call chains and distributed systems built around asynchronous APIs:
- Only the innermost method may actually suspend, while every method above it merely passes the result along.
- An asynchronous method may call a synchronous method that calls another asynchronous method, leaving the entire chain effectively synchronous.
- A distributed system may look thoroughly asynchronous while most of its actual workload remains synchronous.
Traditional async/await still builds a state machine for every asynchronous method it encounters. Even methods that never suspend receive the full bill.
Green Threads
Before getting to Runtime Async, it is worth taking a short detour. The .NET team previously experimented with Green Threads, lightweight user-mode threads similar to goroutines and Java Virtual Threads. The goal was to reduce the cost of thread switching.
The idea was attractive, but the implementation exposed several problems that were hard to avoid.
A Green Thread may be lightweight, but it is still a complete execution context. At a minimum, it must preserve register state, a call stack, and metadata used by the runtime scheduler. A goroutine, for example, starts with roughly a 2 KB user stack and grows it as needed. Two kilobytes is enough space for hundreds of async state machines, so "lightweight" was doing a little work there.
Green Threads also required the runtime to handle scheduling in user mode. In other words, the runtime held the steering wheel. Developers had limited control over the resulting scheduling behavior.
System calls made the situation even less attractive. A Green Thread is not an OS thread, so the OS thread underneath it still had to perform the actual system call. Switching, suspending, and resuming between the Green Thread and the OS thread added work, sometimes making a system call dozens of times more expensive. In the .NET experiment, 100 million system calls went from about 300 ms to about 1,800 ms. That was a slowdown of more than 5x.
Green Threads also interacted poorly with hardware security features. Intel CET Shadow Stack, for example, maintains a protected stack of return addresses in hardware and checks it against the ordinary call stack when a function returns. Because Green Threads switch call stacks in user mode, the runtime would have had to maintain not only the regular stack pointer but also the Shadow Stack state associated with the underlying OS thread. Integrating with hardware control-flow protection became more complicated and could even require dedicated operating-system support.
Thread affinity was another headache. The runtime scheduled Green Threads, so a resumed Green Thread was not guaranteed to run on the same OS thread. Yet plenty of APIs depend on a particular thread, including GUI APIs, parts of the OS, and code built around thread-local state. The runtime would have had to pin a Green Thread to an OS thread or perform extra scheduling and switching whenever such code ran. A GUI message loop can call thread-affine APIs tens of thousands of times per second. At that frequency, Green Thread scheduling could become more expensive than simply using OS threads.
The result in ASP.NET Core was the final blow. RPS (requests per second) did not rise; it actually fell. Accepting all those constraints only to end up slower than traditional async/await was not a compelling bargain. The .NET team ended the Green Thread experiment and moved on to Runtime Async.
Runtime Async
So what is the alternative? The answer is surprisingly simple: do not make the C# compiler build the state machine. Preserve the original asynchronous control flow and hand it directly to the JIT. That idea became Runtime Async.
Runtime Async introduces a new calling convention to the .NET runtime: the Async Calling Convention. Internally, methods using it are marked with MethodImplOptions.Async, but users cannot apply that marker directly. The C# code we write still uses the same familiar async/await syntax:
async Task<int> A()
{
return await B();
}
With traditional async, the JIT sees the MoveNext state machine emitted by the C# compiler. With Runtime Async, it sees the method's original asynchronous control flow and generates code that follows the special Async Calling Convention. In addition to the normal arguments, this convention passes a Continuation object.
Suppose an ordinary method call looks like this:
result = B(args);
Runtime Async adds one more participant, the continuation:
(result, continuation) = B(continuation, args);
The continuation holds the state required to resume the call chain after a suspension.
The first call to an asynchronous method passes null as the Continuation. There is no state to restore yet, so the method begins at the top like an ordinary synchronous method. If it reaches the end without suspending, it returns the normal result together with a null Continuation. The caller can use that null value to tell that the method completed synchronously.
If the method reaches an await whose operation has not completed, the call chain suspends. The runtime stores the state required for resumption in a Continuation object and returns it to the caller. A non-null Continuation tells the caller, "this one suspended." Once the awaited operation completes, the runtime uses that object to resume execution.
At that point, the runtime calls the Runtime Async method again and supplies the saved Continuation as an extra argument. The method resumes where it left off and continues until the entire call chain finishes.
This is the key idea. Both the ordinary result and the extra Continuation are part of the calling convention. They can travel directly through registers rather than being wrapped in an object first. Runtime Async preserves the existing ABI for the ordinary return value and uses another return channel for the Continuation. When the target architecture permits it, both values stay in registers.
For a call chain that never suspends, data moves almost exactly as it would in synchronous code. Arguments, return values, and the Continuation all travel through registers. There is no need to allocate a result wrapper at every async call, so that overhead disappears.
In other words, a method may be declared as returning Task<T>, yet no Task<T> object appears if the call chain never suspends. Internally, the value of type T is returned directly.
Giving the JIT the complete asynchronous control flow matters just as much. It can optimize across method boundaries and may even inline asynchronous calls into one another.
Looking at the generated code
Let us see what Runtime Async actually generates. The example below computes Fibonacci numbers recursively in an asynchronous method:
class Program
{
async Task<int> Fib(int n)
{
if (n <= 1)
return n;
return await Fib(n - 1) + await Fib(n - 2);
}
}
After compiling the assembly, we can decompile its IL with ILSpy and get the following:
internal class Program
{
[MethodImpl(MethodImplOptions.Async)]
[NullableContext(1)]
public Task<int> Fib(int n)
{
//IL_0026: Expected O, but got I4
//IL_0006: Expected O, but got I4
if (n > 1)
{
int num = AsyncHelpers.Await(Fib(n - 1));
int num2 = AsyncHelpers.Await(Fib(n - 2));
return (Task<int>)(num + num2);
}
return (Task<int>)n;
}
}
Apart from the original logic, there is no state machine in sight.
When we run it, the JIT produces code roughly like the following. It is long, but there is no need to panic. We will pick out the important pieces afterward.
Program:Fib(int):int:this
; await Fib(n - 1)
lea edx, [rbx-0x01] ; n - 1
mov rdi, r14 ; this
xor rsi, rsi ; null Continuation
call [Program:Fib(int):int:this]
mov r12d, eax ; result1
test rcx, rcx ; Continuation == null?
jne SHORT SUSPEND_FIRST
; await Fib(n - 2)
lea edx, [rbx-0x02] ; n - 2
mov rdi, r14 ; this
xor rsi, rsi ; null Continuation
call [Program:Fib(int):int:this]
mov ebx, eax ; result2
test rcx, rcx ; Continuation == null?
jne SHORT SUSPEND_SECOND
; Both calls completed synchronously, so return the result directly.
add ebx, r12d
mov eax, ebx ; return value
xor ecx, ecx ; null Continuation
ret
SUSPEND_FIRST:
; Fib(n - 1) suspended, so create a Continuation to save the current state.
mov rdi, rcx
mov rsi, 0x... ; Continuation
call [CORINFO_HELP_ALLOC_CONTINUATION]
mov r12, rax
mov dword ptr [r12+0x48], ebx ; save n
; ... save any other required state ...
mov rcx, r12 ; return Continuation
ret
SUSPEND_SECOND:
; Fib(n - 2) suspended, so create a Continuation to save the current state.
mov rdi, rcx
mov rsi, 0x... ; Continuation type
call [CORINFO_HELP_ALLOC_CONTINUATION]
mov r15, rax
mov dword ptr [r15+0x4C], r12d ; save the result of Fib(n - 1)
; ... save any other required state ...
mov rcx, r15 ; return Continuation
ret
; --------------------------------------------
Program:Fib(int):Task<int>:this
mov rdi, rbx ; this
mov edx, r15d ; n
xor rsi, rsi ; null Continuation
call [Program:Fib(int):int:this] ; call the actual Runtime Async method
mov ebx, eax ; result
test rcx, rcx ; Continuation == null?
jne THUNK_SUSPENDED
; return Task.FromResult(ebx)
mov rax, <Task<int>>
ret
THUNK_SUSPENDED:
; var task = new RuntimeAsyncTask<int>();
; connect the continuation to the task;
; return task;
The first thing to notice is the internal Program:Fib(int):int:this method, which uses the Async Calling Convention. Its return type is int, not the original Task<int>.
On x64, the method passes the this pointer, the Continuation pointer, and the value of n in registers:
mov r14, rdi ; this
mov r15, rsi ; Continuation
mov ebx, edx ; n
On the first call, there is no state to restore, so the method receives a null Continuation.
Consider the first recursive call:
await Fib(n - 1)
The JIT compiles it into:
lea edx, [rbx-0x01] ; n - 1
mov rdi, r14 ; this
xor rsi, rsi ; Continuation = null
call [Program:Fib(int):int:this]
This call to Fib(n - 1) effectively returns two values:
eax = the int result of Fib
rcx = Continuation
This is not a (int, Continuation) tuple. Under the x64 ABI, the int and the Continuation are returned independently in different registers.
The caller needs only a few instructions to collect the result and determine whether the call suspended:
mov r12d, eax
test rcx, rcx ; Is Continuation null?
jne SUSPEND ; If not, the call suspended
If rcx == null, the call completed synchronously and eax contains a valid result. Execution can continue immediately:
lea edx, [rbx-0x02]
mov rdi, r14
xor rsi, rsi
call [Program:Fib(int):int:this] ; second recursive call: Fib(n - 2)
In C#-like pseudocode, the flow looks like this:
var (result1, continuation1) = Fib(null, n - 1);
if (continuation1 != null)
Suspend(continuation1);
var (result2, continuation2) = Fib(null, n - 2);
// ...
What happens when the Continuation is not null? The generated code shows that path just as clearly.
Immediately after the first recursive call, we find this check:
call [Program:Fib(int):int:this]
mov r12d, eax
test rcx, rcx
jne SHORT SUSPEND
If rcx != null, the called Fib did not complete synchronously. The current invocation of Fib must suspend as well.
Only now does the JIT create the Continuation required to preserve the current execution state. It does not allocate one in advance merely because suspension might happen:
mov rdi, rcx
mov rsi, 0x... ; Continuation type
call [CORINFO_HELP_ALLOC_CONTINUATION]
mov r12, rax
It then saves the local state that will still be needed after resumption:
mov dword ptr [r12+0x48], ebx
Finally, it places the new Continuation in rcx and returns it to the caller according to the Async Calling Convention:
mov rcx, r12
ret
Unlike a Green Thread, a Runtime Async Continuation is small. It needs to store only a few pieces of information: local variables that remain live across an await, the resume point, and the result or exception from the awaited operation. This usually takes only a few dozen bytes. Compared with the roughly 2 KB initial stack of a Green Thread, it travels light.
Putting everything together, the C#-like pseudocode looks like this:
var (result1, continuation1) = Fib(null, n - 1);
if (continuation1 != null)
Suspend(continuation1);
var (result2, continuation2) = Fib(null, n - 2);
if (continuation2 != null)
Suspend(continuation2);
return result1 + result2;
In this Fibonacci example, however, every call completes synchronously. The normal path is just recursive calls, so its behavior is equivalent to the following synchronous code:
var result1 = Fib(n - 1);
var result2 = Fib(n - 2);
return result1 + result2;
The entire call chain allocates no Task objects and pays no state-machine overhead. The cost of the async/await abstraction vanishes, leaving code that runs like ordinary synchronous calls. This is fundamentally different from traditional async.
At this point, you may wonder why Program:Fib(int):System.Threading.Tasks.Task`1[int]:this still exists alongside Program:Fib(int):int:this.
Runtime Async uses the new Async Calling Convention internally, but ordinary C# code still sees the method as Task<int> Fib(int). A thin wrapper called an async thunk bridges the two conventions. In this example, it converts the ordinary result plus the Continuation returned by Runtime Async into the Task<int> expected by an external caller.
The thunk contains code similar to what we have already seen:
xor rsi, rsi
call [Program:Fib(int):int:this]
mov ebx, eax
test rcx, rcx ; Is Continuation null?
It first invokes the real Runtime Async method, then checks whether the returned Continuation is null. A null value means the method completed synchronously, so the thunk can wrap the result in a Task<int> and return it. If the thunk can eventually be inlined and the JIT can prove that the Task does not escape, escape analysis may eliminate even this allocation.
Benchmarks
Theory is useful, but numbers are more fun. The benchmark code is available in this GitHub Gist.
The benchmark covers the following cases:
-
Synchronous baselinedirectly calls an ordinary synchronous method. -
Async method, no suspension,Completed Task await, andCompleted ValueTask awaitcover asynchronous methods that complete without suspending. -
Task.Yield suspension,ThreadPool continuation, andTaskCompletionSource continuationsuspend for different reasons. -
Async state-machine chainbuilds a deep asynchronous call chain whose innermost method suspends withTask.Yield.
I compared Runtime Async (Async2) from the latest .NET 11 daily build available when this article was written against traditional async (Async1) on .NET 10. After warmup, each benchmark ran 100 million times. It is a somewhat brute-force approach, but it makes the differences hard to miss.
Below are the raw benchmark result data.
| Benchmark | Ops | Async1 Time/op | Async2 Time/op | Ratio | Async1 Throughput | Async2 Throughput | Async1 Total Alloc | Async2 Total Alloc | Async1 Bytes/op | Async2 Bytes/op | Async1 Gen0 | Async2 Gen0 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Synchronous baseline | 100.0M | 0.33 ns | 0.33 ns | 1.00× | 3.008B ops/s | 3.004B ops/s | 696 B | 696 B | 0 | 0 | 0 | 0 |
| Async method, no suspension | 100.0M | 6.58 ns | 0.34 ns | 19.63× | 152.0M ops/s | 2.984B ops/s | 7.20 GB | 696 B | 72.0000 | 0 | 459 | 0 |
| Completed Task await | 100.0M | 4.01 ns | 0.33 ns | 12.02× | 249.1M ops/s | 2.995B ops/s | 7.20 GB | 696 B | 72.0000 | 0 | 459 | 0 |
| Completed ValueTask await | 100.0M | 0.75 ns | 0.33 ns | 2.25× | 1.329B ops/s | 2.987B ops/s | 696 B | 696 B | 0 | 0 | 0 | 0 |
| Task.Yield suspension | 100.0M | 242.89 ns | 34.68 ns | 7.00× | 4.12M ops/s | 28.84M ops/s | 992 B | 1,000 B | 0 | 0 | 0 | 0 |
| ThreadPool continuation | 100.0M | 324.78 ns | 102.35 ns | 3.17× | 3.08M ops/s | 9.77M ops/s | 16.00 GB | 15.20 GB | 160.0000 | 152.0000 | 1,027 | 969 |
| TaskCompletionSource continuation | 100.0M | 455.50 ns | 114.16 ns | 3.99× | 2.20M ops/s | 8.76M ops/s | 16.00 GB | 16.00 GB | 160.0001 | 160.0000 | 1,027 | 1,021 |
| Async state-machine chain | 100.0M | 678.33 ns | 91.68 ns | 7.40× | 1.47M ops/s | 10.91M ops/s | 30.10 GB | 19.20 GB | 300.9802 | 192.0000 | 1,927 | 1,226 |
The results are striking. When no suspension occurs, Runtime Async almost completely removes the overhead of traditional async. It runs nearly 20 times faster and lands within rounding distance of the synchronous baseline.
The gains are not limited to the synchronous-completion path. The ThreadPool and TaskCompletionSource continuation benchmarks improved by roughly 3x to 4x. The deeper Async state-machine chain improved by 7.4x. The deeper the chain, the more room Runtime Async has to shine.
Runtime Async also allocated less memory in every benchmark. In the no-suspension tests, allocation fell to zero bytes per operation and no Gen0 collections occurred. The numbers confirm that the call chain created no Task objects and paid no state-machine overhead.
Closing thoughts
Runtime Async is a new asynchronous execution model arriving in .NET 11. Instead of having the C# compiler eagerly turn every async method into a state machine, it preserves the original asynchronous control flow until runtime, where the JIT can process and optimize it directly.
This finally delivers pay for play in the literal sense. If the code does not suspend, it pays almost no extra cost for being asynchronous. If it does suspend, it pays only for the state that must actually be preserved.
Async code that does not stop runs like synchronous code. When it does stop, it pays only for what it uses. That, in the end, is the whole idea behind Runtime Async.
Top comments (0)