Generic Methods
Generic methods in .NET are defined with one or more type parameters, allowing them to work with different data types while preserving type safety. They shine when we want code specialization without duplicating the source code for every type.
Consider a generic method that adds two values:
T Add<T>(T a, T b) where T : IAdditionOperators<T, T, T>
{
return a + b;
}
The method can be called with int, float, or any other type that implements IAdditionOperators<T, T, T>.
This flexibility is both powerful and efficient. For value-type instantiations, the JIT normally generates code specialized for the exact type. That lets it use the underlying operation directly, without boxing or runtime type checks. The generated code for int and float is correspondingly small:
Add[int](int,int):int:this (FullOpts):
lea eax, [rsi+rdx]
ret
Add[float](float,float):float:this (FullOpts):
vaddss xmm0, xmm0, xmm1
ret
As you can see, the int instantiation becomes an integer addition, while the float instantiation becomes a scalar floating-point addition. There is no generic dispatch left in either method.
If we have a caller that invokes this method with specific types, the JIT can go further and inline the calls. Suppose we call both instantiations with constants:
void Test()
{
Console.WriteLine(Add(4, 5));
Console.WriteLine(Add(4.0f, 5.0f));
}
The JIT can inline the two calls and then fold both additions. The resulting code contains only the two calls to Console.WriteLine:
Test():this (FullOpts):
push rax
mov edi, 9
call [System.Console:WriteLine(int)]
vmovss xmm0, dword ptr [reloc @RWD00]
add rsp, 8
tail.jmp [System.Console:WriteLine(float)]
RWD00 dd 41100000h ; 9
You can see that both calls to Add have been inlined. Their results are computed directly and passed to Console.WriteLine. Once the exact generic method is known, the abstraction disappears from the generated code.
Virtual Methods
Virtual methods in .NET provide polymorphic behavior. A base class can declare a virtual method:
class Base
{
public virtual void Display()
{
Console.WriteLine("Base Display");
}
}
Now let's create a derived class that overrides it:
class Derived : Base
{
public override void Display()
{
Console.WriteLine("Derived Display");
}
}
When we call Display on an instance of Derived, the overridden method in Derived is executed:
void Test()
{
Base obj = new Derived();
obj.Display(); // Output: Derived Display
}
Every virtual call normally has to select an implementation based on the object's actual runtime type. That dispatch is more expensive than a direct call.
However, a technique called devirtualization can remove that overhead. In the example above, the JIT can see that obj refers to a newly created Derived, so it can replace the virtual target with Derived.Display. Once the target is known, the JIT can inline it as well. The allocation is unused after inlining, so it disappears too:
Program:Test():this (FullOpts):
push rax
mov rdi, 0x7E62ECE02D58 ; 'Derived Display'
add rsp, 8
tail.jmp [System.Console:WriteLine(System.String)]
The JIT "sees" that the actual type of obj is Derived, replaces the virtual call with a direct call to Derived.Display, and then inlines it. The final code calls Console.WriteLine with "Derived Display" directly. Neither the object allocation nor the virtual call survives optimization.
Devirtualization is important for more than removing dispatch overhead. A direct target gives the inliner access to the callee, which often exposes much larger optimization opportunities in the caller.
There are many scenarios where the JIT can devirtualize virtual method calls to improve performance, and the set of scenarios is growing with each new version of .NET.
Generic Virtual Methods
Now let's consider generic virtual methods (GVMs).
Methods can be generic, and they can also be virtual. How about combining both? A generic virtual method has type parameters, but its implementation is also selected according to the receiver's runtime type.
For example, an interface can declare a generic method:
interface ICalculator
{
T Calc<T>(T a, T b) where T : IAdditionOperators<T, T, T>;
}
Then a class can provide the implementation:
class AddCalculator : ICalculator
{
public T Calc<T>(T a, T b) where T : IAdditionOperators<T, T, T>
{
return a + b;
}
}
You might think that the generated code would look similar to the earlier non-virtual generic method. Let's see what happens when we call it through the interface. The receiver is visibly an AddCalculator, and both method type arguments are concrete:
void Test()
{
ICalculator calc = new AddCalculator();
Console.WriteLine(calc.Calc(4, 5));
Console.WriteLine(calc.Calc(4.0f, 5.0f));
}
The generated code in .NET 10 tells a different story:
Program:Test():this (FullOpts):
push rbp
push rbx
push rax
lea rbp, [rsp+0x10]
mov rdi, 0x7A7265C4B8D0 ; AddCalculator
call CORINFO_HELP_NEWSFAST
mov rbx, rax
mov rdi, rbx
mov rsi, 0x7A7265C45A80 ; ICalculator
mov rdx, 0x7A7265D6A6C8 ; token handle
call [CORINFO_HELP_VIRTUAL_FUNC_PTR]
mov rdi, rbx
mov esi, 4
mov edx, 5
call rax
mov edi, eax
call [System.Console:WriteLine(int)]
mov rdi, rbx
mov rsi, 0x7A7265C45A80 ; ICalculator
mov rdx, 0x7A7265D6A8A0 ; token handle
call [CORINFO_HELP_VIRTUAL_FUNC_PTR]
vmovss xmm1, dword ptr [reloc @RWD00]
vmovss xmm0, dword ptr [reloc @RWD04]
mov rdi, rbx
call rax
nop
add rsp, 8
pop rbx
pop rbp
tail.jmp [System.Console:WriteLine(float)]
RWD00 dd 40A00000h ; 5
RWD04 dd 40800000h ; 4
Oh my gosh, that is quite a large piece of code just to compute the sum of two numbers. What happened here?
First, CORINFO_HELP_NEWSFAST allocates the AddCalculator object. The JIT keeps the resulting object reference in rbx because it is needed by both calls.
For each call, the generated code passes three pieces of information to CORINFO_HELP_VIRTUAL_FUNC_PTR: the receiver, the ICalculator interface, and a token that identifies the requested generic method instantiation. The helper resolves the correct implementation and returns its address in rax.
The program then invokes that address with call rax. This is an indirect call. The same resolution sequence runs again for the second instantiation because Calc<int> and Calc<float> are different GVM targets.
The helper calls and indirect calls are only the visible cost. The larger loss is that the JIT never gets a direct target for Calc<T>, so it cannot inline either implementation. As a result, it also misses the constant folding that reduced the earlier generic example to the value 9.
You might say, let's switch AddCalculator from a class to a struct and see whether that helps. Generic code can often call interface members on structs without boxing, so perhaps a struct will make the dispatch cheaper:
struct AddCalculator : ICalculator
{
public T Calc<T>(T a, T b) where T : IAdditionOperators<T, T, T>
{
return a + b;
}
}
The generated code in .NET 10 becomes:
Test():this (FullOpts):
push rbp
push rbx
push rax
lea rbp, [rsp+0x10]
mov rdi, 0x77030962B8B8 ; AddCalculator
call CORINFO_HELP_NEWSFAST
mov rbx, rax
mov byte ptr [rbx+0x08], 0
mov rdi, rbx
mov rsi, 0x770309625A80 ; ICalculator
mov rdx, 0x77030974A6C8 ; token handle
call [CORINFO_HELP_VIRTUAL_FUNC_PTR]
mov rdi, rbx
mov esi, 4
mov edx, 5
call rax
mov edi, eax
call [System.Console:WriteLine(int)]
mov rdi, rbx
mov rsi, 0x770309625A80 ; ICalculator
mov rdx, 0x77030974A8A0 ; token handle
call [CORINFO_HELP_VIRTUAL_FUNC_PTR]
vmovss xmm1, dword ptr [reloc @RWD00]
vmovss xmm0, dword ptr [reloc @RWD04]
mov rdi, rbx
call rax
nop
add rsp, 8
pop rbx
pop rbp
tail.jmp [System.Console:WriteLine(float)]
RWD00 dd 40A00000h ; 5
RWD04 dd 40800000h ; 4
It becomes even worse! The generated code is almost identical to the class implementation, but this version adds boxing. The local variable still has the interface type ICalculator, so assigning the struct to it boxes the value. CORINFO_HELP_NEWSFAST allocates the box, and the following store initializes its payload. After that, the two GVM calls go through the same helper-based resolution and indirect dispatch as before.
This does not mean interface calls on structs always box. A constrained call through a generic type parameter can preserve the value type and avoid boxing. In this particular example, converting the value to ICalculator erases that opportunity before either call occurs.
An ordinary virtual call primarily depends on the receiver type and the virtual slot. A GVM call has another dimension: the method instantiation. Resolving Calc<int> and resolving Calc<float> may produce different entry points, even though both calls use the same receiver and interface method.
The .NET runtime already knows how to resolve this combination at runtime. That is what CORINFO_HELP_VIRTUAL_FUNC_PTR does in the generated code above. The JIT optimization problem is proving during compilation that a particular receiver, interface method, and generic instantiation lead to one exact target.
If the JIT can establish that target, the entire chain can collapse:
- The runtime GVM lookup is no longer needed.
- The indirect call becomes a direct call.
- The direct call can be inlined, exposing the method body to further optimization.
However, the first step was not happening in .NET 10, and the following steps were blocked as a result. Even though the JIT knew that the receiver was an AddCalculator, it could not resolve that information to the exact GVM target. The generated code still needed the helper-based lookup and indirect call, so the inliner never got a chance to see the method body.
Devirtualization of Generic Virtual Method Calls
Now that we understand the problem, let's do something about it in .NET 11.
A GVM call can take several different shapes:
- The receiver can be a class or a struct.
- The instantiation can be a value type or a reference type.
- The exact generic context can be known at compile time or require a runtime lookup.
The first dimension is easy to understand. When a struct is boxed and called through an interface, dispatch may lead to a special piece of code called an unboxing stub. The stub adjusts the boxed object's this pointer so that it points to the struct payload, then transfers control to the actual method body. If the JIT devirtualizes the call, it can target the unboxed entry point directly and may eliminate the box as well.
The second dimension exists because a generic method can be instantiated with either a value type or a reference type. The JIT normally specializes the code for value types, while reference-type instantiations normally share the same native code.
This is called shared generic code. If the type parameter is a reference type, the JIT normally generates one copy of the method body and shares it among all reference-type instantiations.
The shared instantiation is internally represented using a special type called System.__Canon. Because the native code is shared, the callee sometimes needs a hidden generic context argument that identifies the exact instantiation. The method body can use this context to perform type-specific operations.
For example, consider the following shared generic method calls:
void Test()
{
Console.WriteLine(GetType<string>());
Console.WriteLine(GetType<object>());
}
[MethodImpl(MethodImplOptions.NoInlining)]
Type GetType<T>()
{
return typeof(T);
}
There will be only one copy of GetType<T> for all reference types, and the JIT will pass a hidden generic context argument for each exact instantiation. The generated code looks like this:
Test():this (FullOpts):
push rbp
push rbx
push rax
lea rbp, [rsp+0x10]
mov rbx, rdi
mov rdi, rbx
mov rsi, 0x7F694233AAC8 ; Program:GetType[System.String]():System.Type:this
call [Program:GetType[System.__Canon]():System.Type:this]
mov rdi, rax
call [System.Console:WriteLine(System.Object)]
mov rdi, rbx
mov rsi, 0x7F694233AB50 ; Program:GetType[System.Object]():System.Type:this
call [Program:GetType[System.__Canon]():System.Type:this]
mov rdi, rax
add rsp, 8
pop rbx
pop rbp
tail.jmp [System.Console:WriteLine(System.Object)]
GetType[System.__Canon]():System.Type:this (FullOpts):
push rax
mov qword ptr [rsp], rsi
mov rdi, qword ptr [rsi+0x18]
mov rdi, qword ptr [rdi]
call CORINFO_HELP_TYPEHANDLE_TO_RUNTIMETYPE
nop
add rsp, 8
ret
As you can see, the caller passes a hidden argument in rsi to identify the exact instantiation of GetType<T>. In this case, that argument is a method context from which the shared method body retrieves the actual type argument.
This means that devirtualizing a GVM call is not always as simple as changing the target. After turning the GVM call into a direct call, the JIT must also supply the correct hidden context argument when the target uses shared generic code.
Now let's shift to the third dimension. In the example above, the type arguments are known while the caller is being compiled, so the JIT can emit calls to GetType<string> and GetType<object> directly. If the exact context is available only at runtime, however, the JIT must look up the required target or context dynamically. Internally, this is called a runtime lookup.
Improvements in .NET 11
Generic Virtual Method Devirtualization
I have spent almost a year working on this problem, and we have made great progress so far.
Remember the three dimensions from the previous section? In .NET 11, we have improved the first two.
The JIT can now devirtualize GVM calls when the receiver is either a class or a struct, and when the method is instantiated with either value types or reference types. The third dimension, where a runtime lookup is required, is still being worked on and is planned for .NET 12.
Now let's look at the generated code for the same example in .NET 11.
TestClass():this (FullOpts):
push rax
mov edi, 9
call [System.Console:WriteLine(int)]
vmovss xmm0, dword ptr [reloc @RWD00]
add rsp, 8
tail.jmp [System.Console:WriteLine(float)]
RWD00 dd 41100000h ; 9
TestStruct():this (FullOpts):
push rax
mov edi, 9
call [System.Console:WriteLine(int)]
vmovss xmm0, dword ptr [reloc @RWD00]
add rsp, 8
tail.jmp [System.Console:WriteLine(float)]
RWD00 dd 41100000h ; 9
The TestClass and TestStruct methods call Calc on a class and a struct, respectively. As you can see, the generated code is now optimal. The GVM calls have been devirtualized, and the box for the struct has been eliminated. The JIT can inline Calc and fold both additions, so the value 9 is passed directly to Console.WriteLine.
This also works when the method is instantiated with a reference type. string does not fulfill the IAdditionOperators<T, T, T> constraint, so let's change the example.
Suppose we have a generic virtual method that parses an incoming string as the destination type:
void Test()
{
ICanParse p1 = new ClassParser();
Console.WriteLine(p1.Parse<string>("test"));
ICanParse p2 = new StructParser();
Console.WriteLine(p2.Parse<string>("test"));
}
interface ICanParse
{
T Parse<T>(string s) where T : IParsable<T>;
}
class ClassParser : ICanParse
{
public T Parse<T>(string s) where T : IParsable<T>
{
return T.Parse(s, null);
}
}
struct StructParser : ICanParse
{
public T Parse<T>(string s) where T : IParsable<T>
{
return T.Parse(s, null);
}
}
Back in .NET 10, the generated code for Test was:
Test():this (FullOpts):
push rbp
push r15
push r14
push rbx
push rax
lea rbp, [rsp+0x20]
mov rdi, 0x79D16F05B8D0 ; ClassParser
call CORINFO_HELP_NEWSFAST
mov rbx, rax
mov rdi, rbx
mov r15, 0x79D16F05B800 ; ICanParse
mov rsi, r15
mov rdx, 0x79D16F160A10 ; token handle
call [CORINFO_HELP_VIRTUAL_FUNC_PTR]
mov rdi, rbx
mov rbx, 0x79D16C202680 ; 'test'
mov rsi, rbx
call rax
mov rdi, rax
call [System.Console:WriteLine(System.String)]
mov rdi, 0x79D16F05B9B8 ; StructParser
call CORINFO_HELP_NEWSFAST
mov r14, rax
mov byte ptr [r14+0x08], 0
mov rdi, r14
mov rsi, r15
mov rdx, 0x79D16F160A10 ; token handle
call [CORINFO_HELP_VIRTUAL_FUNC_PTR]
mov rdi, r14
mov rsi, rbx
call rax
mov rdi, rax
add rsp, 8
pop rbx
pop r14
pop r15
pop rbp
tail.jmp [System.Console:WriteLine(System.String)]
As you can see, the runtime resolution and indirect calls are still there, and the StructParser is boxed before its interface call.
In .NET 11, the generated code for Test becomes:
Test():this (FullOpts):
push rbx
mov rbx, 0x70FBB5202A70 ; 'test'
mov rdi, rbx
call [System.Console:WriteLine(System.String)]
mov rdi, rbx
pop rbx
tail.jmp [System.Console:WriteLine(System.String)]
The code is now optimal. Both calls end up passing the string literal directly to Console.WriteLine.
Devirtualization for Interface Default Methods in a Generic Context
The same work also applies to default interface methods (DIMs) declared on generic interfaces. Consider the following code:
void Test()
{
IGenericA<string> a = new AImpl();
a.B("GVM");
a.C("Generic DIM");
}
interface IA
{
void A(object o);
void B<T>(T o);
}
interface IGenericA<T> : IA
{
void C(T o) => B(o);
}
class AImpl : IGenericA<string>
{
public void A(object o)
{
Console.WriteLine(o);
}
public void B<T>(T o)
{
Console.WriteLine(o);
}
}
In .NET 10, the generated code was:
Test():this (FullOpts):
push rbx
sub rsp, 32
mov rcx, 0x7FFB383AF518
call CORINFO_HELP_NEWSFAST
mov rbx, rax
mov rcx, rbx
mov rdx, 0x7FFB383AF348
mov r8, 0x7FFB383AF828
call CORINFO_HELP_VIRTUAL_FUNC_PTR
mov rcx, rbx
mov rdx, 0xA025C36200
call rax
mov rcx, rbx
mov r11, 0x7FFB380E0070
mov rdx, 0xA025C36220
call [r11]IGenericA`1[System.__Canon]:C(System.__Canon):this
nop
add rsp, 32
pop rbx
ret
In .NET 11, it becomes:
Test():this (FullOpts):
sub rsp, 40
mov rcx, 0xA0261EE6F0 ; 'GVM'
call [System.Console:WriteLine(System.Object)]
mov rcx, 0xA0261EE710 ; 'Generic DIM'
call [System.Console:WriteLine(System.Object)]
nop
add rsp, 40
ret
All those indirections are gone, and the generated code is now optimal.
A Small Low-Hanging Fruit: where T : new()
How many of you have used where T : new() in your generic code?
Inside a generic method with a new() constraint, new T() is compiled as a call to Activator.CreateInstance<T>(). I also contributed a change to .NET 11 that teaches the JIT that the result of Activator.CreateInstance<T>() has the exact type T. The JIT can then devirtualize subsequent calls on the returned object.
The Final Results in .NET 11
I created a benchmark to compare the performance and allocation costs of GVM calls in .NET 10 and .NET 11. The benchmark and its results are available in a GitHub Gist.
Here is the generated-code difference between .NET 10 and .NET 11:
G_M000_IG01: ;; offset=0x0000
- push rdi
- push rsi
- push rbp
push rbx
- sub rsp, 40
+ sub rsp, 32
-G_M000_IG02: ;; offset=0x0008
- mov rcx, 0x7FFCFE6A32B8
- call CORINFO_HELP_NEWSFAST
- mov rbx, rax
- mov byte ptr [rbx+0x08], 0
- mov rcx, 0x7FFCFE67EBE0
- call CORINFO_HELP_NEWSFAST
- mov rsi, rax
- mov rcx, rbx
- mov rdi, 0x7FFCFE6A2AC8
- mov rdx, rdi
- mov r8, 0x7FFCFE6A35C8
- call CORINFO_HELP_VIRTUAL_FUNC_PTR
- mov rcx, rbx
- mov rdx, rsi
- call rax
- mov rdx, rax
- mov rcx, rbx
- mov r11, 0x7FFCFE3B0100
- call [r11]Test.IFooConsumer`1[System.__Canon]:Consume(System.__Canon):this
- mov rsi, 0x7FFCFE6A3690
- mov rcx, rsi
- call CORINFO_HELP_NEWSFAST
- mov rbp, rax
- mov rcx, rbx
- mov rdx, rdi
- mov r8, 0x7FFCFE6A38F8
- call CORINFO_HELP_VIRTUAL_FUNC_PTR
- mov rcx, rbx
- mov rdx, rbp
- call rax
- mov rdx, rax
- mov rcx, rbx
- mov r11, 0x7FFCFE3B0108
- call [r11]Test.IFooConsumer`1[System.__Canon]:Consume(System.__Canon):this
- mov rcx, rsi
- call CORINFO_HELP_NEWSFAST
- mov rdx, rax
- mov rcx, rbx
- mov r11, 0x7FFCFE3B0110
- call [r11]Test.IFooConsumer`1[System.__Canon]:Consume(System.__Canon):this
- mov rcx, rsi
- call CORINFO_HELP_NEWSFAST
- mov rbp, rax
- mov rcx, rbx
- mov rdx, rdi
- mov r8, 0x7FFCFE6A38F8
- call CORINFO_HELP_VIRTUAL_FUNC_PTR
- mov rcx, rbx
- mov rdx, rbp
- call rax
- mov rdx, rax
- mov rcx, rbx
- mov r11, 0x7FFCFE3B0118
- call [r11]Test.IFooConsumer`1[System.__Canon]:Consume(System.__Canon):this
- mov rcx, rsi
- call CORINFO_HELP_NEWSFAST
- mov rdx, rax
- mov rcx, rbx
- mov r11, 0x7FFCFE3B0128
- call [r11]Test.IFooConsumer`1[System.__Canon]:Consume(System.__Canon):this
- mov rcx, rsi
- call CORINFO_HELP_NEWSFAST
- mov rdx, rax
- mov rcx, rbx
- mov r11, 0x7FFCFE3B0138
-
-G_M000_IG03: ;; offset=0x0138
- call [r11]Test.IFooConsumer`1[System.__Canon]:Consume(System.__Canon):this
+G_M000_IG02: ;; offset=0x0005
+ mov rcx, 0x7FFC55594270
+ call [System.Activator:CreateInstance[System.__Canon]():System.__Canon]
+ cmp byte ptr [rax], al
+ mov rcx, 0x7FFC555943B0
+ mov rdx, 0x224E7636488
+ call [Test.Program:Use[System.__Canon](System.__Canon)]
+ mov rcx, 0x7FFC555943B0
+ mov rdx, 0x224E7636488
+ call [Test.Program:Use[System.__Canon](System.__Canon)]
+ mov rcx, 0x7FFC55594530
+ call [System.Activator:CreateInstance[System.__Canon]():System.__Canon]
+ cmp byte ptr [rax], al
+ mov rcx, 0x7FFC555943B0
+ mov rbx, 0x224E76364D0
+ mov rdx, rbx
+ call [Test.Program:Use[System.__Canon](System.__Canon)]
+ mov rcx, 0x7FFC555943B0
+ mov rdx, rbx
+ call [Test.Program:Use[System.__Canon](System.__Canon)]
+ mov rcx, 0x7FFC555943B0
+ mov rdx, rbx
+ call [Test.Program:Use[System.__Canon](System.__Canon)]
+ mov rcx, 0x7FFC55594530
+ call [System.Activator:CreateInstance[System.__Canon]():System.__Canon]
+ cmp byte ptr [rax], al
+ mov rcx, 0x7FFC555943B0
+ mov rdx, rbx
+ call [Test.Program:Use[System.__Canon](System.__Canon)]
+ mov rcx, 0x7FFC555943B0
+ mov rdx, rbx
+ call [Test.Program:Use[System.__Canon](System.__Canon)]
+ mov rdx, 0x224E76364D0
+ mov rcx, 0x7FFC555943B0
+ call [Test.Program:Use[System.__Canon](System.__Canon)]
+ mov rdx, 0x224E76364D0
+ mov rcx, 0x7FFC555943B0
+ call [Test.Program:Use[System.__Canon](System.__Canon)]
nop
-G_M000_IG04: ;; offset=0x013C
- add rsp, 40
+G_M000_IG03: ;; offset=0x010D
+ add rsp, 32
pop rbx
- pop rbp
- pop rsi
- pop rdi
- ret
As you can see, the indirect virtual calls and boxing are gone. The JIT can now devirtualize the GVM calls, inline their method bodies, and perform further optimizations.
On my machine, .NET 10 took about 6,972 milliseconds and allocated 240 bytes per iteration. With .NET 11, the same benchmark took about 2,498 milliseconds and allocated only 72 bytes per iteration.
That is a 2.79x speedup, while allocation drops by 3.33x.
I would also like to thank the .NET runtime team for their tremendous help throughout this work.
All the above improvements are now available in .NET 11 Preview 7, while more improvements including cases that involve a runtime lookup will likely come in .NET 12. These improvements not only apply to the JIT, but also apply to R2R and NativeAOT.
You can try it out and see the difference yourself!
Top comments (0)