Introduction to Roslyn Source Generator
Roslyn Source Generator is used in various repositories, such as dotnet-isolated-worker. Unfortunately, I have never worked with it and couldn't understand the code. So, I decided to write a simple sample application to try and understand it.
The challenge is to create a mock version of AOP (Aspect Oriented Programming). I wanted to see if I could write a sample that injects logs before and after the execution of a function based on defined attributes.
What are Roslyn APIs?
Source Generator is part of the Roslyn APIs (The .NET Compiler Platform SDK). The Roslyn APIs allow the evaluation of source code during compilation to create a model of the application. These APIs are used to manipulate that model. Specifically, the following can be done:
- Static code analysis
- Code fixes
- Code refactoring
- Code generation
By using these features, Visual Studio can provide code suggestions and assist with code refactoring. This feature enables support for maintaining coding conventions.
.NET executes a pipeline similar to the one shown above during compilation. APIs are provided to support this pipeline:
- Syntax Tree API: Access the tree model created by parsing the source code.
- Symbol API: Access the symbols created as a result of parsing the code.
- Binding and Flow Analysis APIs: Bind symbols to identifiers in the code.
- Emit API: Output assemblies.
Comparing what Visual Studio can do with Roslyn APIs makes it easier to understand the capabilities.
Source Generator
For this investigation, I want to focus on Source Generator.
It allows you to perform the following steps:
- Access the Compilation object to access and analyze the structure of the code.
- Generate source code and compile it to create an assembly.
Specifically, you can:
- Improve performance by performing tasks that are typically done with reflection using code generation.
- Manipulate the order of execution of tasks in MSBuild.
- Convert dynamic routing into static routing using code generation (used by ASP.NET).
Some use cases I have come across are:
- Creating a definition file for logging is cumbersome, so perform code analysis and generate patterns for all of them.
- Improve performance by generating code for methods that should be executed dynamically, referring to attributes.
How to use it
Create a source generator project
Reference the following libraries. Note that it seems to work only with netstandard2.0 currently.
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netstandard2.0</TargetFramework>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="4.7.0" PrivateAssets="all" />
<PackageReference Include="Microsoft.CodeAnalysis.Analyzers" Version="3.3.4" PrivateAssets="all" />
</ItemGroup>
</Project>
Create a generator
To create a generator, implement the ISourceGenerator interface. You need to implement the Initialize(GeneratorInitializationContext context) and Execute(GeneratorExecutionContext context) methods. You can access the current code model and perform code generation through the context objects in each method. You can access the existing model using the context.Compilation object, and perform code generation using the context.AddSource method.
You can read, analyze the model, generate code according to your preferences, and compile it. In the following example, a factory is automatically created.
namespace SourceGenerator
{
[Generator]
public class FactoryGenerator : ISourceGenerator
{
public void Execute(GeneratorExecutionContext context)
{
// How to access the CompilationObject
INamedTypeSymbol attributeSymbol = context.Compilation.GetTypeByMetadataName("AOPLib.FunctionAttribute");
// Create Factory
context.AddSource("FunctionFactory.g.cs", SourceText.From($@"
namespace AOPLib{{
public class FunctionFactory {{
public static IFunction CreateFunction(string name) {{
switch(name) {{
default:
return new SampleFunction();
}}
}}
}}
}}
", Encoding.UTF8));
}
public void Initialize(GeneratorInitializationContext context)
{
}
The caller would look like the following. We haven't created the code for the FunctionFactory class anywhere, but there will be no complaints from the IDE about the missing class.
public static void Main(string[] args)
{
var functionName = Console.ReadLine();
IFunction function = AOPLib.FunctionFactory.CreateFunction(functionName);
function.Execute(new FunctionContext() { Name = "hello", TraceId = Guid.NewGuid() });
}
In other words, the project containing the original Program.cs file, which acts as the caller, would have the following project definition. By setting OutputItemType=Analyzer and ReferenceOutputAssembly="false", the SourceGenerator project's DLL is not imported, and the analyzer is executed. You can find detailed definitions in Common MSBuild Project Items.
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net6.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\SourceGenerator\SourceGenerator.csproj"
OutputItemType="Analyzer"
ReferenceOutputAssembly="false"/>
</ItemGroup>
</Project>
This is the basic usage of Source Generator.
Debugging with Source Generator
Debugging
By default, it is not possible to debug Source Generators directly. However, you can enable debugging in Visual Studio by calling the following method in the Initialize method:
public void Initialize(GeneratorInitializationContext context)
{
Debugger.Launch();
}
Viewing Generated Source
By default, you cannot view the source code generated by the generator. However, you can enable it by adding the following definition to the csproj file of the project where the Program.cs is located. Add it to the PropertyGroup section:
<EmitCompilerGeneratedFiles>true</EmitCompilerGeneratedFiles>
Creating an AOP Sample
The previous example was a simple one, but it would be better to try a slightly more complex sample. So let's implement a fake AOP (Aspect-Oriented Programming) by defining an attribute called Function. If the Logging flag in the attribute is set to true, it automatically outputs logs when the method is executed.
For example, if we write a program like this:
namespace AOPSample
{
[Function(isLogging: true)]
public class LoggingFunction : IFunction
{
public void Execute(FunctionContext context)
{
Console.WriteLine($"[{nameof(LoggingFunction)}]: execute. Name: {context.Name}");
}
}
}
Since the isLogging attribute is set to true, the Source Generator will automatically find this file and inject the logging logic.
To make it easier, let's write an adapter like this:
public class LoggingAdapter : IFunction
{
private readonly IFunction _function;
public LoggingAdapter(IFunction function)
{
_function = function;
}
public void Execute(FunctionContext context)
{
Console.WriteLine($"[{DateTime.UtcNow}] {_function.GetType().Name}: before execute. TraceId: {context.TraceId}");
_function.Execute(context);
Console.WriteLine($"[{DateTime.UtcNow}] {_function.GetType().Name}: after execute. TraceId: {context.TraceId}");
}
}
Then, you can generate the code by creating an instance of LoggingAdapter and passing it to new ***Function().
Code Generation Logic
namespace SourceGenerator
{
[Generator]
public class FactoryGenerator : ISourceGenerator
{
public void Execute(GeneratorExecutionContext context)
{
// Get the symbol for the FunctionAttribute
INamedTypeSymbol attributeSymbol = context.Compilation.GetTypeByMetadataName("AOPLib.FunctionAttribute");
// Get all classes from Compilation.SyntaxTrees
var targetClasses = context.Compilation.SyntaxTrees
.SelectMany(st => st.GetRoot().DescendantNodes().OfType<ClassDeclarationSyntax>());
// Extract only the classes that have the FunctionAttribute
var selectedTargetClasses = targetClasses.Where(p => p.AttributeLists.FirstOrDefault()?.Attributes.FirstOrDefault()?.Name.NormalizeWhitespace().ToFullString() == "Function");
// Save information about the selected classes as metadata in a dictionary
var registeredClasses = new Dictionary<string, FunctionMetadata>();
// Check the information of the Attribute used in the class.
// Store the metadata information of the Function in the dictionary, such as `isLogging`.
foreach (var clazz in selectedTargetClasses)
{
var attribute = clazz.AttributeLists.First()?.Attributes.FirstOrDefault();
var attributeArgument = attribute.ArgumentList.Arguments.FirstOrDefault();
var isLogging = attributeArgument.Expression.NormalizeWhitespace().ToFullString().Contains("true");
var typeSymbol = context.Compilation.GetSemanticModel(clazz.SyntaxTree).GetDeclaredSymbol(clazz);
var className = typeSymbol.ToDisplayString(SymbolDisplayFormat.FullyQualifiedFormat);
var classNameOnly = typeSymbol.ToDisplayString(SymbolDisplayFormat.MinimallyQualifiedFormat);
registeredClasses.Add(classNameOnly, new FunctionMetadata { FullyQualifiedName = className, Name = classNameOnly, IsLogging = isLogging });
}
// Create the code generation template and generate the code
context.AddSource("FunctionFactory.g.cs", SourceText.From($@"
namespace AOPLib{{
public class FunctionFactory {{
public static IFunction CreateFunction(string name) {{
switch(name) {{
{GetFunctionsSection(registeredClasses)}
default:
return new SampleFunction();
}}
}}
}}
}}
", Encoding.UTF8));
}
public void Initialize(GeneratorInitializationContext context)
{
Debugger.Launch();
}
private string GetFunctionsSection(Dictionary<string, FunctionMetadata> metadata)
{
var sb = new StringBuilder();
foreach (var item in metadata)
{
sb.AppendLine($"case \"{item.Key}\":");
if (item.Value.IsLogging)
{
sb.AppendLine($"return new LoggingAdapter(new {item.Value.FullyQualifiedName}());");
}
else
{
sb.AppendLine($"return new {item.Value.FullyQualifiedName}();");
}
}
return sb.ToString();
}
}
class FunctionMetadata
{
public string Name { get; set; }
public string FullyQualifiedName { get; set; }
public bool IsLogging { get; set; }
}
}
Execution Result
Although no code was written in the main program, logs are automatically outputted as expected with the fake AOP.
If isLogging is set to false, no logs will be outputted.
IncrementalGenerators
Now, the interface I used in the above example is an official method, but it seems to be an older method. I received feedback from someone with worldwide expertise. It's amazing to receive feedback from such people when I write a blog!
https://x.com/neuecc/status/1703657375394341330?s=20
You should probably read the article C# in 2022 (Incremental) Source Generator Development Method and the official documentation on Incremental Generators. Let's explain the new interface.
IIncrementalGenerator
Compared to ISourceGenerator, it only has
Porting the AOP Sample
Now that we have learned the new approach, let's rewrite the AOP sample with Incremental Generators.
Great! It has become much easier to understand! I will add explanations within the code. The process follows the same flow as explained earlier: we process the IncrementalValue(s)Provider<T> using a Linq-like interface, manipulate it, and pass the final result to RegisterOutput to generate the files. The key point is to realize that it may look like Linq, but it's not actually Linq. By correctly reading about Incremental Generators, you will understand that it's not about handling collections, so it can also return a single value. That's where IncrementalValueProvider<T> comes in for singular values, and IncrementalValuesProvider<T> for multiple values.
I will add comments as needed throughout the code.
namespace SourceGenerator
{
[Generator(LanguageNames.CSharp)]
public class FactoryGenerator : IIncrementalGenerator
{
public void Initialize(IncrementalGeneratorInitializationContext context)
{
// Uncomment this line if you want to debug.
// Debugger.Launch();
// ExtensionMethod ForAttributeWithMetadataName() is awesome!
// It returns all classes with the specified attribute.
var source = context.SyntaxProvider.ForAttributeWithMetadataName(
"AOPLib.FunctionAttribute",
(node, token) => true,
(ctx, token) => ctx
)
// It can be written in a similar way to Linq. Here we analyze the classes and
// retrieve the values from the attributes attached to the classes and store
// them in FunctionMetadata.
.Select((s, token) => {
var typeSymbol = (INamedTypeSymbol)s.TargetSymbol;
var fullType = typeSymbol.ToDisplayString(SymbolDisplayFormat.FullyQualifiedFormat);
var name = typeSymbol.ToDisplayString(SymbolDisplayFormat.MinimallyQualifiedFormat);
var isLogging = (bool)(s.Attributes.FirstOrDefault()?.ConstructorArguments.FirstOrDefault().Value ?? false);
return new FunctionMetadata { FullyQualifiedName = fullType, Name = name, IsLogging = isLogging };
// Finally, we perform Collect() to convert it to IncrementalValueProvider.
// It was IncrementalValuesProvider<FunctionMetadata> before, but now it
// has been converted to IncrementalValueProvider<ImmutableArray<FunctionMetadata>>.
// We convert it from plural to singular to ensure the Action is only called once.
}).Collect();
context.RegisterSourceOutput(source, Emit);
}
// Since we have converted it to a singular form, this method will be called only once.
// Without Collect(), it would be called multiple times.
private void Emit(SourceProductionContext context, ImmutableArray<FunctionMetadata> source)
{
// Create Factory
context.AddSource("FunctionFactory.g.cs", SourceText.From($@"
namespace AOPLib{{
public class FunctionFactory {{
public static IFunction CreateFunction(string name) {{
switch(name) {{
{GetFunctionsSection(source)}
default:
return new SampleFunction();
}}
}}
}}
}}
", Encoding.UTF8));
}
private string GetFunctionsSection(ImmutableArray<FunctionMetadata> metadata)
{
var sb = new StringBuilder();
foreach (var item in metadata)
{
sb.AppendLine($"case \"{item.Name}\":");
if (item.IsLogging)
{
sb.AppendLine($"return new LoggingAdapter(new {item.FullyQualifiedName}());");
}
else
{
sb.AppendLine($"return new {item.FullyQualifiedName}();");
}
}
return sb.ToString();
}
}
class FunctionMetadata
{
public string Name { get; set; }
public string FullyQualifiedName { get; set; }
public bool IsLogging { get; set; }
}
}
Compared to the previous version, it looks much cleaner. By the way, let's compare the case where IIncrementalGenerator is available and when it's not, like this:
Summary
This generator enables us to execute code that would normally be done using reflection in a static manner, or analyze the structure of the code. It is a very useful and interesting feature. However, I have only scratched the surface, and I think it may be difficult to detect bugs in the generated code. It seems that it would be necessary to write unit tests or other means to cover those cases.
However, since I am still new to this, I can't say that I fully understand it yet. I will continue to investigate and write more samples.
I have provided the sample used in this blog post here:
Resources
This blog is translated from [更新] Source Generator を使って懐かしの AOP を作ってみる Powered by BlogBabel
Top comments (0)