DEV Community: eugenie32b

SmartTraits or lets add «multiple inheritance» to C#

eugenie32b — Mon, 29 Mar 2021 02:04:46 +0000

One of our clients, a developer who used to work with php technologies for quite some time, frequently complained that with the transition to C# and .Net stack, he misses one of his favorite features from the php world - traits and he would like it very much to be able to use such functionality in .Net.

At one point, we decided to make him a present and implemented a proof of concept of similar functionality for C#.

To our surprise, it was quite easy to implement the PoC and the process of development was a lot of fun.

A brief description of what the traits are (for those of us who are lucky to never work with php). Extract from the php.net site:

Traits are a mechanism for code reuse in single inheritance languages such as PHP. A Trait is intended to reduce some limitations of single inheritance by enabling a developer to reuse sets of methods freely in several independent classes living in different class hierarchies. The semantics of the combination of Traits and classes is defined in a way which reduces complexity, and avoids the typical problems associated with multiple inheritance and Mixins.

A Trait is similar to a class, but only intended to group functionality in a fine-grained and consistent way. It is not possible to instantiate a Trait on its own. It is an addition to traditional inheritance and enables horizontal composition of behavior; that is, the application of class members without requiring inheritance.

Many programming languages (including C#) do not support multiple inheritance to avoid the complexity and ambiguity it brings into a development process. But in certain (keep-it-simple) scenarios, it could be quite useful and would reduce the overall project complexity and the size of the codebase. Especially in cases where you have no control over the choice of the base classes and the ability to create a proper class hierarchy.

Recent versions of C# have added support for default interface methods to address these issues, but those methods have some limitations.

After discussion about scope of the PoC, we came up with the minimum requirements for traits functionality:

static typing
integration directly into a project (i.e. not pre / post-processing)
IDE (VisualStudio) compatibility with all its features: autocompletion, error detection during code editing, etc.,
minimum additional effort from a developer

When source generators start to gain popularity, it became clear that they could be a good candidate and might be able to meet our requirements and it would makes sense to try to implement the PoC.

And after couple of weeks of development time, we finally created a tool called SmartTraits.

There are a lot articles on how to implement source generators, so I will not go into details of the implementation of the source generator itself, but will describe the general ideas of the project.

We implemented the functionality in several stages and I will write about them based on the implementation timeline, but first, I would like to describe the conventions we use for the Trait and its destination classes.

Trait is

an abstract class that is declared as partial
the class is marked with the Trait attribute
this class cannot be inherited from another class
the class specification may include an interface that it must implement

Trait destination class (i.e. the class to which the source code from Trait will be added)

must be partial
must include one or more [AddTrait] attribute(s) with a trait type as a parameter

Note: All examples are simplified on purpose, to demonstrate principles of the idea. We are aware that there are alternative ways to meet these requirements (for example, our aspectimum tool).

For example, let's take the simplest class hierarchy

class ExampleA: BaseA 
{}

class BaseA {}

class ExampleB: BaseB
{}

class BaseB {}

Classes ExampleA and ExampleB are derived from different base classes and they do not have a common ancestor (for example base classes from third-party libraries) and as a result, we cannot easily add the required common functionality to all of these classes for one source.

But when unexperienced developer meet such challenge, he/she will use his/her proven solution, copy & paste command and as a result, will get all the consequences it usually brings into the project.

Below is description of iterations that we had during development of our source generator.

(Small disclaimer before we continue - as you may notice, English is not my native language, so please forgive me for any mistakes.)

First iteration - "glorified" #include

Suppose that to the both classes ExampleA and ExampleB in our example above, you need to add code to work with names

public string FirstName { get; set; }

public string LastName { get; set; }

public string GetFullName() { 
   return $"{FirstName} {LastName}"; 
}

First, we create our simplest Trait class

[Trait]
abstract partial class NamesTrait
{
      public string FirstName { get; set; }

      public string LastName { get; set; }

      public string GetFullName() { 
         return $"{FirstName} {LastName}"; 
      }
}

And apply the trait to the classes ExampleA and ExampleB by adding the [AddTrait] attribute

[AddTrait(typeof(NamesTrait))]
partial class ExampleA: BaseA 
{}

That is it, nothing else is required, and our source generator "behind the scene" automatically creates the second part for the ExampleA and ExampleB classes, copying the sources of all methods, properties, etc. from NamesTrait to both of them.

Autogenerated part:

partial class ExampleA
{
     public string FirstName { get; set; }

     public string LastName { get; set; }

     public string GetFullName() { 
        return $"{FirstName} {LastName}"; 
    }
}

The most basic requirements have been met, even at the very first step.

not different from the rest of the code and all the rules that are applied to regular c# code are fully applicable to Traits as well
quite small overhead
full IDE support
immediate changes to ExampleA and ExampleB when the code changes in the NamesTrait

The second iteration is a guarantee of the implementation of the required methods and properties by a trait

We can achieve guarantees in the standard way, by creating an interface and assigning it to both - the Trait class and the destination class.

Defining the interface

interface INames
{
    string FirstName { get; set; }
    string LastName { get; set; }
    string GetFullName();
}

Destination class declarations will look like (added INames interface)

partial class ExampleA: BaseA, INames

The Trait class declaration (we also add the INames interface)

abstract partial class NamesTrait: INames

Now the code will not compile until all the required methods / properties are implemented.

Third iteration - conflict resolution

By default, all the contents of the Trait class will be copied to the destination class, but lets suppose that for some destination classes we would like to be able to implement their own versions of methods, properties, etc.

The solution to achieve this requirement is quite simple, we mark Trait members that could be implemented in destination classes with the [Overrideable] attribute and when do copying from the Trait class, we check if a similar method or property is already implemented by the destination class and in this case do not copy the version from Trait.

If ExampleB has its own implementation of the GetFullName method, then we ignore the version of this method from the NamesTrait

[AddTrait(typeof(NamesTrait))]
partial class ExampleB: BaseB, INames 
{
    public string MiddleName { get; set; }

    public string GetFullName() { 
       return $" {FirstName} {MiddleName} {LastName}"; 
    }
}

Below is a generated part for class ExampleB

partial class ExampleB
{
    public string FirstName { get; set; }

    public string LastName { get; set; }
}

In the NamesTrait class, the GetFullName method looks like (added [Overrideable] attribute)

[Overrideable]
public string GetFullName() { 
   return $"{FirstName} {LastName}"; 
}

Fourth iteration - accessing methods and properties of the destination class from Trait methods

If we know that all destination classes implement some methods or properties that we need to call from a Trait, then there are two ways to achieve this functionality.

In the Trait we add a mock and ignore this mock during copying process. For private and protected methods and properties, this is the only simple option we could think of. To achieve ignore copying of class members, you can mark them with the [TraitIgnore] attribute.

It was suggested that we can use abstracted methods / properties
instead of mocks and ignoring them when copied. And if we still create mocks, it would be better to throw an exception instead of returning default values. This is a more correct and safe option.

In addition to the mock option, for public methods can achieve this requirement in a more elegant way. We create the interface (s) and assign them to both Trait and the destination class. By doing this, we guarantee that there are interface members available in both classes. Also, for these interface members, we can automatically generate a mock for the Trait class. In this case, automatic source generation helps us for both, a destination class and for the Trait itself.

Suppose both ExampleA and ExampleB have a GetNamePrefix (): string method and we would want to be able to call this method from the Trait.

Option 1. Mock

[AddTrait(typeof(NamesTrait))]
partial class ExampleA
{
    private string GetNamePrefix()
    {
        return "mr/mrs. ";
    } 
}

The Trait declaration (you can see the use of GetNamePrefix () in the GetFullName function) and the definition of the GetNamePrefix mock

[Trait]
abstract partial class NamesTrait: INames
{
    public string FirstName { get; set; }

    public string LastName { get; set; }

    public string GetFullName() { 
       return $"{GetNamePrefix()} {FirstName} {LastName}"; 
    }


    [TraitIgnore]
    private string GetNamePrefix()
    {
       throw(new Exception());
    }
}

The generated code for ExampleA will look like

partial class ExampleA: INames
{
    public string FirstName { get; set; }

    public string LastName { get; set; }

    public string GetFullName() { 
       return $"{GetNamePrefix()} {FirstName} {LastName}"; 
    }
}

Option 2. Use of public methods / properties of the destination class via interfaces

We define an interface that the destination class implements (in this example, it is INamePrefix)

interface INamePrefix
{
    string GetNamePrefix();
}

[AddTrait(typeof(NamesTrait))]
partial class ExampleA: INames, INamePrefix
{
    public string GetNamePrefix()
    {
        return "mr/mrs. ";
    } 
}

And assign it to the Trait class

[Trait]
abstract partial class NamesTrait: INames, INamePrefix
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public string GetFullName() { 
       return $"{GetNamePrefix()} {FirstName} {LastName}"; 
     }
}

The generated code for the destination class (Example) will look like

partial class ExampleA: INames, INamePrefix
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public string GetFullName() { 
       return $"{GetNamePrefix()} {FirstName} {LastName}"; 
     }
}

And the automatically generated code for the Trait class will look like

abstract partial class NamesTrait: BaseMockNamesTrait
{
}

abstract class BaseMockNamesTrait: INamePrefix
{
    public abstract string GetNamePrefix();
}

Note: in order to distinguish the interfaces for accessing the members of the destination class (in this example it is INamePrefix) for which it is necessary to generate a mock, from the interface that guarantees the trait contract (in the example it is INames) we introduce an additional condition that the Trait interface of the contract is marked with the attribute [TraitInterface].

Fifth iteration - strict mode

We've included additional support for strict mode to prevent surprises and to reduce the likelihood of introducing of tricky bugs. In the strict mode, the code of Trait and the destination classes must implement the interface marked with the [TraitInterface] attribute. Plus the Trait class must implement ONLY methods and properties from this interface. No additional methods / properties are allowed.

An example of setting strict mode

[Trait(TraitOptions.Strict)]
abstract partial class NameTrait : IName

If you add a method / property that does not exist in the IName interface, there will be an error during compilation and you will not be able to build the project.

Side notes

After five iterations, it turned out to be quite a useful tool, but the curious reader from the very beginning is asking a question of why we added the word "smart" to the project name. So far, it is clearly not enough to be called SmartTraits - I completely agree, so there are still a couple of iterations ahead.

Sixth iteration - smart methods

Working with source generators, we have access to the source code of the project. It is how we were able to implement the Traits functionality.

But if you think about it, if we have the source code, we can copy it, but we also can execute it.

Therefore, we have added a new feature, methods that is

marked with the Process attribute
take ClassDeclarationSyntax object as a parameter
and return string

Will be compiled and executed on the fly. The ClassDeclarationSyntax node of the destination class will be passed to the method as a parameter.

The result of the method execution will be added to the destination class code. If the optional attribute parameter is specified as ProcessOptions.Global, then it will be added not to the class code, but as a separate generated code. This will allow generating additional classes that implement the required functionality. When you start working with this functionality, at first, it really feels like some kind of magic.

Smart methods are mostly useful for generating code depending on external data (api / db / xml accesses, etc.). But you need to understand that this can be quite resource consuming and you need to implement the correct strategy for caching and invalidation.

An example of such code (marked with the [Process] attribute). For the ExampleA class, the GetExampleA method will be generated, and for ExampleB class, will be generated GetExampleB method

[Trait]
abstract partial class NamesTrait: INames, INamePrefix
{
    [Process]
    string BuildMethod(ClassDeclarationSyntax classNode)
    {
        return $"public string Get{classNode.Identifier}() {{ return \"{classNode.Identifier}\"; }}";
    }
}

For example, the generated code in the destination class ExampleA will look like

partial class ExampleA: INames, INamePrefix
{
    public string GetExampleA()
    {
        return "ExampleA";
    }
}

And for ExampleB like

partial class ExampleB: INames, INamePrefix
{
    public string GetExampleB()
    {
        return "ExampleB";
    }
}

Seventh iteration - even smarter

The ability to execute custom code on the fly that immediately changes the code of the same project is definitely cool, but it requires certain qualifications from a developer who writes such code. Generating code by manipulating and navigating through Roslyn takes some getting used to.

Therefore, it was decided to simplify and expand the feature by adding support for executing T4 templates during the source generation process. After all, the T4 was created just for this. It is easy to generate source code and even if a developer has never written T4 scripts before, learning of how to do it, is much easier than manipulating objects using Roslyn.

If you've written aspx or jsp code before, you may consider that you already know T4. Or if you wrote pages in php, the basic idea is the same. It was especially convenient in our case, taking into account the fact that our customer was from the PHP clan.

The idea is the same as in the previous case. We can mark any method, property or class with the [ApplyT4] attribute. It doesn't even need to be a member of the [AddTrait] destination class.

The only difference is that by default the result of template execution will be included as a separate generated code, i.e. T4TemplateScope.Global. But if desired, for the members of the destination class (class with the [AddTrait] attribute), you can specify the T4TemplateScope.Local option and the result of the template will be added to the partial part of the generated destination class.

Due to the fact that the compiled version of the template is cached, execution is very fast. When the T4 template is changed, it sees the changes immediately and is recompiled on the fly and the new version is applied.

It is very convenient in comparison with standard source generators, where on any tiny change, you need to build a new assembly and redeploy the nuget package.

After six months of work, we collected statistics that the version with Process (compilation of embedded code on the fly) is actually not used and it was disabled, leaving only the T4 version.

Some developers who do not require the functionality of the SmartTraits package, have installed a generator and are using it only because of the synergy between source generators and T4, because of the ability to recompile on the fly and seen instant results.

Here is how we define templates:

[AddTrait(typeof(NameTrait))]
[ApplyT4("DemoTemplate", Scope = T4TemplateScope.Local, VerbosityLevel = T4GeneratorVerbosity.Debug)]
partial class ExampleA : BaseA, IName
{
}

class BaseA { }

Sample T4 template:

<#@ include file="SmartTraitsCsharp.ttinclude" #>
<#
    // for this demo, we ignore empty nodes, but in real life you would want the template to fail, to be able to catch issues earlier
    if(ClassNode?.Identifier == null)
        return "";
#>

public string GetT4<#= ClassNode.Identifier.ToString() #>() 
{
    return "T4 <#= ClassNode.Identifier.ToString() #>"; 
}

Generated Code for ExampleA

partial class ExampleA {
    public string GetT4ExampleA() 
    {
        return "T4 ExampleA"; 
    }
}

Generated Code for ExampleB

partial class ExampleB {
    public string GetT4ExampleB() 
    {
        return "T4 ExampleB"; 
    }
}

Conclusion

This was a short description of what was done to demonstrate the capabilities of the source generators technology.

We have published the source code of this proof of concept under a MIT license - in case if anyone might be interested and decide to build a product that could be used for production ready code.

Thinking out loud

In my opinion, such functionality could even be added into the C# language itself, a couple of keywords and it would be ready to use. Based on the experience of this project, everything turns out to be quite convenient and useful.

I would also suggest to guys from Microsoft to add an option so that you can mark your analyzer or generator as "heavy" - i.e.it would be executed only on builds, and not on every tiny change of the code. In certain cases, I would prefer that there was no instant update of the generated files, but much more resource-intensive checks could be added to the analyzer/source generator without being afraid to halt the IDE for a minute.

How to try it?

The project code can be found in the repository

Packages are also available at nuget.org

I want to warn you - the development of this version was stopped right after the demo to the client and after that it was never updated. So this is PoC/MVP with the quality of the code of PoC and with a high probability of multiple bugs. Though, before publishing the article, I have tested this code, and the cases from the demo project were working just fine.

In order for the project to write the generated code to the file system, you need to add to the project file:

<EmitCompilerGeneratedFiles>true</EmitCompilerGeneratedFiles>
<CompilerGeneratedFilesOutputPath>$(BaseIntermediateOutputPath)\GeneratedFiles</CompilerGeneratedFilesOutputPath>

I strongly recommend updating to the latest version of VisualStudio and Resharper before trying the project, source generators are new technology and old versions of the tools are not working with it properly.

Thanks for your time.

Aspect-Oriented Programming (AOP) by source-level weaving

eugenie32b — Mon, 05 Oct 2020 17:14:10 +0000

Aspect-oriented programming is a very appealing concept for simplifying your codebase, creating clean code, improving modularity, structure of code and minimizing copy-paste errors.

Today, in most cases, weaving aspect's advices is implemented at the bytecode level, i.e. after compilation, a certain tool «weave» an additional byte code with the support of the required logic.

Our approach (as well as the approach of some other tools) is modifying the source code to implement aspect logic. With introduction of the .NET Compiler Platform (aka Roslyn), it is quite easy to achieve this goal, and the result gives certain advantages over the modification of the byte code itself.

You may think that aspect-oriented programming is not for you and does not particularly concern you, just a bunch of incomprehensible words, but in fact it is much easier than it seems, this is about the problems of real world product development and if you are a software developer, then you and your project can definitely get benefit from using it.

Especially in medium-large sized projects of a corporate level, where the requirements for the functionality of the products are formalized. For example, there may be a requirement — when a configuration flag is set, log all input parameters for all public methods. Or for all methods in a project to have a notification functionality that will send a message when a certain threshold of the execution time of a method is exceeded.

How would you do it without AOP? Either you ignore the rule and do it only for the most important parts of your program, or when writing new methods, you do copy-paste of similar code from neighboring methods, and you could imagine what copy-paste approach brings with it.

With AOP, it is very easy, you write an advice an apply it to the whole project's code and the job is done. When you need to update the logic, you will again update just the advice and it will be re-applied on the next transformation. Without AOP, some many updates throughout the project code, that most probably you will implement the changes only to the critical parts and leave everything else as is.

On the plus side is that your code will not look like a crater covered moon landscape, because it is sprinkled with such functionality, the same way as moon sprinkled with craters and when reading the code it looks like mixed with white noise.

It is my feeling that aspect-oriented programming in .Net ecosystem is significantly less popular in compared to the Java ecosystem. In my opinion, the main reason is the lack of free and open source tools that are comparable to the functionality and quality of Java ones.

How we start using AOP in our project

To improve developers' performance and at the same time make code's quality better, we widely use the automatic code generation capabilities, and also created several plugins and analyzers for Visual Studio, that are custom tailored to our projects and tasks.

The next logical step was the idea to adopt aspect-oriented programming approach. We evaluated several tools, but the result was far from our expectations. Too complex for a middle level developer and takes too much time to develop even for a senior one. This coincided in time with the release of Roslyn technology, and at a certain moment we had an idea to combine the capabilities of automatic code generation and Roslyn.

In just a couple of weeks, a prototype of the tool was created and this approach seemed more promising. Fast forward several updates of the tool and we can say that our expectations were met and exceeded. We have developed a library of useful templates and we use this approach in most of our projects.

As you may expect, our tool is still not perfect and have some limitations, so I would like to split the article into two parts, the first one is how I see the implementation of this functionality in the perfect world and the second is how it works right now.

Before we dive deep into the details, I would like to make a disclaimer — all examples in this article have been simplified to a level that allows you to show the idea, without being overloaded with irrelevant details.

Also, as you may notice, English is not my native language, so please forgive me for any mistakes.

How it would be done in the perfect world

After several years of using our tool, I have a vision of how I would like this to work in the perfect world scenario.

In my vision, the language specifications allow the use of source code transformations, and there is support of such capabilities for a compiler and IDE.

The idea was inspired by introduction of the «partial» modifier in the C# language specification. This rather simple concept (the ability to define a class, structure or interface in several files) has significantly improved and simplified the support of tools for automatic source code generation. You can view it as a kind of horizontal splitting of the source code of a class between several files.

For those who are not fluent in the C# language, a small example.

Suppose we have a simple form described in the file Example1.aspx

<%@ Page Language="C#" AutoEventWireup="True" %>
// . . .
<asp:Button id="btnSubmit"
           Text="Submit"
           OnClick=" btnSubmit_Click" 
           runat="server"/>
// . . .

A developer implements some custom logic (for example, changing the color of the button to red when it is clicked) in the Example1.aspx.cs file

public partial class ExamplePage1 : System.Web.UI.Page, IMyInterface
{
  protected void btnSubmit_Click(Object sender, EventArgs e) 
  {
    btnSubmit.Color = Color.Red;
  }
}

The «partial» capabilities allows a tool (for example Visual Studio) to parse the Example1.aspx file and automatically generate the Example1.aspx.designer.cs file.

public partial class ExamplePage1 : System.Web.UI.Page
{
  protected global::System.Web.UI.WebControls.Button btnSubmit;
}

In other words, we have the ability to store a part of the code for the ExamplePage1 class in one file (Example1.aspx.cs) that is updated by a programmer and antoher part of the class in the Example1.aspx.designer.cs file, that is updated by an automatically generated tool.

In the end, for a compiler, it looks like as one class, combined from two parts

public class ExamplePage1 : System.Web.UI.Page, IMyInterface
{ 
  protected global::System.Web.UI.WebControls.Button btnSubmit;

  protected void btnSubmit_Click(Object sender, EventArgs e) 
  {
    btnSubmit.Color = Color.Red;
  }
}

As you might notice, in the example with the definition of inheritance of the IMyInterface interface, the final result is a combination of class definitions from different «partial» sections from the different files.

If the language lacks «partial» functionality and the compiler demands storing all the class code in only one file, then we can assume inconvenience and additional hurdles have to overcome for auto-generation of the code.

My idea is to include two additional modifiers in the language specification that will make it easier to embed aspects into the source code.

The first modifier is «original» keyword and we add it to the definition of the class that should be able to be transformed.

The second one is «processed» keyword, and it symbolizes that this is the final class definition that is result of the source code transformation tool and which must be accepted by the compiler to generate the bytecode.

The process sequence is something like this:

An user works with the source code of a class that contains the «original» modifier in the .cs file (for example Example1.cs)
When compiling, the compiler checks the correctness of the source code, and if the class is successfully compiled, it checks for the presence of the «original» modifier
If «original» modifier is present, then the compiler transfer the source code to a transformation process (which is a black box for the compiler)
The transformation process, based on a set of rules, modifies the source code and, upon successful completion of the process, creates .processed.cs file and .processed.cs.map files (.processed.cs.map needed to match the code between the .cs files and the .processed.cs file, to help with debugging and to display differences between the files in IDE)
The compiler reads the code from the .processed.cs file (in our example it is Example1.processed.cs) and compiles it
If the code in the file has been successfully compiled, then it is checked that
- Classes that had the «original» modifier have the «processed» modifier
- The signature of these classes is identical in both the .cs file and the .processed.cs file
If everything is fine, then the byte code produced during compilation of the .processed.cs file is accepted for final byte code generation.

By adding these two modifiers, we were able to introduce support for source code transformation tools at the language level, the same way as «partial» modifier made it possible to simplify support for source code generation.

As I see it, implementation of support of «original»/«processed» feature in the compiler is a week of work for two interns at Microsoft (a joke of course, but only partially). There are no any fundamental difficulties in implementation of the support. From the point of view of the compiler, it is file manipulation and process invocation.

In .NET 5, a new feature was announced — source code generators, which allows you to generate new source code files during compilation process and this is a movement in the right direction. Unfortunately, it only allows you to generate new source code, but not modify the existing one. So Some work still needs to be done.

Here is an example of such process, as I envision it. An user creates file Example2.cs

public original class ExamplePage2 : System.Web.UI.Page, IMyInterface
{ 
  protected global::System.Web.UI.WebControls.Button btnSubmit;

  protected void btnSubmit_Click(Object sender, EventArgs e) 
  {
    btnSubmit.Color = Color.Red;
  } 
}

Starts the compilation process, if it completes successfully and the compiler finds the «original» modifier, then it gives the source code to the transformation process, which generates the Example2.processed.cs file (in the simplest case, it can be just an exact copy of Example2.cs with «original» replaced by «processed»).

In our case, lets assume that the transformation process has added an advice for a logging aspect and the result looks like this:

public processed class ExamplePage2 : System.Web.UI.Page, IMyInterface
{ 
  protected global::System.Web.UI.WebControls.Button btnSubmit;

  protected void btnSubmit_Click(Object sender, EventArgs e) 
  {
    try
    {
      btnSubmit.Color = Color.Red;
    } 
    catch(Exception ex)
    {
      ErrorLog(ex);
      throw;
    }

    SuccessLog();
  } 

  private static processed ErrorLog(Exception ex)
  {
    // some error logic here
  }

  private static processed SuccessLog([System.Runtime.CompilerServices.CallerMemberName] string memberName = "")
  {
    // some success logic here
  }
}

The next step would be to verify the signatures of the «original» vs «processed» classes. The majority of signatures are identical and it satisfies the condition that the definitions in «original» and «processed» versions must be exactly the same.

In this finaly «processed» code, I included an example of one more proposal, this is the «processed» modifier for methods, properties and fields.

For methods, properties, and fields the «processed» modifier identifies them as available only to classes with the «processed» modifier and which are ignored when comparing signatures. This is done for the convenience of aspect logic developers and allows them to move the common logic into separate methods so as not to create unnecessary code redundancy.

The compiler verifies this code and if everything is ok, then takes created bytecode to continue the build process.

It is clear that in this example there is some simplification and in reality the logic may be more complicated (for example, when we include both «original» and «partial» modifires into the same class).

The perfect world's IDE functionality to support source code transformations

Main requirement for IDE to support of source code transformations is correct navigation between the «original»/«processed» classes and support of step-by-step debugging.

The second feature of the IDE that I would like to have is to help in reading the code of processed classes. A «processed» class can contain many pieces of code that have been added by several adivces to the same method/property.

Implementation of displaying such code that is similar to the concept of layers in a graphics editor seems to me as the most convenient option to achieve this goal. Our current plugin implements something similar and the response from its users is quite positive.

Another feature that would help introduce AOP into everyday life is the refactoring functionality. An user, selects a part of the code, and could choose «Extract To AOP Template» option and the IDE creates the necessary template files, scaffolds the initial code and after analyzing the project code, suggests candidates for using the template from other classes.

And the icing on the cake would be support in writing aspect templates, for example, interactively applying an advice to a class / method of your choice so that you can evaluate the results of tranfromation on the fly, without an explicit compilation cycle.

I am sure that if the creators of the Resharper plugin applies their magic, the result is guaranteed.

Creating aspect source code in the perfect world

To paraphrase the TRIZ, the ideal creating of source code for the implementation of aspects logic is the absence of creating of additional code that needed only to support the instrumentation processes.

In the perfect world scenario, we would like to write source code for the aspect advices, without the effort of writing helper logic to achieve that goal. And this source code would be an integral part of the project.

The second desire is the ability to have interactive plug & play, i.e. after we have created a template, we would not need to take additional steps in order for it to be used for transformation. There would be no need to recompile the tool or configure post-compilation options for the project.

After a template was created and a developer writes a couple of lines of code, he/she would immediately see the result and if the template contains errors, their detection and debugging of the errors would be integrated into the process of applying the template, and would not be a separate part that requires additional effort from the developer.

Plus, it would be really good that the syntax of the template language would be as close as possible to the syntax of the C # language, ideally just a minor difference, a few keywords and placeholders.

Our implementation

Our current approach is to create two copies of the project. The first one is for the original source code (the one developers work with) and the second one is the transformed version of the source code, which is used for compilation, debugging, testing and execution.

The scenario is something like this

The programmer works with the original version of the source code, implements the logic, compiles the project to detect compile-time errors, etc.
When he/she is satisfied with the result and ready to start testing, the command line script is launched, which starts the file transformation process and starts the build process (of course, if the transformation is successful).
After the build is complete, depending on the type of project, either a browser is launched that opens a test website for a web project, or a desktop program, if it is a WPF project, or autotests, etc.

For debugging, the second copy (transformed one) of the project is opened in an IDE and a developer continues with it.

The process requires a certain discipline, but after some time it has become a habit, and in certain cases this approach has some advantages (for example, a build could be launched and deployed to a remote server, instead of working with a local computer). Plus, our custom plugin for VisualStudio makes the process easier.

IDE

We use a plugin that is custom tailored for our specific needs and processes and support for the transformation of source code is a fairly small part of its capabilities.

For example, the functionality for displaying code in layers (in a style similar to a graphical editor), allows to hide/show comment layers, by method/properties visiblity (for example, so that only public methods are visible), regions. We surround some of the transformed source code by comments in a special format, and they can also be hidden as a separate layer.

Another possibility is to show a diff between the original and the transformed file. The IDE already knows the relative location of the copy of the files in the project, so it can display the differences between the original and transformed files.

Also, the plugin warns when trying to make changes to the transformed copy (so as not to lose them during subsequent re-transformation). We even published the feature as a free and open sourced plugin for VisualSource Code.

Configuration of pointcuts

A separate topic is configuation of pointcuts for transformation rules, i.e. to which classes and methods we will apply the transformation.

For such purposes we use several levels.

The first level is the top-level configuration file. We can set rules depending on the path on the file system, patterns in the name of files, classes or methods and scopes of classes, methods or properties.

The second level is an indication of the application of transformation rules at the level of attributes of classes, methods or fields in source code of original copy.

The third at the level of the code block and the fourth is an explicit indication to include the results of the transformation of the template in a specific place in the source code.

Templates

Historically, for the purposes of automatic generation, we use templates in the T4 format, so it was quite logical to use the same approach as templates for transformation. T4 templates include the ability to execute arbitrary C# code and have minimal overhead.

For those who have never worked with T4, the simplest analogue would be to present the ASPX or Razor formats, which instead of HTML generates source code in C # and is executed not on IIS, but as a separate tool with outputting the result to the console (or to a file).

Talk is cheap. Show me the code.

To understand how this works for us in real life, the simplest way would be to demonstrate the code before and after the transformation and the source code of the templates that is used during the transformation. I'll demonstrate the simplest options, but the potential of this approach is only limited by your imagination.

An example source code

The example source code after transformation is applied

Step-by-step explanation and template code

AutoComment template

// ##aspect=AutoComment

While processing the source code, if the tool finds a comment in a special format, then it executes the specified template (in this case, it is AutoComment) and insert the transformation result in place of this comment. In this example, it makes sense to automatically insert a special disclaimer that will warn the developer that the code in this file is the result of transformation and it makes no sense to modify this file directly.

AutoComment.t4 template source code

<#@ include file="AopCsharp.ttinclude" #>

//------------------------------------------------------------------------------
// <auto-generated> 
//     This code was generated from a template.
// 
//     Manual changes to this file may cause unexpected behavior in your application.
//     Manual changes to this file will be overwritten if the code is regenerated.
//
//  Generated base on file: <#= FileName #>
//  ##sha256: <#= FileSha256 #>
//  Created By: <#= User #>
//  Created Machine: <#= MachineName #>
//  Created At: <#= Now #>
//
// </auto-generated>
//------------------------------------------------------------------------------

The variables FileName, FileSha256, User, MachineName, and Now are exported to the template from the transformation process.

Result of the transformation

//------------------------------------------------------------------------------
// <auto-generated> 
//     This code was generated from a template.
// 
//     Manual changes to this file may cause unexpected behavior in your application.
//     Manual changes to this file will be overwritten if the code is regenerated.
//
//  Generated base on file: ConsoleDemo.cs
//  ##sha256: PV3lHNDftTzVYnzNCZbKvtHCbscT0uIcHGRR/NJFx20
//  Created By: EuGenie
//  Created Machine: 192.168.0.1
//  Created At: 2017-12-09T14:49:26.7173975-05:00
//
// </auto-generated>

The next transformation is define by a class attribute

[AopTemplate("ClassLevelTemplateForMethods", NameFilter="First")]
public virtual Person FirstDemo(string firstName, string lastName, int age)
{
  Console.Out.WriteLine("FirstDemo: 1");

  return new Person()
      {
        FirstName = firstName,
        LastName = lastName,
        Age = age,
      };
}

This attribute signals that the ClassLevelTemplateForMethods template should be applied to all methods of a class containing the word «First» in the method's name. The NameFilter parameter is a regular expression pattern that is used to determine which methods to include in the transformation.

ClassLevelTemplateForMethods.t4 template source code

<#@ include file="AopCsharp.ttinclude" #>

// class level template
<#= MethodStart() #><#= MethodBody() #><#= MethodEnd() #>

Result of the transformation

// class level template
public virtual Person FirstDemo(string firstName, string lastName, int age)
{
  Console.Out.WriteLine("FirstDemo: 1");

  return new Person()
      {
        FirstName = firstName,
        LastName = lastName,
        Age = age,
      };
}

The next transformations are defined by method attributes to demonstrate abilities of multiple transformations applied to the same method.

[AopTemplate("LogExceptionMethod")]
[AopTemplate("StopWatchMethod")]
[AopTemplate("MethodFinallyDemo", AdvicePriority = 100)]
public Customer[] SecondDemo(Person[] people)
{
    IEnumerable<Customer> Customers;

    Console.Out.WriteLine("SecondDemo: Step 1");

// ...

    return Customers.ToArray();
}

LogExceptionMethod.t4 template source code

<#@ include file="AopCsharp.ttinclude" #>
<# EnsureUsing("System"); #>
<#= MethodStart() #>
try
{
<#= MethodBody() #>
} 
catch(Exception logExpn)
{
    Console.Error.WriteLine($"Exception in <#= MethodName #>\r\n{logExpn.Message}\r\n{logExpn.StackTrace}");
    throw;
}

<#= MethodEnd() #>

StopWatchMethod.t4 template source code

<#@ include file="AopCsharp.ttinclude" #>
<# EnsureUsing("System.Diagnostics"); #>
<#= MethodStart() #>

var stopwatch = Stopwatch.StartNew(); 

try
{
<#= MethodBody() #>
} 
finally
{
    stopwatch.Stop();
    Console.Out.WriteLine($"Method <#= MethodName #>: {stopwatch.ElapsedMilliseconds}");

}

<#= MethodEnd() #>

MethodFinallyDemo.t4 template source code

<#@ include file="AopCsharp.ttinclude" #>

<#= MethodStart() #>
try
{
<#= MethodBody() #>
} 
finally 
{
    // whatever logic you need to include for a method
}

<#= MethodEnd() #>

Result of the transformation

public Customer[] SecondDemo(Person[] people)
{
    try
    {
        var stopwatch = Stopwatch.StartNew();
        try
        {
            try
            {
                IEnumerable<Customer> customers;

                Console.Out.WriteLine("SecondDemo: Step 1");

// ...

                return customers.ToArray();
            }
            catch (Exception logExpn)
            {
                Console.Error.WriteLine($"Exception in SecondDemo\r\n{logExpn.Message}\r\n{logExpn.StackTrace}");
                throw;
            }
        }
        finally
        {
            stopwatch.Stop();
            Console.Out.WriteLine($"Method SecondDemo: {stopwatch.ElapsedMilliseconds}");
        }
    }
    finally
    {
    // whatever logic you need to include for a method
    }
}

The next example of the transformation is defined by a block bounded by a using statement

using (new AopTemplate("SecondDemoUsing", extraTag: "test extra"))
{
    customers = people.Select(s => new Customer()
    {
        FirstName = s.FirstName,
        LastName = s.LastName,
        Age = s.Age,
    });

    foreach (var customer in customers)
    {
        Console.Out.WriteLine($"SecondDemo: 2 {customer.FirstName} {customer.LastName}");
    }
}

SecondDemoUsing.t4 template source code

<#@ include file="AopCsharp.ttinclude" #>

Console.Out.WriteLine("Extra Tag data: <#= ExtraTag #>");

<#= StatementBody() #>

ExtraTag is a string that is passed as a parameter. This can be useful for templates that can have slightly different behavior depending on the input parameters.

Result of the transformation

{
      Console.Out.WriteLine("Extra Tag data: test extra");

      customers = people.Select(s => new Customer()
      {
            FirstName = s.FirstName, 
            LastName = s.LastName, 
            Age = s.Age, 
      });

      foreach (var customer in customers)
      {
          Console.Out.WriteLine($"SecondDemo: 2 {customer.FirstName} {customer.LastName}");
      }
}

The next transformations are defined by class attributes and will be applied on a class and properties levels.

AopTemplate("NotifyPropertyChangedClass", Action = AopTemplaceAction.Classes)]
[AopTemplate("NotifyPropertyChanged", Action = AopTemplaceAction.Properties)]
public class Person
{
    public int Id { get; set; }

// ...
}

This is a classic example, which along with the logging example is given in most examples of aspect-oriented programming.

NotifyPropertyChangedClass.t4 template source code that will be applied on a class level

<#@ include file="AopCsharp.ttinclude" #>
<#
    // the class already implements INotifyPropertyChanged, nothing to do here
    if(ImplementsBaseType(ClassNode, "INotifyPropertyChanged", "System.ComponentModel.INotifyPropertyChanged"))
        return null;

    var classNode = AddBaseTypes<ClassDeclarationSyntax>(ClassNode, "System.ComponentModel.INotifyPropertyChanged"); 
#>

<#= ClassStart(classNode) #>
            public event System.ComponentModel.PropertyChangedEventHandler PropertyChanged;

            protected void NotifyPropertyChanged([System.Runtime.CompilerServices.CallerMemberName] string propertyName = "")
            {
                PropertyChanged?.Invoke(this, new System.ComponentModel.PropertyChangedEventArgs(propertyName));
            }

<#= ClassBody(classNode) #>
<#= ClassEnd(classNode) #>

NotifyPropertyChanged.t4 template source сode that will be applied on properties level

<#@ include file="AopCsharp.ttinclude" #>
<#
    if(!(PropertyHasEmptyGetBlock() && PropertyHasEmptySetBlock()))
        return null;

    string privateUnqiueName = GetUniquePrivatePropertyName(ClassNode, PropertyNode.Identifier.ToString());
#>

    private <#= PropertyNode.Type.ToFullString() #> <#= privateUnqiueName #><#= PropertyNode.Initializer != null ? " = " + PropertyNode.Initializer.ToFullString() : "" #>;

<#= PropertyNode.AttributeLists.ToFullString() + PropertyNode.Modifiers.ToFullString() + PropertyNode.Type.ToFullString() + PropertyNode.Identifier.ToFullString() #>
    {
        get { return <#= privateUnqiueName #>; }
        set 
        {
            if(<#= privateUnqiueName #> != value)
            {
                <#= privateUnqiueName #> = value;
                NotifyPropertyChanged();
            }
        }
    }

Result of the transformation

public class Person : System.ComponentModel.INotifyPropertyChanged
{
    public event System.ComponentModel.PropertyChangedEventHandler PropertyChanged;
    protected void NotifyPropertyChanged([System.Runtime.CompilerServices.CallerMemberName] string propertyName = "")
    {
        PropertyChanged?.Invoke(this, new System.ComponentModel.PropertyChangedEventArgs(propertyName));
    }

    private int _id;
    public int Id
    {
        get
        {
            return _id;
        }

        set
        {
            if (_id != value)
            {
                _id = value;
                NotifyPropertyChanged();
            }
        }
    }

// ...
}

The next transformation for caching property results, it is defined by the attribute

[AopTemplate("CacheProperty", extraTag: "{ \"CacheKey\": \"name_of_cache_key\", \"ExpiresInMinutes\": 10 }")]
public string FullName
{
    get
    {
        return $"{FirstName} {LastName}";
    }
}

CacheProperty.t4 template source code (it is quite long, but most of the code is an example of using JSON from ExtraTag property as a config setting )

Result of the transformation

public string FullName
{
    get
    {
        System.Runtime.Caching.ObjectCache cache = System.Runtime.Caching.MemoryCache.Default;
        string cachedData = cache["name_of_cache_key"] as string;
        if (cachedData == null)
        {
            cachedData = GetPropertyData();
            if (cachedData != null)
            {
                cache.Set("name_of_cache_key", cachedData, System.DateTimeOffset.Now.AddMinutes(10));
            }
        }

        return cachedData;
        string GetPropertyData()
        {
            // FullNameComment FullName
            return $"{FirstName} {LastName}";
        }
    }
}

Non-AOP templates

Although this article focuses on aspect-oriented programming, the source code transformation technique is universal and can be used for tasks that are not directly related to AOP.

For example, it can be used for dependency injection, i.e. we change the resource creation code depending on the build parameters.

The transformation is defined by an attribute

[AopTemplate("DependencyInjection", AdvicePriority = 500, Action = AopTemplateAction.PostProcessingClasses)]
public class ConsoleDemo
{
// ...

    private static IConfigurationRoot _configuration = inject;
    private IDataService _service { get; } = inject;

// ...

    public Customer[] SecondDemo(Person[] people)
    {
         int configDelayMS = inject; // we are going to inject dependency to local variables
         string configServerName = inject;
    }

// ...

    protected static dynamic inject;

// ...
}

As you see, in the source code the dynamic variable feature is used, which allows them to be assigned to a property/field/variable of any types. For the sake of clarity, we have introduced some kind of quasi keyword.

DependencyInjection.t4 template source code

In the template code we use the comparison of GetGlobalSetting ("env") == "test" and depending on this condition, either new DataService() or new MockDataService() will be injected.

Result of the transformation

public class ConsoleDemo
{
// ...

    private static IConfigurationRoot _configuration = new ConfigurationBuilder()
        .SetBasePath(System.IO.Path.Combine(AppContext.BaseDirectory))
        .AddJsonFile("appsettings.json", optional: true)
        .Build();

    private IDataService _service { get; } = new DataService();

// ...

    public Customer[] SecondDemo(Person[] people)
    {
           int configDelayMS = Int32.Parse(_configuration["delay_ms"]);
           string configServerName = _configuration["server_name"];
    }

// ...
}

Also, the tool might be used as a «poor man» static analysis (but it is much, much better to implement analyzers using the native functionality of Roslyn). We analyze the code for our rules and insert #error/#warning directives into the source code when we want to raise an error or warning.

The transformation is defined by

[AopTemplate("StaticAnalyzer", Action = AopTemplateAction.Classes)]
public class ConsoleDemo
{
// ..
         IEnumerable<Customer> Customers;
// ..
         Console.Out.WriteLine(i18(String.Format("SecondDemo: {0}", "Step 3")));
// ..
}

StaticAnalyzer.t4 template source code

Result of the transformation

public class ConsoleDemo
{
// ..
         #error variable "Customers" doesn't match code standard rules
         IEnumerable<Customer> Customers;
// ..
         #warning Please replace String.Format with string interpolation format
         Console.Out.WriteLine(i18(String.Format("SecondDemo: {0}", "Step 3")));
// ..
}

Another way to use the tool for purposes localizing an application, i.e. find all the strings in the classes and replace them with the use of the appropriate resources.

The transformation is defined by

[AopTemplate("ResourceReplacer", AdvicePriority = 1000, ExtraTag = "ResourceFile=Demo.resx,ResourceClass=Demo", Action = AopTemplateAction.PostProcessingClasses)]
public class ConsoleDemo
{
// ..
    Console.Out.WriteLine(i18("SecondDemo: i18"));
// ...
    Console.Out.WriteLine(i18($"First Name {customer.FirstName} Last Name {customer.LastName}"));

    Console.Out.WriteLine("SecondDemo: 2 " + i18("First Name ") + customer.FirstName + i18(" Last Name   ") + customer.LastName);
// ...
    Console.Out.WriteLine(i18(String.Format("SecondDemo: {0}", "Step 3")));

// ...
    protected static string i18(string s) => s;
}

ResourceReplacer.t4 template source code

In the example, we have a resource file Demo.resx, that have created the following lines

<data name="First Last Names Formatted" xml:space="preserve">
  <value>First Name {0} Last Name {1}</value>
</data>
<data name="First Name" xml:space="preserve">
    <value>First Name</value>
</data>
<data name="Last Name" xml:space="preserve">
  <value>Last Name</value>
</data>

And VisualStudio generated C# class for us from content of Demo.resx

Result of the transformation

public class ConsoleDemo
{
// ..
    #error Cannot find resource for a string "SecondDemo: i18", please add it to resources
    Console.Out.WriteLine(i18("SecondDemo: i18"));

// ...
    Console.Out.WriteLine(String.Format(Demo.First_Last_Names_Formatted, customer.FirstName, customer.LastName));

    Console.Out.WriteLine("SecondDemo: 2 " + (Demo.First_Name + " ") + customer.FirstName + (" " + Demo.Last_Name + "   ") + customer.LastName);

// ...

    #error Argument for i18 method must be either string literal or interpolated string, but instead got Microsoft.CodeAnalysis.CSharp.Syntax.InvocationExpressionSyntax
    Console.Out.WriteLine(i18(String.Format("SecondDemo: {0}", "Step 3")));
// ...
}

Please note that the interpolated string was replaced with String.Format and the «First Name {0} Last Name {1}» resource was used. For lines that do not exist in the resource file or do not match our format, an error message is added.

Conclusion

The tool gives you a very powerful and at the same time quite simple way to introduce Aspect-Oriented Programming into your projects.

You can give it a try for a whole project, a folder or even just one file, to evaluate how it goes.

In addition, the transformation tool allows you to work not only with C# files, but also with any file type (of course, with certain limitations). If you have a parser that can build an AST for your language, then you can replace Roslyn with this parser, tweak the implementation of the code handler and it will work. Unfortunately, the number of libraries with functionality close to Roslyn is very limited and their use requires much more effort. In addition to C#, we use transforms for JavaScript and TypeScript projects, but certainly not as comprehensively as for C#. And using T4 which is based on C# and to produce output in a different language is not optimal as well.

Just a reminder, the code of the examples and templates are provided as an illustration of the possibilities of such an approach and in real life, as they say, sky is the limit.

Our original tool is developed for the .Net Framework, but we started work on a simplified open source version named Aspectimum, under the MIT license for .Net Core. At the moment, the result is fully functional and 90% ready, there are need for minor improvements, refactor of the code, the creation of documentation and examples, but without all this, barrier to entry will be quite high and DX might be negative.

[The project's repository at github.com](Conclusion
The tool gives you a very powerful and at the same time quite simple way to introduce Aspect-Oriented Programming into your projects.

You can give it a try for a whole project, a folder or even just one file, to evaluate how it goes.

Just a reminder, the code of the examples and templates are provided as an illustration of the possibilities of such an approach and in real life, as they say, sky is the limit.

The project's repository at github.com

If you have any questions or ideas, please feel free to contact me.