SmartTraits or lets add «multiple inheritance» to C#

One of our clients, a developer who used to work with php technologies for quite some time, frequently complained that with the transition to C# and .Net stack, he misses one of his favorite features from the php world - traits and he would like it very much to be able to use such functionality in .Net.

At one point, we decided to make him a present and implemented a proof of concept of similar functionality for C#.

To our surprise, it was quite easy to implement the PoC and the process of development was a lot of fun.

A brief description of what the traits are (for those of us who are lucky to never work with php). Extract from the php.net site:

Traits are a mechanism for code reuse in single inheritance languages such as PHP. A Trait is intended to reduce some limitations of single inheritance by enabling a developer to reuse sets of methods freely in several independent classes living in different class hierarchies. The semantics of the combination of Traits and classes is defined in a way which reduces complexity, and avoids the typical problems associated with multiple inheritance and Mixins.

A Trait is similar to a class, but only intended to group functionality in a fine-grained and consistent way. It is not possible to instantiate a Trait on its own. It is an addition to traditional inheritance and enables horizontal composition of behavior; that is, the application of class members without requiring inheritance.

Many programming languages ​​(including C#) do not support multiple inheritance to avoid the complexity and ambiguity it brings into a development process. But in certain (keep-it-simple) scenarios, it could be quite useful and would reduce the overall project complexity and the size of the codebase. Especially in cases where you have no control over the choice of the base classes and the ability to create a proper class hierarchy.

Recent versions of C# have added support for default interface methods to address these issues, but those methods have some limitations.

After discussion about scope of the PoC, we came up with the minimum requirements for traits functionality:

• static typing
• integration directly into a project (i.e. not pre / post-processing)
• IDE (VisualStudio) compatibility with all its features: autocompletion, error detection during code editing, etc.,
• minimum additional effort from a developer

When source generators start to gain popularity, it became clear that they could be a good candidate and might be able to meet our requirements and it would makes sense to try to implement the PoC.

And after couple of weeks of development time, we finally created a tool called SmartTraits.

There are a lot articles on how to implement source generators, so I will not go into details of the implementation of the source generator itself, but will describe the general ideas of the project.

We implemented the functionality in several stages and I will write about them based on the implementation timeline, but first, I would like to describe the conventions we use for the Trait and its destination classes.

Trait is

• an abstract class that is declared as partial
• the class is marked with the Trait attribute
• this class cannot be inherited from another class
• the class specification may include an interface that it must implement

Trait destination class (i.e. the class to which the source code from Trait will be added)

• must be partial
• must include one or more [AddTrait] attribute(s) with a trait type as a parameter

Note: All examples are simplified on purpose, to demonstrate principles of the idea. We are aware that there are alternative ways to meet these requirements (for example, our aspectimum tool).

For example, let's take the simplest class hierarchy

class ExampleA: BaseA 
{}

class BaseA {}

class ExampleB: BaseB
{}

class BaseB {}

Classes ExampleA and ExampleB are derived from different base classes and they do not have a common ancestor (for example base classes from third-party libraries) and as a result, we cannot easily add the required common functionality to all of these classes for one source.

But when unexperienced developer meet such challenge, he/she will use his/her proven solution, copy & paste command and as a result, will get all the consequences it usually brings into the project.

Below is description of iterations that we had during development of our source generator.

(Small disclaimer before we continue - as you may notice, English is not my native language, so please forgive me for any mistakes.)

First iteration - "glorified" #include

Suppose that to the both classes ExampleA and ExampleB in our example above, you need to add code to work with names

public string FirstName { get; set; }

public string LastName { get; set; }

public string GetFullName() { 
   return $"{FirstName} {LastName}"; 
}

First, we create our simplest Trait class

[Trait]
abstract partial class NamesTrait
{
      public string FirstName { get; set; }

      public string LastName { get; set; }

      public string GetFullName() { 
         return $"{FirstName} {LastName}"; 
      }
}

And apply the trait to the classes ExampleA and ExampleB by adding the [AddTrait] attribute

[AddTrait(typeof(NamesTrait))]
partial class ExampleA: BaseA 
{}

That is it, nothing else is required, and our source generator "behind the scene" automatically creates the second part for the ExampleA and ExampleB classes, copying the sources of all methods, properties, etc. from NamesTrait to both of them.

Autogenerated part:

partial class ExampleA
{
     public string FirstName { get; set; }

     public string LastName { get; set; }

     public string GetFullName() { 
        return $"{FirstName} {LastName}"; 
    }
}

The most basic requirements have been met, even at the very first step.

• not different from the rest of the code and all the rules that are applied to regular c# code are fully applicable to Traits as well
• quite small overhead
• full IDE support
• immediate changes to ExampleA and ExampleB when the code changes in the NamesTrait

The second iteration is a guarantee of the implementation of the required methods and properties by a trait

We can achieve guarantees in the standard way, by creating an interface and assigning it to both - the Trait class and the destination class.

Defining the interface

interface INames
{
    string FirstName { get; set; }
    string LastName { get; set; }
    string GetFullName();
}

Destination class declarations will look like (added INames interface)

partial class ExampleA: BaseA, INames

The Trait class declaration (we also add the INames interface)

abstract partial class NamesTrait: INames

Now the code will not compile until all the required methods / properties are implemented.

Third iteration - conflict resolution

By default, all the contents of the Trait class will be copied to the destination class, but lets suppose that for some destination classes we would like to be able to implement their own versions of methods, properties, etc.

The solution to achieve this requirement is quite simple, we mark Trait members that could be implemented in destination classes with the [Overrideable] attribute and when do copying from the Trait class, we check if a similar method or property is already implemented by the destination class and in this case do not copy the version from Trait.

If ExampleB has its own implementation of the GetFullName method, then we ignore the version of this method from the NamesTrait

[AddTrait(typeof(NamesTrait))]
partial class ExampleB: BaseB, INames 
{
    public string MiddleName { get; set; }

    public string GetFullName() { 
       return $" {FirstName} {MiddleName} {LastName}"; 
    }
}

Below is a generated part for class ExampleB

partial class ExampleB
{
    public string FirstName { get; set; }

    public string LastName { get; set; }
}   

In the NamesTrait class, the GetFullName method looks like (added [Overrideable] attribute)

[Overrideable]
public string GetFullName() { 
   return $"{FirstName} {LastName}"; 
}

Fourth iteration - accessing methods and properties of the destination class from Trait methods

If we know that all destination classes implement some methods or properties that we need to call from a Trait, then there are two ways to achieve this functionality.

1. In the Trait we add a mock and ignore this mock during copying process. For private and protected methods and properties, this is the only simple option we could think of. To achieve ignore copying of class members, you can mark them with the [TraitIgnore] attribute.

It was suggested that we can use abstracted methods / properties
instead of mocks and ignoring them when copied. And if we still create mocks, it would be better to throw an exception instead of returning default values. This is a more correct and safe option.

1. In addition to the mock option, for public methods can achieve this requirement in a more elegant way. We create the interface (s) and assign them to both Trait and the destination class. By doing this, we guarantee that there are interface members available in both classes. Also, for these interface members, we can automatically generate a mock for the Trait class. In this case, automatic source generation helps us for both, a destination class and for the Trait itself.

Suppose both ExampleA and ExampleB have a GetNamePrefix (): string method and we would want to be able to call this method from the Trait.

Option 1. Mock

[AddTrait(typeof(NamesTrait))]
partial class ExampleA
{
    private string GetNamePrefix()
    {
        return "mr/mrs. ";
    } 
}

The Trait declaration (you can see the use of GetNamePrefix () in the GetFullName function) and the definition of the GetNamePrefix mock

[Trait]
abstract partial class NamesTrait: INames
{
    public string FirstName { get; set; }

    public string LastName { get; set; }

    public string GetFullName() { 
       return $"{GetNamePrefix()} {FirstName} {LastName}"; 
    }


    [TraitIgnore]
    private string GetNamePrefix()
    {
       throw(new Exception());
    }
}

The generated code for ExampleA will look like

partial class ExampleA: INames
{
    public string FirstName { get; set; }

    public string LastName { get; set; }

    public string GetFullName() { 
       return $"{GetNamePrefix()} {FirstName} {LastName}"; 
    }
}

Option 2. Use of public methods / properties of the destination class via interfaces

We define an interface that the destination class implements (in this example, it is INamePrefix)

interface INamePrefix
{
    string GetNamePrefix();
}

[AddTrait(typeof(NamesTrait))]
partial class ExampleA: INames, INamePrefix
{
    public string GetNamePrefix()
    {
        return "mr/mrs. ";
    } 
}

And assign it to the Trait class

[Trait]
abstract partial class NamesTrait: INames, INamePrefix
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public string GetFullName() { 
       return $"{GetNamePrefix()} {FirstName} {LastName}"; 
     }
}

The generated code for the destination class (Example) will look like

partial class ExampleA: INames, INamePrefix
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public string GetFullName() { 
       return $"{GetNamePrefix()} {FirstName} {LastName}"; 
     }
}

And the automatically generated code for the Trait class will look like

abstract partial class NamesTrait: BaseMockNamesTrait
{
}

abstract class BaseMockNamesTrait: INamePrefix
{
    public abstract string GetNamePrefix();
}

Note: in order to distinguish the interfaces for accessing the members of the destination class (in this example it is INamePrefix) for which it is necessary to generate a mock, from the interface that guarantees the trait contract (in the example it is INames) we introduce an additional condition that the Trait interface of the contract is marked with the attribute [TraitInterface].

Fifth iteration - strict mode

We've included additional support for strict mode to prevent surprises and to reduce the likelihood of introducing of tricky bugs. In the strict mode, the code of Trait and the destination classes must implement the interface marked with the [TraitInterface] attribute. Plus the Trait class must implement ONLY methods and properties from this interface. No additional methods / properties are allowed.

An example of setting strict mode

[Trait(TraitOptions.Strict)]
abstract partial class NameTrait : IName

If you add a method / property that does not exist in the IName interface, there will be an error during compilation and you will not be able to build the project.

Side notes

After five iterations, it turned out to be quite a useful tool, but the curious reader from the very beginning is asking a question of why we added the word "smart" to the project name. So far, it is clearly not enough to be called SmartTraits - I completely agree, so there are still a couple of iterations ahead.

Sixth iteration - smart methods

Working with source generators, we have access to the source code of the project. It is how we were able to implement the Traits functionality.

But if you think about it, if we have the source code, we can copy it, but we also can execute it.

Therefore, we have added a new feature, methods that is

• marked with the Process attribute
• take ClassDeclarationSyntax object as a parameter
• and return string

Will be compiled and executed on the fly. The ClassDeclarationSyntax node of the destination class will be passed to the method as a parameter.

The result of the method execution will be added to the destination class code. If the optional attribute parameter is specified as ProcessOptions.Global, then it will be added not to the class code, but as a separate generated code. This will allow generating additional classes that implement the required functionality. When you start working with this functionality, at first, it really feels like some kind of magic.

Smart methods are mostly useful for generating code depending on external data (api / db / xml accesses, etc.). But you need to understand that this can be quite resource consuming and you need to implement the correct strategy for caching and invalidation.

An example of such code (marked with the [Process] attribute). For the ExampleA class, the GetExampleA method will be generated, and for ExampleB class, will be generated GetExampleB method

[Trait]
abstract partial class NamesTrait: INames, INamePrefix
{
    [Process]
    string BuildMethod(ClassDeclarationSyntax classNode)
    {
        return $"public string Get{classNode.Identifier}() {{ return \"{classNode.Identifier}\"; }}";
    }
}

For example, the generated code in the destination class ExampleA will look like

partial class ExampleA: INames, INamePrefix
{
    public string GetExampleA()
    {
        return "ExampleA";
    }
}

And for ExampleB like

partial class ExampleB: INames, INamePrefix
{
    public string GetExampleB()
    {
        return "ExampleB";
    }
}

Seventh iteration - even smarter

The ability to execute custom code on the fly that immediately changes the code of the same project is definitely cool, but it requires certain qualifications from a developer who writes such code. Generating code by manipulating and navigating through Roslyn takes some getting used to.

Therefore, it was decided to simplify and expand the feature by adding support for executing T4 templates during the source generation process. After all, the T4 was created just for this. It is easy to generate source code and even if a developer has never written T4 scripts before, learning of how to do it, is much easier than manipulating objects using Roslyn.

If you've written aspx or jsp code before, you may consider that you already know T4. Or if you wrote pages in php, the basic idea is the same. It was especially convenient in our case, taking into account the fact that our customer was from the PHP clan.

The idea is the same as in the previous case. We can mark any method, property or class with the [ApplyT4] attribute. It doesn't even need to be a member of the [AddTrait] destination class.

The only difference is that by default the result of template execution will be included as a separate generated code, i.e. T4TemplateScope.Global. But if desired, for the members of the destination class (class with the [AddTrait] attribute), you can specify the T4TemplateScope.Local option and the result of the template will be added to the partial part of the generated destination class.

Due to the fact that the compiled version of the template is cached, execution is very fast. When the T4 template is changed, it sees the changes immediately and is recompiled on the fly and the new version is applied.

It is very convenient in comparison with standard source generators, where on any tiny change, you need to build a new assembly and redeploy the nuget package.

After six months of work, we collected statistics that the version with Process (compilation of embedded code on the fly) is actually not used and it was disabled, leaving only the T4 version.

Some developers who do not require the functionality of the SmartTraits package, have installed a generator and are using it only because of the synergy between source generators and T4, because of the ability to recompile on the fly and seen instant results.

Here is how we define templates:

[AddTrait(typeof(NameTrait))]
[ApplyT4("DemoTemplate", Scope = T4TemplateScope.Local, VerbosityLevel = T4GeneratorVerbosity.Debug)]
partial class ExampleA : BaseA, IName
{
}

class BaseA { }

Sample T4 template:

<#@ include file="SmartTraitsCsharp.ttinclude" #>
<#
    // for this demo, we ignore empty nodes, but in real life you would want the template to fail, to be able to catch issues earlier
    if(ClassNode?.Identifier == null)
        return "";
#>

public string GetT4<#= ClassNode.Identifier.ToString() #>() 
{
    return "T4 <#= ClassNode.Identifier.ToString() #>"; 
}

Generated Code for ExampleA

partial class ExampleA {
    public string GetT4ExampleA() 
    {
        return "T4 ExampleA"; 
    }
}

Generated Code for ExampleB

partial class ExampleB {
    public string GetT4ExampleB() 
    {
        return "T4 ExampleB"; 
    }
}

Conclusion

This was a short description of what was done to demonstrate the capabilities of the source generators technology.

We have published the source code of this proof of concept under a MIT license - in case if anyone might be interested and decide to build a product that could be used for production ready code.

Thinking out loud

In my opinion, such functionality could even be added into the C# language itself, a couple of keywords and it would be ready to use. Based on the experience of this project, everything turns out to be quite convenient and useful.

I would also suggest to guys from Microsoft to add an option so that you can mark your analyzer or generator as "heavy" - i.e.it would be executed only on builds, and not on every tiny change of the code. In certain cases, I would prefer that there was no instant update of the generated files, but much more resource-intensive checks could be added to the analyzer/source generator without being afraid to halt the IDE for a minute.

How to try it?

The project code can be found in the repository

Packages are also available at nuget.org

I want to warn you - the development of this version was stopped right after the demo to the client and after that it was never updated. So this is PoC/MVP with the quality of the code of PoC and with a high probability of multiple bugs. Though, before publishing the article, I have tested this code, and the cases from the demo project were working just fine.

In order for the project to write the generated code to the file system, you need to add to the project file:

<EmitCompilerGeneratedFiles>true</EmitCompilerGeneratedFiles>
<CompilerGeneratedFilesOutputPath>$(BaseIntermediateOutputPath)\GeneratedFiles</CompilerGeneratedFilesOutputPath>

I strongly recommend updating to the latest version of VisualStudio and Resharper before trying the project, source generators are new technology and old versions of the tools are not working with it properly.

Thanks for your time.

Comments

[Benjamin Askren]

Most impressive. Thank you for making this awesome tool publicly available!