Alternative view engine for ASP.NET Core: Compiling templates

The last post in this series looked at parsing the template syntax. It also promised the next post will cover supporting inline C# expressions. And that's exactly what we'll do. Be sure to check out the previous posts in the series so that it's easier to follow this one.

In the Razor view engine, you can use the @ symbol to transition from markup into writing plain C#. We're going to do something similar where we treat what's inside mustache tags as C# code. For example:

Your Message: {{1 + 1}}

should output:

Your Message: 2

We'll work with a simple arithmetic expression in this post but we'll still explore the core mechanism through which view engines work. You can find the full contents of the source files discussed below on this GitHub gist.

General Approach

At the end of the day, the .NET runtime expects to run code in one of the languages that it supports. So we have to find a way to get from our template syntax to C# code that can be executed. The general approach that's used in such cases is to dynamically construct a C# class from the view template. Once we have a C# representation of the template, we can use APIs in Microsoft.CodeAnalysis and System.Reflection namespaces to dynamically compile, load, and run it. If you look at the source code for the Razor view engine, it's essentially using this same approach.

View Template to C# Class

As shown in the last post, we can parse the view template to generate a DocumentNode. We can use the structured information in DocumentNode to create a C# class.
For example, given the parsed template of Your Message: {{1 + 1}} the following class can be generated:

using System.Text;
namespace StacheTemplateNamespace{
    public class Bar
    {
        private StringBuilder _output = new StringBuilder();

        public string Execute()
        {

_output.Append("Your Message: ");
_output.Append(1 + 1);

            return _output.ToString();
        }
    }
}

Note that we give the class the same name as the view template. It also has a method named Execute() that computes the contents of the template and outputs a string.

Create a new class Stache/StacheTemplateCompiler.cs and add the following method:

    private static string CreateClassDocument(DocumentNode parsedResult, string templateClassName)
    {
      var result = new StringBuilder();
      result.AppendLine($@"
using System.Text;
namespace StacheTemplateNamespace{{
    public class {templateClassName}
    {{
        private StringBuilder _output = new StringBuilder();");

      result.AppendLine(@"
        public string Execute()
        {
            ");

      foreach(var node in parsedResult.Nodes){
        switch(node){
          case TextNode textNode:
            result.Append("_output.Append(\"");
            result.Append(textNode.Value);
            result.AppendLine("\");");
          break;
          case ExpressionNode expNode:
            result.Append("_output.Append(");
            result.Append(expNode.Value);
            result.AppendLine(");");
          break;
        }
      }

      result.AppendLine(@"
            return _output.ToString();
        }
    }
}
            ");

      return result.ToString();

    }

Since DocumentNode contains information about whether a parsed node is a TextNode or ExpressionNode, CreateClassDocument() can render the node value as a string or expression accordingly.

Creating an Assembly

After creating the C# class in string form, it has to be compiled before it can be executed by the .NET runtime. This compiled code is referred to as an assembly. In StacheTemplateCompiler.cs, add the following method:

internal static Assembly Compile(DocumentNode parsedResult, string templateClassName){
  var classDocument = CreateClassDocument(parsedResult, templateClassName);
  // to be continued...
}

This Compile() method first makes use of the method we defined earlier to generate the class document as a string. Next we take that string and parse it using Microsoft.CodeAnalysis to generate a syntax tree:

var syntaxTree = CSharpSyntaxTree.ParseText(classDocument);

Note that this is analogous to the tokenization and parsing that we performed in the last post. But this time with C# code using Microsoft tooling.

The next step is to compile the parsed syntax to machine code (IL code). This is achieved by what Microsoft calls a compilation. A compilation represents everything needed to compile some code. It includes a name for the assembly, references to other assemblies, and of course the code we want to compile. Add the following method to StacheTemplateCompiler.cs:

private static CSharpCompilation CreateCompilation(SyntaxTree syntaxTree){
    var systemRuntimeAssemblyLocation = typeof(object).Assembly.Location;

    var references = new List<MetadataReference>{
      MetadataReference.CreateFromFile(systemRuntimeAssemblyLocation)
    };

    var compilationOptions = new CSharpCompilationOptions(
        OutputKind.DynamicallyLinkedLibrary);

    var assemblyName = System.IO.Path.GetRandomFileName();

    var compilation = CSharpCompilation.Create(assemblyName, options: compilationOptions, 
        references: references);
    compilation = compilation.AddSyntaxTrees(syntaxTree);

    return compilation;
}

This method prepares the compilation. You can see that we're making a reference to the assembly containing the System.Object class (System.Runtime.dll). We want the output to be a DLL (as opposed to EXE). We also picked a random file path for the assembly name since it's not that important for our case. Finally, we add our parsed C# class. We can then invoke this method in Compile():

var compilation = CreateCompilation(syntaxTree);

We now have all the ingredients that we need to compile the class and create an assembly. We do that in the following method (also in StacheTemplateCompiler.cs):

private static Assembly LoadAssembly(CSharpCompilation compilation){
    using (var assemblyStream = new MemoryStream())
    {
        var result = compilation.Emit(assemblyStream, null);
        if (!result.Success)
        {
            throw new Exception(
                string.Join(" ", result.Diagnostics.Select(
                    d => $"{d.Id}-{d.GetMessage()}\n"))
            );
        }

        assemblyStream.Seek(0, SeekOrigin.Begin);
        return Assembly.Load(assemblyStream.ToArray(), null);
    }
}

This method actually compiles the class and emits the IL code. The result is written to computer memory and loaded to the same domain as our running application. We now have a dynamically created class we can instantiate and whose methods we can invoke! Overall the Compile() method looks like this:

internal static Assembly Compile(DocumentNode parsedResult, string templateClassName){
  var classDocument = CreateClassDocument(parsedResult, templateClassName);
  var syntaxTree = CSharpSyntaxTree.ParseText(classDocument);
  var compilation = CreateCompilation(syntaxTree);
  var assembly = LoadAssembly(compilation);
  return assembly;
}

Using the Assembly

We need to make some modifications to the code discussed in the previous posts to use the assembly. For the templateClassName parameter in StacheTemplateCompiler.Compile(), we will use the view name. This requires changing StachViewEngine.cs slightly to include the view name when constructing the StacheView object:

// in StachViewEngine.FindView() method
return ViewEngineResult.Found(viewName, new StacheView(possibleViewLocation, viewName));

// in StachViewEngine.GetView() method
var viewNameWithExtension = viewPath.Substring(viewPath.LastIndexOf('/')+1);
var viewName = viewNameWithExtension.Substring(0,
    viewNameWithExtension.IndexOf(ViewExtension));
return ViewEngineResult.Found(viewPath, new StacheView(appRelativePath, viewName));

We also modify StacheView.cs to accept the template name in the constructor and store it as an instance variable.

public StacheView(string path, string templateName){
  Path = path;
  TemplateName = templateName;
}

public string TemplateName {get; private set;}

Next we replace the logic in StacheView.RenderAsync() that looped through the document node with a call to StacheTemplateCompiler.Compile() we created in this post:

// in StacheView.RenderAsync()
var assembly = StacheTemplateCompiler.Compile(document, TemplateName);
var instance = assembly.CreateInstance($"StacheTemplateNamespace.{TemplateName}");
var processedOutput = instance.GetType().GetMethod("Execute").Invoke(instance, null);

return context.Writer.WriteAsync(processedOutput.ToString());

The above code compiles the document node and obtains an assembly. It then instantiates the C# class we dynamically generated and invokes the Execute() method on it.

To test our changes, update the contents of Bar.stache to the following:

<h2>Your Message: {{1 + 1}}</h2>

Then running the app (dotnet run) and navigating to https://localhost:5001/Home/Bar you should see:

Your Message: 2

Other simple expressions should work as well. For example,

<h2>Your Message: {{"foobar".Length}}</h2>

<h2>Your Message: {{5 == 4}}</h2>

Summary

As shown above, view engines depend on the ability to dynamically compile template syntax into code that can be executed by .NET. A lot of the Roslyn compiler's functionality is exposed as APIs (Microsoft.CodeAnalysis namespace) which greatly facilitates the analysis of C# (and VB) code. Although it's great that we can show the results of simple expressions, view engines are immensely more useful if they can make use of variable values supplied by the controller. Stay tuned for the next post in which we examine how we can use the values added to the ViewData in the compiled view template.