Dynamics CRM / Any ASP.NET App - Improve C# dynamic compilation times

The Pain

Since I installed Dynamics 365 (CRM) v9.1 On-Premises, I get REALLY slow compile times. The csc.exe (C# Compiler) is so slow to compile whenever I make unique request, I could wait for many seconds or minutes for compilation to finish. So I was searching for a way to improve things. It doesn't do parallel compilation if I make multiple unique requests from multiple browser tabs and it doesn't utilize CPU that much. A "legacy junk" indeed.

The ASP.net does dynamic compilation on first request. You can read more about it at Understanding ASP.NET Dynamic Compilation.. But from Task Manager it looks something like this:

"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\csc.exe" /noconfig /fullpaths @"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\root\0345f868\a571ab7d\mejffpag.cmdline"

The Painkillers?

As Dynamics 365 is a 3rd party app and I thought I could at least precompile the aspx pages and such. So I did just that:

. C:\windows\Microsoft.NET\Framework64\v4.0.30319\aspnet_compiler.exe -m "/LM/W3SVC/2/ROOT"

As it did take a while to complete, I certainly had good feelings about this. Just to find out csc.exe still doing dynamic compilation when I make requests and the performance for my dev environment hasn't improved at all (just a gut feeling, didn't do measurements). That means I have to take first request hit all of my own for any request. :(

But it did something, as it generated tons of files within Temporary ASP.NET Files folder - probably just not enought.

The Painkillers!

After a while I stumbled upon something interesting: Enabling the .NET Compiler Platform (“Roslyn”) in ASP.NET applications.

Eager to try out and do some not-so-much-supported modifications within 3rd party app web.config that will certainly be overwritten when I apply next patch - but it is a dev env, no real risk of breaking it:

# Got nuget.exe and within arbitrary directory:
nuget.exe install Microsoft.CodeDom.Providers.DotNetCompilerPlatform -Version 3.6.0 -OutputDirectory .\packages\

# note that Dynamics 365 (CRM) 9.1 Is .NET Framework 4.6.2 app.
cp -Recurse ".\packages\Microsoft.CodeDom.Providers.DotNetCompilerPlatform.3.6.0\tools\Roslyn46\*" "C:\Program Files\Dynamics 365\CRMWeb\bin\roslyn"
cp ".\packages\Microsoft.CodeDom.Providers.DotNetCompilerPlatform.3.6.0\lib\net45\*" "C:\Program Files\Dynamics 365\CRMWeb\bin\"

Then edited "C:\Program Files\Dynamics 365\CRMWeb\web.config" by adding <system.codedom> node:

<configuration>
  ...
  <system.codedom>
    <compilers>
      <compiler language="c#;cs;csharp" extension=".cs"
        type="Microsoft.CodeDom.Providers.DotNetCompilerPlatform.CSharpCodeProvider, Microsoft.CodeDom.Providers.DotNetCompilerPlatform, Version=3.6.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"
        warningLevel="4" compilerOptions="/langversion:7.3 /nowarn:1659;1699;1701"/>
      <compiler language="vb;vbs;visualbasic;vbscript" extension=".vb"
        type="Microsoft.CodeDom.Providers.DotNetCompilerPlatform.VBCodeProvider, Microsoft.CodeDom.Providers.DotNetCompilerPlatform, Version=3.6.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"
        warningLevel="4" compilerOptions="/nowarn:41008 /define:_MYTYPE=\&quot;Web\&quot; /optionInfer+"/>
    </compilers>
  </system.codedom>
</configuration>

I did validate that it was using the new compiler: csc.exe was from bin/roslyn directory and I saw a VBCSCompiler.exe process too. That verifies that web.config changes are correct and assemblies are in correct places.

Restart-Service W3SVC wasn't needed, as changes to web.config are detected and app pool gets recycled.

Performance improvements

So I had to measure performance improvement. I opted for UIforETW to initiate hassle free etw trace and Windows Performance Analyzer (WPA) to see the results. I wanted more scientific results than a wall clock :)

• I did a run with roslyn compiler and other run with builtin compiler.
• I did open root Dynamics CRM page (a dashboard). I still use Legacy interface, but it doesn't prevent us making a point on csc.exe performance.
• Compilation resulted in generating 440 files.
• I did purge ASP.NET Temporary Files before each run.

Legacy csc.exe

1. Purge files before run

gci -Recurse "C:\windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\root\0345f868\a571ab7d" | ? lastwritetime -ge '2023-04-18' | ? PsIsContainer -eq $false | Remove-Item -Force

1. Refresh page

2. Look at trace with WPA - 6 times csc.exe process was called. Time since first spawn and last one ending 199sec = 3m19sec.

   

Roslynized csc.exe

1. Purge files before run
2. Refresh Page
3. 116sec = 1m56sec

Results

Okay, so that was kind of total refresh for a single page that makes many calls - normally things would be somewhat cached. I feel that navigating to other pages feels faster, but still I wouldn't say that it is fast.

I'll leave roslyn compiler on as any performance improvement will help - I feel that normally navigating through pages is faster. It helps against doing context switching to other tasks while waiting on compilation.

Precompiling ASP.NET Framework app

We covered dynamic compilation - a process where compilation for relevant files happens when making request and compiling appropriate files - and that request is taking the performance impact.

For ASP.NET apps you develop, you can precompile your website, so that dynamic compilation never takes place and your users don't have to wait.

I also have access to a ASP.NET Framework 4.8 application and I did want to improve build times there too. It usually took 34 minutes for aspnet_compiler.exe to do its precompilation job. All I had to do was to install Microsoft.CodeDom.Providers.DotNetCompilerPlatform package (latest 4.1.0 version), it did the web.config changes and copied required files to bin directory - the result was immediatelly visible - instead of 34 minutes, now it did precompilation in ~13 minutes. Solid 20 minutes shaved off!

As for precompilation, it wasn't immediatelly obvjous that it is using roslyn csc.exe - because I didn't see neither VBCSCompiler.exe or csc.exe, just aspnet_compiler.exe doing its job. But the results speak of themselves.

This topic was very helpful in getting up and running with roslyn-ized csc.exe compiler: How to speed up aspnet_compiler.exe?