Build your own debugger using Roslyn's syntax analysis

// This is the 23rd article of C# Advent Calendar 2018 in Japan.
// The original article is Roslyn の構文解析を使ってデバッガーを自作する.

I tried to make something like a debugger.

Motivation

• I normally use Visual Studio, but it is troublesome to step through code manually at debugging
  • When the number of steps is large in a loop or the like
  • When I want to check the state of branching or variables lightly

Solution

• Specify only the time interval of the step and let the debugger run automatically
  • A list of variables is displayed
  • Time interval can be adjusted in real time
• Using the syntax analysis of .NET Compiler Platform (Roslyn), it seems to be realizable if we insert debugging code between each step

Result

So, as a result of trying to make prototype "Tick-tack Debugger" with WPF, it looked like this.
As an example, we are figuring out a square root by the Newton's method.

Commentary

The following is an outline technical explanation.
Before creating a WPF application, first try a console application on the .NET Framework.
To use C# syntax analysis, install Microsoft.CodeAnalysis.CSharp via NuGet.

It is a policy to insert debugging code into the source code to be debugged, compile it dynamically and execute it.
The source code of the console application is shown below (the whole solution is in SyntaxTreeSample).

	using System;
	using System.CodeDom.Compiler;
	using System.Collections.Generic;
	using System.IO;
	using System.Linq;
	using System.Text;
	using System.Threading;
	using DebuggerLib;
	namespace DebuggerConsole
	{
	class Program
	{
	const string SourcePath = @"..\..\..\NumericConsole\Program.cs";
	const string GeneratedPath = @"Program.g.cs";
	static void Main(string[] args)
	{
	// Generates the code for debugging.
	var sourceCode = File.ReadAllText(SourcePath);
	var generatedCode = SyntaxHelper.InsertBreakpoints(sourceCode);
	File.WriteAllText(GeneratedPath, generatedCode, Encoding.UTF8);
	// Compiles and loads the assembly.
	var provider = CodeDomProvider.CreateProvider("CSharp");
	var compilerOption = new CompilerParameters(new[] { "System.Core.dll", "DebuggerLib.dll" }) { GenerateExecutable = true };
	var compilerResult = provider.CompileAssemblyFromFile(compilerOption, GeneratedPath);
	if (compilerResult.Errors.HasErrors) return;
	// Registers the action for breakpoints.
	DebugHelper.InfoNotified += (spanStart, spanLength, variables) =>
	{
	Console.WriteLine(string.Join(", ", variables.Select(v => $"{v.Name}: {v.Value}")));
	Console.WriteLine(sourceCode.Substring(spanStart, spanLength));
	Thread.Sleep(1000);
	};
	// Calls the Main method.
	var entryPoint = compilerResult.CompiledAssembly.EntryPoint;
	entryPoint.Invoke(null, new object[] { new string[0] });
	}
	}
	}

view raw Program.cs hosted with ❤ by GitHub

	using System;
	using System.Collections.Generic;
	using System.Linq;
	using Microsoft.CodeAnalysis;
	using Microsoft.CodeAnalysis.CSharp;
	using Microsoft.CodeAnalysis.CSharp.Syntax;
	using Microsoft.CodeAnalysis.Text;
	using DebugStatement = System.ValueTuple<Microsoft.CodeAnalysis.CSharp.Syntax.StatementSyntax, string[]>;
	namespace DebuggerConsole
	{
	public static class SyntaxHelper
	{
	public static string InsertBreakpoints(string sourceCode)
	{
	var root = ParseText(sourceCode);
	var method = root.DescendantNodes()
	.OfType<MethodDeclarationSyntax>()
	.FirstOrDefault(m => m.Identifier.ValueText == "Main")
	?? throw new FormatException("The Main method is not found.");
	var statements = DetectStatements(method);
	var result = sourceCode;
	foreach (var (statement, variables) in statements.Reverse())
	{
	var (span, debugIndex) = GetSpan(statement);
	result = result.Insert(debugIndex, $"DebugHelper.NotifyInfo({span.Start}, {span.Length}{ToParamsArrayText(variables)});\r\n");
	}
	return result.Insert(root.Usings.FullSpan.End, "using DebuggerLib;\r\n");
	}
	public static CompilationUnitSyntax ParseText(string text)
	{
	var tree = CSharpSyntaxTree.ParseText(text);
	var diagnostics = tree.GetDiagnostics().ToArray();
	if (diagnostics.Length > 0) throw new FormatException(diagnostics[0].ToString());
	return tree.GetCompilationUnitRoot();
	}
	public static DebugStatement[] DetectStatements(MethodDeclarationSyntax method)
	{
	var statements = new List<DebugStatement>();
	DetectStatements(method.Body, statements, new List<(string, SyntaxNode)>());
	return statements.ToArray();
	}
	static void DetectStatements(SyntaxNode node, List<DebugStatement> statements, List<(string name, SyntaxNode scope)> variables)
	{
	// Adds variables.
	if (node is VariableDeclarationSyntax varSyntax)
	{
	var varNames = varSyntax.Variables.Select(v => v.Identifier.ValueText).ToArray();
	var scope = ((node.Parent is LocalDeclarationStatementSyntax) ? node.Parent : node)
	.Ancestors()
	.First(n => n is StatementSyntax);
	variables.AddRange(varNames.Select(v => (v, scope)));
	}
	// Maps variables to the statement.
	if ((node is StatementSyntax statement) &&
	!(node is BlockSyntax) &&
	!(node is BreakStatementSyntax))
	statements.Add((statement, variables.Select(v => v.name).ToArray()));
	// Recursively.
	foreach (var child in node.ChildNodes())
	DetectStatements(child, statements, variables);
	// Maps variables to the last line of the block.
	if (node is BlockSyntax block)
	statements.Add((block, variables.Select(v => v.name).ToArray()));
	// Clears variables out of the scope.
	if (node is StatementSyntax)
	for (var i = variables.Count - 1; i >= 0; i--)
	if (variables[i].scope == node)
	variables.RemoveAt(i);
	else
	break;
	}
	static (TextSpan, int) GetSpan(StatementSyntax statement)
	{
	switch (statement)
	{
	case ForStatementSyntax f:
	var span = new TextSpan(f.ForKeyword.Span.Start, f.CloseParenToken.Span.End - f.ForKeyword.Span.Start);
	return (span, statement.FullSpan.Start);
	case BlockSyntax b:
	return (b.CloseBraceToken.Span, b.CloseBraceToken.FullSpan.Start);
	default:
	return (statement.Span, statement.FullSpan.Start);
	}
	}
	static string ToParamsArrayText(string[] variables) =>
	string.Concat(variables.Select(v => $", new Var(\"{v}\", {v})"));
	}
	}

view raw SyntaxHelper.cs hosted with ❤ by GitHub

In the SyntaxHelper class, C# source code to be debugged is converted to a syntax tree and scanned, and a line of code for debugging is inserted before each statement.

By using the CSharpSyntaxTree.ParseText method, you can convert the source code to a SyntaxTree object.
Also, the superclass representing all nodes, such as methods, statements and expressions, is the SyntaxNode class,

• Parent property
• Ancestors method
• ChildNodes method
• DescendantNodes method

if you know these members, you can scan nodes.

In addition to this, DebuggerLib is created as a class library that defines methods called from code for debugging.
Let this library go through to notify the position of each statement and, the variables and their values that exist immediately before it.

In the Program class, after saving the generated debugging source code to a file, compile it using CodeDomProvider in the System.CodeDom.Compiler namespace, and call its entry point (the Main method).
It also registers the event handler when the debug code is executed, and pauses the program for the specified time using the Thread.Sleep method.

Now, if the original source code to be debugged is Program.cs below, the following Program.g.cs is generated as the debugging source code.

	using System;
	namespace NumericConsole
	{
	class Program
	{
	static void Main(string[] args)
	{
	// Square root by the Newton's method.
	var a = 5.0;
	var x = a;
	for (var i = 0; i < 100; i++)
	{
	var xi = (x + a / x) / 2;
	if (x == xi) break;
	x = xi;
	}
	Console.WriteLine(x);
	}
	}
	}

view raw Program.cs hosted with ❤ by GitHub

	using System;
	using DebuggerLib;
	namespace NumericConsole
	{
	class Program
	{
	static void Main(string[] args)
	{
	DebugHelper.NotifyInfo(188, 12);
	// Square root by the Newton's method.
	var a = 5.0;
	DebugHelper.NotifyInfo(214, 10, new Var("a", a));
	var x = a;
	DebugHelper.NotifyInfo(240, 29, new Var("a", a), new Var("x", x));
	for (var i = 0; i < 100; i++)
	{
	DebugHelper.NotifyInfo(302, 25, new Var("a", a), new Var("x", x), new Var("i", i));
	var xi = (x + a / x) / 2;
	DebugHelper.NotifyInfo(347, 19, new Var("a", a), new Var("x", x), new Var("i", i), new Var("xi", xi));
	if (x == xi) break;
	DebugHelper.NotifyInfo(384, 7, new Var("a", a), new Var("x", x), new Var("i", i), new Var("xi", xi));
	x = xi;
	DebugHelper.NotifyInfo(405, 1, new Var("a", a), new Var("x", x), new Var("i", i), new Var("xi", xi));
	}
	DebugHelper.NotifyInfo(422, 21, new Var("a", a), new Var("x", x));
	Console.WriteLine(x);
	DebugHelper.NotifyInfo(453, 1, new Var("a", a), new Var("x", x));
	}
	}
	}

view raw Program.g.cs hosted with ❤ by GitHub

When we run the console application we created, it will look like the following figure (the time interval is 0.3 seconds).

Based on the above, we created a debugging tool as a WPF application.
The part of the C# source code on the left is a TextBox and can be edited.
When debugging is executed, highlighting is done by setting each statement to the selected state.
The part where the variable list is displayed on the right is a DataGrid.

(An example of figuring out the pi)

This time I tried to make a prototype by the above method, but I think that there is a smarter way to insert debug code and compile.

Remarks

• We can not deal with all possible statements. Also, we only scan the Main method.
• The assemblies generated at compile time (EXE) are saved in the %TEMP% folder (user's AppData\Local\Temp).
• In the TextBox, setting IsInactiveSelectionHighlightEnabled to True may not work. In addition, highlights in the selected state may slip. It might be better to use Run etc in RichTextBox.

Samples Created

• SyntaxTreeSample (GitHub)

Version

• .NET Framework 4.7
• Microsoft.CodeAnalysis.CSharp 2.10.0
• ReactiveProperty 5.3.2

References

• .NET Compiler Platform SDK (Roslyn API)