Parsing in C#

#csharp #parser #dsl #dotnet

Introduction

Silverfly is a robust framework designed for building parsers with a focus on reusability through a composition pattern. This approach allows you to reuse existing parsers or components seamlessly. But why create another parsing framework? As someone who has developed numerous parsers—sometimes by generating them from EBNF, other times by coding from scratch—I often found the process cumbersome. The need to learn a new language and the lack of control over how the code functions or how error messages are generated were frustrating. That’s why I decided to create Silverfly—a framework that maximizes reusability and control.

To demonstrate just how straightforward it is to build a parser with Silverfly, this article will guide you through the creation of a JSON parser.

The JsonGrammar Class

At the core of our JSON parser is the JsonGrammar class, which extends the base Parser class. This class is where we define the rules for lexing and parsing JSON data.

public class JsonGrammar : Parser
{
    protected override void InitLexer(LexerConfig lexer)
    {
        // Lexer configuration
    }

    protected override void InitParser(ParserDefinition def)
    {
        // Parser configuration
    }
}

Configuring the Lexer

In this setup, we define keywords for null, true, and false, instruct the lexer to ignore whitespace, and enable it to recognize boolean values, numbers, and strings.

protected override void InitLexer(LexerConfig lexer)
{
    lexer.AddKeywords("null", "true", "false"); //true and false are only registered for the repl syntax highlighting
    lexer.IgnoreWhitespace();
    lexer.MatchBoolean();
    lexer.MatchNumber(false, false);
    lexer.MatchString("\"", "\"");
}

Configuring the Parser

Next, we configure the parser in the InitParser method:

protected override void InitParser(ParserDefinition def)
{
    def.AddCommonLiterals();
    def.Register("{", new ObjectParselet());
    def.Register("null", new NullParselet());
    def.Register("[", new JsonArrayParselet());
}

Here, we add common literals (likely for numbers, strings, and boolean values) and register specialized parselets to handle objects ({), null values, and arrays ([).

Parselets

Parselets are specialized classes responsible for parsing specific JSON structures:

ObjectParselet: Manages JSON objects, such as {"key": "value"}
NullParselet: Handles null values.
JsonArrayParselet: Parses JSON arrays, such as [1, 2, 3].

Parsing Null-Values

Parsing a null value is straightforward. We implement the IPrefixParselet interface and return a LiteralNode:

public class NullParselet : IPrefixParselet
{
    public AstNode Parse(Parser parser, Token token)
    {
        return new LiteralNode(null, token);
    }
}

Parsing JSON Objects

Now, let's dive deeper into the ObjectParselet class, which handles JSON object parsing:

public class ObjectParselet : IPrefixParselet
{
    public AstNode Parse(Parser parser, Token token)
    {
        // Dictionary to store key-value pairs of the JSON object
        var objectMembers = new Dictionary<string, AstNode>();

        // Continue parsing until we encounter a closing brace
        while (!parser.Match("}"))
        {
            // Parse the key, which must be a string
            var keyToken = parser.Consume(PredefinedSymbols.String);
            var key = keyToken.Text.ToString();

            // Consume the colon separator
            parser.Consume(":");

            // Parse the value associated with the key
            var value = parser.ParseExpression();

            // Add the key-value pair to our dictionary
            objectMembers[key] = value;

            // If we don't find a comma, we've reached the end of the object
            if (!parser.Match(","))
            {
                break;
            }
        }

        // Consume the closing brace
        parser.Consume("}");

        // Create and return a new JsonObject node with our parsed members
        return new JsonObject(objectMembers);
    }
}

This code builds a dictionary of key-value pairs, continually parsing until it encounters a closing brace (}), at which point it returns a JsonObject node containing the parsed members.

Parsing JSON Arrays

Finally, let's explore the JsonArrayParselet class, which is tasked with parsing JSON arrays. Like other parselets, it implements the IPrefixParselet interface, maintaining consistency within the framework.

class JsonArrayParselet : IPrefixParselet
{
    public AstNode Parse(Parser parser, Token token)
    {
        var elements = parser.ParseSeperated(",", "]");

        return new JsonArray(elements);
    }
}

This class efficiently parses an array by identifying elements separated by commas until it reaches a closing bracket (]).

Using the Parser

To use the JSON parser, you would typically create an instance of the JsonGrammar class and then call the Parse method:

var jsonGrammar = new JsonGrammar();
var result = jsonGrammar.Parse(jsonString);

Advantages of this Approach

Flexibility: The framework’s design makes it simple to support new data formats by creating new Grammar classes.
Extensibility: Complex structures can be handled effortlessly by adding new Parselets.
Readability: The declarative style of the Grammar definition ensures the code is easy to read and maintain.

Conclusion

The Silverfly parser framework offers a powerful and adaptable method for parsing JSON. By clearly separating lexing and parsing, and utilizing specialized Parselets and matchers, Silverfly enables efficient and structured handling of JSON data. Moreover, this approach can easily be extended to other data formats or languages, making it a solid foundation for a variety of parsing tasks in your projects.

Appendix

The Silverfly framework is not just limited to parsing JSON—it's a versatile tool with several advanced features that can enhance your development experience.

Pretty Error Messages

Silverfly can display pretty error messages in the console.

REPL

Silverfly also excels in interactive programming environments. It offers built-in support for creating Read-Eval-Print Loops (REPLs) with syntax highlighting. This feature is particularly valuable for developers who want to test and evaluate code snippets on the fly. The syntax highlighting not only makes the code more readable but also helps in identifying syntax errors early, leading to a smoother and more intuitive development process.