DEV Community: Chris

Parsing in C#

Chris — Mon, 12 Aug 2024 12:31:12 +0000

Introduction

Silverfly is a robust framework designed for building parsers with a focus on reusability through a composition pattern. This approach allows you to reuse existing parsers or components seamlessly. But why create another parsing framework? As someone who has developed numerous parsers—sometimes by generating them from EBNF, other times by coding from scratch—I often found the process cumbersome. The need to learn a new language and the lack of control over how the code functions or how error messages are generated were frustrating. That’s why I decided to create Silverfly—a framework that maximizes reusability and control.

To demonstrate just how straightforward it is to build a parser with Silverfly, this article will guide you through the creation of a JSON parser.

The JsonGrammar Class

At the core of our JSON parser is the JsonGrammar class, which extends the base Parser class. This class is where we define the rules for lexing and parsing JSON data.

public class JsonGrammar : Parser
{
    protected override void InitLexer(LexerConfig lexer)
    {
        // Lexer configuration
    }

    protected override void InitParser(ParserDefinition def)
    {
        // Parser configuration
    }
}

Configuring the Lexer

In this setup, we define keywords for null, true, and false, instruct the lexer to ignore whitespace, and enable it to recognize boolean values, numbers, and strings.

protected override void InitLexer(LexerConfig lexer)
{
    lexer.AddKeywords("null", "true", "false"); //true and false are only registered for the repl syntax highlighting
    lexer.IgnoreWhitespace();
    lexer.MatchBoolean();
    lexer.MatchNumber(false, false);
    lexer.MatchString("\"", "\"");
}

Configuring the Parser

Next, we configure the parser in the InitParser method:

protected override void InitParser(ParserDefinition def)
{
    def.AddCommonLiterals();
    def.Register("{", new ObjectParselet());
    def.Register("null", new NullParselet());
    def.Register("[", new JsonArrayParselet());
}

Here, we add common literals (likely for numbers, strings, and boolean values) and register specialized parselets to handle objects ({), null values, and arrays ([).

Parselets

Parselets are specialized classes responsible for parsing specific JSON structures:

ObjectParselet: Manages JSON objects, such as {"key": "value"}
NullParselet: Handles null values.
JsonArrayParselet: Parses JSON arrays, such as [1, 2, 3].

Parsing Null-Values

Parsing a null value is straightforward. We implement the IPrefixParselet interface and return a LiteralNode:

public class NullParselet : IPrefixParselet
{
    public AstNode Parse(Parser parser, Token token)
    {
        return new LiteralNode(null, token);
    }
}

Parsing JSON Objects

Now, let's dive deeper into the ObjectParselet class, which handles JSON object parsing:

public class ObjectParselet : IPrefixParselet
{
    public AstNode Parse(Parser parser, Token token)
    {
        // Dictionary to store key-value pairs of the JSON object
        var objectMembers = new Dictionary<string, AstNode>();

        // Continue parsing until we encounter a closing brace
        while (!parser.Match("}"))
        {
            // Parse the key, which must be a string
            var keyToken = parser.Consume(PredefinedSymbols.String);
            var key = keyToken.Text.ToString();

            // Consume the colon separator
            parser.Consume(":");

            // Parse the value associated with the key
            var value = parser.ParseExpression();

            // Add the key-value pair to our dictionary
            objectMembers[key] = value;

            // If we don't find a comma, we've reached the end of the object
            if (!parser.Match(","))
            {
                break;
            }
        }

        // Consume the closing brace
        parser.Consume("}");

        // Create and return a new JsonObject node with our parsed members
        return new JsonObject(objectMembers);
    }
}

This code builds a dictionary of key-value pairs, continually parsing until it encounters a closing brace (}), at which point it returns a JsonObject node containing the parsed members.

Parsing JSON Arrays

Finally, let's explore the JsonArrayParselet class, which is tasked with parsing JSON arrays. Like other parselets, it implements the IPrefixParselet interface, maintaining consistency within the framework.

class JsonArrayParselet : IPrefixParselet
{
    public AstNode Parse(Parser parser, Token token)
    {
        var elements = parser.ParseSeperated(",", "]");

        return new JsonArray(elements);
    }
}

This class efficiently parses an array by identifying elements separated by commas until it reaches a closing bracket (]).

Using the Parser

To use the JSON parser, you would typically create an instance of the JsonGrammar class and then call the Parse method:

var jsonGrammar = new JsonGrammar();
var result = jsonGrammar.Parse(jsonString);

Advantages of this Approach

Flexibility: The framework’s design makes it simple to support new data formats by creating new Grammar classes.
Extensibility: Complex structures can be handled effortlessly by adding new Parselets.
Readability: The declarative style of the Grammar definition ensures the code is easy to read and maintain.

Conclusion

The Silverfly parser framework offers a powerful and adaptable method for parsing JSON. By clearly separating lexing and parsing, and utilizing specialized Parselets and matchers, Silverfly enables efficient and structured handling of JSON data. Moreover, this approach can easily be extended to other data formats or languages, making it a solid foundation for a variety of parsing tasks in your projects.

Appendix

The Silverfly framework is not just limited to parsing JSON—it's a versatile tool with several advanced features that can enhance your development experience.

Pretty Error Messages

Silverfly can display pretty error messages in the console.

REPL

Silverfly also excels in interactive programming environments. It offers built-in support for creating Read-Eval-Print Loops (REPLs) with syntax highlighting. This feature is particularly valuable for developers who want to test and evaluate code snippets on the fly. The syntax highlighting not only makes the code more readable but also helps in identifying syntax errors early, leading to a smoother and more intuitive development process.

Experience with rewriting a complete application

Chris — Thu, 15 Dec 2022 07:21:53 +0000

Hello friends of programming,

in this post I am going to tell you my experiences of rewriting a working application from scratch.

But first, which application?

I have done a lots of work for my tool for the Remarkable tablet called Slithin. But there were some problems. I have not known that the application would be so complex and big. So I started programming without any plan. Later there were some problems and I have decided to use an IOC-Container to fix the dependency hell. But for some reason now there are such a big amount of code smells from the old synchronisation algorithm that cannot be easily removed and the dependency problem is not complete solved.

Because I have worked most of my time on my programming language I leaved the focus on this project but since I had the idea of making a framework for modular applications I had decided to rewrite Slithin with my custom framework. The application structure is now modular. You can simply replace them and they are seperated by other modules.

But what is a module?

A module consist of two assemblies. One for the logic and the other for the models. A module can register services that other modules can interact with but it can also invoke custom logic with a OnStart method.

Modules can communicate without any dependencies. Every module has an Inbox and an Outbox to send/receive messages. You can simply send data or trigger some actions on specific kind of messages. So modules can be loosly coupled.

A module can also have sub-actions that can be scheduled. A module can invoke for example writing the log cache to file every 10 minutes.

Because the modulesystem is so flexible you can create a model to specify the contract you can implement different modules for different providers like different databases.

The framework has some loosly coupled services/modules implemeted by default for example for logging with different logging frameworks.

So back to slithin:

Slithin has now only a bootstrapper executable that loads the modules and start them. Each functionality is now splitted into it's own module. The userinterface, the file synchronisation, the communication with the device and the communication with the filesystem for example. This modularity gives the opportunity to handle dependencies easily. The only thing I have to think of is which modules have to be loaded first to register the services that other modules need.

I have learned so much with this project and for the future I am going to make a plan for new applications (requirements and code architecture).

If you want to take a look here are the repositories:
Slithin

AuroraModularis - framework for modular applications

This was my little story.

Backlang- Status Report

Chris — Wed, 30 Nov 2022 07:38:01 +0000

Hello friends of programming,

This is the first post about the status of backlang. But first, why making a new language? I can answer this question really quickly. A big reason for me is I want to reduce my daily code that I write. The compiler should generate automaticly what it needs to create a runnable executable. I also really love making languages. In the past I have created some Domain Specific Languages and I have discovered many ways to write parsers. From parser generators to parser combinators over parsers from scratch. Backlang helps me a lot to learn more about formal languages and compilers.

So here are the first things that are already implemented:

Declare classes/structs/enums/functions/unions/discriminated unions/interfaces
Extend types, also types that are not part of your assembly
Generate overloads for different primitve types
Generate ToString, Constructor and GetHashCode for structs
Declare mutable and immutable variables
Assignments
Type deduction for values/members
default and typeof expressions
Macro expansion
Operator overloading
Array declaration
Pointer and reference types/reference
Reference parameters
Type deduction for variables
Property declarations
Conditions
Import namespace with wildcard(import all namespaces one layer deep)
While/Do-While loops

You can see there is a lot implemented but unfortunately not enough to write working programs. There are some features implemented like if-else-statements but they work not as supposed to do.

But I have good news. I make really good progress. I have implemented such new good things like automatic return type deduction for functions. Now you don't need to specify the return type in most of the simple cases. I'll extend the return type deducing so in future you can miss types for complex types like for generics.
But how does the return type deduction work?

The implementation is really simple. First it checks if they can be implicit casted through a simple map or an operator overload if at least one is defined. If they cannot be casted the algorithm looks for the parent types. If the algorithm cannot find a common type between those types it looks recursive on the next layer of inheritance. If the type is System.ValueType or System.Object the algorithm stops and throws an error.

That was the compiler. But what differs a programming language from a good programming language? - The tooling to make it easier to write code and find errors.

I've been working on a visual studio code extension since the parser is functional. The extension has simple syntax highlighting, code folding and for now "simple" code completion for keywords. Simple means that the language server cannot suggest names and other stuff for now. But the completion system in the language server is clever. It suggests not all keywords, only those which are valid in a certain context like the mut keyword it is only valid in a variable declaration.

As you can see I give all my passion to this language. I really want to make it usable. But making a new programming language is really time consuming for one man. There is not only the compiler and the vscode extension. A huge part is also the documentation and the website.

If you like this language please share it with your friends or contribute. I really need help in some parts like integrating the debugger into the vs code extension.

This was a little journey to backlang. I hope you enjoyed it and see you the next post.