3 TIL writing a JSON Parser in Typescript

Greg Clark — Mon, 23 Nov 2020 14:25:34 +0000

I decided one afternoon to write a JSON parser to improve my Typescript skills and reinforce what I learned from Ruslan Spivak's excellent blog series Let's Build a Simple Interpreter.

I used the JSON Interchange Standard - 2nd addition as a reference for the implementation

The JSON Parser has 3 main components:

a Lexer to tokenize the JSON string
a Parser to interpret the JSON grammer, and
a JSONBuilder to create the Javascript representation of the JSON.

Here is what I learned

1. I write better and faster code with Typescript.

I wrote each of the 3 main components in turn, the Lexer, Parser, and JSON builder. Nearly 600 lines of code in total in one afternoon. I'd write maybe 200 lines at a time then test it out. Not once did I receive compile time errors. I was truly shocked, fully expecting some error. I only had logic errors in my code. I was literally expecting to see a misnamed variable or some other error but I didn't see any complaints. All of these issues were caught in the editor while I was typing. This allowed me to fix 'would-be' errors on the fly as I was creating the program. Typescript just makes better code right out of the gate. Since I didn't have these compile time errors to find and fix Typescript allowed me to move faster than I would if I would have used Javascript alone.

2. Its easy to allow Javascript to use your Typescript npm packages but you have to plan for it.

After getting my parser completed, I was excited to publish to npm. This is the first package I ever published to npm. To test it out I created a parse-test project on my local machine and installed ts-jsonparse. It failed spectacularly! I did not configure this project to succeed using npm. I needed to make simple changes to tsconfig.json and package.json.

Summary of tsconfig.json changes:

Add "declaration": true to create .d.ts definition file.
Add "outDir": "dist" to have your output sent to a dist/ folder.

Summary of package.json changes:

Add "main": "dist/index.js","types": "dist/index.d.ts" to use dist/ directory for the main entry point and where Typescript should look for .d.ts files.
Add "build": "tsc" a build script for Typescript.

3. Writing a JSON Parser is simple

If you were ever wondered how a Parser works I encourage you to read through this code as a simple example. The parser consists of 3 main components, the Lexer, Parser, and JSONBuilder.
NOTE: The parser needs to handle exponential numbers and strings with '\' backslash to be complete.

Lexer

The job of the Lexer is to break the parts of the string into tokens. For JSON we look for the following tokens.

export enum eTokens {
  BEGIN_ARRAY = '[', // left square bracket
  BEGIN_OBJECT = '{', // left curly bracket
  END_ARRAY = ']', // right square bracket
  END_OBJECT = '}', // right curly bracket
  COLON = ':',
  COMMA = ',',
  TRUE = 'TRUE',
  FALSE = 'FALSE',
  NULL = 'NULL',
  NUMBER = 'NUMBER',
  STRING = 'STRING',
  EOF = 'EOF',
}

Parser

The goal of the parser is to find structure in the stream of tokens and implement our grammer. I choose to define an Abstract Syntax Tree (AST) to represent an intermediate form of the JSON string. I choose for the AST to include 4 Nodes:

jObject - to represent the object
jArray - to represent the array
jNameValue - to represent a name - value pair
jPrimative - to represent a number, string, true, false, or null

The Parser implements the following JSON Grammer to build the AST.

value : object
| array
| NUMBER
| STRING
| TRUE
| FALSE
| NULL

object : BEGIN_OBJECT name_value_list END_OBJECT

array : BEGIN_ARRAY value_list END_ARRAY

name_value_list: name_value
| name_value COMMA name_value_list

name_value: string COLON value

value_list: value
| value COMMA value_list

JSONBuilder

The JSONBuilder walks the AST and builds the Javascript representation of the JSON string.

If you are interested in the code you can find it in github ts-jsonparse.

DEV Community: Greg Clark