All about Abstract Syntax Tree

Hello there! In this article, I'll tell you what AST is and how it can help you in the context of JavaScript.

What AST is

In computer science, an abstract syntax tree (AST), or just syntax tree, is a tree representation of the abstract syntactic structure of source code written in a programming language. Each node of the tree denotes a construct occurring in the source code.

by Wikipedia

In short, AST is just an object representation of source code.

Example

Let's take a look at the example of source code:

const me = { name: 'Paul' }

For this, AST may look like that:

{
  "type": "Program",
  "start": 0,
  "end": 27,
  "body": [
    {
      "type": "VariableDeclaration",
      "start": 0,
      "end": 27,
      "declarations": [
        {
          "type": "VariableDeclarator",
          "start": 6,
          "end": 27,
          "id": {
            "type": "Identifier",
            "start": 6,
            "end": 8,
            "name": "me"
          },
          "init": {
            "type": "ObjectExpression",
            "start": 11,
            "end": 27,
            "properties": [
              {
                "type": "Property",
                "start": 13,
                "end": 25,
                "method": false,
                "shorthand": false,
                "computed": false,
                "key": {
                  "type": "Identifier",
                  "start": 13,
                  "end": 17,
                  "name": "name"
                },
                "value": {
                  "type": "Literal",
                  "start": 19,
                  "end": 25,
                  "value": "Paul",
                  "raw": "'Paul'"
                },
                "kind": "init"
              }
            ]
          }
        }
      ],
      "kind": "const"
    }
  ],
  "sourceType": "module"
}

Looks pretty simple, right? You can watch this example or write your own in the online playground: astexplorer

Tools

There are many tools for creating AST from javascript source code, for example:

• acorn
• babel-parser
• esformatter
• jscodeshift

In the rest of the article, I'll use jscodeshift because it has a simple API for transforming AST into another state.

When to use

Code analisys

On top of AST have been written many tools that every front-end developer uses every day, such as eslint and prettier.

Migration scripts

Sometimes you need to migrate from one version of a library to another, for example, when React has updated to version 15.5 the PropTypes has moved to different package and the react-developers provides codemod for migration.

Codemod is just a code that takes source code, converts it to AST, does some manipulation, and returns new source code.

You can write your own codemod in easy way, we’ll see it little bit later in this article.

Code generation

For example, you can write script that will generate documentation from you source code using JSDoc or Typescript.

Real-world example

Some time ago, I had a task in which I needed to transform API for a method that checks client metrics (like clicks on elements) in integration tests:

From:

browser.checkMetrics({
    path: '$page.$main.some-block.some-element',
    attrs: {
        action: 'click',
        someData: {
             type: 'kind'
        }
    }
});

To:

browser.checkMetrics({
    path: '/$page/$main/some-block/some-element[@action="click" and @someData@id=1]',
});

In some cases, it can be performed manually, but we have hundreds of this method calls. So, I've decided to write a codemod.

Solution

async function transformMethod(filePath) {
  const source = await asyncReadFile(filePath);

  const root = j(source.toString('utf-8'))
    .find(j.CallExpression)
    .filter(({ value: callExpression }) => {
      const { property } = callExpression.callee;

      return property && property.name === 'checkMetrics';
    })
    .forEach(({ value: callExpression }) => {
      const checkObjectPath = callExpression.arguments[0];
      const checkObject = parseObject(checkObjectPath);
      const attrsString = checkObject.attrs ? `[${attrsIntoString(checkObject.attrs)}]` : '';
      const path = `/${checkObject.path.replace(dotRegexp, '/')}${attrsString}`;

      // Remove `attrs` property from original object
      checkObjectPath.properties = checkObjectPath.properties.filter(({ key }) => key.name !== 'attrs');

      // Find AST representation of `path` property
      const counterPath = checkObjectPath.properties.find(({ key }) => key.name === 'path');

      // Change value of `path` property
      counterPath.value.value = path;
    });

  // For code formatting, try to remove it for see result without it
  const linter = new CLIEngine({ fix: true });
  let newSource = root.toSource({ quote: 'single' });
  let eslintResult;

  try {
    [eslintResult] = linter.executeOnText(newSource, filePath).results;
  } catch (e) {
    console.log(e);
  }

  if (eslintResult.output) {
    newSource = eslintResult.output;
  }

  await asyncWriteFile(filePath, newSource, 'utf-8');
}

Full code you can see in my example repository .

Сonclusion

Thank you for reading! I hope it's been usefull for you and you'll use that information in your tasks.

I'll glad to see any feedback!