Demystification of Macros in Nim

Jason Beetham — Fri, 04 Sep 2020 06:49:47 +0000

At first glance Nim’s metaprogramming certainly may drive you to scream and run away from your computer to the nearest ditch. Luckily though macros are not overly complicated, they are just a way for programmers to write code with code so we can generate logic and behaviour programatically. This write-up will document creating macros.

Implementing Go’s Walrus Operator

Go has an operator which defines a variable and sets the value without having a keyword. The following is what it looks like in Go.

a := 300

This can be done rather simply in Nim, but first let’s look at what we need using dumpTree.

import macros
dumpTree:
  var a = "Test"

The compiler will output anything below dumpTree, so now we can see the AST required for that code.

StmtList
  VarSection
    IdentDefs
      Ident "a"
      Empty
      StrLit "Test"

So if we want to automate this we see that we need the Varsection, an ident, and a value. So let's look at two ways of doing this.

Method #1

import macros
macro `:=`(name, value: untyped): untyped = newVarStmt(name, value)

That's it, we did it, to be certain though we can use the .repr procedure to check.

import macros
macro `:=`(name, value: untyped): untyped = 
  result = newVarStmt(name, value)
  echo result.repr

a := "Test"

Running the above code we will see the compiler echo out the desired var a = "Test"

Method #2

This method is equally as short, but uses a nice tool. quote do lets you write code and it will generate the AST for you, so it's as simple as below. The same checks can be done like above, but I will skip them for the sake of reducing redundancy, as being redundant and repeating yourself is redundant.

import macros
macro `:=`(name, value: untyped): untyped = 
  quote do:
    var `name` = `value`
a := "Test"

Making a Compact If Statement

Sometimes you will want to change the behaviour or function of the language, although Nim does not let you change the syntax you can make your own block logic. One case of this is making a custom if statement. Below is a desired implementation

let a = "Yellow"
expandIf:
  a == "Hello": echo "Good bye"
  a == "Yellow":
    echo "Would be a lot cooler if you liked blue."
    echo "Yellow sucks"
  _: echo "You did not speak."

The first condition will be if, the following conditions will be elif and finally _ will be used for the else branch. This means no repeated keywords in our for fun implementation!

import macros
dumpTree:
  if a == "Hello": echo "Good bye."
  elif a == "Yellow":
    echo "Would be a lot cooler if you liked blue."
    echo "Yellow sucks."
  else: echo "You did not speak."

The above AST looks like such.

StmtList
  IfStmt
    ElifBranch
      Infix
        Ident "=="
        Ident "a"
        StrLit "Hello"
      StmtList
        Command
          Ident "echo"
          StrLit "Good bye."
    ElifBranch
      Infix
        Ident "=="
        Ident "a"
        StrLit "Yellow"
      StmtList
        Command
          Ident "echo"
          StrLit "Would be a lot cooler if you liked blue."
        Command
          Ident "echo"
          StrLit "Yellow sucks."
    Else
      StmtList
        Command
          Ident "echo"
          StrLit "You did not speak."

We can also dumptree the body of our original expandIf idea, and see the resulting AST nodes we can sample from.

StmtList
      Infix
        Ident "=="
        Ident "a"
        StrLit "Hello"
        StmtList
          Command
            Ident "echo"
            StrLit "Good bye"
      Infix
        Ident "=="
        Ident "a"
        StrLit "Yellow"
        StmtList
          Command
            Ident "echo"
            StrLit "Would be a lot cooler if you liked blue."
          Command
            Ident "echo"
            StrLit "Yellow sucks"
      Call
        Ident "_"
        StmtList
          Command
            Ident "echo"
            StrLit "You did not speak."

Looking at both of them you can see in the expandIf body we can get the required infixes for the ElseIfBranch node. If we can seperate the StmtList from each infix, we then can use the newIfStmt macro to generate the elif branch using cond which is the infix and body which is the StmtList.

import macros
macro expandIf(statement: untyped): untyped=
  var
    branches: seq[(NimNode, NimNode)] #Condition, Body
    elseBody: NimNode #Else code
  for cond in statement:
    #Based off the dumpTree, we know this is the else body.
    let ifBody = cond.findChild(it.kind == nnkStmtList)
    if cond.kind == nnkInfix:
      cond.del 3 #Removes Stmtlist
      branches.add((cond, ifBody))
    elif cond.kind == nnkCall and $cond[0] == "_":
      #Based off the dumpTree, we know this is the else body.
      elseBody = ifBody

  result = newIfStmt(branches) #Generates if stmt
  result.add newNimNode(nnkElse).add(elseBody) #Appends else body
  echo result.repr



expandIf:
  11 == 13 : echo "Test"
  12 == 14: echo "Huh"
  _ : echo "duh"

In the above code you can see the extraction using findChild. Removal of the StmtList with the cond.del 3, and finally the creation of the if. When we compile the compiler sends us a nice message thanks to the echo result.repr, which is exactly what we wanted a fully constructed if/else statement.

if 11 == 13:
  echo "Test"
elif 12 == 14:
  echo "Huh"
else:
  echo "duh"

Untyped vs. typed macros

So far we have only looked at the usage of untyped macros, these macros do not have any type information and are parsed then passed to our macro. The other type of macro is a typed macro, these are semantically checked which means they have type information and the code has to be valid. Typed macros are useful for anything that requires introspection.

Our First typed macro

Give me a symbol and I will show you the world. In Nim macros symbols or "syms" are the magic sauce to make our spaghetti, they are semantically checked which means they internally store the type they are. The power this gives is all powerful. The first typed macro we will make is one that resets a variable to it's declared value.

The following is the usage:

var a = 100
a *= 3
assert a == 300
resetToDecl(a)
assert a == 100

To start off we know we need a single typed parameter so our macro header is macro resetToDecl(val: typed): untyped. typed is much like untyped in that it takes any code, but in this case we know it's semantically checked. So to ensure we are only dealing with a variable we must remove all other kinds filtering nnkSym. Using the nifty error tool to give a helpful message, which takes in a string and an optional NimNode as a second parameter to provide the line information for the error.

macro resetToDecl*(val: typed): untyped = 
  if val.kind == nnkSym:
   # We will implement code here later
  else:
   error("This macro only works with variables." val)

Now though we need to filter out all non-var symbols, since symbols can be any type, proc, variable, ... anything you can know by name. So we'll use the symKind of the symbol to filter out everything but var.

macro resetToDecl*(val: typed): untyped = 
  if val.kind == nnkSym and val.symKind == nskVar:
    ## We'll continue here in next block
  else:
    error("This macro only works on variables", val)

We have done it, we just have variables, but now the question is "How do we get the declaration statement?!", through the power of introspection magic! Nim's macros module exposes multiple other avenues of introspection other than symKind it also has getImpl, getTypeImpl, getType, and so much more. The procedure we will use is of course getImpl cause we want to get the implementation of the symbol(Yes having a symbol is required, batteries not included otherwise).

We can quickly just test to see what the AST is like simply with echo val.getImpl, what we see is a lovely declaration.

IdentDefs
  Sym "a"
  Empty
  IntLit 100

If you squint closely you can see this is quit a predictable situation, all we have to do is assign the val to the val.getImpl[^1] so let us do that!

macro resetToDecl*(val: typed): untyped = 
  if val.kind = nnkSym and val.symKind == nskVar:
    result = nnkAsgn.newTree(val, val.getImpl[^1])
  else:
    error("This macro only works with variables", val)

I already hear it "Hey is that it, that was easy, there has to be a catch?" and you're right, simply doing var a: int has caused use to get an error that says Error: illformed AST:. The reason for this error is that the macro does not check if the last entry in the symbol is nnkEmpty so all that needs to be done is to make it so if the last entry is empty we emit a default(typeof(val)).

Let us do that:

import std/macros

macro resetToDecl*(val: typed): untyped = 
   if val.kind == nnkSym and val.symKind == nskVar:
     let impl = val.getImpl
     result = nnkAsgn.newTree(val):
       if impl[^1].kind == nnkEmpty:
         newCall("default", newCall("typeOf", val))
       else:
         impl[^1]
   else:
     error("This macro only works on variables", val)

Now to test we've done it properly:

var a = 100
a *= 3
assert a == 300
resetToDecl(a)
assert a == 100


var b: int
b = 300
assert b == 300
b *= 3
assert b == 900
b.resetToDecl
assert b == 0

If you want to see examples I have a variety of packages with mixed complexity of macros:

Slicerator
Oopsie
Constructor
Nimscripter
Nettyrpc
Kashae