The Happening of A Template Engine for Nim

Zetian Lin — Mon, 06 Jan 2025 20:52:14 +0000

Nothing about this is deliberately designed; everything about this happened by accident.

So I have a habit of writing the back-end part for web apps that I choose to build in my spare time with no dependencies other than the HTTP server library provided by the language itself. It's good fun (if you're a masochist), you get to really feel what kind of things you really want when you're forced to do every little thing your own.

Since I used std/htmlgen in these projects I grew really tired of it and naturally I wished to have some kind of template engine so that I could at least write HTML instead of Nim. The very first version of it was bastardly simple. It reads and parses a file an replaces all the "tags" (in this case, anything surrounded with two brackets {{ ... }} with the corresponding value passed in with a map:

import std/macros

proc parseTemplate(x: string): seq[(bool, string)] {.compileTime.} =
  var res: seq[(bool, string)] = @[]
  var isTag = false
  var currentPiece = ""
  var currentTag = ""
  var i = 0
  let lenx = x.len
  while i < lenx:
    if x[i] == '{' and i+1 < lenx and x[i+1] == '{':
      if isTag:
        currentTag.add('{')
        currentTag.add('{')
        i += 2
      else:
        res.add((false, currentPiece))
        currentPiece = ""
        isTag = true
        i += 2
    elif x[i] == '}' and i+1 < lenx and x[i+1] == '}':
      if not isTag:
        currentPiece.add('}')
        currentPiece.add('}')
        i += 2
      else:
        res.add((true, currentTag))
        currentTag = ""
        isTag = false
        i += 2
    else:
      if isTag:
        currentTag.add(x[i])
        i += 1
      else:
        currentPiece.add(x[i])
        i += 1
  if isTag and currentTag.len > 0:
    currentPiece &= "{{" & currentTag
  if currentPiece.len > 0:
    res.add((false, currentPiece))
  return res

# "macro" because I need to use `staticRead`. 
macro useTemplate*(name: untyped, filename: static[string]): untyped =
  result = nnkStmtList.newTree()
  let s = staticRead(filename)
  let v = s.parseTemplate
  result.add quote do:
    proc `name`(prop: StringTableRef): string =
      let p = `v`
      var res: seq[string] = @[]
      for k in p:
        if k[0]:
          res.add(if prop.hasKey(k[1]): prop[k[1]] else: "")
        else:
          res.add(k[1])
      return res.join("")

Soon there were problems:

How do you if?
Also, how do you for?
How do you send in anything other than a plain StringTableRef, e.g. something like {"name": "John Doe", "address": {"country": "Ireland", "address": "blahblahblah", "postcode": "ABC DEFG"}}? (This problem, due to how Nim has UFCS and operator overloading, turns out to be fatal.)

A normal solution for the first two questions is to have "special tags" like this:

{{if someCondition}}
  ...
{{elif someOtherCondition}}
  ...
{{else}}
  ...
{{/if}}

{{for someVar in someCollection}}
  ...
{{/for}}

To parse the tags themselves is easy even if you don't rely on recursion:

type
  KT = enum
    KIF
    KELIF
    KFOR
    KELSE
  K = ref object
    kOutside: seq[TemplatePiece]
    case kind: KT
    of KIF:
      kIfCond: string
    of KELIF:
      kElifCond: string
      kElifCompletedClause: seq[(string, seq[TemplatePiece])]
    of KELSE:
      kElseCompletedClause: seq[(string, seq[TemplatePiece])]
    of KFOR:
      kForBoundVar: string
      kForExpr: string

# and then at the parser you'll have a while loop and within
# the loop you'll have something like this:
var kstk: seq[K] = @[] 
var res: seq[TemplatePiece] = @[] 
...
if isEndIf:
  if kstk.len <= 0: reportError
  if kstk[^1].kind != KIF and #[KELIF, KELSE]#: reportError 
  let k = kstk.pop
  let newPiece = k.makeIntoIfPiece(res)
  res = k.kOutside
  res.add(newPiece)
elif isIf:  # we save the "outside" part and the condition
  ...
  kstk.add(K(kOutside: res, kind: KIF, kIfCond: cond))
  res = @[] 
elif isElif:
  # whatever is in `res` at this point must belong to an IF
  # or an ELIF clause, which should happen on `k.kIfCond`
  # or `k.kElifCond`.
  if kstk.len <= 0: reportError
  if kstk[^1].kind != KIF and #[KELIF]#: reportError
  ...
  let k = kstk.pop()
  kstk.add(K(kOutside: k.kOutside, kind: KELIF,
             kElifCond: cond,
             kElifCompletedClause:
               case k.kind: 
                 of KIF: @[(k.kIfCond, res)] 
                 of KELIF: k.kElifCompletedClause & @[(k.kElifCond, res)]))
  res = @[] 
elif isElse:
  if kstk.len <= 0: reportError
  if kstk[^1].kind != KIF and #[KELIF]#: reportError
  ...
  let k = kstk.pop()
  kstk.add(K(kOutside: k.kOutside, kind: KELSE,
             kElseCompletedClause:
               case k.kind: 
                 of KIF: @[(k.kIfCond, res)] 
                 of KELIF: k.kElifCompletedClause & @[(k.kElifCond, res)]))              
  res = @[]

(for can be handled similarly.)

The parsing of conditions is a bigger problem. Since this is a template engine meant for Nim, one would understandably wish to support all the constructs available for Nim, but that's a huge undertaking; also, how do you plan to interpret it, since Nim is a language that supports a lot of fancy stuffs? After discovering parseExpr in the same std/macros module that one needs to import to do macros, I've had an epiphany.

Everything is to become macros.

Everything.

The whole template would be expanded into a series of function calls that adds strings to a seq[string], ifs and fors in the template would be expanded into actual if and for statements and conditions would be directly parsed (by Nim instead of me) and expanded into actual expressions.

At the end of the day I had something like this:

proc renderTemplateToAST(s: seq[TemplatePiece], resultVar: NimNode): NimNode =
  result = nnkStmtList.newTree()
  for k in s:
    case k.pType:
      of STRING:
        let key = k.strVal
        result.add quote do:
          `resultVar`.add(`key`)
      of EXPR:
        let v: NimNode = k.exVal.parseExpr
        result.add quote do:
          `resultVar`.add(`v`)
      of FOR:
        let v: NimNode = newIdentNode(k.forVar)
        let e: NimNode = k.forExpr.parseExpr
        let b: NimNode = k.forBody.renderTemplateToAST(resultVar)
        result.add quote do:
          for `v` in `e`:
            `b`
      of IF:
        if k.ifClause.len <= 0:
          raise newException(ValueError, "Cannot have zero if-branch.")
        var i = k.ifClause.len-1
        var lastCond = k.ifClause[i][0].parseExpr
        var lastIfBody = k.ifClause[i][1].renderTemplateToAST(resultVar)
        var lastRes =
          if k != nil and k.elseClause.len > 0:
            let elseBody = k.elseClause.renderTemplateToAST(resultVar)
            quote do:
              if `lastCond`:
                `lastIfBody`
              else:
                `elseBody`
          else:
            quote do:
              if `lastCond`:
                `lastIfBody`
        i -= 1
        while i >= 0:
          var branchCond= k.ifClause[i][0].parseExpr
          var branchBody = k.ifClause[i][1].renderTemplateToAST(resultVar)
          lastRes =
            quote do:
              if `branchCond`:
                `branchBody`
              else:
                `lastRes`
          i -= 1
        result.add lastRes
  return result

macro expandTemplate*(resultVarName: untyped, filename: static[string]): untyped =
  result = nnkStmtList.newTree()
  let resolveBase = (resultVarName.lineInfoObj.filename.Path).parentDir / filename.Path
  var trail: seq[string] = @[]
  # `resolveTemplate` is another function I've had to populate
  # all the `{{include ...}}` tags because I choose to support
  # that as well
  let v = resolveBase.string.resolveTemplate(trail)
  let seqres = newIdentNode("`")
  let procBody = v.renderTemplateToAST(seqres)
  result.add quote do:
    var `resultVarName`: string
    block:
      var `seqres`: seq[string] = @[]
      `procBody`
      `resultVarName` = `seqres`.join("")

which would expand a template like this:

<h1>{{siteName}}</h1>
<p>Hello!
  {{if visitorName == "viktor"}}
    <span style="color: red">Viktor!</span>
  {{elif visitorName != ""}}
    <span style="color: blue">{{visitorName}}!</span>
  {{else}}
    visitor!
  {{/if}}
</p>

into something like this:

var res: string
block:
  var seqres123: seq[string] = @[] 
  seqres123.add("<h1>")
  seqres123.add(siteName)
  seqres123.add("</h1>\n</p>Hello!\n  ")
  if visitorName == "viktor":
    seqres123.add("\n    <span style=\"color: red\">Viktor!</span>\n  ")
  elif visitorName != "":
    seqres123.add("\n    <span style=\"color: blue\">")
    seqres123.add(visitorName)
    seqres123.add("</span>\n  ")
  else:
    seqres123.add("\n    visitor!\n    ")
  seqres123.add("\n</p>")
  res = seqres123.join("")

And that's pretty much it.

What did I lose by doing it this way

The ability to expand and populate a template in runtime! But seriously, Nim don't really have an eval like JavaScript (at least there aren't any that I know of) which will make it much harder and painful to make. Compile-time is good enough for me for now.

DEV Community: Zetian Lin

The Happening of A Template Engine for Nim

What did I lose by doing it this way