more nim for embedded software development

Writing libraries is fun, but at some point you've got to use them. I've been writing one for roughly two years!

Finally, I moved to writing a not-so-small project using said library (avr_io - avr_io@github), so it is with this newfound knowledge that I bring forth some thoughts about my freshly started nim-application-dev days.

This is a loose sequel of a previous article, nim for embedded software development.

Note: the following is mainly about bare-metal firmware for 8-bit microcontrollers, but it can be applied to other domains.

arc memory management

I have been experimenting a bit with arc and I like the "don't pay for what you don't use" kind of thing.

Essentially, if you're not using ref-types you don't really notice arc, and you can use string and seq for relatively low cost in binary size.

Still haven't experimented too much with size differences to have a precise measurement, but it's not too bad if you don't over-use allocations or some portions of the stdlib.

If that does become bad, I have the following incantation that sequeezes quite a bit of KB from your binary:

switch("gc", "none")
switch("opt", "size")
switch("define", "danger")

It also has been suggested to me to use --execptions:quirky, but in all honesty I don't use exceptions a lot.

You have to be careful and really know what you're doing tho, with these flags. And some portions of the language may not work, like..

object variants

One thing were arc (or a mm ≠ none) may be needed is if you're using object variants.

I was kind of surprised by this! Aren't they basically just safer tagged unions from C?

# An example of an object variant in use
type
  CmdKind* = enum
    SetBpm
    SetValue
    SetAmplitude
    SetEnvelope

  Command* = object
    case kind*: CmdKind
    of SetBpm:
      bpm*:       uint16
    of SetValue:
      value*:     NoteValue
    of SetAmplitude:
      amp_chan*:  uint8
      amplitude*: uint8
    of SetEnvelope:
      envelope*:  EnvelopeShape
      frequency*: uint16

The keyword here is safer; nim achieves this by:

• Forcing you to re-create the object instead of just switching the tag and possibly invalidate the underlying data.
• Raising an exception/panic whenever you access a field that is not tied to the active kind.

This second bit is quite nice, because it allows you to catch runtime errors with custom panicoverride handles; for example, I just set up a blinking LED for this kind of stuff, so I have a visual cue that something went wrong.

On the other side, the message passed to the panic handle is built with appendString, which implies at least using -mm:arc --define:useMalloc, so if you want to use object variants AND be safe about them, you cannot go with just --mm:none.

I got three issues with this:

• It would be nice to maybe have a compiler flag that enforces compile-time checking that you are accessing variants through case..of blocks, so to catch these accesses when building. You can actually do this with:

switch("warning", "ProveField:on")
switch("warningAsError", "ProveField:on")

But it's quite picky and it seems like it implies arc, as it requires newObj- to be defined.

• It also would be nice to be able to override the invalid access message creation mechanism, to avoid calling appendString.
• Currently (as I'm writing this post nim v2.2.6 just got released), panicoverride does not support the full extent of nim features, so doing anything more than blinking a LED is a bit cumbersome (I am working to try and put this into upstream).

Anyways, if you want to use object variants, and you are very sure you're great at not writing bad code, you can disable the checks (and the need for arc) with --fieldChecks:off.

Spoiler: I had at least 3 crashes that were caught by these safety features. Even if you are good, everyone does a bit of copy pasting, and mistakes can happen.

trying not to allocate

So, the age-old problem with embedded about not wanting to do dynamic allocations.

I got two nice approaches here:

var arrays

Passing a var array (or a wrapper to it) in nim is quite nice, since you have generics, can enforce """ref-like""" semantics for value types (using {.byref.} to enforce passing by pointer), and arrays do not decade to pointers, unlike in C. Also, I believe var parameters are always passed by pointer in the generated C code.

So you can just use this approach with stack-allocated or global buffers.

non owning views into arrays

Now I had this headache for a day or two, about why I was getting sequences being generated and allocated in my sequence-less code:

let chan = cast[Channel](cmd.scd_chan)
let mode = cast[ChannelMode](cmd.scd_chan)
channels.incl {mode}

ay.channel_on(channels)
ay.set_amplitude(chan, 15)

arpeggio_bufs[cmd.scd_chan].add_arpeggio(
  cmd.root, oct, cmd.buf[0..cmd.size-1], cmd.dir,
  reps
).or_else(failed)

Turns out that slicing an array and passing said slice through an openArray input parameter is not free in nim.

The way around it is using toOpenArray to generate a non-owning view through the array.

compile time tables

This is one of the small things that I love about nim, that almost anything you can use at runtime, you can also use at compile time.

Lots of times, to avoid many allocations or to pre-compute stuff, you want to create arrays or tables with metadata and such.

One pattern I learnt to use and love is the anonymus proc generating comptime arrays (notice that if the user adds or removes an enum entry from ErrorCode, this does not require refactoring):

const
  ErrorStrings = (proc(): array[ErrorCode.high.ord + 1, string] =
    for i in 0..ErrorCode.high.ord:
      result[i] = symbolName(cast[ErrorCode](i))
  )()

Clear, succint, effective.

Metaprogramming

The current project I am working on is made of 3 subprojects:

• An embedded driver
• An embedded application
• A desktop cli tool to interact with the ones above

Before that, I have worked for ~2 years on avr_io, which is mainly a series of wrappers, and easy to use set of API to interact with AVR microcontrollers.

There, I used a lot of macros and templates to perform AST manipulation to e.g. generate ISR handlers easily, and to try to provide 0-overhead abstractions.

Here in these projects?

Macros are incredible in nim, but they are hard to maintain and best kept in libraries. Nim is so expressive that you will rarely need them in application code anyways.

Building

As I said, my current project has 3 projects in 1, with one of these being a CLI app, which means I have different config.nims and targets and such.

nimble, atlas and make

I found nimble to be very cumbersome to use in these scenarios, as config.nims and nims file in general do not seem to work properly with nested nimble projects and custom tasks.

So I very sadly went back to good old make and a single nimble file, which works well enough for me (since I mainly work on linux/macosx). I have a single nimble file in my root directory, and different targets in the Makefile.

Also, I started using atlas a bit and it was quite a positive experience.

Atlas is nim's brand new "package cloner". I would describe it as a really nice way to vendor dependencies and nim compiler versions, it is sort of like if go mod vendor from go, and virtualenv from python had a child.

Not only that, but atlas allow you to link multiple projects together to unlock local-forst workspaces. I am experimenting with it and liking it quite a bit. It works much better than nimble for my personal opinion and ergonomical preferences.