DEV Community: abathargh

more nim for embedded software development

abathargh — Tue, 25 Nov 2025 20:43:24 +0000

Writing libraries is fun, but at some point you've got to use them. I've been writing one for roughly two years!

Finally, I moved to writing a not-so-small project using said library (avr_io - avr_io@github), so it is with this newfound knowledge that I bring forth some thoughts about my freshly started nim-application-dev days.

This is a loose sequel of a previous article, nim for embedded software development.

Note: the following is mainly about bare-metal firmware for 8-bit microcontrollers, but it can be applied to other domains.

arc memory management

I have been experimenting a bit with arc and I like the "don't pay for what you don't use" kind of thing.

Essentially, if you're not using ref-types you don't really notice arc, and you can use string and seq for relatively low cost in binary size.

Still haven't experimented too much with size differences to have a precise measurement, but it's not too bad if you don't over-use allocations or some portions of the stdlib.

If that does become bad, I have the following incantation that sequeezes quite a bit of KB from your binary:

switch("gc", "none")
switch("opt", "size")
switch("define", "danger")

It also has been suggested to me to use --execptions:quirky, but in all honesty I don't use exceptions a lot.

You have to be careful and really know what you're doing tho, with these flags. And some portions of the language may not work, like..

object variants

One thing were arc (or a mm ≠ none) may be needed is if you're using object variants.

I was kind of surprised by this! Aren't they basically just safer tagged unions from C?

# An example of an object variant in use
type
  CmdKind* = enum
    SetBpm
    SetValue
    SetAmplitude
    SetEnvelope

  Command* = object
    case kind*: CmdKind
    of SetBpm:
      bpm*:       uint16
    of SetValue:
      value*:     NoteValue
    of SetAmplitude:
      amp_chan*:  uint8
      amplitude*: uint8
    of SetEnvelope:
      envelope*:  EnvelopeShape
      frequency*: uint16

The keyword here is safer; nim achieves this by:

Forcing you to re-create the object instead of just switching the tag and possibly invalidate the underlying data.
Raising an exception/panic whenever you access a field that is not tied to the active kind.

This second bit is quite nice, because it allows you to catch runtime errors with custom panicoverride handles; for example, I just set up a blinking LED for this kind of stuff, so I have a visual cue that something went wrong.

On the other side, the message passed to the panic handle is built with appendString, which implies at least using -mm:arc --define:useMalloc, so if you want to use object variants AND be safe about them, you cannot go with just --mm:none.

I got three issues with this:

It would be nice to maybe have a compiler flag that enforces compile-time checking that you are accessing variants through case..of blocks, so to catch these accesses when building. You can actually do this with:

switch("warning", "ProveField:on")
switch("warningAsError", "ProveField:on")

But it's quite picky and it seems like it implies arc, as it requires newObj- to be defined.

It also would be nice to be able to override the invalid access message creation mechanism, to avoid calling appendString.
Currently (as I'm writing this post nim v2.2.6 just got released), panicoverride does not support the full extent of nim features, so doing anything more than blinking a LED is a bit cumbersome (I am working to try and put this into upstream).

Anyways, if you want to use object variants, and you are very sure you're great at not writing bad code, you can disable the checks (and the need for arc) with --fieldChecks:off.

Spoiler: I had at least 3 crashes that were caught by these safety features. Even if you are good, everyone does a bit of copy pasting, and mistakes can happen.

trying not to allocate

So, the age-old problem with embedded about not wanting to do dynamic allocations.

I got two nice approaches here:

`var array`s

Passing a var array (or a wrapper to it) in nim is quite nice, since you have generics, can enforce """ref-like""" semantics for value types (using {.byref.} to enforce passing by pointer), and arrays do not decade to pointers, unlike in C. Also, I believe var parameters are always passed by pointer in the generated C code.

So you can just use this approach with stack-allocated or global buffers.

non owning views into arrays

Now I had this headache for a day or two, about why I was getting sequences being generated and allocated in my sequence-less code:

let chan = cast[Channel](cmd.scd_chan)
let mode = cast[ChannelMode](cmd.scd_chan)
channels.incl {mode}

ay.channel_on(channels)
ay.set_amplitude(chan, 15)

arpeggio_bufs[cmd.scd_chan].add_arpeggio(
  cmd.root, oct, cmd.buf[0..cmd.size-1], cmd.dir,
  reps
).or_else(failed)

Turns out that slicing an array and passing said slice through an openArray input parameter is not free in nim.

The way around it is using toOpenArray to generate a non-owning view through the array.

compile time tables

This is one of the small things that I love about nim, that almost anything you can use at runtime, you can also use at compile time.

Lots of times, to avoid many allocations or to pre-compute stuff, you want to create arrays or tables with metadata and such.

One pattern I learnt to use and love is the anonymus proc generating comptime arrays (notice that if the user adds or removes an enum entry from ErrorCode, this does not require refactoring):

const
  ErrorStrings = (proc(): array[ErrorCode.high.ord + 1, string] =
    for i in 0..ErrorCode.high.ord:
      result[i] = symbolName(cast[ErrorCode](i))
  )()

Clear, succint, effective.

Metaprogramming

The current project I am working on is made of 3 subprojects:

An embedded driver
An embedded application
A desktop cli tool to interact with the ones above

Before that, I have worked for ~2 years on avr_io, which is mainly a series of wrappers, and easy to use set of API to interact with AVR microcontrollers.

There, I used a lot of macros and templates to perform AST manipulation to e.g. generate ISR handlers easily, and to try to provide 0-overhead abstractions.

Here in these projects?

Macros are incredible in nim, but they are hard to maintain and best kept in libraries. Nim is so expressive that you will rarely need them in application code anyways.

Building

As I said, my current project has 3 projects in 1, with one of these being a CLI app, which means I have different config.nims and targets and such.

nimble, atlas and make

I found nimble to be very cumbersome to use in these scenarios, as config.nims and nims file in general do not seem to work properly with nested nimble projects and custom tasks.

So I very sadly went back to good old make and a single nimble file, which works well enough for me (since I mainly work on linux/macosx). I have a single nimble file in my root directory, and different targets in the Makefile.

Also, I started using atlas a bit and it was quite a positive experience.

Atlas is nim's brand new "package cloner". I would describe it as a really nice way to vendor dependencies and nim compiler versions, it is sort of like if go mod vendor from go, and virtualenv from python had a child.

Not only that, but atlas allow you to link multiple projects together to unlock local-first workspaces. I am experimenting with it and liking it quite a bit. It works much better than nimble for my personal opinion and ergonomical preferences.

nim, LLMs and me: finding a balance between pragmatism and the joy of programming

abathargh — Mon, 13 Oct 2025 06:49:17 +0000

I recently got one of the new MacBook Air laptops, with the M4 ARM CPU, mainly for music production.

Once I had my hands on it though, I could not resist and I started installing a bit of my usual programming environment on it; one thing led to another, and I found myself enjoying the whole setup quite a bit.

So as usual when finding the right ergonomics, passion gets channeled through the ease of use of the tools you have in your hands, and creativity flows.

I had been working on and off for some months on the tools I use to program avr chips in nim, and I was hoping to add some code to discover serial devices automatically when programming arduino boards.

Nothing I found online was quite where I wanted, as I wished for something akin to pyserial list_ports.comports, which just tells you - in the usual simple and to the point python fashion - what are the serial devices connected to your computer.

So I wrote my own:

proc enumerate_serial_devices(): seq[string] =
  for kind, path in walkDir("/dev/"):
    if kind == pc_file and path.extract_filename.startswith("tty"):
      result.add(path)


# this extracts the vid/pid from /sys/class/tty/.../device/uevent
proc get_vid_pid(dev: string): tuple[vid: uint16, pid: uint16] =
  let dev_name = dev.split("/")[^1]
  let tty_path = fmt"/sys/class/tty/{dev_name}/device"
  let evt_path = fmt"/sys/class/tty/{dev_name}/device/uevent"

  if dir_exists(tty_path) and file_exists(evt_path):
    for line in evt_path.lines:
      if "PRODUCT=" in line:
        let meta = line.split("PRODUCT=")[^1]
        let ids  = meta.split("/")
        return (ids[0].parse_hex_int.uint16, ids[1].parse_hex_nt.uint16)
  (0'u16, 0'u16)

You can find the complete code @github.com/Abathargh/avrman.

Everything was fine, as usual nim is very expressive and manages to embody python's simplicity.

But now to the issue: /sys is a linux feature, and other systems (even UNIX-like ones, like MacOSX) do not provide it. Now, I thought:

How difficult could it be to do the same for Mac?

Turns out that it is not maybe hard, but much less straightforward and immediate.

I studied the issue for a bit, but the combination of this being a "side-problem" and the fact that no one was paying me to do something not fun, led me to something I had not done really seriously up until now: using one of the LLMs to completely generate the code for me.

Now I've been using these tools for a while, like most of us, and I mainly relegated them to a sort of glorified "google when it still was good, but on steroids", but now I wanted to have full code generation, and boy was I surprised.

I fed this problem into Claude (my go to one, I just find it generates better code for my taste and standards, with respect to its competitors), and I was really surprised by the fact that the code actually worked (with minor tweaks) for such a niche use case and programming language, on a specific platform.

These tools have been getting quite better even at less known languages, and I have been surprised recently, because they are just starting to get good for me.

Now, to the real issue at heart.. I love programming, I really do, it's not only a job for me, but some kind of artisanal thing I do with some of my time on this beautiful planet, so let me philosophize and maybe romanticize a bit: I am afraid that we may lose this kind of feeling that the joy of programming gives us by only feeding prompts to a lifeless program.

Not only that, but I fear that learning to program right now will have the double-edged sword that I observe on myself after this experience: I now have a working driver for enumerating serial devices on MacOSX and I cannot explain to you how it works, because I did not think it out myself. I know I can learn it, but why would I?

I have been a kid, a student, and a learner of (sometimes) boring things, so I know that people will take shortcuts for this kind of stuff, and not look back at how it actually is done. How do we solve this issue? I think the biggest problem of the coming years in this landscape will be an educational one.

I have no answer for now, but I for sure have hope.

optimizing c structs layouts

abathargh — Sat, 07 Dec 2024 14:25:38 +0000

Let's consider a simple c struct:

struct foo {
    const char * str;
    unsigned char flag;
    uint64_t len;
};

Suppose that this code is being run as a part of a program executing on a 64-bit machine: what would you expect the result of sizeof(struct foo) to be?

Most people that never had to mess with struct sizes and optimizations would guess that it should be 17...

... but it's 24! Why is that?

The reason for this behavior is that compilers optimize struct layouts for speed, and it's the modern norm that aligned memory accesses are the fastest way to access data.

This means that, depending on the type of field and cpu, your data will have some alignment and will be positioned such that that alignment is respected (or, such that field-address % field-alignment == 0).

size, alignment, padding

In the case of the previous example, pointers and 64-bit fields are 8B aligned on 64-bit machines, and this means that, in order to force a layout where everything in the struct is aligned, the compiler will generate some padding between the flag and len fields:

Now let's consider another example, where a struct like this is defined, on the same machine as before:

struct bar {
    const char * str;
    short s1;
    int i1;
    short s2;
    int i2;
};

How do we go about computing its size?

There are three rules:

Struct fields want to be aligned with their own natural alignment.
The overall struct alignment is equal to the alignment of its widest field
If you had to put two structs of the same type side by side, the second one should be aligned to its alignment -- this means that structs must have trailing padding up to their alignment

Quick recap on basic types alignment and size for 64-bit machines:

type	size	alignment
char	1	1
short	2	2
int	4	4
long	8	8
float	4	4
double	8	8
pointers	8	8

Also remember that:

Arrays have the alignment of their value type, and size that is sizeof(type) * elements number.
Unions have the alignment and size of their widest member.

And you can use the super useful sizeof and _Alignof operators to get this information for your custom types. Note that _Alignof is available from C11 onward, and is called alignof starting with C23. It was always alignof from what I understand in C++, since C++11.

For more information, the bible on this topic is The Lost Art of Structure Packing, from which I learned almost everything I know about this, together with a lot of practice and hands-on experience.

optimizing your structs: stropt

This topic here is something that comes up rather frequently at work, where saving bytes here and there is super important when sending huge loads of data continuously in queues and whatnot.

In order to make my life easier, I wrote a tool that produces some statistics on a type you pass as input, with reference to a source file or code snippet, it's called stropt (struct optimizer).

Abathargh/stropt on GitHub

build & install

If you have a local go installation you can go right ahead and build or install the application directly:

git clone https://github.com/Abathargh/stropt
go build

// or, if you want to install this directly
go install github.com/Abathargh/stropt

Already compiled binaries for a list of os/archs combinations are also provided in the github release page:

stropt binaries

using the tool

You can either use stropt by passing the source to analyze as a string:

stropt "struct meta" "struct meta { int a; double d; float f[100];};"

Or you can pass a file in which the definition is contained:

// test.c
struct test {
  const char * str;
  unsigned char flag;
  const int * const iptr;
  char c;
};

stropt -file test.c "struct test"

The tool can also provide a possible optimization for your types by using the -optimize flag, and it's aware of fields that are structs themselves:

Note that the verbose flag is used to show inner structs (and unions) along with their fields alignment and size.

what's next

This tool was written in go using the great modernc.org/cc C compiler front-end for parsing C code, and lipgloss from charmbracelet for the UI.

I'm writing this for myself but I'm happy to share it publicly; I'd like to make this a webapp for easier use directly in a browser, so probably that's the next thing I'm going to be working on!

harlock v0.5.1 released

abathargh — Sat, 13 Jul 2024 13:24:30 +0000

It is with immense pleasure that I announce that version 0.5.1 off the harlock scripting language is out!

Here is the detailed release log, with a list the artifacts to install the language on debian-like systems, or directly a binary for the supported architectures.

Release note + artifacts @github/Abathargh/harlock

Build from source

Notice that you can always compile and install harlock by executing:

go install github.com/Abathargh/harlock/cmd/harlock

or:

git clone https://github.com/Abathargh/harlock
make install

Release details

This v0.5.1 release is a bugfix one, that solidifies harlock usage within build pipelines.

The main issues that were tackled are:

Correctly handling runtime and evaluation errors being raised at the top-level scope to trigger a non-zero error code. This led to silent errors being passed inside of pipelines using harlock.
Drop support for go 1.15+ unsupported targets.
Adding previously missing .exe suffix for windows executable names when cross-compiling for windows on non-windows
Minor fixes to .gitignore and Makefile.

Last couple of releases included a new error system which was thoroughly tested and fixing it is the main reason for v0.5.1.

Usage and new developments

I've been using harlock a lot to test the avr_io nim library that I work on, alongside personal projects where I need firmware to be updated over the wire/air with huge success.

A nice working project using the language can be found in the bootloader example for avr_io, where it is used to showcase the library capabilities when writing bootloaders for embedded applications.

I wrote an in-depth article on how to use harlock for these kind of scenarios on antima at the following link.

nim for embedded software development

abathargh — Thu, 28 Mar 2024 07:29:20 +0000

While we in embedded-land are mostly working with either C or C++ on our professional ventures, often with proprietary tooling and whatnot, I always find some much appreciated respite in tinkering with other alternatives when wasting time on hobbies and side-projects.

In the last year or so, I delved into exploring other venues to solve my embedded headaches and landed onto the nim programming language.

nim compiles to c (and c++, objective-c, js)

This means that any target with an existing c compiler is automatically supported by nim.

Not only that, but calling c code (and c++, objective-c, etc.) is really simple to do and, from what I see, 0-overhead.

You can also easily check the generated C sources, which are human readable, even if with a lot of noise. This can be achieved by specifying a --nimcache directory when compiling.

writing, building, shipping

Like all modern stuff, we have a sane module system (bye bye text-based #includes), an ergonomic compiler and a nice, basic package manager.

The compiler lets you use a script-like subset of the language as a format for configuration files. These may contain compiler switches and system-specific flags. It integrates seamlessly with the nimble package manager, which lets you write tasks that can be executed both after building or as sorts of standalone targets.

writing a simple hello world for AVR is trivial (ish)

This is a valid program that can be run on a x64 intel CPU and, without any changes, on an 8-bit atmega microcontroller:

proc main = 
  while true:
    discard

main()

To compile on AVR, you just have to provide some more configuration on how to handle critical errors without the os covering our backs:

# panicoverride.nim

proc exit(code: int) {.importc, header: "<stdlib.h>", cdecl.}

{.push stack_trace: off, profiler:off.}

proc rawoutput(s: string) = discard

proc panic(s: string) =
  rawoutput(s)
  while true:
    discard
  exit(1)

{.pop.}

And a couple of compiler flags:

# config.nims

switch("os", "standalone")
switch("cpu", "avr")
switch("gc", "none")
switch("stackTrace", "off")
switch("lineTrace", "off")
switch("passC", "-mmcu=atmega328p")
switch("passL", "-mmcu=atmega328p")
switch("nimcache", ".nimcache")

switch("avr.standalone.gcc.options.linker", "-static")
switch("avr.standalone.gcc.exe", "avr-gcc")
switch("avr.standalone.gcc.linkerexe", "avr-gcc")

when defined(windows):
  switch("gcc.options.always", "-w -fmax-errors=3")

Notice that these is where you specify which c compiler to use, to then actually generate the final binaries.

The compiler is identified by the cpu.os.compiler name (avr.standalone.gcc):

The compiler executable is specified through the exe property.
The linker executable is specified through the linkerexe property.

Let's dump these code snippets into the following files:

Run the compiler...

nim c avr_hw.nim

...and check its output

avr_hw:     file format elf32-avr


Disassembly of section .text:

00000000 <.text>:
   0:   0c 94 38 00     jmp 0x70
   4:   0c 94 4a 00     jmp 0x94
   8:   0c 94 4a 00     jmp 0x94
  ...
  70:   11 24           eor r1, r1
  72:   1f be           out 0x3f, r1
  74:   cf ef           ldi r28, 0xFF
  76:   d0 e1           ldi r29, 0x10
  78:   de bf           out 0x3e, r29
  7a:   cd bf           out 0x3d, r28
  7c:   21 e0           ldi r18, 0x01
  7e:   a0 e0           ldi r26, 0x00
  80:   b1 e0           ldi r27, 0x01
  82:   01 c0           rjmp    .+2 
  84:   1d 92           st  X+, r1
  86:   ae 30           cpi r26, 0x0E
  88:   b2 07           cpc r27, r18
  8a:   e1 f7           brne    .-8
  8c:   0e 94 52 00     call    0xa4
  90:   0c 94 53 00     jmp 0xa6
  94:   0c 94 00 00     jmp 0   
  98:   ff cf           rjmp    .-2     
  9a:   08 95           ret
  9c:   08 95           ret
  9e:   ff cf           rjmp    .-2
  a0:   ff cf           rjmp    .-2
  a2:   ff cf           rjmp    .-2
  a4:   ff cf           rjmp    .-2
  a6:   f8 94           cli
  a8:   ff cf           rjmp    .-2

Worked like a charm!

foreign function interface

Want to use _delay_ms from util/delay.h?

proc delay_ms(us: uint16) {.importc: "_delay_ms", header: "util/delay.h".}

proc main = 
  while true:
    delay_ms(1000)

main()

As shown, it's really easy to integrate existing c code in your programs, you don't have to rewrite everything in nim. Note that there are tools to also translate c code to nim (c2nim, futhark).

metaprogramming

Metaprogramming is nim killer-feature in my opinion.

nim has:

Compile-time functions, which get executed in a VM embedded within the compiler, and supports a subset of the language to be evaluated at compile time.

const data = staticRead("my_file") # the file gets read at compile time!

Generics and concepts, which from what I observed, are completely 0 cost at runtime, and are really only present in nim code, not in the c-generated one. Using them in combination with typeclasses enables quite powerful patterns.

type MappedIoRegister*[T: uint8|uint16] = distinct uint16 ## \
  ## A register that can either contain a byte-sized or word-sized datum.

template ioPtr[T](a: MappedIoRegister[T]): ptr T = 
  cast[ptr T](a)

Templates, which are hygienic (with scoped symbols) c macros, essentially a replace-mechanism that allows you to have "true inlining". Combining this with operator overloading is really nice.

import volatile

template `[]`*[T](p: MappedIoRegister[T]): T =
  volatile.volatileLoad(ioPtr[T](p))

template `[]=`*[T](p: MappedIoRegister[T]; v: T) =
  volatile.volatileStore(ioPtr[T](p), v)

Macros, which are special functions taking in Abstract Syntax Tree (AST) representations of nim code and spewing out ASTs that are transformations of its inputs. This allows for some pretty crazy stuff.

# VectorInterrupt enum definition omitted..
template vectorDecl(n: int): string =
  "$1  __vector_" & $n & 
  """
    $3 __attribute__((__signal__,__used__,__externally_visible__)); 
    $1 __vector_""" & $n & "$3"


macro isr*(v: static[VectorInterrupt], p: untyped): untyped =
  ## Turns the passed procedure `p` into an interrupt 
  ## service routine.
  var pnode = p
  if p.kind == nnkStmtList:
    pnode = p[0]

  expectKind(pnode, nnkProcDef)

  addPragma(pnode, newIdentNode("exportc"))
  addPragma(pnode, 
    newNimNode(nnkExprColonExpr).add(
      newIdentNode("codegenDecl"), 
      newLit(vectorDecl(ord(v)))
    )
  )
  pnode

# Now we can map functions to an interrupt

proc timer0_compa_isr() {.isr(Timer0CompAVect).} =
  # do stuff when the timer0 compare A interrupt gets triggered
  discard

All code snippets are taken from the avr_io library, a small project that I maintain.

Note that we can use exportc to generate code that will interact with c, and codegenDecl to manipulate the c-generated code: this is incredibly powerful, especially for writing libraries.

As a rule of thumb, use these features in this order and go to the next one only if needed: non-meta stuff -> generics -> templates -> macros.

memory management

Memory management is highly configurable in nim.

Starting from v2 onward, the memory management policy used by default is based on a reference counting approach, that also handles cycles (-mm:orc).

You can also choose to use a more easy-to-reason strategy, which also uses reference counting but without support for cycles (-mm:arc). Notice that both are deterministic and not stop-the-world.

By experimenting, I noticed that binary size can become a bit larger with orc/arc, so if it is really a problem, you can always opt in not managing your memory at all (-mm:none).

Why does that matter? Because nim has managed types, but I did not experiment with them enough to have formed an opinion on how good or not they are in bare-metal situations!