How To Use Zig for Elixir NIFs

Elixir excels at building scalable and maintainable applications. However, sometimes Elixir is not the best language to tackle specific tasks, and it can fall short in some areas, like direct system interaction.

Fortunately, Elixir offers NIFs (Native Implemented Functions): a path for integrating with other languages to improve these gaps. NIFs in Elixir act as an interface to call functions written in native languages like C or Rust, enabling developers to optimize performance-critical sections of their code.

NIFs can be written in many languages like Rust, Python, and Zig. For this tutorial, we will use Zig, due to its simplicity, speed, and safety.

Let's get going!

What We'll Cover

We'll cover the following topics in this tutorial:

• Setting up Zig for Elixir NIF development.
• Understanding NIFs in Elixir.
• Writing a simple NIF in Zig.
• Integrating the NIF with an Elixir application.

What Is Zig?

Zig is a general-purpose programming language designed for robustness, optimality, and maintainability.

It is known to provide excellent debugging and error-handling capabilities, making it suitable for various applications (including systems programming, embedded systems, and performance-critical applications).

Setting up Zig for Elixir Development

Let's now look at how we can set up Zig for Elixir NIF development.

Installing Zig

To get started with Zig, download the latest version of the Zig compiler from the official Zig website. Follow the installation instructions for your specific operating system.

Configuring Zig for Elixir NIF Development

To configure Zig for Elixir NIF development, you'll need to include the necessary libraries and dependencies.

Elixir NIFs rely on the Erlang NIF API provided by the erl_nif.h header file. Ensure you have the Erlang development package installed, and include the appropriate path to erl_nif.h in your build script or Zig build file.

Just add the following line to your src/main.zig file:

const erl = @cImport({
    @cInclude("erl_nif.h");
});

Importing this library will allow us to use the Erlang NIF API in our Zig code. It contains the native functions that allow our Zig code to interact with the Erlang VM. Later in this tutorial, we will see how to use this library to create our NIF.

Verifying the Setup

To verify that Zig is correctly set up for Elixir NIF development, we will create a simple "Hello, World!" project in Zig.

Go ahead and run the following commands:

mkdir hello-world
cd hello-world
zig init-exe

If zig has been installed correctly, you should see the following output:

info: Created build.zig
info: Created src/main.zig
info: Next, try `zig build --help` or `zig build run`

Next, we'll run zig build run on our terminal to ensure we can compile and execute a Zig program. If successful, you should see the following output:

All your codebase are belong to us.
Run `zig build test` to run the tests.

Now that we are able to compile and run Zig code, we can move forward.

Understanding NIFs in Elixir

Why Use NIFs in Elixir?

NIFs can be called from Elixir code, providing a performance boost for computationally intensive tasks. While Elixir excels at concurrency and fault tolerance, there might be better choices for performance-sensitive tasks.

By writing NIFs in a lower-level language like Zig, you can leverage the language's speed and efficiency without sacrificing Elixir's maintainability and concurrency features.

Advantages of Using Zig for NIF Development

Zig offers several advantages for NIF development:

• Performance: Zig compiles to efficient native code, providing significant performance improvements over interpreted languages like Elixir.
• Safety: Zig has strong static typing, compile-time checks, and a focus on error handling, reducing the likelihood of runtime errors.
• Simplicity: Zig's straightforward syntax and semantics make it easy to learn and use, especially for developers familiar with C or C++. Zig's standard library is also very comprehensive, providing a wide range of functionality for common tasks.

Writing our First NIF

Let's start by setting up our Elixir project. Run the following command:

mix new example_nif
cd example_nif

This will create a basic Elixir project called ExampleNif. Next, let's initialize the Zig project inside our Elixir project with the following command:

zig init-lib

This will initialize a library inside our project that will create the following files:

• build.zig - which contains the code for compiling and building our zig library.
• src/main.zig - this is the main code of our library and will contain the logic that we will implement.

Writing a Simple NIF in Zig

Our first step when working with Zig is to update the build.zig file with the following code:

const std = @import("std");
const Pkg = std.build.Pkg;

pub fn build(b: *std.build.Builder) void {
    const mode = b.standardReleaseOptions();

    const nif_step = b.step("example_lib", "Compiles erlang library");
    const example_lib = b.addSharedLibrary("example_nif", "./src/main.zig", .unversioned);
    example_lib.setBuildMode(mode);
    example_lib.setOutputDir("build");
    example_lib.addIncludePath("/usr/lib/erlang/usr/include/");

    example_lib.install();
    example_lib.linkLibC();
    nif_step.dependOn(&example_lib.step);
}

This will add the Erlang header path to the build environment and also link with the C library.

Next, we will update the src/main.zig code. For this example, we'll create a simple NIF that takes two integers and returns their sum. This function will be written in Zig and called from Elixir code.

const std = @import("std");
const erl = @cImport({
    @cInclude("erl_nif.h");
});

export fn nif_add(
    env: ?*erl.ErlNifEnv,
    argc: c_int,
    argv: [*c]const erl.ERL_NIF_TERM,
) erl.ERL_NIF_TERM {
    var a: i32 = undefined;
    var b: i32 = undefined;

    if ((argc != 2))
    {
        return erl.enif_make_badarg(env);
    }

    _ = erl.enif_get_int(env, argv[0], &a);
    _ = erl.enif_get_int(env, argv[1], &b);

    const result = a + b;
    return erl.enif_make_int(env, result);
}

const func_count = 1;

var funcs = [func_count]erl.ErlNifFunc{
    erl.ErlNifFunc{
        .name = "nif_add",
        .arity = 2,
        .fptr = nif_add,
        .flags = 0,
    },
};

var entry = erl.ErlNifEntry{
    .major = erl.ERL_NIF_MAJOR_VERSION,
    .minor = erl.ERL_NIF_MINOR_VERSION,
    .name = "Elixir.ExampleNif",
    .num_of_funcs = func_count,
    .funcs = &funcs,
    .load = null,
    .reload = null,
    .upgrade = null,
    .unload = null,
    .vm_variant = "beam.vanilla",
    .options = 1,
    .sizeof_ErlNifResourceTypeInit = @sizeOf(erl.ErlNifResourceTypeInit),
    .min_erts = "erts-10.4",
};

export fn nif_init() *erl.ErlNifEntry {
    return &entry;
}

While this code may look intimidating, it's actually quite simple. Let's break it down:

• We are first importing the erl_nif.h header file, which contains the definitions for the Erlang NIF API and the std library.
• Next, we define our first and only library function, nim_add, which takes three arguments:
  • env is a pointer to an ErlNifEnv struct, which contains the environment for the NIF. This struct is used to allocate memory, create Erlang terms, and more.
  • argc is the number of arguments passed to the NIF.
  • argv is an array of Erlang terms (the arguments passed to the NIF).
  • The function code itself is a very simple and naive implementation that adds two integers.
• After we have defined all our functions, we define a constant func_count (the number of functions in the NIF).
• We then define an array of ErlNifFunc structs, containing the name, arity, and function pointer for each function in the NIF.
• Finally, we define an ErlNifEntry struct, which contains the NIF's version, name, and functions. This struct is used to register the NIF with the Erlang VM.

It is important to highlight that the num_of_funcs field in the ErlNifEntry struct must match the number of functions in the funcs array. The name field must also match the module name and function in the NIF_MOD_FUNCS of the build.zig file.

Compiling the Zig Code Into a NIF

To compile the Zig code into a NIF, use the zig build-lib command, specifying the appropriate target and output file. For example, on a Linux system:

zig build example_lib

This command will produce a shared library file: libexample_nif.so in the build/ directory.

Integrating the NIF with Elixir

To use the NIF in Elixir, we need to follow several steps:

• Load the shared library containing the NIF.
• Define a fallback function to handle cases where the NIF is not loaded.
• Call the NIF function from our Elixir code.

Loading the Shared Library

When using NIFs, Elixir needs to load the shared library containing the native code. To do this, we use the :erlang.load_nif/2 function, which takes two arguments:

• The relative path to the shared library (.so on Linux, .dll on Windows, or .dylib on macOS).
• An optional integer value (usually 0) can be used for versioning purposes.

In our Elixir module, we'll define a load_nif/0 private function that handles loading the shared library. We'll also use the @on_load module attribute to specify that load_nif/0 should be called when the module is loaded.

Defining a Fallback Function

It's important to provide a fallback function that will be called if the NIF fails to load. This function should have the same name and arity as the NIF function and return an error tuple such as {:error, reason} or call the :erlang.nif_error/1 BIF (Built-In Function) with an appropriate error reason.

In our example, we define an add/2 fallback function that calls :erlang.nif_error(:nif_not_loaded).

Calling the NIF Function from Elixir Code

Once the shared library is loaded and the NIF function is linked to the Elixir module, you can call the NIF function just like any other Elixir function. In our example, we call ExampleNif.add/2 to perform the addition using the NIF we wrote in Zig.

By following these steps, you can seamlessly integrate NIFs written in Zig into your Elixir applications, taking advantage of both the performance benefits of Zig and the maintainability and concurrency features of Elixir.

Writing Elixir Code to Use the NIF

Create a new Elixir module called ExampleNif:

defmodule ExampleNif do
  @on_load :load_nif

  def load_nif do
    :erlang.load_nif('./build/libexample_nif', 0)
  end

  def nif_add(_a, _b), do: :erlang.nif_error(:nif_not_loaded)

end

Let's review this. We:

• Create an Elixir module called ExampleNif.
• Specify that the load_nif/0 function should be called when the module is loaded.
• Define a private function, load_nif/0, that loads the shared library containing the NIF.
• Define a fallback function (nif_add/2) that will be called if the NIF fails to load.

Note: :erlang.load_nif loads and links the Zig compiled library to the Elixir module; and it also loads this function before the definition of our nif_add/2 fallback function. This is why we need to define the fallback function nif_add/2 with the same arity and name as the NIF function.

Because of Elixir's pattern-matching capabilities, the fallback function will handle calls to nif_add/2 if the NIF is not loaded, and if the NIF is loaded, the NIF function will handle the call as defined in load_nif/0.

Verifying That the NIF Works

To verify that the NIF is working as expected, create a simple test program in Elixir:

defmodule ExampleNifTest do
  use ExUnit.Case

  test "add/2 NIF" do
    assert ExampleNif.nif_add(1, 2) == 3
    assert ExampleNif.nif_add(-5, 10) == 5
  end
end

Run the test suite using mix test. If the tests pass, it indicates that the NIF is functioning correctly, and our addition is done through our Zig compiled library rather than on Elixir code directly.

Considerations

It is amazing that we can use code from another language through NIF. However, there are a few things you might want to be aware of, starting with the parameters we initially declared on our Zig library:

export fn nif_add(
    env: ?*erl.ErlNifEnv,
    argc: c_int,
    argv: [*c]const erl.ERL_NIF_TERM,
) erl.ERL_NIF_TERM {
    var a: i32 = undefined;
    var b: i32 = undefined;
...
    _ = erl.enif_get_int(env, argv[0], &a);
    _ = erl.enif_get_int(env, argv[1], &b);

We are parsing the first two arguments and casting them into an integer. What happens if we pass a float value to this function? Try it out:

iex(3)> ExampleNif.nif_add(-100.0,-11)
-1431655777

We get junk values, as there are no type checks or logic to handle float values in our Zig function. These can cause unexpected bugs and issues that developers need to be more vigilant to catch.

Wrapping Up

In this post, we set up Zig for Elixir NIF development, wrote a simple NIF in Zig, and integrated the NIF with an Elixir application.

By leveraging the strengths of both Elixir and Zig, developers can create efficient, maintainable, and performant applications.

That said, NIFs should be used with caution and deliberately. Ensure you add the necessary tests and failsafes.

Happy coding!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!