Solve compatibility problems at NixOS vs non-Nix builds space

A few weeks ago I have switched to NixOS completely. Previously I used the Nix store on top of Ubuntu. It is working well, but I had to still configure the host OS for Nix. I'm a big fan of declarative and reproducible systems, so Nix is a paradise for me :)

What is NixOS?

It is a bit hard to define because it is a programming language, a build system, a package manager, and finally a Linux operating system based on all the others.

Everything is defined in .nix files. Nix language is a functional programming language designed for building software. Each program defines its exact dependencies during the build, so when you download a package from the store it always downloads the right version of dependencies. Because of this, Nix is a bit different than any other Linux distribution. On a regular Linux lots of dependency is shipped with the distribution, so distribution maintainers can hard code them. A really good example is the main interpreter for ELF binaries. If you want to know more, please follow my previous post.

What is the issue?

In short for example /lib64/ld-linux-x86-64.so.2 doesn't exist on NixOS, because this file is part of glibc package, but each binary has its version. This isn't an issue with Nix packages, because they had compiled under this philosophy, but once you download something out of the Nix ecosystem, you are in a trouble.

Another problem is /bin/bash also doesn't exist :). I think I don't have to describe deeply why this should be a bad thing.

Kernel modules are also located in the Nix store instead of some well-known locations.

Some other libraries as well, for example, VS Code remote language server can't start because node doesn't find a certain .so file.

How to solve all of this?

I found a nice way to cover all the topics up there. In my main NixOS config /etc/nixos/configuration.nix I made the following changes:

  environment.variables = {
    LD_LIBRARY_PATH = "$(cat ${pkgs.gcc.outPath}/nix-support/cc-ldflags | cut -dL -f2 | sed 's/.$//'):/run/current-system/sw/lib:/run/current-system/kernel-modules/lib";
  };

Configure library search path. The first part is really ugly (VS Code specific), please let me know how to solve it nicely. But the rest is ok because luckily NixOS has reference to the system's main dependencies at /run/current-system.

  system.activationScripts.nonposix.text = ''
    ln -sf /run/current-system/sw/bin/bash /bin/bash
    rm -rf /lib64 ; mkdir /lib64 ; ln -sf ${pkgs.glibc.outPath}/lib/ld-linux-x86-64.so.2 /lib64
  '';

Create a symlink for bash, and another for the main interpreter.

In this way, we fake Nix to look like an average Linux.

You can find my personal script-kiddie Nix config here.

But if you are looking for some rock-star solutions, I suggest to check out these repositories:

• https://github.com/michojel/NixOS by Michal Minář
• https://github.com/sagikazarmark/nix-config by Márk Sági-Kazár

(Special thanks for the inspiration)

Comments

[archite]

I use the following which works on aarch64 as well:

  systemd = {
    tmpfiles = {
      rules = [
        "L+ /lib/${builtins.baseNameOf pkgs.stdenv.cc.bintools.dynamicLinker} - - - - ${pkgs.stdenv.cc.bintools.dynamicLinker}"
        "L+ /lib64 - - - - /lib"
      ];
    };
  };

[Richard Kovacs]

I love it :D thanks