Originaly published at: https://medium.com/att-israel/how-nix-shell-saved-our-teams-sanity-a22fe6668d0e
We're developing a large React Native app which relies heavily on native components already written in Java, C++, and Objective-C. This means that we needed to develop, build, and test many different platforms on complex developer environments and build tools, that change often with platform updates.
This became a burden for our teams, with knowledge spread across many developers, installation guides, readme files, and internal wiki pages. It became expected that installations would take several days, and even a minor change in a dependency version resulted in inconsistent builds with obscure error messages.
Some suggested Dockerizing the environment, but after several attempts, Nix became our tool of choice. Nix allows us to share the same development environment across Linux and macOS with exact dependencies for tools such as CMake, Ninja, Android NDK, etc. With Nix installed, when opening the repository the developer is greeted by all the required dependencies available in their shell. We use Linux for android builds and macOS for both android and apple builds.
So, What’s Nix?
Nix is both a package manager and build tool. Generally, these two are separate things, such as RPM and Make. This unity becomes useful with Nix’s source deployment model in which packages are built from source. Most of the time the package is substituted transparently for a cached binary from a server (as long as the hash of the build instructions is the same.)
Nix prioritizes consistency, and to achieve this it forces you to declare all dependencies and inputs explicitly while sandboxing the build environment from your shell environment and the internet. Not only is the package built from source, but also its dependencies and their dependencies, which can depend on each other, as nodes in a graph mesh.
Nix-env, the Package Manager
With nix-env you can manage user environments. nix-env creates an abstraction layer over bin directories in your PATH with symlinks to /nix/store. As it uses symlink references it can do several important things:
- It keeps track of versions of your environment, and in O(1) time it can roll back to a different version by changing the symlink to a previous profile.
- Installations and uninstallations are atomic. The later version isn’t referenced until the installation is complete.
- As dependencies are not installed in a global folder, multiple users on a machine cannot override or compromise each other’s dependencies, and therefore are allowed to install packages without privileges.
This is possible because each version of a package is installed in a different directory under /nix/store and erasing a dependency doesn’t remove it from disk until it is completely de-referenced and garbage-collected.
Nix takes versioning into its own hands by hashing the build instructions and its input. Even the slightest change constitutes a new version, as the hash is different. Components reside in the Nix Store, alongside all their dependencies, as:
/nix/store/f2rrk276criwxn19bf82cglym4dkv9gr-ninja-1.9.0.drv
/nix/store/iwm3knkdi294rj50w9ai5rdwaglgr362-ninja-1.9.0/
The last characters are the human-readable name attribute. Nix-env is managed with the nix-env command and the .nix-profile directory.
Installation Issue on Mac
Nix can either be installed for a single user (who owns /nix) or as multi-user (in which root owns /nix). However, on a Mac neither will work anymore, as the root filesystem (anything under /) has been read-only since macOS 10.15. Nix can’t trivially change the path for the Nix Store, as all their binary cache has been compiled with /nix/store as its path. The current workaround is to change the path but mount it as an unencrypted (encrypted at rest) APFS Volume.
$ sh <(curl -L https://nixos.org/nix/install) --darwin-use-unencrypted-nix-store-volume --daemon
The installation will explain what it will do, and will request super-user access, which it will call dozens of times. This is how the Nix Store Volume looks with Disk Utility:
And here it is in Finder:
Nix Store Volume in Finder. For some reason, the Unix timestamp is at 0 (and I’ve given out my timezone).
Nix-shell, the Virtual Environment
It was however nix-shell that made an impact for us. With nix-shell, we can create virtual environments per-project, without having to install dependencies on a user or system level with nix-env.
Just add a shell.nix file in your project. Then, when you enter nix-shell, the environment and all the dependencies are ready for use. This file is, of course, committed to source control and shared among all developers. The file lists dependencies, environment variables, and shell hooks to be run when loaded.
Example shell.nix file with two different Nixpkgs sources.
This can be further integrated into the shell with Direnv, which automatically activates the environment when the directory changes; and Lorri, a daemon process that monitors the project’s shell.nix for changes, and automatically reloads the environment if it has. Niv eases the dependency management of a project with a sources.json file, like a higher-order package manager for Nix-shell.
Some prefer the use of Nix-shell over Nix-env for entire user-level environments, as it can be controlled in an isolated, declarative way. Home Manager enables the configuration of user-specific (non-global) packages and “dot-files.” See what you can do in the NixOS wiki. Finally, Nix-drawin enables the configuration of your Mac the way NixOS does with a configuration.nix file.
Nix-shell can be extended into your OS with the tools above, but it can also be used in a narrower, specific way. It’s possible to run commands in Nix-shell without entering its interactive shell with:
nix-shell --run "node ./index.js".
And it’s possible to specify Nix-shell as an interpreter for a file with a shebang at the top of the file:
#! /usr/bin/env nix-shell
#! nix-shell -i real-interpreter -p packages...
The above file will be executed inside of nix-shell, along with its environment.
Nix-build, the Build Tool
Nix-build is a build manager with correctness in its top priority. That is, all builds will be identical given the same build tools and inputs.
Build managers take sources, such as source code and dependencies, and invoke generators such as compilers, to create derivates such as binaries. Both sources and derivates are components. This is the task of tools like Make, CMake, Ant, or Gradle.
Nix builds are based on a derivation, which is a set that lists exact (hashed) dependencies and exact (hashed) build scripts, which look like this:
Derive([("out","/nix/store/winl36i87aydwj5qgrz0nbc7kq3w0yzi-user-environment","","")],[],["/nix/store/kygr761f08l1nanw27lfxkg8qibf0qn1-env-manifest.nix"],"builtin","builtin:buildenv",[],[("allowSubstitutes",""),("builder","builtin:buildenv"),("derivations","true 5 1 /nix/store/9nqninr2aaicvmq83q10d5a1hwagbzyc-hello-2.10 true 5 1 /nix/store/df26nnjiw55rvv6mxy4kapps9h4kfvw7-niv-0.2.19-bin true 5 1 /nix/store/f3swypnb5zi5yd3w7k2ycwyv6b3sv8fa-direnv-2.28.0 true 5 1 /nix/store/vgdizqicd30k4183ssq7g6i07dvys6xl-home-manager-path true -10 1 /nix/store/4023c0ymrxsg1x36jxmnircqjl1y9fkq-nodejs-14.17.6"),("manifest","/nix/store/kygr761f08l1nanw27lfxkg8qibf0qn1-env-manifest.nix"),("name","user-environment"),("out","/nix/store/winl36i87aydwj5qgrz0nbc7kq3w0yzi-user-environment"),("preferLocalBuild","1"),
Nix Expressions, the Language
The above is a minified version of its human-readable version, written functionally with a Nix Expression:
https://gist.github.com/ronenlh/c2c9ca9ed319bfadd212f2eb15e29629#file-default-nix
The entire file is a single function. Lines 1 to 6 describe a set, passed as the only parameter. The set defines all the dependencies needed to build the component. : in line 6 defines the beginning of the function’s body.
The entire body is a call to stdenv.mkDerivation, which will minimize the data into the derivation written above. rec is a function that will enable recursion inside the data set, allowing the definition of values in terms of other keys in the set.
For didactic purposes, the syntax could be rewritten as a JavaScript lambda as:
({ stdenv, ... }) => stdenv.mkDerivation(rec({ ... }))
The value for src is retrieved from a remote URL and validated with a hash. src is the expected key for the standard build tool, which will perform the standard autoconf (./configure ; make ; make install) shell script.
It’s possible to experiment with the Nix language in its interactive shell.
Nixpkgs, the Package Repository
The above function is not yet callable, as we don’t have the parameters for the function. We can achieve the same result with another rec which recursively defines the needed components and its parameters. e.g.,
rec {
lib1 = import package1/default.nix { };
program2 = import package2/default.nix { inherit lib1; };
}
This turns all dependencies into a dependency graph, and as long as they are acyclical, Nix can build all of them. This set can be abstracted with the callPackage function. This is how it is done in the Nix Packages Collection in this amazing file all-packages.nix.
This file is queried implicitly when we install a package in the form:
nix-env -i hello
This is the equivalent of:
nix-env -f .../all-packages.nix -i hello
Both will build and install hello. Nix will represent all the dependencies as a graph and build them as needed. It’s important to note that Nix is Lazy: The parameters are not evaluated until called, which means that dependencies will not be built until (or if) needed.
The file for all-packages can be changed using the nix-channel command. Channels are sorted by stability status.
How Can I Install a Specific Version of a Package with Nix?
The Nixpkgs repository includes the latest versions of packages (according to the selected stability branch). Packages depend on each other and are built as a whole. To pin a specific version of a dependency, you must switch to a different revision of Nixpkgs altogether. A great utility to reverse-search a Nixpkgs revision according to a package’s version is Lazamar’s Nix Package Search.
It’s best practice to always pin your build dependencies to a specific revision of Nixpkgs, for consistency (as you’d do with Docker), and to update to the latest version of Nixpkgs on Nix-env, according to your selected Nix-channel (as you’d do with Homebrew).
Other Nix Tools
- NixOS — using the primitives listed above, builds and configures an entire Linux distribution. The whole of NixOS is defined inside Nixpkgs repository, which is incredible.
- NixOps — related to Cloud deployment, deploys NixOS system configurations to remote machines, as well as provisions cloud resources.
- Hydra — CI tool that periodically checks out the source code of a project, builds it, tests it, and produces reports for developers. Hydra is used to check the stability status of the Nix channels.
- Flakes —an upcoming feature that will remove much of the hassle of pinning dependencies with syntactic sugar. Each dependency’s commit hash will be stored inside a flake.lock file. This is intuitive for NPM/Yarn or Cargo users.
So, Why Not Docker?
Nix and Container engines such as Docker are two very different tools. One is a package and build manager, the other is a resource isolation mechanism that virtualizes the host’s operating system. Both have great caching mechanisms behind them, and both can be used for consistent environments on Linux machines. See below about how Replit migrated from Docker to Nix.
The main abstraction of Docker is the Container: a loosely isolated, lightweight, portable, and encapsulated environment that contains everything needed to run the application. The container — which is runnable — is described by a read-only Image. The image is created by a Dockerfile, where each directive creates a separate Layer, tagged by its cryptographic hash and cached.
Like layers, images can be built one on top of the other and vertically stacked, e.g., the official Node image is built on top of the tiny Alpine Linux image. Your node app would probably be stacked on top of the node image.
Layers of Docker node image (node:slim) from Docker Hub
Containers define the implementation of an image or a layer in terms of another, its parent. Nix creates new functionality by assembling or composing dependencies. Nix requires dependencies to be explicit, and these dependencies are black-boxed and consumed through their interface.
However, Dockerfiles don’t have to be linear. Multi-stage builds introduce a new abstraction: the stage. Docker’s new BuildKit traverses stages from the bottom (of the target stage) to top in a graph data structure, skipping unneeded ones, and building stages concurrently where applicable.
Graph of BuildKit’s Multi-stage build, starting from the bottom (the target stage) to the top, discarding unneeded stages. From ‘Dockerfile Best Practices’ talk: https://youtu.be/JofsaZ3H1qM?t=1169
Favor Composition Over Inheritance
It’s difficult to change layers in Docker, as we’re not sure what each component does or how it will affect the lower layer. Also, developers are disincentivized from changing higher layers as they risk rebuilding all the lower layers in the Dockerfile. This is also a performance bottleneck in terms of concurrency, as Docker builds layers in sequence, and unneeded stages will be unnecessarily built and then discarded.
Docker has a great advantage which is immediately familiar to developers and ops alike. Nix originated as a Ph.D. thesis and it sometimes feels like that. But a design that doesn’t take change into account risks major redesign in the future. Docker hashes machine states, Nix hashes the precise components of a build. As explained earlier, the two tools serve different purposes.
In our case, we were building a library for a client app, so there was no need to ship a machine container as would’ve been the case when developing a Node microservice in Kubernetes. We just needed to share a consistent build environment to create replicable builds. Furthermore, with nix-shell, we can still use our local XCode and the rest of macOS’s walled garden for our tvOS and iOS builds.
The Case of Replit
Replit is a collaborative in-browser IDE with support for a huge number of languages. Replit started with a separate Docker image for each language, but concluded that it was simpler and more efficient to use a single monolithic image: Polygott. This has become a huge burden to maintain, in their own words, as “every new package creates a new exciting way things can break.”
With Nix, Replit users themselves can define infinite combinations of sandboxed environments without the need to maintain a monolithic Docker image. Each machine has /nix/store (with all the binaries cached) mounted, so the instantiation of their environment is immediate.
How Does it Compare with Homebrew?
Homebrew is an incredible tool which has become second nature for most macOS users. Installations work out of the box and is intuitive to use.
Like Nix, Homebrew builds from source unless it finds a “bottle,” that is, a pre-built binary. Similarly — and for the same reason — Homebrew has to be installed into a default path (/opt/homebrew on Apple Silicon or /usr/local on Intel) to enjoy pre-build binaries. This folder is referred to as the cellar.
Homebrew uses Ruby for its formulae, which provides instructions and metadata for Homebrew to install a piece of software. A formula is defined as a class that inherits from Formula. This follows the object-oriented paradigm, unlike the functional Nix derivations which are defined with a function.
class Wget < Formula
homepage "https://www.gnu.org/software/wget/"
url "https://ftp.gnu.org/gnu/wget/wget-1.15.tar.gz"
sha256 "52126be8cf1bddd7536886e74c053ad7d0ed2aa89b4b630f76785bac21695fcd"
def install
system "./configure", "--prefix=#{prefix}"
system "make", "install"
end
end
Homebrew can be used in Linux (formerly Linuxbrew), although Linux distributions often have popular package managers. Similar to nix-channels, brew uses “Taps,” which are third-party repositories.
The immense popularity of Homebrew in Mac gives it an advantage over Nix’s build reliability and thoughtful dependency graph. Most installations are pre-built and “just work.”
Conclusion
From a marketing perspective, I find that Nix lacks branding and distinctive names for their services (except for Hydra and Flakes), which makes it difficult to search for documentation. Nix has merged Nix and NixOS documentation, so trivial beginner searches about nix-env easily lead to solutions about the modification of configuration.nix, which is only applicable to NixOS.
The use of /nix/store has been a bit non-conventional on the part of Nix, as it breaks the FHS guidelines. It would have been more appropriate to put it under /var somewhere. I don’t think macOS follows FHS, but now the root (/) level is read-only in macOS, and Nix had to scratch their head to find workarounds.
Nix isn’t as intuitive as other build tools, but it excels at correctness. As such it aims to have the rigor of science and shows the hard work from academia. It has been embraced by the communities of functional languages such as Haskell and NixOS has piqued the interest of the entire Linux community.
Top comments (2)
Incredibly thorough overview, thanks! As someone in the midst of a nix flake cross-compilation project, it's been a bit of a love/hate thing. When I figure out how to say what I'm trying to say and it works, it's beautiful, but coming to that point is often difficult, and I find myself reading through Nix code way more than documentation. I'm hoping this continues to improve, because the underlying idea is so valuable, better than any alternative. Just not always sold on the implementation.
I’d love to see how you configure your React Native environments in nix. Care to post an example?