Don't Give Up on Nix Just Yet

In my case, I almost did, but I'm glad that I ended up not.

There are many great articles to show how Nix can be useful, so instead of me repeating those points, I will link them at the bottom of this post.

WTH is Nix?

Before I go ahead and talk about why I felt like giving up and why you shouldn't, let me describe what's my perspective on Nix and why I'm interested to keep using it.

You may or may not have come across Built with Nix from something like direnv wondered What Nix is.

Nix is at least these three things:

• Nix, the (programming) language that you write in *.nix files to let Nix work for your liking.
• Nix, the package manager powered by few binaries like nix
• NixOS, Linux "distro" that comes with Nix to configure OS itself

Although I have a super tiny experience about NixOS before, I'm not really using NixOS at the moment so my focus in this post is solely about Nix the package manager and Nix the language.

Why did I even care to try Nix?

There are multiple motivations and below is one of them.

I don't write Go programs often enough that usually Golang is not installed in my machine.
But when I need to, installing Go was a bit painful and confusing enough for me (surprisingly almost every time!). And my usual approach to deal with this issue is using Docker. All though I often use containers like in a way of using VM to avoid polluting my host machine, a container is no localhost, so it's limited and it could "force" you to be container "expert" when you don't need to be just yet.

What you wanted to do was just to write a Go program and run it remember? 😅

Then I discovered Nix and I gave it a Go. 😆

$ go env
zsh: command not found: go

$ nix-shell -p go

# now you are in Nix shell

[nix-shell:~]$ go env 
# ...
GOOS="darwin"
GOPATH="/Users/username/go"
# ...

[nix-shell:~]$ cat <<EOF > hello.go
package main

import "fmt"

func main() {
    fmt.Println("Hello, 세상")
}
EOF

[nix-shell:~]$ go run hello.go
Hello, 세상

[nix-shell:~]$ exit
$ go
zsh: command not found: go

That felt like a breeze to me. I didn't need to do anything like setting the right environment variables and so on. Just nix-shell -p go then I have it and exit then I don't!

Technically you still do but it doesn't pollute the environment. The reality is more complicated than that but let's leave it at that for now.

This, my first experience with Nix and it was probably like a year ago or so. But then I didn't go into Nix much yet until recently.

Since I do like to isolate environments, reuse configurations, and reproduce those setups to multiple devices, I've been using my own dotfiles and sometimes, containers.

What other tools I used before Nix

Since I didn't want to use all different tools for all different OSes (and their own package managers), Homebrew (and Linuxbrew) was my first choice to install necessary packages.

But I find some issues with it in my use case, as it's often the case with any other tools.

To list some:

• installing specific versions with Homebrew can be challenging. (Especially when you need both at the same time)
• many times, it feels slow to update new formulae (brew update) which it always seems to run when I brew install [package]

Enter asdf.

asdf solves those issues mostly for programming language packages.

But in my opinion, Asdf is more of a runtime version manager than a package manager.

However I'm going to skip explaining about asdf since this is a post about Nix! But you should definitely check it out if you are interested :)

However, remember that I mentioned that I used Nix briefly before? At this point, I felt like Nix can be a tool that can complement both tools in a "new way".

So I decided to try Nix to see if it's possible to replace other tools. Therefore my dotfiles started to add Nix files.

Why I felt like giving up

So it felt like everything was actually going ok until it didn't. I reached the point one package didn't install properly on macOS, while it was fine on Linux with the unstable channel. While this can be fixed over time I wanted to make an exception just for few packages only for Darwin (macOS) platform.

And I couldn't manage to actually make that happen for a while.

Why? Because I wanted to do this kind of advanced stuff without actually understanding the language first.

Why did I do that? I think because that's how I approached with other tools.

I usually learn tools first by knowing which command and arguments to use to do this and that.

And Brewfile (in case of brew) and .tool-versions (in case of asdf) are extra that comes after just typing commands. I'm pretty sure I'm not alone on this.

But in the case of Nix, using Nix the language properly could be essential very fast.

What advanced stuff that I tried to do?

Optional overlaying for Darwin

if any of this stuff doesn't make any sense to you feel free to skip to the next section

This is a typical top line you may come across with .nix files

{ pkgs ? import <nixpkgs> {} }:

What if you want to overlay some packages conditionally? You can try something like the below.

{ pkgs ? import <nixpkgs> {
  # to simplify the code, we import `overlay.nix` for now which you will get to see in a minute
  overlays = [] ++ pkgs.lib.optionals pkgs.stdenv.isDarwin [ (import ./overlay.nix) ];
} }:

But how do I get pkgs.lib and pkgs.stdenv if I'm in the import section that doesn't know about pkgs yet?

You can actually import inside import (with let ... in in this case) like below.

{ pkgs ? import <nixpkgs> (
  let
    unstable = import <nixpkgs> {};
    stdenv = unstable.stdenv;
    lib = unstable.lib;
  in
    {
      overlays = [] ++ lib.optionals stdenv.isDarwin [ (import ./overlay.nix) ];
    }    
  )
}:

Seems too many imports in the first import section and it looks confusing? Also, can you even do that?

Yes, it might look weird but this is very valid in Nix. Inside () it just expects any valid Nix expression. Therefore, let ... in { ... } is a perfectly valid because {...} # AttributeSet is expected after import <nixpkgs>. Why import <nixpkgs> expect AttributeSet? We will get to that in the Demystifying Nix files below for details.

Now we solved the issue but because it could look confusing I split the files below.

# my-pkgs.nix
{ pkgs ? import <nixpkgs> (import ./fallback/darwin/optional-overlaying.nix) }:

# ...

# fallback/darwin/optional-overlaying.nix
let
  unstable = import <nixpkgs> {};
  stdenv = unstable.stdenv;
  lib = unstable.lib;
in
  {
    overlays = [] ++ lib.optionals stdenv.isDarwin [ (import ./overlay.nix) ];
  }

# fallback/darwin/overlay.nix
self: super:
let
  # different channel to fallback
  fallback = import <nixpkgs-fallback-darwin> {};
  pkgs = fallback.pkgs;
in with pkgs;
{
  # imagine these two are broken for Darwin in your main channel, `<nixpkgs>`,
  # now you can fallback to a different channel, `<nixpkgs-fallback-darwin>` that may work for Darwin
  inherit hello;
  inherit bash;
}

Like I said earlier if these don't make much sense yet it's ok and the section X will help you

Is Nix the language difficult?

Not really, once you get to know it, it's quite simple but it did look foreign and scary to me for a bit initially.

Because even though it's simple but also it's very flexible like lego blocks.

Nix is an actual programming language not like much simpler configuration files like Brewfile or .tool-versions and even though I've been writing code for a while I couldn't help but notice myself asking but why the whole (programming) language to manage packages?

And not just language itself in a file but also each file look somehow different to each other. For example, why default.nix looks different than shell.nix and all other .nix. What's really expected in .nix files? I got confused.

Therefore I somehow resisted embracing the Nix language. It also didn't even "look" good until I got the aha moment.

Turns out it's because Nix is super flexible. Anything that is a valid Nix expression can be .nix file. Even just the number 3 is a valid Nix expression and you can import it to other .nix files and do anything with it.

Demystifying Nix files

After this section, you will get to know why all nix files kind of somewhat don't look similar to each other except they all look Nix-y.

I tried to come up with why and this is one of my speculations that we are quite used to somewhat (implicitly) structured files.

Some example implicit structure scenarios:

• maybe main.xxx file has to exist in order for your program to work
• import lines should be on the top of the file
• main function somewhere
• have to have one function in a file at least
• could be even a whole main class should exist
• can't just have the number 3 as a whole file content

With Nix, we may need to rethink that, and to simply put, .nix files just need to be valid Nix expression and that's it.

For example, these are all valid.

you can just have 3 and that's it

# number-three.nix
3

just a list if you want

# list-simple.nix
[ 1 2 ]

import can be anywhere, I mean literally

# list-with-import.nix
[ 1 2 (import ./number-three.nix) ]
# [ 1 2 3 ]

You can think of what import does as simply embedding the source file to the target file.

# attributeset-with-import.nix
{
  one = 1;
  two = 2;
  three = import ./number-three.nix;
}

or

# attributeset-with-import-2.nix
let
  number_three = import ./number-three.nix;
in
  {
    one = 1;
    two = 2;
    three = number_three;
  }

even this can be valid

# function-mirror.nix or function-identity.nix
x: x

and you can use like this

# import-identity.nix
2 + (import ./function-identity.nix 3)
# 5

for two arguments, you can:

# function-add.nix
a: b: a + b

# import-add.nix
import ./function-add.nix 2 3
# 5

maybe AttributeSet as an argument

# function-arg-as-attr.nix
{ argInAttr, secondArg ? "with the default value" }:
argInAttr ++ secondArg

# import-arg-as-attr.nix
# I skip secondArg because it's with default value
import ./function-arg-as-attr.nix { 
  argInAttr = "I skip secondArg because it's "; 
}

now you should know these are nothing special

# just a function that has two arguments called `self` and `super`
self: super:
# ...

# just a function that expect AttributeSet that may have `pkgs`, if not `import <nixpkgs> {}` is the default value
{ pkgs ? import <nixpkgs> {} }:
# ...

# perhaps this way helps you to see it better
{ 
  pkgs ? (import <nixpkgs> {}) 
}:

But why import <nixpkgs> {}, why {}?

Because <nixpkgs> will be evaluated to a function that expects AttributeSet as a parameter that can be empty.

For example, let's look at another file below that is a function.

# https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/misc/exa/default.nix

{ lib, stdenv, fetchFromGitHub, rustPlatform, cmake, pandoc, pkg-config, zlib
, Security, libiconv, installShellFiles
}:
# it's just this file is expecting to be called by consumer that will provide all these attributes

So far when I've been with other tools, usually command line proficiency first and conf file after.
With nix, I think it's inevitable to need to be comfortable with Nix language to unleash the potential of Nix. And that's why I think there is a whole programming language to a package manager.

You can find good resources for the language:

• nixcloud.io/tour
• Nix by example
• Any nix code
• Learn Nix in Y minutes

The good thing about Nix is that there are tons of examples you can find on Github. Go ahead and get yourself familiar with them!

OK, so maybe you are convinced that Nix the language is not something we should be afraid of but is that why should I use Nix, though?

Benefits of getting started with Nix

Fresh way of doing things

It could be refreshing if you are willing to learn a new of doing things, just like embracing FP for the first time!

Nix language is actually interesting

you might not welcome less familiar syntax but they actually could help you decouple how you think with syntax.

Nix could look foreign at first just because it's different from what you are used to but it's actually quite simple and works like lego blocks. I hope that becomes more clear in the previous section.

Good opportunity for getting used to Functional Programming

Have you been wanting to use/learn FP but haven't had a chance yet?
Then maybe it could be even more interesting to you.
I don't think I mentioned that Nix is a functional programming language and it is.
It can be a good opportunity to get used to FP since, with Nix, it's very easy to start small.

You don't have to use NixOS or anything else that you don't need/want yet

Start small with nix-shell or nix repl

You get to learn how your packages are built

For example, this is how ponysay is built, https://github.com/NixOS/nixpkgs/blob/d600f006643e074c2ef1d72e462e218b647a096c/pkgs/tools/misc/ponysay/default.nix

My Nix usages so far

A cheap and fast way of installing or trying out new packages

$ nix-shell -p cowsay
[nix-shell:~/tmp]$ cowsay hi
 ____
< hi >
 ----
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

My home that comes with all the packages I want/need across all my devices

https://github.com/ryuheechul/dotfiles/blob/master/nix/home.nix

Also it's reusable to do something else like Building a Docker image

https://github.com/ryuheechul/dotfiles-launchpad/blob/2d70fc6a48a4781e281d579457a0ea5287960252/Dockerfile#L10

Using it in a project

https://github.com/ryuheechul/jmdstack/tree/master/crossplane

So much more you can do

So far I'm not using many things like NixOS, nix-darwin, etc. but you can, whenever you want.

There are also multiple ways of installing your packages even just within Nix ecosystem

• nix-os
• home-manager
• nix-env
• nix-shell
• nix-darwin

Wrapping up

There is nothing wrong with existing tools depends on your perspective/usage.

And of course, I'm still using other package managers like apt-get and brew, asdf, etc. They are good when trying everything with Nix is not quite smooth, but Nix certainly became my default package manager and reduced the usages of other tools significantly!

I'm just glad that I've embraced this new way of managing packages and my environments.

It feels clearer what exactly I need and how it's being installed and what are the dependencies and so on.

It was hard to feel that way with apt-get, etc.

I'm not sure if I will ever be a Nix "expert", but I'm pretty sure Nix will be still useful in anyways.

I'm also collecting useful links for Nix here, https://gist.github.com/ryuheechul/a0bd4e4b69565da86301ee8cc26311e1

Comments

[Benoit de Chezelles]

Great write up!!
Small comment though, where you say Because <nixpkgs> is a function this isn't exactly right, <nixpkgs> is a path, and only when you import that path with import <nixpkgs> you get a function! (:

[Heechul Ryu]

Thanks and you are right! Just revised it :D

[Heechul Ryu]

Discovered great articles for anyone to learn more about overall Nix eco system:

• stephank.nl/p/2020-06-01-a-nix-pri...
• xyno.space/post/nix-darwin-introdu...