Some direnv best practices. Actually just one.

#direnv #nix

I use direnv to configure the environment in most projects I'm working on. In my day job a few projects use direnv to load Nix environments for developers on macOS and Linux, and I did some of the work to make that a good experience. It's in doing that that I've learned what can help, and what might also help you.

Be quick or be dead

There's one rule that should never, ever be broken, and it's almost all I'm going to talk about in this post, and it is: do not block.

By this I mean that .envrc¹ should run in a few hundred milliseconds, no more. 500ms is pushing it. That's not long. There's no way to instantiate a Nix shell² in that time, for example.

So don't. Do not call nix-shell in .envrc.

The same rule holds if you're using NPM – maybe you're tempted to put npm install into .envrc for example – or another package management or build tool. Think twice before using curl or wget. The rule always applies: don't do anything in .envrc that's not going to return quickly.

Why blocking is a problem

$ cd some_directory
direnv: loading .envrc
direnv: ([direnv export bash]) is taking a while to execute. Use CTRL-C to give up.
... a long time passes ...

Your interactive shell is blocked. You wanted to pop in to take a look at README.txt but now you have to wait.

Bright idea #1: open another terminal and `cd some_directory`

Whereupon direnv starts a second long-running process. Facepalm: should've guessed that.

Bright idea #2: hit ctrl-c to give up

The background process was killed. Good. You type git pull to make sure you're up to date.

You have just entered a command: git pull. If direnv is configured correctly its hook will be run just before your shell displays the next prompt. direnv will find the .envrc in the directory and run it, starting another long-running process.

You have been eaten by a grue.

A thing to note: hitting ctrl-c doesn't always work³. The long-running process sometimes keeps running in the background, writing to the terminal, using RAM, CPU cores, and your battery life. At this point you might use pkill on the background process.

You have just entered a command: pkill some_thing. If direnv is configured correctly ...

Long story short: you have been eaten by a grue.

Bright idea #3: look at `README.txt` from your editor

You open README.txt and your editor locks up. You wonder why your editor isn't responding. You force kill it and try again. It locks up. You reboot and try again. It locks ...

Facepalm #2: you installed the direnv plugin to your editor. Your editor has triggered direnv into starting yet another long-running process.

Bright idea #4: get angry

This actually works, but is ultimately unfulfilling and doesn't solve the underlying problem.

Anyway, rm some_directory/.envrc or direnv deny some_directory and you can freely come and go, but you're not getting the benefit of that direnv integration you signed up for. It might be all that you can do when working with someone else's code though.

How to do it instead

Have a separate process to build your environment. With Nix, for example, you could write a build script like:

#!/usr/bin/env bash
nix-shell --run 'direnv dump > .envrc.cache'

Your .envrc could be as simple as:

source <(direnv apply_dump .envrc.cache)

This change, as basic as it looks, gives you back control:

You can cd into any directory without worry that you'll trigger a build.
You can open any file in your editor without worrying that it'll freeze.
You are unlikely to run the build script multiple times concurrently (and ctrl-c will probably work if you do).
You choose when to build, e.g. when you're not running on battery power.

There is more you can do with this. For example, the build script above will show an error if you haven't created .envrc.cache yet. You could make it instead prompt⁴ the user to run the build script.

A couple of other things

If you use ssh-agent then the cache will contain a value for SSH_AUTH_SOCK. This isn't sensitive but it will get out of date if you reboot, say. The symptom is that ssh and commands that use ssh, like git, will always prompt you for your password.

The cache will also contain values for DIRENV_DIFF, DIRENV_DIR, and DIRENV_WATCHES. These are direnv's bookkeeping records. Applying the dump without filtering these variables out can cause weird behaviour.

You can update your build script to address both of these problems like so:

#!/usr/bin/env bash
nix-shell --run \
  'unset ${!SSH_@} ${!DIRENV_@} && direnv dump > .envrc.cache'

At NoRedInk we go further and cache only the difference in the environment so that loading the cache doesn't clobber environment variables unrelated to the project. We also detect if the cache is stale and prompt the user to recreate it.

There's more to talk about

Like: forking background processes from .envrc (be prepared to be eaten by a grue many times before you get it right), tools like lorri, and perhaps some more about NoRedInk's tooling, but there's enough in this post already. I hope it's useful. Please like and subscribe :-)

direnv looks for a .envrc file in a directory you cd into. If it finds one it interprets it as a Bash script, then copies all of the exported environment variables into your shell's environment (which doesn't have to be Bash). It does some bookkeeping too so that it knows how to restore the previous environment when you cd out of that directory. ↩
The nix-shell command is how you start a Nix shell environment with the packages and configuration you've asked for in a shell.nix file. How that works is way out of scope for this post. Suffice it to say that nix-shell almost never returns in less than ~5 seconds, and can take hours. ↩
Maybe this is a bug in direnv, something to do with process groups or some such. Thing is, it's a problem that exists and we have to account for it. ↩
Note that I wrote "prompt", not "run the build script" or "ask the user if they want to run the build script". Either of those would block. ↩