What's in a Makefile?

Gavin Panella — Tue, 28 Jan 2020 08:26:36 +0000

Rules like this:

  target: [prerequisite] [...]
      shell command
      another shell \
         command on multiple lines

The target is the name of a file that can be built by those shell commands. If you run make target and the target is older than any of the prerequisite files, or doesn't exist, Make runs the shell commands. The prerequisites are also file names; Make will look for rules to build them too, recursively. If all of them are up to date, Make will say `target' is up to date and exit without running any commands.

Some things that might trip you up at first:

Those shell commands must be indented by a proper tab character.
Each line of shell commands is $-expanded by Make before being passed to the shell. Make expands $(NAME) and ${NAME} but $NAME would work like ${N}AME.
Each logical shell command is run in a separate shell. The example above would result in something like the following:
$(SHELL) -c 'shell command' && $(SHELL) -c 'another shell command on multiple lines'
This means that shell variables and functions are not passed from one
command to the next.
Execution stops at the first failure.

If the symbols in a Makefile look weird, check the GNU make Quick Reference. For convenience here are a few that are more common:

Symbol	What it means
`$@`	The file name of the target.
`$<`	The name of the first prerequisite.
`$^`	The names of all the prerequisites.
`$(@D)`	The directory part of `$@.`
`$(@F)`	The file-within-directory (`basename`) part of `$@`.
`$$`	A literal dollar sign. Useful when you really need to reference a shell variable.

Make also has variables, recipes, functions, and conditional evaluation, for example, but everything is based around rules like the one at the top. Check out the GNU make manual to learn more. There are other variants of Make, but GNU make seems to be prevalent. Its manual calls out where behaviour is specific to GNU so it's a decent starting point for whatever variant you're working with.

Good luck!

Some direnv best practices. Actually just one.

Gavin Panella — Mon, 04 Nov 2019 13:24:03 +0000

I use direnv to configure the environment in most projects I'm working on. In my day job a few projects use direnv to load Nix environments for developers on macOS and Linux, and I did some of the work to make that a good experience. It's in doing that that I've learned what can help, and what might also help you.

Be quick or be dead

There's one rule that should never, ever be broken, and it's almost all I'm going to talk about in this post, and it is: do not block.

By this I mean that .envrc¹ should run in a few hundred milliseconds, no more. 500ms is pushing it. That's not long. There's no way to instantiate a Nix shell² in that time, for example.

So don't. Do not call nix-shell in .envrc.

The same rule holds if you're using NPM – maybe you're tempted to put npm install into .envrc for example – or another package management or build tool. Think twice before using curl or wget. The rule always applies: don't do anything in .envrc that's not going to return quickly.

Why blocking is a problem

$ cd some_directory
direnv: loading .envrc
direnv: ([direnv export bash]) is taking a while to execute. Use CTRL-C to give up.
... a long time passes ...

Your interactive shell is blocked. You wanted to pop in to take a look at README.txt but now you have to wait.

Bright idea #1: open another terminal and `cd some_directory`

Whereupon direnv starts a second long-running process. Facepalm: should've guessed that.

Bright idea #2: hit ctrl-c to give up

The background process was killed. Good. You type git pull to make sure you're up to date.

You have just entered a command: git pull. If direnv is configured correctly its hook will be run just before your shell displays the next prompt. direnv will find the .envrc in the directory and run it, starting another long-running process.

You have been eaten by a grue.

A thing to note: hitting ctrl-c doesn't always work³. The long-running process sometimes keeps running in the background, writing to the terminal, using RAM, CPU cores, and your battery life. At this point you might use pkill on the background process.

You have just entered a command: pkill some_thing. If direnv is configured correctly ...

Long story short: you have been eaten by a grue.

Bright idea #3: look at `README.txt` from your editor

You open README.txt and your editor locks up. You wonder why your editor isn't responding. You force kill it and try again. It locks up. You reboot and try again. It locks ...

Facepalm #2: you installed the direnv plugin to your editor. Your editor has triggered direnv into starting yet another long-running process.

Bright idea #4: get angry

This actually works, but is ultimately unfulfilling and doesn't solve the underlying problem.

Anyway, rm some_directory/.envrc or direnv deny some_directory and you can freely come and go, but you're not getting the benefit of that direnv integration you signed up for. It might be all that you can do when working with someone else's code though.

How to do it instead

Have a separate process to build your environment. With Nix, for example, you could write a build script like:

#!/usr/bin/env bash
nix-shell --run 'direnv dump > .envrc.cache'

Your .envrc could be as simple as:

source <(direnv apply_dump .envrc.cache)

This change, as basic as it looks, gives you back control:

You can cd into any directory without worry that you'll trigger a build.
You can open any file in your editor without worrying that it'll freeze.
You are unlikely to run the build script multiple times concurrently (and ctrl-c will probably work if you do).
You choose when to build, e.g. when you're not running on battery power.

There is more you can do with this. For example, the build script above will show an error if you haven't created .envrc.cache yet. You could make it instead prompt⁴ the user to run the build script.

A couple of other things

If you use ssh-agent then the cache will contain a value for SSH_AUTH_SOCK. This isn't sensitive but it will get out of date if you reboot, say. The symptom is that ssh and commands that use ssh, like git, will always prompt you for your password.

The cache will also contain values for DIRENV_DIFF, DIRENV_DIR, and DIRENV_WATCHES. These are direnv's bookkeeping records. Applying the dump without filtering these variables out can cause weird behaviour.

You can update your build script to address both of these problems like so:

#!/usr/bin/env bash
nix-shell --run \
  'unset ${!SSH_@} ${!DIRENV_@} && direnv dump > .envrc.cache'

At NoRedInk we go further and cache only the difference in the environment so that loading the cache doesn't clobber environment variables unrelated to the project. We also detect if the cache is stale and prompt the user to recreate it.

There's more to talk about

Like: forking background processes from .envrc (be prepared to be eaten by a grue many times before you get it right), tools like lorri, and perhaps some more about NoRedInk's tooling, but there's enough in this post already. I hope it's useful. Please like and subscribe :-)

direnv looks for a .envrc file in a directory you cd into. If it finds one it interprets it as a Bash script, then copies all of the exported environment variables into your shell's environment (which doesn't have to be Bash). It does some bookkeeping too so that it knows how to restore the previous environment when you cd out of that directory. ↩
The nix-shell command is how you start a Nix shell environment with the packages and configuration you've asked for in a shell.nix file. How that works is way out of scope for this post. Suffice it to say that nix-shell almost never returns in less than ~5 seconds, and can take hours. ↩
Maybe this is a bug in direnv, something to do with process groups or some such. Thing is, it's a problem that exists and we have to account for it. ↩
Note that I wrote "prompt", not "run the build script" or "ask the user if they want to run the build script". Either of those would block. ↩

DEV Community: Gavin Panella