Thinking in Make

Matt Brennan — Sun, 23 Jul 2017 00:00:00 +0000

Okay, but what is a makefile. Actually what is it.

Make is for running tasks

Here’s a simple makefile, as barebones as possible without it literally being an empty file:

do-a-thing:
    echo “did the thing”

Save that as makefile, and in the directory, run make do-a-thing:

⟩ make do-a-thing
echo “did the thing”
did the thing

Tell Make what to run and it runs it. Each line like do-a-thing: specifies a target, and the indented lines (which need to be indented with actual tabs, sorry space-lovers) after the name are the shell commands to run for that target.

Make is for generating files

When the commands in a target create a file, you should let Make know about it. If the name of a target is the same as the name of the file it creates, Make can track that:

foo.txt:
    echo lorem ipsum > foo.txt

⟩ make foo.txt
echo lorem ipsum > foo.txt
⟩ make foo.txt
make: Nothing to be done for ‘foo.txt’

Wait what? Nothing to be done? When the file exists, Make notices, and realises it doesn’t need to do anything. foo.txt isn’t going to change if it runs the commands, so it doesn’t bother running them. Remove the file, and Make will create it again.

⟩ rm foo.txt
⟩ make foo.txt
echo lorem ipsum > foo.txt

Make is for generating files from other files

Say you’re generating a file and it depends on the content of another file. Maybe you’re just copying it:

foo.txt:
    cp bar.txt foo.txt

make doesn’t do anything when you update bar.txt:

⟩ make foo.txt
cp bar.txt foo.txt
⟩ echo hello > bar.txt
⟩ make foo.txt
make: Nothing to be done for ‘foo.txt’

This is the same as the last example: the file exists, so make won’t create it again. If you want foo.txt to update whenever bar.txt does, you need to tell make about it, by marking it as a prerequisite of foo.txt:

foo.txt: bar.txt
    cp bar.txt foo.txt

Everything after the colon in a target’s name is a prerequisite of that target. It can be a file name, the name of another target, or even the name of another target that’s a file to generate. Prerequisites can also be chained: a target that’s a prerequisite can also have prerequisites, and so on.

Running this example again does what you want: if you update bar.txt and run make foo.txt again, it’s updated. If bar.txt hasn’t been updated, make won’t do anything:

⟩ make foo.txt
cp bar.txt foo.txt
⟩ echo hello > bar.txt
⟩ make foo.txt
cp bar.txt foo.txt

Make can work out filenames from patterns

What if you’ve got a whole folder’s worth of targets and prerequisites? You don’t want to have to write a target for each one. I mean, that just sounds boring, and you’d have to remember to update it whenever you added or deleted a file.

So if your file paths are predictable, e.g. if you want to express something like “here’s how to make any file in this folder from the file with the same name in that folder”, you use file patterns. In the target and the prerequisite, write the bit that’s the hole to be filled in as %, e.g. target-files/%.txt: source-files/%.txt.

Then how do you refer to the actual filenames, if all we know is the pattern? make defines a handful of automatic variables to use with pattern rules, which it sets based on the pattern and the filename it’s matching. The two you need to know are $@, which is the matched target filename, and $<, which is the prerequisite filename it infers from the target pattern, the target filename and the prerequisite pattern, by working out what in the filename replaced % in the pattern (the stem).

Given our example above, when you run make target-files/foo.txt, make matches it against target-files/%.txt and runs that target. It sets $@ to target-files/foo.txt, and because the stem is foo (that’s what replaced % in the pattern), it sets $< to source-files/foo.txt.

All together:

target-files/%.txt: source-files/%.txt
    cp $< $@

⟩ make target-files/foo.txt
cp source-files/foo.txt target-files/foo.txt

Don’t tell Make what to do, let it find its own way

Ok, what if you want to make everything? All of the files? make is actually a fully-fledged (if a little arcane, but hey, it’s 40 years old) programming language, with a handful of builtins useful for generating lists of files you want it to build.

One of these is the wildcard function. It takes a shell-like wildcard expression and expands it to a list of files that match. Let’s say your source-files folder contains foo.txt, bar.txt and baz.txt, then $(wildcard source-files/*.txt) expands to source-files/foo.txt source-files/bar.txt source-files/baz.txt. Why does this use * when patterns use %? Because reasons.

Then there’s patsubst. Give it two patterns and a list of files, and it matches and subst itutes the pat terns. So $(patsubst source-files/%.txt, target-files/%.txt, source-files/foo.txt source-files/bar.txt) evaluates to target-files/foo.txt target-files/bar.txt.

I mentioned the automatic pattern variables above. You can define your own variables, and the syntax will look familiar: source-files = $(wildcard source-files/*.txt). Refer to variables using $(variable-name), e.g. $(source-files), and if the name is a single character you can drop the (), like with $< and $@ (which aren’t anything special; make has very lenient syntax for variables, and pretty much the only characters you can’t use are =, #, : or whitespace).

Variables can be used in prerequisites and commands. You’ll see them used in commands to pass long options to command-line programs. When used in prerequisites they can be used to make a bunch of files at once. A common use case is making every file in a folder:

source-files = $(wildcard source-files/*.txt)
target-files = $(patsubst source-files/%.txt, target-files/%.txt, $(source-files))

all: $(target-files)

target-files/%.txt: source-files/%.txt
    cp $< $@

⟩ make all
cp source-files/foo.txt target-files/foo.txt
cp source-files/bar.txt target-files/bar.txt
cp source-files/baz.txt target-files/baz.txt

Breaking this down from the end, it’s the same target we saw before, which makes any .txt file in target-files/. The all target has no commands, and the variable $(target-files) as a prerequisite. When a target has no commands, it’s just there for the prerequisites. make makes those in order to make all, and then makes all by doing nothing. Finally $(target-files) is generated by using wildcard to get a list of files in source-files and patsubst to turn that list into a list of files in target-files. I’ll talk some more about this in a bit.

Alright now check this out. I’ve gone over how make only builds the things it needs to? This works really well when building a bunch of files at once:

⟩ make all
make: Nothing to be done for ‘all’
⟩ echo hello > source-files/baz.txt
⟩ make all
cp source-files/baz.txt target-files/baz.txt

Update baz.txt, leave the other two alone, and that’s the only one that make decides to build. Internally, it’s looking at file modification times. When it discovers a target file depends on a prerequisite file, and both exist, it only makes the target if the prerequisite is newer.

Oh, and if you run just make on its own, it runs the first (non-pattern) target in the file. By convention, people call this all. So you could run the last two examples with make instead of make all.

The part where I actually talk about the title of the post

What is the point of make? By now, I hope you’ll see it’s more than a simple task runner. If you have a bunch of input files, and you want to transform them into a bunch of output files, the tool you should be reaching for is make. If your input filenames look like your output filenames, see if you can write a pattern target.

These are the steps I go through when writing a makefile:

Write a simple rule using concrete filenames
Make the rule generic with patterns
Work out how to generate a list of output files from your input filenames
Make everything all at once
Receive admiration from your friends and colleagues

I’m still on step 4.

I’ll go through a concrete example: building Javascript sources using Babel. To start with, it’s a single file, src/index.js that I want to compile to lib/index.js. To transform the file we run babel src/index.js -o lib/index.js, so the whole rule would look like:

lib/index.js: src/index.js
    babel src/index.js -o lib/index.js

Now I’ve got more .js files, I want to turn it into a pattern rule. The output and input files have a pretty clear pattern of lib/%.js: src/%.js, so using that with automatic pattern variables, we get:

lib/%.js: src/%.js
    babel $< -o $@

To build all of the .js files at once, we need a list of target files that our source files would generate. So get the list of source files:

source-files = $(wildcard src/*js)

And transform the output filenames:

output-files = $(patsubst src/%.js, lib/%.js, $(source-files))

Then we can use the list as a prerequisite:

all: $(output-files)

And so our final makefile looks like:

source-files = $(wildcard src/*js)
output-files = $(patsubst src/%.js, lib/%.js, $(source-files))

all: $(output-files)

lib/%.js: src/%.js
    babel $< -o $@

Extra credit: automatically rebuilding files

Because make doesn’t do anything it doesn’t have to, a really really basic way of remaking things as your change the input files is to run it in a Bash loop:

while true; do
    make
    sleep 1
done

Of course, if you’re not updating anything you’ll spam your terminal with make: Nothing to be done for ‘all’ forever. One alternative is a package called watch-make by (coughs, looks bashful in a british-false-humility way) me. It wraps make, gets the prerequisite files from it, and reruns make when they change:

⟩ wmake

  ⛭ running make
  │ babel src/index.js -o lib/index.js
  ✔︎ make
  ⚲ watching 2 files

  ✏︎ changed src/index.js
  ⛭ running make
  │ babel src/index.js -o lib/index.js
  ✔︎ make
  ⚲ watching 2 files

Make can do a whole lot more

I’ve just scratched the surface here, but this covers way more than most makefiles use in practice. If you’re burning to know more, the GNU Make documentation is exhaustive and very well written, but don’t blame me when you’re up to your elbows in static pattern rules and SECONDEXPANSION.

Before you know it you’ll be writing makefiles that make makefiles, and then no-one can save you.

Iä iä make’file fhtagn.

What about autotools/automake?

No.

DEV Community: Matt Brennan