Makefiles for automation and better dev-UI

As our projects grow in complexity, the list of tools they require also grows, and remembering how to use all of them (with their different syntaxes) can become cumbersome. On top of that, some tasks require multiple steps and tools to be run sequentially. We can create our own aliases and scripts for multi-step tasks, but the downside is remembering all those aliases and scripts.

Wouldn't it be nice if we could have a single CLI entry point to our project that would give us a way to list and run all the tools or tasks it contains? Something consistent across all projects, no matter the technology? Something like this?

Well, turns out we can have this with the help of make. Make is a tool primarily used for automation of code compilation but it can be used for all sorts of automation as we'll see. So let's see it in action.

Building your makefile

make executes a makefile that contains a list of make targets. So let’s start with something simple. Here is a very basic structure of a makefile target:

name: 
    task

It consists of a target name and a task associated with it.

Let’s write a real make target we can use to run our unit tests:

unit-tests:
    vendor/bin/phpspec

If we would run this target with make unit-tests it would run our phpspec tests (same as if we typed vendor/bin/phpspec).

Suppose we have another target for our functional tests:

functional-tests:
    vendor/bin/behat

Now we can use make functional-tests to run our functional tests.

But what if we want to run all our tests? Well, make targets can have a list of dependencies, targets that need to be run before it:

name: dependency dependency

In our case, we can create a new target and list our test suite targets as its dependencies:

tests: unit-test functional-test

Now, when we type: make tests that will run our unit-test and functional-test targets.

And what about those multiple steps tasks? Well, make targets can have more than one task associated with it:

name:
    task
    task
    task

So let’s create a make target to set up our (simple) project:

build-project:
    docker-compose up -d
    docker-compose exec app-comp composer install
    docker-compose exec app-php php artisan migrate

Advantages of using good makefiles

A good makefile provides us with some extra advantages:

• we have a list of aliases for all the tools/tasks we use on a project that is shared between developers (and can be shared across projects also)
• we have a single CLI entry point for our app so no more scouring documentation to find the name/syntax of some tool
• developers don’t need to know the syntax of all tools used on a project— e.g. if you are a backend developer and setting up your project locally requires the usage of some frontend tools, having that wrapped into a make target would be helpful
• multi-step tasks are automated and require a single command to be run

For clarity, using makefile targets doesn’t mean we never touch the underlying tools. We can still use those, especially when we need to run them with some specific configuration. Using make simply gives you a fast way to run them in standard configuration.

Wrap up

Now, in the beginning, I promised a nice Command Line Interface that will give us a list of all make targets that we can run. Well, this doesn’t come with makefiles out of the box, but with the help of a designated help target, makefile comments, and some bash magic we will have it working in no time.

First, we add comments to our make targets:

tests: test-unit test-functional ## Run all tests on the project

As you guessed, comments are prefixed with ##.

Next, we create a special help target that will render a list of our targets with comments as descriptions. There are different ways to achieve this, and you can look for some of them here. I will use the one from user muhmi:

help: ## This help dialog.
    @IFS=$$'\n' ; \
    help_lines=(`fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##/:/'`); \
    printf "%-30s %s\n" "target" "help" ; \
    printf "%-30s %s\n" "------" "----" ; \
    for help_line in $${help_lines[@]}; do \
        IFS=$$':' ; \
        help_split=($$help_line) ; \
        help_command=`echo $${help_split[0]} | sed -e 's/^ *//' -e 's/ *$$//'` ; \
        help_info=`echo $${help_split[2]} | sed -e 's/^ *//' -e 's/ *$$//'` ; \
        printf '\033[36m'; \
        printf "%-30s %s" $$help_command ; \
        printf '\033[0m'; \
        printf "%s\n" $$help_info; \
    donev

As you see, there is some bash wizardry here that I won’t pretend to understand fully, but it gets the job done and if you’re a bash wizard yourself you can make your own cool help target.

We also need to declare this help target as a default target so it would be run by default (if we run make without specifying any target’s name).

.DEFAULT_GOAL := help

Finally, we need to add a special line to bypass some traditional behavior of make. Since make is used for compiling files, if there was a file named “tests” in the folder our command wouldn’t run. This is avoided by using declaring the command as phony. “A phony target is one that is not really the name of a file.” By adding the .PHONY line and declaring all of our commands, this conflict will never occur.

Now our makefile is ready so let’s see how it looks:

.DEFAULT_GOAL := help
.PHONY: help build-project unit-tests functional-tests tests start stop
help: ## This help dialog.
    @echo "Hello to the AwesomeProject\n"
    @IFS=$$'\n' ; \
    help_lines=(`fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##/:/'`); \
    printf "%-30s %s\n" "target" "help" ; \
    printf "%-30s %s\n" "------" "----" ; \
    for help_line in $${help_lines[@]}; do \
        IFS=$$':' ; \
        help_split=($$help_line) ; \
        help_command=`echo $${help_split[0]} | sed -e 's/^ *//' -e 's/ *$$//'` ; \
        help_info=`echo $${help_split[2]} | sed -e 's/^ *//' -e 's/ *$$//'` ; \
        printf '\033[36m'; \
        printf "%-30s %s" $$help_command ; \
        printf '\033[0m'; \
        printf "%s\n" $$help_info; \
    done
build-project: ## Build our project
    docker-compose up -d
    docker-compose exec app-comp composer install
    docker-compose exec app-php php artisan migrate
start: ## Start project containers
    docker-compose up -d
stop: ## Stop project containers
    docker-compose down
unit-tests: ## Run unit tests
    vendor/bin/phpspec
functional-tests: ## Run functional tests
    vendor/bin/behat
tests: unit-test functional-test ## Run all tests.DEFAULT_GOAL := help

If we would run make in a folder containing the makefile we just built, we would get our nice list of targets with their descriptions:

Hello to the AwesomeProject
target                         help
------                         ----
help                           This help dialog.
build-project                  Build our project
start                          Start project containers
stop                           Start project containers
unit-tests                     Run unit tests
functional-tests               Run functional tests
tests                          Run all tests

---

So what are you waiting for? Go supercharge your project with make. It is an easy way to create a consistent command interface so that ramp up time on projects is kept to a minimum.

Comments

[Vlastimil Pospichal]

I use Makefile similarly, but for some actions I chose git aliases - especially for $ PWD sensitive actions.
This Makefile seems too complex to me. I would extract help as a global Bash script for all projects.

help: ## This help dialog.
    @parse_makefile.sh $(MAKEFILE_LIST) "Hello to the AwesomeProject"

[konrad_126]

That's a great idea, thnx :)

[Vlastimil Pospichal]

$ cat parse_makefile.sh

#!/bin/bash
echo "Hello to the $1"
printf "%-30s %s\n" "target" "help"
printf "%-30s %s\n" "------" "----"
sed -ne 's/:.*## */:/p' Makefile |
    while IFS=':' read command info; do
            printf "\033[36m%-30s \033[0m%s\n"  "$command" "$info"
    done