Originally published on kuryzhev.cloud
Makefile infra repo troubleshooting usually starts the same way: make apply exits with status 0, the console output looks clean, and Terraform never actually ran. Nobody touched the pipeline config. Nobody changed the Makefile on purpose. The target just stopped doing anything, and the only clue is a suspiciously fast CI job.
This failure mode shows up constantly in infra repos that wrap Terraform, Docker, or kubectl behind a Makefile, and it goes undiagnosed more often than it should. Make isn't broken — it's doing exactly what its documentation says it will do. The trouble is that most people who write Makefiles today are applying a shell-script mental model to a tool built in the 1970s to decide, based on file timestamps, whether a C source file needed recompiling.
Symptoms
Before touching the Makefile, confirm the pattern matches. The typical signature:
-
make <target>exits 0 but produces no visible effect — no Terraform plan, no Docker build, no output. - A target runs "sometimes" — works after
git clean, fails after a fresh checkout, or vice versa. - CI logs show a full environment dump or secret values that were never explicitly echoed.
-
make helpprints nothing, prints garbage, or lists targets with no descriptions. - A recipe visibly runs two of five commands, then silently moves on — no error, no red text, next target starts anyway.
There's a CI-specific variant worth calling out separately: the target works fine on a developer's laptop but hangs or fails only inside the pipeline. That usually points to a missing bash binary in a minimal container image, a different default SHELL, or a recipe that expects an interactive TTY for a confirmation prompt CI can't provide.
The cost tell is easy to miss: if terraform init or docker build re-runs on every single invocation instead of only when inputs change, that's not a performance quirk — it's CI minutes being burned on work that shouldn't happen at all.
Root cause
Make was built to compile C programs where targets map to output files, and it decides whether to run a recipe by comparing file timestamps. That model leaks into infra repos in three specific ways.
First: Make checks whether a file or directory on disk already matches the target name. If a directory named build, apply, or deploy exists in the repo — common after a stray terraform apply log capture or a leftover build artifact — Make treats the target as up to date and skips it entirely, unless that target is declared .PHONY. No error. No warning. Just silence.
Second: every line in a recipe runs in its own subshell by default. A cd some-dir on line one does not carry over to line two. An export VAR=x set in one line is invisible to the next. This is documented behavior in the GNU Make manual, but it's exactly the assumption most people bring in from writing shell scripts.
Third: the default SHELL on many Linux distributions and CI base images is dash, not bash. Features like set -o pipefail, [[ ]] conditionals, and arrays either error out or silently behave differently depending on which shell actually executes the recipe. Watch out for this one specifically: a Makefile that works on a developer's macOS machine, where bash is the default in most setups, can misbehave inside an Alpine-based CI runner where /bin/sh is dash.
Fix #1: Harden the Makefile shell contract
The fastest way to stop silent skips and partial failures is to make Make fail loudly and consistently, regardless of which machine or CI image runs it.
Pin the shell explicitly and force fail-fast flags:
SHELL := /usr/bin/env bash
.SHELLFLAGS := -eu -o pipefail -c
-e stops the recipe on the first failing command instead of continuing past it. -u errors on unset variables instead of silently expanding them to empty strings. -o pipefail makes a failing command inside a pipe actually fail the recipe, instead of only the last command's exit code mattering.
Declare every logical target .PHONY — this single fix resolves the "target does nothing" symptom described above:
.PHONY: help init plan apply destroy fmt validate clean
Set .DEFAULT_GOAL := help so a bare make invocation never accidentally triggers the first target in the file — which, in many infra Makefiles, is something destructive like init or apply.
Add MAKEFLAGS += --warn-undefined-variables to surface typo'd variable references at parse time rather than letting them expand to nothing. If a recipe genuinely needs shared shell state across multiple lines — loops, conditionals — GNU Make 3.82+ supports ONESHELL:, which runs the whole recipe body in a single shell invocation. That changes how errors propagate partway through the block, so test it deliberately rather than assuming it's a drop-in replacement.
Fix #2: Guard required env vars and secrets
A Makefile that runs terraform apply without checking AWS_PROFILE or TF_VAR_env first is one missing export away from applying against the wrong account with default values. Fail before that happens, not after.
ifndef AWS_PROFILE
$(error AWS_PROFILE is not set — export it before running make)
endif
This check runs at parse time, before any target executes, so a missing profile stops the whole invocation immediately with a readable message instead of a cryptic AWS error three steps into a plan.
Gotcha worth flagging on its own: include .env at the top of a Makefile does not export those variables into recipe subshells — it only makes them available to Make's own variable expansion. If a recipe calls terraform or aws directly and expects .env values as real environment variables, load them inside the recipe instead:
deploy:
set -a; source .env; set +a; terraform apply -auto-approve
Never let a recipe echo secrets. Prefix sensitive lines with @ to suppress command echoing, and treat CI secret-masking as the actual security control — a Makefile convention alone won't stop a stray env or printenv call from dumping credentials into a CI log that a teammate can read later.
Fix #3: Stop redundant re-runs with stamp files (or switch runners)
If terraform init or a Docker image build re-runs on every single pipeline invocation, that's wasted CI minutes on unchanged inputs. Stamp files fix this by giving Make a real file to check timestamps against:
.terraform-init.stamp: .terraform.lock.hcl
terraform init -input=false
@touch .terraform-init.stamp
init: .terraform-init.stamp
Make only re-runs terraform init when the lockfile changes. For multi-module infra repos, order-only prerequisites (target: | dependency) prevent unrelated modules from triggering cascading rebuilds just because they share a dependency chain.
At some point, though, the workaround comments in a Makefile start outnumbering the actual logic — nested $(shell ...) calls doing conditional branching, multi-line escaping for loops that would be three lines in a real script. That's the signal to evaluate a purpose-built task runner instead of adding another patch.
version: '3'
tasks:
init:
desc: Initialize Terraform providers/modules
status:
- test -f .terraform-init.stamp # skips if stamp exists and is current
cmds:
- terraform init -input=false
- touch .terraform-init.stamp
plan:
desc: Show planned infra changes
deps: [init]
cmds:
- terraform plan -input=false -out=tfplan
Task and Just drop the file-timestamp model entirely, support cross-platform execution including Windows, and handle dotenv loading natively. The decision criterion is straightforward: stay with Make if the repo is small, Unix-only, and the team already reads Make comfortably. Migrate once the repo needs native Windows support, YAML readability for non-DevOps contributors, or built-in templating without shell hacks.
Prevention
A hardened Makefile stops rotting only when the checks that fixed it today get enforced automatically tomorrow. Add checkmake as a pre-merge CI gate — it flags missing .PHONY declarations, undocumented targets, and duplicate target names before they reach main. For the shell logic inside recipes, run shellcheck against extracted recipe bodies; it catches the same class of quoting and subshell mistakes that cause the silent partial failures described earlier.
Keep the self-documenting ## comment pattern honest by testing it in CI, not just eyeballing it locally:
help: ## Self-documenting target list
@awk 'BEGIN {FS = ":.*##"} /^[a-zA-Z_-]+:.*##/ {printf " %-15s %s\n", $$1, $$2}' $(MAKEFILE_LIST)
If a new target skips the ## convention, make help silently drops it from the list — add a CI assertion that the output is non-empty and includes every .PHONY target defined.
Document required environment variables, minimum Make version, and any use of ONESHELL: or .RECIPEPREFIX in the repo README. Engineers onboarding onto the repo who don't know a Makefile behaves differently than a shell script are exactly who reintroduces the subshell bugs this troubleshooting exercise just fixed. Revisit the Make-versus-task-runner decision periodically — once conditional logic starts living inside $(shell ...) calls rather than in the recipes themselves, that's a sign the underlying problem is being patched rather than resolved. For broader context on keeping pipelines predictable as repos grow, see the CI/CD reliability notes on kuryzhev.cloud before committing to a migration.
Top comments (0)