Discussion on: How to become more productive using Makefile

View post

Replies for: But you don’t really save time though since the commands are short already. I question the productivity improvements here and would use bash aliase...

Hi, yeah, the example is a little bit too simplistic, however with a bit more involved example, one can start seeing the benefits:

include Makefile.settings

.PHONY: init build clean publish log jenkins slave

# Jenkins settings
export DOCKER_GID ?= 100
export JENKINS_USERNAME ?= admin
export JENKINS_PASSWORD ?= password
export JENKINS_SLAVE_VERSION ?= 2.2
export JENKINS_SLAVE_LABELS ?= DOCKER

# AWS settings
# The role to assume to inject temporary credentials into your Jenkins container
AWS_ROLE ?= `aws configure get role_arn`
# KMS encrypted password - the temporary credentials must possess kms:decrypt permissions for the key used to encrypt the credentials
export KMS_JENKINS_PASSWORD ?=

init:
    ${INFO} "Creating volumes..."
    @ docker volume create --name=jenkins_home

build:
    ${INFO} "Building image..."
    @ docker-compose build --pull
    ${INFO} "Build complete"

jenkins: init
    @ $(if $(and $(AWS_PROFILE),$(KMS_JENKINS_PASSWORD)),$(call assume_role,$(AWS_ROLE)),)
    ${INFO} "Starting Jenkins..."
    ${INFO} "This may take some time..."
    @ docker-compose up -d jenkins
    @ $(call check_service_health,$(RELEASE_ARGS),jenkins)
    ${INFO} "Jenkins is running at http://$(DOCKER_HOST_IP):$(call get_port_mapping,jenkins,8080)..."

publish:
    ${INFO} "Publishing images..."
    @ docker-compose push
    ${INFO} "Publish complete"

slave:
    ${INFO} "Checking Jenkins is healthy..."
    @ $(if $(and $(AWS_PROFILE),$(KMS_JENKINS_PASSWORD)),$(call assume_role,$(AWS_ROLE)),)
    @ $(call check_service_health,$(RELEASE_ARGS),jenkins)
    ${INFO} "Starting $(SLAVE_COUNT) slave(s)..."
    @ docker-compose up -d --scale jenkins-slave=$(SLAVE_COUNT)
    ${INFO} "$(SLAVE_COUNT) slave(s) running"

clean:
    ${INFO} "Stopping services..."
    @ docker-compose down -v || true
    ${INFO} "Services stopped"

destroy: clean
    ${INFO} "Deleting jenkins home volume..."
    @ docker volume rm -f jenkins_home
    ${INFO} "Deletion complete"

log:
    ${INFO} "Streaming Jenkins logs - press CTRL+C to exit..."
    @ docker-compose logs -f jenkins

what I see is that makefiles are less used with some languages, but are still quite heavily used with others.
Though I agree that I also haven't seen complex makefile in a while, but that is probably because more and more languages have their own tooling.

JS: npm scripts, gulp, grunt, etc.
C#: Powershell (Invoke!), maybe Cake
F#: FAKE
Rust: cargo custom tasks
JVM languages had their own for a long time also: sbt (Scala), gradle (a lot of JVM langs)
Python: Fabric, but here I've seen a lot of makefiles :-)
Ruby: Rake
Elixir: mix with custom tasks
Go: maybe Task, but here I've also seen quite a lot of makefiles

Can't tell for the rest, these are the ones I am familiar with.

Ousseynou Diop • Jun 27 '20

Great content, Thank you.

mintyPT • Feb 9 '21

What do you have inside your Makefile.settings?

Štefan Jarina • Feb 9 '21

Hi,

The code in this file is probably a bit more involved and harder to decipher, but it is just parsing functions really.
To get useful data from docker and/or env variables

Makefile.settings:

YELLOW := "\e[1;33m"
NC := "\e[0m"
INFO := @bash -c 'printf $(YELLOW); echo "=> $$1"; printf $(NC)' MESSAGE
ARGS := $(wordlist 2,$(words $(MAKECMDGOALS)),$(MAKECMDGOALS))

SHELL = bash

# Slave arguments
ifeq ($(firstword $(MAKECMDGOALS)),$(filter $(firstword $(MAKECMDGOALS)),slave))
  SLAVE_ARGS := $(wordlist 2,$(words $(MAKECMDGOALS)),$(MAKECMDGOALS))
  SLAVE_COUNT = $(if $(SLAVE_ARGS),$(firstword $(SLAVE_ARGS)),1)
endif

# Docker host settings
DOCKER_HOST_IP := $(shell echo $$DOCKER_HOST | awk -F/ '{printf $$3}' | awk -F: '{printf $$1}')
DOCKER_HOST_IP := $(if $(DOCKER_HOST_IP),$(DOCKER_HOST_IP),localhost)

# Image and Repository Tag introspection functions
# Syntax: $(call get_image_id,<docker-compose-environment>,<service-name>)
# Syntax: $(call get_repo_tags,<docker-compose-environment>,<service-name>,<fully-qualified-image-name>)
get_container_id = $$(docker-compose $(1) ps -q $(2))
get_image_id = $$(echo $(call get_container_id,$(1),$(2)) | xargs -I ARGS docker inspect -f '{{ .Image }}' ARGS)
get_container_state = $$(echo $(call get_container_id,$(1),$(2)) | xargs -I ID docker inspect -f '$(3)' ID)
filter_repo_tags = $(if $(findstring docker.io,$(1)),$(subst docker.io/,,$(1))[^[:space:]|\$$]*,$(1)[^[:space:]|\$$]*)
get_repo_tags = $$(echo $(call get_image_id,$(1),$(2)) | xargs -I ID docker inspect -f '{{range .RepoTags}}{{.}} {{end}}' ID | grep -oh "$(call filter_repo_tags,$(3))" | xargs)

# Port introspection functions
# Syntax: $(call get_port_mapping,<service-name>,<internal-port>)
get_raw_port_mapping = $$(docker-compose ps -q $(1) | xargs -I ID docker port ID $(2))
get_port_mapping = $$(echo $$(IFS=':' read -r -a array <<< "$(call get_raw_port_mapping,$(1),$(2))" && echo "$${array[1]}"))

# Service health functions
# Syntax: $(call check_service_health,<docker-compose-environment>,<service-name>)
get_service_health = $$(echo $(call get_container_state,$(1),$(2),{{if .State.Running}}{{ .State.Health.Status }}{{end}}))
check_service_health = { \
  until [[ $(call get_service_health,$(1),$(2)) != starting ]]; \
    do sleep 1; \
  done; \
  if [[ $(call get_service_health,$(1),$(2)) != healthy ]]; \
    then echo $(2) failed health check; exit 1; \
  fi; \
}

# AWS assume role settings
# Attempts to assume IAM role using STS
# Syntax: $(call assume_role,<role-arn>)
get_assume_session = aws sts assume-role --role-arn=$(1) --role-session-name=admin
get_assume_credential = jq --null-input '$(1)' | jq .Credentials.$(2) -r
define assume_role
    $(eval AWS_SESSION = $(shell $(call get_assume_session,$(1))))
    $(eval export AWS_ACCESS_KEY_ID = $(shell $(call get_assume_credential,$(AWS_SESSION),AccessKeyId)))
    $(eval export AWS_SECRET_ACCESS_KEY = $(shell $(call get_assume_credential,$(AWS_SESSION),SecretAccessKey)))
    $(eval export AWS_SESSION_TOKEN = $(shell $(call get_assume_credential,$(AWS_SESSION),SessionToken)))
endef

mintyPT • Feb 9 '21

Just the first 3 lines are worth it

Štefan Jarina • Feb 9 '21 • Edited

Small updates:

PYTHON:

Fabric was used for this in it's 1.x version.

Since version 2.0 Fabric was split essentially into 2 packages with separate functionalities:

Fabric 2.0 - now used only for remote shell/commands using ssh
Invoke! - task execution tool & library extracted from Fabric 1.x into it's own library

GENERIC:
There is now another tool, that is generic and can be used for many languages, it is even cross-platform

Bazel: bazel.build/

Haven't used it though, in most cases I go with language native solution and if there is none, I usually either:

still write Makefile if only linux/mac/docker/kubernetes is needed
use Powershell 7 if I need it to run on MAC/Linux/Windows