DEV Community

Discussion on: How to become more productive using Makefile

Collapse
 
stefanjarina profile image
Štefan Jarina • Edited

Hi, yeah, the example is a little bit too simplistic, however with a bit more involved example, one can start seeing the benefits:

include Makefile.settings

.PHONY: init build clean publish log jenkins slave

# Jenkins settings
export DOCKER_GID ?= 100
export JENKINS_USERNAME ?= admin
export JENKINS_PASSWORD ?= password
export JENKINS_SLAVE_VERSION ?= 2.2
export JENKINS_SLAVE_LABELS ?= DOCKER

# AWS settings
# The role to assume to inject temporary credentials into your Jenkins container
AWS_ROLE ?= `aws configure get role_arn`
# KMS encrypted password - the temporary credentials must possess kms:decrypt permissions for the key used to encrypt the credentials
export KMS_JENKINS_PASSWORD ?=

init:
    ${INFO} "Creating volumes..."
    @ docker volume create --name=jenkins_home

build:
    ${INFO} "Building image..."
    @ docker-compose build --pull
    ${INFO} "Build complete"

jenkins: init
    @ $(if $(and $(AWS_PROFILE),$(KMS_JENKINS_PASSWORD)),$(call assume_role,$(AWS_ROLE)),)
    ${INFO} "Starting Jenkins..."
    ${INFO} "This may take some time..."
    @ docker-compose up -d jenkins
    @ $(call check_service_health,$(RELEASE_ARGS),jenkins)
    ${INFO} "Jenkins is running at http://$(DOCKER_HOST_IP):$(call get_port_mapping,jenkins,8080)..."

publish:
    ${INFO} "Publishing images..."
    @ docker-compose push
    ${INFO} "Publish complete"

slave:
    ${INFO} "Checking Jenkins is healthy..."
    @ $(if $(and $(AWS_PROFILE),$(KMS_JENKINS_PASSWORD)),$(call assume_role,$(AWS_ROLE)),)
    @ $(call check_service_health,$(RELEASE_ARGS),jenkins)
    ${INFO} "Starting $(SLAVE_COUNT) slave(s)..."
    @ docker-compose up -d --scale jenkins-slave=$(SLAVE_COUNT)
    ${INFO} "$(SLAVE_COUNT) slave(s) running"

clean:
    ${INFO} "Stopping services..."
    @ docker-compose down -v || true
    ${INFO} "Services stopped"

destroy: clean
    ${INFO} "Deleting jenkins home volume..."
    @ docker volume rm -f jenkins_home
    ${INFO} "Deletion complete"

log:
    ${INFO} "Streaming Jenkins logs - press CTRL+C to exit..."
    @ docker-compose logs -f jenkins
Enter fullscreen mode Exit fullscreen mode

what I see is that makefiles are less used with some languages, but are still quite heavily used with others.
Though I agree that I also haven't seen complex makefile in a while, but that is probably because more and more languages have their own tooling.

  • JS: npm scripts, gulp, grunt, etc.
  • C#: Powershell (Invoke!), maybe Cake
  • F#: FAKE
  • Rust: cargo custom tasks
  • JVM languages had their own for a long time also: sbt (Scala), gradle (a lot of JVM langs)
  • Python: Fabric, but here I've seen a lot of makefiles :-)
  • Ruby: Rake
  • Elixir: mix with custom tasks
  • Go: maybe Task, but here I've also seen quite a lot of makefiles

Can't tell for the rest, these are the ones I am familiar with.

Collapse
 
xarala221 profile image
Ousseynou Diop

Great content, Thank you.

Collapse
 
mintypt profile image
mintyPT

What do you have inside your Makefile.settings?

Thread Thread
 
stefanjarina profile image
Štefan Jarina

Hi,

The code in this file is probably a bit more involved and harder to decipher, but it is just parsing functions really.
To get useful data from docker and/or env variables

Makefile.settings:

YELLOW := "\e[1;33m"
NC := "\e[0m"
INFO := @bash -c 'printf $(YELLOW); echo "=> $$1"; printf $(NC)' MESSAGE
ARGS := $(wordlist 2,$(words $(MAKECMDGOALS)),$(MAKECMDGOALS))

SHELL = bash

# Slave arguments
ifeq ($(firstword $(MAKECMDGOALS)),$(filter $(firstword $(MAKECMDGOALS)),slave))
  SLAVE_ARGS := $(wordlist 2,$(words $(MAKECMDGOALS)),$(MAKECMDGOALS))
  SLAVE_COUNT = $(if $(SLAVE_ARGS),$(firstword $(SLAVE_ARGS)),1)
endif

# Docker host settings
DOCKER_HOST_IP := $(shell echo $$DOCKER_HOST | awk -F/ '{printf $$3}' | awk -F: '{printf $$1}')
DOCKER_HOST_IP := $(if $(DOCKER_HOST_IP),$(DOCKER_HOST_IP),localhost)

# Image and Repository Tag introspection functions
# Syntax: $(call get_image_id,<docker-compose-environment>,<service-name>)
# Syntax: $(call get_repo_tags,<docker-compose-environment>,<service-name>,<fully-qualified-image-name>)
get_container_id = $$(docker-compose $(1) ps -q $(2))
get_image_id = $$(echo $(call get_container_id,$(1),$(2)) | xargs -I ARGS docker inspect -f '{{ .Image }}' ARGS)
get_container_state = $$(echo $(call get_container_id,$(1),$(2)) | xargs -I ID docker inspect -f '$(3)' ID)
filter_repo_tags = $(if $(findstring docker.io,$(1)),$(subst docker.io/,,$(1))[^[:space:]|\$$]*,$(1)[^[:space:]|\$$]*)
get_repo_tags = $$(echo $(call get_image_id,$(1),$(2)) | xargs -I ID docker inspect -f '{{range .RepoTags}}{{.}} {{end}}' ID | grep -oh "$(call filter_repo_tags,$(3))" | xargs)

# Port introspection functions
# Syntax: $(call get_port_mapping,<service-name>,<internal-port>)
get_raw_port_mapping = $$(docker-compose ps -q $(1) | xargs -I ID docker port ID $(2))
get_port_mapping = $$(echo $$(IFS=':' read -r -a array <<< "$(call get_raw_port_mapping,$(1),$(2))" && echo "$${array[1]}"))

# Service health functions
# Syntax: $(call check_service_health,<docker-compose-environment>,<service-name>)
get_service_health = $$(echo $(call get_container_state,$(1),$(2),{{if .State.Running}}{{ .State.Health.Status }}{{end}}))
check_service_health = { \
  until [[ $(call get_service_health,$(1),$(2)) != starting ]]; \
    do sleep 1; \
  done; \
  if [[ $(call get_service_health,$(1),$(2)) != healthy ]]; \
    then echo $(2) failed health check; exit 1; \
  fi; \
}

# AWS assume role settings
# Attempts to assume IAM role using STS
# Syntax: $(call assume_role,<role-arn>)
get_assume_session = aws sts assume-role --role-arn=$(1) --role-session-name=admin
get_assume_credential = jq --null-input '$(1)' | jq .Credentials.$(2) -r
define assume_role
    $(eval AWS_SESSION = $(shell $(call get_assume_session,$(1))))
    $(eval export AWS_ACCESS_KEY_ID = $(shell $(call get_assume_credential,$(AWS_SESSION),AccessKeyId)))
    $(eval export AWS_SECRET_ACCESS_KEY = $(shell $(call get_assume_credential,$(AWS_SESSION),SecretAccessKey)))
    $(eval export AWS_SESSION_TOKEN = $(shell $(call get_assume_credential,$(AWS_SESSION),SessionToken)))
endef
Enter fullscreen mode Exit fullscreen mode
Thread Thread
 
mintypt profile image
mintyPT

Just the first 3 lines are worth it

Collapse
 
stefanjarina profile image
Štefan Jarina • Edited

Small updates:

PYTHON:

Fabric was used for this in it's 1.x version.

Since version 2.0 Fabric was split essentially into 2 packages with separate functionalities:

  • Fabric 2.0 - now used only for remote shell/commands using ssh
  • Invoke! - task execution tool & library extracted from Fabric 1.x into it's own library

GENERIC:
There is now another tool, that is generic and can be used for many languages, it is even cross-platform

Haven't used it though, in most cases I go with language native solution and if there is none, I usually either:

  • still write Makefile if only linux/mac/docker/kubernetes is needed
  • use Powershell 7 if I need it to run on MAC/Linux/Windows