DEV Community

Hamdi (KHELIL) LION
Hamdi (KHELIL) LION

Posted on

🧰 Bootstrapping a Whole Dev Environment With chezmoi and a Makefile

#automation #cloud #kubernetes #git

If you work as a consultant, you rebuild your environment far more often than you buy a laptop. 😅

New client, new WSL2 distro, new jumpbox, new VM someone spun up for a migration weekend. Every single time: reinstall the toolchain, rebuild the shell, fix the git identity, re-add the forty small tweaks you only notice are missing a week later, mid incident.

chezmoi solves the dotfiles half of that beautifully. The toolchain half needs a bit of design on top of it.

This post is about that design: the patterns that make a one command bootstrap actually survive contact with reality. Every example below is illustrative, but every trap is real. 🚀

📋 What we will cover

✅ Why dotfiles alone are not enough
✅ The bootstrap script, and the flag that stops it running twice
✅ Why installer scripts need numbers in their names
✅ Failure isolation, and how it plays with run_once_
✅ The GitHub API rate limit that breaks a cold bootstrap
✅ Knowing when not to use chezmoi
✅ CI that catches rotted download URLs before a machine does

Everything here was verified against chezmoi v2.72.0.

😤 The problem: dotfiles are the easy half

Copying .bashrc around is a solved problem. That was never what cost a day.

What costs a day is the toolchain. A platform or SRE workstation typically needs something like:

✅ Kubernetes: kubectl, helm, k9s, kubectx, stern, kind, flux or argocd, krew plugins
✅ IaC: terraform or opentofu, terragrunt, helmfile, sops, tflint
✅ Cloud CLIs and their auth plugins
✅ A language toolchain or three, plus a pile of pip and npm tools
✅ Editor extensions and settings

Every one of those has a different install story. Some ship a raw binary, some a tarball, some a zip with a different internal layout, some ship nothing at all and need go install. And the download URLs rot: assets get renamed between releases and your bootstrap breaks months later, on a machine you needed working right now.

👉 chezmoi manages the files. The interesting engineering is making the toolchain install reliably and repeatably.

🗂️ The shape of the repo

A layout that works well:

.chezmoi.toml.tmpl          prompts for the few values that really vary
.chezmoiignore              repo-only files, never written to $HOME
.chezmoiscripts/
  run_once_10_install_base.sh
  run_once_20_install_lang.sh
  run_once_40_install_k8s.sh
  run_once_50_install_iac.sh
scripts/lib.sh              shared helpers: try/finish, gh_latest, fetch_*
dot_bashrc.tmpl             -> ~/.bashrc
dot_gitconfig.tmpl          -> ~/.gitconfig
dot_config/                 -> ~/.config/...
private_dot_ssh/config      -> ~/.ssh/config  (0600)
setup.sh                    one command bootstrap for a new machine
Makefile                    the interface: bootstrap / apply / diff / update
Enter fullscreen mode Exit fullscreen mode

Two things worth noticing. Files in .chezmoiscripts/ are executed but never create a matching directory in your home dir. And private_ strips group and world permissions on the target, which is how ~/.ssh/config lands at 0600 without any extra step.

🚀 The bootstrap, and the flag that matters

A setup.sh for a fresh machine does three things: install the chezmoi binary, apply the dotfiles, install the toolchain.

The interesting line is the second one:

chezmoi init "$REPO" --apply --exclude=scripts
Enter fullscreen mode Exit fullscreen mode

That --exclude=scripts is not cosmetic.

Without it, --apply runs every installer. Then step three runs the toolchain install and runs them all again. Excluding scripts at apply time and installing separately means the toolchain is built exactly once.

--exclude takes entry types, and scripts is one of them. It is the cleanest way to say "give me the files, hold the side effects". The same flag is worth knowing for chezmoi apply --exclude=encrypted on a machine that has no key yet.

🔢 Why installer scripts need numbers

chezmoi executes run_ scripts in alphabetical order. That single fact drives the naming:

run_once_10_install_base.sh
run_once_20_install_lang.sh
run_once_40_install_k8s.sh
run_once_45_install_krew.sh
run_once_50_install_helmrepos.sh
Enter fullscreen mode Exit fullscreen mode

Without the NN_ prefix, helmrepos sorts before k8s and tries to register chart repositories before helm exists. Same for krew before kubectl, and every go install before the Go toolchain is there.

It is worth enforcing the convention in CI so an unordered script cannot sneak in:

for f in .chezmoiscripts/run_once_*.sh; do
  case "$(basename "$f")" in
    run_once_[0-9][0-9]_install_*.sh) ;;
    *) echo "bad name (needs a NN_ order prefix): $f"; fail=1 ;;
  esac
done
Enter fullscreen mode Exit fullscreen mode

A quick recap of the prefixes while we are here:

✅ run_ runs on every apply
✅ run_once_ runs once per unique script content, tracked by SHA256 in chezmoi's state
✅ run_onchange_ runs whenever the content changes
✅ before_ and after_ control ordering relative to file updates

Note that run_once_ keys on contents, not filename. Reformat the script and it runs again. Rename it and it does not.

🛡️ Failure isolation, and why it pairs with run_once_

Early versions of this kind of script have a nasty failure mode: one dead download URL kills the whole group. The Kubernetes installer dies on tool number seven and you silently lose everything after it.

So wrap every tool:

_FAILED=()

try() {
  local label="$1"; shift
  echo; echo "==> $label"
  if ( set -e; "$@" ); then
    return 0
  fi
  echo "  !! $label FAILED, continuing"
  _FAILED+=("$label")
  return 0
}

finish() {
  if [ ${#_FAILED[@]} -gt 0 ]; then
    echo "==> ${#_FAILED[@]} step(s) FAILED: ${_FAILED[*]}"
    exit 1
  fi
  echo "==> all steps completed"
}
Enter fullscreen mode Exit fullscreen mode

The subshell with set -e isolates each tool, so a failure gets recorded rather than being fatal. The group runs to completion, then finish exits non zero.

That non zero exit is the clever part. 🔁

chezmoi only records a run_once_ script as done when it succeeds. A failing group is never marked complete, so the next chezmoi apply retries it automatically. Transient network failures heal themselves without you doing anything.

Usage stays flat and readable:

try "kubectl"  install_kubectl
try "helm"     install_helm
try "k9s"      fetch_tarball_bin "$(gh_latest derailed/k9s 'Linux_amd64.tar.gz')" k9s k9s
try "stern"    fetch_tarball_bin "$(gh_latest stern/stern 'linux_amd64.tar.gz')" stern stern
try "kind"     fetch_raw_bin "$(gh_latest kubernetes-sigs/kind 'kind-linux-amd64$')" kind
finish
Enter fullscreen mode Exit fullscreen mode

👉 Match release assets by regex, not by a pinned filename. When upstream renames a file, a loose pattern often still matches and your bootstrap survives.

🔑 The rate limit that breaks a cold bootstrap

This one is easy to miss and painful to debug.

The anonymous GitHub API limit is 60 requests per hour. A full bootstrap that resolves a few dozen release assets blows straight through it. Halfway down the list, every remaining lookup returns nothing and tools start failing for no visible reason. There is no obvious error, just empty URLs.

Authenticate whenever a token is available:

_gh_curl() {
  local token="${GH_TOKEN:-${GITHUB_TOKEN:-}}"
  [ -z "$token" ] && command -v gh >/dev/null && token=$(gh auth token 2>/dev/null || true)
  if [ -n "$token" ]; then
    curl -fsSL -H "Authorization: Bearer $token" "$@"
  else
    curl -fsSL "$@"
  fi
}

gh_latest() {
  _gh_curl "https://api.github.com/repos/$1/releases/latest" \
    | jq -r ".assets[] | select(.name | test(\"$2\")) | .browser_download_url" \
    | head -1
}
Enter fullscreen mode Exit fullscreen mode

The gh auth token fallback is nice because the GitHub CLI is usually already authenticated on a developer machine.

One more variant earns its keep. Some projects tag asset-less release candidates as latest, so releases/latest hands you a release with nothing to download. Scan recent releases instead:

gh_latest_recent() {
  _gh_curl "https://api.github.com/repos/$1/releases?per_page=30" \
    | jq -r "[.[] | select(.draft|not) | .assets[]? | select(.name | test(\"$2\")) | .browser_download_url] | first // empty"
}
Enter fullscreen mode Exit fullscreen mode

🪟 Knowing when not to use chezmoi

chezmoi only writes inside $HOME. That is a deliberate safety property, not a limitation to work around.

The classic case is VS Code under WSL. The editor runs on Windows and only its server lives in the distro, so the User directory that actually gets read sits under /mnt/c/Users/<you>/AppData/Roaming/Code/User/. That is outside $HOME, so chezmoi is the wrong tool for it.

The clean answer is to keep those files in the repo but tell chezmoi to leave them alone:

# .chezmoiignore
README.md
setup.sh
Makefile
.github
vscode
Enter fullscreen mode Exit fullscreen mode

Then let a run_once_ script copy them wherever they actually belong, backing up whatever was there first and making a second run a no-op.

Remember that .chezmoiignore is itself a template, and that the logic is inverted from what your brain wants. chezmoi installs everything by default, so you write "ignore this unless" rather than "install this if":

{{- if ne .chezmoi.os "darwin" }}
Library/Application Support/SomeApp/config.json
{{- end }}
Enter fullscreen mode Exit fullscreen mode

Check your work with chezmoi ignored.

❓ Ask only the questions that matter

Keep the init prompts small. Most things do not actually vary:

[data]
  git_name        = "{{ promptStringOnce . "git_name" "Git full name" "Your Name" }}"
  git_email       = "{{ promptStringOnce . "git_email" "Git email" "you@example.com" }}"
  git_signing_key = "{{ promptStringOnce . "git_signing_key" "GPG signing key ID" "" }}"
Enter fullscreen mode Exit fullscreen mode

promptStringOnce only asks if the value is not already stored, and the fourth argument is the default.

For per client or per project identity, resist the urge to prompt. Git already solves it with conditional includes, and the per client file can live next to the projects instead of in your dotfiles:

[includeIf "gitdir:~/workspace/clients/acme/**"]
    path = ~/workspace/clients/acme/.gitconfig
Enter fullscreen mode Exit fullscreen mode

👉 Switching client context becomes just cd, and no client name ever enters the dotfiles repo. That matters when the repo might one day be shared or published. 🔒

🧪 CI that catches rot before a machine does

This is the part that pays off most and gets built least.

Run it on every push and on a weekly schedule, because upstream assets get renamed on their timetable, not yours:

✅ bash -n and shellcheck on every script
✅ Enforce the NN_ ordering convention
✅ Render every dotfile and assert the expected files exist
✅ Resolve every release asset pattern and ping the non-GitHub endpoints
✅ Run the full install on a clean runner, then assert the binaries are on PATH
✅ Run it a second time and assert it is idempotent

The render job is easy because chezmoi makes it easy. You can apply the whole source state somewhere harmless:

mkdir -p /tmp/dest /tmp/cfg
cat > /tmp/cfg/chezmoi.toml <<'EOF'
[data]
  git_name = "CI Test"
  git_email = "ci@example.com"
  git_signing_key = "0000000000000000"
EOF

chezmoi -S . -D /tmp/dest --config /tmp/cfg/chezmoi.toml \
  apply --exclude=scripts --force
Enter fullscreen mode Exit fullscreen mode

-S sets the source directory and -D the destination. Point both at scratch paths and you can assert on the rendered output without touching a real home directory. It is also a good place to check that sensitive files land at the right permissions and that no private keys are tracked.

The weekly run is the whole point. An asset gets renamed, CI goes red on Monday morning, you fix it while nothing is on fire. The alternative is discovering it during a bootstrap on a client machine. 🔥

🔁 The daily interface

Wrap it in a Makefile and you will rarely type a raw chezmoi command again:

apply:   ## Apply the dotfiles
    chezmoi apply --no-pager

diff:    ## Show what apply would change, writing nothing
    chezmoi diff --no-pager

update:  ## Pull the latest dotfiles and bump every tool
    chezmoi update
    @bash scripts/update.sh
Enter fullscreen mode Exit fullscreen mode

Because every installer is re-runnable, "update all my tools" is just "run them all again". Keep each group individually addressable too, so make install-k8s rebuilds only what you need.

And when you tweak something directly on a machine:

chezmoi re-add
Enter fullscreen mode Exit fullscreen mode

pulls the local edits back into the source state. 🙂

Teach chezmoi diff before chezmoi apply. It is the single thing that makes people trust the tool.

🧠 Traps worth knowing

✅ --apply runs your scripts. If you also install separately, use --exclude=scripts or everything runs twice
✅ Scripts run alphabetically, so ordering is a naming problem, not a config problem
✅ A group that exits non zero is not marked done, which is a feature: it retries. Make sure your success path really exits zero
✅ Do not set RETURN traps inside helper functions. A RETURN trap is not function local, so it fires again when the calling wrapper returns, and set -u turns your cleanup into a failure
✅ The anonymous GitHub API limit will bite a cold bootstrap. Authenticate
✅ Scripts need a #! line, but you do not need to set the executable bit
✅ A script template that renders to only whitespace is not executed at all, which is a clean way to disable one per platform
✅ Scripts break chezmoi's declarative model. Every time you can express something as a managed file instead, the result is better

🧭 Wrapping Up

chezmoi is a small tool and you can learn the useful part of it in an afternoon.

The leverage does not come from the tool though. It comes from treating your environment as software: a Makefile as the interface, failure isolation in the installers, and CI that runs the whole thing on a clean machine every week so it cannot silently rot.

The payoff is not really the minutes saved either. It is that rebuilding a machine stops being a small project you put off, so you stop working in a half configured shell for a week every time you change context.

👉 Your environment is infrastructure. Version it, test it, and make rebuilding it boring.

Happy bootstrapping and stay safe! 🚀

Top comments (0)