DEV Community

Cover image for Testable Dotfiles Management: Building Development Environment with Chezmoi
Shunsuke Kitada
Shunsuke Kitada

Posted on • Originally published at shunk031.me

Testable Dotfiles Management: Building Development Environment with Chezmoi

#dotfiles #chezmoi #bash #githubactions

This article explains an approach to dotfiles management that emphasizes testability, using the author's dotfiles repository shunk031/dotfiles as a case study.

GitHub logo shunk031 / dotfiles

💻 My dotfiles powered by chezmoi; Primarily built with Rust-based applications (sheldon/startship/mise etc.)

shunk031's

📂 dotfiles

Snippet install Unit test codecov

zsh-users/zsh tmux/tmux rossmacarthur/sheldon starship/starship

🗿 Overview

This dotfiles repository is managed with chezmoi🏠, a great dotfiles manager The setup scripts are aimed for MacOS, Ubuntu Desktop, and Ubuntu Server. The first two (MacOS/Ubuntu Desktop) include settings for client machines and the latter one (Ubuntu Server) for server machines.

The actual dotfiles exist under the home directory specified in the .chezmoiroot. See .chezmoiroot - chezmoi more detail on the setting.

📥 Setup

To set up the dotfiles run the appropriate snippet in the terminal.

💻 MacOS MacOS

  • Configuration snippet of the Apple Silicon MacOS environment for client macnine:
bash -c "$(curl -fsLS http://shunk031.me/dotfiles/setup.sh)"
Enter fullscreen mode Exit fullscreen mode

🖥️ Ubuntu Ubuntu

  • Configuration snippet of the Ubuntu environment for both client and server machine:
bash -c "$(wget -qO - http://shunk031.me/dotfiles/setup.sh)"
Enter fullscreen mode Exit fullscreen mode

Minimal setup

The following is a minimal setup command to install chezmoi and my dotfiles from the github repository on a new empty machine:

sh -c…

View on GitHub

Introduction

Dotfiles and Dotfiles Repositories

The dotfiles refer to configuration files that start with a "." (dot) such as .bashrc, .vimrc, and .gitconfig. In recent years, dotfiles repositories that manage these files using Git repositories have become widely popular among developers.

GitHub logo webpro / awesome-dotfiles

A curated list of dotfiles resources.

Awesome Dotfiles Awesome

A curated list of dotfiles resources. Inspired by the awesome list thing Note that some articles or tools may look old or old-fashioned, but this usually means they're battle-tested and mature (like dotfiles themselves). Feel free to propose new articles, projects or tools!

Articles

Introductions

Tutorials

Shell startup

Using specific tools

View on GitHub

The dotfiles repositories often function not just as configuration file management tools, but as automated development environment setup tools that include configuration files, installation scripts, and setup scripts. This enables quick and consistent setup on new machines and environments.

The Problem of Untested Scripts

Most setup and installation scripts included in dotfiles repositories are not tested for proper functionality (which is painful). As a result, various problems can occur when setting up in new environments. Script errors, installation failures of some tools due to dependency issues, and script malfunctions due to OS updates may go unnoticed until actually executed. It's extremely stressful when you get a new computer or environment and excitedly start the setup process, only to have errors occur midway through, preventing the environment setup from completing.

This lack of quality assurance results in what should be automated environment construction consuming a lot of time on manual problem-solving and debugging.

My Repository's Approach: Testable Configuration

To solve the above problems, my dotfiles repository builds an architecture that emphasizes testability. Setup scripts are managed as independent files to enable individual testing, quality is ensured through automated testing with Bats, and continuous testing and code coverage measurement are performed in macOS and Ubuntu environments using GitHub Actions.

For managing dotfiles, I've adopted chezmoi. chezmoi is a modern dotfiles management tool with high popularity on GitHub (10,000+⭐️). Written in Go as a single, dependency-free binary, chezmoi is easy to install even on a brand-new, clean environment.

GitHub logo twpayne / chezmoi

Manage your dotfiles across multiple diverse machines, securely.

chezmoi logo chezmoi

GitHub Release

Manage your dotfiles across multiple diverse machines, securely.

chezmoi's documentation is at chezmoi.io.

If you're contributing to chezmoi, then please read the developer guide.

Contributors

Contributor avatars

License

MIT


View on GitHub


Environment setup on a new machine can be executed with the following very simple one-liner using chezmoi's official installer1.

sh -c "$(curl -fsLS get.chezmoi.io)" -- init --apply $GITHUB_USERNAME
Enter fullscreen mode Exit fullscreen mode

Environment-specific settings can be dynamically generated using chezmoi's template functionality based on Go's text/template as follows.

[user]                           # Can be dynamically specified via template functionality
    name = "{{.name}}"           # - User name
    email = "{{.email}}"         # - Email address etc.
{{- if eq .chezmoi.os "darwin"}} # macOS-specific settings
[credential]
    helper = osxkeychain
{{- end}}
Enter fullscreen mode Exit fullscreen mode

https://www.chezmoi.io/user-guide/templating/

In this way, we aim to achieve reliable dotfiles management by ensuring script quality through testable configuration and flexibly managing environment-specific settings through chezmoi's template functionality.

Architecture Design: Testable Configuration

Repository Structure

My repository is broadly divided into three directories: home/, install/, and tests/, managing dotfiles, environment setup scripts, and automated tests independently.

.
├── ...
│
├── home/                   # dotfiles under chezmoi management
│   ├── dot_bashrc          # - deployed as ~/.bashrc
│   ├── dot_vimrc           # - deployed as ~/.vimrc
│   ├── dot_config/         # - deployed as ~/.config/
│   └── .chezmoi.yaml.tmpl  # - chezmoi configuration file
│
├── install/                # setup scripts (testable)
│   ├── common/             # - common installation scripts
│   ├── macos/              # - macOS-specific scripts
│   └── ubuntu/             # - Ubuntu-specific scripts
│
├── tests/                  # automated tests with Bats
│   ├── install/            # - tests for installation scripts
│   └── files/              # - tests for files after chezmoi deployment
│
└── ...
Enter fullscreen mode Exit fullscreen mode

Design Philosophy

The core of this architecture lies in "separation of concerns" and "maximizing testability". Traditional dotfiles repositories mix configuration files and setup scripts, making testing difficult, but this configuration clearly separates each element.

The install/ directory: Easy Unit Testing Through Script Separation

By making setup scripts independent from chezmoi, individual testing becomes possible.

Platform-specific configuration separates OS-specific logic, allowing each to be tested independently. Each script follows the single responsibility principle, handling only the installation of specific tools or packages.

The home/ directory: chezmoi Templates and dotfiles

These are the actual dotfiles under chezmoi management. They follow chezmoi's unique file naming conventions (dot_ prefix etc.) and utilize template functionality. This repository specifies home as the source directory using the .chezmoiroot file2.

It's independent from the scripts in the install/ directory, separating configuration file placement and environment construction.

The tests/ directory: Automated Testing with Bats

I use Bash Automated Testing System (Bats) to test scripts in the install/ directory. The test directories and files are configured to be consistent with the script.

{{< blogcard url="https://github.com/bats-core/bats-core" >}}

Each test file verifies the script's behavior and confirms that expected results (package installation, configuration file generation, etc.) are obtained.

Test & CI/CD Strategy

This repository adopts a test strategy based on the fundamental policy of "continuous verification". We can verify that scripts in the install/ directory work correctly in various environments and discover problems in advance to prevent failures during actual environment construction.

Unit Test Implementation with Bats

My repository adopts Bash Automated Testing System (Bats) to verify the behavior of each installation script. Bats is a testing framework specifically for shell scripts that allows writing tests with simple syntax.

#!/usr/bin/env bats

@test "brew installation script exists" {
  [ -f "install/macos/common/brew.sh" ]
}

@test "brew installation script is executable" {
  [ -x "install/macos/common/brew.sh" ]
}

@test "brew installation script runs without errors" {
  run bash install/macos/common/brew.sh
  [ "$status" -eq 0 ]
}

@test "brew command is available after installation" {
  run command -v brew
  [ "$status" -eq 0 ]
}
Enter fullscreen mode Exit fullscreen mode

Each test progressively verifies script existence, executable permissions, actual execution, and expected results (command availability, etc.).

Comprehensive Verification with GitHub Actions

My repository uses GitHub Actions for multi-stage verification. In addition to unit tests, the workflow regularly executes actual end-to-end setup to achieve comprehensive quality assurance.

Unit Test Execution

My repository runs automated tests in macOS and Ubuntu environments to detect platform-specific issues early.

name: Test
on: [push, pull_request]

jobs:
  test:
    strategy:
      matrix:
        os: [ubuntu-latest, macos-latest]
    runs-on: ${{ matrix.os }}

    steps:
      - uses: actions/checkout@v4
      - name: Install Bats
        run: |
          if [[ "${{ matrix.os }}" == "ubuntu-latest" ]]; then
            sudo apt-get update && sudo apt-get install -y bats
          else
            brew install bats-core
          fi

      - name: Run tests
        run: bats tests/install/

      - name: Upload coverage reports
        uses: codecov/codecov-action@v3
Enter fullscreen mode Exit fullscreen mode

Regular Execution of Actual Setup

More importantly, we should verify in environments identical to the actual user experience. This repository's workflow automatically executes the setup process using setup.sh on macOS and Ubuntu runners every Friday. This script wraps the chezmoi environment construction one-liner mentioned earlier.

name: Snippet install
on:
  schedule:
    - cron: "0 0 * * 5"  # Every Friday

jobs:
  build:
    strategy:
      matrix:
        os: [ubuntu-latest, macos-14]
        system: [client, server]
        exclude:
          - os: macos-14
            system: server

    runs-on: ${{ matrix.os }}
    steps:
      - name: Setup dotfiles with snippet
        run: |
          if [ "${OS}" == "macos-14" ]; then
            bash -c "$(curl -fsLS https://shunk031.me/dotfiles/setup.sh)"
          elif [ "${OS}" == "ubuntu-latest" ]; then
            bash -c "$(wget -qO - https://shunk031.me/dotfiles/setup.sh)"
          fi
Enter fullscreen mode Exit fullscreen mode

This regular execution continuously monitors the impact of external dependency changes, OS updates, and package manager changes on environment construction, ensuring reliability when actual users execute the setup.

Code Coverage Measurement and Codecov Integration

My repository measures shell script code coverage using kcov and visualizes it with Codecov. This helps identify untested code paths and improve testing. Actual measurement uses scripts/run_unit_test.sh.

GitHub logo SimonKagstrom / kcov

Code coverage tool for compiled programs, Python and Bash which uses debugging information to collect and report data without special compilation options

Coveralls coverage status Codecov coverage status Coverity Scan Build Status Docker Pulls

PayPal Donate Github All Releases

kcov

Kcov is a FreeBSD/Linux/Mac OS code coverage tester for compiled languages, Python and Bash. Kcov was originally a fork of Bcov, but has since evolved to support a large feature set in addition to that of Bcov.

Kcov, like Bcov, uses DWARF debugging information for compiled programs to make it possible to collect coverage information without special compiler switches.

For a video introduction, look at this presentation from SwedenCPP

Installing

Refer to the INSTALL file for build instructions, or use our official Docker image (kcov/kcov):

Docker images and usage is explained more in the docker page.

How to use it

Basic usage is straight-forward:

kcov /path/to/outdir executable [args for the executable]
Enter fullscreen mode Exit fullscreen mode

/path/to/outdir will contain lcov-style HTML output generated continuously while the application runs. Kcov will also write cobertura- compatible XML output and generic JSON coverage information and can easily be…

View on GitHub

https://about.codecov.io/

# Example of coverage measurement
kcov --clean --include-path=install/macos/common/ \
  coverage/ \
  bats tests/install/macos/common/brew.bats
Enter fullscreen mode Exit fullscreen mode

Coverage reports are automatically commented on Pull Requests using codecov/codecov-action, allowing immediate understanding of the impact of changes.

GitHub logo codecov / codecov-action

GitHub Action that uploads coverage to Codecov ☂️

Codecov GitHub Action

GitHub Marketplace FOSSA Status Workflow for Codecov Action

Easily upload coverage reports to Codecov from GitHub Actions

v5 Release

v5 of the Codecov GitHub Action will use the Codecov Wrapper to encapsulate the CLI. This will help ensure that the Action gets updates quicker.

Migration Guide

The v5 release also coincides with the opt-out feature for tokens for public repositories. In the Global Upload Token section of the settings page of an organization in codecov.io, you can set the ability for Codecov to receive a coverage reports from any source. This will allow contributors or other members of a repository to upload without needing access to the Codecov token. For more details see how to upload without a token.

Warning

The following arguments have been changed

  • file (this has been deprecated in favor of files)
  • plugin (this has been deprecated in favor of plugins)

The following arguments have been added:

  • binary
  • …
View on GitHub

Performance Measurement and Benchmark Automation

To continuously monitor shell startup performance after dotfiles application and detect the impact of configuration changes early, I've automated benchmark measurement in the workflow of GitHub Actions.

This implementation references the following article using benchmark-action/github-action-benchmark. The GitHub Actions workflow measures both initial shell startup time and average startup time (measured 10 times) to quantify the impact of dotfiles configuration on shell startup.

GitHub logo benchmark-action / github-action-benchmark

GitHub Action for continuous benchmarking to keep performance

GitHub Action for Continuous Benchmarking

Action Marketplace Build Status codecov

This repository provides a GitHub Action for continuous benchmarking If your project has some benchmark suites, this action collects data from the benchmark outputs and monitor the results on GitHub Actions workflow.

  • This action can store collected benchmark results in GitHub pages branch and provide a chart view. Benchmark results are visualized on the GitHub pages of your project.
  • This action can detect possible performance regressions by comparing benchmark results. When benchmark results get worse than previous exceeding the specified threshold, it can raise an alert via commit comment or workflow failure.

This action currently supports the following tools:

View on GitHub

Measurement results are published on GitHub Pages, achieving continuous performance monitoring. We can numerically confirm the impact of adding new plugins or configurations on shell startup time, preventing performance degradation before it occurs.

{{< blogcard url="https://shunk031.me/my-dotfiles-benchmarks/" >}}

Implementation Details and Operational Flow

Structure and Implementation Examples of Setup Scripts

Scripts in the install/ directory are designed following the single responsibility principle. Each script handles only the installation and configuration of specific tools, creating an independently testable structure.

As a basic script structure, OS-specific processing is separated into different files and implemented as platform-specific scripts. All installation scripts follow the following common pattern. For shell script writing practices, Minimal safe Bash script template is helpful.

https://betterdev.blog/minimal-safe-bash-script-template/

#!/usr/bin/env bash
set -Eeuo pipefail

# Debug mode setting
if [ "${DOTFILES_DEBUG:-}" ]; then
    set -x
fi

# Tool-specific functions
function is_tool_exists() {
    command -v tool_name &>/dev/null
}

function install_tool() {
    if ! is_tool_exists; then
        # Platform-specific installation process
        # macOS: brew install tool_name
        # Ubuntu: sudo apt-get install -y tool_name
    fi
}

# Main process
function main() {
    install_tool
    # Execute additional configuration process if needed
}

if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
    main
fi
Enter fullscreen mode Exit fullscreen mode

The conditional statement if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then executes the main function only when the script is directly executed. When a script is loaded from another file using the source command (e.g., when calling functions from test files), only function definitions are loaded and main is not executed3. This allows the same script to be used both for "execution" and "library" purposes, greatly improving testability.

For example, Homebrew installation is separated into install/macos/common/brew.sh, and chezmoi Ubuntu installation is in install/ubuntu/common/chezmoi.sh. This structure achieves platform optimization and testability.

Development and Maintenance Flow

New Application Addition Procedure

(1). Create installation script

# Create install/macos/common/new_tool.sh
# Implement following the basic structure above
Enter fullscreen mode Exit fullscreen mode

(2). Create test file

# Create tests/install/macos/common/new_tool.bats
@test "new_tool installation script exists" {
  [ -f "install/macos/common/new_tool.sh" ]
}

@test "new_tool can be installed" {
  run bash install/macos/common/new_tool.sh
  [ "$status" -eq 0 ]
}
Enter fullscreen mode Exit fullscreen mode

(3). Run local tests

bats tests/install/macos/common/new_tool.bats
Enter fullscreen mode Exit fullscreen mode

Test-Driven Development Process

Development always proceeds test-first.

  1. Create test cases: First write expected behavior as tests
  2. Minimal implementation: Implement minimal script that passes tests
  3. Refactoring: Improve code while maintaining behavior
  4. Integration testing: Verify operation in CI environment

This operational flow allows continuous improvement of dotfiles while maintaining quality and maintainability. Each change is necessarily covered by tests and verified in the CI pipeline, minimizing the risk of problems occurring in actual environments.

Conclusion

This article explained the approach of "testable dotfiles management" combining chezmoi and test-driven development. We presented a comprehensive solution to the fundamental problem that traditional dotfiles repositories face: "not knowing if setup scripts work correctly until execution". Specifically, this was an approach combining unit testing with Bats, continuous verification with GitHub Actions, and regular execution of actual end-to-end setup. Please consider incorporating testability into your own dotfiles management to achieve reliable development environment construction.

GitHub logo shunk031 / dotfiles

💻 My dotfiles powered by chezmoi; Primarily built with Rust-based applications (sheldon/startship/mise etc.)

shunk031's

📂 dotfiles

Snippet install Unit test codecov

zsh-users/zsh tmux/tmux rossmacarthur/sheldon starship/starship

🗿 Overview

This dotfiles repository is managed with chezmoi🏠, a great dotfiles manager The setup scripts are aimed for MacOS, Ubuntu Desktop, and Ubuntu Server. The first two (MacOS/Ubuntu Desktop) include settings for client machines and the latter one (Ubuntu Server) for server machines.

The actual dotfiles exist under the home directory specified in the .chezmoiroot. See .chezmoiroot - chezmoi more detail on the setting.

📥 Setup

To set up the dotfiles run the appropriate snippet in the terminal.

💻 MacOS MacOS

  • Configuration snippet of the Apple Silicon MacOS environment for client macnine:
bash -c "$(curl -fsLS http://shunk031.me/dotfiles/setup.sh)"
Enter fullscreen mode Exit fullscreen mode

🖥️ Ubuntu Ubuntu

  • Configuration snippet of the Ubuntu environment for both client and server machine:
bash -c "$(wget -qO - http://shunk031.me/dotfiles/setup.sh)"
Enter fullscreen mode Exit fullscreen mode

Minimal setup

The following is a minimal setup command to install chezmoi and my dotfiles from the github repository on a new empty machine:

sh -c…

View on GitHub

  1. chezmoi starts setup by referencing $GITHUB_USERNAME/dotfiles on GitHub. ↩

  2. By default, chezmoi uses the repository root as the source directory. ↩

  3. This is the same mechanism as Python's if __name__ == "__main__". ↩

Top comments (0)