Using PyYaml to support YAML and JSON configuration files in your CLI tools

In 2019, I wrote the wslrun tool as a proof of concept for a testing framework (i.e. for git hooks, local testing, etc.) CLI that used the Windows Subsystem for Linux (WSL). I designed it to be guided by a JSON file, like so:

{
    "name": "PS Test",
    "description": "Tests building in multiple environments",
    "pullPolicy": "Never",
    "ciMode": "False",
    "stages": [
        {
            "image": "AlpineDub",
            "name": "Build on distro with a passing job",
            "steps": [
                "sudo apk --update add musl-dev",
                "env CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build -v -x -o package-amd64-alpine"
            ]
        },
        {
            "image": "UbuntuDub",
            "name": "Build on distro I know will fail",
            "steps": [
                "GOOS=linux GOARCH=amd64 go build -v -x -o package-amd64-ubuntu"
            ]
        },
        {
            "image": "AlpineLub",
            "name": "Build on image I know doesn't exist",
            "steps": [
                "env GOOS=linux GOARCH=amd64 go build -v -x -o package-amd64-alpine"
            ]
        }
    ]
}

Where stages is just a nested object containing the WSL distro (image) to execute the respective steps (the commands to be run in the WSL environment).

However, I don't always want to write JSON by hand, and while it's easily generated by automation, which makes it a great format for handing this information over to another automated system (like a CI workflow), I wanted to support YAML as well, which was substantially easier for me to compose by hand. In that case, the above JSON became:

--------
name: PS Test
description: Tests building in multiple environments
pullPolicy: Never
ciMode: 'False'
stages:
- image: AlpineDub
  name: Build on distro with a passing job
  steps:
  - sudo apk --update add musl-dev
  - env CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build -v -x -o package-amd64-alpine
- image: UbuntuDub
  name: Build on distro I know will fail
  steps:
  - GOOS=linux GOARCH=amd64 go build -v -x -o package-amd64-ubuntu
- image: AlpineLub
  name: Build on image I know doesn't exist
  steps:
  - GOOS=linux GOARCH=amd64 go build -v -x -o package-amd64-alpine

Having my CLI tool handle this was exceptionally trivial to accomplish: The Python pyyaml package has a method, safe_load which can be used to, either, read JSON, or, for our purposes, render inputted YAML to be returned as JSON.

In this case, the wslrun tool takes an argument, like:

wslrun --pipeline config.json

or

wslrun --pipeline config.yaml

and in both cases, this file is read, and passed to a function like this:

def manifestRead(manifestPath):
    try:
        with open(manifestPath, "rt") as manifest_json:
            manifest = yaml.safe_load(manifest_json)
        return manifest
    except OSError as e:
        return e

If the manifestPath can be read into a string (manifest_json) and, itself, read as valid YAML (or is already valid JSON), the safe_load function will return this object as manifest, which the rest of the CLI flow will operate on as described above.

The benefit to an implementation like this is: even if you are providing a JSON file, it acts as a check on the content's validity, but also a way to provide multiple formats for users to write configurations in, while remaining a comparatively low-effort yet effective way to accomplish this.