Let CI check & fix your yamls

👉 Intro

A few months ago, we have started to migrate our sourcecode CI & CD to Github.com.

Then, both DEV and OPS started to migrate source codes.

It appeared that OPS had to migrate a huge amount of Ansible related repos from an onPrem Gitlab instance to Github.com, and we wanted to take advantage of linting and code quality of the yamls.

OPS started to develop the migration process so it could be entirely code driven and tested.

What appeared was that we needed to manage that huge amount of code to :

• 1️⃣ lint (syntax validity, but for weirdnesses like key repetition and cosmetic problems such as lines length, trailing spaces, indentation, etc)
• 2️⃣ fix/implement lints feedbacks in a massive way (they were so huge that we could not imagine do that by hand)

OPS did choose to use the following approach :

• 1️⃣ Lint with yamllint
• 2️⃣ Fix with yamlfixer, which in facts uses yamllint in his core

As DEV & OPS are using yaml everyday a -bit- lot more (k8s, CI configuration, Spring Boot configuration, ...), we started to

think about a common toolbox that we could embed in our CI pipeline, on Github.com.

What we needed to do was to :

1. Integrate OPS sourcecode (DevOPS teamwork)
2. Package it inside as a dedicated GH Action that everyone could add to their pipeline within a few lines of code

🛠️ Yaml Fixer

An important work has been done by OPS on yamlfixer so it could be used conveniently from the cli :

opt-nc / yamlfixer

Automates the fixing of problems reported by yamllint by parsing its output

yamlfixer

❔ About

yamlfixer automates the fixing of problems reported by yamllint by parsing its output.

🖼️ Screenshot

Here'a screenshot of yamlfixer launched on yaml-test-suite :

📑 Installation

The easiest way to install yamlfixer is from pypi, as described below.

🐧 Linux install

python3 -m pip install yamlfixer-opt-nc

🪟 Windows install

python -m pip install yamlfixer-opt-nc

pipx setup

For an optimal experience we recommand using pipx.

To install :

pipx install yamlfixer-opt-nc
pipx list

To upgrade :

pipx upgrade yamlfixer-opt-nc

To uninstall :

pipx uninstall yamlfixer-opt-nc

🚀 Usage

This software automatically fixes some errors and warnings reported by yamllint.

usage: yamlfixer [-h] [-v] [-b] [-B BACKUPSUFFIX] [-d] [-D DIFF_FILE] [-e EXTENSIONS] [-f]
                 [-F] [-l] [-N] [-n] [-r LEVEL] [-j | -p | -s] [-t TABSIZE]
                 [-c CONFIG_FILE | -C CONFIG_DATA]
                 [FILE_or_DIR [FILE_or_DIR ...]]
Fix formatting problems in YAML documents. If no file is specified, then reads input

…

View on GitHub

Still this cli needed a Python runtime to be used.

🐳 Deliver as a Docker image

Then we wanted to distribute it an easier way, which meaned :

package it as a Docker image.

At that point, DEVs took the relay and implemented Docker release on GH CI, and pushed images on DockerHub under optnc/yamlfixer :

🏃 Deliver as a GH Action

Now, we could use the Docker image to implement the following Public GH Action :

opt-nc / yamlfixer-action

Github action that fixes some errors and warnings reported by yamllint and create a pull request with fixes.

❔ About

This github-action is based on the tool Yamlfixer :

• Developed by @tamere-allo-peter
• Integrated as a Github Action by @mbarre

It automatically fixes some errors and warnings reported by yamllint and creates a pull request that embeds the fixes.

🧰 Usage

See action.yml

You need a Github account that is allowed to create pull request on the inspected repository.

Find below an example that allows to check yaml files on each push with verbose mode :

name: Lint yaml files

on: [push]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout my app
        uses: actions/checkout@v3
      - name: Lint yaml files
        uses: opt-nc/yamlfixer-action
        with:
            options: --summary
            user: ${{secrets.my_user}}
            token: ${{secrets.my_user_password}}

The github-action creates :

1. A new branch named yamlfixer/patch/$branch_name
2. The pull request to be merged into the working branch

🔖 Resources

Here are some useful resources…

View on GitHub

... that could be released on the Marketplace :

🤖 Action usage

Here is the code (and... it's yaml 🐔🥚🐣) required to use the action in any workflow :

name: Lint yaml files

on: [push]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout my app
        uses: actions/checkout@v2
      - name: Lint yaml files
        uses: opt-nc/yamlfixer-action
        with:
            yaml_file: .github/*.yml
            options: --verbose
            user: ${{secrets.my_user}}
            token: ${{secrets.my_user_password}}

👉 Notice that the action creates a new branch and the pull request to be merged into the working branch so this code activity can be managed and reviewed as any other source code proposal.

🎦 Live demo

As a live video worths a thousand words : discover how the Github Action runs on a dedicated repository :

📷 Examples in details

❌ Rejected PR because of lint failure

✔️ Valid PR coming from GH Action

And code change review :

🪙 ROI

Now, both DEVs and OPs are using it to enhance yaml quality proactively with the help of CI and yaml quality issues are managed on our daily workflow on GH Issue.

👉 Also notice that all the teams using the GH Action have the same level of quality accross the whole organization.

🙏 Acknowledgments 🧑‍🤝‍🧑

A lot of thanks to the team that made this work come true, all on our DEVOPS pipeline :

• Jerome ALET as core Yaml Fixer developer & integrator
• Michèle BARRE for Docker packaging and GH Action development
• Daniel SANTOS for its test on DockerHub publish and his continuous feedbacks on this new Gh Action