Create cloud-native buildpacks using Github Actions

I've been working with CNB technology a lot during the last few months particularly with pack and PackIt library. Overall it's been a lot of fun and learning along the way, I'm not going to lie here, the learning curve seemed propelled by a rocket but turned out to be quite simple once you grab your head around the new terminology and concepts . Just like anything in tech.

Once I understood the hows and whys then I was buildpacking this and buildpacking that, pretty soon I found myself doing repetitive jobs that were begging to be automized. I'm talking about simple script bundled into buildpacks for better distribution, organization, and versioning plus all the benefits CNB brings to the table.

I decided to invest some time in the making of a more generic tool to help me with this problem.

I needed a little something that took a list of commands and turn them into a valid build pack. In my mind, I could see and almost touch the form and shapes of this new toy, how the sequence of events was executed to coordinate the creation of this on the fly _buildpack.

Idea

The easiest way to explain the process is with the diagram below.

# From a working PackIt project, parametrize and map input commands 
# taken from a YML file
┌───────────────────────┐
│      Base file        │ 
└───────────────────────┘
          ▼
# Compile the modified project so once the build process kicks 
# https://buildpacks.io/docs/concepts/components/lifecycle/build/
# our buildpack executes one by one the **commands**
┌────────────────────────┐
│      Build.go          │ 
└────────────────────────┘
          ▼
# Pack it for distribution and release at will
┌────────────────────────┐
│    buildpack.cnb       │
└────────────────────────┘

Whiteboard time

What I first did was draft a very simple definition that allowed me to see how a regular user such as me, in this case, would like to define a list of commands. I decided to use YAML for this. Here's how a command file looks like.

This isn't a Github Action file although I borrowed some of the key names (name and run)

laraboot-commander:
  commands:
    - name: Init
      run: |
        echo "Init 🔥"
        pwd
    - name: NoOps
      run: |
        echo "Hello 🗺️"
        ls -ltah
    - name: Install deps
      run: |
        echo "installing 🤖"

This list could as well be defined as a list of "strings" but I foresee probably as a next step eventually I'd like to be able to pass extra parameters, environment variables, etc. Just like Github allows it. So the final definition looks something like the one below.

// type Commander struct {
//  WorkingDir string `yaml:"directory"`
//  Commands   []struct {
//      Name string `yaml:"name"`
//      Run  string `yaml:"run"`
//  } `yaml:"commands"`
// }

Then I thought, what would be the inputs I'd like to accept for the creation of buildpacks from Gh pov? That was a question easy to answer. Being familiar with pack cli I decided to ask just for name and version.

Detect

For the scope of the project, let's agree the detection will always occur and the buildpack will always participate in the build plan.

Build

The build part was quick because honestly executing a bash command or script in GO might be the easiest thing to do. Nevertheless, I found some blocks along the road that I didn't foresee in my planning.

Such as this one:

ERROR: failed to build: the launch, cache and build flags should be in the types table of /layers/xxx. Basically, if we're authoring a buildpack from a template then allowing the user to provide a name for it, the name provided has to be considered during the build process. We need to grab the name of a buildpack directly from the buildpack.toml file.

Also, the constant unknown of where I am stood (what context or path) and why it fails while running:

• Locally
• Locally with WSDL & ubuntu
• Same repo remotely with Github actions
• Another repository (consumer) using the Github action

At the end, I can say that for this being a quick prototype it actually works really good and might even be a more robust thing down the road.

Usage

Ok, how do we use this Github Action that creates buildpack for us? I'm glad you ask, couldn't be simpler.

First, create your commander file. It can contain several or just one command.

#commander.yml
laraboot-commander:
  commands:
    - name: Init
      run: |
        echo "Hello World"

As a part of my workflow I will use 3 actions:

• The setup-pack action. This will set up pack cli and other util tools. Check out the repo if you feel interested.
• Our commander action
• actions/upload-artifact@v2

# .github/workflows/worklfow.yml
      - uses: buildpacks/github-actions/setup-pack@v4.1.0
        with:
          pack-version: 0.20.0

      - name: Gh action
        uses: laraboot-io/laraboot-commander/actions/commander@master
        with:
          name: my-buildpack
          version: 0.0.2
          file: commander.yml

      - name: Upload dist
        uses: actions/upload-artifact@v2
        with:
          name: dist
          path: dist

And if everything works fine it will look like this:

    'create'  buildpack.toml
    'create'  bin/build
    'create'  bin/detect
Successfully created 'my-buildpack'
Successfully created package 'dist/my-buildpack.cnb'

Then as a part of your workflow or in another project you might use this buildpack as follows:

mkdir -p app
pack build sample-app --path app --buildpack my-buildpack.cnb --builder cnbs/sample-builder:bionic

Conclusion

We've seen how to create a buildpack from a YML file using a custom GitHub action but the same can be replicated for any other CI runner easily.

We're going to hear more and more about buildpacks in the years to come. It's a game-changer for developers and operators in an era of automation and continuous integration. The source code of this post is available here

Where to go from here?

• Developing a third party CLI action
• Creating container images with Cloud Native Buildpacks using AWS CodeBuild and AWS CodePipeline