Event-driven scripting on your K8s infrastructure with Brigade

What is Brigade?

Brigade is a tool from the team that brought you Helm and Draft which are designed to assist developers and operations teams to accomplish more work with event-driven scripting. The tool enables teams to build out any ordered workflow of containers in Kubernetes and trigger the workflow by listening for arbitrary events.

Using Kubernetes allows users to not only manage long-standing services, but also run quick short-term tasks that are commonly used for things like event-driven programming, code quality testing, and batch processing.

By leveraging Brigade, users are able to string multiple tasks together and execute them inside containers. It follows the same patterns found in other forms of serverless computing, where a Brigade pipeline is triggered by an event.

At first glance, Brigade:

• is Kubernetes-native, with one-line installation.
• supports projects and a customizable event model.
• uses JavaScript, making a Brigade pipeline an application.
• enables arbitrarily complex (parallel or synchronous) pipelines.
• executes functions in containers.
• fully supports Kubernetes volumes, pods, role-based access control (RBAC), and secrets.

What can I build with Brigade?

You can use Brigade to easily automate just about any process you can think of in your workflow using Javascript. If it can be run with Javascript, you can likely automate it with Brigade. Many users have automated things such as:

• Batch processors
• Unit tests
• Slack notifications (Chat Ops)
• Cron Jobs
• CI/CD

How it works

Brigade is an in-cluster runtime environment. It interprets scripts and executes them often by invoking resources inside of the cluster. Brigade is event-based scripting of Kubernetes pipelines.

• Event-based: A script execution is triggered by a Brigade event.
• Scripting: Programs are expressed as JavaScript files that declare one or more event handlers.
• Pipelines: A script expresses a series of related jobs that run-in parallel or serially.

Setting up a CI pipeline with Brigade

Throughout this tutorial, we’ll walk you through the creation of a basic web application with a Brigade CI pipeline for testing the application.

It’ll consist of two parts:

• A public site that lets people generate UUIDs.
• A brigade.js that tests the site

We’ll assume you have Brigade, Git (a version control system), and pip (a package management system for Python) installed already.

You can tell if Brigade is installed and its version by running the following command in a shell prompt (indicated by the $ prefix):

$ helm status brigade-server

Creating your first application

For this tutorial, we’ll be creating an example application written in Python which uses Flask to provide a very simple UUID generator web server. Flask is a microframework for Python based on Werkzeug, Jinja 2 and good intentions. (Note that this application is written for Python 3, specifically.)

Let’s write the web server. To create your app, type this command:

$ mkdir -p uuid-generator/app

Open the file uuid-generator/app/__init__.py and put the following Python code into it:

from flask import Flask, Response
app = Flask(__name__)


@app.route("/")
def hello():
    return "Hello World!"

This is the simplest application possible in Flask. To call hello(), we need to map it to a URL - in this case we want to map it to the root path (/).

Start the development server

Now that we’ve written the app, let’s run it!

$ echo "Flask" > requirements.txt
$ pip install -r requirements.txt
$ FLASK_APP=uuid-generator/app/__init__.py flask run
 * Serving Flask app "app"
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

Now, open a Web browser and go to “/” on your local domain – e.g., http://127.0.0.1:5000/. You should see “Hello World!”:

If you look back to the running flask app’s logs, you should see new logs pop up:

127.0.0.1 - - [18/Aug/2017 12:56:16] "GET / HTTP/1.1" 200 -

Generating UUIDs

Let’s make the application generate a random UUID on every request. Edit the uuid-generator/app/__init__.py file so that it looks like this:

from flask import Flask, Response
import uuid

app = Flask(__name__)


@app.route("/")
def hello():
    return Response(str(uuid.uuid4()), status=200, mimetype='text/plain')

Re-run the web server and open the browser back to http://127.0.0.1:5000/. You should now see a random UUID:

Keep refreshing the page. You should see new UUIDs being generated every time you refresh the page.

Create a Github account

To use Github, you will need a Github account.

In your own browser:

1. Open a new browser tab
2. Navigate to https://github.com/
3. Create an account

If you already have Github account, continue to the next exercise.

After you sign up, you will receive a verification e-mail. Be sure to verify your e-mail address to Github by following the instructions in that e-mail.

Initialize your repository

Now that you have a Github account and a working application, let’s create a git repository and push it up to Github!

Let’s first change working directories over to the Flask app we made in Tutorial 1.

$ cd uuid-generator

Now, we’ll create a git repository and push the project to Github.

$ git init
$ echo -e "*.pyc\n*.egg-info" > .gitignore
$ git commit -am "initial commit"
$ git push origin master

We created a .gitignore file so all compiled python files and python eggs (noted with the .pyc and .egg-info extensions, respectively) are not added to source control.

Configuring GitHub

We’ll now walk through the process for configuring your newly created Github repository with Brigade for testing new features. We’ll configure a new Brigade project, and have Github push events to trigger Brigade builds.

We want to build our project each time a new commit is pushed to master, and each time we get a new Pull Request.

To do this, follow the Brigade GitHub App documentation to set up a GitHub App. During configuration, copy the shared secret above (mDXUDZyDsTUHw4KZIMPOQMN1) and set this as the “Webhook secret” value for the App.

Create a Brigade project

The Brigade server tracks separate configuration for each project you set up. To create and manage these configurations, we use the brig cli.

Here we create a project for our GitHub repo:

$ brig project create
? VCS or no-VCS project? VCS
? Project Name bacongobbler/uuid-generator
? Full repository name github.com/bacongobbler/uuid-generator
? Clone URL (https://github.com/your/repo.git) https://github.com/bacongobbler/uuid-generator.git
? Add secrets? No
Auto-generated a Shared Secret: "mDXUDZyDsTUHw4KZIMPOQMN1"
? Configure GitHub Access? No
? Configure advanced options No
Project ID: brigade-5ea0b3d7707afb5d04d55544485da6aff4f58006c1633f4ae0cb11

Note: To explore the advanced options, each prompt offers further info when ? is entered.

We’ll need to upgrade our Brigade server with our brigade-github-app sub-chart configuration filled in:

$ helm inspect values brigade/brigade > brigade-values.yaml
$ # Add configuration under the `brigade-github-app` section
$ helm upgrade brigade brigade/brigade -f brigade-values.yaml

We can then get the IP needed to update the “Webhook URL” entry for our App. Run this command on your Kubernetes cluster, and look for the brigade-github-app line:

$ kubectl get service
NAME                                TYPE           CLUSTER-IP     EXTERNAL-IP    PORT(S)        AGE
brigade-server-brigade-api          ClusterIP      10.0.34.228    <none>         7745/TCP       1d
brigade-server-brigade-github-app   LoadBalancer   10.0.69.248    10.21.77.9     80:31980/TCP   1d

You will use the EXTERNAL-IP address to form the full URL: http://10.21.77.9/events/github. Update the App’s “Webhook URL” with this value. (Note: it is preferred that DNS be set up instead of a hard-coded IP. See the Brigade GitHub App docs for more.)

The next time you push to the repository, the webhook system should trigger a build.

Final Thoughts

Using Brigade can add a really useful tool for those trying to implement Kubernetes-native automation into their pipelines and processes.