What is Concourse?
Concourse is an open-source continuous thing-doer.
Concourse is a CI/CD tool that is configurable with a simple YAML file.
The Walkthrough
Pre-Requisities
Follow the steps on https://concourse-ci.org/ to install Fly & Concourse.
If this is your first time checking out Concourse, I would highly recommend reviewing the tutorials & docs they have posted.
Exploring a basic Resource Type
You can follow along using or (fork my repo)[https://github.com/thehandsomezebra/concourse-resourcetype-helloworld].
A concourse resource_type works well using a minimum of 4 files:
+-- resource
| +-- check
| +-- in
| +-- out
+-- Dockerfile
Each of the scripts are required to have the output of its execution in json appropriate format.
Let's walk thru each file at a high level and get everything set up!
check
The check file is going to check for new versions.
The check script we will report back a message of Hello World.
#!/bin/bash
set -eu
set -o pipefail
echo '[{ "revision": "N/A", "message": "The check script says Hello World!" }]'
in
This fetches the given resource.
For the In script, let's add some metadata. This metadata will be viewable in a few places in the pipeline.
#!/bin/bash
set -eu
set -o pipefail
echo '{ "version": { "revision": "N/A", "message": "The in script version message says Hello World" }, "metadata": [{ "name": "message", "value": "The in script metadata message says Hello World." }] }'
out
For the Out script, let's get a little more complex. We'll be getting a variable greeting from the pipeline, and let's echo it out.
We'll be using jq to grab it from the params that get passed via the pipeline.
#!/bin/bash
echo '{ "version": { "revision": "N/A", "message": "The out script version message says Hello World" }, "metadata": [{ "name": "message", "value": "The out script metadata message says Hello World." }] }'
set -e -u
exec 3>&1 # make stdout available as fd 3 for the result
exec 1>&2 # redirect all output to stderr for logging
echo "Echo says hi from the out script!" >&2
source=$1
if [[ -z "$source" ]]; then
echo "usage: $0 <path/to/source>"
exit 1
fi
# for jq
PATH=/usr/local/bin:$PATH
export TMPDIR=${TMPDIR:-/tmp/rclone}
mkdir -p "${TMPDIR}"
payload=$(mktemp "$TMPDIR/resource-request.XXXXXX")
cat > "$payload" <&0
greeting=$(jq -r '.params.greeting // ""' < "$payload")
echo "Param greeting says: $greeting" >&2
Dockerfile
This Dockerfile will be short and sweet.
- Spins up a tiny Alpine Linux instance
- Loads in bash & jq
- Adds in our script files to the /opt/resource folder
- Makes those files executable
- Specifies that we will be running bash in root
FROM alpine:3.13
RUN apk add --update --upgrade --no-cache bash
RUN apk update \
&& apk upgrade \
&& apk add --update bash curl unzip jq
ADD resource/ /opt/resource/
RUN chmod +x /opt/resource/*
WORKDIR /
ENTRYPOINT ["/bin/bash"]
One last file to get it running: the pipeline yaml file
This file does not need to be in the docker - but it's inluded in my repo, so you can follow along.
Create a file called hello-pipeline.yml
resource_types:
- name: helloworld
type: docker-image
source:
# set repo of your helloworld resource
#repository: Your-Docker-Acct/Your-Docker-Repository
## or use my helloworld resource
repository: thehandsomezebra/concourse-test
resources:
- name: helloworld
type: helloworld
jobs:
- name: hello-job
plan:
- get: helloworld
- put: put-helloworld
resource: helloworld
inputs:
- helloworld
params:
greeting: "Hey y'all!"
This is going to create a new pipeline called hello-job that will get the in and put the out.
Get it running
Following the steps above, create a github repo (or fork mine) & create a docker hub repo. It's fast to connect your github repo to docker and even set it up to autobuild (learn more about that here)).
Once your repo is ready in Docker Hub, let's start the concourse pipeline by using this line
Let's start the hello-world pipeline!
Run the following in your terminal.
fly -t ci set-pipeline -c hello-pipeline.yml -p hello-world
ci will be the name of your target (if you followed a the tutorial, it might be named tutorial or example)
If we look at the pipeline diagram, we can also see the result of the check step in the box with the dashed lines (dependency - trigger).
Checking our job, we can see our messages in each of the stages.
There's a lot you can do creating your own resource types for concourse - be sure to check out the Resource Types catalog, contribute to the community and get assistance (or lurk thru questions that have been already answered) in Discord in the #resource-types channel.
Note: I'm not affiliated with Concourse - I am just a nerd that is fascinated by the continuous thing-doer continuously doing things.
Top comments (0)