DEV Community: Liam Conroy Hampton

What is the difference between the Azure Command Line Interface and Azure Developer Command Line Interface?

Liam Conroy Hampton — Mon, 24 Jul 2023 14:18:30 +0000

Let's start by saying they are both great developer tools for Azure and they both allow you to interact with services in two different ways. So let's dig into what they are and when you should use them.

What is the Azure Command Line Interface (Az CLI)?

The Azure (CLI) is a command-line tool that you can use to manage and control your Azure resources directly from the terminal. It is a cross-platform tool that can be installed in a number of ways (Windows, Mac, Linux, Docker and Azure Cloud Shell).

However, it doesn't stop there. You are also able to install a number of extensions to add additional functionality to the Azure CLI. Some examples of extensions are:

Azure Interactive Mode
Azure DevOps
Azure Kubernetes Service

A full list of extensions can be found here.

My personal favourite is the Azure Interactive Mode, an AI based extension. You can checkout my blog post on this here: https://dev.to/liamchampton/step-by-step-how-to-setup-and-use-azure-cli-interactive-mode-292e

When using the Azure CLI you would use the alias az and a subcommand typically follows for an action to take. This is just a few of the commands available to you and you can manage almost all of your Azure resources using the Azure CLI.

Example use case of the Azure CLI

A typical example of when to use the Azure CLI is if you are creating a simple resource group that will hold a Virtual Machine and you need complete control over the commands being executed and how this Virtual Machine will be deployed.

What is the Azure Developer Command Line Interface (Azd CLI)?

The Azure Developer CLI is a tool primarily developer-centric. This tool is aimed at creating and deploying Azure applications and infrastructure using only a handful of commands. The Azd CLI is open-source and has 3 main goals:

Reduce the time required for a developer to be productive
Demonstrate opinionated best practices for Azure development
Help developers understand core Azure development constructs

When using the Azd CLI you would typically follow this workflow:

Select a template - This template will be written in Bicep or Terraform and it outlines the infrastructure you want to deploy within Azure.
Login to Azure using the command azd auth login
Run the command azd init and it will initialise the project with the selected template.
Package it all up, provision and deploy the application using the command azd up

As you continue to iterate, write code and develop your solutions you can continue to update your deployed application by only running the command azd deploy.

If you happen to change the underlying infrastructure files then you only need to run the command azd provision again and it will handle the rest for you.

Here are some of the commands available to you when using the Azd CLI:

Example use case of the Azure Developer CLI

A typical use case of the Azd CLI is if you have a full application consisting of multiple complex services that all connect to one anther and will be a very arduous task to set them up manually in the portal or through many Azure CLI commands. This would also bode well if you are working in a distributed team and you want to ensure everyone is using the same infrastructure and application code.

When to use the Azure CLI and when to use the Azure Developer CLI?

Both CLI's allow you to deploy services and infrastructure to Azure. However, the Azure CLI is more of a general purpose tool that allows you to manage and control your Azure resources directly from the terminal with a lot of freedom to edit, change and have more of a granular control over each command executed. The Azure Developer CLI is more of a developer-centric tool with a focus on speeding up deployments by having a lot of the dots connected for you within a single application. This isn't to say you lack control but you certainly have less to worry about when it comes to the deployment process.

Happy terminal'ing!

If you have any questions or some feedback, I'd love to hear so feel free to drop me a line 📝

Twitter
LinkedIn

Step-by-Step: How to setup and use Azure CLI Interactive mode

Liam Conroy Hampton — Thu, 13 Jul 2023 11:55:47 +0000

The Azure Command Line Interface (CLI) is a developer tool that allows users to control Azure services using command line commands. It is a cross-platform tool that can be installed in a number of ways (Windows, Mac, Linux, Docker and Azure Cloud Shell).

It also has the ability to have a number of extensions installed to add additional functionality, assisting with your development and deployments to the cloud. One of these extensions is the newly released AZ Interactive extension that is now in preview.

Useful links:

What is AZ Interactive?

Using any CLI can sometimes be cumbersome, especially if you are new to the tool. AZ Interactive is a new AI-based extension for the Azure CLI and somewhat of a cheat code for developers. It allows you to interact with Azure resources in a more interactive and intuitive way than usual. It is a great tool for those who are new to the Azure CLI and want to learn how to use it, or for those who are more experienced and want to be more efficient.

It sets out to solve a few problems that developers face when using the Azure CLI:

Discoverability of services, commands and parameters
Reducing the time taken to learn the Azure CLI and its commands
Speed up the process of writing commands and deploying resources

Prerequisites

To use AZ Interactive, you will need to have the Azure CLI installed. You can find instructions on how to install the Azure CLI here.

Installing AZ Interactive

To install the AZ Interactive extension, you will need to run the following command from your terminal (once you have the Azure CLI installed):

az extension add --name interactive --upgrade

This command will install the extension and any dependencies that are required. Once the installation is complete, you can run the following command to start the interactive mode:

az interactive

Those are the only commands you need to install and start using AZ Interactive. It is that simple!

Once you have it running, your terminal will look something like this:

Using AZ Interactive

Now you have this awesome tool installed, what sort of super power does it provide? Let's take a look at some of the features.

Command Suggestions

One of the great features of AZ Interactive is the command suggestions. As you type, it will suggest commands that match what you have typed. It is somewhat similar to tab completion you may find in other tools, but it is a lot more intuitive and it will match commands based on what you have typed so far and allow you to chose from a list.

Command Scenarios

Another great feature of AZ Interactive is the command scenarios. This is where AZ Interactive allows you to select a scenario and then it will guide you through the process of creating the resources required for that scenario and provide you with the commands needed to do so.

Search for commands and scenarios

You also have a search capability that allows you to search for commands, parameters and resources. This is great if you know what you are looking for and want to have the CLI generate the command(s) for you. For instance, if you search with the prompt "connect an app service to mongodb", it will generate a number of commands within a scenario that you can run.

Running commands outside of interactive mode

You may notice that you are abstracted away from your normal terminal shell at this point and tucked into an "az interactive" shell. This does not mean that you cannot run commands outside of interactive mode. You can run any command you like by prefixing it with "#". This will run the command as normal and return the output to the terminal for you. For example, if I want to know my current working directory, I would typically run the command "pwd". In interactive mode, I would run "#pwd" and it would return the output to the terminal in the same manner.

JMESPath query support

This is a powerful feature and probably one of my favourites. JMESPath is a query language for JSON that allows you to extract and transform elements of JSON. AZ Interactive supports JMESPath queries and allows you to run them against the output of commands. This is great if you want to filter the output of a command to only return the data you are interested in. For example, if I want to get a the name of my resource group from the JSON object returned in a previous command I can do just that by using ??name as seen in the example below.

This is not all either. You can also use this to inject jmespath query into future commands allowing you to use this data throughout your session. For example, if I want to get the name of my resource group and use it in a command to delete the resource group, I can do that by using ??name in the command as seen below.

Status codes

When running terminal commands you may be used to seeing a status code returned. This is a number that indicates whether the command was successful or not. This is also supported within this extension and you can find the previous status code of each command by simply using the $ symbol. This will return the status code of the previous command.

I previously ran a command to delete a resource group and it returned a status code of 0, indicating that the command was successful.

Setting Scope

When you are working within a certain command group you can set the scope of the commands. This means that you can run commands without having to specify the command group every time. For example, if I want to run a number of commands to work within resource groups, I can set the scope to be groups using the command %% group. From there, I can run any command within that group without having to specify the prefix every time. This saves time and acts as a shorthand for you.

Fun colour themes

This is a fun way to spice up your terminal. You have the ability to change the colours you want to use with a number of different styles. This changes the text colour, drop down lists and more!

By default, all of the AI capabilities are turned on but if you are not a fan of having all the noise in your terminal and you want to get back to the basics, you can turn it off by running the command az config set interactive.enable_recommender=False.

Overall this is a great tool to use and I would highly recommend it to anyone who is new to the Azure CLI or wants to speed up their development and deployment processes from within the terminal. It is a great tool to have in your arsenal and I am sure it will only get better as it is developed further.

For more examples of how to use the Azure CLI Interactive mode, you can checkout the documentation here.

Happy terminal'ing!

If you have any questions or some feedback, I'd love to hear so feel free to drop me a line 📝

Twitter
LinkedIn

Unit Testing - Table Tests in Go

Liam Conroy Hampton — Thu, 27 May 2021 15:29:14 +0000

There are many types of testing that takes place throughout the software development lift cycle (SDLC) but in this post I will concentrate on table testing within unit tests.

First let’s look at WHAT test cases are, specifically unit testing. Unit testing is a granular level of testing within your project. It tests the individual parts of the puzzle for fitment and correctness - "Does this function behave how I expect it to behave?".

WHY do we use unit testing? - The aim of unit testing is to get high code coverage and check each moving part of your program. You shouldn't need to run your program everytime you want to check one small piece of functionality. That is only a good way to test "the happy path", its manual and time consuming. So, what if things don’t go as expected? You should be able to test failures and handle unexpected output from the operating function. With unit testing, you can do this, and you should do this.

In practise, unit testing is a must but it can get messy very quickly. I will show you using an example function below.

Function

package main

import "fmt"

func main() {
    str := "Liam"
    reversedStr := reverseString(str)
    fmt.Println(reversedStr)
}

func reverseString(str string) string {
    var reversedStr string
    for _, s := range str {
        reversedStr = string(s) + reversedStr
    }
    return reversedStr
}

This function will take the string "Liam" and reverse it. The result of running this program should be "maiL".

So lets write some tests for the reverseString() function.

Test Function

package main

import "testing"

func TestReverseString(t *testing.T) {
    t.Run("reverse of Liam", func(t *testing.T) {
        str := "Liam"
        expected := "maiL"
        got := reverseString(str)
        if got != expected {
            t.Errorf("expected %s, got %s", expected, got)
        }
    })

    t.Run("reverse of foobar", func(t *testing.T) {
        str := "foobar"
        expected := "raboof"
        got := reverseString(str)
        if got != expected {
            t.Errorf("expected %s, got %s", expected, got)
        }
    })

    t.Run("reverse of testing", func(t *testing.T) {
        str := "testing"
        expected := "gnitset"
        got := reverseString(str)
        if got != expected {
            t.Errorf("expected %s, got %s", expected, got)
        }
    })
}

You can see by the tests above; with only 3 test cases this is quite a lot of code. This is still valid, but it is long and messy. Just imagine what these tests might look like if the original function was a bit more complex.

Let’s simplify these cases with a table test.

Test Function Simplified

func TestReverseString(t *testing.T) {
    tests := []struct {
        input  string
        output string
    }{
        {input: "Liam", output: "maiL"},
        {input: "foobar", output: "raboof"},
        {input: "testing", output: "gnitset"},
    }

    for _, testCase := range tests {
        got := reverseString(testCase.input)
        expected := testCase.output
        if got != expected {
            t.Errorf("expected %s, got %s", expected, got)
        }
    }
}

That looks much better but what did I just do?

To begin with, I created a struct that houses each individual test case input and expected output of the type string. This is what the original function expects as input and what it returns as an output.

Next, I created a range loop. This loop iterates through the test cases in the struct containing the input / output. Inside the loop is a single test case that will be fed the input / output for each test case from the struct. The code inside the loop is the same code used in the first set of test cases but introducing the loop removes the need to duplicate test cases if the only part changing is the input / output.

Just like that, I have simplified the tests, made them easier to follow and easier to ammend.

This code can be found at https://github.com/liamchampton/go-string-manipulation

If you have any questions or want to see more content like this, feel free to drop me a line 📝

GitHub | Twitter | Instagram | LinkedIn

Deploy your Application using GitHub Actions

Liam Conroy Hampton — Mon, 10 May 2021 15:49:08 +0000

To make it easier for you to consume I have created this tutorial by breaking it down into bite size chunks in GitHub. It makes a bit more sense and makes reading / following the workshop a little easier, as opposed to having a long thread of pictures and code snippets. Everything in this tutorial is free - no credit card is needed at all

Workshop Links:

Workshop Tutorial

IBMDeveloperUK / Deploy-To-IBM-Cloud-With-GitHub-Actions

Deploy applications to IBM Cloud Foundry using GitHub Actions

Working Example

liamchampton / github-actions-demo

Demo the use of GitHub Actions into IBM Cloud Foundry

The purpose of this tutorial is to show you how to create a small Go application on your system and then deploy it into IBM Cloud Foundry using GitHub Actions.

In this tutorial there are a few bits of tech involved.

IBM Cloud - The cloud provider of choice
IBM Cloud Foundry - The cloud service
Golang Application (or an application of your choice)
GitHub platform - The code repository platform
GitHub Actions - The platform service

IBM Cloud Foundry is a neat tool that allows you to run your apps in the cloud with very little hassle. You essentially create the app you wish to run in the cloud, set up a small config file (this is optional) and then you push it into IBM Cloud via the CLI. It's a sleek process as far as Cloud deployments are concerned. IBM Cloud Foundry is free to use, and it is included in the Lite Tier account which you can sign up for free. Check out the docs for more information and supported runtimes.

GitHub is a great code repository platform which is widely used by developers all over the world. A fantastic feature within GitHub is called GitHub Actions. GitHub Actions is essentially a pipeline tool integrated into GitHub allowing you to deploy your code into an environment of your choice. This is the beginning of your CI/CD journey with your application.

To do this, you need to create a workflow configuration file outlining the platform and all the configurations you need and then specify how to deploy your application. Much like other pipeline tooling, it has images and buildpacks available for you to use.

A possible use case

You have just built your amazing personal website and manually deployed it into the cloud hosting provider. It outlines your achievements, your personal projects, and your work history.

This works well for the time being as it holds current, up-to-date information. But let's say you have just earned a new qualification and you want it to add it to your website. You will now need to write the new code, wrap it up, push the code into GitHub and then manually deploy it again.

This is where GitHub Actions comes in. It will automate the last part for you. All you should do is focus on writing the code. Let the deployment be taken care for you.

This is just one example and running the risk of turning this blog into a DevOps discussion but that is another can of worms for another time. I hope you enjoy the workshop tutorial and find is useful! 🎉

If you have any questions or want to see more content like this, feel free to drop me a line 📝

GitHub | Twitter | Instagram | LinkedIn

Interacting with IBM Cloud

Liam Conroy Hampton — Mon, 29 Mar 2021 16:20:23 +0000

Many people think the best and only way to interact with a cloud vendor is via the web browser.

I do not disagree that it is a great method, but it isn't the only way and in some cases it might not be the best way. This article will outline the 3 main ways you can interact with IBM Cloud services. If you don't already have an account, you can sign up for a free tier Lite account and try them for yourself.

1 - Web browser cloud management console
The web browser is great for visualising what you are doing in your cloud account. Here you can visit your account dashboard with an overview of active and inactive services, keep up to date with recent IBM news and gain access to quick start templates. By using this method, you will feel more in control of what you are doing and therefore put aside any axiety of spending cloud credits by accident or starting the wrong service with the wrong resources.

Pros:

Visualise your workload
Feel more in control
Easily find quick start guides
Easily calculate costs with available tools

Cons:

Time consuming
Repetitive tasks become arduous
Can take time to get used to the console layout

Example use case: If you wish to set up a 30 days free Kubernetes cluster and visualise the progress and visit the Kubernetes dashboard this would be a great method of interaction.

2 - Command Line Interface (CLI)
Using a CLI is great for efficiency, once logged into the CLI you have access to IBM Cloud services via the API's. If you are using this method, you may already know what services you are wanting to use and how to use them. That being said, even if you don't you can always find your way through the plain text interface by using the -h flag. I find the best part about cloud vendor CLI's is more often than not they will be available within a Docker container. For example, IBM Cloud Developer Tools. This makes them super portable and doesn't require you to install them onto your local machine. Second to that, some even have a CLI integrated into the browser that allows you to use the CLI through a browser window. If you have an IBM Cloud Account then you can visit and use the IBM Cloud Shell to interact with yours.

Pros:

Great for repetitive tasks (you can write scripts around them)
Can be containerised
Standard layout
Quick implementations

Cons:

Not every product offering will have an API or have a specific command embedded into the released CLI
No visual representation of actions
Usually require some level of command line confidence

Example use case: You are wanting to create a new IBM Cloud Function with an Action and a Trigger to process a batch job on request.

3 - Software Development Kit's (SDK's)
This one is less frequently used and less frequently spoken about. Thus making it the method most often forgotten about. An SDK is a set of tools and libraries that you can directly inject into your own source code of your chosen language to create and communicate between applications. Certain product offerings will have their own specific SDK's. An SDK usually contains many API's. There is an amazing video explaining the difference between API's and SDK's - https://www.youtube.com/watch?v=kG-fLp9BTRo

You can see the IBM Cloud API's and SDK's list showing which product offerings have an associated SDK ready for you to use.

Pros:

Language specific
Lots of SDK's available
Great to import and use straight away

Cons:

Can be difficult to understand
Can be large in size
Sometimes challenging to work out if you need one or not

Example use case: You have written a Go program and want to interact directly with IBM Watson. You can use the Go SDK to do this.

For this example you would install it with the following bash command (this is just the starting point for the purpose of this article and not a complete code example):

$ go get -u github.com/watson-developer-cloud/go-sdk/v2

And to use it, it needs to be imported it into your application code directly:

import "github.com/watson-developer-cloud/go-sdk/v2/servicev1"

service, serviceErr := servicev1.NewServiceV1(&servicev1.ServiceV1Options{
    Version:   "2018-03-05",
})

If you have any questions or want to see more content like this, drop me a line!

GitHub
Twitter
Instagram
LinkedIn

2 Clouds 1 Pipeline

Liam Conroy Hampton — Fri, 05 Feb 2021 18:39:59 +0000

You've probably heard of CI/CD or DevOps. Maybe you have read my Toolchain article or listened to the Tech Jam podcast I co-host.

Build/Test/Deploy your Go App using IBM Cloud Toolchain

Liam Conroy Hampton ・ Sep 24 '20

#cloud #devops #go #beginners

Wherever you have heard it, it is a hot topic and there are so many elements to it. In this blog I will show you how I deploy two microservices into two different clouds. However, I have a plot twist. I am only using 1 pipeline. This is called multi-cloud deployment, a method that has many benefits from cost savings to security and reliability. This is becoming a more popular method of cloud deployment and this article just scratches the surface.

The 3 takeaways from this article:

Learn something new about CI/CD
Learn how to write a Tekton pipeline
Understand how easy it is to deploy into more than one cloud

For context, I have written a client app in Golang and a server app in Node.js. The Go app will submit/POST a piece of data through a form input box and the Node app will receive this and display it. Pretty simple but it does the job.

If you also want to do this, here are some pre-reqs:

IBM Cloud Account
DigitalOcean Cloud Account
GitHub Account
2 microservices (ensure they are both in GitHub or equivalent). Feel free to use mine and just sub in the values throughout the demo

Building a TekTon pipeline

TekTon is purpose build for Knative applications. It handles containers and deployment into Knative environments seemlessly. Whereas you would previously have a Jenkinsfile or travis.yaml in the root of your project you dont neceessarily need this in TekTon (at least for this demo anyway).

A TekTon pipeline provides new resources through Custom Resource Definitions (CRD's). These are:

Pipeline
Task
TaskRuns
PipelineRuns
PipelineResources

A TekTon pipeline can consist of many tasks and within the tasks can be many steps. This will become clearer as we go along.

The most fundamental part to always remember is the pipeline is decoupled and the building blocks are isolated containers.

Step 1 - Set up an IBM and DigitalOcean Cloud Accounts and acquire access keys

Just follow these links to sign up (they are free)

IBM token creation

Sign into the IBM Cloud Managemant Console
Manage -> Access (IAM) -> API keys -> Create an IBM Cloud API key

DigitalOcean token creation

Sing into the DigitalOcean Management Console
Select API from the list on the left
Select the Token/Keys tab
Click Generate New Token

Keep both of these api keys safe and ensure you know which is which. You will not be able to see them again.

Step 2 - Write the pipeline yaml

File 1 - deployment-pipeline.yaml

In this file you can see the main flow of the pipeline. It will firstly execute pipeline-ibm-cloud-install-task and then afterwards it will execute pipeline-do-cloud-install-task.

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: deployment-pipeline
spec:
  params:
    - name: ibmcloudapikey
      description: The IBM Cloud API Key
    - name: doauth
      description: The DigitalOcean API Key
    - name: file
      description: The node app repository contents to download
    - name: uuid
      description: A unique identifier
  tasks:
    - name: pipeline-ibm-cloud-install-task
      taskRef:
        name: ibm-cloud-install-task
      params:
        - name: ibmcloudapikey
          value: $(params.ibmcloudapikey)
        - name: file
          value: $(params.file)
        - name: uuid
          value: $(params.uuid)
    - name: pipeline-do-cloud-install-task
      runAfter: pipeline-ibm-cloud-install-task
      taskRef:
        name: do-cloud-install-task
      params:
        - name: doauth
          value: $(params.doauth)
        - name: uuid
          value: $(params.uuid)

File 2 - ibm-cloud-install-task.yaml

This is a Task and it has a single step. The primary image declared is ibmcom/ibm-cloud-developer-tools-amd64:1.2.3 and it has the ibmcloud cli pre-installed and set up. First it downloads a zip of my project from GitHub. Secondly, it uses the ibmcloud cli to authenticate my IBM Cloud account and pushed my project up to CloudFoundry.

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: ibm-cloud-install-task
spec:
  params:
    - name: ibmcloudapikey
      type: string
    - name: file
      type: string
    - name: uuid
      type: string
  steps:
    - name: login-and-deploy-function
      image: ibmcom/ibm-cloud-developer-tools-amd64:1.2.3
      script: |
        #!/usr/bin/env sh
        apk add zip
        mkdir -p ./application
        wget $(params.file) -O nodeapp.zip
        unzip ./nodeapp.zip -d ./application
        rm nodeapp.zip
        echo $(params.uuid)
        cd application/microservice-node-demo-main

        ibmcloud login -a cloud.ibm.com -r eu-gb -g Default -apikey $(params.ibmcloudapikey)
        ibmcloud cf install
        ibmcloud target --cf
        ibmcloud cf push liams-node-app -m 128M
        echo PUSHED TO IBM CLOUD CF

Be sure to change the name of your app by changing the line ibmcloud cf push.

File 3 - do-cloud-install-task.yaml

This Task is using a vanilla Alpine image, installing the DigitalOcean CLI (doctl), creating an app spec (a deployment .yaml file DigitalOcean expects) and then finally deploying the app. The app spec consists of the constraints I wish to impose on my application upon deployment. As you can see, I have hard coded an environment variable (APPROUTE) for my application to use to connect to my other app deployed in IBM Cloud. This could be subsituted with a parameter but this is just a basic demo.

In the code below, make sure you copy your own values into the bits that need changing (repo_clone_url and APPROUTE value) and tailor it to your applications needs. DigitalOcean App Spec.

If you are using my repositories just sub in the values.

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: do-cloud-install-task
spec:
  params:
      - name: doauth
        type: string
      - name: uuid
        type: string
  steps:
    - name: login-and-deploy-function
      image: alpine:3.13.0
      script: |
        #!/usr/bin/env sh
        echo pipeline-uuid-$(params.uuid)
        apk add wget
        wget https://github.com/digitalocean/doctl/releases/download/v1.54.0/doctl-1.54.0-linux-amd64.tar.gz
        tar xf doctl-1.54.0-linux-amd64.tar.gz
        mv doctl /usr/local/bin
        mkdir -p ./appspec
        cd appspec/
        touch client-go-app.yaml
        ls -la
        cat >> client-go-app.yaml <<EOF
        name: client-go-app
        services:
          - name: go-app
            git:
              repo_clone_url: <your-github-project-url-to-deploy-into-digital-ocean>
              branch: do-deploy
            build_command: go build -o goapp
            run_command: ./goapp
            envs:
              - key: APPROUTE
                value: https://<your-ibmcloud-app-name>.eu-gb.mybluemix.net
            instance_count: 1
        EOF
        pwd
        doctl auth init -t $(params.doauth)
        doctl apps create --spec client-go-app.yaml
        echo App deployed to DigitalOcean

As it stands this will not work and you'll find out why shortly

Once these files have been created, create a new repository in GitHub and push these files into it.

Step 3 - Build your Toolchain in IBM Cloud

Login to your IBM Cloud account
Navigate to the DevOps section in IBM Cloud using the navigation menu on the left (its about half way down the list)
Select your region (I chose London) and then click on Create toolchain
Scroll down to the bottom and select "Build your own toolchain"
Give it a name and leave the region and resource group as the defaults

You have now created a foundation to build on. At this point your toolchain will be empty, so lets fill it.

In the top right, click on the button Add tool and search for GitHub. This will be used to access the microservices in GitHub or an equivalent server.

Fill out the details:

Server - Where the projects lives (default is GitHub).
Repository type - Change this to Existing since your projects already exists.
Repository URL - This is the HTTPS project URL (not the .git URL).
Integration Owner - This is your GitHub username. You may be propted to allow Toolchain access to your GitHub profile. Once connected, it will auto-populte.

Ignore the checkboxes Enable GitHub Issues and Track deployment of code changes as we do not need them for this demo.

Repeat these steps and add a new GitHub tool in the Toolchain for each GitHub repository. 1 for each microservice and 1 for the Tekton pipeline repository.

Once you have done this, add one final tool. Thats right, FINAL tool! Click Add tool and search for Delivery Pipeline. This is the workhorse in this Toolchain.

Select it and proceed to fill out the following:

Pipeline name - Give it a unique name
Pipeline type - Here you have 2 options. Classic, which provides a graphical interface and allows you to deploy your applications into CloudFoundry and Tekton, a cloud native CI/CD pipeline tool. Select Tekton from the list.

Now click on Create Integration.

If you get an error box saying "Continuous Delivery service required" dont panic. In the main IBM Cloud search bar, search for continuous delivery and select "Continuous Delivery" - You only need to do this once!
Its very simple to set up this service. Make sure the region is set to the same as your toolchain (mine is London), give it a sensible name such as "toolchain delivery service" and click Create. Now head back to your Toolchain and the error should be gone.

It should now look like this:

Step 4 - Configure the Tekton pipeline

Click on the Delivery Pipeline tool and this will take you to the TektonPipeline Configuration dashboard.

On the left side of the screen you will see a list. These are as follows:

Definitions - This specifies the pipeline code to be run upon a trigger. We created 3 Tekton .yaml files in Step 2. Click on Add and you will be given a popout. For the Repository select the Tekton pipeline repository that has the .yaml files we created earlier in. Chose the main branch in the next box and leave Path blank if you have left the files in the root directory of the project repository. Click Add. Once the code has loaded in, click on Validate, wait for this to finish and then click Save.

Worker - This is essentially how your pipeline will run. You can have dedicated private workers or ones that from a shared pool of resources. We will just use a shared resource so select IBM Managed workers (Tekton Pipelines vx.xx.x) in LONDON and click Save.

Triggers - These specify what happens when an event occurs. More often than not, they will be event triggers from a webhook on GitHub, such as a pull request or a new commit or merge. However in this blog we will be using a manual trigger. We'll set this up shortly.

Environment properties - Key:Value pairs that can be used from within your pipeline. These are referenced as $(PARAMS) in the .yaml files. In here we need to Add 4 evironment properties for this pipeine to run correctly and they are:

Key	Type	Value
doauth	Secure	DigitalOcean api key
file	Text	.zip url of 1 project repsitory (mine is my node project)
ibmcloudapikey	Secure	IBM Cloud api key
uuid	Text	Unique identifier e.g 1234

.zip url = url + /archive/main.zip for example: https://github.com/liamchampton/microservice-node-demo/archive/main.zip

Other settings - Use these settings for more granular control over builds. For this demo we will not be touching these.

Now everything has been set up. The pipeline code has been imported, the workers have been set and the evironment properties have been set. Seems like it is good to go.. right? Try and click on Run Pipeline in the top right of your screen.

PSYCH! Got cha! This will not work because we have not got a trigger. It doesn't know what to do when that button is clicked so we need to tell it.

Like I mentioned, it would usually be a trigger from an event that happens on a repository, such as a pull request or merge but we will create a manual trigger.

We need to revisit our pipeline code repository and add some more files. I have named these hello-xxx.yaml to give a sense I am envoking a new conversation with it but you can call them whatever you like, just make sure to change the corrosponding code.

File 4 - hello-lister.yaml

apiVersion: tekton.dev/v1beta1
kind: EventListener
metadata:
  name: hello-listener
spec:
  triggers:
    - binding:
        name: hello-trigger-binding
      template:
        name: hello-trigger-template

File 5 - hello-trigger-binding.yaml

apiVersion: tekton.dev/v1beta1
kind: TriggerBinding
metadata:
  name: hello-trigger-binding
spec:
  params:
    - name: ibmcloudapikey
      value: $(event.ibmcloudapikey)
    - name: doauth
      value: $(event.doauth)
    - name: file
      value: $(event.file)
    - name: uuid
      value: $(event.uuid)

File 6 - hello-trigger-template.yaml

apiVersion: tekton.dev/v1beta1
kind: TriggerTemplate
metadata:
  name: hello-trigger-template
spec:
  params:
    - name: ibmcloudapikey
      description: The IBM Cloud API Key
    - name: doauth
      description: The DigitalOcean API Key
    - name: file
      description: The node app repository to be downloaded
    - name: uuid
      description: A unique identifier
  resourcetemplates:
    - apiVersion: tekton.dev/v1beta1
      kind: PipelineRun
      metadata:
        name: pipelinerun-$(params.uuid)
      spec:
        pipelineRef:
          name: deployment-pipeline
        params:
          - name: ibmcloudapikey
            value: $(params.ibmcloudapikey)
          - name: doauth
            value: $(params.doauth)
          - name: file
            value: $(params.file)
          - name: uuid
            value: $(params.uuid)

Push these new files into your main branch on GitHub and inside your pipeline dashboard in IBM Cloud, go to Definitions -> click the 3 dots on the repository already imported -> Edit -> click on the Update button -> Validate -> Save

Head to the Triggers page and then click Add trigger. Select Manual. You should only have 1 EventListener to choose from and it should be called hello-listener. On the Worker box, select the option Inherit from Pipeline Configuration as this will use the shared pool or works, just like we set up earlier. Click Save.

Wahoo! You have finished setting up your Tekton pipeline!

Now all thats left to do is run it.. click Run Pipeline, enter a uuid value for the pipeline run and click Run. Watch the magic happen 🎉

My challenge for you

Create a new Task to clean up each deployment at the end of the pipeline run and let me know how you got on 😄

If you have any questions or want to see more content like this, drop me a line!

GitHub
Twitter
Instagram
LinkedIn

Code Engine - A developers dream!

Liam Conroy Hampton — Mon, 23 Nov 2020 17:50:18 +0000

Code Engine? What is Code Engine? This is a slick new tool that IBM Cloud has released into Beta. It takes your already containerised app OR source code (that's right, it takes both), and then deploys that into the wild. This is literally a developers dream. You code it, they deploy it. It really is that simple. Look, I'll show you the only steps you need to take..

1) Sign into IBM Cloud or create an IBM Cloud account
2) Write some code and put it into GitHub OR containerise your application and store it in a container registry - literally anywhere you want
3) Link up Code Engine and watch that puppy deploy

As always, I will be using a Golang application for this tutorial as that's my favourite flavour but you can use anything you like.

Before we start, you need to get an API key set up. Follow these easy steps:

Login or Sign up for a free IBM Cloud account and then head to Manage -> Access (IAM) -> View All of your API keys -> Create an IBM Cloud API key

Give the key a name and description and MAKE A NOTE OF THE KEY. You will not see this key again and you need it later.

Now for the juicy stuff.

Head to the search bar and type in Code Engine and select the catalogue option as shown below

Now let's be real, playing with containers is great but not everyone knows how to containerise their app or know the ins and outs of containerisation. Since lots of developers just deliver code, we will select the option to "Run your source code". All this needs is a github link and Code Engine does the heavy lifting here.

Select this option and click on "Start creating ->"

Next you will be prompted to select "Application" or "Job".

Application - A project you intend to keep alive, such as a HTTP server
Job - A block of code you intend to run to completion, such as a function

For this tutorial, we are going to select "Application" as we want to make a HTTP server. Beneath that option, you will be promped to create a project. A project is what houses this deployed application. Click on "Create" and fill in the details on the panel that pops out and then click "Create" to make the project (shown below).

Now we can carry on with the application. Give it a name, select "Source Code" because that's what we are using, specify the .git repository url and then click on "Specify build details" - this is the most important part.

Note I am using a simple golang http server I had in GitHub - feel free to use it too

A window will pop out and the details will be pre-configured - only change these if you need to specify something other than the root of the directory and a different branch.

Next, on the "Strategy" stage you will be asked how you would like this built. Dockerfile or Cloud Native Buildpack? This might seem a bit confusing but fear not, this is just how you want the code to be bundled up and run. For ease, we will just leave it pre-configured as the Dockerfile.

Finally, the output stage. If you do not already have a container registry / repository in place you need to set one up. This is just so that Code Engine has a place to store the built image of your source code.

It might look a little garish and something like this

Click on "Add" to add a registry and you will the be presented with a bunch of fields to fill out that look like this..

Now, lets take a second to walk through these bits

Registry name - This is the name you will identify the registry with, you can choose whatever you like. As seen above I have chosen "http-project-registry"
Registry server - The eagle eyed reader might have noticed that this project is based on a server in Dallas. At the point of writing this, Code Engine is in beta meaning Dallas is the only available deployment location. I am using IBM Cloud container registry so my sever address is us.icr.io. You can choose Dockerhub or any other container registry.
Password - If you are not using IBM Cloud container registry, you will need to provide your user/password. If you ARE using IBM Cloud container registry like me, you will only be prompted for the password as the "Username" is pre-filled. Remeber that API I asked you to make earlier? - that API key goes in here as the password.
Email - This is optional, I haven't put it in but there is no harm in specifying a valid email address.

Click "Add" and you will then get a pop out that looks similar to the one below. You may have to enter a repository name if this is your first project. You can call this anything you like.

Click on "Done" and then click on "Deploy".

Wait for the applicaiton to be built and deployed. Click on "Application URL" to view the deployed application.

.. And just like that, you have a cloud native application built from raw source code. As I said earlier, you just need to focus on pushing your code and let Code Engine do the rest!

Now close the browser tab / window that is running the application. Have you spotted anything weird in the project window? There might be some lag to this but you might notice you have "0 instances" running.

This is just another one of those cool perks where you would only pay for what you use on a larger project. Under the covers, Code Engine uses Knative which has this awesome feature of being able to scale down to 0 if no traffic is being received. Checkout the Knative docs to read more about this!

Disclaimer

At the time of writing this, Code Engine is in beta so you may experience fluctuating behaviour and projects are kept alive for only 7 days.

If you have any questions or want to see more content like this, feel free to reach out!

GitHub
Twitter
Instagram
LinkedIn

Get GitHub's service status using Golang on IBM Cloud

Liam Conroy Hampton — Thu, 22 Oct 2020 16:11:37 +0000

How many times have you had a service go down leaving you searching your Twitter feed to find out if it's really down or if it's just you?

Golang has a knack and good reputation for many things but one that stands out like a sore thumb is the low memory footprint.

For this project, it is perfect for a lightweight HTTP server and client. The low data usage makes it compatible for most cloud providers free tier accounts and it compiles to binaries that are cross-platform. This means you can deploy your server to the cloud and share your client binary and it will run on all systems.

In this article I will show you how to create the following elements and connect them:
1) A cloud-hosted Golang HTTP server
2) A Golang client to consume the HTTP server data
3) How to host the server in IBM Cloud Foundry
4) How to build and install the client binary onto your own machine

This is really fun because it can be adjusted for other services really easily, after all it is just providing and consuming JSON data!

What will you need for this?
1) IBM Cloud Lite account
2) IBM Cloud CLI + Cloud Foundry CLI
3) GitHub account + git installed (optional)
4) Golang installed/setup on your machine

Pre-req set up

1) Sign up to IBM Cloud
2) Login/signup to GitHub. This is optional should you wish to push your code to a repository or clone mine. Install git to interact with GitHub
3) Golang installation can be tricky but the official docs are where it's at!
4) Install the IBM Cloud CLI + Cloud Foundry CLI (use command ibmcloud cf install once you have the IBM Cloud CLI installed)

Create a Go HTTP server

First, we are going to create a server to act as a stepping stone endpoint that the client will call instead of going straight to the GitHub API. For this, it might seem a little overkill but if you were to expand on this project it would be very useful.

First, let's make a simple server and get it into IBM Cloud Foundry, OR you can fork / clone this one from my github

In a terminal, navigate into $HOME/go/src/github.com and create your project folder. Call it github-status-server. If you are copying from GitHub, make sure you clone the repository into this directory.

Note - if you do not have this folder tree already (go/src/github.com), then create it and make sure go is installed on your system correctly. To test your installation and check the installed version, enter the command go version into a terminal window.

cd into this new project folder and enter the command go mod init (in some cases you may need to specify the exact project path after init). This will initialise the project to use go modules to help with dependency tracking.

Make a new file in this directory and call it main.go.

Open this project in your preferred editor. I use VSCode or GoLand.

In main.go you will need the following code (if you are copying from GitHub, you will just need to amend the code to suit):

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io/ioutil"
    "log"
    "net/http"
    "os"
)

// Response struct will pick out the elements of JSON we want to display
type Response struct {
    Components []struct {
        Name        string `json:"name"`
        Description string `json:"description"`
        Status      string `json:"status"`
    }
    Status struct {
        Description string `json:"description"`
    }
}

// Setup the HTTP server in one housing function
func SetUpRoutes() {
    // Create the route handler listening on '/'
    http.HandleFunc("/", Home)
    fmt.Println("Starting server on port 8080")

    // Start the server
    http.ListenAndServe(":8080", nil)
}

func main() {
    SetUpRoutes()
}

func Home(w http.ResponseWriter, r *http.Request) {
    fmt.Println("Calling GitHub API for Data")

    // GitHub API
    response, err := http.Get("https://kctbh9vrtdwd.statuspage.io/api/v2/summary.json")

    if err != nil {
        fmt.Print(err)
        os.Exit(1)
    }

    // Pull out the body of the response
    responseData, err := ioutil.ReadAll(response.Body)
    if err != nil {
        log.Fatal(err)
    }

    defer response.Body.Close()

    // Unmarshal the reponse data into the struct specified at the top
    var responseObject Response
    json.Unmarshal(responseData, &responseObject)

    // Turn unmarshalled json into a byte array for the write stream
    responseBodyBytes := new(bytes.Buffer)
    json.NewEncoder(responseBodyBytes).Encode(responseObject)
    byteArray := []byte(responseBodyBytes.Bytes())
    w.Write(byteArray)
}

Tada! We have got a simple HTTP server that will return only the part of the GitHub status API that we have stated in the struct. You can adjust this to taste.

To run this and see it in action, all you have to do is enter the command go run main.go into a terminal from within the root of the project directory. You may be asked to allow a network connection (at least on Mac you might).

If you go to localhost:8080 in your web browser you should see the following JSON output:

{"Components":[{"name":"Git Operations","description":"Performance of git clones, pulls, pushes, and associated operations","status":"operational"},{"name":"API Requests","description":"Requests for GitHub APIs","status":"operational"},{"name":"Webhooks","description":"Real time HTTP callbacks of user-generated and system events","status":"operational"},{"name":"Visit www.githubstatus.com for more information","description":"","status":"operational"},{"name":"Issues","description":"Requests for Issues on GitHub.com","status":"operational"},{"name":"Pull Requests","description":"Requests for Pull Requests on GitHub.com","status":"operational"},{"name":"GitHub Actions","description":"Workflows, Compute and Orchestration for GitHub Actions","status":"operational"},{"name":"GitHub Packages","description":"API requests and webhook delivery for GitHub Packages","status":"operational"},{"name":"GitHub Pages","description":"Frontend application and API servers for Pages builds","status":"operational"}],"Status":{"description":"All Systems Operational"}}

This looks gross, I know! But only the client app will consume this so you don't really need it to be readable. By all means, feel free to make it readable, nicely formatted JSON.. make it a personal challenge.

Push the server into IBM Cloud Foundry - its so easy!

Make sure you are signed up to IBM Cloud and have the CLI installed.

Once it is installed, make sure you are logged into IBM Cloud via your terminal. You can log in with the simple command ibmcloud login or if you have a federated ID use the command ibmcloud login --sso. Signup if you don't already have an account.

After you have logged in, make sure you also have the Cloud Foundry CLI installed. To install it, just use the command ibmcloud cf install. This will allow you to interact with Cloud Foundry quick and easily through the terminal.

Now all you need to do is target Cloud Foundry. Again, this is easily done with the command ibmcloud target --cf. Now you are able to interact with IBM Cloud Foundry.

It's time to push the app up. To do this, use the command ibmcloud cf push github-status -m 128mb

Broken down, ibmcloud cf push will connect to Cloud Foundry through IBM Cloud and push up your code. Using a build pack and the name you give it, in this case I called it github-status, it builds and creates a service for you. The -m 128mb just states the amount of memory you wish to allocate to this service. Since it is a basic HTTP server you do not need much memory at all.

Create a Go (or a node, python, swift etc..) Client for local system use

This is the magic of microservices, they don't all have to be the same language, they just need a common way to chat to one another. For this example I am using Go so you can see how easy it is for you to use it from anywhere on your local system environment.

Just like before, in a terminal navigate into $HOME/go/src/github.com and create your project folder. Call it get-github-status.

You guessed it.. make a new file in this directory and call it main.go and open this project in your preferred editor.

In main.go you will need the following code (you may notice it is very similar to the server code). The main difference is that this will make a single request when you want it to, not just sit running like the server does. Make sure you change the endpoint link in the first line of the main() function to that of your Cloud Foundry route. This will be found in the terminal output after your push up to the cloud.

My example is below:

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io/ioutil"
    "log"
    "net/http"
    "os"
)

type Response struct {
    Components []struct {
        Name        string `json:"name"`
        Description string `json:"description"`
        Status      string `json:"status"`
    }
    Status struct {
        Description string `json:"description"`
    }
}

func main() {

    // Calling the hosted HTTP server
    // Edit this with your route
    response, err := http.Get("https://YOU-ROUTE-ENDPOINT")

    // Check the request didn't return an error
    if err != nil {
        fmt.Print(err)
        os.Exit(1)
    }

    // Pull out the body of the response
    responseData, err := ioutil.ReadAll(response.Body)
    if err != nil {
        log.Fatal(err)
    }

    defer response.Body.Close()

    // Unmarshal the data into the format given in the struct
    var responseObject Response
    json.Unmarshal(responseData, &responseObject)

    // Pretty print the JSON making it easier to read
    responseBodyBytes := new(bytes.Buffer)
    json.NewEncoder(responseBodyBytes).Encode(responseObject)
    byteArray := []byte(responseBodyBytes.Bytes())
    byteBuffer := &bytes.Buffer{}
    if err := json.Indent(byteBuffer, byteArray, "", "  "); err != nil {
        panic(err)
    }
    fmt.Println(byteBuffer.String())
}

Wahoo 🎉

You now have a server providing data and a client to consume it.

Run the client locally using the command go run main.go and you should hopefully see the following output:

{
  "Components": [
    {
      "name": "Git Operations",
      "description": "Performance of git clones, pulls, pushes, and associated operations",
      "status": "operational"
    },
    {
      "name": "API Requests",
      "description": "Requests for GitHub APIs",
      "status": "operational"
    },
    {
      "name": "Webhooks",
      "description": "Real time HTTP callbacks of user-generated and system events",
      "status": "operational"
    },
    {
      "name": "Visit www.githubstatus.com for more information",
      "description": "",
      "status": "operational"
    },
    {
      "name": "Issues",
      "description": "Requests for Issues on GitHub.com",
      "status": "operational"
    },
    {
      "name": "Pull Requests",
      "description": "Requests for Pull Requests on GitHub.com",
      "status": "operational"
    },
    {
      "name": "GitHub Actions",
      "description": "Workflows, Compute and Orchestration for GitHub Actions",
      "status": "operational"
    },
    {
      "name": "GitHub Packages",
      "description": "API requests and webhook delivery for GitHub Packages",
      "status": "operational"
    },
    {
      "name": "GitHub Pages",
      "description": "Frontend application and API servers for Pages builds",
      "status": "operational"
    }
  ],
  "Status": {
    "description": "All Systems Operational"
  }
}

Providing you have set up the Golang environment correctly on your system, you should now be able to build/install a binary of the client. This will allow you to call the program from wherever you are on your system.

An example use case - you are developing a really cool app and you need to push your code up to GitHub but you are getting an error in your terminal. Well, now you can use your chosen command to call this program instead of having to search the internet for a service status 😉

To do this, you simply use the command go install. This will build a binary and store it within the directory go/src/bin. Providing this is on your PATH you can call it from anywhere.

Official docs specify this. To see the full explanation use the command go help install in your terminal from within your project.

usage: go install [-i] [build flags] [packages]

Install compiles and installs the packages named by the import paths.

Executables are installed in the directory named by the GOBIN environment
variable, which defaults to $GOPATH/bin or $HOME/go/bin if the GOPATH
environment variable is not set. Executables in $GOROOT
are installed in $GOROOT/bin or $GOTOOLDIR instead of $GOBIN.
etc...

Here is my example of me calling it from my $HOME directory:

➜  ~ get-github-status
{
  "Components": [
    {
      "name": "Git Operations",
      "description": "Performance of git clones, pulls, pushes, and associated operations",
      "status": "operational"
    },
etc...

If you have issues calling the binary from outside of this directory, use the following command and try again:
export PATH=$PATH:$(go env GOPATH)/bin

And there you have it, a simple example of 2 microservices running in a hybrid like scenario. Neither one is dependant on the other. You can switch them up and modify this to fit your needs.

For example, the following code is for a python client:

import requests
import json

r = requests.get('https://YOU-ROUTE-ENDPOINT')

pretty_json = json.loads(r.text)
print(json.dumps(pretty_json, indent=2))

If you have any questions or want to see more content like this, feel free to reach out!

GitHub
Twitter
Instagram
LinkedIn

Alternatively, you can see what my team get up to and our other projects on our website IBM Developer UK

Build/Test/Deploy your Go App using IBM Cloud Toolchain

Liam Conroy Hampton — Thu, 24 Sep 2020 12:45:08 +0000

Whether you're a project manager, part of a dev team, or flying solo, there is always the looming issue of incremental changes to your project. Gone are the days of waterfall development - Agile is the 'in' thing. You are running fortnightly sprints and releasing on a regular basis. You've probably seen a lot of options for deployment methods, lots of continuous integration/continuous delivery (CI/CD) pipelines, and much more.

On the face of it, this is a journey for getting the application on your local system deployed into the cloud in stages using a pipeline.

A pipeline allows you to work on small features independently and deploy them one by one without taking down the application, or better yet, prevent you from having to write the application all in one go and then deploying it in one big push. This will allow you to fail small and fail fast. In return, leaving less technical debt and allowing faster development.

The diagram below outlines the aim of this using a simple pull request example

Now you can see how simple it really is, in this blog, I will show you how to build, test, and deploy your application under one roof in a matter of minutes using IBM Cloud Toolchain. That's right.. in a matter of minutes.

But first, you will need some prerequisites:
1) IBM Cloud Lite account - sign up here
2) GitHub account
3) Golang installed on your machine

Let's get started!

Create your Go app and get it into GitHub

Step 1 - Create an empty repository in GitHub

Navigate to your GitHub profile, click on the + in the top right corner and select "New repository". Give it a name and make sure it is public. Feel free to also add a license .gitignore file and a README.md if you like

Step 2 - Let's create a simple Golang HTTP Server with an accompanying test

In a terminal clone your new repository into $HOME/go/src/github.com

Navigate into the cloned project using the cd command and initiate go modules with the command go mod init (in some cases you may need to specify the exact project path) and then open it in your preferred editor (I use Visual Studio Code)

Note - if you do not have this folder tree already (go/src/github.com), then create it

Create a test file in the root of the project (main_test.go):

package main

import (
    "net/http"
    "net/http/httptest"
    "testing"
)

func TestHomeRoute(t *testing.T) {
    t.Run("returns message", func(t *testing.T) {
        request, _ := http.NewRequest(http.MethodGet, "/", nil)
        response := httptest.NewRecorder()

        Home(response, request)

        got := response.Body.String()
        want := "This is a toolchain test PR"

        if got != want {
            t.Errorf("got %q, want %q", got, want)
        }
    })
}

Run the test and watch it fail using the following command in your terminal
go test ./...

Now it is time to write just enough code for the test to pass (this is called TDD - test-driven development)

Create a main file in the project root (main.go). This will be home to the sever code

package main

import (
    "fmt"
    "net/http"
)

func main() {
    // Create the route handler listening on '/'
    http.HandleFunc("/", Home)
    fmt.Println("Starting server on port 8080")

    // Start the sever
    http.ListenAndServe(":8080", nil)
}

func Home(w http.ResponseWriter, r *http.Request) {
    // Assign the 'msg' variable with a string value
    msg := "This is a toolchain test PR"

    // Write the response to the byte array - Sprintf formats and returns a string without printing it anywhere
    w.Write([]byte(fmt.Sprintf(msg)))
}

Run the test command go test ./... and your test should pass

To see this in action, you can run the program with the go run main.go command and navigating to localhost:8080 in your web browser

To quit the program use the commands control + C in your terminal

Step 3 - Push your code into the GitHub repository

Let's get this code into GitHub.. Don't worry it's pretty simple, just follow the following commands

git add . - this command will add all of the new/modified files into staging ready to be committed

git commit -m "initial commit" - this command commits the changes ready to be pushed with a message so they can easily be identified

git push origin master - this will push the changes into the master/main branch - this is fine for an initial commit but going forward you will use feature branches

Awesome - you now have a simple Golang HTTP server with a test that you can push to any cloud, but what if you wanted to branch out and have others work on this? What if you had to add incremental changes and automate the deployment of it? - manual deployments need precision, they are time-consuming and weight on any chain for a team!

Let's use IBM Cloud Toolchain for this.
"What is IBM Cloud Toolchain?" you ask...

Well, that's quite simple. It is a tool that allows you to concatenate actions, called stages. Each stage will have 'jobs' and these allow you to choose what you want to do with your application in that stage, be it build, test, or deploy - heck, why not all of them?

In this blog, we will run a test stage to ensure the code compiles and the tests pass on the pull request. Then once the pull request is merged into the main production branch another stage will compile the code, run the tests, and then automatically deploy the application into Cloud Foundry. Sound good? - let's go!

Creating a toolchain

Navigate to the DevOps section in IBM Cloud using the navigation menu on the left

Select your region and then create a new toolchain

Scroll down to the bottom and select "Build your own toolchain"

Choose an appropriate name and select the correct region for your toolchain

Cool, you now have the foundation - add a new tool to your toolchain

We want to have GitHub integration so search for "GitHub" and select the GitHub tool

Set up the GitHub tool to have the correct information for your project repository, as an example, mine is

You should now see that the GitHub tool on the toolchain overview has ticks and the word "configured". This will give it access to the repository information (you may be asked to confirm this on GitHub)

Next, you need to add a pipeline. On the toolchain overview, just like you did with the GitHub tool, select "Add tool" and search for "pipeline". Select "Delivery Pipeline"

Configure the pipeline by giving it a name and ensuring the pipeline type is "Classic" and not "TekTon"

If you haven't used a pipeline in IBM Cloud before, you may see the following error. You just need to add a continuous delivery service which will enable the CI/CD

Search for continuous delivery and select "Continuous Delivery" - You only need to do this once!

Setup the service. Make sure the region is set to the same as your toolchain, give it the same name as what you called your pipeline service. In this example, it is "CI-CD-Service". Once filled out, create it

Now head back to your toolchain and click on the Pipeline box itself. You will be taken to a blank page. In the top right select "Add Stage"

Call it the "Test Stage" and this will build the code and then run the tests. Make sure the input type is the git repository for your project, the branch is set to "master" and you want this stage to run "When a pull request is opened or updated"

Once the "Input" has been set up, move along to the "Jobs" and create 2 jobs. 1 x Build and 1 x Test. There is no Golang buildpack in the toolchain so we need to use a docker image with the environment set up for us

Add the following script into the build script box on the "Build" stage and modify it to suit your project. I called my directory "gohttpserver" but you can change it to be the name of your repository/project. This script just moved the code from a temporary directory on the virtual machine into the directory tree much like you would have locally on your machine. It then builds a binary to test the code compiles

#!/bin/bash
pwd
ls -la
cd $HOME
mkdir -p go/src/github.com
mkdir -p go/pkg
mkdir -p go/bin
export GOPATH=$HOME/go
export PATH=$PATH:$GOPATH/bin
cd $HOME/go/src/github.com
cp -ar /home/pipeline/* /root/go/src/github.com
mv ./* gohttpserver
cd gohttpserver
go build -o serverbinary
chmod -v +x serverbinary
pwd
ls -la
echo code compiled successfully

Move onto the "Test" stage and do the same. Copy the script from your "Build" stage and modify the end of it to look something like this

#!/bin/bash
pwd
ls -la
cd $HOME
mkdir -p go/src/github.com
mkdir -p go/pkg
mkdir -p go/bin
export GOPATH=$HOME/go
export PATH=$PATH:$GOPATH/bin
cd $HOME/go/src/github.com
cp -ar /home/pipeline/* /root/go/src/github.com
mv ./* gohttpserver
cd gohttpserver
go test ./...
echo test run successful

This is just running the tests within the repository and not building a binary

That is the Build/Test stage complete, now we need to create the deploy stage. This stage will run when the pull request is merged into the main/master branch

Go back to the stages overview and clone the first stage we created by clicking the cog on the stage and selecting "Clone stage"

Click the cog on the newly cloned stage and click "configure". Rename it to "Deploy" and we want this stage to be triggered by a new commit into the master/main branch and not on a new pull request. On the "Input" tab, check the box "When a commit is pushed"

On the "Jobs" section, add a new "Deploy" job and fill out the details for your project. You want to make sure the deployer type is Cloud Foundry (you can choose Kubernetes if you wish but the setup will be different). You will also need to create a new API key for the integration to work (shown below)

Make sure the region selected is the same as your toolchain and the application name is the same as your pipeline. The script is very simple and should be pre-populated for you. Once completed, save and exit

Your toolchain is now set up!

Time for a code change...
First, within your project, checkout a new local feature branch using the following command in a terminal
git checkout -b test-new-toolchain

Next, change the test file main_test.go

And then change the corresponding code in main.go and push the code up to the new remote branch using the commands seen below

Now you can open a pull request in GitHub. Ensure the pull request is from the new branch you have just created, give it a meaningful title and description. Then click "Create pull request"

Once open, you will see the IBM Cloud toolchain integration. If you check the toolchain stages you will see it running the first stage you called "Test Stage"

Once the test stage has finished, you will be able to merge the pull request. Go ahead and merge it. Then check the toolchain and you will see

Once the build stage has finished, click on the link "View logs and history". Scroll to the bottom and you will see the result of the build and where it is deployed in Cloud Foundry. Navigate to the route in a browser and you will see your app running

If you check the merged pull request, you will see a message of successful deployment and a tag from the toolchain

You've now got a CI/CD IBM Cloud Toolchain set up.. how easy was that?!

Although this blog uses a Docker image for the Golang environment, many other projects can be built using the readily available buildpacks in IBM Cloud (a list of these can be found in the stage configurator)

If you have any questions or want to see more content like this, feel free to reach out!

GitHub
Twitter
Instagram
LinkedIn