DEV Community: Juell

Docker: An easier way around

Juell — Thu, 05 Nov 2020 09:40:54 +0000

Containers & Softwares

Containers are all about Software!
Traditionally we use the following process to run software:

Find the software, usually a standalone web site.
Download the software, usually a zip file or some sort of installer.
Then we install the software, often extracting a zip file or running an installer.
Then we run the installed software.

You can learn a lot about containers by relating them to the process above. Here's what it looks like to run software with containers:

Find the software, on Docker Hub.
Download the software with docker pull, comes down as an image which is much like a zip file or msi installer. An image is an application packaging format.
Instead of installing the software, we create a container. So, a container--a stopped container--is like installed software. Docker unpacks the image onto the computer, creating a container. Note: if you just want to create a container, you can use docker create.
Then we run the container which is exactly like running an exe. It's the same thing under the covers!!!
We often use docker run to orchestrate all of these steps with one command, how convenient!
docker exec can be thought of as running another copy of our installed software, like when we launch an executable twice. For example, two copies of Microsoft Word. Or with MongoDB, we might run two mongo clients. After a container is created and running, we can use docker exec to run multiple applications, or multiple copies of the same app, inside the container.

Why Docker?

With traditional softwares, we are faced with many challenges, such as:

Software Discovery

Problems:

Where to get software?
- App stores
- Package Managers
- Standalone Websites
- Software Stats and Metadata
- Trust and Security
- Download Availability
- Payment & Licensing

Solution:

Where:

With Docker, we have just one place to go to get our software: Dockerhub.com. Here you can find any popular software repository that holds images you can download and use.

Stats and Metadata:

Dockerhub provides download stats and data on how many people likes a software. Softwares have documentation and other instructions on use. There is provision for comments too from images users.

Trust and security:

Dockerhub has what's called Official Images that Docker Inc certifies. On the Tags page of these images you can find the result of security scans on these images, showing discovered vulnerabilities, if any.

Download Availability:

DockerHub can be relied upon to provide access to our software images for download. No need to search the whole internet for a simple software.

Payment and Licensing:

DockerStore is a new way of accessing and paying for commercial images.

Software Installation:

Problem:

Is it compatible? Is it cross-platform?
What format is it?
How to install?
What did it install?
Updates/uninstall

Solution:

Compatibility:

With Docker you no longer need to worry if software is cross-platform as you can run Linux containers on windows and vice-versa.

Format:

This is all taken care of in the Image for you. You can decide to peek into how the image was built by visiting Dockerhub page and inspecting the Dockerfile.

Installation:

No more lengthy instructions on how to install a software. Just Docker Run.

What was installed:

The publicly available Dockerfile can be access to find out what exactly was installed in a container.

Updates/Uninstall

Docker remove command is used to uninstall software. As for updates, you can easily pull down an updated image. Images are tagged, making its update and versioning consistent and easier to understand and follow.

All other problems such as documentation, where its installed, starting service, stopping application, licensing, installing dependencies, and security, are equally taken care of by docker. Docker run starts all needed dependencies and performs all needed configs.

Breaking Changes

This used to be a huge one. An application update, operating system patch or upgrade of a shared library meant an end to a properly function software in the past. No more. Docker solves this as well.

Installation

Docker’s installation is very straightforward.

Visit: https://docs.docker.com/get-docker/
Click on your host type, Mac, Windows or Linux
Follow the simple installation instruction to get started.
On Ubuntu, the installation can be as simple as running:

Sudo apt install docker.io

Make sure your machine has support for virtualization. If you’re running a windows machine, ensure it has Hyper-V as this is the technology Docker leverages to function. You might have to turn-on the support in the machine’s BIOS.

Working with Docker

Common commands

	Docker Command
After installation, use the following to check the success of the process:	Docker version Or Docker info
To start a container	Docker start < container-identifier>
To view all running containers in docker, you use the Process Status command:	Docker ps
To view all containers running or not:	Docker ps -a
To view all images, use:	Docker images
To run a container, you use:	Docker run
To start a container	Docker start
To stop a running container:	Docker stop
To uninstall a container:	Docker rm
To remove an image:	Docker rmi
To run another instance of an image:	Docker exec
To create a container	Docker create < container-identifier>
To search for an image	Docker search
To find all details about a container or image	Docker inspect

Docker Demystification

Enter the following docker command into your terminal

docker run msoap/ascii-art cowsay 'Hello'

If you’re running this for the first time, this will:

Download the software Image of Ascii-art from the msoap repository in Dockerhub.
When this process is done, it’ll extract the files it just downloaded,
Docker will the create a container and start up the application
The ‘cowsay’ argument of the Ascii-art app takes a string input that it will display.

Attaching the “-rm” tag after the run, removes the container whenever it is done running.

Run the following to test:

docker run --rm -it wernight/funbox asciiquarium

To close the animation/container press Ctrl + C.

Inverted Learning

Execute this in your terminal:

Docker run -p 80:80 nginx

The -p flag specifies the port Docker should forward from the container it’ll launch to which port in your host machine.

This command pulls down Nginx server in layers. Since this comes in a compressed form to save bandwidth, docker then extracts them to save time in subsequent runs. Finally, docker starts the Nginx server and does the port forwarding.

A simple process like installation and configuration of Nginx server can take a considerable length of time, and require some level of expertise. With Docker, all you have to do is run a line of command and the process is set up. This is called Inverted Learning.

With Inverted learning one does not need to understand a technology to use it. With the Nginx example above, with just one line of code, you can have Nginx up and running, even without knowing how to set it up traditionally.

This enables people to preview/use software and decide if they want to invest the time to learn it, saving wasted time.

Images, and containers

In a Windows parlance, an image can be likened to a downloaded software in say, .zip format. By extracting this software, we can access the ‘.msi’ or ‘.exe’ we can use to install our software. The same with docker. To download a software we use:

Docker pull <image-name:tag>

This downloads the file to our machine. If we do not specify a tag in our image identifier, the ‘latest’ tag is automatically used. I.e. .

When we execute:

Docker run <image-name>

docker looks for the specified image locally first, and on locating it, it’ll extract it, create a container and run the app image inside it.

A container is essentially a running instance of an image, the same way a running application is an instance of its installer file. Traditionally, we can’t install an application without its installer. In docker, when we execute the run command with an image name, docker first searches locally in our machine for the image, if not found, it proceeds to search Dockerhub. If the image is found, it’ll download, extract, install and run the image.

Just as we can close a running application, we can also terminate a running container with:

Docker stop <container-name>

This doesn’t remove the container but merely stops its execution. To confirm this, you can use the command:

Docker ps -a

This shows you all your docker equivalent of installed applications, both running and terminated, with their running status specified.

If you want to ‘uninstall’ a container, you can use:

Docker rm <container-identifier>

The container identifier can be the ID or name. The full ID doesn’t need to be supplied, just enough to uniquely identify the container is enough. The output of this command is an echo of the affected container(s). To remove all containers, consider using:

Docker rm $(docker ps -aq)

This echoes out all the affected container IDs. The additional -q flag ensures only the container IDs are supplied to the rm command.

Just as uninstalling a software in Windows doesn’t delete its downloaded installer, so also does removing a container doesn’t remove its image. To view all pulled images use:

Docker images -a

To remove the image, you use:

Docker rmi <image-identifier>

The identifier can be the image ID or the repository name. Running this command spits out a lot of outputs. This is because an image is often layered and while removing them, each layer must be deleted for the images to go.

Isolation

Containers take isolation to a whole new level, and understandably so.

Let us step back and examine a running application. A running software is made up of the following layer:

The Application (with supporting runtimes and frameworks)
OS Apps—cmd, powershell, etc
The libraries (dependencies, etc.)
Kernel, drivers
Hardware

When an image is pulled down, it contains the needed files in the User space. This is to ensure the application has access to all libraries, components and dependencies it’ll need. This is what makes the image large most times as it attempts to err on the part of being over careful. And well, sometimes you never know what else an application might be needing, so why not have it all.

Since it’s the Kernel that has full access to the machine’s computing resources, the software makes all request through API calls to the Kernel. The Kernel is responsible for telling the running container all the processes, network or files that it can see. This isolation enable container to be more independent and to not compromise the host system. It’s basically the Kernel lying to containers in order to provide security to the host.

All containers are processes with the same Process ID in a host system. Processes inside a container can only see and interact with processes within the same container.

Drive Mounting

In docker, there are times when we need a container to interact with files in a host machine. This can be to process, access or output data. To achieve this, we use the Drive mounting feature of docker:

Docker run -v c:/users/folder_to_share:/data

This command mounts the c:/users/folder to the container’s /data folder thanks to the -v (volume) flag. This way we can specify some extra command that can perform the desired operation.

Docker run –rm -v c:/Users/folder:/data alpine ls /data

Mounting is the process of adding a reference to a host filesystem for permission to access some host files.

Docker Docs

You can get the docker docs for local reference with:

Docker run -p 4000:4000 -d docs/docker.github.io

The -d flag is for it to run in detached mode. When the process is done, you can access the full docker docs in your browser at http://0.0.0.0:4000

Thanks for your time!

Vagrant-up with ease

Juell — Fri, 23 Oct 2020 20:54:18 +0000

What & why

Vagrant is an open-source tool that allows you to create, configure, and manage boxes of virtual machines through an easy to use command interface. It is a tool for building complete development environments with a focus on automation and easy workflow.

It is used in software development to ensure all team members are building with the same configuration. Not only does Vagrant make it possible to share environments, it also shares code as well. This allows the code from one developer to work on the system of another, making collaborative and cooperative development possible. With Vagrant, you can create virtual machines on-the-fly via a set of reusable configuration files. Developers can track environment configuration changes and share their environment so that others can spin up an identical environment.

Installation

Download the latest version of Vagrant from:

https://www.vagrantup.com/downloads.html

Download the latest version of VirtualBox:

https://www.virtualbox.org/wiki/Downloads

Install and restart your PC. Confirm they are properly set up:

Vagrant -v

Example

Run:

Vagrant init generic/alpine310

Vagrant up

Vagrant

Vagrant Boxes

The basic unit in a Vagrant setup is called a “box” or a “Vagrantbox.” This is a complete, self-contained image of an operating system environment. It is a clone of a base operating system image. Using a clone speeds up the launching and provisioning process.

Adding a VagrantBox:

You can add a box with the command:

vagrant box add ubuntu/trusty64

This downloads the box and stores it locally in your machine.

To see all your OS boxes, use:

Vagrant box list

To make use of a box, you need to configure the Vagrantfile and point it to the box. Make sure you’re at the desired folder. Initialize vagrant with:

Vagrant init trusty64

A “Vagrant” file will be created in the directory. Open It with the command:

sudo nano vagrantfile

Once the Vagrantfile is open, locate and change the “config.vm.box” string from what it is to “ubuntu/trusty64”:

config.vm.box = "ubuntu/trusty64"

To remove a VagrantBox:

vagrant box remove ubuntu/trusty64

VagrantFile Demystification

Vagrantfile is the file that Vagrant uses to specify the configuration of the VagrantBox. It’s a Ruby Script but doesn’t require the knowledge of Ruby to create or understand one. Note that VagrantFile has no extension.

The primary function of the Vagrantfile is to describe the type of machine required for a project, and how to configure and provision these machines. Vagrantfiles are called so because the actual literal filename for the file is Vagrantfile.

Here is a sample vagrant file (you can generate one by executing “Vagrant Init ”):


Vagrant.configure("2") do |config|

 config.vm.box = "trusty64"

 config.vm.box_check_update = false

 # **NETWORK SETTING**

 config.vm.network "forwarded_port", guest: 80, host: 8080

 config.vm.network "forwarded_port", guest: 80, host: 8080, host_ip: "127.0.0.1"

 config.vm.network "private_network", ip: "192.168.33.10"

 config.vm.network "public_network"

 # **SHARED FOLDER**

 config.vm.synced_folder "../data", "/vagrant_data"

 # **PROVIDER-SPECIFIC CONFIGURATION**

 config.vm.provider "virtualbox" do |vb|

 #  # Display the VirtualBox GUI when booting the machine

  vb.gui = true

 # Customize the amount of memory on the VM:

  vb.memory = "1024"

  end

 # **PROVISIONING**

 config.vm.provision "shell", inline: <<-SHELL

   apt-get update

   apt-get install -y apache2

  SHELL

end

“#” is used to comment out a line. These can be uncommented to apply the implied setting.
Versioning: Configuration versions are the mechanism by which Vagrant is able to remain backward compatible. It is specified by the **Vagrant.configure(“version_number”)**
Box: A box can be used on any platform that Vagrant supports to bring up an identical working environment.
Shared folder: Vagrant automatically synchronizes content that is in your project directory with a special directory in the guest (virtual) system
Networking: Vagrant includes options to place your virtual machine on a network. This makes it more accessible. You can create a forwarded port for the guest system, define private networks, public networks, and other more advanced options.
Provisioning: Provisioners in Vagrant allow you to automatically install software, alter configurations, and more on the machine as part of the vagrant up process. Some supported Provisioners includes Ansible and Shell script.

Vagrant Up

After making any modification to a VagrantFile, it is advised you save and commit it to a Version Control system. To apply the changes made, execute this command:

vagrant up

subsequently, to apply any other change while the provisioned VM is running, use the command:

vagrant reload

Vagrant SSH

After the vagrant up has successfully completed, you can connect to the created virtual machine by using an SSH connection, with:

vagrant ssh

This opens a secure shell connection to the new virtual machine. Your command prompt will change to “vagrant@trusty64” to indicate that you are logged into the virtual machine.

Once you are done exploring the virtual machine, you can exit the session with CTRL-D. The virtual machine will still be running in the background, but the SSH connection will be closed.

Vagrant on Azure

Demo

First visit www.portal.azure.com and provision a VM.

SSH into the VM

Install Vagrant on it with:

wget https://releases.hashicorp.com/vagrant/2.2.10/vagrant_2.2.10_x86_64.deb

sudo dpkg -i vagrant_2.2.10_x86_64.deb

Install the Vagrant-azure package with:

vagrant plugin install vagrant-azure

Add the Azure Dummy Box:

vagrant box add azure-dummy https://github.com/azure/vagrant-azure/raw/v2.0/dummy.box --provider azure

Install Azure-cli with:

curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash

Login to your Azure Account with:

az login

Create Azure Service Principal with:

az ad sp create-for-rbac

The output of “az ad sp create-for-rbac” should look like the following:

{

 "appId": "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",

 "displayName": "some-display-name",

 "name": "http://azure-cli-2017-04-03-15-30-52",

 "password": "XXXXXXXXXXXXXXXXXXXX",

 "tenant": "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"

}

The values tenant, appId and password map to the configuration values azure.tenant_id, azure.client_id and azure.client_secret in your Vagrant file or environment variables.

Create a Vagrantfile and add this config:

Vagrant.configure('2') do |config|

 config.vm.box = 'azure-dummy'

 # use local ssh key to connect to remote vagrant box

 config.ssh.private_key_path = File.join(File.dirname(__FILE__), 'azure_private_key.id_rsa')

 config.vm.provider :azure do |azure, override|

 # each of the below values will default to use the env vars named as below if not specified explicitly

 azure.tenant_id = ENV['AZURE_TENANT_ID']

 azure.client_id = ENV['AZURE_CLIENT_ID']

 azure.client_secret = ENV['AZURE_CLIENT_SECRET']

 azure.subscription_id = ENV['AZURE_SUBSCRIPTION_ID']

 end

####### Provisioner #######

 config.vm.provision "ansible" do |ansible|

 ansible.playbook = "playbook.yml"

 ansible.verbose = true

 end

end

Since we specified Ansible as our provisioned, we need to install it on our host machine:

sudo apt update
sudo apt install software-properties-common
sudo apt-add-repository ppa:ansible/ansible
sudo apt install ansible

Set up the ENV variables with:

export AZURE_CLIENT_ID=“***************************”
export AZURE_CLIENT_SECRET=“***********************”
export AZURE_TENANT_ID=“***************************”
export AZURE_SUBSCRIPTION_ID=“*********************”

Generate your SSH key:

ssh-keygen -t rsa  -f azure_private_key.id_rsa

Create a “playbook.yml” that we’ll be using for provisioning:

---
    - name: install nginx on a Azure VM instance
      hosts: all
      become: yes
      become_method: sudo
      gather_facts: yes
      tasks:

        - name: install nginx            
          apt:
            name: nginx
            update_cache: yes
            state: present

        - name: stop nginx service
          service:
            name: nginx
            state: stopped
          register: nginx_down


        - name: delete the default html file
          file:
            path: /var/www/html/index.nginx-debian.html
            state: absent


        - name: copy new file from template folder
          template:
            src: ./index.html
            dest: /var/www/html/index.html
            owner: root
            group: root


        - name: restart nginx service
          service:
            name: nginx
            state: restarted

Create a “templates” folder and add an index.html file. Paste this inside:

<!DOCTYPE html>

<html>
    <head>
        <title>Welcome to DevOps - Powered by Telios</title>

        <style>
            body {
                width: 35em;

                margin: 0 auto;

                font-family: Tahoma, Verdana, Arial, sans-serif;
            }
        </style>
    </head>

    <body>
        <h1>Welcome to Vagrant</h1>

        <p>Powered By Telios</p>

        <p>If you see this page, Our page is hosted on Nginx server on our VM with Vagrant.</p>

        <br />
    </body>
</html>

Return back to the base folder you were with:

cd ..

Now Run:

vagrant up

When the process is done, ssh into the provisioned VM with:

vagrant ssh

Access the localhost to check if our provisioning was successful. We can use curl but lets use lynx for a more formatted output:

sudo apt install lynx -y
lynx localhost

If we get the custom page then the process was a success.

Clean Up Vagrant

Once you are done working on your guest system, you have a few options on how to end the session.

To stop the machine and save its current state-run:

vagrant suspend

You can resume by running vagrant up again. This is much like putting the machine in sleep mode.

To shut down the virtual machine use the command:

vagrant halt

Again, vagrant up will reboot the same virtual machine, and you can resume where you left off. This is much like shutting down a regular machine.

To remove all traces of the virtual machine from your system type in the following:

vagrant destroy

Anything you have saved in the virtual machine will be removed. This frees up the system resources used by Vagrant. The next time you vagrant up, the machine will have to be re-imported and re-provisioned. This is much like formatting a hard drive on a system, then reloading a fresh image.

Thanks for your time!

Questions??

Github Actions: From Scratch to Azure Deployment

Juell — Wed, 21 Oct 2020 17:37:23 +0000

Github Actions Training

Introduction

GitHub is a development platform that enables you to host and review code, manage projects, and build software alongside 50 million developers. It provides some important DevOps features that organizations of all sizes need for their software projects. Be it planning of features, fixing bugs, or collaborating on changes, GitHub is the place where developers gather to make things.

In the October of 2018, Github was added to the Microsoft stack and has served to increase enterprise use of GitHub alongside other Microsoft's developer tools and services.

Github Actions are a relatively new feature to Github. It enables you to build, test, package and deploy your code, right from your repository. It is a task automation system fully baked into GitHub. It came out of beta and reached general availability in November 2019. In any organization, Github Action can significantly scale down the amount of work and time required to build and maintain software projects.

Github Flow: is a workflow that simplifies the way changes are made to a software project. It is branch based. We start off with the Master. When there is need to add a feature, we branch off into a parallel feature branch to add the changes. This is not to disrupt the master. When done, we create a Pull Request for team members review. If approved, it is merged with the Master branch.

Github Actions can massively improve an organization’s Github flow and SDLC.

Setting Up Github Action

Github Actions are YAML files located at the “.github/workflows” of your Github repository, that describes actions or tasks to be taken on your code based off an event.

There are two ways of setting up Github Actions in your repo, and we shall consider both:

From the Actions Tab
Manually Adding a .github/workflows folder

Method 1: Automatic Process

From the Actions Tab:

Create a new Github repo, or use an existing one without Github Actions yet.
Navigate to the Actions tab.
Locate the “Simple Workflow” Github Action and click on the Set up this workflow button.
This would open the Simple Workflow. Click on Commit.
You now have this Github Action in your .github/workflows directory.

Example 1:

# This is a basic workflow to help you get started with Actions

name: CI

# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the master branch
on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # This workflow contains a single job called "build"
  build:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
      - uses: actions/checkout@v2

      # Runs a single command using the runners shell
      - name: Run a one-line script
        run: echo Hello, world!

      # Runs a set of commands using the runners shell
      - name: Run a multi-line script
        run: |
          echo Add other actions to build,
          echo test, and deploy your project.

Field Dymystification

The file is called an Action File. It is a YAML file that specifies the triggers, runners, steps and other details needed to run a job. Lets consider the terms you can find in the above Action file.

Name: Optional - The name of the workflow as it will appear in the Actions tab of the GitHub repository. It is a label.
On: Specifies the event that automatically triggers the workflow file. This example uses the push and pull_request events on the Master branch, so that the jobs run every time someone pushes a change to that branch.
Jobs: This Groups together all the jobs that run in the CI workflow file.
Runs-on: Configures the job to run on a runner. A runner is a VM that has the Github Actions Application runner installed. Here an Ubuntu Linux runner is specified. This means that the job will execute on a fresh virtual machine hosted by GitHub. A runner can be Github hosted or self hosted.
Steps: Steps represent a sequence of tasks that will be executed sequentially as part of the job. Each line nested under this section is a separate action.
Uses: The uses keyword tells the job to retrieve v2 of the community action named actions/checkout@v2. This is an action that checks out your repository and downloads it to the runner, allowing you to run actions against your code.
Run: The run keyword tells the job to execute a command on the runner. In this case, it uses the runner’s shell to echo texts.

A job contains steps with actions, run commands or api calls that is triggered when a workflow runs.

Steps run actions or commands on the same runner, meaning they can share data.

You can target any event and activity type on a repo with the syntax:

 On: 
<event_type>:
   Activity type: [activity1, activity2, …]

The full list of all activities can be found at developer.github.com/v3

You can also schedule a workflow to run at a given UTC time:

On: 
  Schedule:
   - cron: ‘*/15 * * *’

This will trigger a run on the last commit on the default branch.

Method 2: Manual Process

The manual way of adding a Github Action to Github is to add a ‘.Github/workflow’ folder to your new or existing Github repo. Here you’ll create your Action Files.

Github will automatically pick up these action files and run them when the specified even and activity occurs.

Example 2

In the .github/workflows folder, create an action file. Name it ‘Greet.yml’. In the file enter the following.

# your-repo-name/.github/workflows/greet.yml
name: Second Example 
on: 
Push: [ master ]                                                  
jobs:                         
  greet-job:                           
    name: Say hi                           
    runs-on: ubuntu-latest                           
    steps:                           
      - name: Print a greeting                             
        run: echo Hi from our first workflow!   

      - name: Show ASCII greeting                             
        uses: mscoutermarsh/ascii-art-action@master   
        with:                               
          text: 'HELLO!'

This workflow is triggered whenever there is a push event. Lets commit this file to the master and observe the run at the Action tab.

CI

CI stands for Continuous Integration, and is the idea that as different members of the team work on code on different git branches, the code is merged to a single working branch which is then built and tested with automated workflows before merging to the production ready Master branch. This helps to constantly make sure everyone's code is working properly together and is well-tested.

From the point of view of CI, the main goal of GitHub Actions and Workflows is to build and test our software every time something changes. This way we can spot bugs and correct things as soon as the errors come up.

CI Demo: A NodeJS CI pipeline

Create a CI.yaml file in your .github/workflows folder.

Paste the following into it:

name: Node Continuous Integration

on:
  Push:
    branches: [ master ]

  pull_request:
    branches: [ master ]


jobs:
  test_pull_request:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v1
        with:
          node-version: 12
      - run: npm ci
      - run: npm test
      - run: npm run build`

Branch Protection

Define branch protection rules to ensure builds pass before it can be merged to master.

To set up branch protection:

Go to setting
Click on branches
Under Branch Protection Rules click on Add Rule
Check “Require status checks to pass before merging” and other additional rules you want.
Click on create.

Now test your rule by intentionally breaking your code such that build fails.

CD

CD stands for Continuous Deployment (or Delivery). Continuous Deployment takes things a step further and takes the automation to the deployment level. Where with the CI process, you automate the testing and the building, Continuous Deployment will automate the process of deploying the project to a desired environment.

CICD Demo: A NodeJS CICD Pipeline

Clone this repo for a Sample NodeJS app [https://github.com/udujoel/nodejs_sample.git ]
Create a .github/workflows folder
Add a cicd.yaml file
Copy the workflow details below into it and Commit.

# File: .github/workflows/cicd.yml

on:
  push:
    branches:
      - master

env:
  AZURE_WEBAPP_NAME: app_name_here   # set this to your application's name
  AZURE_WEBAPP_PACKAGE_PATH: '.'      
  NODE_VERSION: '10.x'                

jobs:
  build-and-deploy:
    name: Build and Deploy
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@master
    - name: Use Node.js ${{ env.NODE_VERSION }}
      uses: actions/setup-node@v1
      with:
        node-version: ${{ env.NODE_VERSION }}
    - name: npm install, build, and test
      run: |
        # Build and test the project, then
        # deploy to Azure Web App.
        npm install
        npm run build --if-present
        npm run test --if-present
    - name: 'Deploy to Azure WebApp'
      uses: azure/webapps-deploy@v2
      with: 
        app-name: ${{ env.AZURE_WEBAPP_NAME }}
        publish-profile: ${{ secrets.AZURE_WEBAPP_PUBLISH_PROFILE }}
        package: ${{ env.AZURE_WEBAPP_PACKAGE_PATH }}

Checking the Actions page, you’ll see our deploy is failing. We’ll fix that next. Now we need to understand how to setup Secrets and how to use it for Job authentication.

Secrets and Authentication

You can use encrypted secrets, variables and tokens in your workflow to protect your repo and store sensitive info.

You can set them by visiting your Github repo settings tab,
click on ‘Secrets’.
Click on ‘New Secret’ to add a new one.
Set the name and value
Click on ‘add secret’ to save the addition.

To make use of a added secret:

Steps:
  With:
Super_secret_input: ${{ secret.SECRET_NAME }}

To set these secrets as environment variable:

Steps:
  env:
Super_secret_input: ${{ secret.SECRET_NAME }}

Now to fix our workflow:

Create an Azure App Service
Download, open and copy the App service's Publish Profile.
Add a repo secret, with title "AZURE_WEBAPP_PUBLISH_PROFILE" and paste the Publish Profile.
Update the CICD.yaml file in .github/workflows with the App name.

Commit this file to trigger a CICD process. Open the Actions tab to observe the build and deploy.

When done.

View the changes on the site address: [your-app-name].azurewebsites.net

Thanks for your time!

Ansible: Enders Game

Juell — Tue, 20 Oct 2020 04:18:47 +0000

Configuration Management

The purpose of configuration management is to maintain systems in a desired state. It aims to eliminate configuration drift as much as possible.
Configuration drift occurs when ad-hoc configuration changes and updates result in a mismatched development, test, and deployment environments. This can result in issues at deployment, security vulnerabilities, and risks when developing applications and services that need to meet strict regulatory compliance standards.
Configuration management makes it possible to scale infrastructure without worrying about increasing staff or workload. It ensures consistency across all infrastructure, improves automation and mitigates human error.

Ansible

Ansible is a very powerful and popular configuration management tool. Here are some quick facts that will interest you about Ansible:

The name “ansible” originally refers to a category of fictional technology capable of faster-than-light communication. It can send and receive messages to and from a corresponding device over any distance or obstacle whatsoever with no delay.
Its name is used in sci-fi movies (e.g Enders Game)and novels (Ursula).
In the Enders Game, the story depends on the ansible to centrally control a huge, complex fleet of distant ships. This was what inspired Ansible automation software founders to choose the term as the name for their product.
Developed by Michael DeHaan, the creator of Cobbler and Func.

Ansible is an agentless push-based configuration tool that
functions by connecting via SSH to its managed nodes. It doesn’t need
a special agent on the node and operates by pushing modules to the
clients. The modules are then executed locally on the node, and the
output is pushed back to the controller or host.

Drift is the process of making changes on a host that makes it differ from the synced version.

Nodes for Demo:

Spin-up 3 Ubuntu Linux VMs on portal.azure.com
Make user=agent, password=password@101.
Copy out the IP addresses of all 3 VMs

Ssh into any one that would serve as our control node, and proceed as follows.

Ansible Installation :

Step 1: Perform an update to the packages

   sudo apt update

Step 2: Install the software-properties-common package

 sudo apt install software-properties-common

Step 3: Add ansible personal package archive

sudo apt-add-repository ppa:ansible/ansible

Step 4: Install ansible

 sudo apt install ansible

To test your installation, do:

ansible --version

Ansible Ad-hoc Commands:

This is one of the simplest ways Ansible can be used to issue some commands on a server or a bunch of servers. Ad-hoc commands are not version controlled but represent a fast way to interact with the desired servers.

ansible -m ping localhost

-m ==> module
Ping ==> module name
Localhost ==> target node

ansible -m copy -a 'src=./file1.txt dest=~/file.txt' localhost

To see if some changes would be implemented, do a dry run. Add ‘--check’ to the command. To see the changes that would be implemented, add ‘--diff’. ‘ diff flag alone implements the changes after pointing them out to you.

Idempotency:

This is an IaC property that ensures a defined infrastructure
deployment sets the target environment to the desired state, every
time it is run, regardless of its starting state. The IaC provider
will decide what needs to be stood-up, torn down or reconfigure to
match your described target state.

Ansible Playbook

The ansible playbook is a more readable, versionable way of executing tasks using ansible. In comparison with ad-hoc commands, playbooks are used in complex scenarios, and they offer increased flexibility and capability. Playbooks use YAML format, so there is not much syntax needed, but its indentation rules are strict. Ansible playbooks tend to be more of a configuration language than a programming language.

Like its name, a playbook is a collection of plays. A play is a set of
orchestration instructions.

    ---
      #ansible -m copy -a “src=./file1.txt dest=~/file1.txt” localhost
    - name: Play name
    hosts: localhost
      tasks:
      - name: Copy a file
        copy:
          src: ./file1.txt
          dest: ~/file1.txt

An ansible playbook starts with the 3 dashes.
"#" indicates a comment line.
The - name indicates a non-executed name that serves a descriptive purpose.
Hosts specifies the target node device, or groups.
A task is made up of an action and its args.
Under tasks you can specify your modules and its parameter.

If the action you desire requires root/sudo, use the ‘Become’ keyword
for privilege escalation.

Ansible Inventory

Ansible uses the Inventory to identify its managed hosts. The default host file is at:

/etc/ansible/hosts

Here you define your nodes, node alias, node groups and variables. Create a group with the ip addresses of the VMs you created.

[master]
host1 ansible_host=192.168.100.2 ansible_user=agent ansible_password=password@101
[agents]
node1 ansible_host=192.168.100.3 ansible_user=agent ansible_password=password@101
node2 ansible_host=192.168.100.4 ansible_user=agent ansible_password=password@101

The ‘ansible-inventory’ command is used for working with the managed nodes.
To see a full list of all our nodes type:

ansible-inventory --list

For a more graphical layout use:

ansible-inventory --graph

@localhost refers to the current/control node
@all refers to all the nodes
@group_name targets all nodes under a group.

To simplify our host file, define all variables in the [all:vars] group:

[all:vars]
ansible_user=agent
ansible_password=password@101

[master]
host1 ansible_host=192.168.100.2 

[agents]
node1 ansible_host=192.168.100.3 
node2 ansible_host=192.168.100.4

NOTE: Since Ansible is going to be making an SSH connection with a password to these machines, we need to install “sshpass”.

sudo apt install sshpass -y

Let us try to test our hosts with the ping command:

ansible -m ping all

Gives us an error. We need to modify some ansible configuration.

Ansible CFG

The Ansible configuration file is exactly what you expect it to be. Its located at:

/etc/ansible/ansible.cfg

We’ll need to open it and modify a setting. Open the file and uncomment: host_key_checking = false
Or enter it.
Now you can run your ping command:

ansible -m ping all

Writing a Playbook

Create a file:

touch playbook.yml

Open it and add the following to it:

    ---
    - name: play one - install vim from custom inventory
      hosts: agents
      become: true
      tasks:
      - name: apt 
        apt:
          name: vim
          state: latest

      - name: zsh install
        apt:
          name: zsh
          state: latest

      - name: vagrant install
        apt:
          name: vagrant
          state: latest

      - name: nodejs install
        apt:
          name: nodejs
          state: latest

    - name: play two server config
      become: true
      hosts: master
      tasks:
        - name: ensure Nginx is at latest
          apt:
            name: nginx
            state: latest
        - name: start server
          service:
            name: nginx
            state: started

Save it.

Executing a Playbook

To execute the play we just wrote, we use the ‘ansible-playbook’ command. Enter:

ansible-playbook playbook.yml -v

The -v is for verbosity. The more the v’s the more details that would be output.
We can also execute a playbook from a custom inventory file.
Create a custom inventory file.

touch inventory

Add nodes to it.

Ping test it with the following:

ansible -I inventory -m ping all

Run a playbook off it with:

ansible-playbook -I inventory playbook.yml

Shut-down our VMs

Now, as a final demo, let's shut down our VMs to avoid extra charges. Since we know ansible can do all sorts of things, let's use it for this demo.
Can you write the YAML script for this? Try it out first. It's a simple one.

    ---
    - name: shut down all my agent VMs
      hosts: all
      become: true
      tasks:
        - name: run shutdown command
          command: shutdown

Save and execute this:

ansible-playbook shutdownVMs.yml -vv

Ansible on Azure Cloudshell

Azure Cloudshell has all we need to execute both Ansible Ad-hoc command and playbooks. We shall next create a VM with its underlying resources from the Azure Cloudshell.

Visit HTTP://portal.azure.com and open the Cloudshell. Choose Bash and enter the following to open its embedded VS Code editor:

code .

inside the editor, create a azvmplaybook.yml and paste the following inside:


- name: Create Azure VM
  hosts: localhost
  connection: local
  tasks:
  - name: Create resource group
    azure_rm_resourcegroup:
      name: myResourceGroup
      location: eastus


  - name: Create virtual network
    azure_rm_virtualnetwork:
      resource_group: myResourceGroup
      name: myVnet
      address_prefixes: "10.0.0.0/16"


  - name: Add subnet
    azure_rm_subnet:
      resource_group: myResourceGroup
      name: mySubnet
      address_prefix: "10.0.1.0/24"
      virtual_network: myVnet


  - name: Create public IP address
    azure_rm_publicipaddress:
      resource_group: myResourceGroup
      allocation_method: Static
      name: myPublicIP
    register: output_ip_address

  - name: Dump public IP for VM which will be created
    debug:
      msg: "The public IP is {{ output_ip_address.state.ip_address }}."


  - name: Create Network Security Group that allows SSH
    azure_rm_securitygroup:
      resource_group: myResourceGroup
      name: myNetworkSecurityGroup
      rules:
        - name: SSH
          protocol: Tcp
          destination_port_range: 22
          access: Allow
          priority: 1001
          direction: Inbound


  - name: Create virtual network interface card
    azure_rm_networkinterface:
      resource_group: myResourceGroup
      name: myNIC
      virtual_network: myVnet
      subnet: mySubnet
      public_ip_name: myPublicIP
      security_group: myNetworkSecurityGroup


  - name: Create VM
    azure_rm_virtualmachine:
      resource_group: myResourceGroup
      name: myVM
      vm_size: Standard_DS1_v2
      admin_username: agent
      admin_password: password@101
      network_interfaces: myNIC
      image:
        offer: UbuntuServer
        publisher: Canonical
        sku: '18.04-LTS'
        version: latest

To set the ball rolling, execute this on the Cloudshell:

ansible-playbook azvmplaybook.yml

Wait for it...
When it's done, go to your Resource Group to check it out. You can even SSH into your created VM. When done playing with it, we need to clean our tracks. Conveniently, simply deleting our resource group will take all the created resources with it. So let's do just that. Create a rmrgplaybook.yml:

---
- hosts: localhost
  tasks:
    - name: Deleting resource group
      azure_rm_resourcegroup:
        name: myResourceGroup
        state: absent
        force_delete_nonempty: yes

Here we set the force_delete_nonempty flag to yes to enable it to delete all underlying resources.
Now to put it in play:

ansible-playbook rmrgplaybook.yml

Thanks for your time!