DEV Community: Hussein Alamutu

Demystifying For Loops in Terraform: A Practical Guide

Hussein Alamutu — Thu, 15 Jun 2023 12:56:39 +0000

Terraform is a popular infrastructure as a code tool that allows you to define and manage cloud resources.

One of the powerful features of Terraform is the ability to use for loops.

The for loops in Terraform provide a powerful mechanism for iterating over a list of items and performing actions or creating resources dynamically.

This capability allows for efficient and flexible infrastructure provisioning, reducing manual effort and enhancing scalability.

In this guide, I walk you through the syntax and benefits of for loops in Terraform, providing practical insights and examples to help you leverage their power effectively.

Understanding For Loops in Terraform

To leverage for loops, you start by defining a variable that represents the list of items you want to iterate over, then, you use either the for_each or count expressions to control the iteration.

The count expression is suitable for iterating a fixed number of times, while the for_each expression is particularly useful when you have a dynamic set of items, such as a map or set.

But, what do I mean by the map and set?

The for_each map is like dictionaries in Python, where variables have attributes with keys mapped to a value, while the set is similar to a list - an ordered collection of objects.

Here's an example of a Terraform code snippet that demonstrates the use of for_each and count expressions in different use cases:

# Dynamic set of items using for_each
variable "dynamic_items" {
  type    = map(string)
  default = {
    ubuntu_server = "ami-01dd271720c1ba44f“ 
    windows_server = "ami-0274fd9e256dea7b1“ 
    rhel_server = "ami-013d87f7217614e10“ 
  }
}

resource "aws_instance" "servers" {
  for_each = var.dynamic_items

  ami           = each.value
  instance_type = "t2.micro"

  tags = {
    Name = each.key
  }
}

In this example, the for_each expression is used to create AWS EC2 instances based on the dynamic set of items defined in the dynamic_items variable. Each item in the map represents a server name and its corresponding types - machine image (ami). The aws_instance resource will be created for each server in the map, and the instance tags will be set using the for_each iterator.

On the other hand, the count expression is used to create AWS EBS volumes iteratively based on the fixed count defined in the fixed_count variable. The aws_ebs_volume resource will be created three times, with each volume having a different name specified using the count.index variable (DataVolume-1, DataVolume-2, DataVolume-3).

# Fixed number of iterations using count
variable "fixed_count" {
  type    = number
  default = 3
}

resource "aws_ebs_volume" "data_volume" {
  count          = var.fixed_count
  availability_zone = "us-west-1a"
  size           = 100
  volume_type    = "gp2"

  tags = {
    Name = "DataVolume-${count.index + 1}"
  }
}

By using for loops, instead of manually defining multiple resource blocks or modules, you can dynamically generate resources, this eliminates the need for repetitive code and reduces the chances of errors.

They also enable easy scalability, this allows you to adapt to changing requirements and easily handle larger infrastructure deployments. Whether you need to create multiple EC2 instances, provision multiple subnets, or configure multiple security groups, for loops provide a convenient mechanism to scale your infrastructure.

In addition to automation, they also support conditional statements, you can incorporate if conditions and logical expressions within for loops to control the iteration process and make dynamic decisions based on specific criteria.

Best Practices

When working with for loops in Terraform, maintaining code readability and organization is essential for ensuring long-term maintainability and collaboration. Here are some best practices to consider:

Use meaningful variable names: Choose descriptive names for your variables to enhance code readability. This makes it easier for others (and yourself) to understand the purpose and context of the loop.
Add comments: Include comments within your code to provide explanations and document the intention behind the for loop. This helps others grasp the logic and purpose of the iteration.
Indentation and formatting: Consistently apply proper indentation and formatting to your for loops.
Limit loop complexity: Avoid overly complex for loops with nested iterations or extensive conditional statements. Excessive complexity can make code difficult to understand and maintain.
Test and validate: Before deploying for loops in production environments, thoroughly test and validate their functionality.

The End -

Summarily, the power of for loops in Terraform empowers you to automate resource creation and configuration while providing flexibility through conditional statements. Moreover, the inclusion of conditional statements allows you to adapt to different scenarios, making your infrastructure setups more dynamic and adaptable to changing requirements.

For a more in-depth tutorial, take a look at this article.

How to automate the installation of web servers on AWS EC2 using Terraform and Github Actions

Hussein Alamutu — Sun, 30 Apr 2023 22:58:26 +0000

As more and more businesses are moving their applications to the cloud, the need to automate the deployment and management of infrastructure has become increasingly important. One of the common tasks in cloud computing is setting up web servers on virtual instances, and managing them, which can be a tedious and error-prone task if done manually.

This is where automation tools like Terraform and Github Actions come in handy.

Terraform is an open-source infrastructure as code software tool that allows developers to manage their infrastructure using code, and is particularly useful for managing cloud resources such as AWS EC2 instances.

Github Actions, on the other hand, is a powerful continuous integration and continuous deployment (CI/CD) tool that can be used to automate the deployment of code changes to cloud environments.

In this article, I will teach you how to use Terraform and Github Actions to automate the installation of web servers(apache and nginx) on AWS EC2 instances. By automating this process, we can save time and reduce the risk of human error, while also allowing us to easily replicate our infrastructure across multiple instances.

Prerequisites

Be familiar with AWS and it's services
Understand how Github works
Know what web servers are
Familiarity with Terraform

Setting up AWS and Terraform
Creating the Github repository and setting up Github Actions
Writing the Terraform code

I. Setting up AWS Access Key and Secret Key

To get started with automating the installation of web servers on AWS EC2 using Terraform and Github Actions, you need to set up an AWS account.

Creating an AWS account and setting up access keys

If you don't have an AWS account already, create an AWS account by visiting the AWS website and following the instructions to sign up.

Once you have an account, you will need to create an access key that Terraform can use to interact with your AWS account. To do this, go to the AWS console and navigate to the IAM service.

Under the "Access management" section on the left hand pane, select "Users" and create a new user. Give the user a name and choose "Programmatic access" as the access type. Next, assign the user to a group that has the necessary permissions or directly attach the permission policies necessary for managing EC2 instances, load balancers, and auto scaling, then create the user.

Finally, you will be prompted to download the access keys for the user. Keep these keys safe as they will be used to configure Github Actions and Terraform.

If you got stuck in any part, you should watch this short tutorial on how to create AWS access keys. You can also follow AWS guide on creating access keys here.

II. Creating the Github repository and setting up Github Actions

Once we have set up our AWS account, the next step is to create a Github repository for our project and set up Github Actions to automate the deployment of the web server on AWS EC2.

Creating a new Github repository for the project

Go to the Github website and sign in to your account. Create a new repository by clicking on the "New" button in the top right corner of the screen. Give your repository a name, choose any other settings that you prefer, and click on the "Create repository" button to create the repository.

Configuring Github Actions to automatically deploy the web server on AWS EC2

Github Actions allows us to automate various tasks in our repository, including the deployment of our web server on AWS EC2.

To set up Github Actions, go to the "Actions" tab in your repository and click on the "Set up a workflow yourself" button. This will allow you to create a new workflow file that defines the actions you want to perform.

In the workflow file, you can define the triggers for when the actions should be executed, such as on pushes to a specific branch or on a schedule. Then, you can define the steps that should be executed as part of the workflow.

For this example:

Define the triggers to be when you make a push to your main branch.

on:
  push:
    branches: main

Then, define a step to configure AWS credentials using the access keys you created earlier, and another step to run Terraform commands to deploy the web server on an EC2 instance.

# configuring AWS access keys
env:
  AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
  AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

For this part of the code to work, define the environment secrets that are required which is the values for the secret defined in the above github actions code.

Secrets are encrypted and can be securely stored in Github so that they can be used in your workflow without exposing sensitive information.

To do this, navigate to your Github repository for this task and click on the settings tab, then navigate to the secrets and variables tab in the left pane,

After that click on actions in the drop down menu from the secrets and variables tab, then click on the green button in the middle of the page, labelled "New repository secret"

This takes you to a new page to configure your secret environment variables used in your Github Actions workflow.

Using AWS_ACCESS_KEY_ID as the name, next, get the access key you got from AWS in step one, copy and paste into the Secret* box, and click on "Add secret", now repeat the same process for AWS_SECRET_ACCESS_KEY.

Configuring the job for the Github workflow to deploy the web server on an EC2 Instance

# configuring steps to be run, for terraform various commands 
jobs:
  tf:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - uses: hashicorp/setup-terraform@v2
    - name: Terraform fmt
    id: fmt
    run: terraform fmt -check
    - name: Terraform Init
    id: init
    run: terraform init
    - name: Terraform Plan
    id: plan
    run: terraform plan
    - name: Terraform Apply
    id: apply
    run: terraform apply -auto-approve

Once you have defined your workflow, save the file and Github Actions will automatically start running the defined steps whenever the triggers are met(a push to the main branch). You can monitor the progress of your workflow in the "Actions" tab and view the logs to troubleshoot any issues that may arise.

With Github Actions configured, you now have an automated process for deploying web servers on AWS EC2 instances using Terraform, making it easier to manage your infrastructure and ensure consistent deployments across environments.

Note: Your final Github Actions code should look like this,

name: Terraform AWS

# setting up triggers
on:
  push:
    branches: main

# setting up the access key to aws-cli
env:
   AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
   AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

jobs:
  tf:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - uses: hashicorp/setup-terraform@v2
    - name: Terraform fmt
      id: fmt
      run: terraform fmt -check
    - name: Terraform Init
      id: init
      run: terraform init
    - name: Terraform Plan
      id: plan
      run: terraform plan
    - name: Terraform Apply
      id: apply
      run: terraform apply -auto-approve

III. Writing Terraform code

In this section I teach you how to write Terraform code to configure an EC2 instance and install a web server on it.

Writing Terraform code to configure the EC2 instance and install the web server:

To begin, create a main.tf file, where to define the resources that will be created. First, define the provider module in this case AWS provider, you can get it from the official terraform documentation.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.0"
    }
  }
}

# Configure the AWS Provider
provider "aws" {
  region = "us-east-1"
}

Next, define the two ec2 instance, one for NGINX and the other one for Apache.

resource "aws_instance" "nginx" {
    ami = "ami-014d05e6b24240371"
    instance_type = "t2.micro"
    **user_data = file("nginx.sh")**
    tags = { 
    Name  = "NGINX"
  }
}

resource "aws_instance" "apache" {
    ami = "ami-014d05e6b24240371"
    instance_type = "t2.micro"
    **user_data = file("apache.sh")**
    tags = { 
        Name  = "APACHE"
    }
}

The resource block is used to define the type of resource you need terraform to provision for you in AWS, in this case aws_instance and the next label stands for an identifier, in case you need to use the resource in other parts of your code i.e "aws_instance.apache.instance_type*" or "aws_instance.nginx.ami"

Next, to get the ami(amazon machine image) value for your main.tf instance resource configuration, navigate to EC2 dashboard on AWS and click on ami catalog under Images dropdown in the left pane, choose any ami id of your choice, copy and paste into your instance configuration, in this case I make use of an Ubuntu image which has a free tier.

instance_type is used to select the appropriate combination of CPU, memory and networking resource. The t2.micro instance type has an attached free tier, I recommend you use it. Learn more about instance type

user_data in the code is use to run a shell script that installs and configures nginx or apache. For this example, I created two files, apache.sh and nginx.sh with bash commands in the files to install both apache and nginx respectively.

Follow these steps to create your user data's file

Create a file with the name nginx.sh in your parent folder, copy and paste the below code.

#!/bin/bash

# install nginx
apt-get update
apt-get -y install nginx

# make sure nginx is started
service nginx start

Next, create another file with the name apache.sh in your parent folder, copy and paste the below code.

#!/bin/bash

# install apache
apt-get update
apt-get -y install apache2

# make sure apache is started
service apache2 start

Final note, your final main.tf file should look like this, you can also get access to the code here

terraform {
  required_providers {
    aws = {
        source = "hashicorp/aws"
        version = "~>4.0"
    }
  }
}

provider "aws" {
  region = "us-west-1"
}

resource "aws_instance" "nginx" {
    ami = "ami-014d05e6b24240371"
    instance_type = "t2.micro"
    user_data = file("nginx.sh")
    tags = { 
    Name  = "NGINX"
  }
}

resource "aws_instance" "apache" {
    ami = "ami-014d05e6b24240371"
    instance_type = "t2.micro"
    user_data = file("apache.sh")
    tags = { 
        Name  = "APACHE"
    }
}

With these resources defined, we can run "terraform init" to initialize our project, "terraform plan" to see what resources will be created, and "terraform apply" to create the resources on AWS.

The Github Actions Workflow created in section II contains the above mentioned commands, which will be triggered to run in succession when you git push to your repo's main branch.

Bonus: Using variables and modules to make the code more modular and reusable

As our infrastructure becomes more complex, it becomes more difficult to manage all the configuration options in a single file. To make the code more modular and reusable, we can use variables and modules.

Variables allow us to define values that can be passed into our Terraform code at runtime. We can define variables for the instance type, the key pair, and any other configurable settings.

variable "instance_type" {
  default = "t2.micro"
}

variable "key_pair" {
  default = "my-key-pair"
}

Modules, on the other hand, allow us to encapsulate groups of resources into reusable units that can be easily shared and reused across different Terraform projects. We can create a module for configuring an EC2 instance with a web server installed, including all of the necessary resources and configuration options.

module "web_server" {
  source = "./modules/web_server"

  instance_type = var.instance_type
  key_pair      = var.key_pair
}

In this example, we have created a module in the "modules/web_server" directory, which defines the resources and configuration options required for the web server. The module can be shared and reused across different Terraform projects, making it easier to manage and maintain the infrastructure.

In conclusion, writing Terraform code to configure an EC2 instance and install a web server is a powerful way to manage infrastructure in a declarative way. By using variables and modules, we can make our Terraform code more modular and reusable, allowing us to manage more complex infrastructure with ease.

That's it folks if you are interested in seeing more terraform task related to this you can check out my github .

Understanding Virtual CPUs: Exploring the Differences Between Cores and Threads

Hussein Alamutu — Thu, 13 Apr 2023 23:09:07 +0000

In today's world, computers are more powerful than ever before. However, the speed and performance of a computer are often limited by its hardware components, such as its CPU.

In recent years, virtual CPUs have emerged as a solution to this problem, enabling faster and more efficient computing.

Virtual CPUs is the creation of multiple CPUs from a single physical CPU.

This allows for more efficient use of hardware resources, and can significantly improve the performance of computers and servers.

In this article, we will explore the concept of virtual CPUs, and explain the differences between cores and threads.

Virtual CPU

A virtual CPU, also known as a vCPU, is a portion of a physical CPU that is allocated to a virtual machine. Virtual machines are software programs that mimic the behavior of a physical computer, and are commonly used in cloud computing and virtualization. By creating virtual CPUs within a virtual machine, it is possible to run multiple virtual machines on a single physical server.

The virtual CPUs within a virtual machine can be configured to use a different number of cores and threads, depending on the requirements of the application being run. A core is a physical processing unit within a CPU, while a thread is a virtual processing unit that can be created within a core. By using multiple cores and threads, it is possible to improve the performance of the virtual machine, and ensure that it can handle multiple tasks at once.

One of the key benefits of virtual CPUs is that they enable greater efficiency and resource utilization. It also makes more efficient use of hardware resources, and reduce the cost of running a data center or cloud computing service.

Additionally, virtual CPUs enable faster and more efficient processing of data, which can help to reduce latency and improve performance.

Cores vs Threads

Virtual CPUs are a technology that allow for more efficient use of hardware resources, and can significantly improve the performance of computers and servers.

However, to fully understand the benefits of virtual CPUs, it's important to first understand the differences between CPU cores and threads, and how they impact the performance of virtual machines.

Cores

CPU cores are physical processing units within a CPU that are responsible for executing instructions. Most CPUs today have multiple cores, which enables them to execute multiple instructions simultaneously. This allows for faster and more efficient processing of data, and is particularly important in applications that require high levels of computing power, such as gaming, video editing, and scientific computing.

Threads

Threads, on the other hand, are virtual processing units that can be created within a core.

Threads allow for parallel processing of data within a core, which can significantly improve the performance of applications that require multiple tasks to be executed simultaneously. By using threads, it is possible to maximize the use of CPU cores, and achieve faster and more efficient processing of data.

Impact of Cores and Threads

When it comes to virtual CPUs, the number of cores and threads allocated to a virtual machine can have a significant impact on its performance.

Virtual machines with multiple cores and threads can handle more tasks simultaneously, which can result in faster and more efficient processing of data. However, allocating too many cores and threads can also result in overhead, and can reduce the efficiency of the virtual machine.

One of the key advantages of virtual CPUs is that they enable flexible allocation of resources.

For example, applications that require high levels of computing power, such as video editing or scientific computing, can benefit from more cores and threads, while applications that require less computing power, such as web browsing or email, may only require a few cores and threads.

Advantages of Virtual CPUs in Cloud Computing and Virtualization

Virtual CPUs have become an essential technology for cloud computing and virtualization. By enabling multiple virtual CPUs to be created from a single physical CPU, virtualization can provide many benefits, including improved performance, greater efficiency, and more flexibility.

This section explores the advantages of virtual CPUs in cloud computing and virtualization.

Reduced Cost

Virtual CPUs enable more efficient allocation of resources, which help to reduce the cost of running a data center or cloud computing service and also ensure resources are used effectively, and applications perform as efficiently as possible.

Greater Efficiency

By creating multiple virtual machines on a single physical server, it is possible to make more efficient use of hardware resources. This can help to reduce the cost of running a data center or cloud computing service, as fewer physical servers are required to run the same number of applications.

More Flexibility

Finally, virtual CPUs provide more flexibility in cloud computing and virtualization. With virtual machines it is possible to isolate applications and improve their security. This help to reduce the risk of security breaches and protect sensitive data.

Virtual CPUs also enable greater flexibility in resource allocation. By adjusting the number of cores and threads allocated to a virtual machine, it is possible to optimize its performance, this can help to ensure that resources are used effectively, and that applications perform as efficiently as possible.

Limitations of Virtual CPUs

Virtual CPUs offer many benefits, but they also have limitations that can impact performance. In this section, I will discuss some of the limitations of virtual CPUs and their impact on performance.

Overhead

Virtualization software adds overhead to the system. The overhead is the additional processing power required to manage the virtual machines.

This overhead can be significant, especially for applications that require low latency and high performance. Overhead can lead to performance degradation, as the processing power required to manage the virtual machines can reduce the amount of processing power available to applications.

Resource Allocation Issues

Resource allocation can also be a challenge when using virtual CPUs. In some cases, it can be difficult to allocate the appropriate amount of resources to virtual machines. If a virtual machine is allocated too few resources, it can result in poor performance. On the other hand, if a virtual machine is allocated too many resources, it can lead to wasted resources and unnecessary costs.

I/O Bottlenecks

Virtualization software can create I/O bottlenecks, which can impact performance. I/O bottlenecks occur when multiple virtual machines compete for the same physical resources, such as network bandwidth or disk I/O. This can lead to slow response times and poor performance.

Licensing

Another limitation of virtual CPUs is licensing. Some software vendors require licensing based on the number of physical CPUs in a system. If a virtual machine is running on a system with multiple virtual CPUs, it can be challenging to determine how many physical CPUs the software vendor requires for licensing purposes.

The impact of these limitations on performance can be significant. Overhead can reduce the processing power available to applications, leading to slower response times and degraded performance. Resource allocation issues can result in poor performance if a virtual machine is not allocated enough resources. I/O bottlenecks can lead to slow response times and poor performance, especially for applications that require high I/O throughput. Licensing can also impact performance, as software vendors may require licensing based on the number of physical CPUs in a system.

Conclusion

I started by introducing the concept of virtual CPUs and how they differ from physical CPUs. I then discussed the differences between cores and threads and how they impact virtual machine performance. I also explained the advantages of virtual CPUs in cloud computing and virtualization, as well as their applications.

Likewise, I also discussed the limitations of virtual CPUs, such as overhead and resource allocation issues, which can impact performance.

In conclusion, virtual CPUs play a critical role in modern computing environments. They allow multiple virtual machines to run on a single physical server, reducing hardware costs and improving resource utilization.

Virtual CPUs offer many benefits, but they also have limitations that can impact performance. By understanding these limitations and how they impact performance, we can ensure that virtual machines perform as expected, and make the most out of this powerful technology.

Congratulations! You have gotten to the end of the guide.

I’m currently in search of paid technical writing gigs related to cloud & devops, if you got one, reach out or refer me.

Send me an e-mail via husseinalamutu@gmail.com

The How, Why and When of Github Actions.

Hussein Alamutu — Sat, 08 Apr 2023 11:50:23 +0000

DevOps is a broad field, with what seems like a non ending pool of tools to use.

Github Actions is one of those tools. If you want to automate your code pipeline from integration to delivery without complicating things, Github Actions comes in handy.

At the end of this article you will gain the understanding of why Github Actions is used in DevOps, when to use it and how to get started with it.

Prerequisites

In order to get the most out of this article, you need to have a basic understanding of Github and continuous integration and continuous delivery(CI/CD) pipeline.

Why Use Github Actions?

Github Actions is a CI/CD tool that allows you to automate your workflow by creating custom scripts that can be triggered by different events. These events can include things like code changes, pull requests, or other Github activities.

Note: I will be using workflows a lot as we move along.

In the context of Github Actions, a workflow refers to a set of automated steps or processes that are triggered by an event, such as a code push or pull request, in a Github repository.

A workflow consist of one or more jobs, and each job can have one or more steps that define specific actions to be executed, such as running tests, building a project, or deploying code to a server.

The workflow is defined in a YAML file that is committed to the repository, and Github automatically executes the workflow when the defined event, such as code push or merged pull request occurs.

Github Actions was first launched in 2018, and since then it has been adopted by a large number of organizations and individuals. Github Actions is based on YAML syntax and uses Docker containers to run your code in a variety of different environments(Linux, Windows or MacOS).

Github Actions offers several advantages over other CI/CD tools.

These advantages include:

Integration with Github: Because Github Actions is integrated with Github and Github is very popular amongst programmers as a code hosting platform for version control and collaboration, this makes Github Actions easy to set up and use if you're already familiar with Github.
Scalability: Github Actions is suitable for small teams and can scale to meet the needs of large organisations.
Flexibility: Github Actions is highly customizable, which means you can create workflows that fit your specific needs.
Event-based workflows: With Github Actions, you can set up workflows to run when specific events occur, which can help you automate your workflow more efficiently.
Time-based workflows: You can set up workflows to run at a specific time or on a recurring schedule, rather than being triggered by an event in a Github repository. This is useful for performing tasks like automated backups, running tests on a regular basis, or performing routine maintenance on a server or application.
Github Actions can also be a more cost-effective solution than other CI/CD tools, as it doesn't require any additional infrastructure or third-party services. Additionally, Github Actions provides a wide range of pre-built actions that you can use to get started quickly.

When to use Github Actions?

Github Actions is a powerful tool for automating continuous integration and deployment workflows, but it's not always the best choice for every situation. When considering whether to use Github Actions for your project, it's important to evaluate your needs and compare Github Actions to other CI/CD tools on the market.

This section explores when it's appropriate to use Github Actions, how to evaluate whether it's the right choice for your project, and some examples of successful use cases.

Choosing between Github Actions and other CI/CD tools

When it comes to choosing between Github Actions and other CI/CD tools, there are a few factors to consider. One key factor is the level of customization and control you need over your workflows. Github Actions offers a high level of customisation and control, allowing you to define complex workflows and use a wide range of third-party integrations.

However, other tools may offer more specialised features or integrations that better meet your specific needs.

Another factor to consider is the level of expertise and resources required to set up and maintain your CI/CD pipeline.

Github Actions is relatively easy to set up and use, especially if you're already familiar with Git and Github, other tools may require more expertise or specialised knowledge to configure and maintain.

Evaluating when Github Actions is appropriate

To determine whether Github Actions is appropriate for your project, it's important to evaluate your needs and goals. Here are a few questions to consider:

What level of automation and testing do you need for your project?
What are your deployment goals, and how frequently do you need to deploy code?
What level of customisation and control do you need over your workflows?
What level of expertise and resources do you have available to set up and maintain your CI/CD pipeline?

If you need a high level of automation and testing, frequent deployments, and a high level of customisation and control over your workflows, Github Actions may be a good choice.

However, if you have more specialised needs or require a higher level of expertise to set up and maintain your pipeline, other tools may be a better fit.

Examples of successful use cases

There are many successful use cases for Github Actions, including:

Automating testing and deployment for web applications
Building and deploying Docker images.
Running linters and other code quality checks.
Automating documentation generation and publishing
In each of these cases, Github Actions provides a powerful and flexible tool for automating complex workflows and ensuring that code is tested, built, and deployed reliably and efficiently.

In conclusion, Github Actions can be a powerful tool for automating CI/CD workflows, but it's important to evaluate your needs and compare Github Actions to other tools before making a decision.

By carefully considering your requirements and goals, you can choose the tool that best meets your needs and ensures that your development process is efficient, reliable, and scalable.

The How of GitHub Actions

This section explores how to set up Github Actions, including a step-by-step guide, an explanation of workflows and actions, and examples of YAML syntax.

Step-by-Step Guide to Setting up Github Actions

Create a new repository in Github, or select an existing repository.

For this step, I am using an existing repository(devops_hands-on)

Navigate to the "Actions" tab in your repository and click "set up a workflow yourself" to create a new YAML file for your workflow.

You should see something similar to what's in the image below, Github will suggests some sample workflow depending on what's in your repository.

Define your workflow by specifying the event that will trigger your workflow, the jobs that will be executed, and the steps that will be performed within each job.

This is what the workflow YAML file looks like.

Test your workflow by committing changes to your repository and observing the actions taken by Github.

This image shows what goes on underneath the specified workflow, basically, anytime I push to my repo or merge a pull request, this workflow runs and perform the actions "echo Hello, world!" and some other multi-line script that was defined in the workflow.

Explaining Workflows and Actions

A workflow in Github Actions is a set of automated steps or processes that are triggered by an event, such as a code push or pull request, in a Github repository.

A workflow can consist of one or more jobs, and each job can have one or more steps that define specific actions to be executed, such as running tests, building a project, or deploying code to a server.

An action is a specific step that is executed within a job in a workflow. Actions can be defined using pre-existing actions from the Github Marketplace, or by writing custom actions in Docker containers.

Examples of YAML Syntax

Here are some examples of YAML syntax that you might use when defining your workflow:

Specifying a trigger for your workflow:

on:
  push:
    branches: [ main ]

When you push any changes to your main branch, your workflow fires up.

Defining a job that runs a command:

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v2
      - name: Install dependencies
        run: npm install
      - name: Build project
        run: npm run build

This code defines a single job in a Github Actions workflow called "build". The job runs on the latest version of the Ubuntu operating system, as specified by the runs-on key.

The job consists of three steps:

The first step, named "Checkout code", uses the actions/checkout action to fetch the latest version of the repository code. This action is commonly used in Github Actions workflows as a first step to ensure that the code being built or tested is up-to-date.

The second step, named "Install dependencies", runs the npm install command to install any dependencies required by the project. This step assumes that the project is a Node.js application and uses the Node Package Manager (npm) to install the dependencies.

The third and final step, named "Build project", runs the npm run build command to build the project. This step assumes that the project has a build script defined in its package.json file, and that running npm run build will produce a compiled version of the project ready for deployment.

Overall, this code defines a simple Github Actions job that checks out the latest code, installs dependencies, and builds the project.

This is a common workflow for many types of projects and can serve as a starting point for more complex workflows that incorporate testing, deployment, and other tasks.

Defining a job that uses a pre-existing action:

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v2
      - name: Deploy to server
        uses: easingthemes/ssh-deploy@v2.1.1
        with:
          server: ${{ secrets.SERVER }}
          username: ${{ secrets.USERNAME }}
          key: ${{ secrets.KEY }}
          port: ${{ secrets.PORT }}
          source: "dist/"
          dest: "/var/www/my-app"

This code defines a Github Actions job called "deploy" that runs on the latest version of Ubuntu operating system.

The job consists of two steps.

The first step, named "Checkout code", uses the actions/checkout action to fetch the latest version of the repository code. This step is similar to the first step in the previous example, and ensures that the code being deployed is up-to-date.

The second step, named "Deploy to server", uses the easingthemes/ssh-deploy action to deploy the code to a server. This action allows for secure SSH-based deployment of a project to a remote server. The action takes several parameters that are stored as secrets in Github, including the server's ip address, the username to log in with, and the path to the SSH key file.

The action also specifies the source and destination paths for the deployment. In this case, it is deploying the contents of the "dist/" directory to the "/var/www/my-app" directory on the remote server.

Overall, this code defines a simple Github Actions job that deploys a project to a remote server using SSH-based deployment. This is a common workflow for many types of projects that require deployment to a remote server, and can be modified to suit different deployment scenarios.

What a final workflow looks like

The final workflow for Github Actions will depend on the specific needs and goals of your project.

However, to give you an idea of what a final workflow might look like, here's an example:

name: Build and Deploy

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v2
      - name: Install dependencies
        run: npm install
      - name: Build project
        run: npm run build
  deploy:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - name: Setup SSH
        uses: webfactory/ssh-agent@v0.4.1
        with:
          ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
      - name: Deploy to server
        run: ssh user@server "cd /var/www && git pull"

This workflow is triggered by a push to the main branch or a pull request on the main branch.

It consists of two jobs: a build job that installs dependencies and builds the project, and a deploy job that deploys the project to a server. The deploy job has a dependency on the build job, meaning it won't be executed until the build job has completed successfully.

In the deploy job, the workflow sets up an SSH connection to the server using a private key stored as a secret in Github. It then uses SSH to run a command on the server that pulls the latest changes from the Github repository.

This is just one example of a Github Actions workflow, and yours will likely look different depending on your specific needs.

However, by following the steps outlined in this article, you can create powerful and flexible automation for your Github repository.

Bonus: Best practices for Github Actions

Github Actions is a powerful tool that can help automate many aspects of software development, from continuous integration and deployment to testing and monitoring.

However, with great power comes great responsibility, and it's important to follow best practices when writing Github Actions workflows to ensure that they are efficient, effective, and secure.

This section covers some of the best practices for Github Actions that you should keep in mind when creating workflows.

Keep workflows simple and focused.

One of the best practices for writing Github Actions workflows is to keep them simple and focused on a specific task. This means that each workflow should only do one thing, such as running tests or deploying code, and should not try to do too many things at once. This approach makes it easier to understand and maintain the workflow, and reduces the risk of errors or failures.

Use caching to speed up workflows.

Github Actions provides caching functionality that can be used to speed up workflows by caching files and dependencies between workflow runs.

This can significantly reduce the time it takes to run a workflow, especially if the workflow requires downloading and installing dependencies or compiling code. To use caching, you can add a caching step to your workflow that specifies which files or directories to cache.

Use reusable actions.

Github Actions allows you to create and reuse actions, which are reusable steps that can be used in multiple workflows.

Reusing actions can help simplify your workflows and make them more modular, as you can reuse the same action in multiple workflows rather than writing the same steps multiple times. You can create your own actions or use actions from the Github Marketplace.

Keep secrets and sensitive information secure.

Github Actions allows you to store secrets and sensitive information as encrypted variables, which can be used in your workflows. However, it's important to keep these secrets and sensitive information secure, as they can be accessed by anyone with access to your repository.

To keep them secure, you should only grant access to authorised users, limit the scope of the secrets and sensitive information, and avoid storing them in plain text in your workflows.

Test and validate workflows.

Github Actions workflows should be thoroughly tested and validated before they are deployed to production. This includes testing the workflows on different platforms and environments, validating the inputs and outputs of the workflows, and ensuring that the workflows are reliable and error-free.

You can use Github Actions' built-in testing functionality, such as the jobs.<job_id>.steps.run syntax, to test and validate your workflows.

In conclusion, Github Actions is a powerful tool that can help automate many aspects of software development, but it's important to follow best practices when writing workflows to ensure that they are efficient, effective, and secure.

By keeping workflows simple and focused, using caching, reusing actions, keeping secrets and sensitive information secure, and testing and validating workflows, you can create robust and reliable workflows that help streamline your development process.

Congratulations! You have gotten to the end of the guide.

I’m currently in search of paid technical writing gigs related to cloud & devops, if you got one, reach out or refer me.

You can check out - My portfolio.

Bash vs Python Scripting: A Simple Practical Guide

Hussein Alamutu — Mon, 20 Mar 2023 16:14:00 +0000

Bash and Python are two popular scripting languages used for automation and system administration tasks.

This article aims to provide a simple, practical guide for beginners to understand the differences between Bash and Python scripting and when to use each one.

By the end of this article, readers will have a better understanding of the basics of Bash and Python scripting, as well as their strengths and weaknesses in different scenarios.

Prerequisites

To follow this tutorial you need the following:

Access to a linux machine(Ubuntu, Cent OS etc.) to enable you run bash commands on the terminal, since Bash is unix-based. If you don’t have that, git bash or Windows Subsystem for Linux(wsl) will do.
Python installation, whether Linux, Windows or MacOS, most times python comes pre-installed, you can confirm that by running python —-version on your system command line interface, if python exists, you will get the version number of the python on your system. If not you can install it by following this Python guide.

With that been said, let’s dive into the guide properly, fasten your seatbelts.

Bash Scripting

What is Bash & Bash Scripting?

Bash is a command-line shell used on Linux, macOS, and other Unix-like operating systems. While, Bash scripting is commonly used for automation, system administration, and software deployment tasks.

Bash scripts are easy to write and execute, and they can perform complex operations using a few lines of code. Bash provides many built-in commands, such as "echo"(used to print), "cd"(to change directory), "ls"(list), and "grep"(searches for a match to a given pattern), that can be used in scripts.

Bash scripts can be used to manipulate files, perform backups, and configure system settings, among other tasks.

Basics of Bash Scripting

Here's an explanation of the basics of Bash scripting:

1.Variables:

In Bash, a variable is used to store a value, such as a number or a string.
Variables can be declared and assigned a value using the equals sign (=).
For example, x=10 assigns the value 10 to the variable x.

2.Conditionals:

In Bash, conditionals are used to make decisions based on a certain condition.
The if statement is used to check whether a condition is true, and the else statement is used to specify what to do if the condition is false.
For example:



x=10
if [ $x -gt 5 ]
then
    echo "x is greater than 5"
else
    echo "x is less than or equal to 5"
fi

3.Loops:

In Bash, loops are used to iterate over a sequence of values or to repeat a block of code a certain number of times.
The for loop is used to iterate over a sequence of values, while the while loop is used to repeat a block of code as long as a certain condition is true.
For example:



# Using a for loop to iterate over a list of values
fruits=("apple" "banana" "cherry")
for fruit in "${fruits[@]}"
do
    echo "$fruit"
done

# Using a while loop to repeat a block of code
x=0
while [ $x -lt 10 ]
do
    echo "$x"
    x=$((x+1))
done

4.Functions:

In Bash, functions are used to encapsulate a block of code that can be called repeatedly with different inputs.
Functions are defined using the function keyword, and they can have inputs and outputs.
For example:



# Defining a function to calculate the area of a rectangle
function calculate_rectangle_area {
    width=$1
    height=$2
    area=$((width * height))
    echo $area
}

# Calling the function with different inputs
echo $(calculate_rectangle_area 3 4) # Output: 12
echo $(calculate_rectangle_area 5 6) # Output: 30

These are the basic building blocks of Bash scripting, and they can be combined to create more complex scripts.

More Bash Scripts

Here are some examples of Bash scripts for common tasks:

1. File manipulation:

A script that renames all files in a directory with a specific extension to have a new prefix.
A script that searches for a particular string in a file and replaces it with a new value.
A script that compresses all files in a directory into a single archive file.

Check this github repo for sample scripts of the above bash scripting.

2. System administration:

A script that backs up all files in a directory to a remote server using secure copy (SCP).
A script that monitors system logs for a particular error and sends an email alert when it occurs.
A script that automates the installation of software packages and updates on multiple servers.

For the system administration part, here's an example of a Bash script that automates the installation of software packages and updates on multiple servers:



#!/bin/bash

# List of servers to update
servers=("server1" "server2" "server3")

# Software packages to install
packages=("apache2" "mysql-server" "php")

# Update package lists on all servers
for server in "${SERVERS[@]}"; do
  ssh $server "sudo apt-get update"
done

# Install packages on all servers
for server in "${SERVERS[@]}"; do
  ssh $server "sudo apt-get install ${PACKAGES[@]} -y"
done

In this script, we first define a list of servers to update and a list of packages to install or update. We then loop through each server in the list and run the "apt-get update" command to update the system packages. We then loop through each package in the list and install or update it using the "apt-get install" command.

The -y option is used with apt-get install to automatically answer "yes" to any prompts during the installation process.

Note that this script assumes that you have SSH access to the servers and that you have sudo privileges to install software packages.

Python Scripting

What is Python & Python Scripting?

Python is a general-purpose programming language used for a wide range of applications, including web development, data analysis, AI and machine learning. Python provides a clear, concise syntax that is easy to read and write
Python has a large standard library and many third-party modules that can be used to perform complex operations

Python scripting is commonly used for automation, data processing, and scientific computing tasks. Python scripts can be used to scrape data from websites, process large datasets, and automate repetitive tasks, among other things.

Basics of Python Scripting

Here's an explanation of the basics of Python scripting:

1.Variables:

In Python, a variable is used to store a value, such as a number or a string.
Variables can be declared and assigned a value using the equals sign (=).
For example, x = 10 assigns the value 10 to the variable x.

2.Conditionals:

In Python, conditionals are used to make decisions based on a certain condition.
The if statement is used to check whether a condition is true, and the else statement is used to specify what to do if the condition is false.
For example:



x = 10
if x > 5:
    print("x is greater than 5")
else:
    print("x is less than or equal to 5")

3.Loops:

In Python, loops are used to iterate over a sequence of values, such as a list or a string.
The for loop is used to iterate over a sequence of values, while the while loop is used to repeat a block of code as long as a certain condition is true.
For example:



# Using a for loop to iterate over a list
fruits = ["apple", "banana", "cherry"]
for fruit in fruits:
    print(fruit)

# Using a while loop to repeat a block of code
x = 0
while x < 10:
    print(x)
    x += 1

4.Functions:

In Python, functions are used to encapsulate a block of code that can be called repeatedly with different inputs.
Functions are defined using the def keyword, and they can have inputs and outputs.
For example:



# Defining a function to calculate the area of a rectangle
def calculate_rectangle_area(width, height):
    area = width * height
    return area

# Calling the function with different inputs
print(calculate_rectangle_area(3, 4)) # Output: 12
print(calculate_rectangle_area(5, 6)) # Output: 30

These are the basic building blocks of Python scripting, and they can be combined to create more complex programs.

Python Modules & How to Use Them in Scripts

Python modules are pre-written pieces of code that can be imported into a script to add functionality such as working with files, processing data, sending emails, and more. . Here are some common Python modules and how to use them in scripts:

1.os module:

This module provides a way to interact with the underlying operating system.
Functions like os.chdir() to change the current working directory, os.mkdir() to create a new directory, and os.path.exists() to check if a file or directory exists can be used.
Example:



import os

# Change the current working directory
os.chdir('/path/to/new/directory')

# Create a new directory
os.mkdir('new_directory')

# Check if a file exists
if os.path.exists('/path/to/file'):
    print('File exists')
else:
    print('File does not exist')

2.datetime module:

-This module provides a way to work with dates and times.

Functions like datetime.datetime.now() to get the current date and time, datetime.timedelta() to calculate the difference between two dates or times, and datetime.datetime.strptime() to convert a string to a date or time object can be used.
Example:



import datetime

# Get the current date and time
current_time = datetime.datetime.now()
print(current_time)

# Calculate the difference between two dates or times
time_diff = datetime.timedelta(days=1)
yesterday = current_time - time_diff
print(yesterday)

# Convert a string to a date or time object
date_string = '2023-03-20'
date_object = datetime.datetime.strptime(date_string, '%Y-%m-%d')
print(date_object)

3.csv module:

This module provides a way to read and write comma-seperated-value(CSV) files.
Functions like csv.reader() to read a CSV file, and csv.writer() to write to a CSV file, can be used.
Example:



import csv

# Read a CSV file
with open('data.csv', 'r') as f:
    reader = csv.reader(f)
    for row in reader:
        print(row)

# Write to a CSV file
with open('output.csv', 'w') as f:
    writer = csv.writer(f)
    writer.writerow(['Name', 'Age', 'City'])
    writer.writerow(['Alice', '25', 'New York'])
    writer.writerow(['Bob', '30', 'San Francisco'])

4.shutil module:

The shutil module provides a higher-level interface for working with files and directories than the os module. It provides functions for copying, moving, and deleting files and directories.
For example:



import shutil

# Copy a file from one directory to another
shutil.copy("source/file.txt", "destination/file.txt")

# Move a file from one directory to another
shutil.move("source/file.txt", "destination/file.txt")

# Delete a file
os.remove("file.txt")

5.requests module:

The requests module provides a way to send HTTP requests and handle responses.
You can use it to download files, interact with web APIs, and scrape web pages.
Example usage:



import requests

# Download a file
url = "https://example.com/file.txt"
response = requests.get(url)
with open("file.txt", "wb") as file:
    file.write(response.content)

# Get data from a web API
url = "https://api.example.com/data"
response = requests.get(url, headers={"Authorization": "Bearer YOUR_TOKEN"})
data = response.json()

These are just a few examples of common Python modules and their usage. There are many other modules available that can help you accomplish various tasks in your scripts. You can search for them in the Python Package Index (PyPI) or through the Python documentation.

More Python Scripts

1. Web Scraping

Python is a popular language for web scraping, as it provides easy-to-use libraries like BeautifulSoup and requests.
Here's an example script that scrapes the top headlines from the BBC News website:



import requests
from bs4 import BeautifulSoup

url = "https://www.bbc.com/news"
response = requests.get(url)
soup = BeautifulSoup(response.text, "html.parser")

headlines = soup.find_all("h3", class_="gs-c-promo-heading__title")
for headline in headlines:
    print(headline.text)

This script uses the requests library to send an HTTP GET request to the BBC News website and fetch the HTML content. It then uses the BeautifulSoup library to parse the HTML and extract the headlines using a CSS selector.

2. Data Analysis

Python is a popular language for data analysis, with many powerful libraries such as pandas, numpy, and matplotlib available for use.
Here's an example Python script for data analysis using the pandas library:



import pandas as pd
import matplotlib.pyplot as plt

# Load data from a CSV file
data = pd.read_csv('data.csv')

# Print the first five rows of the data
print(data.head())

# Compute descriptive statistics
print(data.describe())

# Compute the correlation matrix
corr_matrix = data.corr()
print(corr_matrix)

# Plot a histogram of one of the variables
data['variable_name'].hist()

# Plot a scatter plot of two variables
data.plot.scatter(x='variable1', y='variable2')

# Group data by a categorical variable and compute summary statistics
grouped_data = data.groupby('category')['variable'].agg(['mean', 'std', 'count'])
print(grouped_data)

# Plot a bar chart of the means for each category
grouped_data['mean'].plot(kind='bar')

# Save a plot to a file
plt.savefig('output.png')

This script loads data from a CSV file, computes descriptive statistics and a correlation matrix, plots a histogram and a scatter plot of two variables, groups data by a categorical variable and computes summary statistics, plots a bar chart of the means for each category, and saves a plot to a file.

You can modify this script to analyse your own data by replacing the file name, variable names, and categorical variable with the appropriate names for your data.

Bash vs Python: A Fair Comparison

This section covers various aspects that should be considered while comparing Bash and Python scripting languages.

A. Syntax and readability:

The syntax of Bash scripting language is more complex and harder to read and understand than Python. Bash uses various special characters and symbols to represent different actions, which can make it harder to read and maintain. In contrast, Python's syntax is more straightforward, using indentation to denote code blocks, and a clean, easy-to-read syntax that is more accessible to beginners and experts alike.

B. Functionality and capabilities:

Python provides a wide range of libraries and modules that allow for a wide range of functionalities, such as data analysis, web development, artificial intelligence, and more. On the other hand, Bash scripting is mostly used for automating system-level tasks and commands, such as file management and system administration.

C. Performance and execution time:

Bash scripts tend to run faster than Python scripts because they do not require an interpreter to execute the code. Bash scripts are directly executed by the shell, which makes them faster than Python scripts that need to be interpreted by a Python interpreter. However, Python has several modules that can be used to optimize the code and improve performance.

D. Portability and compatibility:

Bash is available on most Unix-based systems and can be executed on any system that supports the Bash shell. In contrast, Python scripts may require the installation of Python and its dependencies on each system where the script will be executed, making it less portable. Additionally, Bash scripts may be more compatible with other shell commands and utilities used in the Unix shell environment.

E. Security and safety:

Python provides better error handling and a more secure execution environment than Bash. Bash scripts are more susceptible to shell injection attacks and other security vulnerabilities. Python's built-in security features, such as the capability to handle exceptions, prevent code injection, and other security features make it a more secure option for scripting tasks.

F. Maintenance and scalability:

Python's object-oriented approach, modular structure, and extensive libraries make it easier to maintain and scale than Bash. In contrast, Bash scripts can be more difficult to maintain as they tend to be more complex and lack the object-oriented structure that Python provides. Python scripts are more scalable as they can be easily extended and modified using libraries and modules.

Overall, these points highlight the key differences between Bash and Python scripting languages, making it easier for you to choose the appropriate language for your specific use case.

Congratulations! You have gotten to the end of the guide. Also, I’m currently in search of paid technical writing gigs related to cloud & devops, if you got one, reach out or refer me.

You can check out - My portfolio.

Serverless and its not so Server-less Nature

Hussein Alamutu — Mon, 08 Aug 2022 16:23:00 +0000

If you have ever wondered what serverless means and why it was built, then you are at the right place, in this article, I would share a bit of:

what serverless is(a little dive into history),
and, my favorite part - debunking some myths that has to do with serverless.

What you should know?

Why this article might not be so technical, it requires you to at least have a background knowledge of cloud computing.

What is Serverless?

Serverless has been gaining increasing interest over time and it seems like a very promising tech. it's not a technology or architecture, serverless is a bunch of solutions that promise great benefits in administering your application e.g. low entry barrier, cost efficiency, high availability and scalability.

How Serverless came to be, the evolution

Some people claim that serverless is the next stage of evolution in cloud computing.

How true is that?

To know the answer to this, I need to take you back to the era before the great wall of china was built, I'm kidding.

Before the era of the cloud, to build applications, you have to think of so many concepts (e.g. capacity management, installing hardware, managing servers, physical security, software updates, virtualization etc.) and all these distract you from the software development goal itself.

By 2006, Amazon launched Infrastructure as a Service, in which, with just a few clicks and an API request, I can provision a Linux-based server somewhere in North California while eating Jollof rice in my hometown in Nigeria, yeah, it's that easy. Now, all I have to focus on is software provisioning, OS management, scaling etc.

In just a small amount of time, cloud computing became very popular and everyone wanted to hop on the train, from Google and Microsoft creating their own IaaS service to traditional hosting companies also leveraging the cloud, the software development landscape started to change drastically.

Around 2013 things got even better, containers were introduced, let's call it Container as a Service(CaaS). Containers helped DevOps teams to accelerate software development, test and production cycle. This allowed us to package our applications in containers and run them on an orchestration platform like Kubernetes. CaaS reduced the amount of time spent on infrastructure, but not totally we still had to provision infrastructures, manage auto-scaling, monitor and log events etc.

Why Serverless was created.

With all the immense benefits the cloud has to offer, serverless took it even further. Serverless was introduced around 2014, which was mostly pioneered by Amazon.

Serverless promise is - you don't have to think about managing infrastructures, just provision your desired infrastructures using FaaS(Function as a Service) i.e. Azure functions and your service provider(i.e. Azure) will take it up from there.

However, while serverless has great benefits, it also has some hurdles, new technologies, tools and architectural patterns to be learnt.

Some companies that currently use serverless are Coca-Cola, Netflix, Codepen, Nordstream etc. Coca-Cola was able to cut the cost of processing vending machine transactions by 99% when they switched to serverless. As I said, it's a promising tech.

Now to my favourite part...

Some Myths about serverless.

1. Serverless and it's not so serverless tech

Myth: For the language gurus reading this, you should know linguistically, serverless means without servers. Well, that name doesn't tell the full story, it is just like cloud computing when it was first introduced and we asked questions like are there computers or servers in the cloud?

The answer is No, the servers are on the earth but can be accessed from anywhere.

Truth: There are still servers in the background running the backend services. Serverless just means you can focus on your work and let a cloud provider handle the provision of servers and infrastructure that runs seamlessly.

Why was it names serverless then?

Personal opinion, buzzwords like these(cloud, serverless) are more of marketing tactics.

2. It takes more than one click

Myth: Deploy your code and get it running with just one click.

Truth: It takes several processes to get your application to run on a serverless platform. There are technologies you still need to master, configurations and designs you need to make.

Using Azure as a case study. With Azure Functions, you can provision a fully managed compute platform for processing data, integrating systems, and building simple APIs and microservices. These automate administrative tasks from development through to deployment and maintenance.

Azure functions is a file-like manual that tells Azure what services should be provisioned i.e the number of servers, CPU size, Database to be integrated, event handlers or triggers etc.

Serverless reduces a lot of relative friction for the developer in terms of going through required steps that add no direct value to their workflow, like building a container for every new code release or monitoring, and logging.

If you would like me to write an ultimate guide on deploying with serverless or functions, let me know in the comment section.

3. Serverless is suitable for all applications

One major drawback of serverless computing is its non-persistent state or response latency.

Response latency is the time between when a request is stimulated and when a program reacts. In the cloud, serverless computing is not continually running, it gets powered down between requests.

Learn more about serverless response latency and its work-around here.

Phew!

Those are some myths debunked. Hopefully, now you have a better understanding of what serverless is and its historical background.

Congratulations! You have gotten to the end of the guide. Also, I’m currently in search of paid technical writing gigs related to cloud & devops, if you got one, reach out or refer me.

You can check out - My portfolio.

A Simple Guide to Container Orchestration with Kubernetes

Hussein Alamutu — Wed, 27 Jul 2022 08:55:38 +0000

Introduction

You'll get the most out of this guide if your desire is to learn container orchestration with Kubernetes.

The world of container orchestration is complex and ever-changing, but you can easily understand the basics, and that basic knowledge can make a big difference. There are also free Kubernetes courses widely available on the web, including guides like this.

Combine the information gotten from all these guides and courses with some practice and you are well on your way to becoming proficient in container orchestration.

What you'll learn

This guide focuses on an intermediate topic in Linux system administration. After reading through, you are expected to understand:

What is container orchestration
What is Kubernetes and
Container orchestration with Kubernetes using AWS.

The basics of container orchestration

Container orchestration is a part of a CI/CD pipeline in DevOps, it is used to automate the management, deployment, scaling and networking of containers.

In the CI/CD process, there are several container orchestration platform, the two major ones are Kubernetes and Docker swarm. This guide will focus on Kubernetes.

If you are familiar with CI/CD or DevOps, you should already know what containers are, they are a form of virtual machines that houses an operating system, they can be used to run a small microservice application and even large applications.

Why Orchestration?
Orchestration is the automated management of the lifecycle of our application

Orchestration helps us automate our deployment process for continuous deployment
Orchestration helps us handle complicated workflows in deploying our application

Understanding how containers work

Containers are like a standardized packaging for microservices applications with all needed application code and dependencies.

Prior to containerization, the traditional way of building and deploying codes was to run your software on a physical server where you have to install or use an existing operating system and also install dependencies for the software.

Meanwhile, containers are similar to virtual machines(VM) you create on your local machine when building applications, with VM you are able to create a replicate of your local machine, this allows you to run different versions of software dependencies.

In simpler words, using virtual machines, I am able to build two different applications that requires two different versions of, let's say node i.e. node v17.2.0 and node v18.6.0. If you build the two apps on your local machine it's impossible to use two different node versions at once.

Now, the best part, while containers are similar to VM, containers are way cooler. Containerization enables you to deploy multiple applications using the same operating system on a single virtual machine or server.

What is Kubernetes?

Kubernetes is a container orchestration system packed with features for automating an application’s deployment, it easily scales applications and ship new codes.

Well, read what Kubernetes' docs have to say

Kubernetes is a portable, extensible, open source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation.

Kubernetes consists of two major components for the continuous deployment process, pods and services.

Pods are abstractions of multiple containers and are also ephemeral. It's not uncommon to see a deployment involving a few containers to be deployed, hence the formation of pods.

Services are an abstraction of a set of pods to expose them through a network. Applications are often deployed with multiple replicas to help with load balancing and horizontal scaling.

NOTE:

Load Balancing: handling traffic by distributing it across different endpoints
Horizontal Scaling: handling increased traffic by creating additional replicas so that traffic can be divided across the replicas
Replica: a redundant copy of a resource often used for backups or load balancing

A practical overview

Prerequisites
This practical aspect assumes you have a previous knowledge on creating images and containers with docker and basic knowledge on cloud computing with AWS.

What you will need

A command line interface
A docker hub account
A docker image and container
An AWS account

Time to get dirty, wear your gloves, get your tools ready, let's get started.

Installing Kubernetes

There are many services that can be used to setup Kubernetes but this guide will focus on using AWS.

Why AWS?
Setting up Kubernetes from scratch it's a bit complicated, but AWS makes it easier. This guide will walk you through how to create Kubernetes cluster using AWS and how to create a node on AWS.

Note:

Kubernetes cluster consists of a set of nodes, that run containerized applications. When you deploy Kubernetes you get a cluster and every cluster has at least one worker node.
Nodes is an abstraction of pods managed by a control plane, the control plane handles scheduling of the pods across the Nodes in the cluster.

Getting our container ready

The Underlying Process
Docker images are loaded from the container registry into Kubernetes pods. Access to the pods are exposed to consumers through a service.

To deploy an application on Kubernetes, there are two basic configuration files that need to be created, can either be written in YAML or JSON, but YAML is the most used one. YAML is a great way to define our configurations because it is simple and efficient.

The first configuration is the "deployment.yaml" file.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
  labels:
    app: my-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: simple-node
        image: YOUR_DOCKER_HUB/simple-node
        ports:
        - containerPort: 80

Then the second one "service.yaml" file;

apiVersion: v1
kind: Service
metadata:
  name: my-app
  labels:
    run: my-app
spec:
  ports:
  - port: 80
    protocol: TCP
  selector:
    run: my-app

Once this file is available the next thing to do is to create a Kubernetes cluster using Elastic Kubernetes Service on AWS. Once logged into your AWS console, you can easily find it by typing into the search bar eks, then click on it.

The next step is to create an EKS cluster, navigate to the cluster section in the left panel, click on create cluster,

But before that, open a new tab, search for IAM, now...

Step 1: Create EKS Cluster IAM role:

Navigate to the Roles tab in the Identity and Access Management (IAM) dashboard in the AWS Console

Click Create role
Select type of trusted entity:
Choose EKS as the use case
Select EKS-Cluster
Click Next: Permissions
Click Next: Tags
Click Next: Review
Give the role a name, e.g. EKSClusterRole
Click Create role

Step 2: Create an SSH Pair

Navigate to the Key pairs tab in the EC2 Dashboard
Click Create key pair
Give the key pair a name, e.g. mykeypair
Select RSA and .pem
Click Create key pair

Step 3: Create an EKS Cluster

Navigate to the Clusters tab in Amazon EKS dashboard in the AWS Console
Click Create cluster
Specify: a unique Name (e.g. MyEKSCluster), Kubernetes Version (e.g. 1.21), Cluster Service Role (select the role you created above, e.g.EKSClusterRole)
Click Next
In Specify networking look for Cluster endpoint access, click the Public radio button
Click Next and Next
In Review and create, click Create

It may take 5-15 minutes for the EKS cluster to be created.

Troubleshooting: If you get a message like this:

Cannot create cluster the targeted availability zone does not currently have sufficient capacity to support the cluster

choose another availability zone and try again. You can set the availability zone in the upper right corner of your AWS console.

Step 4: Create a Node Group
Now, go back to the opened IAM tab,

- Create EKS Cluster Node IAM role

In the IAM Roles tab, click Create role
Select type of trusted entity:
Choose EC2 as the use case
Select EC2
Click Next: Permissions
In Attach permissions policies, search for each of the following and check the box to the left of the policy to attach it to the role: "AWSAmazonEC2ContainerRegistryReadOnly, AmazonEKSWorkerNodePolicy, AmazonEKS_CNI_Policy".
Click Next: Tags
Click Next: Review
Give the role a name, e.g. NodeRole
Click Create role

Now, go back to the Cluster tab...

Creating the node

Click on the Compute tab in the newly-created cluster
Click Add Node Group
Specify: a unique Name (e.g. MyNodeGroup)
Cluster Service Role (select the role you created above, e.g.NodeRole)
Create and specify SSH key for node group
In Node Group compute configuration, set instance type to t3.micro and disk size to 4* to minimize costs
In Node Group scaling configuration, set the number of nodes to 2
Click Next
In Node Group network configuration, toggle on Configure SSH access to nodes
Select the EC2 pair created above (e.g. mykeypair)
Select All
Click Next
Review the configuration and click "Create"

At this point, we have a Kubernetes cluster set up and understand how YAML files can be created to handle the deployment of pods and expose them to consumers.

Moving forward, the Kubernetes command-line tool (kubectl), will be used to interact with our cluster. The YAML files that we created will be loaded through this tool.

Interacting With Your Cluster

The next step is to load the Infrastructure as Code(IaC) configurations.

Loading YAML files, these are the deployment.yaml and service.yaml files. Load them to Amazon EKS resource using the following commands:

kubectl apply -f deployment.yaml

Step 1: Deploy resources

Send YAML files to Kubernetes to create resources. This will create the number of replicas of the specified image:

kubectl apply -f deployment.yaml

and create the service:

kubectl apply -f service.yaml

Step 2: Confirm deployment

Verify that the resources have been created:

kubectl get pods

and check to see that they were created correctly:

kubectl describe services

To get more metadata about the cluster, run:

kubectl cluster info dump

By loading these configuration files to the Kubernetes cluster, you have set up Kubernetes to pull your created Docker image from your DockerHub's account.

Troubleshooting

If you ran into some issues:

Check to see if node groups was successfully created in your EKS cluster. If it wasn't, Kubernetes won't have enough pods setup.
If you get an error message concerning not been able to pull your Docker images, confirm that your DockerHub repo is set to public.

What Next?

The next step is to secure and tune Kubernetes services for production.

Managing Cost

We can start with configuring the clusters to greatly reduce costs(i.e specifying the number of replicas to be created and also minimize the resources used such as CPU).

Security Consciousness
Make sure your Kubernetes cluster are secured from those with malicious intent, you can do that by configuring who has access to the Kubernetes pods and services.

Applications deployed for production use varies differently from the ones for development. In the case of production the application is no longer running in an isolated environment. It has to be configured with least access privileges to prevent unexpected traffic and allow only expected traffic to access it.

Preparing for Scaling and Availability

Ensure the Kubernetes service is able to handle the number/size of user requests and the application is responsive i.e. able to be used when needed.

One way to ascertain this prior to releasing your application to a production environment is to use load testing mechanism, this simulates a large number of requests to our application, which helps us set a baseline understanding of the limits of our application.

Handling Backend Requests with Reverse Proxy

A reverse proxy is a single interface that forwards requests from the frontend(i.e. user of our application) and this appears to the user as the origin of the responses.

An API gateway functions as a reverse proxy that accepts API requests from users, get the requested services and return the right result.

Nginx is a web server that can be used as a reverse proxy, configurations can be specified with an nginx.conf file.

Accessing the Kubernetes Cluster using Reverse Proxy

First, check the pods to see what is running, using:
kubectl get pods

Then, check the services to find the entry point for accessing the pod:

kubectl describe services

In the output you will find the service Name(i.e. alamz-app-svc) and Type(ClusterIP) which means that the service is only accessible within the cluster.

Now, create an nginx.conf file. A sample nginx.conf file looks like this:

events {
}
http {
    server {
        listen 8080;
        location /api/ {
            proxy_pass http://alamz-app-svc:8080/;
        }
    }
}

The Nginx service listens for requests at port 8080. Any requests with endpoints prefixed with /api/ will be redirected to the Kubernetes service alamz-app-svc. alamz-app-svc is a name that our Kubernetes cluster recognizes internally.

Put this in your Dockerfile:

FROM nginx:alpine

COPY nginx.conf /etc/nginx/nginx.conf

The nginx.conf sets up the service to listen for the requests that come in to port 8080 and forward any requests to the API endpoint to http://my-app-svc endpoint in the app.

Setting Up the YAML files for the Reverse Proxy

The setup is similar to the previous yaml file created in this guide. The basic functionality is to: Create a single pod named reverseproxy and configure it to limit resources.

Your "deployment.yaml" file should look like this:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    service: reverseproxy
  name: reverseproxy
spec:
  replicas: 1
  template:
    metadata:
      labels:
        service: reverseproxy
    spec:
      containers:
      - image: YOUR_DOCKER_HUB/simple-reverse-proxy
        name: reverseproxy
        imagePullPolicy: Always          
        resources:
          requests:
            memory: "64Mi"
            cpu: "250m"
          limits:
            memory: "1024Mi"
            cpu: "500m"       
        ports:
        - containerPort: 8080
      restartPolicy: Always

Your "service.yaml" file should look like:

apiVersion: v1
kind: Service
metadata:
  labels:
    service: reverseproxy
  name: reverseproxy-svc
spec:
  ports:
  - name: "8080"
    port: 8080
    targetPort: 8080
  selector:
    service: reverseproxy

Deploying Reverse Proxy

Finally, we are ready to deploy our reverse proxy. Deploy the reverse proxy using:

kubectl apply -f reverseproxy_deployment.yaml
kubectl apply -f reverseproxy_service.yaml

kubectl is the command line tool used to interact with our kubernetes cluster and the YAML file is the Infrastructure as Code(IaC) that specifies the configuration for our reverse proxy.

Gracias Amigos

Congrats, If you have carefully followed up to this point, gracias amigo!, while I am fully aware this guide isn't an all inclusive one, I make sure to point you to the right direction, where you can get the information that wasn't included.

Meanwhile, I am putting this guide out here to serve as the MVP(minimum viable product)... what? Is this now a product management class? Well.. No. I am just a Cloud DevOps Engineer with a Product Design background that somehow fell in love with writing. Okay, enough of me.

Let me cut to the chase, if there is anything you want me to go deeper on, or an information you feel I skipped, please, make sure to inform me, so I can Include it in the next release.

Okay, that's it fellow techies. See Ya! Ehhmm, one sec, to supplement this guide I will write a short article that talks about securing clusters and containers, and resource quota management.

Linux: Managing RPM Packages with YUM

Hussein Alamutu — Tue, 19 Jul 2022 20:21:23 +0000

Intro

Package managers for Linux are basically tools or software applications that allows users to install, uninstall, update, configure and manage software packages on Linux. There are various package managers for Linux. However, this article will focus on YUM.

Target audience

This article assumes you have basic or intermediate knowledge of Linux, RPM and/or navigating the Linux CLI, If you have no previous knowledge of these topics, I recommend you read them prior to reading this.

What you'll learn

This article focuses on an intermediate topic in Linux system administration. After reading through, you are expected to understand:

What is YUM and how to install it
Difference between YUM and RPM
Managing software with the YUM command

What you'll need

A computer running Fedora or some other version of Red Hat-based Linux

What is YUM and how to install it

The YellowDog Updater Modified(YUM) is a command line package management utility for the Linux operating system, it is used for managing Linux RPM software packages.

While there are differences between YUM and RPM, YUM still uses the RPM package format.

Look at what Wikipedia have to say:

Under the hood, YUM depends on RPM, which is a packaging standard for digital distribution of software, which automatically uses hashes and digital signatures to verify the authorship and integrity of said software.

Yum evolved from Yellowdog Updater(YUP). YUP was created between 1999-2001 to serve as a back-end engine for Linux's graphical installer.

Now that you know a little history of yum, let's get down to using it. Fedora should come pre-installed with YUM as a default package, run the following command to confirm if YUM is installed.

which yum

but just in case you don't have YUM installed on your system, you can install it with...

sudo dnf install yum

Now type in the previous command to confirm if yum is installed. Before diving deeper into using the YUM package manager, get a grasp of the similarities and differences between YUM and RPM.

Difference between YUM and RPM

Both YUM and RPM are package managers, the biggest drawback for RPM is that it cannot resolve package dependencies, for this amongst many other reasons YUM was created.

It varies differently from RPM in various ways, to mention a few:

YUM resolves package dependencies automatically
It can install multiple versions of a package
It automatically upgrades packages
With YUM, you can go back to previous versions of a package

Package management with YUM

yum help - Displays list of YUM commands and options
yum install <package_name> - Installs whatever package name was given in the command.
yum update - Updates the package
yum downgrade - Returns the package to an earlier version
yum remove - Removes the package and it's dependencies
yum info - Displays information about the package.
yum list - List package names

Conclusion

By reading this article, you should have learnt about YUM package manager, how to use it and it's common commands. I urge you actively run the commands yourself, to make sure the learning sticks.

However, YUM has been replaced with Dandified YUM(DNF), which means YUM is no more the primary package manager in Fedora. For more information on DNF check out the following articles:

DEV Community: Hussein Alamutu

Demystifying For Loops in Terraform: A Practical Guide

Understanding For Loops in Terraform

Best Practices

The End -

How to automate the installation of web servers on AWS EC2 using Terraform and Github Actions

Prerequisites

Contents

I. Setting up AWS Access Key and Secret Key

II. Creating the Github repository and setting up Github Actions

III. Writing Terraform code

Understanding Virtual CPUs: Exploring the Differences Between Cores and Threads

Virtual CPU

Cores vs Threads

Advantages of Virtual CPUs in Cloud Computing and Virtualization

Limitations of Virtual CPUs

Conclusion

The How, Why and When of Github Actions.

Why Use Github Actions?

When to use Github Actions?

The How of GitHub Actions

Bonus: Best practices for Github Actions

Bash vs Python Scripting: A Simple Practical Guide

Prerequisites

Bash Scripting

Basics of Bash Scripting

More Bash Scripts

Python Scripting

Basics of Python Scripting

Python Modules & How to Use Them in Scripts

More Python Scripts

Bash vs Python: A Fair Comparison

Serverless and its not so Server-less Nature

What is Serverless?

How Serverless came to be, the evolution

Why Serverless was created.

Some Myths about serverless.

A Simple Guide to Container Orchestration with Kubernetes

Introduction

What you'll learn

The basics of container orchestration

Understanding how containers work

What is Kubernetes?

A practical overview

Installing Kubernetes

Getting our container ready

Interacting With Your Cluster

What Next?

Gracias Amigos

Linux: Managing RPM Packages with YUM

Intro

Target audience

What you'll learn

What you'll need

What is YUM and how to install it

Difference between YUM and RPM

Package management with YUM

Conclusion