DEV Community: Ivy Jeptoo

How to Create AWS EKS Cluster

Ivy Jeptoo — Thu, 14 Sep 2023 08:19:08 +0000

Amazon Elastic Kubernetes Service (Amazon EKS) is a managed Kubernetes service provided by Amazon Web Services (AWS). It simplifies the deployment, management, and scaling of containerized applications using Kubernetes.

Advantage of EKS

EKS is fully managed, so AWS handles control plane maintenance, scaling, and updates, allowing you to focus on your applications.
It offers high availability across multiple AWS Availability Zones, ensuring uptime and fault tolerance for your Kubernetes clusters.
Security, It integrates with AWS IAM for authentication and authorization, and you can apply IAM policies for fine-grained access control.
EKS seamlessly integrates with AWS services, simplifying application deployments and operations.
EKS is user-friendly, compatible with standard Kubernetes tools, and simplifies Kubernetes cluster management.

Elastic Kubernetes Service(EKS) can be created in two ways,

Web Console
AWS CLI tool

I Web Console

Prerequisites

i) Ensure you have a default VPC. This will automatically createa size /20 default subnet in each availability zone. If you don't have one follow this instructions to create one.

A default VPC will look like so:

ii) Create an IAM role that your cluster and the node group will assume. A role is a set of permissions to be assigned to an entity.

Below are the steps you will follow after selecting create a new role
- Click on the Create role button to start the wizard.
- Choose AWS service as the trusted entity.

Click on the EKS to see EKS use cases. (See the snapshot below)
Choose EKS - Cluster. It will allow access to other AWS service resources that are required to operate clusters managed by EKS. Click Next.

The needed policy, AmazonEKSClusterPolicy, will be selected. This policy provides Kubernetes the permissions it requires to manage resources on your behalf.
Click Next, and ignore the Tags.
Click Next, and name the role

iii) Create an IAM role for the worker nodes, this wil give permisssion to kubelet running on the worker node to make calls to other APIs on your behalf. The steps will be the same as above only that:

In the Use case you will select EC2 instead of EKS case.

In the attach policy you need to give choose the following
- AmazonEKSWorkerNodePolicy
- AmazonEC2ContainerRegistryReadOnly
- AmazonEKS_CNI_Policy

iv) Create an SSH key Pair that we'll use to log into EC2 Instance, the public key is placed automatically on the EC2 instances, whereas you use the private key instead of a password to access your instances securely.

To create, go to EC2 service → Networkk & Security → Key Pairs.
- Click on Create key pair
- name your key pair then chose a format. (.pem format is used by Mac/Linux users, and a .ppk format is used by Windows users.)

private key file will be downloaded locally.

Create EKS Cluster

An EKS cluster consists of:
Control place which has nodes running the K8 software like the kubernetes API and the etcd which run in AWS-owned accounts.
Data plane is made up of worker nodes which run in customer accounts.

Create a Control Plane
Step 1
Under EKS Service→ Amazon EKS→ Clusters, click on create cluster.

Give your cluster a name and choose kubernetes version, select the IAM role we created earlier Step 2 Chose the default VPC, subnets and security group in your account. Mark the cluser endpoints as public.

step 3
Accept the default set for the rest of the steps and create the cluster.

Create a Node Group

Node groups are worker nodes(VMs) used to run the pods that your cluster will be serving. We'll create a node group and attach it to the cluster.
step 1
Once the cluster that we created earlier is Active, click on the name for more details

step 2
Click on Compute under the new cluster the click on Add Node Group

Step 3
Give it a name then attache the IAM node role we created earlier.

Step 4
Under Node group and compute and Scaling Configuration, choose the OS,hardware config and worker node count.

Field	Value	Purpose
AMI type	Amazon Linux 2 (AL2_x86_64)	OS
Capacity type	On-Demand	Instance purchasing option
Instance types	t3.micro	2 vCPU, 1 GiB memory
Disk size	20 GiB	---
Scaling configuration
Min size	2	Min number of nodes for scaling in.
Max size	2	Max number of nodes for scaling out.
Desired size	2	Initial count

step 5
Choose the subnets we created earlier while creating the cluster and also choose the SSH key pair we created earlier. Allow remote access from anywhere on the internet.

Clean Up

Delete the Node Group. Explore how you'd do the deletion. If you need help, refer to the instructions here.
Delete the cluster.
Delete the custom IAM roles you have created in this exercise.

II AWS CLI Tool

Creating EKS using the AWS CLI involves resources and is way ekctls CLI is used to to simplify cluster creation. eksctl uses services of AWS CloudFormation internally to create clusters on AWS.
- AWS CloudFormation is an AWS service for creating, managing, and configuring ANY resource on the AWS cloud using a YAML/JSON script. In the script file, you can define the properties of the resource you want to create in the cloud.

In the case of a simple cluster, eksctl will not need to create a script but for a more complex one you will be needed to a minimal YAML script.

eksctl Installation

Linux

curl --silent --location "https://github.com/weaveworks/eksctl/releases/download/latest_release/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
sudo mv /tmp/eksctl /usr/local/bin

Windows

# Install Chocolatey. Refer to the https://chocolatey.org/install  for detailed steps
Set-ExecutionPolicy AllSigned 
Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))
# Exit and re-run Powershell as an Admin
chocolatey install eksctl
# Verify
choco -?

Mac OS

# Check Homebrew 
brew --version
# If you do not have Homebrew installed - https://brew.sh/ 
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
# Install eksctl
brew tap weaveworks/tap
brew install weaveworks/tap/eksctl

If you face any error due to ownership permission, you can change the ownership of those directories to your user.

sudo chown -R $(whoami) /usr/local/<directory_name>

Create a basic cluster

Once you have you have installed eksctl, create a basic cluster,

eksctl create cluster

The cluster will generate:

An auto-generated name
Two m5.large worker nodes. Recall that the worker nodes are the virtual machines, and the m5.large type defines that each VM will have 2 vCPUs, 8 GiB memory, and up to 10 Gbps network bandwidth.
Use the Linux AMIs as the underlying machine image
Your default region A dedicated VPC

You can specify it on one command:

eksctl create cluster --name myCluster --nodes=4

Create an advanced cluster

you will need to write the configurations in a YAML file separately then run

eksctl create cluster --config-file=<path>

List the details

This is specific to a cluster

eksctl get cluster [--name=<name>][--region=<region>]

Delete Cluster

This will delete a cluster and all the resources associated to it

eksctl delete cluster --name=<name> [--region=<region>]

How to deploy AWS Serverless Application in Cloud9

Ivy Jeptoo — Tue, 12 Sep 2023 09:46:39 +0000

Howdy!

In this tutorial we are going to initialize, build and deploy a simple Hello world application using AWS Serverless Application.
The Hello World application contains resources which includes:

AWS Lambda – Function that processes the HTTP API GET request and returns a hello world message.
AWS Identity and Access Management (IAM) role - Provisions permissions for the services to securely interact.
- Provisions permissions for the services to securely interact - API endpoint that you will use to invoke your function.

Prerequisites:

Before installing AWS SAM CLI ensure that
- Create an AWS account, AWS Identity, IAM credials and access key pair.
Install AWS CLI
Use AWS CLI to configure AWS credentials

Initialize Hello World Application

We are going to use the AWS SAM CLI to create the application on our Cloud9.

i). Run the command below on your desired directory

sam init

ii). The prompts will guide you through the initializing a new application.

Select AWS Quick Start Templates to choose a starting template.
Choose the Hello World Example template and download it.
Use the Python3.8 runtime and image package type.
Opt out of AWS X-Ray tracing. See What is AWS X-Ray? in the AWS X-Ray Developer Guide.
Opt out of monitoring with Amazon CloudWatch Application Insights. Learn about Amazon CloudWatch Application.
Name your application

Use the image below for reference

cont:

iii). The AWS SAM CLI then downloads your the template then creates the application directory.

iv). Navigate to the newly created directory. Now, let's breakdown the files:

hello_world/app.py – Contains your Lambda function code.
hello_world/requirements.txt – Contains any Python dependencies that your Lambda function requires.
samconfig.toml – Configuration file for your application that stores default parameters used by the AWS SAM CLI.
template.yaml – The AWS SAM template that contains your application infrastructure code.

Build the Application

Building the app will dockerize the lambda function in cloud9 environment. Use the command:

sam build

After building AWS SAM CLI creates a .aws-samdirectory,it organizes your function dependencies, project code and files there.

.aws-sam
├── build
│   ├── HelloWorldFunction
│   │   ├── __init__.py
│   │   ├── app.py
│   │   └── requirements.txt
│   └── template.yaml
└── build.toml

this is what each file does:

build/HelloWorldFunction – Contains your Lambda function code and dependencies. The AWS SAM CLI creates a directory for each function in your application.
build/template.yaml – Contains a copy of your AWS SAM template that is referenced by AWS CloudFormation at deployment.
build.toml – Configuration file that stores default parameter values referenced by the AWS SAM CLI when building and deploying your application.

Deploy the application to an ECR image repository

Before building ensure that you have configured your AWS credentials.
The application files will be uploaded to S3 and the SAM template is transformed into a CloudFormation. The template is then uploaded the template to the AWS CloudFormation service to provision your AWS resources.
To deploy, run the command below:

sam deploy --guided

Be sure to configure the deployment steps here is an example of how to

Confirm the deployment changes
After successful deployment an API gateway endpoint URL that you can curl or paste in a browser to see the function output.

Alternatively

You can use the sam list endpoints-output json command to get information

Testing

Ensure that you are in the correct directory before running the testing command. Run the command to test the function

sam local invoke

The expected output is as below:

Since sam local invoke does not test API we are going to use the command below.

sam local start-api

To check on what we are calling, you can open a new terminal the run curl and the endpoint.

curl http://127.0.0.1:3000/hello

The payload is successful as below:

This is the basis of getting started with SAM!
Happy Coding!

EC2 Instance with an Admin Role

Ivy Jeptoo — Tue, 18 Jul 2023 13:08:22 +0000

Howdy!!!
Some time back we learnt how to create and EC2 Instance and we even connected to it, don't remember? Check it out here.

Today we are going to create an Instance based on Amazon Linux AMI that will be connected via SSH. Using Security Groups, you will ensure that access to the instance is limited to your IP address only.
The CLI will be pre-installed on the instance by default. This instance only needs permissions assigned. Once the instance is up and running, create an IAM role with admin access for your account. Add the role to your EC2 after that.

Table of Content

Objectives
Create a default Virtual Private Cloud
Launch an EC2 instance
Create an IAM Role
Attach the Role to the EC2 Instance
Connect to your EC2 instance
Conclusion

Objectives

By the end of this article, you'll be able to:

Launch a secure EC2 instance.
Create IAM role with admin previleges
Attach the IAM role to the your Instance.
Connect to your EC2 instance via SSH
Use CLI tool in the Instance.

Create a default Virtual Private Cloud

VPC is a private cloud computing environment contained within a public cloud and once can launch AWS resources in.
Check in your account if you already have a default VPC and if not, go to the VPC dashboard and create a default VPC.

Launch an EC2 instance

I already have an article that covers launching EC2 but just to touch briefly on the steps:

Under security limit access to your IP address only.

If you do not have a SSH key be sure to download a new one.

N/B
This key-pair will allow you to log into your instance, using SSH, from your local machine. Save the key-pair carefully, because the same private key cannot be re-generated.

Once you have launched your Instance, verify that it is running successfully.

Create an IAM Role

Identity and Access Management is used to specify who and what can access services and resources in AWS.

On the IAM dashboard, select Roles
Click on Create role button

Select AWS service as the trusted entity and EC2 as the use case. This will allow the instance to whom the role will be attached to to be able to call any AWS service on your behalf.

Under the permissions, search for AdministratorAccess in Filter policies textbox to apply to the role.

Under the review section, provide a name to the new role.

Attach the Role to the EC2 Instance

On the EC2 dashboard, check on the running instances and select the checkbox on the Instance we created earlier.
Click the Actions button which will open a drop-down options, select the Security → Modify IAM role

Select and apply the newly created role to your Instance

Connect to your EC2 instance

We are going to connect to the EC2 instance using SSH, under Actions, click on Connect

Follow the SSH steps to connect to the Instance.

After connecting to the instance, you need to verify installation of AWS CLI

Conclusion

This is a practical method for having a well configured, secure server that you can use for testing without worrying about credentials.

Happy cloud adventures!

AWS Cloud Fundamentals

Ivy Jeptoo — Thu, 13 Jul 2023 17:00:55 +0000

Cloud Computing is the delivery of computing services (servers, storage, database, networking, software etc) over the internet using pay-as-you-go pricing. With this you can access technology services depending with your needs on cloud providers like Amazon Web Service, Google Cloud Platform, Microsoft Azure etc.

Advantages of Cloud Computing

Cost Effectiveness - you pay for the resources that you use this helps avoiding overbuilding and over provisioning.
Security - Top notch security is provided through Virtual private cloud, encryption and API keys which help keep data secure
Scalability - You can easily scale resources and storage up to meet your business demands without investing in physical infrastructure, one can also scale down if resources aren't being used.
Flexibility - It allows users to select the operating systems, language, database, and other services as per their requirements.

you can read more on the advantages here

Table of Content

Compute
Database
Security
Networking
Messaging
Management Services
Conclusion

Overview of AWS Cloud Fundamentals

AWS provides several services which include: Compute, Storage, Security, Networking, Messaging, management services etc. Many more services are provided but we'll discuss the above.

Compute

Amazon EC2 - provides secure, resizable capacity in the cloud based on the user requirements. It can shrink or expand resources in accordance to the load.
Amazon Elastic Beanstalk - scales and deploys web applications made in programming languages like java, python, ruby. It deals with capacity provisioning, load balancing and auto scaling.
Amazon LightSail - It enables Virtual Private Server(VPS) to be launched and managed with ease. It automatically deploys and manages the computer, storage, and networking capabilities required to run your applications.
EKS (Elastic Container Service for Kubernetes) - The tool allows you to Kubernetes on Amazon cloud environment without installation.
AWS Lambda - This AWS service allows you to run functions in the cloud. The tool is a big cost saver for you as you to pay only when your functions execute.

Storage

AWS S3 (Simple Storage Service) - an object storage that can store and retrieve data from anywhere: websites, mobile apps, IoT sensors, and so on. It is durable, provides comprehensive security, and flexibility in managing data.
Amazon Glacier - is used for archiving data and long-term backup. The glacier is used for data archiving and long term backup.

Amazon EBS(Elastic Book Store) - provides block storage volumes for instances of Amazon EC2. EBS is a reliable storage volume that can be attached to any running instance that is in the same availability zone. I wrote an article on EBS
Amazon Elastic File System - provides elastic file storage, which can be used with AWS Cloud Services and resources that are on-premises. It is easy to use and offers a simple interface that allows you to create and configure file systems quickly and easily.

More on storage

Database

Amazon RDS(Relational Database Service) - Helps in administration and management of database It frees managing the hardware and enables us to focus on the application.
- DynamoDB - is a fully managed, serverless, key-value NoSQL database designed to run high-performance applications at any scale. DynamoDB offers built-in security, continuous backups, automated multi-Region replication, in-memory caching, and data import and export tools.
Amazon Redshift - users to analyze their data using SQL. It is a fast, fully managed data warehouse. It also allows users to run complex analytical queries against structured data using sophisticated query optimizations.

More on database

Security

Identity and access management (IAM) - allows one to configure who can access AWS account. It controls access to resources by authenticating and authorizing users, apps and services.
Web Access Control List - monitors HTTPs requests for AWS resources
AWS Firewall Manager - Firewall is a network security mechanism that monitors and controls incoming and outgoing traffic while the firewall manager allows configuration and management of firewall rules access accounts.
AWS Shield - provides continuous DDoS(Distributed Denial of Service) attack detection and automatic mitigations. This safeguards applications running on AWS.

More on security

Networking

Elastic Load Balancing - automatically diverts traffic to multiple targets, balances load between servers and also provide redundancy and performance.
Autoscaling - automatially adjust resource usage to ensure steady performance at the lowest cost while EC2 autoscaling monitors EC2 instance and automatically adjusts by adding or removing EC2 instance based on conditions you define.
Route 53 - is a cloud domain name system that has servers distributed around the globe, it routes end users to internet application.

More on networking

Messaging

SNS (Simple Notification Service) - allows sending of notification to users on your application(large subscribers) through various protocols like email, SMS, and HTTP/S.
SQS (Simple Queue Service) - allows you to integrate queuing functionality in the application. It can be Standard of FIFO(First In First Out).

Queuing is a data structure that holds requests(messages). Processing data using FIFO improves scalability, performance, user experience.

More on messaging

Management Services

AWS CloudWatch - monitors and manages AWS resources and application by collecting data in form of logo, metric and events
AWS CloudFormation - models infrastructure text files template allowing provisioning of AWS resources based on the written scripts. Infrastructure as code allows description and provision of all infrastructure resources in cloud environment

Conclusion

Cloud fundamentals are essential to grasp the basics of cloud computing, a transformative technology for data storage, access, and processing. Understanding cloud fundamentals is crucial for leveraging scalability, flexibility, cost-efficiency, and collaboration benefits. It enables informed decision-making, selection of service models, and consideration of security and privacy. Additionally, it facilitates cloud-native application development, big data utilization, and effective disaster recovery planning. Mastering cloud fundamentals is vital for competitiveness in the digital landscape.

Elastic Block Store(EBS)

Ivy Jeptoo — Fri, 23 Jun 2023 14:42:46 +0000

We previously learnt how to create an EC2 instance and connect to it. you can recap here.

While still on this there are two types of storage in EC2:

i) In Memory(instance store) - which provides temporary block-level storage that is directly attached to the EC2 instance. It is ideal for temporary data, cache, or scratch space that doesn't need to persist beyond the lifespan of the instance.

ii) EBS( elastic block store) - provides persistent block-level storage volumes that can be attached to EC2 instances. It offers durable and reliable storage that persists independently of the instance's lifespan.

Let us dive more into EBS.

Amazon Elastic Block Store (EBS) provides storage volumes for use with EC2 instances. These volumes act like raw, unformatted block devices that can be mounted on your instances. They persist independently from the life of the instance. You can create a file system or use them like a hard drive. EBS volumes offer fast access to data and long-term persistence, making them great for file systems, databases, and applications that need frequent updates and direct access to block-level storage. They work well for both random reads and writes and continuous read and write operations.

Features of EBS

a). EBS provides General Purpose SSD, Provisioned IOPS SSD, Throughput Optimized HDD, and Cold HDD as volume types.

General Purpose SSD volumes (gp2 and gp3):
- Ideal for transactional workloads.
- Balance price and performance.
- Suitable for boot volumes, medium-size single instance databases, and development/test environments.
Provisioned IOPS SSD volumes (io1 and io2):
- Designed for I/O-intensive workloads.
- Offer consistent and predictable IOPS rate.
- Scale to tens of thousands of IOPS per instance.
- io2 volumes provide highest volume durability.
Throughput Optimized HDD volumes (st1):
- Low-cost magnetic storage.
- Performance measured in terms of throughput.
- Suitable for large, sequential workloads like Amazon EMR, ETL, data warehouses, and log processing.
Cold HDD volumes (sc1):
- Low-cost magnetic storage.
- Performance measured in terms of throughput.
- Ideal for large, sequential, infrequently accessed data.
- Cost-effective solution for storing cold data.

b). EBS Volume Encryption - This allows one to encrypt their EBS volumes to meet data-at-rest encryption requirements for regulated or audited data. It also ensures that the stored data in the volume. disk I/O and snapshot is encrypted.

Encryption happens on the servers hosting EC2 instance which secures data both at rest and during transit.

c). EBS Snapshots allows you to create point-in-time copies of your EBS volumes which are stored in S3. The snapshot provide long-term durability and serve as a starting point for new EBS volumes.

The same snapshot can be used to create multiple volumes as needed and the snapshots can be copied across different AWS Regions.

d). Performance Metrics such as bandwidth, latency, and average queue length are provided by Amazon Cloudwatch and it allows you to monitor the performance of your EBS volume.

Monitoring helps ensure that you are allocating sufficient performance resources for your applications without paying for unnecessary resources.

e). When you create an EBS volume, it is linked to a specific Availability Zone (AZ). This means that the volume is located in a particular data center within a specific geographic region. However, if you want to use the volume in a different AZ within the same AWS Region, you can create a snapshot.

A snapshot is a copy of your EBS volume's data and settings. It captures all the information stored on the volume at a specific point in time. By creating a snapshot, you essentially create a backup of your volume.
Once you have a snapshot, you can use it to restore the volume to a new EBS volume in any AZ within the same AWS Region. This means you can move your volume's data from one AZ to another without losing any information. The restored volume will have the same data and settings as the original volume at the time the snapshot was taken.

Creating EBS Volume

AWS allows us to create a volume from either of the following three methods:

i) Create and attach EBS volumes while creating an EC2 instance using the Launch Instance wizard.
ii) Create an empty EBS volume, and later you can attach it to a running instance.
iii) Create an EBS volume from a previously created snapshot, and later you can attach it to a running instance.

We are going to use option 2 to create EBS

STEPS

On the EC2 Dashboard select the Elastic Block Store → Volumes service on the left navigation pane.
Select the Create Volume button as on the screenshot below:

Specify the volume details on the set-up wizard page.
You will have to specify the following details:

i)Volume type - AWS offers various types of volumes, as described in the table below.
ii)Size (GB) - Mention the size with-in limits of the type you have chosen above.
iii) Availability Zone - It has a default value, or you can choose your preferred AZ.
iv)Snapshot ID - Specify the ID of the snapshot if you wish to create a volume from an existing snapshot. Remember, a snapshot is the saved state of another volume at a particular moment.
v) Tag - Specify the key-value pair, such as {Name: U-test}

Click on the Create Volume and your EBS will be created.

EBS Dashboard

On the EC2 Dashboard select the Elastic Block Store → Volumes service in the left navigation pane.

List of all Volumes

On the image above it is labeled as 1, here you can view all the volumes available under your account in a specific region.

There is ID, size, type, I/O per second, snapshot ID of that volume, date of creation, availability zone, current status, whether the volume is encrypted and the EC2 instance to which it is attached.

You can do several operations after creating a volume which includes:
- Attach a volume to one or more EC2 instance(s)
- Detach a volume from an instance
- Replace a volume
- View the volume details, and monitor the current status
- Delete a volume

Details of Selected Volume

This is the second part of the EBS Dashboard, Select the checkbox against the name of any volume.

Details - you can view the specific information, such as volume ID, snapshot ID, size, date of creation, instance to which it is attached, type of volume, and much more.
Status Check - Here you can view the health status of the selected volume. There are four possible status: Okay, Warning, Impairedor insufficient-data. See more details about the status here. You can view IO status, pre-defined IO performance, dates, and further textual description (selective).

Monitoring - Here you can view the I/O performance metrics for the selected volume, such as:
- read/write bandwidth (kB/sec),
- read/write throughput (Operations/sec),
- average queue (Operations)
- Idle time (%)
- avg read/write size (kB/Operation)
- avg read/write latency (msec/Operation)

Tags - Here, you can have a look at the associated tag. In the snapshot above, it shows the Name tag with U-test value.

Conclusion

Amazon Elastic Block Store (EBS) provides versatile storage for Amazon EC2 instances. With features like volume encryption, snapshots, and performance monitoring, EBS offers secure data-at-rest, reliable backups, and efficient resource allocation. It allows flexibility by associating volumes with Availability Zones (AZs) and restoring snapshots to different AZs within the same AWS Region for workload distribution, migration, and disaster recovery. EBS is essential for scalable and reliable architectures on AWS.
You can read more on EBS here

Site Reliability Engineering (SRE) and DevOps: A Comparative Study for Beginners

Ivy Jeptoo — Mon, 05 Jun 2023 09:55:09 +0000

I am pretty sure you have heard of DevOps and SRE in your technological journey if you are a beginner it can be very confusing. Both SRE and DevOps share a goal of bridging development and operations.

It is hard to say one is better than the other since they are both similar yet different in some ways. To simplify this, let's look at some key points.

SRE is viewed as a specific implementation of DevOps.
Thy both share the same foundational principles.
They both aim to deliver reliable software.
DevOps determines what needs to be done, whereas SRE determinesDevOps determines what needs to be done, whereas SRE determines how it will be done. DevOps captures a vision of a system that is developed efficiently and reliably. SRE builds processes and values that result in this system.
You can establish your goals using DevOps principles, and then implement SRE to achieve them. it will be done. DevOps captures a vision of a system that is developed efficiently and reliably. SRE builds processes and values that result in this system. You can establish your goals using DevOps principles, and then implement SRE to achieve them.

TABLE OF CONTENT

Introduction
Methods and Practices
Team Structure and Roles
Tools
How SRE connects to DevOps
Conclusion

Introduction

What is DevOps?

DevOps reflects two parts Developement and Operations which originated from the need for faster software delivery and more streamlined collaboration. This promotes shared responsibilities, collaboration and automation.
The main goal of DevOps is to reduce the time between making a change in code and that change reaching customers without having an impact on reliability.
DevOps has its main focus on on collaboration, integration and automation of system services to enable faster and more efficient software delivery. It helps stream line software development lifecycle,encompassing development, testing, deployment and operations.

What is SRE?

As we had mentioned earlier Site Reliability Engineering is an implementation of DevOps where it's goal is to align engineering goals with customer satisfaction. SRE originated from google where it was developed to maintain the reliability and scalability of large scale systems.
SRE introduced practices like error budgets and defined service level objects(SLOs) to align the goals of engineering and operations team.
SRE's focuses on the reliability, availability and performance of systems and services with emphasis on monitoring, engineering practices and response to high reliability.

Methods and Practices

DevOps methods and Practices

Practices in DevOps are based on continuous, incremental improvements achieved by automation.The methodology focuses on the following elements:

Continuous Integration and Continuous Delivery(CI/CD)

One goal that DevOps aims to achieve is to deliver updates and applications to customers rapidly and frequently, CI/CD pipelines connect processes and practices.
DevOps automates updating and code release to production. CI/CD means continuous monitoring and deployment to ensure that code is consistent in deployment environments and also in the software versions.

Infrastructure as code

In order for IT infrastructure to be managed using software engineering techniques and provisioned automatically, DevOps places a strong emphasis on its abstraction.This ensures the system can efficiently:
- Monitor infrastructure configurations.
- Track changes.
- Roll back changes with unintended effects.

Automated Testing

After being written or changed, code is automatically and continually tested. The continuous process speeds up deployment by removing the delays brought on by pre-release testing.

SRE methods and Practices

SRE routine includes analysis of logs, incidence response, testing production environments, patch management etc. Let's break it down:

Service Level Objectives (SLOs) and Service Level Indicators (SLIs)

Reliability is crucial for building customer trust and satisfaction and SRE allows the measure of how satisfied a customer is by using SLIs so we can say that SLIs are measurements used to quantify the performance and reliability of a service. It helps assess the user experience such as response time, error rates and availability.
A well established SLIs the team gains insights into the overall health of the system and use then define SLOs.
SLOs are targets set for key performance indicators(KPIs) which measure the reliability and performance of a service. They are set based on user expectations and also business requirements, by monitoring and measuring the actual performance against SLOs there is ease in identification of issues and drive continous improvement. In short SLOs sets a limit for how much unreliability the customer will tolerate for that SLI.

Error Budgeting

This is basically the acceptable level of unreliability or downtime of a system. The SRE team establishes a measure to determine when to prioritize stability or new feature development. We can say that error budget is the room you have before your SLO is breached
Error budget helps in decisons about prioritization, take an example services with lots of remaining error budget can accelerate development. When the error budget depletes, the team knows it's time to focus on reliability. This allows operations to influence development in a way that reflects customer needs.

Incident Management

By responding to incidents faster there is a reduction in customer impact. To achieve this there are components that need to be in place this includes:
- Runbooks: These are documents that guide responders through a particular task. They include things to check for, steps to take for each possibility which are always straightforward to reduce toil. Automating it is also a plus.
- On-call systems and Alerting: This determines the people available to respond to incidents as needed.
- Incident classification: sorts incidents into categories based on severity and area affected this allows you to triage incidents and alert the right people.
- Incident retrospectives: Learn a lot from each incident and review the documentations to determine follow-up tasks or revise runbooks.

Team Structure and Roles

Team Structure

SRE teams consist of software engineers with a focus on reliability engineering. They work closely with development and operations teams to balance reliability and feature development. SREs often have expertise in coding, systems, and operations.
DevOps encourages cross-functional teams that include developers, operations engineers, and sometimes QA engineers. This fosters collaboration and shared responsibilities, blurring the lines between traditional roles.

Roles

DevOps Engineer

Connecting micro services and tools to smooth the development cycle.
Sharing operation needs with development
Introducing new tools and processes.
Assessing risk to deployment targets.
Aligning teams on development goals

Site Reliability Engineer

Developing, configuring, and deploying software to be used by operations teams
Handling support escalation issues
Conducting and reporting on incident reviews
Developing system documentation
Change management
Determining and validating new features and updates

Tools

SRE Tools

In the SRE role, the most widely used tools are Prometheus and Grafana for collecting and visualizing the different metrics (CPU usage, memory, disk space, etc.), incident alert tools (OP5, PageDuty, xMatters, etc.), Ansible, Puppet, or Chef, Kubernetes and Docker for container orchestration, cloud platform AWS, GCP, Azure, JIRA, SVN, GitHub.

DevOps Tools

In the DevOps role, the most widely used tools are – Integrated Development Environment (IDEs) for development purposes, Jenkins for Continuous Integration and Development, JIRA for change management, Splunk for log monitoring, SVN, GitHub.

How SRE connects to DevOps

An organization can implement both DevOps and SRE and this can be achieved by considering SRE as a way of achieving DevOps goals.

SRE as an implementation of DevOps

Here are some of the practical approaches that SRE uses to solve DevOps goals:

Remove Silos

DevOps works to ensure that different departments/software teams are not isolated from each other, ensuring they all work towards a common goal.
SRE achieves this by creating documentation that the entire organization can use and learn from. Lessons from incidents are fed back into development practices through incident retrospectives.

Implementing Change gradually

DevOps embraces slow, gradual change to enable constant improvements. SRE supports this by allowing teams to perform small, frequent updates that reduce the impact of changes on application availability and stability.
SRE teams use CI/CD tools to perform change management and continuous testing to ensure the successful deployment of code alterations.

Accepting failure as normal

While DevOps aims to handle runtime errors and allow teams to learn from them, SRE enforces error management through Service Level Commitments (SLx) to ensure all failures are handled.
SRE strategically uses error budgets, accelerate development while maintaining reliability.

Leveraging tools & automation

Both DevOps and SRE use automation to improve workflows and service delivery. SRE enables teams to use the same tools and services through flexible application programming interfaces (APIs). While DevOps promotes the adoption of automation tools, SRE ensures every team member can access the updated automation tools and technologies.
Whenever you automate or simplify a process, you reduce toil and increase consistency. You also accelerate the process, achieving DevOps goals.

Metric-based decisions

SRE practices encourage monitoring everything and then constructing deep metrics. These will give you the insights you need to make smart decisions.
DevOps gathers metrics through a feedback loop. On the other hand, SRE enforces measurement by providing SLIs, SLOs, and SLAs to perform measurements. Since Ops are software-defined, SRE monitors toil and reliability, ensuring consistent service delivery.

Conclusion

SRE and DevOps are two sides of the same coin, with SRE tooling and techniques complementing DevOps philosophies and practices. SRE involves the application of software engineering principles to automate and enhance ITOps functions while DevOps model enables the rapid delivery of software products through collaboration between development and operations teams.
The goal of both the methodologies is to enhance the end-to end cycle of an IT ecosystem—the application lifecycle through DevOps and operations lifecycle management through SRE.

Git as a DevOps Tool

Ivy Jeptoo — Fri, 26 May 2023 22:10:48 +0000

I know when we say Git what comes to the mind of most beginners is Github. Well just to make it clear, Github or bitbucket are built on top of git with some additional functionalities that help with hosting and version control of code in the remote Git repository.

Git provides developers with shared workspace hence visualization of works is made easy. It comes with good integrations to work with CI/CD tools.

Why Git is needed in DevOps

Effective CI/CD Discussions with Developers

As a DevOps engineer you design and develop CI/CD pipelines and Git plays a vital role in having productive discussions about Continuous Integration and Continuous Deployment (CI/CD) with developers.

Git is essential in this context:

It enables seamless collaboration among developers, facilitating discussions on merging changes, resolving conflicts, and integrating code during the CI/CD pipeline.
Git's branching feature allows developers to work on separate branches for different features or bug fixes*(git branching). This supports discussions on branch readiness, **code review,* and ensuring quality code integration.
Git stores deployment configurations, such as infrastructure as code files and deployment scripts. Discussions can involve reviewing and modifying these configurations to ensure smooth deployments in the CI/CD pipeline.
Git helps manage release versions and tags, enabling discussions on release planning, versioning strategies, and feature inclusion. This ensures a coordinated approach to releases within the CI/CD workflow.

Infrastructure as Code

Development and maintenance of infrastructure code is done in Git and the it is important to note that infra code is treated the same as application code hence it goes through unit testing and integration tests before deployment to environments. This means that infra code needs CI/CD pipeline which translates to git-based workflows.
With Git, you can manage changes to your infrastructure code in a controlled and auditable manner. Each change is tracked, allowing you to understand who made the change, when it was made, and why. This helps maintain a clear history of modifications and simplifies change management processes.
Git facilitates collaboration among team members working on infrastructure code. Multiple developers can work on the same codebase simultaneously, enabling seamless collaboration, code review, and the ability to merge changes efficiently.

Gitops

GitOps uses Git as the one point of truth to control the deployment of infrastructure and applications. Git plays a crucial part in GitOps by offering features for version control and collaboration

GitOps uses a pull-based methodology, in which the system's ideal state is specified in Git and changes are constantly compared to the real state. As the only source of truth, Git repositories, the GitOps tools extracts the desired state from Git and applies it to the target.
GitOps uses the branching capabilities of Git to handle several environments. In the Git repository, each environment (such as development, staging, and production) is allowed to have a separate branch. As a result, distinct workflows, change isolation, and controlled promotion of settings and applications between environments are made possible.
Git's pull request and code review functions are advantageous to GitOps. A review procedure can be applied to changes to deployment manifests and infrastructure code to guarantee best practices compliance and code quality. Reviewers can comment on modifications, debate them, and make sure that only those that have been authorized are merged into the main branch and released.

GIT ROADMAP

Learning Resources

Smooth Sailing with Docker: A Beginner's Guide to Containerization

Ivy Jeptoo — Sun, 14 May 2023 09:31:52 +0000

Welcome to Docker world where containers bring magic to the world of software development and deployment!

Why did the Docker container never ask for help?
Because it couldn't find a container support group – it was too self-contained!

Now that we've shared a giggle or a laugh let's embark on Docker exploration. In this article you'll learn docker fundamentals and it's installation...let's dive in!

TABLES OF CONTENTS

Introduction
Docker Features
Docker Architecture
Installing Docker Desktop
Conclusion

Introduction

Docker is a container management server that is used in development, packaging and deployment of applications automatically.

Container - is an instance of an image that allows developers package the application with all parts needed such as libraries and other dependencies.
Image - is a file that has multiple layers used to execute code in a docker container. They are a set of instructions used to create docker containers

How Docker works

Docker uses containerization where applications are packaged into containers that has everything they need to run(code, libraries, dependencies). Docker uses OS-level virtualization to create the containers and ensuring that each container operates as an isolated and self-contained unit.

Docker Engine - is a client-server based application that hosts containers.
It has three main components:

Server(Docker deamon): Creates, and manages docker images, containers, network and volumes on docker.
REST API(Docker Client): Allows users to interact with server, issuing commands to build, run, and manage containers.
Client: is a docker command-line interface(CLI), that allows interaction with docker using docker commands.

Docker Features

Scalability
Docker containers are lightweight hence easily scalable. Its portability makes it simple to manage workloads, scaling up or down apps and services as demands in real-time.

Swarm
It is a clustering and scheduling tool for docker containers. Swarm uses docker API as its front-end hence various tool to control it. It also helps us control cluster of docker hosts as a single virtual host. It's a self-organizing group of engines used to enable pluggable backends.

Security
Docker saves secret into the swarm itself. Docker container provide a high level of isolation between different application preventing them fro interacting with or affecting each other hence a more secure and stable platform for running multiple apps on a single host.

Routing Mesh
It enables connection even if there is no task running on the node. It routes the incoming requests for published ports on available nodes to an active container.

Docker Architecture

In a nutshell, the client talks with the docker deamon which helps in building, running and distributing docker containers. The client runs with the deamon on the same system or connected remotely. With the help of REST API over a network the client and deamon interacts.

DOCKER CLIENT
Docker client uses commands and REST API to communicate with the server(Docker Deamon).
When a client runs any docker client terminal, the client terminal sends the docker commands to the docker deamon which receives the commands in form of command and REST API's request
Docker Client can communicate with more than one docker deamon and it uses CLI to run the docker build, docker pull, docker run.

DOCKER HOST
Provides an environment to execute and run applications. It contains docker deamon, containers, images, network and storage.

DOCKER REGISTRY
Manages and stores docker images.
It is of two types:

Public Registry - Also called Docker Hub is used by everyone.
Private Registry - uses to share images within the enterprise.

DOCKER OBJECTS

Docker Image - are the read-only binary templates used to create Docker Containers. Images enables collaboration between developers.
Docker Containers - is used to hold the entire package that is needed to run the application. containers need less resources which is a plus. Containers is a copy of a template while the image is a template.
Docker Networking - provides isolation for docker containers, it links docker container to many networks.
Types of Docker Network
Bridge - default network driver for the container, used when when multiple docker communicates with the same docker host.
Host - used when we don't need network isolation between container and host.
None - disables all the networking.
Overlay - enables containers to run on different docker host. Offers Swarm services to communicate with each other.
Macvlan - Used when we want to assign MAC (Media Access Control)addresses to the container.

Docker Storage
It is used to store data in containers. Docker has the following storage options;-
Data Volume - provides the ability to create persistence storage, it allows us to name volumes, list volumes and containers associated with the volume.
Directory Mounts - it mounts host's directory with a container, it is the best option.
Storage Plugins - it provides ability to connect to external storage platforms.

Installing Docker Desktop

We can install docker on any operating system but docker runs natively on Linux distributions. We will install docker engine for linux Ubuntu.

Prerequisites
To Install docker Desktop ensure that:

Have a 64-bit version of either Ubuntu Jammy Jellyfish 22.04 (LTS) or Ubuntu Impish Indri 21.10. Docker Desktop is supported on x86_64 (or amd64) architecture.
Meet the system requirements

Steps

Set up Docker's package repository
Download latest DEB package
Install the package with apt:

`sudo apt-get update`
`sudo apt-get install ./docker-desktop-<version>-<arch>.deb`

Launch Docker Desktop using the terminal:

`systemctl --user start docker-desktop`

or search Docker Desktop on Applications menu and open it.
Check docker binary versions by running:

`docker compose version` 
`docker --version`
`docker version`

To login to docker

`systemctl --user enable docker-desktop`

To stop Docker Desktop

systemctl --user stop docker-desktop

If you are using Windows you can install it from here or if you are using Mac you can find docker installation here

Conclusion

Remember, Docker is a powerful tool with numerous advanced features and use cases. Continue learning by referring to the official Docker documentation and community resources. Embrace the world of containerization and elevate your software development and deployment workflows with Docker. Happy containerizing!

My First Month in the SCA Cloud School: A journey of Learning and growth.

Ivy Jeptoo — Mon, 01 May 2023 21:23:48 +0000

I've always been fascinated by the field of SRE and the crucial part it plays in maintaining the availability and smooth operation of digital services. Despite the fact that I had no prior expertise in SRE, I had worked in software engineering for a while and had witnessed firsthand the value of having a solid infrastructure.

I came across the She Codes Africa SRE boot-camp sponsored by Deimos which was an ideal chance to expand my knowledge of SRE tenets and best practices when I learned about it. The program's focus on practical knowledge and abilities, as well as its reputation for producing excellent SREs, attracted my attention in particular.

The program began on 31st of March and it runs for two months. There are evaluations and projects that are to be covered during this period.I will break down what I have learnt during the first four weeks.

WEEK ONE

We were introduced to cloud computing and AWS fundamentals, including the different cloud deployment models and AWS services such as EC2,S3 and IAM.
We learned how to set up an AWS account, create an EC2 instance and configure it with appropriate security settings.
We further explored on security best practices for AWS resources such as access control policy and monitoring.
I had an opportunity to practice my skills by creating an EC2 instance and writing and article about it, you can check it out here

By the end of week one I had a solid understanding of cloud computing and AWS fundamentaals and I had gained practical experience with creating and securing an EC2 instance.

WEEK TWO

We learned about Azure and its concepts including the architectural components of Azure and how to create an Azure account.
We explored further into Azure core services which included Azure compute services, storage services,Azure analytics, Azure databases.
We delved into core solutions and management on Azure, where we learnt about IoT services, AI services, serverless technology and management and configuration on Azure environment.
We finally learned about monitoring services for Azure and how to use them to keep your Azure resources secure and performant.

By the end of the second week I had a full understanding of Azure's services and how to manage them effectively. I then wrote an article on how to create an Azure App Service to host the web application

WEEK THREE

Learned about the general security and network security features on Azure, including how to protect against threats and secure network connectivity.
We explored Identity, governance, privacy, and compliance features on Azure, which are essential for maintaining the security and compliance of your Azure resources.
We also learned about Azure cost management and service level agreements (SLAs), which are critical for managing your Azure resources effectively and ensuring that you are getting the best value for your money.
Finally, we deployed a virtual machine on the Azure portal, which gave me hands-on experience with creating and managing Azure resources. I also wrote an article on How to Deploy a virtual machine on Azure portal

At the end of week three I gained clarity on how to secure Azure resources and manage their cost effectively. I practiced creating and managing a virtual machine on Azure portal.

WEEK FOUR

We were introduces to SQL and learned about its fundamental concepts and how it is used in the context of Azure.
Learned about deployment and configuration of servers, instances, and databases for Azure SQL, which involved creating and configuring SQL instances and databases on the Azure platform.
We practiced connecting user-created instances to SQL Server Management Studio (SSMS), which is a crucial tool for managing and querying SQL databases.
We learned how to backup and restore databases using SSMS, which is essential for ensuring data integrity and recoverability.
We explored on how to secure data using Azure SQL, including data encryption and access control measures.
Finally, you learned how to deliver consistent performance with Azure SQL, which is critical for ensuring that your SQL databases are fast and reliable.

By the end of week four I had solid understanding on Azure SQL and how it is used in the context of Azure.

CONCLUSION
The first four weeks of the She Codes Africa SRE boot-camp have been a fantastic experience for me. I have learned a lot about cloud computing, Azure, general security, network security, SQL, and database management, and I have gained practical experience with creating and managing cloud resources on Azure. I am now looking forward to the next phase of the boot-camp, where I will deepen my knowledge of cloud technologies and gain more practical experience with using them.

One of the things that have impressed me the most about this boot-camp is the amazing community of learners and facilitators that I have had the privilege of interacting with. The community is incredibly supportive and always ready to help each other out, which has made the learning process both enjoyable and productive. The facilitators are also knowledgeable and experienced, and they have provided excellent guidance and feedback throughout the boot-camp.

As a woman in tech, I would like to recommend She Code Africa to other women who are interested in cloud computing and want to gain practical experience with using cloud technologies. SCA is a fantastic organization that is committed to increasing the number of women in tech and providing them with the support and resources they need to succeed. I am grateful for the opportunity to be part of this boot-camp, and I am confident that the skills and knowledge that I am gaining will be invaluable in my future career as an SRE.

How to create a Linux VM and Install MySQL Server using Cloudshell

Ivy Jeptoo — Thu, 27 Apr 2023 16:27:51 +0000

Data management is critical to every organization, and MySQL database has proven to be a reliable tool for storing and retrieving information. Pairing MySQL with Azure VMs provides a robust, flexible environment for running your workloads. In this article, we'll explore the process of setting up VMs and using MySQL on Azure VMs.

The features provided by Azure Virtual Machine include:-

Create, start, stop, restart or terminate Virtual Machine instances.
Implementation of Load Balancing and Auto Scaling for multiple Virtual Machine.
Provides additional storage to Virtual Machine instances
Manages Network Connectivity to Virtual Machine.

Why MySQL? - It is a relational database that stores and manages data. It is known for its speed, reliability and ease of use.
MySQL is a good fit for running on Azure Virtual Machine because:

Flexibility: it allows you to customize the VM to your specific requirements(choosing CPU, memory config, storage size).
Compatibility: it is compatible with a range of programming languages hence ease of integration in the environment.
Scalability: it can handle large data volumes thus easy to take advantage of automatic scaling, load balancing in Azure.
Security: it have great security features like SSL encryption, user authentication and access control running it on Azure adds Azure Security features advantages.

TABLE OF CONTENT

Create Virtual Machine
Connect to Virtual Machine
Install MySQL
Connect to MySQL

Create Virtual Machine

Search for Virtual Machine on the search bar and click on it, you should also see it under Azure Services.
Once you click on it, You will see a create button that has a drop down, select on Azure Virtual Machine.
On the Basics, Provide a resource group in which you VM will belong to, you can create one as well.
Give your VM a unique name.
Select a region you'd want to deploy you VM
We'll go with the standard security
Choose an Operating System you'd want under image
The size is how big we want our VM to be, be sure to select your desired size.
Leave the default authentication type (SSH Public Key)
You can use the default username or provide a new one.
Under SSH public key source we are going to generate a new key pair.
Be sure to provide a Key Pair Name
Once all is set click on Review and Create and when the validation is passed click on Create.

There will be a Generate new key pair pop up where you will download the key pair that we will use later on in connecting to the VM.

Once the deployment is complete click on Go to Resource so as to see the newly created VM.

Connect to Virtual Machine

We are going to connect to the VM using Azure Cloud Shell which is used for managing Azure resources from anywhere with an inernet connection
On the top of VM page, click on the connect button.
At the top right click on the first icon to open the cloudshell.
Once you click connect, a new page will be opened that has the SSH guide to connect to the VM.
Upon clicking the cloud shell icon, at the bottom of the page you'll see a section that requires a storage mounting. Click on the create button which will open the shell terminal successfully.
We need to upload our private key(we downloaded earlier) to the cloud shell.
Click the upload icon then upload the file.
After it has uploaded you will see the notification at the bottom right of the page.
To confirm the file has been uploaded run ls and you will be sure to see your file name.
To ensure we have read-only access to private key run chmod 400 <keyname>.pem.
Replace .pem with your actual file name.
Confirm you have read-only access running ls -ltrh
Provide a path to the private key. Run pwd to get path of the SSH private key then run ssh -i <private key path>/<filename>.pem azureuser@52.191.61.164
Replace private key path with the path and also the private key file name.
Confirm that you want to continue with the connection by clicking yes and you will soon be connected!
You can confirm you are connected to the VM by running hostname.

Install MySQL

sudo apt-get update
sudo apt-get install mysql-server
mysql -V to confirm installation

Connect to MySQL

We are going to connect to the root user account since we have not created any user account

sudo mysql too start MySQL client with admin privileges.

Create a new User

CREATE USER 'newuser'@'localhost' IDENTIFIED BY 'password';
Replace newuser with desired username for the new user and password with a unique password.
@ localhost specifies that the user account can only be used to connect to MySQL server from the local machine.(it is a recommended approach in azure).
Grant the new user necessary privileges to the databases and tables on the server GRANT ALL PRIVILEGES ON * . * TO 'newuser'@'localhost';
You can now connect to MySQL server using the new user account created by running: mysql -u newuser -p -h hostname .
Replace the username with the username of the MySQL user account created earlier, hostname with the hostname/IP address of the machine running MySQL server
It will prompt you to enter the password for the new user account.
Once connected you can create a new database CREATE DATABASE dbname;
Replace dbname with your new database name.

N/B
Remember to stop your Virtual Machine from running to avoid incurring extra costs!

Congratulations!! You have just create your own Virtual Machine and connect MySQL server to it!

How to Build HTTP API using AWS Serverless Application Model (SAM), AWS Lambda and Node.js

Ivy Jeptoo — Mon, 24 Apr 2023 08:30:56 +0000

AWS Serverless Application Model(SAM) provides short hand syntax to specify serverless resources such as lambda functions, API Gateway and dynamodb. The syntax can be used to model the application you want create in AWS using yaml.
AWS Lambda is a serverless, event-driven service that runs your code on demand.

In this article we are going to develop a HTTP API locally with AWS SAM, deploy it to AWS and test the deployed API.

Create Lambda Function
API Gateway
HTTP API AWS SAM on VS Code
Deploying to AWS
Testing
CloudWatch Logs
Conclusion

Create Lambda Function.

Search for Lambda on the search bar, on the left side bar click on Functions then on Create Function.
Provide the needed details i.e Function name, runtime and architecture, then create function.

Use image for referral

API Gateway

This is a fully managed service that makes it easy for developers to create, deploy, and manage APIs at any scale.
On the page click on API so as to pick on the API you want to create.(We are building HTTP API)

Use image for reference

Once you click on build:-

Step 1 - Add an Integration which will be Lambda, select the AWS and select the lambda function you created earlier. Provide an API name.

Step 2 - We shall Configure routes select a GET method and provide a resource path and the Intergration target is the lambda function we created.

Step 3 - Stages is the deployment environment where our API is going to be hosted. leave it to default and ensure that auto-deploy is activated, it comes in handy.
Step 4 - Review everything you have done and click on create if you are satisfied with it.

HTTP API AWS SAM on VS Code

Prerequisites

Ensure you have an AWS account.
Install AWS CLI using this link.
Configure your AWS using the aws configure where you'll be needed to provide your AWS Key details.
In case you want to generate new ones use this link to generate new ones.
Install the AWS SAM CLI using this documentation.
Install Docker here (Local Testing)
Install Nodejs here
Install AWS Toolkit VS Code Extension.
Install Thunder Client CLI extension on VS Code.

Create SAM project

Create a folder and open it in VS Code.
Generate a default sam application using sam init. Pick on AWS Quick Start application template. Hello world Example, nodejs14.x runtime and package type, zip package, Hello World Example starter template, N to x-ray tracing to avoid extra charges finally give your sam project a name.
A folder with the sam project name will be generated, let's discuss it in details:-
events/: contains sample events that can be used for local testing of Lambda functions.
hello-world/: contains an example AWS SAM app that is used to test that everything is working correctly.
.gitignore: used by git to determine which files and directories to ignore when committing changes to a repository.
README.md: a markdown file that contains documentation for the project.
samconfig.toml: a configuration file used by the SAM CLI to specify options and settings for deploying the application.
template.yaml: defines the resources and configuration for the application.

Template.yaml Code Explanation

Check out the full code in template.yaml here. Under the Resources section, HttpApi defines the resources that are going to be created on AWS. We've kept our default HelloWorldFunction that has a type of serverless function There are three events:

GetUsers: Gets all the users from the API. Has type HttpApi, path of /user, method used is get, ApiId with a ref of HttpApi

Note
ApiId Ref(HttpApi) is defined above the HelloWorldFunction and has a type of serverless HttpApi with a stage name of nonprod(default)

GetUser:
Gets a single user by id from the the API.
Has type HttpApi, path of /user/{id}, method used is get, ApiId with a ref of HttpApi
PostUser
Creates a new user.
Has type HttpApi, path of /user, method used is post, ApiId with a ref of HttpApi

app.js Code Explanation

Check out the full code in app.js here.
This file defines the lambda handler i.e the code that will run when our lambda is invoked. Response is the object we get back from the lambda(API Response) const USERS is an array of objects that contain the users it has a property of name and userid. lambdaHandler has event and context passed in. In the event is where the http request will be injected console.log the event to get the http in cloudwatch logs. result will be passed in the response object.
First condition uses the event object passed in earlier and the routeKey(has the method used) Query parameters can be passed in for start and end(default for start is 0 and total array length for the end.) Validation
first validation is to ensure the start is greater or equal to 0.
second validation is to ensure the end is less or equal to the array length.
third validation ensures the start is less than end.
Second Condition we have the POST method and /user path where we will post new users to the user array
Third Condition returns a single user using the method get and path of /user/{id} with a personalized greeting.

Deploying to AWS

Ensure you are in the directory that contains template.yaml.
Run sam build command to build the Lambda function and generate a deployment package.
aws-sam folder will be generated successfully.

N/B
Verify that your IAM user has the necessary permissions to perform the deployment. The required permissions are:

AWSLambdaFullAccess
IAMFullAccess
AmazonAPIGatewayAdministrator
AmazonS3FullAccess
CloudFormationFullAccess
CloudWatchLogsFullAccess
Run sam deploy --guided for step by step creation of CloudFormation stack using the SAM template and Lambda function code.
Provide a name for the app, use the default region, confirm the changes,allow IAM role creation, don't disable rollback, we do not have authorization so agree to it, we want our arguments saved in the configuration file and use default for the rest.
Confirm that we want to deploy the changes(it should be successful).
On your AWS account under API Gateway you will see the just created HttpApi once you refresh it.

HttpAPi overview

Under the routes section we can also see the paths we created as so:

All the routes have a configuration integration(same Lambda) as so:
Copy the stages copy the Invoke URL which we will use it for testing.

Testing

We will use Thunder Client CLI extension for testing our API

GetUsers
We are doing a get request and the response will be all the users in our API.
To test the query string you can add the specifications at the end /user?start=1&end=3.

GetUser
This is also a get request with id specification the response is a personalized greeting with the user's name.

postUser
Using the post methos it adds a new user to the user array.

CloudWatch Logs

Earlier we printed the events to the logs in CloudWatch.
Once in cloudwatch go to log groups then find the correct log group then click on it.
You will see an overview of it then at the bottom there is log streams which has all the calls made.
The log has START, END and a REPORT like so:

Conclusion

AWS Serverless Application Model (SAM), AWS Lambda and Node.js make building APIs a breeze. With the steps outlined in this article, you can get started with your own simple API endpoint in no time. If you found this article helpful, be sure to connect with me and leave your comments and feedback. Happy coding!

How to attach a data disk to a Linux VM

Ivy Jeptoo — Fri, 14 Apr 2023 11:42:19 +0000

Hello there,
We learnt how to create a virtual machine(you can recap here) and today we are going to cover how to create a data disk and attaching it to the virtual machine.

Introduction

Attach Disk

Find Disk

Prepare new disk

Mount Disk

Verify Disk.

Introduction.

Data disk are separate storage units attached to VMs to enhance the storage capacity and performance without affecting operating system or application.
We will explore the step-by-step process of using the Azure portal to create and attach a data disk to a Linux VM, and then mount it so that it can be used for storing data or running applications.
Select VM from the Azure portal and select one that you'll use or create a new one if need be.

Use image for reference

Attach Disk.

In the Virtual Machine page, Under Settings choose Disk options

Attaching a new disk.

Select Create and attach a new disk under the Data disks pane
Give your managed disk a name and configure the default setting.
Select save at the top of page to save your new disk and update the VM configuration.

Use image for reference

Attaching an existing disk.

Select Attach existing disks under the Data disk.
From the drop down choose the desired disk you would want to work with.
Select save at the top of page to save your new disk and update the VM configuration.

Find Disk

Make sure you are connected to your Virtual Machine

Follow the steps from the image below to connect to your Virtual Machine.
You are now connected to the VM and we need to find the just created disk. Run the following command on the terminal

lsblk -o NAME,HCTL,SIZE,MOUNTPOINT | grep -i "sd"

N/B
lsblk - lists information about all available block devices, including disks and partitions.

-o NAME,HCTL,SIZE,MOUNTPOINT - specifies the columns of information to display in the output.We are asking lsblk to display the device name, host controller target and logical unit number (HCTL), size, and mount point (if any) for each block device.

| - a pipe symbol that redirects the output of the lsblk command to the next command in the pipeline.

grep -i "sd" - searches for lines in the output of lsblkthat contain the characters "sd" (case-insensitive).

"sd" is typically used to indicate SCSI disks in Linux.

The expected output should be:

sda     0:0:0:0      30G 
├─sda1             29.9G /
├─sda14               4M 
└─sda15             106M /boot/efi
sdb     0:0:0:1       4G 
└─sdb1                4G /mnt
sdc     1:0:0:0       4G

Prepare new disk

If you are using an existing disk that contains data, skip to mounting the disk. The following instructions will delete data on the disk.
If you are using a new disk you need to partition it because:

You can **organize data **into logical units, making it easier to manage and locate specific files.
It allows you to isolate certain data sets from others so if one partition is compromised, the other partitions remain safe.
Improve performance by allowing you to separate frequently accessed files from those that are rarely accessed. Hence optimize the disk usage and reduce the time it takes to access data.

-To partition run this command, make sure to replace sdc with the correct option for your disk

sudo parted /dev/sdc --script mklabel gpt mkpart xfspart xfs 0% 100%
sudo mkfs.xfs /dev/sdc
sudo partprobe /dev/sdc

Mount disk

Once the file system is created, the disk needs to be mounted to a specific directory in the file system hierarchy to be accessible to users and applications.
Create a directory to mount the file system using mkdir. The following example creates a directory at /datadrive:
sudo mkdir /datadrive
Mount the /dev/sdc partition to the /datadrive mount point:
sudo mount /dev/sdc /datadrive
We need to find the UUID of the newly attached drive:

sudo blkid

The expected output:

Verify disk

use lsblk command again to see the disk and the mountpoint.

lsblk -o NAME,HCTL,SIZE,MOUNTPOINT | grep -i "sd"

The expected output:

-Well done!! You can see that sdc is now mounted at /datadrive.

These steps are crucial for ensuring a smooth and efficient computing experience, and taking the time to properly prepare and mount a new disk can save time and effort in the long run.

DEV Community: Ivy Jeptoo

How to Create AWS EKS Cluster

I Web Console

Prerequisites

Create EKS Cluster

Create a Node Group

Clean Up

II AWS CLI Tool

eksctl Installation

Create a basic cluster

Create an advanced cluster

List the details

Delete Cluster

How to deploy AWS Serverless Application in Cloud9

Prerequisites:

Initialize Hello World Application

Build the Application

Deploy the application to an ECR image repository

Testing

EC2 Instance with an Admin Role

Table of Content

Objectives

Create a default Virtual Private Cloud

Launch an EC2 instance

Create an IAM Role

Attach the Role to the EC2 Instance

Connect to your EC2 instance

Conclusion

AWS Cloud Fundamentals

Advantages of Cloud Computing

Table of Content

Overview of AWS Cloud Fundamentals

Compute

Storage

Database

Security

Networking

Messaging

Management Services

Conclusion

Elastic Block Store(EBS)

Features of EBS

Creating EBS Volume

Conclusion

Site Reliability Engineering (SRE) and DevOps: A Comparative Study for Beginners

TABLE OF CONTENT

Introduction

Methods and Practices

DevOps methods and Practices

SRE methods and Practices

Team Structure and Roles

Team Structure

Roles

Tools

How SRE connects to DevOps

SRE as an implementation of DevOps

Conclusion

Git as a DevOps Tool

Why Git is needed in DevOps

GIT ROADMAP

Learning Resources

Smooth Sailing with Docker: A Beginner's Guide to Containerization

TABLES OF CONTENTS

Introduction

How Docker works

Docker Features

Docker Architecture

Installing Docker Desktop

Conclusion

My First Month in the SCA Cloud School: A journey of Learning and growth.

How to create a Linux VM and Install MySQL Server using Cloudshell

TABLE OF CONTENT

Create Virtual Machine

Connect to Virtual Machine

Install MySQL

Connect to MySQL

Create a new User

How to Build HTTP API using AWS Serverless Application Model (SAM), AWS Lambda and Node.js

Table of Contents

Create Lambda Function.