DEV Community: Ashutosh Mallick

Bastion Host Setup In Azure Console

Ashutosh Mallick — Tue, 09 Aug 2022 06:59:19 +0000

Bastion Host

A bastion host is a server whose purpose is to provide access to a private network from an external network, such as the Internet. Because of its exposure to potential attack, a bastion host must minimize the chances of penetration.

What is the difference between a firewall and a bastion host?

A bastion host is a dedicated server that lets authorized users access a private network from an external network such as the internet. Placed outside the firewall or within a DMZ, the bastion host becomes the only ingress path to those internal resources.

If we want to SSH into a VM on the private subnet from our home/office (or using a development machine), currently we can’t. Our instance has no public IP, it is in a Private Subnet (no direct route from the internet). This is where we can use a Bastion Server or Jump server.

Azure Bastion is deployed to a virtual network and supports virtual network peering.
Specifically, Azure Bastion manages RDP/SSH connectivity to VMs created in the local or peered virtual networks.

Exposing RDP/SSH ports over the Internet isn't desired and is seen as a significant threat surface. This is often due to protocol vulnerabilities. To contain this threat surface, you can deploy bastion hosts (also known as jump-servers) at the public side of your perimeter network.

Bastion host servers are designed and configured to withstand attacks. Bastion servers also provide RDP and SSH connectivity to the workloads sitting behind the bastion.

Network Security Group (NSG)

A network security group, or NSG, allows or denies inbound network traffic to your Azure resources. Think of a NSG as a cloud-level firewall for your network.

For example, notice that the a VM allows inbound traffic on ports 22 (SSH) and 80 (HTTP). This VM’s network security group allows inbound traffic over these ports from all sources as default. But we can configure a network security group to accept traffic only from known sources, such as IP addresses that you trust or your local PC IP address.

NSGs can be associated with subnets or individual virtual machine instances within that subnet. When an NSG is associated with a subnet, the ACL rules apply to all Virtual Machine instances of that subnet.

NSG Architecture:

Bastion Host Setup Architecture:

Deployment of Resources:

Create a Resource group (NSG-RG):

Create a Virtual network (NSG-vnet) [CIDR- 10.0.0.0/16]

Add subnet-1 (NSG-subnet-1) with CIDR value “10.0.1.0/24"

Add subnet-2 (NSG-subnet-2) with CIDR value “10.0.2.0/24”

Create VM-1 (Bastion host in Subnet-1). Allow public IP to it.

In this case NSG is applied to server by default. We can also attach NSG at subnet level. There are two rules i.e, Inbound rules and outbound rules in NSG.

Create another VM (Web-server) inside Subnet-1. We can assign public Ip to the server.

Create DB-server inside Subnet-2. Assign private ip to it.

Inside inbound port there is a RDP port of source any and destination any , this means anyone can RDP into my server.

Now let’s create an inbound rule so that it can only allow my local desktop IP , to do that we have to put our ipv4 ip inside source IP addresses. Delete the previous inbound port security rule.

Add a new inbound port rule with source as your local PC ip address. Select service as RDP and protocol as TCP.

After editing this inbound port rule, NSG will only allow my local server IP to access the Bastion-host server via RDP. We can’t access the bastion-host from any other PC.

If we want to allow multiple PC to be able to access the bastion host, we can edit the inbound port rule and in place of source we can put the respective IP address of the PCs.

Suppose we don’t want a particular IP to access my Virtual machine, then create an inbound port and put that IP inside the source IP addresses and inside the destination IP addresses put this VM IP and select deny option in action.

We cannot directly RDP into our DB-server because it has only private IP, so in order to RDP we need a server that has a public ip and should present in the same network.

So, we use the Bastion-host or Jump-server to access the DB-server as they exist in the same network. Let’s Connect DB-server from inside Bastion-host.

Connection established to DB-server via RDP from Bastion-host.

Similarly, connection to DB-server can be established by WEB-server as they exist in the same network.

Connection established with DB-server from Web-server via RDP.

But our Connection to DB-server should only be restricted to Jump-server only. To solve this, we need to change inbound rule configurations of NSG of DB-server.

First delete the existing rule for RDP.
Add inbound rule as following :
Source: IP address of web-server
Destination: IP address of DB-server
Service: RDP
Protocol: Any
Priority: 210(say)
Name: 3389_port_block
Action: Deny

Now let’s try connecting DB-server again from inside WEB-server via RDP. We can see that the connection can’t be established.

If we try connecting internet from DB-server from via Jump server we can see that DB-server has internet access.

To restrict internet access to DB-server we need to edit the Outbound rules of DB-server.
Add Outbound rule as following:
Source: Any
Destination: Service Tag
Destination Service Tag: Internet
Destination port range: 8080
Action: Deny
Priority: 200(say)
Add rule.

Now our deployment and setup for connecting a database server via bastion host is successfully established.

Drop your views regarding this.
Thank you!!

VPC Networking

Ashutosh Mallick — Thu, 21 Jul 2022 05:45:08 +0000

What is VPC?

The AWS VPC is essentially a private virtual network inside the AWS network. While AWS is
physically a shared network, each VPC is logically isolated from other AWS customers. The AWS
network supports private and public addresses for each of its customers
AWS customers are logically separated, there is no contention for IP address space between VPCs. Each VPC will
have its own routing table that is responsible for directing traffic.
Below image shows the logical isolation of AWS customers.

** What is Subnetting?**
To understand what Subnetting is, first we have to understand what
exactly is the Network and Subnet

What is Network?
A network is a group of two or more connected computing devices. Usually all devices in the network are connected to a central hub — for instance, a router. A network can also include subnetworks, or smaller subdivisions of the network

What is Subnet ?
A subnet, or subnetwork, is a network inside a network. Subnets make networks more efficient. Through subnetting, network traffic can travel a shorter distance without passing through unnecessary routers to reach its destination.

What is Subnetting ?

Subnetting/Subnetworking is how very large networks, such as those
provided by ISPs, are able to manage thousands of IP addresses and
connected devices.
In simple words: Dividing bigger networks into smaller networks called Subnetting.

CIDR
CIDR stands for (Classless Inter-Domain Routing) -- also known as supernetting. It's a method of assigning Internet Protocol (IP) addresses that improves the efficiency of address distribution and replaces the previous system based on Class A, Class B and Class C networks.

For each VPC we setup, we have to assign a CIDR value say(10.0.0.0/16).

CIDR range depends upon the no. of servers going to be deployed.
Here, the variable values of CIDR vary from 0 to 255.
But first four values i.e, from "10.0.0.0" to "10.0.0.3/16" are reserved and "10.0.0.255/16" is also reserved.

For each subnet we create, we restrict public and private resources to different subnets. Each subnet is assigned to different IP.

Here '/24' means first 3 bits of IP will be kept constant.
For Subnet-1 Ip address ranges from 10.0.0.4 to 10.0.0.254.
For Subnet-2 Ip address ranges from 10.0.1.4 to 10.0.1.254.
For Subnet-3 Ip address ranges from 10.0.2.4 to 10.0.2.254.

Basically, we deploy our different services into different subnets. Such as we keep web-servers in a single subnet, Database servers in a different private subnet.

VPC DEPLOYMENT:
Create a VPC "Silicon-Vpc" (say).
Give an IPV4 CIDR to it (10.0.0.0/16).

Create two Subnets inside it, Web-subnet and DB-subnet.
Assign CIDR values to the subnet as "10.0.1.0/24" and "10.0.2.0/24" respectively.

Now select "Web-subnet" and edit subnet settings. Enable the auto-assign public IP in order to allow public access to all the web-servers inside the subnet.

We can see that for each subnet we can allocate 251 servers (According to the CIDR values available).

Now let's launch instance say linux servers. Set VPC as "Silicon-vpc" and subnet as "Web-subnet".
Create a new key pair and launch.
Similarly launch another instance into DB-subnet.

Connect web-server using SSH. We can see that we won't be able to connect.
It is because we have not attached the internet gateway to give the web-server internet access.

What is Internet Gateway?
An internet gateway is a horizontally scaled, redundant, and highly available VPC component that allows communication between your VPC and the internet.

An internet gateway serves two purposes:
1) to provide a target in your VPC route tables for internet-routable traffic.
2) to perform network address translation(NAT) for instances that have been assigned public IPv4 addresses.

Now we'll deploy an IGW.
Create an IGW " Silicon-IGW". Attach a vpc to it.

IGW Creation:

Now we still won't be able to connect as Route table is undefined.
So we have to create and define the route table.

What is VPC Route Table?
Your VPC has an implicit router, and you use route tables to control where network traffic is directed. Each subnet in your VPC must be associated with a routing table, which controls the routing for the subnet (subnet route table).

You can explicitly associate a subnet with a particular route table. Otherwise, the subnet is implicitly associated with the main route table.

A subnet can only be associated with one route table at a time, but you can associate multiple subnets with the same subnet route
table.

Public Subnet: If a subnet is associated with a route table that has a route to an internet gateway, it's known as a public subnet.

Private Subnet: If a subnet is associated with a route table that does not have a route to an internet gateway, it's known as a private subnet.

In the public subnet's route table, we can specify a route for the internet gateway to all destinations not explicitly known to the route table (0.0.0.0/0 for IPv4 or ::/0 for IPv6).

Alternatively, we can scope the route to a narrower range of IP addresses; for example, the public IPv4 addresses of your company’s public endpoints outside of AWS, or the Elastic
IP addresses of other Amazon EC2 instances outside your VPC.

Route Table Creation:

Now, we'll be able to connect to the web-server via ssh.

But we can't access the DB-server as it doesn't have public IP.
Similarly, we can't connect DB-server from Web-server console as we don't have access to "Db-server-key.pem' file.

So let's make a pem file using vi editor and save contents of Db-server-key.pem file to vi editor.

Now, we've to give read-write permission to the server.

Type: "chmod 600 DB-server-key.pem"

Now we'll be able to access the Db-server via Web-server.

We can verify that Web-server has access to internet by using command "ping 8.8.8.8".

But DB-server can't access the internet.

If we want to give internet access to DB-server we have to configure it in such a manner that no one else from the internet can access it except us.

To have such kind of setup we deploy "NAT-Gateway".
"Nat-gatweway has both private and public IP associated with it.
We need to deploy it in Public subnet. Hence, the subnet having IGW access is called public subnet.

If DB-Subnet wants to have internet access then it has to connect to the public ip of NAT gateway.

Static Website hosting on AWS S3

Ashutosh Mallick — Wed, 13 Jul 2022 06:08:19 +0000

On continuation to my previous documentation regarding S3 services and their usage, we'll discuss about static website hosting using s3 services.
The first step to hosting a static website on AWS S3 is to create an S3 bucket in your account.
After creating the bucket, we will upload the website contents and files in our bucket. The website content will then be assigned specific permissions to be accessible to the public.

create the bucket and the bucket name should be unique for all AWS accounts around the world:

Since we want the website to be publicly accessible, we need to grant the public access to the objects of this S3 bucket. For that, uncheck the Block all public access checkbox in the “Block Public Access setting for this bucket” section.

After configuring the public access settings, a section will appear to acknowledge the S3 bucket and its content being made public. Check the box to acknowledge it.
Now download a sample website. Extract it and copy the contents of the folder into your TNT drive folder you created earlier.
Go to the Objects section, and then Click on the upload button. Now, browse your system for the directory you want to upload into the S3 bucket. Select the static website directory and upload it to the S3 bucket.

Uploading the static site content may take some time depending upon the size of the folder:

After a successful upload, click close at the right corner. You will be directed back to the object section.
Now go to AWS S3 console. Select your bucket and select all objects, make them public.
After uploading the static site content, enable hosting on your S3 bucket. In order to allow static website hosting on your S3 bucket, go to the properties tab from the top menu in the S3 bucket.
Scroll down in properties tab and look for the Static Website Hosting section.
Click on the Edit button in the Static website hosting section and enable the hosting.
After enabling static website hosting, specify the index file of your project (the opening page of your website or webapplication). In this case, it is index.html.
Also, if there is an error file in your project, you must specify it in the error document field. This will appear in case your actual web page is not reachable. Now. click on the Save changes button to apply the changes to your S3 bucket.
Now, our S3 bucket is hosting the website content uploaded to it and is publicly accessible. In order to access the website, we need a public URL that AWS itself provides. This URL can be seen in the static website hosting section of the S3 bucket.

To make our content accessible publicly, we need to add a bucket policy for which we have to go to the permissions tab of our S3 bucket to make some changes to the permissions of our S3 bucket.

Paste the following JSON in the editor to allow the public to read files from the bucket:

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PublicRead",
"Effect": "Allow",
"Principal": "",
"Action": [
"s3:GetObject",
"s3:GetObjectVersion"
],
"Resource": "arn:aws:s3:::YOUR-S3-BUCKETNAME/"
}
]
}

-Make sure to replace “YOUR-S3-BUCKETNAME” with your S3 bucket name in the JSON policy.

After setting the permissions for the bucket, it’s time to access the webpage through the URL. For this, go to the Objects tab of the S3 bucket and go to the static site directory.
Look for the index.html file in the folder, which you defined as the index document for this project. Click on the index.html file.

Now, in the object overview section under the properties tab, you can find the URL of the static website.

Go to this URL, and the static website hosted on the AWS S3 bucket will be accessible via browser.

In this article, we went through the configuration of a static website using the AWS S3 bucket. We can use a custom domain to launch the static website via configuration using AWS Route 53 services.
You can create the website pages, upload them into an S3 bucket, and users can access the website. You do not require to maintain any backend servers. It is a serverless architecture, and AWS manages them for you automatically.

Thank you!! Stay tuned for more exciting stuff on AWS services and provide your feedback in the comments.

AWS Simple Storage Services(s3). S3 buckets, mapping of S3 objects as network drive.

Ashutosh Mallick — Wed, 06 Jul 2022 05:31:06 +0000

Amazon S3

Simple Storage Service is a scalable, high-speed, low-cost web-based service designed for online backup and archiving of data and application programs.
We can use S3 to store and retrieve any amount of data at any time, from anywhere on the web. It allows to upload, store, and download any type of files up to 5 GB in size.
This service allows the subscribers to access the same systems that Amazon uses to run its own web sites. The subscriber has control over the accessibility of data, i.e. privately/publicly
accessible.
Since S3 is an object storage, it's not divided into multiparts or exists as a single storage.
When you modify a file in block storage, only the pieces that are changed are updated. When a file in object storage is modified, the entire object is updated.
In object storage, each object consists of data, metadata, and a key.
The data might be an image, video, text document, or any other type of file. Metadata contains information about what the data is, how it is used, the object size, and so on. An object’s key is its unique identifier.

Provisioning before S3

It's important to keep backup of your data. But before s3 for backup of data, we used to attach the servers to on premises SAN storage.
For SAN storage, due to its hardware lifetime, we need to regularly change the on premises services or hardware disk drives, which was a costlier process. Thus AWS introduced S3 services.
S3 buckets stores backup of your servers and it's very cheap. Such as for storing 1GB data costs you $0.01/month. But cost varies accordingly.
According to Aws SLA, for S3 bucket the data is 99.999999% available and durable. AWS makes three copies of data present in S3 bucket. One copy remains with the user while the other two copies are stored in other data centers. Hence you'll have high SLA.

S3 Buckets

A bucket is a container for objects stored in Amazon S3. You can store any number of objects in a bucket and can have up to 100 buckets in your account.
When you create a bucket, you enter a bucket name and choose the AWS Region where the bucket will reside. After you create a bucket, you cannot change the name of the bucket or its Region.
Bucket names must follow the bucket naming rules i.e, Bucket name should be unique and having small case letters only. You can also configure a bucket to use S3 Versioning or other storage management features.
Each object has three main components; the object’s content, the object’s unique identifier, and the object’s metadata (including its name, size, URL).

An Object cannot be independent, it must exist within a bucket. There can be hundreds of buckets in each Amazon account and within each bucket, there can be hundreds of objects.

Bucket creation process:

Create a bucket, give a unique bucket name. If ACL is disabled, enable it. Untick "Block all public access".
Leave other things as default and create.

Upload an Object in the bucket. Select a file from your local directory and upload it.

Mapping of S3 bucket object using TNT drive

In the bucket, click on object actions dropdown and select make public using ACL. So that the object will be publicly accessible.

Now download TNT drive in order to mount S3 bucket in our local server. Now add account in TNT drive.
Go to IAM and create a user with S3 full access.

IAM user creation from AWS console.

Now in TNT click add a account. Create account with access key and secret key ID for your IAM user created.

Now select "add a new mapped drive" in TNT drive. Browse for the S3 bucket you created and select your bucket. Click on add a new drive.

Go to my PC, find the mapped drive and see if the object is listed inside the bucket directory.( We can see that the object will be present inside the mapped drive).
Now add a text file in the local folder for your bucket and save it. Now you can the same changes updated in AWS S3 console. That means in S3 bucket, you'll be having two objects.
Hence, we can also mount S3 objects as network drives also.

In the upcoming blogs we'll see how to host a static website using S3, S3 versioning and different storage classes of S3.
Stay tuned. :)

Thank You!!!

AWS IAM

Ashutosh Mallick — Thu, 30 Jun 2022 05:23:03 +0000

AWS Identity and Access Management (IAM)

What is IAM?

IAM stands for Identity and Access Management.
AWS Identity and Access Management (IAM) enables you to manage access to AWS services and resources securely.
IAM is a web services that enable you to manage users and group
permissions in AWS
IAM gives you the flexibility to configure access based on your company’s specific operational and security needs.
It is targeted at organizations with multiple users or systems that use AWS products such as Amazon Elastic Compute Cloud, Amazon Relational Database Service, and the AWS Management Console.
You do this by using a combination of IAM features, which are explored in detail in this lesson:
• IAM users, groups, and roles
• IAM policies
• Multi-factor authentication.

Why we go for IAM?

To avoid a security and logistical headache.
When you create an AWS account, it has permissions to do
anything and everything with all the resources.
IAM Allows you to limit access as needed and gives you the
peace of mind that approved people are accessing the right
resources in the desired manner.
IAM will allow us to create multiple users with individual security credentials and permissions, with this IAM, each user is allowed to do only what they need to do.

Each user in the AWS account must have a unique set of credentials to access the console. Different types of users have different set of permission.
Administrators need to access all AWS resources
Developers need only access on Amazon Elastic Compute Cloud (EC2)
We can use IAM to create a unique user for each employee and define their permissions.

AWS Root user and IAM user

*AWS Account Root User *
When you first create an AWS account, you begin with an identity known as the root user.
The root user is accessed by signing in with the email address and password that you used to create your AWS account.
You can think of the root user as being similar to the owner of the coffee shop. It has complete access to all the AWS services and resources in the account.

Best practices

Do not use the root user for everyday tasks. Instead, use the root user to create your first IAM user and assign it permissions to create other users.
Then, continue to create other IAM users, and access those identities for performing regular tasks throughout AWS.
Only use the root user when you need to perform a limited number of tasks that are only available to the root user.
Examples of these tasks include changing your root user email address and changing your AWS support plan.

IAM users

An IAM user is an identity that you create in AWS. It represents the person or application that interacts with AWS services and resources. It consists of a name and credentials.
By default, when you create a new IAM user in AWS, it has no permissions associated with it. To allow the IAM user to perform specific actions in AWS, such as launching an Amazon EC2 instance or creating an Amazon S3 bucket, you must grant the IAM user the necessary permissions.

Best practice:

You should create individual IAM users for each person who needs to access AWS. Even if you have multiple employees who require the same level of access, you should create individual IAM users for each of them.
This provides additional security by allowing each IAM user to have a unique set of security credentials.
IAM user Creation:

IAM policies

An IAM policy is a document that allows or denies permissions to AWS services and resources.
IAM policies enable you to customize users’ levels of access to resources. For example, you can allow users to access all of the Amazon S3 buckets within your AWS account, or only a specific bucket.
By having IAM policies, you help to prevent users or roles from having more permissions than needed to perform their tasks.
For example, if an employee needs access to only a specific bucket, specify the bucket in the IAM policy.
We do this instead of granting the employee access to all of the buckets in your AWS account.
Here’s an example of how IAM policies work. Suppose that the coffee shop owner has to create an IAM user for a newly hired cashier. The cashier needs access to the receipts kept in an Amazon S3 bucket
The below image represents an IAM policy writyten in json format.

In this example, the IAM policy is allowing a specific action within Amazon S3: ListObject. The policy also mentions a specific bucket ID: maven-repo-tutorial.asimi.net.
When the owner attaches this policy to the cashier’s IAM user, it will allow the cashier to view all of the objects in the maven-repo-tutorial.asimi.net bucket.
If the owner wants the cashier to be able to access other services and perform other actions in AWS, the owner must attach additional policies to specify these services and actions.
Now, suppose that the coffee shop has hired a few more cashiers. Instead of assigning permissions to each individual IAM user, the owner places the users into an IAM group.

IAM Groups

We can create IAM groups for smilar type of sevices like EC2 and S3. We can add custom policies for the group.
Policies will be applied to every user associated in the group, so we don't have to assign policies to each user manually.
An IAM group is a collection of IAM users. When you assign an IAM policy to a group, all users in the group are granted permissions specified by the policy.
Here’s an example of how this might work in the coffee shop. Instead of assigning permissions to cashiers one at a time, the owner can create a “Cashiers” IAM group. The owner can then add IAM users to the group and then attach permissions at the group level.
Below pictures include the creation process of an IAM user group in AWS console.

Assigning IAM policies at the group level also makes it easier to adjust permissions when an employee transfers to a different job.
For example, if a cashier becomes an inventory specialist, the coffee shop owner removes them from the “Cashiers” IAM group and adds them into the “Inventory Specialists” IAM group. This ensures that employees have only the permissions that are required for their current role.

IAM Roles

Using IAM roles we can attach S3 and EC2 with each other(kind of pipelining), in this case we don't have to use ID and Password.
An IAM role is an identity that you can assume to gain temporary access to permissions.
Before an IAM user, application, or service can assume an IAM role, they must be granted permissions to switch to the role.
When someone assumes an IAM role, they abandon all previous permissions that they had under a previous role and assume the permissions of the new role.
IAM roles are ideal for situations in which access to services or resources needs to be granted temporarily, instead of long-term.

Example of IAM Roles

An example of how IAM roles could be used in the coffee shop:

First, the owner grants the employee permissions to switch to a role for each workstation in the coffee shop.
Next, the employee begins their day by assuming the “Cashier” role. This grants them access to the cash register system.
Later in the day, the employee needs to update the inventory system. They assume the “Inventory” role. This grants the employee access to the inventory system and also revokes their access to the cash register system.

Multi-factor Authentication

When you sign in to Gmail(say) it requires you to provide multiple pieces of information to verify your identity. Such as you might have needed to provide your password and then a second form of authentication, such as a random code sent to your phone.
This is an example of multi-factor authentication. In IAM, multi-factor authentication (MFA) provides an extra layer of security for your AWS account. This is also called as Hardening of AWS account.

Setup:

ON IAM dashboard click on activate MFA.
Then click on virtual MFA device. We use an authenticator app in our cell phone as MFA device to scan the QR code and to activate MFA.
After scanning the QR code via authenticator app we get MAF1 and MFA2 codes.
On next login for every user they'll have to undergo MFA authentication.
We can also activate MFA via AWS CLI. But we have to provide access key and secret key in this case. So it's better not to go ahead with this step as anyone can access our ervices if they get the keys.

Thank you !!
Follow me for more detailed contents on AWS services :)

Website Hosting on Ec2 instance with SSL enabled using an Application Load Balancer and Route53.

Ashutosh Mallick — Tue, 28 Jun 2022 03:56:04 +0000

Amazon EC2: The Amazon Elastic Compute Cloud is a web service that helps you to run virtual machines in the cloud by configuring its capacity, security, and networking.

EC2 instance: A virtual server on Amazon’s Elastic Compute Cloud (EC2) to run your business software.

Create an EC2 instance and install a web server
First, you create an EC2 instance in the public subnet of your VPC. [Here we used a default public subnet and default VPC].

Choose EC2 Dashboard, and then choose Launch instance,
Choose the Amazon Linux 2 AMI.

Choose the t2.micro instance type, as shown following, and then choose Next: Configure Instance Details.
On the Configure Instance Details page : You can select number of instances you need and Go with Your Default setup or [Choose you VPC if you created].
Choose Next: Add Storage.
On the Add Storage page, keep the default values and choose

Next: Add Tags.

Choose Next: Configure Security Group.

On the Configure Security Group page, shown following, choose Select an existing security group.

Choose Review and Launch.
On the Select an existing key pair or create a new key pair

To launch your EC2 instance, choose Launch Instances.
Name those server as

Now connect these servers with xshell and so to root user.

sudo su , cd [command for root user]

In-order to host a website in ec2 , First you need to download web server [ Here we are going to install apache server ].
yum install httpd , type yes if required

We can use Github commands to clone our website code from our repo , so You need to install git. yum install git -y

Now we have to clone that repo to our web server folder name [ html ] as follow

cd /var/www/html [ path ].

We need to start Apache Server using command below

[In case if we stop and start our server , we need start our apache again , to avoid this we can simply enable that server]
Continue this process with 3 server [You can use single or multiple server ]
Paste public IP to browser , You can able to see your website

Some other ways to upload your website

You can use s3 bucket to upload your website and using IAM we can access to this or using WinSCP application you can simply copy paste your code to that folder.

What is a load balancer ?
A load balancer serves as the single point of contact for clients. The load balancer distributes incoming application traffic across multiple targets, such as EC2 instances, in multiple Availability Zones.

This increases the availability of your application. You add one or more listeners to your load balancer.

A listener checks for connection requests from clients, using the protocol and port that you configure. The rules that you define for a listener determine how the load balancer routes requests to its registered targets.

Each target group routes requests to one or more registered targets, such as EC2 instances, using the protocol and port number that you specify.

Elastic Load Balancing supports the following load balancers: Application Load Balancers, Network Load Balancers, Gateway Load Balancers, and Classic Load Balancers.

Create Load Balancer

Search for load balancer and click on create.

Select Application load balancer.

Name it as follow

Select VPC and subnet

Create a Security group enable ssh,http,https.

In the navigation pane, under Load Balancing, choose Target Groups.
Choose Create target group.

Under Basic configuration, keep the Target type as instance.

For Target group name, enter a name for the new target group.
Keep the default protocol (HTTP) and port (80).
Select the VPC containing your instances. Keep the protocol version as HTTP1.
For Health checks, keep the default settings.

Choose Next.

On the Register targets page, complete the following steps. This is an optional step for creating the load balancer. However, you must register this target if you want to test your load balancer and ensure that it is routing traffic to this target.

For Available instances, select one or more instances.
Keep the default port 80, and choose Include as pending below.

Choose Create target group

Leave default setup
Create load balancer

Paste that DNS name of your load balancer in browser you can see your website.

How to attach a domain name

you require a free or purchased domain to fully complete all the steps. If you already have a domain, then awesome, if not, don't worry, you can get a free domain! You can visit the following site and get yourself a free domain.
https://www.freenom.com/en/index.html?lang=en

Once you sort out your domain, you should go back to the AWS console and navigate to the “Route 53” service. You should then navigate to “Hosted zones” and create a new hosted zone.
Amazon Route 53 is a highly available and scalable Domain Name System (DNS) web service. You can use Route 53 to perform three main functions in any combination: domain registration, DNS routing, and health checking.

You have to make sure to enter the exact domain name and select “Public hosted zone” for the type when creating the new hosted zone.

Once you have created the hosted zone, it should contain two records, NS (Name Server) record and SOA (Start Of Authority) record. You will need to use the NS record in the next step!

Next, you should head over to the admin panel of your domain provider, in my case it is freenom.com. You should find the section which enables you to configure the name servers for the domain! For different domain providers, this would look a bit different!

_- You should be aware that sometimes Nameservers takes a couple of hours to Sync in. So if your domain doesn't work at the end of this article, be patient and try again in a few hours _

You can also create record name , record type , record traffic

After that you can type www.yourDomainName.ml in browser , But its not secured , to make it secured we need to attach a SSL to it.

How to attach a SSL to our domain

AWS Certificate Manager is a service that lets you easily provision, manage, and deploy public and private Secure Sockets Layer/Transport Layer Security (SSL/TLS) certificates for use with AWS services and your internal connected resources

Search for Certificate Manager

Request a public certificate

Enter your domain name you can use * before your domain name for host based routing.

Click on certificate and create records

Create DNS records in Amazon Route 53.

A CNAME record is added to your record

After that go to load balancer .. Click listener and Add Listener.

Add HTTPS Protocol 443 port , Select the target group .

In default SSL certificate part :- Add the certificate [ That certificate you have created in Certificate Manager ]

Edit the HTTP part Click on add condition select Host add www.domanname.ml and redirect to HTTPS 443 port [save it] add domanname.ml and redirect to HTTPS 443 port [save it]

Now it will be redirect to HTTPS protocol,

*Connection Draining
*
When Connection Draining is enabled and configured, the process of deregistering an instance from an Elastic Load Balancer gains an additional step. For the duration of the configured timeout, the load balancer will allow existing, in-flight requests made to an instance to complete, but it will not send any new requests to the instance. During this time, the API will report the status of the instance as in Service, along with a message stating that “Instance deregistration currently in progress.” Once the timeout is reached, any remaining connections will be forcibly closed.

Stickyness

Suppose two separate web browsers each request three separate web pages in turn. Each request can go to any of the EC2 instances behind the load balancer, like this:

When a particular request reaches a given EC2 instance, the instance must retrieve information about the user from state data that must be stored globally. There’s no opportunity for the instance to cache any data since the odds that several requests from the same user / browser will go down as more instances are added to the load balancer.
With the new sticky session feature, it is possible to instruct the load balancer to route repeated requests to the same EC2 instance whenever possible.

In this case, the instances can cache user data locally for better performance.
A series of requests from the user will be routed to the same EC2 instance if possible.
If the instance has been terminated or has failed a recent health check, the load balancer will route the request to another instance.

IP addressing

Ashutosh Mallick — Sat, 09 Apr 2022 03:24:50 +0000

What is Ip address?
IP address or internet protocol address is an unique number assigned to a server when it's connected to the internet.
Basically your internet service provider (ISP) will assign a geologically identifiable number when you connect to Internet.

Purpose of Ip address

Ip address is the unique identity off a server. -Each domain like "Apple.com" is bounded with an ip address.
Let's say we bind "apple.com" domain with ip address of the server where application is deployed. Hence, when a user will hit the domain name, they can directly navigate to the particular website.
Most of the larger websites like Apple has it’s own IP address like 17.172.224.47. You can enter this IP address directly on the browser’s address bar to resolve the domain name of the site.
Smaller websites may share single server address amongst hundreds of other websites.
On the other hand, the websites and apps also can track your details based on your IP address.
Let us take the same example of accessing Apple’s website. When you open the site from China, it will automatically redirects to the Chinese version applce.com/cn/ instead of the English version apple.com.

Types of ip address

When we launch a server, it has two ip address : public ip and private ip address. -Public ip is unique for the particular server only when the server is running.
By using public ip, we can connect to the servers even if we are outside of the network where the server is deployed.
But private ip is only accessible within the same network where the server is deployed.
For a database, only private ip is assigned to it. That means only internal resources of that network will be able to communicate and access the server.
Hence, by using private ip for the servers, we restrict access to the servers and allow specified users only.

Create an instance "Web-1" and note down its public ip address. If we reboot the instance, we can see that the public ip remains unchanged.

If we stop the instance it'll have only private ip address and again if we restart the instance then we'll see that the public ip of the instance gets changed.

A public IP address or primary address represents the whole network of devices associated with it.
Every device included within with your primary address contains their own private IP address. ISP is responsible to provide your public IP address to your router.
Public IP addresses are further classified into two categories- dynamic and static.

Dynamic IP addresses

As the name suggests, Dynamic IP addresses change automatically and frequently. With this types of IP address, ISPs already purchase a bulk stock of IP addresses and allocate them in some order to their customers.

Static IP addresses

In comparison to dynamic IP addresses, static addresses are constant in nature.
The network assigns the IP address to the device only once and, it remains consistent.
Though most firms or individuals do not prefer to have a static IP address, it is essential to have a static IP address for an organization that wants to host its network server.
It protects websites and email addresses linked with it with a constant IP address.

Elastic Ip address

Elastic IP addresses are most commonly used to help with fault-tolerant instances or software.
For example, if you have an EC2 instance that has an Elastic IP address and that instance is stopped or terminated, you can remap the address and re-associate it with another instance in your account.
On termination or rebooting the public ip of instance remains fixed if the instance is associated with an elastic ip address.
AWS provides 5 elastic ip in a single region.
Let's configure elastic ip setup :
Launch an instance and allocate an elastic ip to it.
Mark the public ip of the elastic ip.
Associate the elastic ip with the running instance (Web-1).
Now try stopping and restarting the instance and check that the public ip remains unchanged.

IPV4 and IPV6

An IPv4 address (Internet Protocol version 4) comprises four sets of numbers, each ranging from 0 to 255, which are separated by periods. For example, a site's IP address is 104.103.88.45.
IPv4 is the current standard for IP addresses in the TCP/IP model, while IPv6 is a newer IP version on the rise.
Internet Protocol version 6, or IPv6, was first introduced in the late 1990s as a replacement for IPv4.
It uses 128-bit addresses formatted as eight groups of four hexadecimal numbers separated by colons.
IPv6 is the solution that addresses the relatively limited number of IP addresses possible under IPv4. Under IPv6, there will no longer be a shortage of the total number of possible addresses.

AWS EFS (Elastic File Storage)

Ashutosh Mallick — Sun, 27 Mar 2022 05:42:50 +0000

What is EFS?

Amazon EFS provides scalable file storage for use with Amazon EC2. You can create an EFS file system and configure your instances to mount the file system.
You can use an EFS file system as a common data source for workloads and applications running on multiple instances.

Instance store lifetime

You can specify instance store volumes for an instance only when you launch it. You can't detach an instance store volume from one instance and attach it to a different instance.
The data in an instance store persists only during the lifetime of its associated instance.
If an instance reboots, data in the instance store persists. However, data in the instance store is lost under any of the following circumstances: • The underlying disk drive fails • The instance stops • The instance terminates Therefore, we generally do not rely on instance store for valuable, long-term data. Instead, we use more durable data storage, such as Amazon S3, Amazon EBS, or Amazon EFS. Here we'll discuss about features and implementation of EFS.

Characteristics of EFS volume

In EFS, we mount the volume in multiple instances like a shared storage.
EBS volume can only be mounted on AWS Ec2 whereas EFS can be mounted on AWS Ec2, Azure Vms, GCP vms and also on premises servers.
EFS can be attached with servers irrespective of availability zones.
For EFS, we don't have to define storage and aws only charges for the storage that's used i.e. EFS is synced with the data copied.
Amazon EFS is a fully managed service for hosting Network File System (NFS) filesystems in the cloud.
It is an implementation of a NFS file share and is accessed using the NFS protocol.
It provides elastic storage capacity and pay for what you use (in contrast to Amazon EBS with which you pay for what you provision).
You can configure mount-points in one, or many, AZs.
You can mount an AWS EFS filesystem from on-premises systems ONLY if you are using AWS Direct Connect or a VPN connection.
Typical use cases include big data and analytics, media processing workflows, content management, web serving, home directories etc.
AWS EFS can scale up to petabytes. AWS EFS is elastic and grows and shrinks as you add and remove data.
You can concurrently connect up to thousands of Amazon EC2 instances, from multiple AZs.

The following diagram depicts the various options for mounting an EFS filesystem:

Need for EFS in AWS

In EBS, we clubbed ebs volumes and mounted those volumes on a server.
But to attach that same volume to another server/instance we have to first unmount the volume from server-1 and then attach vol. to server-2.
Similarly for multiple servers, we can unmount and detach volume from one server and attach it to another.

On premises setup for EBS

Load balancers are used to distribute traffic into different servers so that each server can handle requests.
Data from each server gets stored in the database, we've to give the database an endpoint. If we don't have an database, we can store the data locally.
Such as let's consider an e-commerce app. If today you accessed the app and you get response from server-1. then your details will be stored in EBS volume of server-1.
The next day if you access the app and you get response from server-2, then you can't see your details listed over there as server-2 has different EBS volume.
To solve this issue we use EFS volume and mount it to multiple instances in order to make the app more user friendly.

On premises setup For EFS :

-First we establish a hardware rack and then mount hardware on it.
Linux and NFS are configured on the servers.
Then we attach the server to SAN storage by a LAN cable.
NFS basically converts the physical volumes into logical volumes. So that we can attach the logical volumes into different servers. Changes in each server gets reflected on logical volumes.
But, On premises setup is costly, so Aws provides these services as functionality of NFS using EFS.

EFS VS EBS
We often get confused about EFS and EBS volumes and their characteristics. Here I've mentioned some basic and key differences between ebs and efs.

EBS can be mounted on EC2 instances only. Whereas, EFS can be mounted on Virtual machine(VMs) of AZURE, Compute engines of GCP, and also with the on premises setup too.
For EBS, the instance attached with ebs should be in the same availability zone as that of the ebs vol. If not, then we've to create a snapshot of ebs in the same region as that of instance and attach it.
But in case of EFS, the efs vol can be attached to instance irrespective of the availability zone specified.
If we use 10 GB of EBS volume, AWS charges us for the entire 10GB, even if all of it isn't utilized.
But in EFS volume we don't specify the storage. Rather, it is synced with the data copied. Which means, if you use 1GB of data then you'll be charged for that 1GB only.
So, EFS is cost effective than EBS.

PROVISIONING:

Launch an ec2 instance with security group configured to allow only your IP to ssh into the server.
To mount EFS on server, we need to allow NFS port i.e, port 2049.

Now go to EFS on AWS console and create a file system. For the network part click manage -> security groups -> select the security group you configured (say my-sg) and save the changes.
Here you can choose any availability zone for your efs vol. configuration. (My instance is at ap-south-1a and I've configured my efs vol. at ap-south-1b).

After configuring network part, click on attach -> mount via ip. Then copy the command to attach efs via nfs.
Run the command in xshell

-After mounting the volume create some files in the instance "my-server" by using touch command.

Similarly, if we create two instances (VM-1, and VM-2) and configure efs volume via NFS client following the above steps, we've to connect both the instances separately and mount the efs volume by creating a folder (say mkdir /efs1 or mkdir /efs2 respectively.)
Now if we create some files on VM-1 by using touch command (say touch abc{1..100}) , these files will also be listed over VM-2 server as we're using the same EFS volume for both the servers.

CONCUSION:

Basically, we learned that in EFS, we can attach volume to multiple servers and if we make changes in one server, then the same changes are reflected on the other servers too.

Basics of Cloud Computing

Ashutosh Mallick — Fri, 11 Feb 2022 03:58:52 +0000

Cloud Computing

In 2006, Amazon Web Services (AWS) started to offer IT services to the market in the form of web services, which is nowadays known as cloud computing.
With this cloud, we need not plan for servers and other IT infrastructure which takes up much of time in advance. Instead, these services can instantly spin up hundreds or thousands of servers in minutes and deliver results faster.
We pay only for what we use with no up-front expenses and no long-term commitments, which makes AWS cost efficient.

Computing Models

1. Desktop Computing :

In traditional desktop computing, each employee accesses business software from his or her own individual computer. Desktop applications like Microsoft Word or Adobe Acrobat are installed directly on employee computers and business applications are installed on back end servers.
Main disadvantage of desktop computing is that it isn't remotely accessible.

2. Client-Server Computing model :

In client server computing, the clients requests a resource and the server provides that resource.
A server may serve multiple clients at the same time while a client is in contact with only one server.
Both the client and server usually communicate via a computer network but sometimes they may communicate internally within the same network.

Characteristics of Client Server Computing:

The client server computing works with a system of request and response. The client sends a request to the server and the server responds with the desired information.
A server can only accommodate a limited number of client requests at a time. So it uses a system based to priority to respond to the requests.
Denial of Service attacks hinders server ability to respond to authentic client requests by increasing the Cpu utilization of server with false requests.
An example of a client server computing system is a web server. It returns the web pages to the clients that requested them.

Advantages of Client Server Computing :

All the required data is concentrated in the server. So it is easy to protect the data and provide authorization and authentication.
The server need not be located physically close to the clients. Yet the data can be accessed efficiently.
It is easy to replace, upgrade or relocate the nodes in the client server model because all the nodes are independent and request data only from the server.

Disadvantages of Client Server Computing :

If all the clients simultaneously request data from the server, it may get overloaded. This may lead to latency in the network.
If the server fails for any reason, then requests of the clients can't be fulfilled.
The cost of setting and maintaining a client server model are quite high.

3. Cluster Computing :

Cluster computing is a collection of tightly or loosely connected computers that work together so that they act as a single entity.
The connected computers execute operations all together thus creating the idea of a single system. The clusters are generally connected through fast local area networks (LANs).
Cluster computing gives a relatively inexpensive, unconventional to the large server or mainframe computer solutions.
It resolves the demand for content criticality and process services in a faster way. Many organizations and IT companies are implementing cluster computing to augment their scalability, availability, processing speed and resource management at economic prices.

Load Balancing Clusters :

This Cluster distributes all the incoming traffic/requests for resources from nodes that run the same programs and machines.
In this Cluster model, all the nodes are responsible for tracking orders, and if a node fails, then the requests are distributed amongst all the nodes available. Such a solution is usually used on web server farms.

High Availability (HA) and Failover Clusters:

The basic idea in this form of Cluster is that if a node fails, then applications and services can be made available to other nodes.
These types of Clusters serve as the base for critical missions, mails, files, and application servers.

Advantages of Cluster Computing :

High Performance :

The systems offer better and enhanced performance than that of mainframe computer networks.

Easy to manage :

Cluster Computing is manageable and easy to implement.

Scalable :

Resources can be added to the clusters accordingly.

Expandability :

Computer clusters can be expanded easily by adding additional computers to the network. Cluster computing is capable of combining several additional resources or the networks to the existing computer system.

Availability :

The other nodes will be active when one node gets failed and will function as a proxy for the failed node. This makes sure for enhanced availability.

Flexibility :

It can be upgraded to the superior specification or additional nodes can be added.

Disadvantages of Cluster Computing :

High cost :

It is not so much cost-effective due to its high hardware and its design.

Problem in finding fault :

It is difficult to find which component has a fault.

More space is needed :

Infrastructure may increase as more servers are needed to manage and monitor.

4. Grid Computing :

Grid computing is the practice of leveraging multiple computers, often geographically distributed but connected by networks, to work together to accomplish joint tasks.
It is typically run on a “data grid,” a set of computers that directly interact with each other to coordinate jobs.
Grid computing works by running specialized software on every computer that participates in the data grid. The software acts as the manager of the entire system and coordinates various tasks across the grid.
Specifically, the software assigns subtasks to each computer so they can work simultaneously on their respective subtasks.
After the completion of subtasks, the outputs are gathered and aggregated to complete a larger-scale task.
The software lets each computer communicate over the network with the other computers so they can share information on what portion of the subtasks each computer is running, and how to consolidate and deliver outputs.

5. What is Cloud Computing?

Cloud computing is an internet-based computing service in which large groups of remote servers are networked to allow centralized data storage, and online access to computer services or resources.
Using cloud computing, organizations can use shared computing and storage resources rather than building, operating, and improving infrastructure on their own.
Cloud computing is a model that enables the following features:
Users can provision and release resources on-demand
Resources can be scaled up or down automatically, depending on the load.
Resources are accessible over a network with proper security.
Cloud service providers can enable a pay-as-you-go model, where customers are charged based on the type of resources and per usage.

Cloud Computing Models

Types of Clouds :

There are three types of clouds - Public, Private, and Hybrid cloud.

Public Cloud

In public cloud, the third-party service providers make resources and services available to their customers via Internet. Customer’s data and related security is with the service providers’ owned infrastructure.

Private Cloud

A private cloud also provides almost similar features as public cloud, but the data and services are managed by the organization or by the third party only for the customer’s organization. In this type of cloud, major control is over the infrastructure so security related issues are minimized.

Hybrid Cloud

A hybrid cloud is the combination of both private and public cloud. The decision to run on private or public cloud usually depends on various parameters like sensitivity of data and applications, industry certifications and required standards, regulations, etc.

Cloud Service Models :

There are three types of service models in cloud - IaaS, PaaS, and SaaS.

IaaS:

IaaS stands for "Infrastructure as a Service".
It provides users with the capability to provision processing, storage, and network connectivity on demand.
Using this service model, the customers can develop their own applications on these resources.

PaaS:

PaaS stands for "Platform as a Service".
Here, the service provider provides various services like databases, queues, workflow engines, e-mails, etc. to their customers.
The customer can then use these components for building their own applications. The services, availability of resources and data backup are handled by the service provider that helps the customers to focus more on their application's functionality.

SaaS:

SaaS stands for Software as a Service.
As the name suggests, here the third-party providers provide end-user applications to their customers with some administrative capability at the application level, such as the ability to create and manage their users.
Also some level of customizability is possible such as the customers can use their own corporate logos, colors, etc.

Advantages of Cloud Computing:

Cost-Efficient:
Building our own servers and tools is time-consuming as well as expensive as we need to order, pay for, installing, and configure expensive hardware, long before we need it. However, using cloud computing, we only pay for the amount we use and when we use the computing resources. In this manner, cloud computing is cost efficient.
Reliability:
A cloud computing platform provides much more managed, reliable
and consistent service than an in-house IT infrastructure.
It guarantees 24x7 and365 days of service. If any of the server fails, then hosted applications and services can easily be migrated to any of the available servers.
Unlimited Storage:
Cloud computing provides almost unlimited storage capacity,
i.e., we need not worry about running out of storage space.
We can easily increase our current storage space availability. We can access as much or as little as we need. We can also decrease our storage space as per our requirement and cost of storage.
Backup & Recovery:
Storing data in the cloud, backing it up and restoring the
same is relatively easier than storing it on a physical device.
The cloud service providers also have enough technology to recover our data, so there is the convenience of recovering our data anytime.
Easy Access to Information:
Once you register yourself in cloud, you can access
your account from anywhere in the world provided there is internet connection at that point.
There are various storage and security facilities that vary with the account type chosen.

Disadvantages of Cloud Computing:

Security issues:
Security is the major issue in cloud computing. The cloud service providers implement the best security standards and industry certifications, however, storing data and important
files on external service providers always bears a risk.
AWS cloud infrastructure is designed to be the most flexible and secured cloud network.
It provides scalable and highly reliable platform that enables customers to deploy applications and data quickly and securely.
Technical issues:
As cloud service providers offer services to number of clients each day, sometimes the system can have some serious issues leading to business processes temporarily being suspended.
Additionally, if the internet connection is offline then we will not be able to access any of the applications, server, or data from the cloud.

Configuring EBS volume (Extending and reducing volume, Concept of snapshots)

Ashutosh Mallick — Mon, 31 Jan 2022 06:39:55 +0000

In the last blog we talked about AWS Block storage, EBS to be specific. We also discussed the types, uses of EBS volume, how to attach it to an instance and how to mount it in a disk.
Let's move further and discuss about resizing, reducing the volume and some more interesting facts about EBS volume.

Resizing EBS volume

Now go to volumes in aws console. Select the EBS volume we created "Extra-vol" and select modify volume. Change the volume size from (15GB to 20GB). -Now go to xshell for your instance "Test-1" and t ype "resize2fs /dev/xvdf" command to resize your volume from 15 to 20 GB.
In xshell prompt you can list the volume attached with your instance by typing "df -h". You can see the 20GB vol. listed over there.

Attaching volume to different instances

For attaching volume (20GB) to another instance we've to first unmount "Extra-vol" from "Test-1" instance.
We should never go for forced detaching of volume. Rather we should first unmount it as per the best industrial practices.
To unmount the vol, go to the root folder and then type "umount /india" or "umount /dev/xvdf" any of these two will do. -Now detach the "Extra-vol" from "Test-1" server and create another instance in same region as that of the EBS volume (say "Test-2" server in ap-south-1a region).
Connect the "Test-2" server via SSH. Attach "Extra-vol" to "Test-2". Verify the same by typing "lsblk" in xshell prompt. -Now mount the EBS volume by creating a directory in Test-2 console(We don't have to format the disk again as it was formatted once).
Now create another instance "Test-3" in another region (ap-south-1b). Now let's unmount and detach the volume from Test-2 and attach it to Test-3.
To unmount follow the same procedure as discussed earlier. Go to root folder and type umount /dev/xvdf. Now detach the volume from "Test-2" server.
Now the challenge is how to attach "Extra-vol" to "Test-3". Because, the server and volume belong to different regions. Hence, we take the help of snapshots in this case.

Amazon EBS snapshots

You can back up the data on your Amazon EBS volumes to Amazon S3 by taking point-in-time snapshots.
Snapshots are incremental backups, which means that only the blocks on the device that have changed after your most recent snapshot are saved. This minimizes the time required to create the snapshot and saves on storage costs by not duplicating data.
Each snapshot contains all of the information that is needed to restore your data (from the moment when the snapshot was taken) to a new EBS volume.

Creating EBS volume from snapshots

When you create an EBS volume based on a snapshot, the new volume begins as an exact replica of the original volume that was used to create the snapshot.
The replicated volume loads data in the background so that you can begin using it immediately. If you access data that hasn't been loaded yet, the volume immediately downloads the requested data from Amazon S3, and then continues loading the rest of the volume's data in the background.

Snapshots Pricing

Charges for your snapshots are based on the amount of data stored. Because snapshots are incremental, deleting a snapshot might not reduce your data storage costs.
Data referenced exclusively by a snapshot is removed when that snapshot is deleted, but data referenced by other snapshots is preserved.
Now let's go back to aws console and create a snapshot for the EBS volume. Name the snapshot as (Extra-vol-snapshot).

-Now create volume "New-vol" from snapshot that you've created in the earlier step. remember that we've to create the volume in the same region as that of "Test-3" server i.e, in ap-south-1b region.
Attach the "New-vol" to the server "Test-3". In Test-3 console mount the volume.

You can list the volumes in the disk by "df-h". Now go inside "india" directory and type "ls". You can see the 100 files that you created earlier are present.
Now let's create another volume say 1GB under the name "Reduced-vol". Deploy the volume in same region as that of "Test-3" server.
Now attach the "Reduced-vol" to "Test-3" server. Mount the volume to a disk you've created (say software).
Now let's sync all the files present in "india" directory to "software" directory by using the command "rsync -aHAXxSP /india/ /software".
Now type ls inside the software directory. You can see all the 100 files listed over there that you created earlier.
After you're done with testing, unmount the volume , detach it from "Test-3" server. After checking the above steps you can delete the EBS volumes and the snapshots you've created in ealier steps and terminate the "Test-3" server.

Amazon Data Lifecycle Manager

You can use Amazon Data Lifecycle Manager to automate the creation, retention, and deletion of EBS snapshots and EBS-backed AMIs. When you automate snapshot and AMI management, it helps you to:
Protect valuable data by enforcing a regular backup schedule.
Create standardized AMIs that can be refreshed at regular intervals.
Retain backups as required by auditors or internal compliance.
Reduce storage costs by deleting outdated backups.
Create disaster recovery backup policies that back up data to isolated accounts.

How Amazon Data Lifecycle Manager works

Snapshots

Snapshots are the primary means to back up data from your EBS volumes. Snapshots are incremental, containing only the volume data that changed since the previous snapshot.
When you delete one snapshot in a series of snapshots for a volume, only the data that's unique to that snapshot is removed. The rest of the captured history of the volume is preserved.

EBS-backed AMIs

An Amazon Machine Image (AMI) provides the information that's required to launch an instance. You can launch multiple instances from a single AMI when you need multiple instances with the same configuration.
Amazon Data Lifecycle Manager supports EBS-backed AMIs only. EBS-backed AMIs include a snapshot for each EBS volume that's attached to the source instance.
Now let's enable a Lifecycle manager policy. First create an instance(Test-server).
For lifecycle manager select type of backup. Here I'll take the instance backup. Under policy description, type "My-backup-policy". Keep policy status "enabled".
Now set the frequency of backup as per your convenience such as: Daily, weekly or monthly. Set the UTC time and expire time.
Establishing lifecycle manager will help you to store the data of "Test-server" in RDS database for easier queries. Hence, even if your server gets corrupted, backup will be always available in RDS.

Thank You. :)

Amazon Elastic Block Storage (EBS)

Ashutosh Mallick — Mon, 24 Jan 2022 12:19:50 +0000

What is Instance storage?

Block-level storage volumes behave like physical hard drives.
An instance store provides temporary block-level storage for an Amazon EC2 instance.
An instance store is disk storage that is physically attached to the host computer for an EC2 instance, and therefore has the same lifespan as the instance. When the instance is terminated, you lose any data in the instance store.
Let's create two ec2 instances such as "Ticked-server" and "Unticked-server" based upon their delete on termination protection for instance storage is kept on or off. -But if we terminate instances we are likely to lose our instance volumes. -Now let's terminate both the instances. After termination we can see that the volume for unticked server remains as such. Thus even on termination we can have our data secured if we untick the "delete on termination section". Amazon Elastic Block Store (EBS)
EBS is a block storage system used to store persistent data.
Amazon EBS is suitable for EC2 instances by providing highly available block level storage volumes.
It has three types of volume, i.e. General Purpose (SSD), Provisioned IOPS (SSD), and Magnetic. These three volume types differ in performance, characteristics, and cost.

EBS General Purpose (SSD)

This volume type is suitable for small and medium workloads like Root disk EC2 volumes, small and medium database workloads, frequently logs accessing workloads, etc. By default, SSD supports 3 IOPS (Input Output Operations per Second)/GB means 1 GB volume will give 3 IOPS, and 10 GB volume will give 30 IOPS. Its storage capacity of one volume ranges from 1 GB to 1 TB. The cost of one volume is $0.10 per GB for one month.

Provisioned IOPS (SSD)

This volume type is suitable for the most demanding I/O intensive, transactional workloads. By default, IOPS SSD supports 30 IOPS/GB, means 10GB volume will give 300 IOPS. Its storage capacity of one volume ranges from 10GB to 1TB. The cost of one volume is $0.125 per GB for one month for provisioned storage.

EBS Magnetic Volumes

It was formerly known as standard volumes. This volume type is suitable for ideal workloads like infrequently accessing data, i.e. data backups for recovery, logs storage. Its storage capacity of one volume ranges from 10GB to 1TB. The cost of one volume is $0.05 per GB for one month for provisioned storage and $0. 05 per million I/O requests.

Amazon EBS Benefits

Reliable and secure storage: Each of the EBS volume will automatically respond to its Availability Zone to protect from component failure.
Secure: Amazon’s flexible access control policies allows to specify who can access which EBS volumes.
Higher performance: Amazon EBS uses SSD technology to deliver data results with consistent I/O performance of application.
Easy data backup: Data backup can be saved by taking point-in-time snapshots of Amazon EBS volumes.

Adding an EBS volume to an instance

Let's create an instance "Test-1" and connect it via SSH using Xshell.
In Xshell type "sudo su" to login using root user.
Now to add extra volume to the instance, create an EBS volume of size say(15GB) and the volume should be created in the same region as that of the instance.
Now attach the extra volume to your instance "Test-1". If you type "lsblk" on Xshell, you can see your 15GB vol. is addaed to your instance but it's not mounted yet. -To mount the Extra-vol, first make a directory in the root then change path to that directory and mount your volume there.
In order to mount your EBS volume, first format the ddisk "india" using command "mkfs.ext4 /dev/xvdf". After formatting the disk mount your vol. using the command "mount /dev/xvdf /india".
You can list the volumes attached to your disk by running "df -h" in Xshell. -Now create 100 files inside your folder using command "touch abc{1..100}
Now let's shutdown and restart the instance, we can observe that the particular volume that we mounted (15GB vol.) will be unmounted from the disk.
If we again mount the volume to our disk, we can't see our files listed. Hence, In order to make the changes permanent, we'll use Vi editor in fstab. It's for permanently mounting the volume to the disk.
To use the vi editor use the command "vi /etc/fstab". Enter i to insert. In the editor write " /dev/xvdf /india ext4 defaults 0 0". Then press esc and :wq to save the file.
now if we again restart our instance, we can see our 100 files listed in india folder as we made the changes permanent.

What is EC2 instance and Setup of EC2 instances.

Ashutosh Mallick — Tue, 18 Jan 2022 05:41:08 +0000

WHAT IS AN EC2 INSTANCE?

Amazon EC2 stands for Amazon Elastic Compute Cloud.
Cloud service providers like AWS, Azure and GCP etc. use hypervisors in their physical servers to virtualize it.
EC2 instances are virtual machines (VMs) using those Hypervisors.
EC2 provides scalable computing capacity in the AWS Cloud and eliminates your need to invest in hardware up front, so you can develop and deploy applications faster. -EC2 helps to launch as many virtual servers as you need, configure security and networking, and manage storage.
EC2 provides a wide range of instance types optimized to fit different use cases.
Instances are virtual servers and have varying combinations of CPUs, memory storage and networking capacity.

FEATURES OF EC2 INSTANCE

• Virtual computing environments, known as instances AND Preconfigured templates for your instances, known as Amazon Machine Images Or AMIs.
• Various configurations of CPU, memory, storage, and networking capacity for your instances, known as instance types.
• Secure login information for your instances using key pairs (AWS stores the public key, and you store the private key in a secure place)
• Storage volumes for temporary data that's deleted when you stop, hibernate, or terminate your instance, known as instance store volumes.
• Persistent storage volumes for your data using Amazon Elastic Block Store (Amazon EBS), known as Amazon EBS volumes.
• Multiple physical locations for your resources, such as instances and Amazon EBS volumes, known as Regions and Availability Zones.
• A firewall that enables you to specify the protocols, ports, and source IP ranges that can reach your instances using security groups.
• Metadata, known as tags, that you can create and assign to your Amazon EC2 resources.

SETUP OF EC2 INSTANCE

Step 1: Sign-in to AWS account and open IAM console by using the following link: https://console.aws.amazon.com/iam/.

Step-2: In the navigation panel select EC2 service. On EC2 dashboard select instances from the left panel then click on launch instance.

-Step-3: Select an AMI from the list of various AMIs like( Amazon AMI, Linux AMI, Red hat AMI, Windows Server AMI etc.), let's select Amazon AMI.

Step-4: Select an Instance type from the list of instances, let's use "t2.micro" as it is eligible for free tier.

-Step-5: Configure instance details such as no. of instances you want to run or specify VPC network or Subnet. Here I've kept the specifications as default.

Step-6: Add storage for your instance as per your requirement or you can select the default root volume. -Step-7: Add tags for your instance. A tag consists of a case-sensitive key-value pair. you could define a tag with key=Name and value = Webserver .A copy of a tag can be applied to volumes, instances or both. -Step-8: Configure security groups. security group is a set of firewall rules that control the traffic for your instance. you can add rules to allow specific traffic to your instance. If you want to set up a web server and allow Internet traffic to reach your instance, add rules that allow unrestricted access to the HTTP and HTTPS ports. You can create a new security group or select from an existing one.

-Step-9: Review Instance Launch.
You can Cancel Previous Launch, review your instance launch details, go back to edit changes for each section. Click Launch to assign a key pair to your instance and complete the launch process. Don't forget to download your key pair file.

-Step-10: If your instance is successfully launched you can see Green check mark with instance state as "Running" and status as "2/2 checks passed" respectively.

Step-11: You can connect your instance using public DNS of your instance via SSH client using command prompt or Xshell or using PUTTY/PUTTYGEN. Here I've shown how to connect your instance using Xshell. First Install XSHELL from this link: https://www.netsarang.com/en/free-for-home-school/
paste your Public DNS in Xshell prompt and click accept. Then select Browse and select your private key pair file and open it. Your instance will now be connected via SSH.

We can also perform the above steps for an windows server instance using RDP client.

-After launching the instance select RDP client and download the RDP file. Then generate password and decrypt the password.

-Now open the RDP file and login using the decrypted password. When you connect you will be redirected to the remote desktop server and you can can connect your instance remotely.

Finally terminate the instance you have created otherwise you'll end up with bills from AWS.

Thank you :)