DEV Community: Piyush Chaudhari

Deploying Sample Docker Container to AWS Elastic Beanstalk (PAAS)

Piyush Chaudhari — Mon, 27 Mar 2023 12:22:49 +0000

Elastic Beanstalk is an AWS service that leverages Amazon EC2 and S3 and deploys, manages and scales your web applications for you. It uses managed containers that support Node.js, Java, Ruby, Docker and more. There are multiple ways to deploy applications using Elastic Beanstalk. You can deploy using the management console, the CLI, or the API. The amazing thing about Elastic Beanstalk is that once you upload your application, it takes care of things like load balancing, health monitoring, scaling, and more. Check out more information on Elastic Beanstalk
Overall, Elastic Beanstalk builds your docker image using your uploaded source code, and then it is deployed to EC2 instances running in an Elastic Beanstalk environment.

In this demonstration, we will:

Create an IAM User with sufficient privileges to it to manage AWS Elastic Beanstalk environment and generate Access keys, Secret keys. These keys will help us to configure AWS CLI. Here, I have created IAM User named "eb-demo-piyush". IMAGE
Deploy our Docker container with Elastic Beanstalk using a single container configuration (run 1 container per EC2 instance), using the Elastic Beanstalk CLI via AWS Cloud Shell because it is prebuilt with EB CLI along with AWS CLI. Check out more information on AWS Cloud Shell
Use a Dockerfile for our configuration for simplicity.
I will be using eu-central-1 as my AWS region.

So, Let’s start!

Step 1: Lets check the EB CLI version and lets start configuring the AWS CLI via AWS Cloud shell to talk to respective AWS services via APIs

I attached AdministratorAccess-AWSElasticBeanstalk AWS managed policy to my IAM User "eb-demo-piyush".

Lets check AWS CLI version and configure it with generated access and secret keys of my IAM User "eb-demo-piyush".

Step 2: In order to deploy a docker container in AWS Elastic Beanstalk we need an application and dockerfile for that application. So, here we will create a simple application and its dockerfile.

Paste the below code and save the file.

<!DOCTYPE html>
<html>
<body>
<center>
<h1> <font color="GREEN"> Don't Stop Trying, Until You Succeed in Doing!!! </font> </h1>
</center>
</body>
</html>

Now we will create a dockerfile of the application.
Paste the below content in Dockerfile.

FROM nginx
ADD eb-app-piyush.html /usr/share/nginx/html/
EXPOSE 80

Step 3: Initialize your Elastic Beanstalk Environment. By initializing Elastic Beanstalk knows about what type of application you are about to deploy here.

It will ask for an Application name.
It will give options for selecting a platform branch. Here default is option 1 so, let’s move to the default one.
Now it will ask if you want to set up a SSH for your instance. I say No to it.

Initialization is done now. On AWS Management console select Elastic Beanstalk. You will see the new Application created. But, when you open it, no environment exists.

Step 4: Lets create the environment for the application created.

Enter their environment name as per your wish.

Select the type of load balancer. I will go with the default one.

I will say No to Spot Fleet requests.

Now the application version will be uploaded to S3 and it will start the process of creating the Environment.

This process of createEnvironment will take at least 2-3 minutes. You will see here that resources are being created like target groups, security groups etc.

After some time the environment got created.

You can recheck it on Elastic Beanstalk. It is successfully created now.

You will find one EC2 instance created.

Lets check S3 Bucket which got created.

You can check the status of the application by using the following command.

Now, since the status of Elastic Beanstalk environment is healthy, if you want, we can browse the Application Load Balancer URL.

Bingo..!!! We have successfully deployed a Sample docker container based app to AWS Elastic Beanstalk (PAAS) !!!

Its time to wrap up..

OK, folks that’s it for this post. Have a nice day guys…… Stay tuned…..!!!!!

Don’t forget to like & share this post on social networks!!! I will keep on updating this blog. Please do follow me on "LinkedIn" & my other blogs -
cpiyush151 - Wordpress
cpiyush151 - Hashnode

Deploy an AWS EKS Cluster using Terraform (IaC)

Piyush Chaudhari — Mon, 20 Mar 2023 09:54:01 +0000

In this blog I’ll explain you about AWS EKS (Elastic Kubernetes Service) and how to deploy EKS cluster on AWS using Terraform.

What is AWS EKS?

Amazon Elastic Kubernetes Service(EKS) is a managed service that you can use to run Kubernetes on AWS without needing to install, operate, and maintain your own Kubernetes Control Plane/Nodes. AWS EKS is a managed AWS Kubernetes service that scales, manages and deploys containerized applications. It typically runs in the Amazon public cloud, but can also be deployed on premises.

The Kubernetes management infrastructure of Amazon EKS runs across multiple Availability Zones (AZ). AWS EKS helps you provide highly available and secure clusters and automates key tasks such as patching, node provisioning, and updates.

How Does AWS EKS Work?

AWS EKS Clusters are composed of the following components:

Control Plane: Composed of 3 master nodes, each running in a different AZ to ensure High-Availability.
Worker Nodes: Run on Amazon EC2 instances located in a VPC, which is not managed by AWS. You can control and configure the VPC allocated for worker nodes. You can use a SSH to give your existing automation access or to provision worker nodes.

There are 2 main deployment options, you can deploy one cluster for each environment/application. Or alternatively, you can define IAM security policies and Kubernetes namespaces to deploy one cluster for multiple applications/environments.

And for restricting the traffic between control-plane and your cluster, EKS also provides support of Amazon VPC network policies. Only authorized clusters and accounts, defined by Kubernetes role-based access control (RBAC), can view or communicate with control plane components.

You can read more about AWS EKS from here

What is Terraform?

Terraform is a free and open-source infrastructure as code (IaC) that can help to automate the deployment, configuration, and management of the remote servers. Terraform can manage both existing service providers and custom in-house solutions.

You can read more about Terraform from here

Now, I’m going to create an EKS Cluster with the help of Terraform (IaC).

Prerequisites

An AWS Account
Basic Knowledge of AWS Cloud, Terraform & Kubernetes

Now, let’s start creating terraform code files for our AWS EKS based Kubernetes cluster.

Step-1 Start with Creating Terraform Files

Here, I will be using Visual Studio Code on my local machine. I have already installed Terraform and authenticated with necessary IAM user with sufficient privileges to interact with my AWS account programmatically.

Create vars.tf file and add the below content in it:



variable "access_key" {
  default = "<YOUR-AWS-ACCESS-KEY>"
}
variable "secret_key" {
    default = "<YOUR-AWS-SECRET-KEY>"
}

Kindly replace the necessary AWS Access keys, Secret keys according to your IAM user. Make sure you have sufficient privileges.

Create main.tf file and add the below content in it:



provider "aws" {
    region = "eu-central-1"
    version = ">= 3.40.0"    
    access_key = "${var.access_key}"
    secret_key = "${var.secret_key}"
}
data "aws_availability_zones" "azs" {
    state = "available"
}

Here, I am using "eu-central-1" region here but you can use any region as per the business requirement.

Create vpc.tf file and add the below content in it:



variable "region" {
    default = "eu-central-1"
}
data "aws_availability_zones" "available" {}
locals {
    cluster_name = "Piyush-EKS-Cluster"
}
module vpc {
    source = "terraform-aws-modules/vpc/aws"

    name = "Piyush-EKS-VPC"
    cidr = "10.0.0.0/16"

    azs = data.aws_availability_zones.available.names
    private_subnets = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
    public_subnets =  ["10.0.4.0/24", "10.0.5.0/24", "10.0.6.0/24"]

    enable_nat_gateway = true
    single_nat_gateway = true

  enable_dns_hostnames= true
tags = {
    "Name" = "Piyush-EKS-VPC"
}
public_subnet_tags = {
    "Name" = "EKS-Public-Subnet"
}
private_subnet_tags = {
    "Name" = "EKS-Private-Subnet"
}
}

Let’s understand this file.
I am using the AWS VPC (Virtual Private Cloud) module for the VPC creation.
Once you run the above code it will create an AWS VPC named Piyush-EKS-VPC having 10.0.0.0/16 as a CIDR range in the eu-central-1 region.
This AWS VPC has 3 Private [10.0.1.0/24, 10.0.2.0/24, 10.0.3.0/24] & 3 Public [10.0.4.0/24, 10.0.5.0/24, 10.0.6.0/24] subnets.
I have also enabled the NAT Gateway & DNS HOSTNAME in our VPC.
And, data aws_availability_zones and azs will provide the list of the Availability zone for the eu-central-1 region.

Create sg.tf for AWS Security Group and add the below content in it:



resource "aws_security_group" "worker_group_one" {
    name_prefix = "worker_group_one"
    vpc_id = module.vpc.vpc_id
ingress {
        from_port = 22
        to_port = 22
        protocol = "tcp"
cidr_blocks = [
            "10.0.0.0/8"
        ]
    }
}
resource "aws_security_group" "worker_group_two" {
    name_prefix = "worker_group_two"
    vpc_id = module.vpc.vpc_id

    ingress {
        from_port = 22
        to_port = 22
        protocol = "tcp"
cidr_blocks = [
            "10.0.0.0/8"
        ]
    }
}
resource "aws_security_group" "all_worker_management" {
    name_prefix = "all_worker_management"
    vpc_id = module.vpc.vpc_id
ingress {
        from_port = 22
        to_port = 22
        protocol = "tcp"
cidr_blocks = [
            "10.0.0.0/8"
        ]
    }
}

Now here,
I am creating 2 Security Groups for 2 Worker Nodes Groups
Port 22 is open for SSH Connections but I’ve restricted access for 10.0.0.0/8 CIDR only.

Create eks.tf file EKS-Cluster and add the below content in it:



module "eks"{
    source = "terraform-aws-modules/eks/aws"
    version = "17.18.0"
    cluster_name = local.cluster_name
    cluster_version = "1.23"
    subnets = module.vpc.private_subnets
tags = {
        Name = "Piyush-EKS-Cluster"
    }
vpc_id = module.vpc.vpc_id
    workers_group_defaults = {
        root_volume_type = "gp3"
    }
worker_groups = [
        {
            name = "Worker-Group-1"
            instance_type = "t2.medium"
            asg_desired_capacity = 2
            additional_security_group_ids = [aws_security_group.worker_group_one.id]
        },
        {
            name = "Worker-Group-2"
            instance_type = "t2.medium"
            asg_desired_capacity = 1
            additional_security_group_ids = [aws_security_group.worker_group_two.id]
        },
    ]
}

data "aws_eks_cluster" "cluster" {
    name = module.eks.cluster_id
}
data "aws_eks_cluster_auth" "cluster" {
    name = module.eks.cluster_id
}

Here,
For EKS Cluster creation, I have used Terraform AWS EKS Module.
This will create 2 worker groups (worker_group_one & worker_group_two) with the desired capacity of 3 instances of type t2.medium.

Create kubernetes.tf file for the Kubernetes Provider and add the below content in it:



provider "kubernetes" {

    host = data.aws_eks_cluster.cluster.endpoint
    token = data.aws_eks_cluster_auth.cluster.token
    cluster_ca_certificate = base64encode(data.aws_eks_cluster.cluster.certificate_authority.0.data)
}

From the above code you can ensure that we’re using the very recently created EKS cluster as the host and we’re using token for the authentication and cluster_ca_certificate for the CA certificate.

Create output.tf file for the outputs:



output "cluster_id" {
    value = module.eks.cluster_id
}
output "cluster_endpoint" {
    value = module.eks.cluster_endpoint
}

Now, we are done with writing all the Terraform files.

Step-2: Initialize Directory with Terraform

I am going to run terraform init inside the working directory and then it’ll download all the necessary providers and all the other required modules. Run the following command in our VSCode:



terraform init

Step-3: Create Terraform Plan

Run terraform plan command in the working directory and it’ll give you the execution plan.

Now, let’s check the plan first and make sure that everything that we’ve written is what plan has suggested. We can redirect the output to a text file as well. :-)

Step-4: Create EKS Cluster using Terraform Command

Run terraform apply command and it will create the entire Kubernetes Cluster on AWS i:e; AWS EKS cluster.

After running this command terraform has created the below resources in my AWS account :

IAM Role
VPC
NAT Gateway
Security Group
Route Table
Public-Private Subnets
EKS Cluster

Step-5 Check EKS Cluster on AWS

Now, I’m gonna log-in into my AWS Account to verify all the resources.

VPC

Subnets

NAT Gateway

Route Tables

Security Groups

AWS EKS Cluster

EC2 Instances

Bingo!!!! Our EKS cluster is up & running now.
We have successfully provisioned an AWS EKS Cluster using Terraform (IaC).
Now you can play around it and make some changes and then modify it accordingly.

OK, folks that’s it for this post. Have a nice day guys…… Stay tuned…..!!!!!

Provisioning a Kubernetes Cluster Using Rancher in AWS EC2

Piyush Chaudhari — Thu, 16 Mar 2023 10:23:41 +0000

In this blog, I will be demonstrating how to use a container tool to create a gossip-based Kubernetes cluster using Rancher (RKE).

The rancher is an enterprise open source tool much like Kubernetes and Swarm container orchestration and it’s very simple to use.

Two steps we will be performing:

Installing Rancher
Creating a k8s cluster using rancher

Installing Rancher

We have two options available to install Rancher:

Single node installation
High availability installation

Single Node Installation: Install rancher in single Linux node; this is for development and testing purposes. Click here for more details on Rancher single node installation.

High Availability Installation: Installing and configuring Rancher on a cluster mode for production mode is recommended by Rancher. Click here for more details on Rancher high availability installation.

Here in this article, we will perform a single node installation.

Launching an EC2 Instance and Install Docker :-

Here, I have already launched an EC2 instanced based on the Ubuntu Server 22.04 LTS (HVM) AMI.
I have created a security group for this Rancher EC2 instance. I opened the necessary ports according to the official documentation. Click here

Now, I will start installing Docker on this.

Step 1: Update system repositories

sudo apt update

sudo apt upgrade

Step 2: Install required dependencies
After updating the system packages, next step is to install required dependencies for Docker:

sudo apt install lsb-release ca-certificates apt-transport-https software-properties-common -y

Step 3: Adding Docker repository to system sources
When Docker repository is added to the system sources, it makes the Docker installation easier and provides faster updates.
To add the Docker repository to the system sources, first, import the Docker GPG key required for connecting to the Docker repository:

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

Then, execute the following command for adding the Docker repository to your Ubuntu 22.04 system sources list:

echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

Step 4: Update system packages
After adding Docker repository to the system sources, again update the system packages:

sudo apt update

Step 5: Install Docker on Ubuntu 22.04
If you have carefully followed the previously given steps, then at this point, your Ubuntu 22.04 system is all ready for the Docker installation:

sudo apt install docker-ce

Note that we are utilizing the “docker-ce” package instead of “docker-ie” as it is supported by the official Docker repository:

Enter “y” to permit the Docker installation to continue:

The below-given error-free output indicates that Docker is successfully installed on our Ubuntu 22.04 system:

Step 6: Verify Docker status
Now, execute the below-given “systemctl” command to verify if the Docker is currently active or not on your system:

sudo systemctl status docker

Executing the Docker Command Without Sudo (Optional)

By default, the docker command can only be run the root user or by a user in the docker group, which is automatically created during Docker’s installation process.
If you want to avoid typing sudo whenever you run the docker command, add your username to the docker group:

sudo usermod -aG docker ${USER}

Here, I will add my default user ubuntu to docker group.

Bingo!! our installation of Docker is complete.

Installing Rancher on a Single Node Using Docker :-

Rancher can be installed by running a single Docker container.
In this installation scenario, we already installed Docker on a single Linux host EC2 instance, and now we will deploy Rancher on our host using a single Docker container.
When the Rancher server is deployed in the Docker container, a local Kubernetes cluster is installed within the container for Rancher to use. Because many features of Rancher run as deployments, and privileged mode is required to run containers within containers, you will need to install Rancher with the --privileged option.
Let's install Rancher using the self-signed certificate that it generates. This installation option omits the hassle of generating a certificate yourself.

docker run -d --restart=unless-stopped -p 80:80 -p 443:443 -v /opt/rancher:/var/lib/rancher --privileged rancher/rancher:latest

Note: In above command, we are adding -v option to persist the data.

Make sure if our container is running. Let's fire below command.

docker ps -a

After the container is up and running, you can access the UI on “https” and the first screen will ask you to set password.

Lets perform the steps to retrieve the Bootstrap password to get started.

We can now set the new password. Default user is "admin".

After setting password you will see below page as shown below:

Bingo!! our Rancher setup is ready..!!!

Creating a k8s cluster using rancher

Once Rancher is up and running, it makes the deployment and management of Kubernetes clusters quite easy.

Before you start with this, make sure, that you meet these requirements:

The host on which you run Rancher needs to communicate with all instances you deploy on EC2, in both directions. If you have Rancher running locally this will only work if the EC2 instances will be able to reach your local Rancher installation.
You need to setup the correct IAM policies and groups. If you don’t get this right you will not be able to deploy the cluster.

Because this is the most important point, lets start with the IAM user & policies. I created a new IAM user named "piyush" which I will be using for deploying the cluster through Rancher.
Additionally, I have generated the Access Key and Secret Key that will be used to create the instances.

I’ve created three IAM policies:

piyush-rancher-controlplane-policy: This is the policy that will be used for the control plane
piyush-rancher-etcd-worker-policy: This is the policy that will be used for the etcd and worker nodes
piyush-rancher-passrole-policy: This is the policy that will be attached to the AWS user that will be registered in Rancher with the cloud credentials

Here is the piyush-rancher-controlplane-policy (replace [YOUR_AWS_ACCOUNT_ID] with your AWS account ID):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "ec2:AttachVolume",
                "ec2:AuthorizeSecurityGroupIngress",
                "ec2:DescribeInstances",
                "autoscaling:DescribeLaunchConfigurations",
                "ec2:DescribeRegions",
                "elasticloadbalancing:DescribeLoadBalancerPolicyTypes",
                "elasticloadbalancing:SetWebAcl",
                "elasticloadbalancing:DescribeLoadBalancers",
                "ec2:DeleteVolume",
                "elasticloadbalancing:DescribeListeners",
                "autoscaling:DescribeAutoScalingGroups",
                "ec2:CreateRoute",
                "ec2:CreateSecurityGroup",
                "ec2:DescribeVolumes",
                "elasticloadbalancing:DescribeLoadBalancerPolicies",
                "kms:DescribeKey",
                "elasticloadbalancing:DescribeListenerCertificates",
                "elasticloadbalancing:DescribeInstanceHealth",
                "ec2:ModifyInstanceAttribute",
                "ec2:DescribeRouteTables",
                "elasticloadbalancing:DescribeSSLPolicies",
                "ec2:DetachVolume",
                "ec2:ModifyVolume",
                "ec2:CreateTags",
                "autoscaling:DescribeTags",
                "ec2:DeleteRoute",
                "elasticloadbalancing:*",
                "ec2:DescribeSecurityGroups",
                "ec2:CreateVolume",
                "elasticloadbalancing:DescribeLoadBalancerAttributes",
                "ec2:RevokeSecurityGroupIngress",
                "iam:CreateServiceLinkedRole",
                "elasticloadbalancing:DescribeTargetGroupAttributes",
                "ec2:DescribeVpcs",
                "elasticloadbalancing:DescribeAccountLimits",
                "ec2:DeleteSecurityGroup",
                "elasticloadbalancing:DescribeTargetHealth",
                "elasticloadbalancing:DescribeTargetGroups",
                "elasticloadbalancing:DescribeRules",
                "ec2:DescribeSubnets"
            ],
            "Resource": "*"
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": "elasticloadbalancing:*",
            "Resource": "*"
        },
        {
            "Sid": "VisualEditor2",
            "Effect": "Allow",
            "Action": "elasticloadbalancing:*",
            "Resource": "arn:aws:elasticloadbalancing:*:[YOUR_AWS_ACCOUNT_ID]:loadbalancer/*"
        },
        {
            "Sid": "VisualEditor3",
            "Effect": "Allow",
            "Action": "elasticloadbalancing:*",
            "Resource": [
                "arn:aws:elasticloadbalancing:*:[YOUR_AWS_ACCOUNT_ID]:targetgroup/*/*",
                "arn:aws:elasticloadbalancing:*:[YOUR_AWS_ACCOUNT_ID]:listener-rule/app/*/*/*/*",
                "arn:aws:elasticloadbalancing:*:[YOUR_AWS_ACCOUNT_ID]:listener-rule/net/*/*/*/*",
                "arn:aws:elasticloadbalancing:*:[YOUR_AWS_ACCOUNT_ID]:listener/net/*/*/*",
                "arn:aws:elasticloadbalancing:*:[YOUR_AWS_ACCOUNT_ID]:listener/app/*/*/*",
                "arn:aws:elasticloadbalancing:*:[YOUR_AWS_ACCOUNT_ID]:loadbalancer/net/*/*",
                "arn:aws:elasticloadbalancing:*:[YOUR_AWS_ACCOUNT_ID]:loadbalancer/app/*/*"
            ]
        },
        {
            "Sid": "VisualEditor4",
            "Effect": "Allow",
            "Action": "elasticloadbalancing:*",
            "Resource": [
                "arn:aws:elasticloadbalancing:*:[YOUR_AWS_ACCOUNT_ID]:targetgroup/*/*",
                "arn:aws:elasticloadbalancing:*:[YOUR_AWS_ACCOUNT_ID]:listener-rule/app/*/*/*/*",
                "arn:aws:elasticloadbalancing:*:[YOUR_AWS_ACCOUNT_ID]:listener-rule/net/*/*/*/*",
                "arn:aws:elasticloadbalancing:*:[YOUR_AWS_ACCOUNT_ID]:listener/net/*/*/*",
                "arn:aws:elasticloadbalancing:*:[YOUR_AWS_ACCOUNT_ID]:listener/app/*/*/*",
                "arn:aws:elasticloadbalancing:*:[YOUR_AWS_ACCOUNT_ID]:loadbalancer/net/*/*",
                "arn:aws:elasticloadbalancing:*:[YOUR_AWS_ACCOUNT_ID]:loadbalancer/app/*/*"
            ]
        },
        {
            "Sid": "VisualEditor5",
            "Effect": "Allow",
            "Action": "elasticloadbalancing:*",
            "Resource": [
                "arn:aws:elasticloadbalancing:*:[YOUR_AWS_ACCOUNT_ID]:loadbalancer/app/*/*",
                "arn:aws:elasticloadbalancing:*:[YOUR_AWS_ACCOUNT_ID]:loadbalancer/net/*/*",
                "arn:aws:elasticloadbalancing:*:[YOUR_AWS_ACCOUNT_ID]:targetgroup/*/*",
                "arn:aws:elasticloadbalancing:*:[YOUR_AWS_ACCOUNT_ID]:listener-rule/app/*/*/*/*",
                "arn:aws:elasticloadbalancing:*:[YOUR_AWS_ACCOUNT_ID]:listener-rule/net/*/*/*/*",
                "arn:aws:elasticloadbalancing:*:[YOUR_AWS_ACCOUNT_ID]:listener/net/*/*/*",
                "arn:aws:elasticloadbalancing:*:[YOUR_AWS_ACCOUNT_ID]:listener/app/*/*/*"
            ]
        }
    ]
}

Here is the piyush-rancher-etcd-worker-policy (replace [YOUR_AWS_ACCOUNT_ID] with your AWS account ID):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "ec2:*",
            "Resource": "*"
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": "secretsmanager:*",
            "Resource": "arn:aws:secretsmanager:*:[YOUR_AWS_ACCOUNT_ID]:secret:*"
        }
    ]
}

Finally, here is the content of piyush-rancher-passrole-policy (here you need to reference the other two policies):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "ec2:ModifyInstanceMetadataOptions",
                "ec2:AuthorizeSecurityGroupIngress",
                "ec2:Describe*",
                "ec2:ImportKeyPair",
                "ec2:CreateKeyPair",
                "ec2:CreateSecurityGroup",
                "ec2:CreateTags",
                "eks:*",
                "ec2:DeleteKeyPair"
            ],
            "Resource": "*"
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": "ec2:RunInstances",
            "Resource": [
                "arn:aws:ec2:eu-central-1::image/ami-*",
                "arn:aws:ec2:eu-central-1:[YOUR_AWS_ACCOUNT_ID]:security-group/*",
                "arn:aws:ec2:eu-central-1:[YOUR_AWS_ACCOUNT_ID]:subnet/*",
                "arn:aws:ec2:eu-central-1:[YOUR_AWS_ACCOUNT_ID]:network-interface/*",
                "arn:aws:iam::[YOUR_AWS_ACCOUNT_ID]:role/piyush-rancher-controlpane-role",
                "arn:aws:iam::[YOUR_AWS_ACCOUNT_ID]:role/piyush-rancher-etcd-worker-role",
                "arn:aws:ec2:eu-central-1:[YOUR_AWS_ACCOUNT_ID]:instance/*",
                "arn:aws:ec2:eu-central-1:[YOUR_AWS_ACCOUNT_ID]:volume/*",
                "arn:aws:ec2:eu-central-1:[YOUR_AWS_ACCOUNT_ID]:placement-group/*",
                "arn:aws:ec2:eu-central-1:[YOUR_AWS_ACCOUNT_ID]:key-pair/*"
            ]
        },
        {
            "Sid": "VisualEditor2",
            "Effect": "Allow",
            "Action": [
                "ec2:RebootInstances",
                "ec2:TerminateInstances",
                "ec2:StartInstances",
                "ec2:StopInstances"
            ],
            "Resource": "arn:aws:ec2:eu-central-1:[YOUR_AWS_ACCOUNT_ID]:instance/*"
        },
        {
            "Sid": "VisualEditor3",
            "Effect": "Allow",
            "Action": "iam:PassRole",
            "Resource": [
                "arn:aws:iam::[YOUR_AWS_ACCOUNT_ID]:role/piyush-rancher-controlpane-role",
                "arn:aws:iam::[YOUR_AWS_ACCOUNT_ID]:role/piyush-rancher-etcd-worker-role"
            ]
        }
    ]
}

Once you have that ready, create two IAM roles with the same name as the policies you created above. This is required, because you need to specify those later when you setup the node templates in Rancher:
piyush-rancher-controlplane-role
piyush-rancher-etcd-worker-role

The final step for the permissions in AWS is to assign the last policy (piyush-rancher-passrole-policy) as a permission to the AWS IAM user "piyush" which I will be using for deploying the cluster:

First, you will set up your EC2 cloud credentials in Rancher. Then you will use your cloud credentials to create a node template, which Rancher will use to provision new nodes in EC2.

Then you will create an EC2 cluster in Rancher, and when configuring the new cluster, you will define node pools for it. Each node pool will have a Kubernetes role of etcd, controlplane, or worker. Rancher will install RKE Kubernetes on the new nodes, and it will set up each node with the Kubernetes role defined by the node pool.

The steps to create a cluster differ based on your Rancher version.

Create your cloud credentials
Create a node template with your cloud credentials and information from EC2
Create a cluster with node pools using the node template

1. Create your cloud credentials :
In the Rancher UI, Left panel contains Cluster Management.
Go to Cluster management and click Cloud Credentials.

Click Create Cloud Credential and select Amazon.

Enter a name for the cloud credential.
In the Region field, select the AWS region where your cluster nodes will be located.
Enter your AWS EC2 Access Key and Secret Key.
Click Create.

2. Create a node template with your cloud credentials and information from EC2

Creating a node template for EC2 will allow Rancher to provision new nodes in EC2. Node templates can be reused for other clusters.

In the Rancher UI, Cluster management, click Node Templates.
Click Add Template.

Fill out a node template for EC2. For Account Access, Select AWS region, Cloud credentials added previously. Click Next: Authenticate and configure nodes.

Select appropriate AZ & VPC. Click Next: Select a security group.

Lets choose Standard which will automatically create a security group for this demo. Click Next: Set Instance options.

This is the most important section: The AMI ID you see, is the latest Ubuntu 20.04 AMI. The user for that AMI is "ubuntu". If you want to go with a Debian, CentOS or whatever AMI you need to adjust those (The user for Debian would be "admin", for CentOS it would be "centos" ). The "IAM instance profile name" is the role you created above, and this is important. Here you see "piyush-rancher-controlplane-role" because this will be the node template for the control plane:

Rest, keep all defaults. Click Create. This will create a control plane node template in Rancher.

By following the same steps mentioned above, you will create 2 more node templates 1 for etcd node and worker node EC2. Now, I have already created those as shown below -

Now we are ready to deploy a brand new Kubernetes cluster on top of EC2:
On Clusters page, Click Create and select EC2.

Here you reference the node templates. Make sure you use the control pane template for the control pane, and the other templates for etcd and worker nodes:

Go with the default and select “AWS” as cloud provider:

Before you press “Create”, it is a good idea to log into your Rancher host and tail the logs of the Rancher container. If anything goes wrong it shows up there.

Once you started the cluster creation, you can also monitor the EC2 console and watch the EC2 instances coming up:

Check AWS Console for the progress of launching EC2 cluster -

Now, the cluster state is "Active" as shown -

The cluster is fully ready and you can drill into the cluster Explore section.

It is also possible to get into the Kubectl shell via Rancher UI. Lets initiate Kubectl shell via UI.

kubectl get all

Bingo!!!! We are now able to manage our k8s cluster based on EC2 instances provisioned from Rancher UI!!!!!!!!!!! :-)

OK, folks that’s it for this post. Have a nice day guys…… Stay tuned…..!!!!!