Automating an AWS Study Environment with Rancher and Kubernetes using Terraform

João Vitor Freitas — Tue, 18 Jul 2023 16:11:49 +0000

In this article, we explore automating the creation of an AWS study environment with Rancher and Kubernetes using Terraform. Manual infrastructure setup can be time-consuming and error-prone. By leveraging Terraform's power as an infrastructure-as-code tool, we'll streamline the process, making it easier to deploy and manage AWS resources for studying Kubernetes and Rancher. The article will guide you through the steps, ensuring a successful deployment and helping you focus on learning and development tasks. Let's get started on your journey to cloud-native technologies!

Requirements:

AWS Account: To follow this guide and create a study environment on AWS, you will need an active AWS account. If you don't have an account, you can create one for free by visiting the following link: Create an AWS Account.
Configured AWS CLI Profile: You need to have the AWS CLI (Command Line Interface) installed and properly configured with your AWS account credentials. If you need assistance with AWS CLI configuration, refer to the official AWS documentation: Configuring the AWS CLI.
Installed Terraform: Ensure that you have Terraform installed on your machine. Terraform is an infrastructure-as-code tool used in this guide to automate the creation and management of AWS resources. To install Terraform, follow the instructions provided in the official documentation: Installing Terraform.

Once all requirements are completed, let's start our journey to create an environment.

What will we do?

Our flow will work with Terraform creating 4 instances (t3.medium) on AWS EC2. Three will be k8s nodes and the one as Rancher Server to manage the nodes.

Like the image below, look…

Starting Terraform code

First, let's create the main.tf file using the code below

terraform {
  required_version = "1.3.7"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "4.28.0"
    }
  }

  backend "local" {
    path = "terraform.tfstate"
  }
}

Running the terraform init command, you'll see that will be created a .terraform.lock.hcl file and .terraform folder.

Terraform uses the dependency lock file .terraform.lock.hcl to track and select provider versions.

The .terraform directory, Terraform uses to manage cached provider plugins and modules, record which workspace is currently active, and record the last known backend configuration in case it needs to migrate state on the next run. This directory is automatically managed by Terraform, and is created during initialization.

So, now let's create the variables.tf file to save some values to use in the future.

I'm using the us-east-2 region, but you can use the region you prefer

variable "aws_region" {
  type        = string
  description = "AWS region"
  default     = "us-east-2"
}

With the variable that we created, we can configure the aws provider in our main.tf. Your main.tf should look like this:

provider "aws" {
  region  = var.aws_region
  profile = "terraform-aws-pessoal"
}

! Don't forget to change the profile parameter to put your aws cli profile !

Rancher and K8s Module

To better organization we will create a folder to manage the files about the instances.

Create a modules folder in root path and create a folder rancher_k8s inside.

.
├── main.tf
├── variables.tf
└── modules/
    └── rancher_k8s

Now, let's create a file to save our variables. Create the variables.tf in rancher_k8s module

variable "aws_region" {
  type        = string
  description = "AWS region"
  default     = "us-east-2"
}

variable "ami_id" {
  description = "AWS instance ami id"
  default     = "ami-00149760ce42c967b"
}

variable "instance_type" {
  description = "AWS instance type"
  default     = "t3.medium"
}

variable "subnet_id" {
  description = "AWS subnet id"
  default     = "subnet-"
}

variable "vpc_id" {
  description = "AWS VCP id"
  default     = "vpc-"
}

variable "key_name" {
  type        = string
  description = "ssh key name"
  default     = "pem"
}

! Change the vpc_id to your vpc's id
! Change the subnet_id to your subnet's id
! Change the key_name to your ssh key registered on AWS

We will create a file to use for our tags. That's easier. Create a locals.tf in the module.

locals {
  common_tags = {
    ManagedBy = "Terraform"
  }
}

I usually use this pattern to know when the resource is managed with IaC (Infrastrcuture as Code) in projetcs.

Here, let's start creating the ec2.tf file to configure our instances.

We will start with k8s nodes.

resource "aws_instance" "k8s" {
  count                  = 3
  ami                    = var.ami_id
  instance_type          = var.instance_type
  subnet_id              = var.subnet_id
  key_name               = var.key_name
  ebs_optimized          = true

  root_block_device {
    volume_size = 30
    volume_type = "standard"
    tags = merge(
      local.common_tags,
      {
        Name = "Rancher K8s ${count.index + 1}"
      }
    )
  }

  tags = merge(
    local.common_tags,
    {
      Name = "Rancher K8s ${count.index + 1}"
    }
  )
}

And now the rancher server instance. It's looks like the above

resource "aws_instance" "rancher_server" {
  count                  = 1
  ami                    = var.ami_id
  instance_type          = var.instance_type
  subnet_id              = var.subnet_id
  key_name               = var.key_name
  ebs_optimized          = true

  root_block_device {
    volume_size = 30
    volume_type = "standard"
    tags = merge(
      local.common_tags,
      {
        Name = "Rancher Server"
      }
    )
  }

  tags = merge(
    local.common_tags,
    {
      Name = "Rancher Server"
    }
  )
}

Before the test, let's create the security group config

resource "aws_security_group" "rancher_k8s" {
  name        = "Rancher K8s"
  description = "Security group to Rancher K8s Instances"
  vpc_id      = var.vpc_id

  ingress {
    description = "SSH"
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    description = "All"
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port        = 0
    to_port          = 0
    protocol         = "-1"
    cidr_blocks      = ["0.0.0.0/0"]
    ipv6_cidr_blocks = ["::/0"]
  }

  tags = merge(
    local.common_tags,
    {
      Name = "rancher-k8s"
    }
  )
}

Now let's associate this security group to our instances. Your file should look like this:

resource "aws_instance" "k8s" {
  count                  = 3
  ami                    = var.ami_id
  instance_type          = var.instance_type
  subnet_id              = var.subnet_id
  vpc_security_group_ids = [aws_security_group.rancher_k8s.id]
  key_name               = var.key_name
  ebs_optimized          = true

  root_block_device {
    volume_size = 30
    volume_type = "standard"
    tags = merge(
      local.common_tags,
      {
        Name = "Rancher K8s ${count.index + 1}"
      }
    )
  }

  tags = merge(
    local.common_tags,
    {
      Name = "Rancher K8s ${count.index + 1}"
    }
  )
}

resource "aws_instance" "rancher_server" {
  count                  = 1
  ami                    = var.ami_id
  instance_type          = var.instance_type
  subnet_id              = var.subnet_id
  vpc_security_group_ids = [aws_security_group.rancher_k8s.id]
  key_name               = var.key_name
  ebs_optimized          = true

  root_block_device {
    volume_size = 30
    volume_type = "standard"
    tags = merge(
      local.common_tags,
      {
        Name = "Rancher Server"
      }
    )
  }

  tags = merge(
    local.common_tags,
    {
      Name = "Rancher Server"
    }
  )
}

resource "aws_security_group" "rancher_k8s" {
  name        = "Rancher K8s"
  description = "Security group to Rancher K8s Instances"
  vpc_id      = var.vpc_id

  ingress {
    description = "SSH"
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    description = "All"
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port        = 0
    to_port          = 0
    protocol         = "-1"
    cidr_blocks      = ["0.0.0.0/0"]
    ipv6_cidr_blocks = ["::/0"]
  }

  tags = merge(
    local.common_tags,
    {
      Name = "rancher-k8s"
    }
  )
}

Validating our code

Before to run the commands, we should add the rancher_k8s module on main.tf

terraform {
  required_version = "1.3.7"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "4.28.0"
    }
  }

  backend "local" {
    path = "terraform.tfstate"
  }
}

provider "aws" {
  region  = var.aws_region
  profile = "terraform-aws-pessoal"
}

module "rancher_k8s" {
  source = "./modules/rancher_k8s"
}

Let's running some commands to validate what we did.

The commands below should run in the root path;

The first is the command to start the terraform and dependencies. Run terraform init;

Now, let's format the code and fix the identation. Run terraform fmt -recursive.

Run terraform validate to terraform check the files.

In case there are not errors, we can run the terraform plan.

You should see something like this:

Plan: 5 to add, 0 to change, 0 to destroy.

If you agree with what you read in terraform plan, we can run terraform apply and create our instances on cloud.

Apply complete! Resources: 5 added, 0 changed, 0 destroyed.

If you open the AWS console to see your EC2 instances, you are able to see these instances we created.

Now, we can run the terraform destroy… What? Why?

Because we have some things to automate before :3

Destroy complete! Resources: 5 destroyed.

Scripts and automations

Let's create one more folder to save some scripts. Create 3 files.

.
├── main.tf
├── variables.tf
├── scripts/
│   ├── docker.sh
│   ├── kubctl.sh
│   └── rancher.sh
└── modules/
    └── rancher_k8s

#!/bin/bash

#Installing Docker and Docker Compose
sudo apt update
sudo apt install apt-transport-https ca-certificates curl software-properties-common -y
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable"
apt-cache policy docker-ce
sudo apt install docker-ce docker-ce-cli containerd.io -y
sudo groupadd docker
sudo usermod -aG docker ubuntu
newgrp docker
sudo apt install python3-pip -y
sudo pip install docker-compose

#!/bin/bash

#Installing Kubectl
sudo apt-get update && sudo apt-get install -y apt-transport-https gnupg2
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
echo 'deb https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee -a /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubectl

#!/bin/bash

docker run -d --name rancher --restart=unless-stopped -v /opt/rancher:/var/lib/rancher  -p 80:80 -p 443:443 rancher/rancher:v2.4.3

docker.sh: It's to install and configure docker and docker-compose
kubectl: To install kubectl
rancher.sh: To run a rancher server container

How will we use these scripts?

We will create a file to rendered this scripts and running automatically when the instance be created. That's great, isn't it?

Create the data.tf file in rancher_k8s module.

In this file we are creating a unique config file with these merged files to run as a sequence

data "template_cloudinit_config" "rancher_k8s_just_docker_install" {
  part {
    content_type = "text/x-shellscript"
    content      = data.template_file.docker_config.rendered
  }
}

data "template_cloudinit_config" "rancher_k8s_install" {
  part {
    content_type = "text/x-shellscript"
    content      = data.template_file.docker_config.rendered
  }
  part {
    content_type = "text/x-shellscript"
    content      = data.template_file.run_rancher.rendered
  }
  part {
    content_type = "text/x-shellscript"
    content      = data.template_file.kubectl_install.rendered
  }
}

data "template_file" "docker_config" {
  template = file("scripts/docker.sh")
}

data "template_file" "run_rancher" {
  template = file("scripts/rancher.sh")
}

data "template_file" "kubectl_install" {
  template = file("scripts/kubectl.sh")
}

And we need to add in ec2.tf file to talk to terraform use these scripts for our instances.

It's the user_data param. And I add the lifecycle to terraform understand that this param should be ignored when the resource will be updated.

resource "aws_instance" "k8s" {
  count                  = 3
  ami                    = var.ami_id
  instance_type          = var.instance_type
  subnet_id              = var.subnet_id
  vpc_security_group_ids = [aws_security_group.rancher_k8s.id]
  key_name               = var.key_name
  ebs_optimized          = true

  user_data = data.template_cloudinit_config.rancher_k8s_just_docker_install.rendered
  lifecycle {
    ignore_changes = [user_data]
  }
  root_block_device {
    volume_size = 30
    volume_type = "standard"
    tags = merge(
      local.common_tags,
      {
        Name = "Rancher K8s ${count.index + 1}"
      }
    )
  }

  tags = merge(
    local.common_tags,
    {
      Name = "Rancher K8s ${count.index + 1}"
    }
  )
}

resource "aws_instance" "rancher_server" {
  count                  = 1
  ami                    = var.ami_id
  instance_type          = var.instance_type
  subnet_id              = var.subnet_id
  vpc_security_group_ids = [aws_security_group.rancher_k8s.id]
  key_name               = var.key_name
  ebs_optimized          = true

  user_data = data.template_cloudinit_config.rancher_k8s_install.rendered
  lifecycle {
    ignore_changes = [user_data]
  }
  root_block_device {
    volume_size = 30
    volume_type = "standard"
    tags = merge(
      local.common_tags,
      {
        Name = "Rancher Server"
      }
    )
  }

  tags = merge(
    local.common_tags,
    {
      Name = "Rancher Server"
    }
  )
}

resource "aws_security_group" "rancher_k8s" {
  name        = "Rancher K8s"
  description = "Security group to Rancher K8s Instances"
  vpc_id      = var.vpc_id

  ingress {
    description = "SSH"
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    description = "All"
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port        = 0
    to_port          = 0
    protocol         = "-1"
    cidr_blocks      = ["0.0.0.0/0"]
    ipv6_cidr_blocks = ["::/0"]
  }

  tags = merge(
    local.common_tags,
    {
      Name = "rancher-k8s"
    }
  )
}

Outputs and finishing code

We will create two files about outputs, one inside the module and another one in root path.

In the output.tf inside the module, put this code:

output "k8s_security_group_ingress" {
  value = aws_security_group.rancher_k8s[*].ingress
}

output "k8s_aws_ec2_name" {
  value = aws_instance.k8s[*].tags.Name
}

output "k8s_aws_ec2_type" {
  value = aws_instance.k8s[*].instance_type
}

output "k8s_public_ip" {
  value = aws_instance.k8s[*].public_ip
}

output "k8s_security_group_all_tags" {
  value = aws_security_group.rancher_k8s[*].tags_all
}

output "rancher_aws_ec2_name" {
  value = aws_instance.rancher_server[*].tags.Name
}

output "rancher_aws_ec2_type" {
  value = aws_instance.rancher_server[*].instance_type
}

output "rancher_public_ip" {
  value = aws_instance.rancher_server[*].public_ip
}

And in output.tf file at root path, put that:

output "rancher_aws_ec2_name" {
  value = module.rancher_k8s.rancher_aws_ec2_name
}

output "rancher_aws_ec2_type" {
  value = module.rancher_k8s.rancher_aws_ec2_type
}

output "rancher_public_ip" {
  value = module.rancher_k8s.rancher_public_ip
}

output "k8s_aws_ec2_name" {
  value = module.rancher_k8s.k8s_aws_ec2_name
}

output "k8s_aws_ec2_type" {
  value = module.rancher_k8s.k8s_aws_ec2_type
}

output "k8s_public_ip" {
  value = module.rancher_k8s.k8s_public_ip
}

output "k8s_security_group" {
  value = module.rancher_k8s.k8s_security_group_all_tags
}

output "k8s_security_group_rules" {
  value = module.rancher_k8s.k8s_security_group_ingress
}

These files are used when you want to know some details about the resource you created without needing to go to the cloud console.

Let's Run!

Now, we can run the terraform apply command and see our infrastructure being created and our scripts installing the dependencies for rancher, k8s and docker working correctly.

After applying it you must see something like this:

With our outputs working correctly, you maybe want to connect with ssh in your instances to check if the scripts working rightly

! Don't forget to use your ssh key created on aws and used on our terraform code

Our instances were created

Your instances must be able to run docker commands and your rancher server must be able to run docker commands and kubectl commands.

Check if all your nodes are working right and if rancher container is running in your Rancher Server Instance.

Come on to Rancher Interface to setting your cluster

Open your rancher serve using the your Rancher Server's IP on Port 80

You must see that screen:

Here, you need to create a password to admin user.

Before that, you need to confirm your Rancher Server URL

So, let's create an our cluster. Click on the "Add cluster" button on top of screen

And select the "From existing nodes (Custom)" option

In next screen, only change this to "Disabled"

In the next step, we will choose all the option and add a name to our Node

Copy this command and paste on each node instance that you have changing only the name for each one. Like this:

After running, if you go to the rancher interface, you see your cluster is provisioning status

After some minutes, our nodes will be available

And you are able to see the cluster metrics

Destroying

To complete this flow, I'll destroy this infrastructure. But if you want continue with it to study more, you can.

But remember costs in cloud are on-demand, it means you will pay for the time you use of theses resources.

The good side of the Infrastructure as Code (IaC) is the ability to destroy and create the infrastructure when you need. So, saving time and money!

Conclusion

Here, we can see the potential of automation that the combined use of Rancher, Kubernetes and Terraform can offer in creating and managing a study environment in AWS, being efficiently and at scale.

It was evident how the integration of these technologies can significantly simplify the task of provisioning and managing cloud resources, allowing developers to focus more on application and service development rather than manual and repetitive tasks.

Also, the Infrastructure as Code (IaC) provides many benefits such as change traceability and the ability to easily reproduce the environment in different scenarios. This makes the learning and experimentation process more agile and secure, as it enables the creation of isolated environments for testing and evaluation without impacting the production environment.

As more organizations adopt the cloud and container-based architectures, the skills demonstrated in this article will become increasingly valuable in the job market. Therefore, this guide can serve as a valuable resource for students, developers, and professionals looking to enhance their skills in modern cloud environments.

In summary, the combination of Rancher, Kubernetes, and Terraform offers a powerful and effective way to automate and manage environments in AWS, opening doors to countless opportunities in the field of cloud computing and DevOps. Learning and applying these tools and concepts can lead to significant improvements in the efficiency, reliability, and scalability of IT solutions, both at a personal and corporate level.

Go ahead and give it a read - you won't regret it! Happy cloud adventures! 🚀🌟

DEV Community: João Vitor Freitas