Imagine a world where deploying and managing Kubernetes clusters in the cloud isn't a complex task but a seamless and efficient experience. This is where AWS Elastic Kubernetes Service (EKS) comes into play, simplifying the lives of developers and cloud architects. By leveraging the powerful combination of EKS and Terraform, you not only automate infrastructure deployment but also ensure consistency and scalability with just a few commands.
This article will guide you step by step through the process of how to deploy a Kubernetes cluster in AWS using Terraform, we will have a product and order API example to show you the results!
We will cover:
- What is AWS EKS?
- A guide to deploy a cluster using Terraform
What is AWS EKS?
Amazon Elastic Kubernetes Service (Amazon EKS) is a managed service that simplifies running Kubernetes on AWS without needing to install, operate, and maintain your own Kubernetes control plane. Some key features that makes great AWS EKS are:
- Secure Networking and Authentication: Integrates with AWS networking and security services, including AWS Identity and Access Management (IAM).
- Easy Cluster Scaling: Supports horizontal pod autoscaling and cluster autoscaling based on workload demand.
- Managed Kubernetes Experience: Allows you to manage clusters using eksctl, AWS Management Console, AWS CLI, API, kubectl, and Terraform.
- High Availability: Provides high availability for your control plane across multiple Availability Zones.
- Integration with AWS Services: Seamlessly integrates with other AWS services for a comprehensive platform to deploy and manage containerized applications.
Guide to deploy a cluster using Terraform
Using Terraform with AWS EKS simplifies provisioning, configuration, and management of Kubernetes clusters. These are the steps to deploy a cluster using terraform:
- Create an AWS account: If you don’t have one you can sign up here Cloud Computing Services - Amazon Web Services (AWS)
- Install Terraform: Depending on the operating system you have, you can refer to this guide to install it. Install Terraform | Terraform | HashiCorp Developer
- Install AWS CLI: Tool to manage your AWS services and resources from command line,depending on the operating system you have, you can refer to this guide to install it. Install or update to the latest version of the AWS CLI - AWS Command Line Interface (amazon.com)
- Install Kubectl: Tool for interacting with Kubernetes clusters from command line, depending on the operating system you have, you can refer to this guide to install it. Install Tools | Kubernetes
- Configure AWS CLI: You will have to configure your AWS CLI using the
aws configure
command with access credentials to AWS Account, you will have to provide Access Key ID, Secret Access Key and Session Token (you can find those in AWS Portal).
- Clone Repository: You can clone this repository Citrux-Systems/aws-eks-terraform-demo , it will have the setup for EKS cluster and an example with app implementation.
- Configure set up: Once you have cloned the repository, you can modify the region, cluster name and namespace where you need to deploy in variables.tf file, for our demo we used
'us-west-2'
,“citrux-demo-eks-${random_string.suffix.result}'
and“ecommerce'
variable 'region' {
description = 'AWS region'
type = string
default = 'us-west-2'
}
resource 'random_string' 'suffix' {
length = 8
special = false
}
data 'aws_availability_zones' 'available' {}
locals {
name = 'citrux-demo-eks-${random_string.suffix.result}'
region = var.region
cluster_version = '1.30'
instance_types = ['t3.medium'] # can be multiple, comma separated
vpc_cidr = '10.0.0.0/16'
azs = slice(data.aws_availability_zones.available.names, 0, 3)
tags = {
Blueprint = local.name
GitHubRepo = 'github.com/aws-ia/terraform-aws-eks-blueprints'
}
namespace = 'ecommerce'
}
Terraform files:
variables.tf : In this file you’ll find the configuration of all variables you need to use. Here yoou can customize the cluster name, region, EC2 instance type for nodes (more information here), tags and Kubernetes namespace (change to the namespace your manifest files look for deploy the app).
variable 'region' {
description = 'AWS region'
type = string
default = 'us-west-2'
}
resource 'random_string' 'suffix' {
length = 8
special = false
}
//fetches a list of available Availability Zones (AZs) in the specified AWS region.
data 'aws_availability_zones' 'available' {}
locals {
name = 'citrux-demo-eks-${random_string.suffix.result}'
region = var.region
cluster_version = '1.30'
instance_types = ['t3.medium'] # can be multiple, comma separated
vpc_cidr = '10.0.0.0/16' //The CIDR block for the VPC where the EKS cluster will be deployed
azs = slice(data.aws_availability_zones.available.names, 0, 3)
tags = {
Blueprint = local.name
GitHubRepo = 'github.com/aws-ia/terraform-aws-eks-blueprints'
}
namespace = 'ecommerce' //represents the Kubernetes namespace where resources will be deployed.
}
providers.tf : In this file you’ll find a provider configuration where allows Terraform to interact with a specific cloud provider in the region you previously define.
provider 'aws' {
region = local.region
}
terraform.tf : In this file you’ll find a Terraform configuration with all the required providers and versions needed to install and use for managing resources
terraform {
required_providers {
aws = {
source = 'hashicorp/aws'
version = '~> 5.47.0'
}
random = {
source = 'hashicorp/random'
version = '~> 3.6.1'
}
nullres = {
source = 'hashicorp/null'
version = '>= 3.1'
}
kubernetes = {
source = 'hashicorp/kubernetes'
version = '>= 2.20'
}
helm = {
source = 'hashicorp/helm'
version = '>= 2.9'
}
}
}
main.tf : In this file you’ll find the definition and resources configuration in Terraform. We are using the module pattern. In our case we have 2 modules: VPC and EKS. In the Main.tf you need to mapped the modules and pass the value for the variables the modules are expecting.
module 'vpc' {
source = './modules/vpc'
name = local.name
vpc_cidr = local.vpc_cidr
azs = local.azs
tags = local.tags
}
module 'eks' {
source = './modules/eks'
region = var.region
name = local.name
cluster_version = local.cluster_version
instance_types = local.instance_types
vpc_id = module.vpc.vpc_id
private_subnets = module.vpc.private_subnets
tags = local.tags
namespace = local.namespace
}
VPC Module:
variable.tf : You’ll find the variables definition for the VPC module.
variable 'name' {
description = 'name for the VPC'
type = string
}
//This variable specifies the CIDR block (IP address range) for the VPC
variable 'vpc_cidr' {
description = 'CIDR block for the VPC'
type = string
}
//This variable is used to specify the Availability Zones (AZs) in which the VPC resources will be distributed
variable 'azs' {
description = 'Availability zones'
type = list(string)
}
//This variable is used to specify a set of tags (key-value pairs) that will be applied to the resources created within the VPC
variable 'tags' {
description = 'Tags to apply to resources'
type = map(string)
}
main.tf : You’ll find a terraform configuration that creates a Virtual Private Cloud (VPC) using terraform modules with public and private subnets, a NAT Gateway, and configurations for running a Kubernetes cluster on Amazon EKS.
module 'vpc' {
source = 'terraform-aws-modules/vpc/aws'
version = '5.0.0'
name = var.name
cidr = var.vpc_cidr
azs = var.azs //Availability Zones (AZs) to be used for the VPC
public_subnets = [for k, v in var.azs : cidrsubnet(var.vpc_cidr, 8, k)]
private_subnets = [for k, v in var.azs : cidrsubnet(var.vpc_cidr, 8, k + 10)]
enable_nat_gateway = true
single_nat_gateway = true
enable_dns_hostnames = true
# Manage so we can name
manage_default_network_acl = true
default_network_acl_tags = { Name = '${var.name}-default' }
manage_default_route_table = true
default_route_table_tags = { Name = '${var.name}-default' }
manage_default_security_group = true
default_security_group_tags = { Name = '${var.name}-default' }
public_subnet_tags = {
'kubernetes.io/cluster/${var.name}' = 'shared'
'kubernetes.io/role/elb' = 1
}
private_subnet_tags = {
'kubernetes.io/cluster/${var.name}' = 'shared'
'kubernetes.io/role/internal-elb' = 1
}
tags = var.tags
}
output 'vpc_id' {
value = module.vpc.vpc_id
}
output 'private_subnets' {
value = module.vpc.private_subnets
}
EKS module:
variables.tf : You’ll find the variables definition for the EKS module.
variable 'region' {
description = 'AWS region'
type = string
}
variable 'name' {
description = 'name for the EKS cluster'
type = string
}
variable 'cluster_version' {
description = 'EKS cluster version'
type = string
}
//This variable is used to specify the EC2 instance types that will be used for the worker nodes in the EKS cluster.
variable 'instance_types' {
description = 'EC2 instance types'
type = list(string)
}
//This variable is used to specify a set of tags that will be applied to the resources created within the EKS cluster.
variable 'tags' {
description = 'Tags to apply to resources'
type = map(string)
}
variable 'vpc_id' {
description = 'VPC ID'
type = string
}
variable 'private_subnets' {
description = 'Private subnet IDs'
type = list(string)
}
variable 'namespace' {
description = 'Kubernetes namespace'
type = string
}
providers.tf : You’ll find the configuration for Kubernetes provider (which allows Terraform to interact with a Kubernetes cluster) and helm (which allows Terraform to manage Helm releases (packages of pre-configured Kubernetes resources) in a Kubernetes cluster).
# Kubernetes provider
# You should **not** schedule deployments and services in this workspace.
# This keeps workspaces modular (one for provision EKS, another for scheduling
# Kubernetes resources) as per best practices.
provider 'kubernetes' {
host = module.eks.cluster_endpoint
cluster_ca_certificate = base64decode(module.eks.cluster_certificate_authority_data)
exec {
api_version = 'client.authentication.k8s.io/v1beta1'
command = 'aws'
# This requires the awscli to be installed locally where Terraform is executed
args = ['eks', 'get-token', '--cluster-name', module.eks.cluster_name]
}
}
provider 'helm' {
kubernetes {
host = module.eks.cluster_endpoint
cluster_ca_certificate = base64decode(module.eks.cluster_certificate_authority_data)
exec {
api_version = 'client.authentication.k8s.io/v1beta1'
command = 'aws'
# This requires the awscli to be installed locally where Terraform is executed
args = ['eks', 'get-token', '--cluster-name', module.eks.cluster_name]
}
}
}
main.tf : You’ll find the configuration that creates an EKS cluster with the cluster name, Kubernetes version, VPC and subnet settings, node groups, and add-ons. It also uses the eks_blueprints_addons
(More information here) to allow the creation of load balancers and allow access from browser to the services we’ll deplloy. Finally, it updates the local kubeconfig file to allow the use of kubectl and creates a Kubernetes namespace for deploying resources.
module 'eks' {
source = 'terraform-aws-modules/eks/aws'
version = '19.15.1'
cluster_name = var.name
cluster_version = var.cluster_version
cluster_endpoint_public_access = true //allows public access to the EKS cluster's API server endpoint.
vpc_id = var.vpc_id
subnet_ids = var.private_subnets
control_plane_subnet_ids = var.private_subnets
# EKS Addons
cluster_addons = {
aws-ebs-csi-driver = {
most_recent = true
}
coredns = {}
kube-proxy = {}
vpc-cni = {}
}
eks_managed_node_group_defaults = {
# Needed by the aws-ebs-csi-driver
iam_role_additional_policies = {
AmazonEBSCSIDriverPolicy = 'arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy'
}
}
eks_managed_node_groups = {
one = {
node_group_name = 'node-group-1'
instance_types = var.instance_types
min_size = 1
max_size = 3
desired_size = 2
subnet_ids = var.private_subnets
}
two = {
node_group_name = 'node-group-2'
instance_types = var.instance_types
min_size = 1
max_size = 2
desired_size = 1
subnet_ids = var.private_subnets
}
}
tags = var.tags
}
//This module installs and configures various add-ons for the EKS cluster, such as the AWS Load Balancer Controller, Metrics Server, and AWS CloudWatch Metrics.
module 'eks_blueprints_addons' {
source = 'aws-ia/eks-blueprints-addons/aws'
version = '1.16.3'
cluster_name = module.eks.cluster_name
cluster_endpoint = module.eks.cluster_endpoint
cluster_version = module.eks.cluster_version
oidc_provider_arn = module.eks.oidc_provider_arn
# K8S Add-ons
enable_aws_load_balancer_controller = true
aws_load_balancer_controller = {
set = [
{
name = 'vpcId'
value = var.vpc_id
},
{
name = 'podDisruptionBudget.maxUnavailable'
value = 1
},
{
name = 'enableServiceMutatorWebhook'
value = 'false'
}
]
}
enable_metrics_server = true
enable_aws_cloudwatch_metrics = false
tags = var.tags
}
# To update local kubeconfig with new cluster details
resource 'null_resource' 'kubeconfig' {
depends_on = [module.eks_blueprints_addons]
provisioner 'local-exec' {
command = 'aws eks --region ${var.region} update-kubeconfig --name ${var.name}'
environment = {
AWS_CLUSTER_NAME = var.name
}
}
}
resource 'null_resource' 'create_namespace' {
depends_on = [null_resource.kubeconfig]
provisioner 'local-exec' {
command = 'kubectl create namespace ${var.namespace}'
}
}
- Run terraform: Now when is all set, you can create the resources using Terraform.
First, run this command to initialize Terraform:terraform init -upgrade
Then run these commands to add and apply the resources we need: terraform plan -out terraform.plan , terraform apply terraform.plan
After that, it will take about 10-15 minutes to get it done. It will create the resources in AWS and you will get your cluster information:
Then, you can validate the status in AWS Console:
Go to search bar and type 'Elastic Kubernetes'
And then you will see the cluster created and it’s status, it has to be 'Active'
- Connect with Kubectl: run the following command to allow kubectl modify your eks cluster so you can deploy containers has needed, you will have to give it your EKS information. Don’t forget to modify your region and cluster name.
aws eks --region <your-region> update-kubeconfig --name <cluster_name>
- Deploy containers with manifest files: now you can go to the raw-manifests folder to apply all manifest files in order to deploy containers with your application. Don’t forget that all .yaml files you should review it and modify according to your app: your namespace, load balancer name, services name and paths.
cd raw-manifests
kubectl apply -f ingress.yaml,products-service.yaml,products-deployment.yaml,orders-service.yaml,orders-deployment.yaml
It will appears something like this when the containers are deployed
ingress.networking.k8s.io/alb-ingress created
service/products-service created
deployment.apps/products created
service/orders-service created
deployment.apps/orders created
- Get endpoint to make http requests: Use the following command you’ll obtain the address for your application deployment:
kubectl get ingress -n ecommerce
NAME CLASS HOSTS ADDRESS PORTS AGE
alb-ingress alb * new-ecommerce-alb-406866228.us-west-2.elb.amazonaws.com 80 8m22s
Now, you just have to modify your address for what you want to see from your application. In our case, we have 3 endpoints, one for orders and one for products with paths: ‘v1/orders’ ,‘v1/products’ and ‘v1/orders/products’, and we’ll see the information we have in each one:
http://new-ecommerce-alb-406866228.us-west-2.elb.amazonaws.com/v1/orders
[
{
'id': '1',
'productId': '1a',
'orderFor': 'Herbert Kelvin Jr.',
'deliveryAddress': 'Asphalt Street',
'deliveryDate': '02/11/2023',
'deliveryType': 'STANDARD'
},
{
'id': '2',
'productId': '1b',
'orderFor': 'John Zulu Nunez',
'deliveryAddress': 'Beta Road',
'deliveryDate': '10/10/2023',
'deliveryType': 'FAST DELIVERY'
},
{
'id': '3',
'productId': '1c',
'orderFor': 'Lael Fanklin',
'deliveryAddress': 'Charlie Avenue',
'deliveryDate': '02/10/2023',
'deliveryType': 'STANDARD'
},
{
'id': '4',
'productId': '1d',
'orderFor': 'Candice Chipilli',
'deliveryAddress': 'Delta Downing View',
'deliveryDate': '22/09/2023',
'deliveryType': 'FAST DELIVERY'
},
{
'id': '5',
'productId': '1d',
'orderFor': 'Tedashii Tembo',
'deliveryAddress': 'Echo Complex',
'deliveryDate': '12/12/2023',
'deliveryType': 'FAST DELIVERY'
}
]
http://new-ecommerce-alb-406866228.us-west-2.elb.amazonaws.com/v1/products
[
{
'id': '1a',
'name': 'Hoodie'
},
{
'id': '1b',
'name': 'Sticker'
},
{
'id': '1c',
'name': 'Socks'
},
{
'id': '1d',
'name': 'T-Shirt'
},
{
'id': '1e',
'name': 'Beanie'
}
]
http://new-ecommerce-alb-406866228.us-west-2.elb.amazonaws.com/v1/orders/products
[
{
'id':'1',
'productId':'1a',
'orderFor':'Herbert Kelvin Jr.',
'deliveryAddress':'Asphalt Street',
'deliveryDate':'02/11/2023',
'deliveryType':'STANDARD',
'product':{
'id':'1a',
'name':'Hoodie'
}
},
{
'id':'2',
'productId':'1b',
'orderFor':'John Zulu Nunez',
'deliveryAddress':'Beta Road',
'deliveryDate':'10/10/2023',
'deliveryType':'FAST DELIVERY',
'product':{
'id':'1b',
'name':'Sticker'
}
},
{
'id':'3',
'productId':'1c',
'orderFor':'Lael Fanklin',
'deliveryAddress':'Charlie Avenue',
'deliveryDate':'02/10/2023',
'deliveryType':'STANDARD',
'product':{
'id':'1c',
'name':'Socks'
}
},
{
'id':'4',
'productId':'1d',
'orderFor':'Candice Chipilli',
'deliveryAddress':'Delta Downing View',
'deliveryDate':'22/09/2023',
'deliveryType':'FAST DELIVERY',
'product':{
'id':'1d',
'name':'T-Shirt'
}
},
{
'id':'5',
'productId':'1d',
'orderFor':'Tedashii Tembo',
'deliveryAddress':'Echo Complex',
'deliveryDate':'12/12/2023',
'deliveryType':'FAST DELIVERY',
'product':{
'id':'1d',
'name':'T-Shirt'
}
}
]
Notice how the last endpoint include products information from the products endpoint. This means the pods can communicate with each other, thanks to private subnet settings.
Conclusion
AWS Elastic Kubernetes Services is a great alternative to deploy kubernetes cluster in the cloud, reducing the weight of manage the control plane with the self-managed nodes. Nonetheless, the amount of required kubernetes knowledge needed is quite high, making it better for migrate running kubernetes or create really big application with a need for certain infraestructure management. In our study case, the application just retrieve information, this case is commonly solved with API calls, in this kind of cases using lambda functions and api-gateways for orders and products would be better for quickly development, less infraestructure knowledge and reduced costs.
Top comments (0)