DEV Community

Cover image for CI/CD Pipeline for Amazon ECS/FARGATE with Terraform
erozedguy
erozedguy

Posted on • Edited on

CI/CD Pipeline for Amazon ECS/FARGATE with Terraform

#terraform #aws #devops #cloud

DESCRIPTION

In this post I am going to explain how to build the infrastructure on AWS with Terraform to implement a CI / CD pipeline for ECS / Fargate.
The Architecture consists of a VPC with 2 public subnets in different Availability Zones. The desired tasks are 2 and each task is deployed on each public subnet with Fargate and each task belongs to the same ECS service.
An Application Load Balancer is used to balance the load between the two tasks.
In this case, the main goal is to implement a docker container that contains a simple HTTP server built with GOLANG. This HTTP server allows you to obtain the private IP of each task.
If I push new changes to the CodeCommit repository, CodePipeline detects those changes, triggers the pipeline and creates a new docker image and then deploys it to the ECS service to update the tasks.

ARCHITECTURE

aws cicd pipeline

RESOURCES

https://github.com/erozedguy/CICD-Pipeline-for-Amazon-ECS-Fargate

STEPS

STEP 01 - Create a IAM Role and CodeCommit Credentials

  • Create a Service Role for Elastic Container Service Task (
    Allows ECS tasks to call AWS services on your behalf.)
    taskRole

  • Attach AWSCodeCommitPowerUser Policy to my USER
    commitpolicy

  • Generate HTTPS Git credentials for AWS CodeCommit to clone, push, pull to the CodeCommit Repository

credentials

STEP 02: Terraform scripts to build the infrastructure

PROVIDERS 



terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.51"
    }
  }
}
provider "aws" {
  profile = "default"
  region  = "us-east-1"
}
Enter fullscreen mode Exit fullscreen mode

VPC

The vpc script has VPC, SUBNETS and INTERNET GATEWAY resources.



resource "aws_vpc" "ecs-vpc" {
  cidr_block = "${var.cidr}"

  tags = {
    Name = "ecs-vpc"
  }
}

# PUBLIC SUBNETS
resource "aws_subnet" "pub-subnets" {
  count                   = length(var.azs)
  vpc_id                  = "${aws_vpc.ecs-vpc.id}"
  availability_zone       = "${var.azs[count.index]}"
  cidr_block              = "${var.subnets-ip[count.index]}"
  map_public_ip_on_launch = true

  tags = {
    Name = "pub-subnets"
  }
}

# INTERNET GATEWAY
resource "aws_internet_gateway" "i-gateway" {
  vpc_id = "${aws_vpc.ecs-vpc.id}"

  tags = {
    Name = "ecs-igtw"
  }
}
Enter fullscreen mode Exit fullscreen mode

VARIABLES TO VPC 



variable "cidr" {
  type    = string
  default = "145.0.0.0/16"
}

variable "azs" {
  type = list(string)
  default = [
    "us-east-1a",
    "us-east-1b"
  ]
}

variable "subnets-ip" {
  type = list(string)
  default = [
    "145.0.1.0/24",
    "145.0.2.0/24"
  ]

}
Enter fullscreen mode Exit fullscreen mode

IAM ROLES & POLICIES

For the CodeBuild is necessary to create a IAM Role&Policy to allow access to ECR to push and pull the Docker images in the ECR repository. Also, is necessary a permission to access a S3 bucket to store the artifacts.



resource "aws_iam_role" "codebuild-role" {
  name = "codebuild-role"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "codebuild.amazonaws.com"
        }
      },
    ]
  })
}

resource "aws_iam_role_policy" "codebuild-policy" {
  role = "${aws_iam_role.codebuild-role.name}"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action   = ["codecommit:GitPull"]
        Effect   = "Allow"
        Resource = "*"
      },
      {
        Action = [
          "ecr:BatchCheckLayerAvailability",
          "ecr:GetDownloadUrlForLayer",
          "ecr:BatchGetImage",
          "ecr:CompleteLayerUpload",
          "ecr:GetAuthorizationToken",
          "ecr:InitiateLayerUpload",
          "ecr:PutImage",
        "ecr:UploadLayerPart"]
        Effect   = "Allow"
        Resource = "*"
      },
      {
        Action = [
          "logs:CreateLogGroup",
          "logs:CreateLogStream",
        "logs:PutLogEvents"]
        Effect   = "Allow"
        Resource = "*"
      },
      {
        Action = [
          "s3:PutObject",
          "s3:GetObject",
          "s3:GetObjectVersion",
          "s3:GetBucketAcl",
        "s3:GetBucketLocation"]
        Effect   = "Allow"
        Resource = "*"
      }
    ]


  })

}
Enter fullscreen mode Exit fullscreen mode

ROUTE TABLES

A single Route Table for both Public Subnets



# TABLE FOR PUBLIC SUBNETS
resource "aws_route_table" "pub-table" {
  vpc_id = "${aws_vpc.ecs-vpc.id}"
}

resource "aws_route" "pub-route" {
  route_table_id         = "${aws_route_table.pub-table.id}"
  destination_cidr_block = "0.0.0.0/0"
  gateway_id             = "${aws_internet_gateway.i-gateway.id}"
}

resource "aws_route_table_association" "as-pub" {
  count          = length(var.azs)
  route_table_id = "${aws_route_table.pub-table.id}"
  subnet_id      = "${aws_subnet.pub-subnets[count.index].id}"
}
Enter fullscreen mode Exit fullscreen mode

SECURITY GROUPS

The first Sec-Group is for the ECS Service



resource "aws_security_group" "sg1" {
  name        = "golang-server"
  description = "Port 5000"
  vpc_id      = aws_vpc.ecs-vpc.id

  ingress {
    description      = "Allow Port 5000"
    from_port        = 5000
    to_port          = 5000
    protocol         = "tcp"
    cidr_blocks      = ["0.0.0.0/0"]
    ipv6_cidr_blocks = ["::/0"]
  }

  egress {
    description = "Allow all ip and ports outboun"
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}
Enter fullscreen mode Exit fullscreen mode

The second Sec-Group is for the Application Load Balancer



resource "aws_security_group" "sg2" {
  name        = "golang-server-alb"
  description = "Port 80"
  vpc_id      = aws_vpc.ecs-vpc.id

  ingress {
    description      = "Allow Port 80"
    from_port        = 80
    to_port          = 80
    protocol         = "tcp"
    cidr_blocks      = ["0.0.0.0/0"]
    ipv6_cidr_blocks = ["::/0"]
  }

  egress {
    description = "Allow all ip and ports outboun"
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}
Enter fullscreen mode Exit fullscreen mode

APPLICATION LOAD BALANCER 



resource "aws_lb" "app-lb" {
  name               = "app-lb"
  internal           = false
  load_balancer_type = "application"
  security_groups    = [aws_security_group.sg2.id]
  subnets            = ["${aws_subnet.pub-subnets[0].id}", "${aws_subnet.pub-subnets[1].id}"]

}
Enter fullscreen mode Exit fullscreen mode

Port #5000 is used in the Target Group because that port is used for the container



resource "aws_lb_target_group" "tg-group" {
  name        = "tg-group"
  port        = "5000"
  protocol    = "HTTP"
  vpc_id      = "${aws_vpc.ecs-vpc.id}"
  target_type = "ip"

}
Enter fullscreen mode Exit fullscreen mode

Port #80 is used for the Listener




resource "aws_lb_listener" "lb-listener" {

  load_balancer_arn = "${aws_lb.app-lb.arn}"

  port              = "80"

  protocol          = "HTTP"



default_action {

    type             = "forward"

    target_group_arn = "${aws_lb_target_group.tg-group.arn}"

  }

}
Enter fullscreen mode Exit fullscreen mode




ECS & ECR

ECR repository 



resource "aws_ecr_repository" "ecr-repo" {

  name = "ecr-repo"

}
Enter fullscreen mode Exit fullscreen mode



ECS Cluster




resource "aws_ecs_cluster" "ecs-cluster" {

  name = "clusterDev"

}
Enter fullscreen mode Exit fullscreen mode



Task Definition
  • In this part is important to specify the containerPort
  • Create a ENV VAR: export TF_VAR_uri_repo = <ID_ACCOUNT>.dkr.ecr.<REGION>.amazonaws.com/<ECR_REPOSITORY_NAME> 



resource "aws_ecs_task_definition" "task" {

  family                   = "HTTPserver"

  network_mode             = "awsvpc"

  requires_compatibilities = ["FARGATE"]

  cpu                      = 256

  memory                   = 512

  execution_role_arn       = data.aws_iam_role.ecs-task.arn



container_definitions = jsonencode([

    {

      name   = "golang-container"

      image  = "${var.uri_repo}:latest" #URI

      cpu    = 256

      memory = 512

      portMappings = [

        {

          containerPort = 5000

        }

      ]

    }

  ])

}
Enter fullscreen mode Exit fullscreen mode



ECS Service

Specify the load balancer block




resource "aws_ecs_service" "svc" {

  name            = "golang-Service"

  cluster         = "${aws_ecs_cluster.ecs-cluster.id}"

  task_definition = "${aws_ecs_task_definition.task.id}"

  desired_count   = 2

  launch_type     = "FARGATE"



network_configuration {

    subnets          = ["${aws_subnet.pub-subnets[0].id}", "${aws_subnet.pub-subnets[1].id}"]

    security_groups  = ["${aws_security_group.sg1.id}"]

    assign_public_ip = true

  }



load_balancer {

    target_group_arn = "${aws_lb_target_group.tg-group.arn}"

    container_name   = "golang-container"

    container_port   = "5000"

  }

}
Enter fullscreen mode Exit fullscreen mode




CI/CD PIPELINE

CodeCommit Repository 



resource "aws_codecommit_repository" "repo" {

  repository_name = var.repo_name

}
Enter fullscreen mode Exit fullscreen mode



CodeBuild Project




resource "aws_codebuild_project" "repo-project" {

  name         = "${var.build_project}"

  service_role = "${aws_iam_role.codebuild-role.arn}"



artifacts {

    type = "NO_ARTIFACTS"

  }



source {

    type     = "CODECOMMIT"

    location = "${aws_codecommit_repository.repo.clone_url_http}"

  }



environment {

    compute_type    = "BUILD_GENERAL1_SMALL"

    image           = "aws/codebuild/standard:5.0"

    type            = "LINUX_CONTAINER"

    privileged_mode = true

  }

}
Enter fullscreen mode Exit fullscreen mode



buildspec.yml
  • This file is very important to create the Docker Image and Pull it to ECR repository
  • To update the ECS service is important to specify the containerName and imageUri in a JSON file with the name imagedefinitions.json. This file is an artifact
  • This file must be in the CodeCommit repository



version: 0.2



phases:

  pre_build:

    commands:

      - echo Logging in to Amazon ECR...

      - echo $AWS_DEFAULT_REGION

      - aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin 940401905947.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com

      - REPOSITORY_NAME="ecr-repo"


      - REPOSITORY_URI=940401905947.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$REPOSITORY_NAME

      - COMMIT_HASH=$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | cut -c 1-7)

      - IMAGE_TAG=${COMMIT_HASH:=latest}

  build:

    commands:

      - echo Building the Docker image...

      - docker build -t $REPOSITORY_NAME:latest .

      - docker tag $REPOSITORY_NAME:latest $REPOSITORY_URI:latest

      - docker tag $REPOSITORY_NAME:latest $REPOSITORY_URI:$IMAGE_TAG

  post_build:

    commands:

      - docker push $REPOSITORY_URI:latest

      - docker push $REPOSITORY_URI:$IMAGE_TAG

      - printf '[{"name":"golang-container","imageUri":"%s"}]' $REPOSITORY_URI:$IMAGE_TAG > imagedefinitions.json



artifacts:

  files: imagedefinitions.json
Enter fullscreen mode Exit fullscreen mode



S3 Bucket to store the artifacts




resource "aws_s3_bucket" "bucket-artifact" {

  bucket = "eroz-artifactory-bucket"

  acl    = "private"

}
Enter fullscreen mode Exit fullscreen mode



CodePipeline

Specify Source, Build, Deploy Stages

NOTE: to code the stages check the official documentation
https://docs.aws.amazon.com/codepipeline/latest/userguide/action-reference.html




resource "aws_codepipeline" "pipeline" {

  name     = "pipeline"

  role_arn = "${data.aws_iam_role.pipeline_role.arn}"



artifact_store {

    location = "${aws_s3_bucket.bucket-artifact.bucket}"

    type     = "S3"

  }

  # SOURCE

  stage {

    name = "Source"

    action {

      name             = "Source"

      category         = "Source"

      owner            = "AWS"

      provider         = "CodeCommit"

      version          = "1"

      output_artifacts = ["source_output"]




  <span class="nx">configuration</span> <span class="o">=</span> <span class="p">{</span>
    <span class="nx">RepositoryName</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${var.repo_name}</span><span class="dl">"</span>
    <span class="nx">BranchName</span>     <span class="o">=</span> <span class="dl">"</span><span class="s2">${var.branch_name}</span><span class="dl">"</span>
  <span class="p">}</span>
<span class="p">}</span>






    Enter fullscreen mode
    


    Exit fullscreen mode
    









}

  # BUILD

  stage {

    name = "Build"

    action {

      name             = "Build"

      category         = "Build"

      owner            = "AWS"

      provider         = "CodeBuild"

      version          = "1"

      input_artifacts  = ["source_output"]

      output_artifacts = ["build_output"]




  <span class="nx">configuration</span> <span class="o">=</span> <span class="p">{</span>
    <span class="nx">ProjectName</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${var.build_project}</span><span class="dl">"</span>
  <span class="p">}</span>
<span class="p">}</span>






    Enter fullscreen mode
    


    Exit fullscreen mode
    









}

  # DEPLOY

  stage {

    name = "Deploy"

    action {

      name            = "Deploy"

      category        = "Deploy"

      owner           = "AWS"

      provider        = "ECS"

      version         = "1"

      input_artifacts = ["build_output"]




  <span class="nx">configuration</span> <span class="o">=</span> <span class="p">{</span>
    <span class="nx">ClusterName</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">clusterDev</span><span class="dl">"</span>
    <span class="nx">ServiceName</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">golang-Service</span><span class="dl">"</span>
    <span class="nx">FileName</span>    <span class="o">=</span> <span class="dl">"</span><span class="s2">imagedefinitions.json</span><span class="dl">"</span>
  <span class="p">}</span>
<span class="p">}</span>






    Enter fullscreen mode
    


    Exit fullscreen mode
    









}

}
Enter fullscreen mode Exit fullscreen mode




DATA

This section is for using created IAM roles




data "aws_iam_role" "pipeline_role" {

  name = "codepipeline-role"

}



data "aws_iam_role" "ecs-task" {

  name = "ecsTaskExecutionRole"

}
Enter fullscreen mode Exit fullscreen mode




OUTPUTS

To get the ALB DNS and the CodeCommit Repository URL




output "repo_url" {

  value = aws_codecommit_repository.repo.clone_url_http

}



output "alb_dns" {

  value = aws_lb.app-lb.dns_name

}
Enter fullscreen mode Exit fullscreen mode




EXTRA VARIABLES





variable "repo_name" {

  type    = string

  default = "dev-repo"

}



variable "branch_name" {

  type    = string

  default = "master"

}



variable "build_project" {

  type    = string

  default = "dev-build-repo"

}



variable "uri_repo" {

  type = string

  #The URI_REPO value is in a TF_VAR in my PC

}
Enter fullscreen mode Exit fullscreen mode




STEP 03: HTTP Simple Server with GOLANG

This code is useful to get the PRIVATE IP of the ecs tasks




package main



import (

    "fmt"

    "log"

    "net"

    "net/http"

)



func main() {

    log.Print("HTTPserver: Enter main()")

    http.HandleFunc("/", handler)

    log.Fatal(http.ListenAndServe("0.0.0.0:5000", nil))

}



// printing request headers/params

func handler(w http.ResponseWriter, r *http.Request) {




<span class="nx">log</span><span class="p">.</span><span class="nc">Print</span><span class="p">(</span><span class="dl">"</span><span class="s2">request from address: %q</span><span class="se">\n</span><span class="dl">"</span><span class="p">,</span> <span class="nx">r</span><span class="p">.</span><span class="nx">RemoteAddr</span><span class="p">)</span>
<span class="nx">fmt</span><span class="p">.</span><span class="nc">Fprintf</span><span class="p">(</span><span class="nx">w</span><span class="p">,</span> <span class="dl">"</span><span class="s2">%s %s %s</span><span class="se">\n</span><span class="dl">"</span><span class="p">,</span> <span class="nx">r</span><span class="p">.</span><span class="nx">Method</span><span class="p">,</span> <span class="nx">r</span><span class="p">.</span><span class="nx">URL</span><span class="p">,</span> <span class="nx">r</span><span class="p">.</span><span class="nx">Proto</span><span class="p">)</span>

<span class="nx">fmt</span><span class="p">.</span><span class="nc">Fprintf</span><span class="p">(</span><span class="nx">w</span><span class="p">,</span> <span class="dl">"</span><span class="s2">Host = %q</span><span class="se">\n</span><span class="dl">"</span><span class="p">,</span> <span class="nx">r</span><span class="p">.</span><span class="nx">Host</span><span class="p">)</span>
<span class="nx">fmt</span><span class="p">.</span><span class="nc">Fprintf</span><span class="p">(</span><span class="nx">w</span><span class="p">,</span> <span class="dl">"</span><span class="s2">RemoteAddr = %q</span><span class="se">\n</span><span class="dl">"</span><span class="p">,</span> <span class="nx">r</span><span class="p">.</span><span class="nx">RemoteAddr</span><span class="p">)</span>
<span class="k">if</span> <span class="nx">err</span> <span class="p">:</span><span class="o">=</span> <span class="nx">r</span><span class="p">.</span><span class="nc">ParseForm</span><span class="p">();</span> <span class="nx">err</span> <span class="o">!=</span> <span class="nx">nil</span> <span class="p">{</span>
    <span class="nx">log</span><span class="p">.</span><span class="nc">Print</span><span class="p">(</span><span class="nx">err</span><span class="p">)</span>
<span class="p">}</span>
<span class="k">for</span> <span class="nx">k</span><span class="p">,</span> <span class="nx">v</span> <span class="p">:</span><span class="o">=</span> <span class="nx">range</span> <span class="nx">r</span><span class="p">.</span><span class="nx">Form</span> <span class="p">{</span>
    <span class="nx">fmt</span><span class="p">.</span><span class="nc">Fprintf</span><span class="p">(</span><span class="nx">w</span><span class="p">,</span> <span class="dl">"</span><span class="s2">Form[%q] = %q</span><span class="se">\n</span><span class="dl">"</span><span class="p">,</span> <span class="nx">k</span><span class="p">,</span> <span class="nx">v</span><span class="p">)</span>
<span class="p">}</span>
<span class="nx">fmt</span><span class="p">.</span><span class="nc">Fprintf</span><span class="p">(</span><span class="nx">w</span><span class="p">,</span> <span class="dl">"</span><span class="se">\n</span><span class="s2">===&gt; local IP: %q</span><span class="se">\n\n</span><span class="dl">"</span><span class="p">,</span> <span class="nc">GetOutboundIP</span><span class="p">())</span>






    Enter fullscreen mode
    


    Exit fullscreen mode
    









}



func GetOutboundIP() net.IP {

    conn, err := net.Dial("udp", "8.8.8.8:80")

    if err != nil {

        log.Fatal(err)

    }

    defer conn.Close()




<span class="nx">localAddr</span> <span class="p">:</span><span class="o">=</span> <span class="nx">conn</span><span class="p">.</span><span class="nc">LocalAddr</span><span class="p">().(</span><span class="o">*</span><span class="nx">net</span><span class="p">.</span><span class="nx">UDPAddr</span><span class="p">)</span>

<span class="k">return</span> <span class="nx">localAddr</span><span class="p">.</span><span class="nx">IP</span>






    Enter fullscreen mode
    


    Exit fullscreen mode
    









}
Enter fullscreen mode Exit fullscreen mode




STEP 04: Dockerfile

  • This Dockerfile create a image with a HTTP Server with GOLANG
  • This file must be in the CodeCommit repository



FROM golang:alpine AS builder



ENV GO111MODULE=on </span>

    CGO_ENABLED=0 </span>

    GOOS=linux </span>

    GOARCH=amd64

WORKDIR /build

COPY ./HTTPserver.go .



# Build the application

RUN go build -o HTTPserver ./HTTPserver.go



WORKDIR /dist

RUN cp /build/HTTPserver .



# Build a small image

FROM scratch

COPY --from=builder /dist/HTTPserver /

EXPOSE 5000

ENTRYPOINT ["/HTTPserver"]
Enter fullscreen mode Exit fullscreen mode




STEP 05: Create TF_VAR

uri_repo

STEP 06: Create the Infrastructure

Commands

  • terraform init
  • terraform validate
  • terraform plan
  • terraform apply -auto-approve

When the creation is finished we get the OUTPUTS
outputs

STEP 07: Upload Dockerfile, Code and buildspect files to the CodeCommit repository

  • Clone the repository
    Alt Text

  • Copy buildspect.yml, Dockerfile and Golang Code to the cloned repository folder and then do a commit
    git

  • Git push to the CodeCommit Repository
    git push

STEP 08: Check the Pipeline

pipeline

  • When the "Build" stage is done, check the docker image in the ECR repository build

repo

STEP 09: Check the ECS Service

When the "Deploy" stage is done, check the Tasks in the ECS Service

deploy

service

tasks

STEP 10: Check the Target Group

target_group

STEP 11: Check the operation of the Application Load Balancer

alb1

alb2

FINAL STEP: Delete the Infrastructure

terraform destroy -auto-approve

Top comments (4)

Collapse
 
amarg_12 profile image
Amar •

Through terraform i have created ecs and services and codepipeline integrated with org git as source but when i deploy through GitHub the task definition is updated with latest version but when i modify any different resources in terraform and ecs get deployed with initial task definition which is old there in tf file how to overcome this issue.

Collapse
 
erozedguy profile image
erozedguy •

Hey, It's been a long since I created it but, I'm not sure maybe you should revie the container_definitios block in terraform (task_definition resource), there you have the image you are using and the tag. As best practices for docker in general we shouldn't use latest tag, also take a look in the CI/CD pipeline, the image should be tagged using the commit hash. I hope I've helped you :)

Collapse
 
amarg_12 profile image
Amar •

I didn't get you sorry , commit has unique value every time , i want to tell tf about the changes happened at outside and stop it to deploy and maintain the state .

Thread Thread
 
erozedguy profile image
erozedguy •

What did you get as outputs during the tf plan ?. You must review what changes will be applied. Maybe the problem is in the task definition resource, you need to review if the declaration you have in the tf code corresponds to the current configuration you have in AWS, for example the tag of the docket image in the containers_definitions block

Read next

joaomarques profile image

Error when retrieving token from sso Token has expired and refresh failed

Joao Marques -

skorfmann profile image

The Journey of CDK.dev: From Static Site to Bluesky

Sebastian Korfmann -

apilover profile image

Top 10 API Load Testing Tools

Wanda -

igventurelli profile image

Embracing Automated Versioning with Semantic Release

Igor Venturelli -