Creating a CI/CD pipeline with a git repository in ~30 seconds using Terraform and AWS

Terraform is a powerful tool for provisioning and managing cloud infrastructure. I recently wrote a blog post that provisions a CI/CD pipeline integrated with GitHub repositories to continuously deploy static websites. After going through the exercise of creating that template and blog post I realized I could abstract all of this.

So I created a new Terraform module that provisions an entire CI/CD pipeline and AWS CodeCommit Git repository in roughly 30 seconds.

To leverage the module we can create a new template called my-stack.tf and add the following to it.

module "codecommit-cicd" {
    source                 = "git::https://github.com/slalompdx/terraform-aws-codecommit-cicd.git?ref=master"
    repo_name              = "learn-by-doing-newsletter"              # Required
    organization_name      = "kylegalbraith"                     # Required
    repo_default_branch    = "master"                     # Default value
    aws_region             = "us-west-2"                  # Default value
    char_delimiter         = "-"                          # Default value
    environment            = "dev"                        # Default value
    build_timeout          = "5"                          # Default value
    build_compute_type     = "BUILD_GENERAL1_SMALL"       # Default value
    build_image            = "aws/codebuild/nodejs:6.3.1" # Default value
    test_buildspec         = "buildspec_test.yml"         # Default value
    package_buildspec      = "buildspec.yml"              # Default value
    force_artifact_destroy = "false"                      # Default value
}

output "repo_url" {
    depends_on = ["module.codecommit-cicd"]
    value = "${module.codecommit-cicd.clone_repo_https}"
}

There are quite a few variables that can be passed into the module. Here is a brief explanation of what each one does:

• repo_name: The name of our new Git repository.
• organization_name: The organization name provisioning the template.
• repo_default_branch: The default branch of our Git repository.
• aws_region: The AWS region we want to provision our infrastructure in.
• char_delimiter: The delimiter to use for unique names.
• environment: What our development team calls this environment (i.e. dev v.s. prod).
• build_timeout: How long CodeBuild will wait for a build to complete in minutes.
• build_compute_type: The instance type CodeBuild will use to run our builds on.
• build_image: The base AMI our build instances will use to run our builds on.
• test_buildspec: The path to the test runner buildspec.yml file.
• package_buildspec: The path to the package & deploy buildspec.yml file.
• force_artifact_destroy: Specify whether or not we want to forcibly delete our build artifacts when this template is destroyed.

To provision the resources in our AWS account we run the following commands from our command line.

$>terraform init
.....this will initialize terraform template.....
$>terraform plan
.....this will show you what the module is going to provision.....
$>terraform apply
.....this will create all of the resources.....

Once the apply command is complete we will have the following resources created in our Amazon Web Services account.

• AWS CodeCommit git repository for source code.
• AWS CodePipeline with a build and test stage.
• AWS CodeBuild projects, one that is looking for a buildspec_test.yml file and one for the build process that is looking for a buildspec.yml file.

An important thing to note here is that our new repository is not yet initialized. We need to clone the repository using the URL from our template output, add a file, and then push.

$>git clone https://git-codecommit.us-west-2.amazonaws.com/v1/repos/learn-by-doing-newsletter
$>cd learn-by-doing-newsletter
$>echo 'hello world' > touch.txt
$>git commit -a -m 'init master'
$>git push -u origin master

With this module and about 30 seconds on our command line, we have created a new git repository and provisioned a CI/CD pipeline all in AWS.

It might not seem like much on the surface but infrastructure as code like this is a huge time saver. Take a look at this accompanying blog post that has manual steps for creating our CI/CD pipeline. There are 21 different steps to create the pipeline. Here in our template we create the pipeline and a new repository in 20 lines of code.

This comes in very handy when we need to provision new micro-services frequently. We could turn this into a template that other teams use to quickly stand up their own CI/CD pipelines and repositories. Terraform is a great tool for provisioning our infrastructure and there is more power in encapsulating common infrastructure into its own module.

If you are looking to start treating your infrastructure as code, start thinking about using Terraform. Give this module a try, and let us know if there is more you would like to see us add to it!

Learn AWS by actually using it

If you enjoyed this post and are hungry to start learning more about Amazon Web Services, I have created a new course on how to host, secure, and deliver static websites on AWS! It is a book and video course that cuts through the sea of information to accelerate your learning of AWS. Giving you a framework that enables you to learn complex things by actually using them.

Head on over to the landing page to get a copy for yourself!

Comments

[Vincent Fulco (It / It's)]

Phenomenal tutorial. Looking forward to diving into the course in 1-2 weeks.

[Jitendra Thakur]

This is really good, I have ever seen, can you please provide some links? specially related to Terraform (to get depth knowledge). And thanks for this informative block of lines.