AWS DevOps Essentials

Developing a CI/CD Pipeline, is often a monotonous task and requires a lot of manual work to automate the development process. AWS introduces the solution to all our CI/CD problems by providing simple, easy-to-use, and highly scalable services. In this post, we'll be going through the main AWS services used for CI/CD including CodeCommit, CodeBuild, CodeDeploy, and CodePipeline.

Services Overview

CodeCommit

CodeCommit is an AWS-managed Version Control service, which allows users to manage and version their code efficiently and securely. GitHub is a popular alternative to CodeCommit and there are many advantages and disadvantages with using either service, but one advantage of CodeCommit is that it provides free private repositories, which can be useful based on your use case.

CodeBuild

CodeBuild is AWS's Continuous Integration service, that allows users to test their code with environments similar to their production environment. Users write tests in a buildspec.yml file, detailing what should occur
in the install pre-build, build, and post-build phases (see sample buildspec.yml file below)

version: 0.2

phases:
  install:
    runtime-version:
      python: 3.7
    commands:
      - echo "installing something"
  pre_build:
    commands:
      - echo "pre build phase"
  build:
    commands:
      - echo "build phase"
  post_build:
    commands:
      - echo "post build phase"

artifacts:
  type: zip
  files:
    - file1.yml
    - file2.yml

The logs returned from CodeBuild can be seen in the AWS console.

Notice the artifacts specified with the "artifacts" key can be used by AWS's deployment service CodeDeploy as an input to the deployment and artifacts are saved in Amazon S3 (Amazon's Simple Storage Service).

CodeDeploy provides continuous delivery and is easily integrated with the services outlined above. CodeDeploy deploys your application to your server and allows for in-place and blue-green deployments.

In-place deployments spin down all instances that run your old code, then start new instances with the new deployed code, so it has some downtime, but it's cheap and fast, so it would be ideal for a Development environment.

Blue-green deployments allow for gradual deployment of your new application by creating a separate set of instances (without terminating the old instances) and slowly redirecting traffic to the new set of instances using Amazon Route 53 and Auto Scaling Groups. Although this costs more, there is no downtime, which is ideal for a Production environment.

In CodeDeploy, you create deployment groups where you specify the type of deployment as well as where you want to deploy your application.

Important Note: The server where you want to deploy your application should already exist. In AWS, you can deploy your application using Amazon EC2 (Amazon's Elastic Compute Service), Amazon ECS (Amazon's Container Service), or Lambda (Amazon's Serverless Service).

The actions CodeDeploy takes are defined in an application.yml file. This file details what happens when the application stops, before installing the application, after installing, and what to do when starting and testing the application (see example application.yml file below)

version: 0.0
os: linux
files:
  - source: /app/
    destination: /var/www/html/
hooks:
  ApplicationStop:
    - location: scripts/stop_server.sh
      timeout: 300
      runas: root

  BeforeInstall:
    - location: scripts/install_dependencies.sh
      timeout: 300
      runas: root

  AfterInstall:
    - location: scripts/after_install.sh
      timeout: 300
      runas: root

  ApplicationStart:
    - location: scripts/start_server.sh
      timeout: 300
      runas: root

  ValidateService:
    - location: scripts/validate_service.sh
      timeout: 300

Notice the script files that run can setup a httpd server for example and validate whether it was successfully setup. Also note the timeout of 5 minutes (300 seconds) and for all hooks (other than ValidateService), we run the scripts in the scripts folder as root.

We covered the AWS services that allow for CI and CD, but how do we combine these services together? The answer is CodePipeline

CodePipeline

CodePipeline allows us to create a pipeline that combines all the services we discussed in this topic, which gives us an easy and automated manner of updating our pipeline and infrastructure.

In CodePipeline, we can specify the input to our pipeline as CodeCommit or GitHub and have our pipeline be triggered when code is pushed to our source code repository (for example, when code is pushed to the master branch in our repo).

When code is pushed, we want to make sure our code works as expected, so we run some tests (CI). This is where CodeBuild comes in, grabs the artifacts (zipped source code) stored in Amazon S3, creates the build environment and runs the tests based on whatever is specified in our buildspec.yml file.

Important Note: We can create an Amazon CloudWatch Event, which gets triggered if a test fails (or anything fails in CodePipeline) and when it's triggered, it can do a variety of things, such as notify developers via SNS or Slack that the build failed.

If the build succeeds, we can now deploy our code to where we're hosting our application using CodeDeploy. Once again, the inputs would come from the output of the previous step (in this case CodeBuild), which comes from S3.

Tip: You can have a Development deployment, which you'd deploy to your test instances and can verify the application running smoothly and after it successfully deploys, the pipeline would wait in a "Manual Approval" state, so it waits for someone to manually approve the change. If "Approve" is chosen, the code is then deployed the Production environment (see example diagram below).

Conclusion

The services AWS provides are great tools that can greatly simplify and enhance your CI/CD workflow.

For more information on CI/CD for AWS, I recommend clicking the following link to start your AWS DevOps journey: Begin Journey