Making Lambda/API Gateway Environments for each git Branches

To test easily, our team tried to make Lambda/API Gateway environments for each git branches.

Background / Requirements

• We are using CircleCI for test and deployment.
• Web Application that is created with serverless and serverless-express.
• Your domain is managed by Route53, and SSL certificate is issued by Amazon Certificate manager on us-east-1 region

Install plugin

To manage custom domain, install plugin:

yarn add -D serverless-domain-manager

Configure serverless.yml

• To avoid creating S3 bucket creation for each branches, set deploymentBucket into serverless.yml in order to specify bucket place.
• To switch environment, serverless.yml load configuration from ./config/sls-config.yml. (If you need to set secured variable, use KMS to encrypt it.)

# serverless.yml

service:
  name: ${self:custom.name}

provider:
  name: aws
  region: ap-northeast-1
  runtime: nodejs6.10
  stage: ${opt:stage, 'dev'}
  deploymentBucket:
    name: ${self:custom.name}.${self:provider.region}.deploy
  environment:
    XXX: ${self:custom.config.XXX, ''}
    YYY: ${self:custom.config.YYY, ''}

custom:
  name: project-name
  customDomain:
    domainName: ${self:provider.stage}.sls.example.com
    basePath: ''
    stage: ${self:provider.stage}
    certificateName: '*.sls.example.com'
    createRoute53Record: true
  config: ${file(./config/sls-config.yml)}

plugins:
  - serverless-domain-manager

functions:
  render:
    handler: server.render
    events:
      - http:
          path: '/'
          method: 'get'
          private: false
      - http:
          path: '{proxy+}'
          method: 'get'
          private: false

Make configuration for each environments

Make branch layout. For example,

• master branch = for production environment.
• develop branch = for development environment.
• Other branches = preview environment.

And make configuration file for each environments. And put object to any S3 Bucket with environment prefix:

• production => s3://config-bucket/production/sls-config.yml
• development => s3://config-bucket/development/sls-config.yml
• preview => s3://config-bucket/preview/sls-config.yml

Make deployment script

• Make sure you cannot use -, _ to API Gateway stage name or CloudFormation template name. So this script removes these characters from git branch name. (e.g. 123-test-branch => 123testbranch) It will be used for stage name for serverless framework.
• Set script file permission to 755.

#!/bin/bash

# scripts/deploy.sh

set -eu

export NODE_ENV="preview"

if [ "${CIRCLE_BRANCH}" == "master" ]; then
    export NODE_ENV="production"
elif [ "${CIRCLE_BRANCH}" == "develop" ]; then
    export NODE_ENV="development"
fi

STAGE=${CIRCLE_BRANCH//[-\/]/}
echo stage=${STAGE}

# get config file from S3
aws s3 cp s3://config-bucket/${NODE_ENV}/sls-config.yml config/

# make Route53 configuration
./node_modules/.bin/sls create_domain --stage=$STAGE
# deploy
./node_modules/.bin/sls deploy --stage=$STAGE

# remove old CloudFormation stacks when you delete remote branch
STACKS=`aws cloudformation list-stacks --stack-status-filter CREATE_COMPLETE CREATE_FAILED UPDATE_COMPLETE ROLLBACK_COMPLETE ROLLBACK_FAILED UPDATE_ROLLBACK_FAILED --max-items 1000 | jq '.StackSummaries[].StackName' -r | grep '^project-name-'`
BRANCHES=`git branch -r | sed -e "s/^[[:space:]]*origin\///" | grep -v "^HEAD" | uniq`

echo "stacks----"
echo "${STACKS}"
echo "branches----"
echo "${BRANCHES}"

while read line
do
    found=0

    lineStage=`echo $line | sed -e 's/^project-name-//'`

    while read branch
    do
        stage=${branch//[-\/]/}

        if [[ "$lineStage" = "$stage" ]]; then
            found=1
            break;
        fi
    done <<END
$BRANCHES
END

    if [ $found -eq 0 ]; then
      ./node_modules/.bin/sls remove --stage=$lineStage || true
      ./node_modules/.bin/sls delete_domain --stage=$lineStage || true
    fi

done <<END
$STACKS
END

Configure .circleci/config.yml

Your docker image for the deployment needs following:

• nodejs6
• python
• pip
• jq
• awscil

And make .circleci/config.yml

# .circleci/config.yml

version: 2
jobs:
  build:
    docker:
      - image: my_docker_image
    parallelism: 1
    working_directory: ~/project-name
    steps:
      - checkout

      - restore_cache:
          key: project-name-yarn-{{ checksum "yarn.lock" }}

      - run:
          name: Install node modules
          command: yarn install

      - run:
          name: lint
          command: yarn run lint

      - run:
          name: unit
          command: yarn run unit

      - save_cache:
          key: project-name-yarn-{{ checksum "yarn.lock" }}
          paths:
            - ~/.yarn-cache

      - run:
          name: deploy
          command: scripts/deploy.sh

Done!

By this configuration, CircleCI will make Lambda/API Gateway environment to ${stagename}.sls.example.com. At first time to mapping domain, Route53 & CloudFront requires 40 minutes.

Comments

[Frank Wang]

Awesome post!

For anyone interested in extending this setup, here is a post on building a CI/CD pipeline with PR workflow and auto-removing stages on merge for Serverless monorepo apps with CircleCI - seed.run/blog/how-to-build-a-cicd-...