DEV Community

Cover image for Building a super fast serverless container deployment pipeline on Google Cloud
Blair Hudson for shirtctl

Posted on

Building a super fast serverless container deployment pipeline on Google Cloud

One of our driving principles for shirtctl is #frugalbydesign - we simply don’t want to be paying for anything that we don’t use.

Our architecture needs to balance cost alongside other core capabilities like application security 🔒, design flexibility 💪 and developer collaboration 👩‍💻👨‍💻.

In this post, we’ll be sharing the some of the details of our continuous deployment pipeline. We’ve combined BitBucket with Google's Cloud Build service, which deploys our applications onto Cloud Run in an average of 1-2 minutes per build!

For development, we’ve also created a local build workflow to:

  1. Speed up local code iteration 🏎💨
  2. Minimise the number of Cloud Build jobs and Cloud Run revisions (#frugalbydesign) ☁️
  3. Keep our commit log tidy! 🧹

Here’s a high level view of our approach:


Now let’s take a closer look at some of the major components. 🔎

Speedy local builds

Our MVP sign-ups API is a Python Flask app. It relies on a few various Python packages that provide the REST framework, email, storage and other capabilities. Right now it’s a simple file and a requirements.txt that represents our package dependencies.

Our Dockerfile for local and cloud deployment is purposefully identical, so we can focus on API development.

FROM python:slim

# install python dependencies
RUN python3 -m venv /app/env
COPY requirements.txt .
RUN /app/env/bin/pip install -r requirements.txt

# configure port (Cloud Run requires 8080)

# setup application runtime
WORKDIR /app/src

RUN chmod +x


CMD ["sh", "-c", "./"]
Enter fullscreen mode Exit fullscreen mode

We have a script that emulates Cloud Run deployment locally using Docker, which means we can iterate our development tasks very quickly without having to redeploy to Cloud Run.

REPO=$(basename -s .git $(git config --get remote.origin.url))
BRANCH=$(git rev-parse --abbrev-ref HEAD)

gcloud iam service-accounts keys create sa-key.json \
SA_KEY_FILE_BASE64=$(cat sa-key.json | base64)

docker build -t shirtctl-${REPO}-${BRANCH}:latest .

docker run --rm -it \
 -e K_SERVICE=localbuild \
 -p 8080:8080 \
 -v $(pwd):/app/src \
Enter fullscreen mode Exit fullscreen mode

We can “hot reload” 🔥 our changes to develop even faster! determines at run time whether to run Flask or Gunicorn depending on the value of $K_SERVICE. This way our Flask service restarts automatically when changes to the source code are detected:


if [ "$K_SERVICE" = "localbuild" ] ; then
    export FLASK_APP=""
    export FLASK_DEBUG=1
    /app/env/bin/flask run --host= --port=$PORT
    /app/env/bin/gunicorn --bind=$PORT api:app
Enter fullscreen mode Exit fullscreen mode

BitBucket to Cloud Source Repository

Code is committed and pushed to a private BitBucket repo. Our branching structure is simple:

  • ⚙️ dev for feature-based development (we can have as many of these as required!)
  • test where all feature dev branches are merged to (by pull request only)
  • 🚀 prod where test is released to (also by pull request only, with dual approval required)

The BitBucket repo is automatically synced to a Cloud Source Repository of the same name and branch structure.

Deploying with Cloud Build

Cloud Build allow a build job to trigger on a push to our repo. This runs submits the cloudbuild.yaml file from our repo to Cloud Build, which accomplishes the following steps for the current branch:

Pulls the previous Docker image from Google Container Registry

Enter fullscreen mode Exit fullscreen mode

Builds and tags a new Docker image from our Dockerfile above:

docker build . \
 --cache-from$PROJECT_ID/$REPO_NAME-$BRANCH_NAME:latest \
Enter fullscreen mode Exit fullscreen mode

Pushes the latest image to Google Container Registry:

Enter fullscreen mode Exit fullscreen mode

Deploys the latest image to Cloud Run, and maps the appropriate domains to access the service:

gcloud beta run deploy $REPO_NAME-$BRANCH_NAME \
gcloud beta run domain-mappings create \
         --service $REPO_NAME-$BRANCH_NAME \
         --domain $BRANCH_NAME.$
Enter fullscreen mode Exit fullscreen mode

That's all for now! Keep an eye on for our MVP sign-ups launch! 👕👚

Top comments (0)