Deploying a Gatsby site to Netlify using Azure Pipelines

Deploying a site to Netlify is super simple and has served me well so far. But, it has a few limits which resulted in my builds failing after my site grew to its current size.

• a 15-minute timeout limit for builds
• the free tier has 300 free minutes in a month. In most cases this is fine, but gatsby takes a fair amount of time to process images and I was about to hit the limit.

First Attempt - Cache the gatsby folders

For some time, I was able to solve the issue by using the unofficial gatsby-plugin-netlify-cache plugin. The plugin uses an undocumented shared folder to cache the .cache and public folders across builds.

Unfortunately, this solution had 2 main issues

1. A clear cache and deploy site would fail.
2. Since my site is image-heavy, I started reaching close to the limit again.

Second Attempt - Use Azure Piplelines

Azure Pipelines uses a YAML config file to build your project. We will use templates so that we can have separate workflows for pr and release builds. Let’s create 3 files:

1. azure-pipelines.common.yaml: The template file with the job to build the gatsby site and deploy to netlify.
2. azure-pipelines.pr.yaml: The pipeline to run for PR(Pull Request) builds.
3. azure-pipelines.ci.yaml: The pipeline to run for Live builds.

Common Job to Build Gatsby Site

In azure-pipelines.common.yaml we create the job to build the gatsby site.

jobs:
    - job: Build

      pool:
          vmImage: 'ubuntu-latest'

      steps:
          - checkout: self
            lfs: true

          - task: NodeTool@0
            displayName: 'Install Node.js'
            inputs:
                versionSpec: '10.x'

          - script: yarn install
            displayName: 'Install dependencies'

          - script: yarn build
            displayName: 'Build Gatsby'

We name the job as Build and use the default ubuntu-latest image. This is followed by a series of steps

1. Checkout: Check out the repo and download the files from git-lfs.
2. Install Node.js : Install Node 10.x.
3. Install dependencies : Execute yarn install.
4. Build Gatsby: Execute the yarn build script which calls gatsby build.

Live Builds

Let’s add the trigger for live builds in azure-pipelines.ci.yaml.

name: 'Live'

trigger:
    batch: false
    branches:
        include:
            - master

pr: none

jobs:
    - template: azure-pipelines.common.yml

1. name: The name for this pipeline.
2. trigger: Conditions for running the pipeline for when a push is made.
   1. batch: false: trigger the pipeline for each commit i.e disable batching of builds.
   2. branches: Include only the master branch so that the pipeline is triggered only for commits pushed to_master_.
3. pr: none: Disable triggering the pipeline for when a PR is opened or changes pushed to an open PR.
4. jobs: The jobs to run.
   1. template : Use the job template created in azure-pipelines.common.yml.

PR(Pull Requests) Builds

I also want a build to be triggered every time a PR is made against master.

name: 'PR $(System.PullRequest.PullRequestId)'

trigger:
    branches:
        exclude:
            - '*'
pr:
    autoCancel: false
    branches:
        include:
            - master

    - template: azure-pipelines.common.yml

1. name: The name for this pipeline.
2. trigger: Conditions for running the pipeline for when a push is made.
   1. branches: Exclude all branches so that the build is not triggered for commits on any branch including master.
3. pr:: Conditions for running the pipeline for when a PR is opened or changes pushed to an open PR.
   1. autoCancel : Additional pushes should not cancel in-progress runs.
   2. include: Include only the master branch so that the pipeline is triggered only for PR’s opened against master.
4. jobs: The jobs to run.
   1. template : Use the job template created in azure-pipelines.common.yml.

At this point, you might be wondering for having 2 separate files for pr and Live builds. This will become important in the next section when we add the steps to deploy to netlify.

Deploying to Netlify

In order to deploy to Netlify, we need a few variables which will be passed to the template as parameters and then passed as flags to the netlify deploy command.

First, let’s see what changes we need to make to the template azure-pipelines.common.yml.

parameters:
    deployScriptParams: '' # defaults for any parameters that aren't specified netlifySiteId: 'defaultID' netlifyToken: 'defaultToken' message: ''jobs:
    - job: Build_And_Deploy
      pool:
          vmImage: 'ubuntu-latest'

      steps:
          #unchanged from before

          - script: yarn build
            displayName: 'Build Gatsby'
          # prettier-ignore - script: yarn netlify deploy --site ${{ parameters.netlifySiteId }} --auth ${{ parameters.netlifyToken }} --message '${{ parameters.message }}' ${{ parameters.deployScriptParams }} --dir=public displayName: 'Deploy to Netlify'

1. parameters: The environment variables to pass as flags to the netlify deploy command.
   1. deployScriptParams: Any additional parameters that should be passed to the deploy command.
   2. netlifySiteId : The site ID for the netlify site.
   3. netlifyToken: The access token to authorize Azure Pipeline to deploy the build to netlify.
   4. message: The message to include in the deploy log.
2. Deploy to Netlify: The netlify deploy command which takes the siteId, token, message and deployScriptParams as flags. We also set the dir flag to public so that the public folder is deployed.
   1. # prettier-ignore: I have added this so that prettier does not try to break up the command into multiple lines.

Now we update azure-piplelines.ci.yaml for the live build.

# unchanged from before
variables: - template: azure-pipelines.vars.yml
jobs:
    - template: azure-pipelines.common.yml
      parameters: deployScriptParams: '--prod' netlifySiteId: $(netlify.siteId) netlifyToken: $(netlify.token) message: 'Live $(Build.SourceVersion) $(Build.SourceVersionMessage)'

1. variables: We set a template file to get the environment variables.
2. parameters: The parameters to be passed to the job template
   1. deployScriptParams: For live builds, I pass the -—prod flag which netlify to deploy the build to production.
   2. netlifySiteId : The site ID from environment variables.
   3. netlifyToken : The token from environment variables.
   4. message: The message is created using the commit ID and the comment of the commit.

Next azure-pipelines.pr.yaml for the PR builds. The new lines are highlighted.

# unchanged from before
variables: - template: azure-pipelines.vars.yml
jobs:
    - template: azure-pipelines.common.yml
      parameters: netlifySiteId: $(netlify.siteId) netlifyToken: $(netlify.token) message: 'PR $(System.PullRequest.PullRequestNumber) $(System.PullRequest.SourceBranch) $(Build.SourceVersionMessage)'

1. variables: We set a template file to get the environment variables.
2. parameters: The parameters to be passed to the job template
   1. netlifySiteId : The site ID from environment variables.
   2. netlifyToken : The token from environment variables.
   3. message: The message is created using the Pull Request Number, Source branch and the comment of the commit.

Conclusion

By using Azure Pipelines, the netlify limits are no longer a problem. In a later post, we will see how we can add tests and run them against the deploy url.