Deploying a site to Netlify is super simple and has served me well so far. But, it has a few limits which resulted in my builds failing after my site grew to its current size.
- a 15-minute timeout limit for builds
- the free tier has 300 free minutes in a month. In most cases this is fine, but gatsby takes a fair amount of time to process images and I was about to hit the limit.
First Attempt - Cache the gatsby folders
For some time, I was able to solve the issue by using the unofficial gatsby-plugin-netlify-cache plugin. The plugin uses an undocumented shared folder to cache the .cache and public folders across builds.
Unfortunately, this solution had 2 main issues
- A clear cache and deploy site would fail.
- Since my site is image-heavy, I started reaching close to the limit again.
Second Attempt - Use Azure Piplelines
Azure Pipelines uses a YAML config file to build your project. We will use templates so that we can have separate workflows for pr and release builds. Let’s create 3 files:
- azure-pipelines.common.yaml: The template file with the job to build the gatsby site and deploy to netlify.
- azure-pipelines.pr.yaml: The pipeline to run for PR(Pull Request) builds.
- azure-pipelines.ci.yaml: The pipeline to run for Live builds.
Common Job to Build Gatsby Site
In azure-pipelines.common.yaml we create the job to build the gatsby site.
jobs:
- job: Build
pool:
vmImage: 'ubuntu-latest'
steps:
- checkout: self
lfs: true
- task: NodeTool@0
displayName: 'Install Node.js'
inputs:
versionSpec: '10.x'
- script: yarn install
displayName: 'Install dependencies'
- script: yarn build
displayName: 'Build Gatsby'
We name the job as Build and use the default ubuntu-latest image. This is followed by a series of steps
- Checkout: Check out the repo and download the files from git-lfs.
- Install Node.js : Install Node 10.x.
- Install dependencies : Execute yarn install.
- Build Gatsby: Execute the yarn build script which calls gatsby build.
Live Builds
Let’s add the trigger for live builds in azure-pipelines.ci.yaml.
name: 'Live'
trigger:
batch: false
branches:
include:
- master
pr: none
jobs:
- template: azure-pipelines.common.yml
- name: The name for this pipeline.
-
trigger: Conditions for running the pipeline for when a push is made.
- batch: false: trigger the pipeline for each commit i.e disable batching of builds.
- branches: Include only the master branch so that the pipeline is triggered only for commits pushed to_master_.
- pr: none: Disable triggering the pipeline for when a PR is opened or changes pushed to an open PR.
-
jobs: The jobs to run.
- template : Use the job template created in azure-pipelines.common.yml.
PR(Pull Requests) Builds
I also want a build to be triggered every time a PR is made against master.
name: 'PR $(System.PullRequest.PullRequestId)'
trigger:
branches:
exclude:
- '*'
pr:
autoCancel: false
branches:
include:
- master
- template: azure-pipelines.common.yml
- name: The name for this pipeline.
-
trigger: Conditions for running the pipeline for when a push is made.
- branches: Exclude all branches so that the build is not triggered for commits on any branch including master.
-
pr:: Conditions for running the pipeline for when a PR is opened or changes pushed to an open PR.
- autoCancel : Additional pushes should not cancel in-progress runs.
- include: Include only the master branch so that the pipeline is triggered only for PR’s opened against master.
-
jobs: The jobs to run.
- template : Use the job template created in azure-pipelines.common.yml.
At this point, you might be wondering for having 2 separate files for pr and Live builds. This will become important in the next section when we add the steps to deploy to netlify.
Deploying to Netlify
In order to deploy to Netlify, we need a few variables which will be passed to the template as parameters and then passed as flags to the netlify deploy command.
First, let’s see what changes we need to make to the template azure-pipelines.common.yml.
parameters:
deployScriptParams: '' # defaults for any parameters that aren't specified netlifySiteId: 'defaultID' netlifyToken: 'defaultToken' message: ''jobs:
- job: Build_And_Deploy
pool:
vmImage: 'ubuntu-latest'
steps:
#unchanged from before
- script: yarn build
displayName: 'Build Gatsby'
# prettier-ignore - script: yarn netlify deploy --site ${{ parameters.netlifySiteId }} --auth ${{ parameters.netlifyToken }} --message '${{ parameters.message }}' ${{ parameters.deployScriptParams }} --dir=public displayName: 'Deploy to Netlify'
-
parameters: The environment variables to pass as flags to the netlify deploy command.
- deployScriptParams: Any additional parameters that should be passed to the deploy command.
- netlifySiteId : The site ID for the netlify site.
- netlifyToken: The access token to authorize Azure Pipeline to deploy the build to netlify.
- message: The message to include in the deploy log.
-
Deploy to Netlify: The netlify deploy command which takes the siteId, token, message and deployScriptParams as flags. We also set the dir flag to public so that the public folder is deployed.
- # prettier-ignore: I have added this so that prettier does not try to break up the command into multiple lines.
Now we update azure-piplelines.ci.yaml for the live build.
# unchanged from before
variables: - template: azure-pipelines.vars.yml
jobs:
- template: azure-pipelines.common.yml
parameters: deployScriptParams: '--prod' netlifySiteId: $(netlify.siteId) netlifyToken: $(netlify.token) message: 'Live $(Build.SourceVersion) $(Build.SourceVersionMessage)'
- variables: We set a template file to get the environment variables.
-
parameters: The parameters to be passed to the job template
- deployScriptParams: For live builds, I pass the -—prod flag which netlify to deploy the build to production.
- netlifySiteId : The site ID from environment variables.
- netlifyToken : The token from environment variables.
- message: The message is created using the commit ID and the comment of the commit.
Next azure-pipelines.pr.yaml for the PR builds. The new lines are highlighted.
# unchanged from before
variables: - template: azure-pipelines.vars.yml
jobs:
- template: azure-pipelines.common.yml
parameters: netlifySiteId: $(netlify.siteId) netlifyToken: $(netlify.token) message: 'PR $(System.PullRequest.PullRequestNumber) $(System.PullRequest.SourceBranch) $(Build.SourceVersionMessage)'
- variables: We set a template file to get the environment variables.
-
parameters: The parameters to be passed to the job template
- netlifySiteId : The site ID from environment variables.
- netlifyToken : The token from environment variables.
- message: The message is created using the Pull Request Number, Source branch and the comment of the commit.
Conclusion
By using Azure Pipelines, the netlify limits are no longer a problem. In a later post, we will see how we can add tests and run them against the deploy url.
Top comments (0)