Background
We were nearing the alpha release of one of our new modules which includes the release of a new mobile application. Our web app, new mobile app and our desktop app, all consume our graphQL API.
To increase the reliability of our platform, we needed to formalize and implement a backwards compatibility and deprecation policy for our GraphQL API.
The rules to add or delete from schema were
- Only add new fields.
- If you want to remove a field, deprecate it and remove it when it’s no longer in use.
- If you want to change an existing field, create a new field, deprecate the old field and remove it once it’s no longer in use.
However, since the mobile app has a much larger footprint as users might be using the app with the older version with old graphQL schema, having a test for it seemed inevitable. Our existing e2e automation on mobile did not cover all graphQL endpoints, and it would have been too expensive and time consuming to cover all endpoints from a user interface perspective. Moreover, we wanted backward compatibility test to be a test that runs before we run e2e tests to detect breaking changes earlier in the cycle.
Since we needed something light weight and graphQL compatible, we decided to use GraphQL Inspector Tool to ensure backward compatibility and prevent schema breaking changes.
Tech Stack
- Bitbucket (Git)
- Codeship Pro for CI/CD workflow
- Docker
- GraphQL
Code Structure
Why GraphQL Inspector Tool?
We decide to use GraphQL Inspector Tool for graphQL backward capability test because of following reasons:
- GraphQL compatibility
- Ease of implementation with custom rules
- Easy CLI command for CI integration
Steps to Implement
Step1: Create a dockerfile for GraphQL Inspector
- Dockerfile template can be found here at graphQL inspector docker hub
- Dockerfile can be named as
Dockerfile.graphqlInspectorin./codeshipfolder
FROM node:10-slim
ENV LOG_LEVEL "debug"
ARG PRIVATE_SSH_KEY
RUN apt update
RUN apt install -y ssh
RUN yarn global add @graphql-inspector/cli@2.1.0 graphql
RUN apt install -y git-all
# Copy the decrypt ssh private key for Bitbucket Readonly User to allow codeship access to
# Our repository from codeship container
RUN mkdir -p /root/.ssh
RUN echo $PRIVATE_SSH_KEY >> /root/.ssh/id_rsa
RUN chmod 600 /root/.ssh/id_rsa
RUN ssh-keyscan -H bitbucket.org >> /root/.ssh/known_hosts
RUN mkdir /app
WORKDIR /app
Step2: Create an entrypoint file
- This file will be triggered after docker image is built
- Name file
entrypoint-graphql-inspector.shin./codeshipfolder- To add git remote for our repository
- Note: it uses remote name as
apibecauseoriginalready taken and it doesn't work in codeship
- Note: it uses remote name as
- To fetch the master branch which will use for
graphql inspector commandin codeship
- To add git remote for our repository
#!/bin/bash
git remote add api git@bitbucket.org:<owner_name>/<repo_name>.git
git fetch api master
exec "$@"
Step3: Create graphQL inspector test service
-
graphql-inspector-testservice will be placed in./codeship-services.yml - Build from
Dockerfile.graphqlInspector - Include entrypoint link with
entrypoint-graphql-inspector.shto trigger the command line after finishing pulling and building the docker image from dockerfile - For
volumes, it can be different depending on your code structure or same as created above in dockerfile.- How to find which volume to mount
- Install jet if you have not (for locally running commands for codeship)
cd <repo>jet run SERVICE_NAME pwd- Example:
jet run graphql-inspector-test pwd - With
pwd, it will show the path of the volume where it should be mounted
services:
graphql-inspector-test:
build:
context: ./
dockerfile: ./codeship/Dockerfile.graphqlInspector
encrypted_args_file: <path_to_encrypted_git_ssh_key>
encrypted_env_file: <path_to_encrypted_git_ssh_key>
entrypoint: ["sh", "./codeship/entrypoint-graphql-inspector.sh"]
cached: true
volumes:
- .:/app
Step4: Create graphql-inspector-test step
-
GraphQL Schema Inspector Test test step under Jest Tests in
./codeship-steps.yml - Use
graphql-inspector-testservice - To trigger command
graphql-inspector diff git:api/master:./src/generated/graphql/schema.graphql ./src/generated/graphql/schema.graphql --rule ./src/generated/graphql/custom-rule.js
- type: serial
name: Continuous Integration
encrypted_dockercfg_path: dockercfg.encrypted
steps:
- name: Tests
type: parallel
exclude: ^(production)
steps:
- name: Jest Tests
type: serial
steps:
- name: Jest Tests for E2E Tests
service: e2e-runner
command: yarn test
- name: ESLint for E2E Tests
service: e2e-runner
command: yarn run lint
- name: Graph QL Test
service: graphql-test
command: yarn run graphql-test
- name: Graph QL Schema Inspector Test
service: graphql-inspector-test
command: graphql-inspector diff
git:api/master:./src/generated/graphql/schema.graphql
./src/generated/graphql/schema.graphql --rule
./src/generated/graphql/custom-rule.js
- name: Deploy to Dev and Production
Step5: How to create custom rules for graphQL inspector test
- For details, you can see graphql inspector custom rules
- We use custom-rules to skip certain attributes that we don’t want the test to run
- Example: the field may not be available to mobile apps yet and still under development, so there may be many changes to the field.
- Create file name
custom-rules.jsin the folder where you have yourschema.graphql
// custom-rules.js
/**
* @summary exceptionSet is an array of schema attributes that can be skipped during graphql schema diff test
* for breaking changes.
* !important: by adding to exceptionSet, there is a risk of graphql schema test will skip these attributes
* example: new Set(['Form.user]);
*/
const exceptionSet = new Set([]);
/**
* @summary changes is an object array of all breaking changes [{error1},{error2}]
* Each error object contains these key value pairs as follow
* critical: { level: 'BREAKING', reason: "some reason"}
* type: 'FIELD_TYPE_CHANGED' or 'FIELD_REMOVED' or 'ENUM_VALUE_REMOVED', etc...
* message: "Field 'XYZ' was removed from interface 'ABC'"
* path: graphql schema attribute, eg: 'Comment.description'
*/
module.exports = ({ changes }) => {
const included = [];
const excluded = [];
for (const change of changes) {
if (exceptionSet.has(change.path)) {
excluded.push(change);
continue;
}
included.push(change);
}
if (excluded.length > 0) {
const warnColor = '\x1b[33m';
console.log('The following changes have been excluded from breaking changes test:');
for (const change of excluded) {
// print something sensible - should show that the field was excluded
// and what potential problem we're ignoring ;-)
const message = `!!! ${change.path}: ${change.message}`;
console.log(warnColor, message);
}
}
return included;
};
How to execute test locally
Prerequisite
npm install --global @graphql-inspector/cli graphql
Steps
cd <repo_folder>
graphql-inspector diff git:origin/master:./src/generated/graphql/schema.graphql ./src/generated/graphql/schema.graphql --rule ./src/generated/graphql/custom-rule.js
Conclusion
We were successfully able to add graphQL backward capabilities test that is easy to maintain, and is platform agnostic by using GraphQL Inspector Tool. We are able to gain a better understanding of how API should be handled for projects, where the same API is used across multiple platforms (i.e. web, desktop and mobile).
Top comments (3)
We would love to give the same experience to Bitbucket as it is today on GitHub with our GitHub Application but the API of Bitbucket is super awkward to deal with.
I recommend to use a branch created by bitbucket (not sure if there's one like on github
refs/pull/1234/merge) where you get the combination of the target branch and source branch. This way you won't end up in a state where Pull Request is behind the target branch.Very cool!
Very nice! Lots of useful details! Thanks!