DEV Community: Paul B.

Bump diff, the missing piece for an API “design-first” approach

Paul B. — Wed, 07 Jul 2021 08:55:21 +0000

While building APIs, are you more of a design-first or a code-first type of person? The former approach is usually done when designing APIs for clients, while the latter one is usually used for internal APIs.

At Bump we like to focus on design-first APIs because we think an API - be it RESTful, Messaging, GraphQL, … - needs more love during its design phase. This is why we have improved our tools to go in that direction.

By focusing on the API contract first, we step into the shoes of API consumers. And it is a great way to make sure we build consumer-friendly APIs. Thus making the API clearer, easier to use, and improving its adoptability.

However, until today, we had one missing piece to offer a good design-first approach for our users. Read on for some good news if you are a “design-first” person or maybe a good argument to convince you to become one 😊

API contract development

For clarity we will focus on a tiny API from our demo train company which retrieves the current logged-in users account details. It’s made up of a single GET /account endpoint.

Now, imagine you want to change the API to add a list of tickets in the user details, and also want to add a new endpoint to be able to fetch a specific ticket. The changes are easily described in human words, but what does it represent from your API contract point of view? You will modify the JSON or YAML contract with your favorite tool, but now what? git diff will probably be a bit hard to read…

This is where our new bump diff command comes into play.
Make sure to upgrade your bump-cli package to at least v2.1.0 to test it.

Straight from your CLI, you are now able to describe the changes made in your contract:

> bump diff --doc users-account api-specification.yaml
Updated: GET /account
  Response modified: 200
    Body attribute added: ticket_ids
Added: GET /tickets/{ticket_id}

The new command will output a quick summary of what has changed between your latest deployed API contract and the one you have changed locally.

If you need more context, with a visual diff for instance, you can even use the --open flag to open the diff directly in your browser:

> bump diff --open --doc users-account api-specification.yaml

The command will return the diff summary and open this documentation diff in your browser

Collaborating on API design

So, you have made some changes to the contract locally but now you want to receive suggestions and reviews from your team. It’s a good time to open a pull request.

If you use Github Actions to launch your automation workflows, we have some more good news for you: we now offer a stable bump-sh/github-action@v1 action. And this release includes automatic API contract changelog pushed as a comment for each pull request.

Your team can now directly discuss on the pull request and suggest improvements by reading the diff summary without needing to dig into the JSON diff - which you’ll have to admit is a PITA to read 😅 -. If you update the contract again, the initial comment will be updated accordingly.

We find this bot comment really useful because it helps the team to jump in API design decisions quickly without having to step into a full code review. And it’s a win-win situation, because the person implementing the changes doesn’t have to focus on the code directly to receive quick feedback.

More to come soon

If you are still unsure whether you want to work with a design-first or code-first approach when building your APIs here are two good reads from the community:

A design-first example from the AsyncAPI community
a comparison of both methods from the OpenAPI community.

Regarding our two novelties, if you want to try them, you can visit our dedicated help pages or contact us anytime:

You will soon see some more improvements, on the way we display diff summaries, helping you identify breaking changes by warning the reviewers or even break the Continuous Integration checks if you desire… So stay tuned.
Until then we will continue to work on one of our initial mission: “Git diff, for your APIs”.

Have fun, stay free and stay kind.

✨

Playing with Twilio's OpenAPI specifications

Paul B. — Thu, 22 Apr 2021 12:46:47 +0000

This week, Twilio published OpenAPI definitions for all of their APIs. Let's see together the small steps we can take to transform those definitions into a nice looking API Reference documentation page which tracks API changes automatically.

If you wonder what Twilio is, here's a paragraph taken from their about page:

Twilio has democratized communications channels like voice, text, chat, video, and email by virtualizing the world’s communications infrastructure through APIs that are simple enough for any developer to use, yet robust enough to power the world’s most demanding applications.

Note: this is an experiment demonstration which is not endorsed, nor affiliated by, the Twilio company.

Definition files hierarchy

First let's take a look at the repository with their published files. The OpenAPI definition files are available in the spec/ folder in both JSON or YAML formats. Here's a sneak peek of some of the JSON files available:

spec/
└── json/
    ├── twilio_accounts_v1.json
    ├── twilio_chat_v2.json
    ├── twilio_conversations_v1.json
    ├── twilio_events_v1.json
    ├── twilio_fax_v1.json
    ├── twilio_messaging_v1.json
    ├── twilio_notify_v1.json
    ├── twilio_numbers_v2.json
    ├── twilio_verify_v2.json
    ├── twilio_video_v1.json
    ├── twilio_voice_v1.json
    └── …

At first glance we can imagine a quick way of generating the documentation pages:

For each file from the available *.json files
- Extract the API name from filename twilio_<api_name>_vN.json
- Create documentation api_name from the api definition file twilio_<api_name>_vN.json within a single Hub named Twilio
Enjoy the result and customise the UI!

Automatically publish the documentation on changes

In order to demonstrate the magic on each commit I've forked their repository on my Github namespace where I will be able to push to my main branch and use Github Actions on my fork.

Now let's write a tiny shell script - Don't worry it's only an experiment - which will do what we imagined in the previous paragraph:

#!/usr/bin/env sh
set -eu

# Install bump-cli (published as a Ruby gem)
gem install bump-cli

# For each file from the available `*.json` files
for api in spec/json/*.json; do
    # Extract the API name from filename `twilio_<api_name>_vN.json`
    apiName="${api%_*}" # remove everything after the last '_'
    apiName="${apiName#*_}" # remove everything before the first '_'

    # Auto-create documentation <apiName> from the api definition file
    # in the 'twilio' hub
    bump deploy \
      --doc "${apiName}" --token "${BUMP_TOKEN}" \
      --hub twilio --auto-create \
      "${api}"
done

The bump deploy command is where all the magic happens: it will push the api definition file to generate its documentation on Bump. By passing the --auto-create flag together with the --hub twilio and --doc ${apiName} flags we are going to create a documentation called ${apiName} if it doesn't exist. On all future changes of the definition files, it will simply be updated.

And now the Github Action workflow to use the script above on each commit of the main branch of our git repository:

name: Deploy twilio API documentations on Bump

on:
  push:
    branches:
      - main

jobs:
  deploy-doc:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - uses: ruby/setup-ruby@v1
        with:
          ruby-version: 2.7
          bundler-cache: true
      - run: sh ./.github/scripts/deploy-all.sh
        env:
          BUMP_TOKEN: ${{ secrets.BUMP_TOKEN }}

We are almost there! You might notice we are using a secrets.BUMP_TOKEN variable which needs to be added in the settings of my Github repository. Let's get our hub access token from Bump.

Creating the Twilio hub page on Bump.sh

OK let's leave the terminal and head to my Bump.sh account in order to create the hub. I'm headed to bump.sh/hubs/new to do so.

Now my hub is created, I can retrieve my hub token from my hub Settings > CI deployment page

And adding the token to my Github repository secrets at the following location: https://github.com/paulRbr/twilio-oai/settings/secrets/actions (Don't forget to change your project namespace in the URL if you're trying it out)

We are now ready to push our two files - the sh script and the Github Action workflow - of the previous paragraph.

Shall we try to git push origin main?

The screenshot above shows the Github Action logs that ran successfully on my main branch after pushing. We can see everything seems to have worked as expected.

Refining the resulting Twilio API reference hub

It's live! https://bump.sh/hub/twilio/ 🎉

The result is already pretty nice. We can browse each API documentation and have a good overview of what Twilio's API can offer. Let's see what we can change on Bump and in Twilio's definition files to make it even better.

Grouping by categories and UI customisation

Here are some of the configuration option I changed in my hub settings page:

Select the grouping mode of API docs to “by category” from the hub Settings > Customize UI page
Adding a category to each documentation from their respective Settings > Hubs settings page
- Moving the main api documentation into a dedicated category called “Main”
- Moving all docs containing the word TODO: into a dedicated “Todo” category (Might be useful for the Twilio dev team to track which api definition files needs attention)
- Moving all tools related docs into a category “Tools”
- All the other docs will land in a category that I named “Uncategorized”
Changing the hub defaults from the Default settings page, which will apply to all documentations of that hub
- Uploading Twilio's logo
- Change the color scheme to red
- Change default navigation to Groups and operations with verbs to display all API operations with their associated HTTP verb in the navigation bar of each documentation page

Updating the api definition files

Now the hub is looking all good. We can get back to API definitions in our favorite text editor.

To demonstrate the automatic documentation updates I have commited some changes on all API *.json files. The goal was to change their description fields and mark all docs as “unofficial” to not confuse end-users.

After 1min our Github Action workflow reports a successful run. Meaning our hub and all subsequent documenation pages have been updated.

https://bump.sh/hub/twilio/

That's about it. If you have a set of different APIs and need a centralised public page for your docs Bump can help you out. Give it a try and let us know what you think or how we can improve to better fit your needs.

Have fun, stay free and stay kind.

✨