DEV Community: Yuliya Bagriy

Using openapi-cli: custom rules

Yuliya Bagriy — Mon, 06 Sep 2021 20:35:43 +0000

If you’re planning to lint OpenAPI description documents (you should!), always check whether a linter supports adding custom rules. And I mean not just changing severity or disabling predefined rules, but actually adding new ones specific to your API standards.

openapi-cli, as any respectable OpenAPI linter, allows that. The process is very similar to adding preprocessors.

Example: enforcing case style

Let’s look at our previous enumeration example:

state:
    type: string
    nullable: true
    x-aviskase-enum:
      - OK
      - DESTROYED
      - BURIED

Typically, there are guidelines for case style of enumeration values. Imagine that our rule is: all values in enumerations and extensible enumerations MUST be CAPITAL_CASE.

Adding a rule

First, here is our rule file ./plugins/rules/uppercase-schema-enums.js:

//@ts-check
module.exports = UppercaseSchemaEnums;

/** @type { import('@redocly/openapi-core/src/visitors').Oas3Rule } */
function UppercaseSchemaEnums(options) {
  const enumProperties = ['enum', ...(options.enumLikeProperties || [])];

  return {
    Schema: {
      skip(node) {
        return node.type !== 'string';
      },
      enter(node, { report, location }) {
        for (const prop of enumProperties) {
          if (node[prop] !== undefined) {
            if (node[prop].some(s => s !== s.toUpperCase())) {
              return report({ message: `All enum values should be uppercase`, location: location.child(prop) });
            }
          }
        }
      }
    }
  };  
}

Export a rule function UppercaseSchemaEnums that can be configured with enumeration-like properties in addition to the standard enum.
All rules (like preprocessors) should return a visitor. We use Schema visitor with two methods:
- skip method to ensure that we check enumerations only for properties with type: string
- normal enter method to check the rule
To report a problem we use the second argument of enter method: context. It’s destructured to report and location:
- report is a function to report a problem (duh!)
- location has several properties and methods, in our example we use location.child() to ask reporter to pinpoint the problem to the node’s child location. Thus, the problem will be pointed to state.x-aviskase-enum instead of simply state.

Enabling the rule via a custom plugin

Similarly to the preprocessor plugin, we keep all custom rules bundled as a plugin in ./plugins/custom-rules.js:

//@ts-check
const UppercaseSchemaEnums = require('./rules/uppercase-schema-enums');
const id = 'custom-rules';

/** @type { import('@redocly/openapi-core/src/config/config').CustomRulesConfig } */
const rules = {
  oas3: {
    'uppercase-schema-enums': UppercaseSchemaEnums,
  },
};

module.exports = {
  id,
  rules,
};

Next, enable the plugin and configure the rule in the .redocly.yaml file:

...
  plugins:
    - './plugins/custom-preprocessors.js'
    - './plugins/custom-rules.js'
...
  rules:
    custom-rules/uppercase-schema-enums:
      severity: error
      enumLikeProperties:
        - x-aviskase-enum
...

The final result is in commit 45a99c7.

Example: checking `description` property style

Let’s add one more housekeeping rule: all description properties should start with a capital letter and end with a punctuation mark.

//@ts-check
module.exports = DescriptionStyle;

/** @type { import('@redocly/openapi-core/src/visitors').Oas3Rule } */
function DescriptionStyle() {
  return {
    Info: checkStyle(),
    Server: checkStyle(), 
    ServerVariable: checkStyle(),
    PathItem: checkStyle(),
    Operation: checkStyle(), 
    ExternalDocs: checkStyle(),
    Parameter: checkStyle(),
    RequestBody: checkStyle(),
    Response: checkStyle(),
    Example: checkStyle(),
    Header: checkStyle(),
    Tag: checkStyle(),
    Schema: checkStyle(),
    SecurityScheme: checkStyle(),
  };  
}

function checkStyle(attribute = 'description') {
  return {
    skip(node) {
      return typeof node[attribute] !== 'string';
    },
    enter(node, { report, location, type }) {
      const loc = location.child([attribute]);

      const content = node[attribute].trim();
      if (content.length === 0){
        report({message: `The ${type.name} description should not be empty string.`, location: loc});
      }

      const firstChar = content.slice(0,1);
      if (firstChar !== firstChar.toLocaleUpperCase()){
        report({message: `The ${type.name} description should start with capital letter.`, location: loc});
      }

      const lastChar = content.slice(-1);
      if (!(['.', '!'].includes(lastChar))) {
        const suggest = `"<...>${content.slice(-10)}."`;
        report({message: `The ${type.name} description should end with punctuation.`, location:loc, suggest: [suggest]});
      }
    }
  };
}

description property can be present in several objects. Thus, we target several visitors at once.
report can also show suggestions: here we show a possible fix for the punctuation problem.

Don’t forget to enable it in .redocly.yaml. Here is what we have after adding the rule and fixing most of the errors: commit 33e6a46

But there are still two errors to fix:

validating /openapi/gates.yaml...
[1] openapi/components/schemas/Stargate.yaml:10:5 at #/properties/state/description

The Schema description should end with punctuation.

Did you mean: "<...>, `BURIED`." ?

 8 | - nullable: true
 9 | state:
10 | type: string
11 | nullable: true
 … | < 4 more lines >
16 | environment:
17 | type: string
18 | description: Last known place where gate is situated.

Error was generated by the custom-rules/description-style rule.

[2] openapi/components/schemas/Stargate.yaml:18:18 at #/properties/environment/description

The Schema description should end with punctuation.

Did you mean: "<...>DE_OBJECT`." ?

16 | environment:
17 | type: string
18 | description: Last known place where gate is situated.
19 | nullable: true
20 | x-aviskase-enum:

Error was generated by the custom-rules/description-style rule.

Hmm, looks confusing. Thanks to suggest we can at least see that something was added to the original description… Oh, right, that was our preprocessor! That’s why I suggested to use it in the first place: if we were to modify descriptions via decorators, the rule wouldn’t be able to catch this problem.

Finally, to make linter passing, we need to fix the preprocessor plugin code commit 81a4332.

Rules are cooler than preprocessors and decorators: they support nested visitors. I’ll cover this concept in the next article. Until then, try to come up with an example where the uppercase rule in its current form might have undesired behavior.

Using openapi-cli: custom preprocessing

Yuliya Bagriy — Tue, 17 Aug 2021 03:44:37 +0000

The key feature of openapi-cli is its extensibility. There are three ways to extend it: preprocessors, rules, and decorators. In comparison, Spectral supports only custom rules.

In this tutorial, we’ll start with preprocessors. They are used to transform OpenAPI description document before validation and linting. Mind you, documentation says they should be avoided, because custom preprocessors tend to be error prone.

Example: extensible enumerations

In our Stargate Network API there is a Stargate.yaml schema with two properties defined with enumerations: state and environment. We might think that we captured all potential values, but we can’t be sure.

Zalando API guidelines and The Design of Web APIs by Arnaud Lauret warn us that enumerations can cause API compatibility breaks, especially with outputs.

Imagine the scenario. We generated SDK for the initial version of the API, and the generator was smart enough to use enumerations in the selected programming language. Some time later we push a new API version where environment can also be STAR. Customers, who are still on the first version of the SDK, won’t be able to process API responses containing this value, because most probably their clients will crash with something-something-uprocessable-entity exceptions.

Thus, similarly to Zalando, we will use a custom vendor extension to document possible values. I’m calling it x-aviskase-enum just in case not to introduce conflicts with other extensions. Commit fa78766.

Surfacing values in the documentation

Oops, now there is no way to see known values in the generated reference docs. The manual fix is to add these values to the description. But this is too cumbersome, especially if you’ll ever want to change presentation format.

Preprocessor for the rescue! What we want it to do:

Find all properties with x-aviskase-enum property.
Grab the list of values, format them nicely, and add it to the description. Make sure not to overwrite existing description if it’s already present!

Adding a preprocessor

Let’s create a preprocessor first ./plugins/preprocessors/add-enum-to-description.js:

//@ts-check
module.exports = AddEnumToDescription;

/** @type { import('@redocly/openapi-core/src/visitors').Oas3Preprocessor } */
function AddEnumToDescription(options) {
  const vendorExtension = options.vendorExtension || 'x-enum';
  return {
    Schema(schema) {
      if (schema[vendorExtension]) {
        schema.description = appendToDescription(schema[vendorExtension], schema.description);
      }
    },
  };
}

function appendToDescription(values, description) {
  let additionalDescription = `Possible values: ${values.map(v => `\`${v}\``).join(', ')}`;
  if (description) {
    return `${description}\n\n${additionalDescription}`;
  }
  return additionalDescription;
}

What’s happening here:

We export a preprocessor function AddEnumToDescription that accepts configuration options:
- vendorExtension option is used to define extension name for extensible enumeration. If not provided, it equals to x-enum.
All openapi-cli preprocessors should return a visitor. In our case, we specifically indicate to take into account only schema nodes.
If a schema has defined extensible enumeration property, we update its description with formatted list of values.

Note: Per documentation, for type support you can use import('@redocly/openapi-cli').Oas3Preprocessor}. It never worked for me, so I use a “full path” import('@redocly/openapi-core/src/visitors').Oas3Preprocessor.

Enabling our preprocessor via custom plugin

Let’s keep our preprocessor (and the future ones) bundled inside one plugin via ./plugins/custom-preprocessors.js:

//@ts-check
const AddEnumToDescription = require('./preprocessors/add-enum-to-description');
const id = 'custom-preprocessors';

/** @type { import('@redocly/openapi-core/src/config/config').PreprocessorsConfig } */
const preprocessors = {
  oas3: {
    'add-x-enum-to-description': AddEnumToDescription,
  },
};

module.exports = {
  id,
  preprocessors,
};

Here we defined plugin’s id and a mapping between preprocessor’s name (add-x-enum-to-description) and its implementation.

Then we need to add this custom plugin to the .redocly.yaml and to enable the preprocessor:

...
lint:
  plugins:
    - './plugins/custom-preprocessors.js'
  preprocessors:
    custom-preprocessors/add-x-enum-to-description:
      vendorExtension: x-aviskase-enum
...

The final result is in commit 6654926.

How it looks in the docs

Now, when we render a reference documentation, we can see all values with simple markdown formatting.

By the way, can you spot operations where we could have left normal enum and why?

Why use a preprocessor here?

As mentioned before, preprocessors might be brittle. We could have used decorator instead. I’ll show you the reason in the next article, but here is a sneak peek: we want to be able to lint modified descriptions!

Using openapi-cli during API design: part two

Yuliya Bagriy — Tue, 23 Mar 2021 04:30:57 +0000

Part one showed the basics of using openapi-cli for a single-definition project. Let’s see how it works for multi-definition projects.

Multi-what?

By multi-definition project I mean a project with several OpenAPI description documents. For example, it can be useful if you want to keep contracts for all services in one place. Then, you can reuse common schemas between definitions.

Another use case is keeping parts of your APIs hidden from the final description document (internal and external APIs). This approach will mostly work only if separation is on path-level (see more here).

Configuration

In our small Stargate API example we are splitting gates and addresses APIs into two description docs (commit). Next step is to change configuration file, so that openapi-cli would know that we have two definitions (commit).

apiDefinitions:
  gates: openapi/gates.yaml
  addresses: openapi/addresses.yaml

But now we need to fix our scripts in the package.json so that we could work with both definitions!

Preview

If you simply run openapi preview-docs as we did before, it will start server only for the first definition in config file (in our case: gates). To preview a specific definition, you need to pass a definition label from configuration file: openapi preview-docs addresses.

But what if we want to preview both definitions? The simple fix will be adding separate scripts to run each definition on different ports:

"scripts": {
    "preview:gates": "openapi preview-docs -p 8080 gates",
    "preview:addresses": "openapi preview-docs -p 8081 addresses",
    ...
}

Then, run both commands (either setting them to background processes or in the different terminals).

Linting

Nothing to change here! openapi lint without any definition labels will check all of them.

Bundling

Remember how we had to use -o dist/openapi to properly generate bundled desciption doc for single-definition project? Now we should simply leave it as -o dist. This will generate dist/gates.yaml and dist/addresses.yaml based on definitions labels.

Generating reference docs

To generate static reference docs we should split npm run docs into separate scripts to target each definition:

"scripts": {
    ...
    "docs:gates": "node plugins/prepareOptions.js && redoc-cli bundle dist/gates.yaml -o dist/redoc-gates.html --options .redoc.json",
    "docs:addresses": "node plugins/prepareOptions.js && redoc-cli bundle dist/addresses.yaml -o dist/redoc-addresses.html --options .redoc.json",
    "docs": "npm run docs:gates && npm run docs:addresses",
    ...
}

There is no need to run prepareOptions.js twice, but I’m keeping it in case I’d want to generate only one reference doc.

The final working package.json is here. As you can see, we have to manually manage some building processes, but the most important pieces (linting & bundling) are quite transparent.

In the next article we’re gonna finally see juicy linting magic!

Using openapi-cli during API design: part one

Yuliya Bagriy — Mon, 01 Feb 2021 01:57:57 +0000

Obviously, API design is much more than writing OpenAPI description doc. First of all, should it even be OpenAPI-based? If yes, using openapi-cli will make your life a bit easier.

Example API

Let’s imagine you’ve been asked to create an API for Stargate Network and you already did some preliminary analysis.

Stargate network consists of points in space identifiable via special addresses. When you dial an address (like a phone number), a wormhole is established between your gate and a gate at the address. Gates are physical objects, so they can be destroyed or moved to another address.

First, resources:

Address
- Id
- Is the address accessible? Can we dial it?
- last known position in human terms (some addresses aren’t fixed points in space)
- Symbols used to dial it
  - in galaxy and out of galaxy addresses
  - unique address if it assigned to the gate
- Available gates at this address
Stargate
- Id
- Environment: is it located above the surface, underwater, on the surface, or inside the ship?
- State: is the gate functional, destroyed, or buried?
- Address id

Next, operations. Nothing fancy, the bare minimum:

List all addresses
Get address info
Add new gate info
Get gate info
Update gate info

Note: Don’t forget, not all APIs should be OpenAPI-based. If I needed to design a real Stargate system, I would go with a combo of something like GraphQL for lookup and event-based APIs for anything else (perhaps described with AsyncAPI).

Let’s create a place to store our OpenAPI description doc: YAML-based, multi-file git repo. You can read more about why’s in the Redocly docs.

The example repo is on the GitHub. At each step of the tutorial you’ll see links to commits as Commit <commit_hash>.

Repository structure

We’re gonna use create-openapi-repo to generate the initial structure and then tweak it a bit. Run npx create-openapi-repo inside a directory where you want to generate API structure. One caveat to keep in mind: this tool will attempt to initialize a git repository and make a commit. Commit e5b40a5.

Let’s explore the generated files:

docs directory contains HTML template for reference doc. I’m deleting it, because I don’t need any custom template at the moment.
.gitignore set up to ignore our beloved node_modules and dist directory used for generating the final OpenAPI description doc.
.redocly.yaml is a configuration file for openapi-cli. Because I removed docs directory, I also need to remove referenceDocs.htmlTemplate option here.
Don’t forget to update the LICENSE.
package.json set up with the most common actions.
Take your time to read README.md. It explains what actions are available and provides a sample contribution guidelines. In the real project, you probably don’t want to have all guidelines there in one file.
And last, but not least, openapi directory. This is where your API definitions live. Almost all subdirectories have a relevant README.md explaining why these particular files were created as well as showing you other alternatives. I don’t want to copy-paste them here, so, go ahead and explore ;)

Now, let’s rewind the time. Stargate Network API contract is defined, commit c2b8d11. For more information about keeping your structure DRY read this and this. If you’re interested in reading/watching about it from me, just ping!

Note: The definitions are not state of art, even for this shallow design. Some problems with it were added consciously to simplify the example, others for fixing in later articles.

And I’m not even talking about how many details are omitted lore-wise!

Preview

In the package.json file there is already an action for previewing the docs: openapi preview-docs. To execute it, run in the terminal npm run start.

This starts a server with the generated Redoc reference documentation (docs). It watches changes from the disk; usually I keep it running while writing definitions and use it like a UI of Stoplight Studio or Insomnia Designer.

As you may (or may not) know, you can customize how Redoc looks like. This is done withreferenceDocs section in .redocly.yaml (docs). Beware, some options are available only for premium edition.

Options can be divided into two categories: those that affect a theme (colors, fonts) and those for disabling or enabling certain features. For example, requiredPropsFirst: true will display required properties before others:

See at commit c873f15.

Linting

From time to time you should lint with openapi lint (docs). Let’s execute npm run test.

➜ npm run test

> stargate-network@1.0.0 test /home/aviskase/Projects/openapi-cli-examples
> openapi lint

validating /home/aviskase/Projects/openapi-cli-examples/openapi/openapi.yaml...
[1] openapi/openapi.yaml:1:1 at #/

Servers must be present.

 1 | openapi: 3.0.3
 2 | info:
 … | < 30 more lines >
33 | $ref: components/securitySchemes/api_key.yaml
34 |

Error was generated by the no-empty-servers rule.

/home/aviskase/Projects/openapi-cli-examples/openapi/openapi.yaml: validated in 27ms

❌ Validation failed with 1 error.
run with `--generate-ignore-file` to add all problems to ignore file.

Oops! We have an error! Wait a second, no-empty-servers rule complains that we must have servers defined. But OpenAPI specification doesn’t state this as a required field, so our description doc is valid. Where does this error come from?

When .redocly.yaml was generated, it included lint section:

lint:
  extends:
    - recommended
  rules:
    no-unused-components: warning

The no-empty-servers comes from extending recommended set of rules. If it’s too strict, you may want to change to minimal. Or change it to all to get even more errors.

I prefer keeping recommended rules and explicitly disabling unneeded ones. Per docs you need to set no-empty-servers: off in rules section, commit ee2e3ff. Or you can change severity to warning.

You can read more about other built-in rules in the docs. In the later article I’ll show how to add your own custom rules.

Bundling

Most of the tools require single-file OpenAPI doc. That’s where bundling comes in. The package.json is preconfigured with the script to run openapi bundle -o dist which will save a generated file to the dist directory. Usually it’s enough, but in some cases you may want to run with --derefenreced flag to produce a file with internally inlined definitions (docs).

➜ npm run build

> stargate-network@1.0.0 build /home/aviskase/Projects/openapi-cli-examples
> openapi bundle -o dist

bundling /home/aviskase/Projects/openapi-cli-examples/openapi/openapi.yaml...
📦 Created a bundle for /home/aviskase/Projects/openapi-cli-examples/openapi/openapi.yaml at dist.yaml 28ms.

Hmm, what just happened? Instead of ./dist/<filename>.yaml we’ve got ./dist.yaml. Why?

Well, this is kinda bug or feature situation. If you check the code, this looks like an intended behavior, but nevertheless confusing. In short, if you have only one API definition, -o seems to be expecting file path. If you have multiple API definitions (more about that in the upcoming part two), this option can point to directory.

In our example we have one API definition, so to fix we should change the script either to -o dist/openapi or to -o dist/openapi.yaml. Both will work the same, as long as --ext equals to yaml by default. If you want to generate file in JSON format, you can use -o dist/openapi --ext json or -o dist/openapi.json. If you want both formats, you need to run commands separately, see at commit a4e1e23.

Generating reference doc

One of the features that openapi-cli doesn’t have, but I need all the time is to generate static Redoc html file. Yes, you’d want to invest in better solutions and processes at some point, but at smaller scale it’s ok to send a simple HTML file when someone from non-dev team wants to see the latest or prototype API thingy.

First, we need to install redoc-cli via npm i redoc-cli. Then, add a new script to package.json to generate an HTML file inside dist directory:

"docs": "redoc-cli bundle dist/openapi.yaml -o dist/redoc.html"

After running npm run docs and opening the generated file you’ll notice that all our custom options are gone. You see, redoc-cli doesn’t support .redocly.yaml configuration file. To fix this, add prepareOptions.js which reads referenceDocs from .redocly.yaml and saves it to .redoc.json Next, modify docs script to run preparation script and use .redoc.json:

"docs": "node plugins/prepareOptions.js && redoc-cli bundle dist/openapi.yaml -o dist/redoc.html --options .redoc.json"

See at commit ad6dacc.

IMHO

Personally, I prefer different names for scripts:

preview instead of start
lint instead of test
bundle instead of build
build for npm run lint && npm run bundle && npm run docs

Then, all I do is jump between npm run preview and npm run build.

See at commit 51c078e.

That’s it for today. In the part two, I’ll show how to manage multiple definitions per repository.

Using openapi-cli for API exploration

Yuliya Bagriy — Sun, 10 Jan 2021 19:28:31 +0000

As promised, let’s dive into the usage of openapi-cli. The first topic is semi-non-technical: API exploration. You might be interested if:

you’re a tech writer
you’re a tester
you don’t know what the heck API exploration is

What is API exploration

I use this term akin to exploratory testing. You can search for works of Cem Kaner, James Bach, Michael Bolton, Elisabeth Hendrickson, James Whittaker, but beware, that’s a nice little serpentarium of a field.

In the simplest words, it is a process of learning something about API I don’t know or have a limited information about. That’s it. The main contexts for this activity are:

Necessity. I want to use some API.
Curiosity. Just looking at how other APIs are done.
Assessment. I need to check APIs for certain characteristics.

There are many ways to perform exploration, countless tools and sources to use. In this article I’ll be looking only at OpenAPI description documents, because that’s where openapi-cli comes handy.

Note: When exploring unknown APIs, I run a globally installed package: npm i -g @redocly/openapi-cli. Then it is accessible in the terminal as openapi <args>.

`openapi stats`

Gathers some basic statistics for an OpenAPI description document. Can be in “stylish” format (as in the example below) or in JSON.

➜ openapi stats http://localhost:8888/api/v1/swagger.json
Document: http://localhost:8888/api/v1/swagger.json stats:

🚗 References: 12 
📦 External Documents: 0 
📈 Schemas: 12 
👉 Parameters: 9 
🔗 Links: 0 
➡️ Path Items: 4 
👷 Operations: 7 
🔖 Tags: 1

Nothing useful for me at the moment. The only use case that comes to mind is when you have a bunch of documents for an assessment and you’re not sure where to start. Then, you can check stats for each doc and proceed with the smallest one.

`openapi preview-docs`

This command is the primary reason I have openapi-cli installed as a global package: I use it all the time. The command accepts path or link to the existing OpenAPI description document and starts a local HTTP server with the generated Redoc reference doc.

➜ openapi preview-docs http://localhost/api/v1/swagger.json
Using Redoc community edition.
Login with openapi-cli login or use an enterprise license key to preview with the premium docs.

🔎 Preview server running at http://127.0.0.1:8080

👀 Watching http://localhost/api/v1/swagger.json and all related resources for changes

Bundling...

Created a bundle for http://localhost/api/v1/swagger.json successfully

Why you’d want that?

First, sometimes there is just an OpenAPI description document and nothing else. No documentation whatsoever. Rare, but I do stumble upon this with internal APIs.

The second reason is when there are some reference docs, but they don’t fit you.

For example, it’s a SwaggerUI, and you don’t like it, or the description doc has advanced features like oneOf or discriminator. In such case, SwaggerUI cannot generate fancy form-based presentation, so you have to read bare model definitions.

Or perhaps you have special needs. Lorna Jane Mitchell in her presentation on Delightful SDKs mentions that she often generates her own local reference docs because official documentations aren’t accessible. AFAIK, she uses Redoc and made several pull requests specifically for better accessibility support.

Note: You can generate reference docs with redoc-cli, but I prefer using openapi-cli as a one-stop shop. The difference is when you want to generate static HTML with your theme: right now you have to use a small hack from this issue.

`openapi split`

Reading a big OpenAPI description doc is cumbersome. Especially when you need to jump back and forth between references.

Split for the rescue! It’s an opposite of the bundling operation and separates references into several files. Though, it’s a bit smarter; for example, it splits paths into separate files too.

Warning! This command doesn’t support OpenAPI 2.

Let’s split a description doc into out directory: openapi split --outDir out openapi.json. Here is a sample result:

~/out 
➜ tree
.
├── components
│ ├── parameters
│ │ ├── pageNumber.yaml
│ │ ├── pageSize.yaml
│ │ └── resourceId.yaml
│ ├── requestBodies
│ │ ├── User.yaml
│ ├── responses
│ │ ├── Error.yaml
│ │ ├── Forbidden.yaml
│ │ ├── Ok.yaml
│ │ └── Unauthorized.yaml
│ └── schemas
│ ├── Id.yaml
│ ├── ResourceId.yaml
│ ├── UserCollection.yaml
│ └── User.yaml
├── openapi.yaml
└── paths
    ├── users@{id}.yaml
    └── users.yaml

Another reason to use split is when you’re not just exploring API, but you plan to work on it. For example, when you’re switching from annotations-driven to handcrafted approach; split creates a good starting point for refactoring.

`openapi lint`

You may want to do linting in assessment or curiosity contexts:

Assessment:
- Is this old API at least valid per OpenAPI standard or it’s complete garbage?
- We came up with new API guidelines. Do our existing APIs conform to it? How many don’t?
Curiosity:
- I wonder how many APIs out there follow this rule? Is it common and should we apply it in our API?

As you can see, the biggest benefits of openapi lint come when you use custom rules and configurations. This topic I’ll be covering in later articles, stay tuned.

If you want to read more about OpenAPI-based API exploration, I recommend Arnaud Lauret’s series of articles about jq.

Using openapi-cli: intro

Yuliya Bagriy — Mon, 21 Dec 2020 03:26:21 +0000

Before we jump into the usage of this mysterious tool with way too generic name, let me give you an introduction.

There are many tools for supporting API design via OpenAPI description documents. You can check the list here.

The most common tasks are:

dereferencing
linting
reference documentation preview and/or generation

I talked about dereferencing not long ago. In short, it’s needed to produce a final OpenAPI description doc from more DRY and granular schemas. You can read more about the process here. One of the tools that can help you is json-schema-ref-parser.

Linting is used to check OpenAPI description doc for errors and style guide violations. Spectral comes to mind; Arnaud Lauret has a great talk about using it for design reviews.

SwaggerUI is the most common choice for reference documentation generation. If you need something more aesthetically pleasing you can try Redoc or RapiDoc.

But there is a tool that combines all of these and more: welcome to openapi-cli (GitHub, docs). It supports both OpenAPI 2 (fka Swagger) and OpenAPI 3; support for 3.1 is coming soon. At the time of the writing, the tool’s version 1 is still in beta phase, so keep it in mind. But I’m fairly confident in using it anyway. Why? Two reasons.

First of all, it’s developed by the same team as the famous Redoc, so you already know how reference docs look like =) Redoc is famous. You cannot read a book or attend a conference without seeing it mentioned by someone. A lot of people use it, so it would make sense for a openapi-cli to be developed further.

The second reason is stated on the team’s site: “built because we needed it”. All tools developed by Redocly are used in the other company of its founder, Adam Altman, Rebilly. These tools weren’t created in vacuum; they are constantly dogfooded by the developers whose primary business is making good APIs. By the way, check their api definitions for an inspiration on how to structure description doc development.

So, in this series of articles I’m gonna write about my experience of using openapi-cli.

We gonna go through themes:

using openapi-cli for API exploration
basic API doc structuring and preparation (dereferencing, previewing, linting, handing off results)
advanced topic: preprocessing
advanced topic: custom linting rules
advanced topic: decorators

And while I’m writing next posts (hehe), install and play with it!

P.S. It’s a pity that Redocly team isn’t very active about advertising themselves. While everyone knows and praises Redoc, openapi-cli isn’t that well-known. Maybe the team wants to finish the beta first? Anyway, I was using it for almost a year, and I think it’s time to break the silence and give it some public love!

Crash course into API-related terminology

Yuliya Bagriy — Tue, 08 Dec 2020 01:59:28 +0000

Anyone trying to dive into the world of APIs is doomed to be confused by conflicting terminology. In this post I’m gonna touch upon the main definitions and processes. I already tried it once during lunch&learn session at work, but the time has come to revisit that old presentation in the written form.

One last note: the basis for my definitions goes from OpenAPI Glossary started by Phil Sturgeon.

Main definitions

Specification is some standard or RFC or whatever describing a particular format or protocol.

Schema (aka: data model) is a metadata describing the data type, formats, and validation rules. Schema can be defined for content body, header, path or query parameters, etc.

Hypermedia controls is an easier way to say HATEOAS. What’s that? It’s a special affordance that allows a client to traverse API without hardcoding links. The simplest example would be including pagination links in the response data. E.g., a client asked to get collection of items — the server responded with first N items and a link to the next page of results — the client can use the link to get the next page without constructing a link on their own.

{
    "items": [
        {"id": 1},
        {"id": 2}
    ],
    "links": {
        "next": { "href": "/api/items?cursor=ae12fb2" }
    }
}

Description (aka: definition, contract) is a metadata about API, its endpoints, resources, operations, headers, parameters, and etc.

API description document is a file that contains API description. It is usually written with a particular format from specifications like OpenAPI, WSDL, RAML.

API documentation is a collection of information pieces that allows clients to successfully use an API. It includes, but not limited to, an API reference documentation (which is often generated from description document). Other important pieces are: tutorials, concept explanations, registration helpers, SDKs.

Specifications (some of them)

There are tons of API-related specifications. I’ll cover only some.

OpenAPI is used to describe both the service model and the request/response body kinda based on JSON Schema. It was called Swagger before being donated to Linux Foundation. You probably noticed that this created a sore spot: people still call it Swagger, perhaps, because “OpenAPI” is a bit mouthful. Anyway, know that Swagger is just a part of the SwaggerHub branding of tooling like Swagger Editor or SwaggerUI.

Note: You may stumble upon people using the word “swagger” even in weirder situations:

as a synonym for “API”: “We developed swagger for creating users!”, “Use this swagger to export files!”
as a synonym for response body: “We make a request and the swagger we got back…”

JSON Schema is for describing an instance of JSON data. OpenAPI was using an extended subset of it, but this was fixed in OAS 3.1, and now they are fully compatible. Why does it matter? Well, JSON Schema have additional tooling for client-side validations or HATEOAS support.

JSON: API is an anti-bikeshedding spefication for building APIs. Imagine not inventing your own guidelines for naming and other patterns! A notable feature is ingrained support for HATEOAS.

AsyncAPI is like OpenAPI but for event-driven architectures. Think Kafka, AMQP, WebSocket.

Notable mentions: gRPC, GraphQL, OData, HAL, Siren.

Processes

API-first vs. API design-first vs. API code-first

Read this article for more details. Here is a TL;DR.

API-first is about how you approach your API as a product: not an afterthought on top of already created system, but thought through right from the start. The good example came recently from Obsidian’s devs:

For the developer the biggest issue is migrating an existing app that wasn’t designed with plugin/API in mind. Having experienced this first hand with our previous product, we designed Obsidian from the ground up with a plugin API from the first version.

API design-first means prototyping and testing a design before any coding is started. API code-first (or implementation-first) means having some code written even at the the design stage. Often though, a more restricted definition is used and it’s about the source of truth: description doc or code.

Design-first: “we write description doc first and then implement”.
Code-first: “we annotate code and generate a description doc from these annotations”.

Beware, as I said, this is a more restricted definition! The article I’ve linked uses a broader one.

Description doc validation vs. linting vs. preprocessing

API description doc is often written in machine-readable format, thus, we can manipulate it and check for errors.

Validation checks that description doc conforms to some specification.

Linting checks custom rules like style. For example: check that all paths are lowercase.

Preprocessing allows transforming the doc based on additional rules before using it in code or documentation generation. For example, hiding some endpoints from the published version or injecting descriptions from markdown files.

`$ref`

When working with OpenAPI or JSON Schema you will stumble upon $ref’s a great deal. These are pointers to JSON structures in different locations: inside the same file or to some other file:

For example, this reference points to the schema from the other file.

...
application/json:
  schema:
    type: object
    properties:
      data:
        $ref: ../components/schemas/Users/User.yaml

References help with structuring schemas, separating them into reusable pieces.

The process of looking for value of $ref is called resolution (or lookup). Not all tools and libraries play well with multi-file resolutions, so you may want to do one of these:

Bundling (aka: external inlining) replaces $ref’s to external files with internal references. This creates one file that is easier to share.
Dereferencing (aka: internal inlining) replaces $ref’s to external or internal data with actual data. The resulting file will have no $ref’s, but will be noticeably bigger.

This post was triggered by discussion with Areti Panou in AB Testing podcast’s slack. I hope it will help you!

DEV Community: Yuliya Bagriy

Using openapi-cli: custom rules

Example: enforcing case style

Adding a rule

Enabling the rule via a custom plugin

Example: checking description property style

Using openapi-cli: custom preprocessing

Example: extensible enumerations

Surfacing values in the documentation

Adding a preprocessor

Enabling our preprocessor via custom plugin

How it looks in the docs

Why use a preprocessor here?

Using openapi-cli during API design: part two

Multi-what?

Configuration

Preview

Linting

Bundling

Generating reference docs

Using openapi-cli during API design: part one

Example API

Repository structure

Preview

Linting

Bundling

Generating reference doc

IMHO

Using openapi-cli for API exploration

What is API exploration

openapi stats

openapi preview-docs

openapi split

openapi lint

Using openapi-cli: intro

Crash course into API-related terminology

Main definitions

Specifications (some of them)

Processes

API-first vs. API design-first vs. API code-first

Description doc validation vs. linting vs. preprocessing

$ref

Example: checking `description` property style

`openapi stats`

`openapi preview-docs`

`openapi split`

`openapi lint`

`$ref`