Good design lends itself to being Easier To Change [ETC]. However, this principle of ETC tends to get ignored when it comes to API documentation and service validation. Here the tenants of Do Not Repeat Yourself [DRY] are often neglected leaving services with multiple files potentially spanning hundreds if not thousands of lines of code with copious amounts of duplication.
Development in services with monolithic validation engines and swagger documents then become a form of tech-debt. As these engines and documents often live outside of the code surface area that is getting changed the probability of them becoming out of sync increases.
So What's The Fix?
I propose a new design pattern for developing your swagger documentation, and then letting your OpenAPI specification drive your validation.
With our mission statement above, let us make sure we are all on the same page with our tool-chain. The NodeJS and the JavaScript ecosystem being what it is, it is an important step to understand our end goals.
Service Documentation: Swagger 3.0 -- OpenAPI
Service Validation Engine: AJV
node-modules: swagger-jsdoc, openapi-validator-middleware
NodeJS Framework: Express
While I acknowledge other validations engines exist (JOI and express-validator to name a few) AJV lends itself to a simple JSON feed and one that people already have written OpenAPI wrappers for! As for NodeJS frameworks, I chose to use express because that is what I am more familiar with. There is no reason this wouldn't work with koa as the package openapi-validator-middleware even supports koa!
So How Exactly Are You Removing Duplication?
Each of the above packages has a specific goal.
With swagger-jsdoc we are going to adhere to the earlier statement of Easier To Change. We will co-locate our swagger definitions in the route files themselves. This will allow future developers to see specification living with the code, making it more obvious to them that when they change the code in the route, to change that specification.
openapi-validator-middleware has the ability to consume a generated OpenAPI Swagger document and use that for the validation engine. This package is a wrapper around AJV that allows us to have minimal code-changes for a large duplication removal.
So What's This Look Like?
So let us start with the validation piece, and for that, we take a peek at the file app.js where we describe our express app.
First things first then; let's import our module
const swaggerValidation = require('openapi-validator-middleware');
After it is imported, we simply need to point it at our Swagger doc to configure it.
swaggerValidation.init('swagger.yml');
With the validation engine configured with our swagger, we just need to enforce it in our route definitions as middleware.
api.get('/simple', swaggerValidation.validate, getSimple)
With those 3 lines of code, we have configured our validation engine, tweaked it to our swagger specification and it is now enforcing its rules against the /simple route. No longer do you have to maintain a separate file Joi/AJV file to maintain your service validations - cool huh?
OK, but about the swagger file? Won't that be monstrous now?
The answer is yes; because your swagger file will now have to have all your validation logic in it, it will be huge - but it should have had that info already. So with that in mind, we will let our other package swagger-jsdoc worry about maintaining the swagger file. Our goal here is Easier To Change remember? So we will co-locate our swagger definitions with our route file logic. With the code and documentation living in a single place when developers make changes they will hopefully be more encouraged to keep everything in sync. Not to mention any requirement to change the validation requirements of parameters/request-bodies instantly get reflected in the swagger doc as well.
So here's our get-simple.js that we defined earlier
/**
* @openapi
* /v1/acme:
* get:
* description: a simple get route that returns the `foo` query param
* parameters:
* - in: query
* name: foo
* schema:
* type: string
* minimum: 3
* responses:
* 200:
* description: a object witth the echoed query param.
* content:
* type: object
* properties:
* foo:
* type: string
* minimum: 3
*/
const getSimple = (req, res) => {
const { foo } = req.query;
return res.status(200).json({ foo });
};
module.exports = getSimple;
Wait I Have Some Questions!
Is that 20 lines of comments for a 4 line route file? And how come foo is duplicated I thought we were removing duplication?
To answer those questions yes, you will have a pretty large chunk of documentation here. It is inevitable, as we need to have the shell of swagger here, but it should help serve new developers looking at that file to know what the expectations for both the requests and responses are.
As for the duplication you saw, I'm getting to it! That was showing the duplication for ease. Using the features of YAML we can actually remove some of that duplication all the while compartmentalizing our definitions even more.
OK - just get to it, how do you do it?
Leveraging YAML anchors we can create variable-like atomic definitions of our fields. But first, let's scaffold out our service a bit more and make some files/directories.
mkdir swagger
touch swagger/first-name.yml
touch swagger/last-name.yml
touch swagger/user-id.yml
This swagger folder, as you can see, will contain all of our swagger component definitions. This will ensure our definitions remain consistent as they get used across the various routes while removing duplication as they can now all share a single-source of truth - this folder.
The Files
# swagger/first-name.yml
x-template:
firstName: &firstName
type: string
minimum: 1
maximum: 30
description: the first name of our acme user
# swagger/last-name.yml
x-template:
lastName: &lastName
type: string
minimum: 1
maximum: 30
description: the last name of our acme user
# swagger/user-id.yml
x-template:
userId: &userId
type: string
minimum: 4
maximum: 4
pattern: '[0-9]{4}'
description: the unique identifier of our acme user
With our swagger field-components created, let's spin up some new routes using our new fields!
put-create.js
/**
* @openapi
* /v1/acme/create:
* put:
* description: creates a fake user of the acme service
* requestBody:
* content:
* application/json:
* schema:
* type: object
* required:
* - firstName
* - lastName
* properties:
* firstName: *firstName
* lastName: *lastName
* responses:
* 200:
* description: a object with the echoed firstName, lastName and a random userId.
* content:
* type: object
* properties:
* firstName: *firstName
* lastName: *lastName
* userId: *userId
*/
const putCreate = (req, res) => {
const { firstName, lastName } = req.body;
const userId = Math.floor(1000 + Math.random() * 9000);
return res.status(200).json({ firstName, lastName, userId: `${userId}` });
};
module.exports = putCreate;
Look at that, we've made a more complicated request/response object and our total line count for the comments has 3 more lines! On top of that, even if you had no experience in the file you could determine its use-case and request/response contract by simply reading the first comment. See the Easier To Change benefits yet? Hypothetically if you had the requirement to allow for 60 character last names, you can simply change the swagger file last-name.yml and you would get both the Swagger Document updated as well as a validation rule in place enforcing it!
OK - I'm Sold, But How Do You Turn Those Comments Into A Swagger Doc?
swagger-generator.mjs
import fs from 'fs';
import swaggerJsdoc from 'swagger-jsdoc';
import { dirname } from 'path';
import { fileURLToPath } from 'url';
import packageJson from './package.json';
const __dirname = dirname(fileURLToPath(import.meta.url));
const options = {
format: '.yml',
definition: {
openapi: '3.0.0',
info: {
title: packageJson.name,
version: packageJson.version,
},
},
apis: ['./src/routes/*.js', './swagger/**/**.yml'], // files containing annotations
};
const runtime = async () => {
try {
const openapiSpecification = await swaggerJsdoc(options);
fs.writeFileSync(`${__dirname}/swagger.yml`, openapiSpecification);
} catch (e) {
console.log('broke', e);
}
};
runtime();
The above script is the magic that will generate the OpenAPI Specification and generate the swagger.yml that the validation engine will consume. To help enforce good practices, and because all developers (myself included) are bad at remembering things I personally leverage Husky to ensure this file is generated. This would done as a pre-commit hook that will run the above script followed by a git add swagger.yml command.
But how could you enforce that?
CI CI CI! Because we only have a pre-commit hook to generate our swagger.yml, there is a valid concern. After all, the only worse than no documentation is bad/out-of-date documentation.
What if a developer commits with a -n or simply makes the commit from the web-ui
Well let me start by saying that they are a monster (especially if they are committing with -n!). But to help enforce this, it should be a build step when creating/bundling your application. Right with the test cases, we can re-run the swaggerJsDoc command and compare its output directly against the swagger.yml output. Any differences and stop the execution.
Examples/References
Repo Showcasing this process:
ms-acme-openapi-ajv
Article Link: https://gem-ini.medium.com/de-duping-the-duplication-in-services-featuring-swagger-openapi-and-ajv-abd22c8c764e
The purpose of ths repo is to be an aide for the medium article. The code in this repo does not represent production-quality code, as such the individual code-sample should be taken with a grain of salt, but the pattern itself is what should be inspected.
The Pattern
This repo shows you how to co-locate your swagger docs with your express route files. With this co-location we go into having a pre-commit hook to generate the swagger output. This swagger output will then become the validation file that will protect your express routes (see article for more details)
Packages Used
Surnet
/
swagger-jsdoc
Generates swagger/openapi specification based on jsDoc comments and YAML files.
swagger-jsdoc
This library reads your JSDoc-annotated source code and generates an OpenAPI (Swagger) specification.
Getting started
Imagine having API files like these:
/**
* @openapi
* /:
* get:
* description: Welcome to swagger-jsdoc!
* responses:
* 200:
* description: Returns a mysterious string.
*/
app.get('/', (req, res) => {
res.send('Hello World!');
});
The library will take the contents of @openapi (or @swagger) with the following configuration:
const swaggerJsdoc = require('swagger-jsdoc');
const options = {
definition: {
openapi: '3.0.0',
info: {
title: 'Hello World',
version: '1.0.0',
},
},
apis: ['./src/routes*.js'], // files containing annotations as above
};
const openapiSpecification = swaggerJsdoc(options);
The resulting openapiSpecification will be a swagger…
PayU
/
openapi-validator-middleware
Input validation using Swagger (Open API) and ajv
openapi-validator-middleware
This package provides data validation within an Express, Koa or Fastify app according to a Swagger/OpenAPI definition. It uses Ajv under the hood for validation.
NOTICE: As this package gone through a long way, as we added support for OpenAPI definitions, while also adding support for more frameworks such as Koa and Fastify, we finally took the step of changing express-ajv-swagger-validation name to something that describes it better. As of now we'll be using the name openapi-validator-middleware instead.
There are no code changes in openapi-validator-middleware@2.0.0 compared to express-ajv-swagger-validation@1.2.0 apart from the name change.
Table of Contents
- openapi-validator-middleware
Installation
Install using the node package registry:
npm install --save openapi-validator-middleware
Then import the module in your…
Top comments (0)