Handling API validation with OpenAPI (Swagger) documents in NodeJS.

#swagger #api #openapi #node

This article was originally posted on my Medium blog.

I always found the hardest thing about API work was the documentation.

Sure, there are loads of nice tools out there to help you define it, provide nice front-ends and the like, but maintaining that isn’t anywhere nearly as fun as getting the actual work done. So soon enough, you’ve got stale documentation with little errors, and validation rules that don’t quite match up.

A recent NodeJS API project came my way which had out-of-date OpenAPI 3 documentation for the few endpoints it already had, but the understanding that we where going to start using it a lot more, so it needed to get up to scratch.

OpenAPI Specification (formerly Swagger Specification) is an API description format for REST APIs

I figured that if we where going to maintain this OpenAPI spec, which contained all the validation rules for the endpoints, then there must be a way we could use that to save us some time.

What if we could use that spec to enforce the validation? What if we could use it as the basis of the endpoint testing?

If we could get these two things, we have the wonderful combo of the OpenAPI spec needing to be write for the validation to work, and the validation being unable to deviate from the spec — so no more dodgy documentation where that param is documented as an int but it’s actually a float..

.. and if we can build tests based from the documentation then all our outputs have to be as defined, so the consumers of the API don’t get urked if we send an object and they’re expecting an array.

Using the OpenAPI spec to enforce the validation and be the crux of the tests enforces good definition of the API and removes all the nasty little ‘Ohh yea, that only returns X if Y’ that plagues API development IMHO.

So let’s stop waffling here and create something simple to prove how this works.

First we’re going to spec our endpoint. To save some time, I’ve used one of the sample specifications as a base. There’s a very nice editor / visualization tool at https://editor.swagger.io/ to work with your spec files.

Here’s the sub-set of the specification we’re going to look at:

	/{dataset}/{version}/records:
	post:
	summary: 'Provides search capability.'
	parameters:
	- name: version
	in: path
	description: Version of the dataset.
	required: true
	schema:
	type: string
	default: v1
	- name: dataset
	in: path
	description: 'Name of the dataset.'
	required: true
	schema:
	type: string
	default: oa_citations
	requestBody:
	content:
	application/json:
	schema:
	type: object
	properties:
	criteria:
	type: string
	description: 'Search string'
	start:
	description: Starting record number.
	type: integer
	minimum: 0
	rows:
	type: integer
	default: 100
	minimum: 1
	maximum: 100
	required:
	- criteria
	responses:
	'200':
	description: successful operation
	content:
	application/json:
	schema:
	type: array
	items:
	type: object
	properties:
	id:
	type: number
	example: 4011,
	title:
	type: string
	example: 'Latest report 2019'
	created:
	type: string
	format: date-time
	archived:
	type: boolean
	default: false
	ISBN:
	type: string
	pattern: 'ISBN\x20(?=.{13}$)\d{1,5}([- ])\d{1,7}\1\d{1,6}\1(\d\|X)$'
	example: 'ISBN 1-56389-016-X'

	'404':
	description: No matching record found for the given criteria.

view raw api.spec.yaml hosted with ❤ by GitHub

An endpoint that expects two variables in the path, {dataset} and {version} that are both strings. There are three possible variables in the post body also, one of which is required. It has two responses, a 200 that returns an array of records, and a 404. The response has a bunch of criteria also.

Let’s store this thing as /spec/api.spec.yaml

Now, a quick build of an Express app to handle responses to the path as documented:

	const express = require('express')
	const app = express()
	const port = 3000

	app.use(express.json());

	app.post('/:dataset/:version/records',(req, res, next) => {
	const {
	dataset,
	version,
	} = req.params
	res.send(`Should handle the search for ${dataset} / ${version} : ${JSON.stringify(req.body)}`)
	})

	app.listen(port, () => console.log('App listening on port ' + port))

view raw index.js hosted with ❤ by GitHub

This is as simple as it gets. So lets run it and check out if it works in Postman.

As expected, our API responds by telling us what we told it.

This all so-far-so-normal. Lets add the good stuff. Looking at the spec, we should now start adding validation into the endpoint we’ve just created — ensuring all those numbers are numbers, that the criteria is there etc.. But we don’t want to do that as we’ve already spent the time writing that all into the spec.

We’re going to install a node module called express-openapi-validate ( along with js-yaml) to handle this for us. Now we’ve got that installed, let’s change up the code a little:

	const express = require('express')
	const fs = require('fs')
	const jsYaml = require("js-yaml");
	const { OpenApiValidator } = require("express-openapi-validate");

	const app = express()
	const port = 3000

	// Load the validator and the spec
	const openApiDocument = jsYaml.safeLoad(
	fs.readFileSync("./spec/api.spec.yaml", "utf-8")
	);

	// Construct the validator with some basic options
	const validator = new OpenApiValidator(openApiDocument,
	{
	ajvOptions: {
	allErrors: true,
	removeAdditional: "all",
	}
	}
	);

	// we need JSON
	app.use(express.json());

	// Our post endpoint
	app.post('/:dataset/:version/records',
	validator.validate("post", '/{dataset}/{version}/records'),
	(req, res, next) => {
	const {
	dataset,
	version,
	} = req.params
	res.send(`Should handle the search for ${dataset} / ${version} : ${JSON.stringify(req.body)}`)
	})

	// add some error handling
	app.use((err, req, res, next) => {
	const statusCode = err.statusCode \|\| 500;
	res.status(statusCode).json({
	error: {
	name: err.name,
	message: err.message,
	data: err.data,
	},
	});
	});

	// listen to the port
	app.listen(port, () => console.log('App listening on port ' + port))

view raw index.js hosted with ❤ by GitHub

Lost more going on here!

We’ve loaded the app.spec.yaml file and we’re creating an OpenApiValidator object with it, along with some interesting looking options. The options are all inherited from the parsing app ajv . Our two are :

allErrors: true, // makes it return all errors, not just the 1st
removeAdditional: "all", // Removes any undocumented params

We’re validating the spec against the request as middleware, where we’re telling it what method we’re looking for and the path, and we’ve added some error handling to give us something to display if it doesn’t all go according to plan.

Let’s mess up our request, and try again.

What, an API that returns with useful error messages?!

Okay! We’ve just added validation against the OpenAPI spec! It’s capturing the two things I broke: The removal of the required field criteria , and the incorrect type of .body.rows . It’s not a very graceful error message, but it’s telling the consumer what’s gone wrong, and you’ve not had to write any of it. It’s also returning the right status code of 400 for us. Sweet!

Let’s fix the request and try once again.

All looks as before.. but it’s stripped out the foo: "bar" from the body, because it wasn’t documented. The validator stripped it out because it was undocumented. No more sneaking in properties in post bodies and not telling anyone.

This means now, if you format your OpenAPI spec correctly, the data arrives to your code validated and correct. Now, I’m not saying it’s perfect — there is a known problem with trying to parse numbers in the path, and Express handles everything as a string, but it’s much faster than having to maintain the OpenAPI spec document -and- the validation on the endpoint.

I’m hoping that gives you enough grounding in how to approach this so you can start using your OpenAPI spec documents as the amazing resource that it is. Treat the spec well and it’ll not only provide you with documentation for the consumers of the API, but will also do a lot of the work for you.

The next thing to look at, which I’ll link to once I write it, is the other side of this, which is writing tests that ensure that the output of your API conforms to the OpenAPI spec — thus forcing you to write API responses that your consumers expect!

I would love to hear how you use this in your projects! Get me on https://twitter.com/Scampiuk