AsyncAPI for documentation and validation of event-driven architecture

In this blog post, you’ll read about what is AsyncAPI and how at relayr we use it for validation of documentation of messaging between dozens of Node.js services.

OpenAPI and Swagger

Let’s first talk a little about OpenAPI and Swagger. The Swagger specification was founded back in 2010 as a design specification for RESTful APIs. It got a lot of attention from the whole development community and within a few years, it was being used by many big companies. In 2015, the Swagger specification was donated to Linux Foundation and renamed to OpenAPI and is now managed by OpenAPI Initiative.

AsyncAPI

The Swagger specification was created in 2010. Nowadays, more and more companies are moving towards event-driven/message-driven and microservices architecture where multiple services communicate with each other over some messaging protocol, such as Kafka, RabbitMQ or even web sockets.

This brought the need to have a specification for messaging between services. The AsyncAPI specification solves this problem. It lets you define the schema for messages between services with any kind of protocol. I once called it "Swagger for Kafka" 😄. @fmvilas started the project in 2017 at Hitch and now works 24/7 for it 💪. We have around 16 backers on OpenCollective 🎉.

Let’s now talk about the advantages of AsyncAPI!

A contract between multiple parties

AsyncAPI provides a contract for communication between consumers and producers. In our case, there could be multiple consumers and producers so it’s a contract between all of them. It tells what should be included in the payload when a service produces a message and also tells the consumer about the properties in the message.

Understandable by humans and machines

As AsyncAPI is a schema definition, it is understandable and readable by both machines and humans. The schema can be written in YAML or JSON format which can be easily used with Node.js. It also provides a nice HTML or Markdown document which is human readable.

Testing and validation

AsyncAPI can be used for testing and validation of the payload. At relayr, we wrote the validation library asyncapi-validator which I describe in detail in the Architecture at relayr section of this article.

Documentation

One other big benefit of using AsyncAPI is that it provides nice documentation for all the messages in-between your services. You can tag them as you like and also define custom properties.

You can use the AsyncAPI Playground here: playground.asyncapi.io

In the above example, I have defined a simple userCreated event. It has the three properties id, userName and role, where id is the required field. The schema definition is almost the same as the Swagger file. In the screenshot, you can see the generated example and schema view, which is easily readable and understandable.

Architecture at relayr

At relayr, we have multiple services running with Node.js and communicating using Kafka. We receive data from various client devices through MQTT. A Node.js service receives all these messages and then publishes relative Kafka messages. We wanted to validate that clients are sending the right payload, so we wrote an AsyncAPI schema for MQTT messages and also one for Kafka messages.

asyncapi-validator

To validate incoming MQTT messages, I wrote the asyncapi-validator with which you can validate any message against its AsyncAPI schema definition. First, you have to load the AsyncAPI schema definition using the fromSource method and then you can validate any message with its key and payload using the validate method. Here is an example of validation for the schema we defined earlier for the userCreated event.

This validation will give the below error because id should be a UUID and userName should be an email.

We added the asyncapi-validator where we receive messages from the MQTT broker. If the messages pass the validation, we let them through. If they fail, we generate related error messages back to the device, so that the client can fix the payload. We also log the failures in our system for debugging.

AsyncAPI v2.0

I started working with the AsyncAPI core team. We recently released version 2.0 of the AsyncAPI schema which has a lot of improvements. We are looking for more contributors. There is a lot of development going on around the schema definition and tooling. For details, check asyncapi.io or join our slack.

The asyncapi-validator is also open source: npmjs.com/package/asyncapi-validator. If you have any questions, feel free to reach out to me on twitter: @waledashraf01

Comments

[Simran Kaur]

What's the most effective way to validate schemas on the consumer side when the schema of an outbound message changes due to the event envelope? I assume these changes aren't typically included by producer services in the JSON Schema payloads. Could you provide specific examples, perhaps for SNS or SQS?

[Lukasz Gornicki]

Hey man, great article. Can you please add also #asyncapi tag to it, there are few related posts already dev.to/t/asyncapi and I'm working on another one.

[Waleed Ashraf]

Done. 🎉
Good to know that we have #asyncapi tag now.