In conversations with software developers, I’ve noticed that most of them claim to be API-first in their API development strategy. In fact, Postman’s 2021 State of API Report found that 67% of respondents ranked themselves with a score of 5 or higher (on a scale of 0 to 10) in terms of embracing API-first development.
In the past years, I’ve helped many teams build and deliver APIs. Most of them claimed to be API-first. But as it turned out, being API-first took on a different meaning in every project. This isn’t surprising. Postman’s 2021 State of the API Report includes the following definitions of API-first (with the percentage of respondents that subscribed them in parentheses):
- “Defining and designing APIs and schema before beginning development” (42% or respondents)
- “Developing APIs before developing applications or integrations” (31% of respondents)
- “Defining business requirements before defining and designing APIs” (16% of respondents)
The remaining 10% of respondents admitted they don’t know what API-first means.
Despite the diversity of perceptions around what API-first means, over the years I’ve noticed an increasing consolidation of the concept. When I’ve had the opportunity to work with the same team for a long period of time, I’ve noticed that over time the definition of API-first becomes more clear and solid. And as the concept of API-first matures, I also notice how API development methods become more sophisticated and standardized. In view of this, I’d like to propose a framework for analyzing API-first development strategies. I call it the API-first maturity framework (yes, inspired by Leonard Richardson).
I want to make it clear that this is a framework for analyzing API-first development strategies. I don’t analyze API-first strategies in general, but focus specifically on API-first development: the strategy you use to build your APIs. Since we’re talking API-first, the framework assumes that there’s always an element of API design before heading on to the implementation. However there’re no assumptions about how you design the API or how and when you document it. In fact, this framework considers different strategies for documenting APIs and evaluates them in terms of their benefits and constraints. The discussion also focuses mainly on REST since it’s the most prevalent type of API, but the same conclusions in my experience apply to GraphQL and other types of APIs.
In the early days of API development, very few people had a clear idea of how to design and document their APIs, and the API development workflow was full of hiccups. I used to say that no API server survives first contact with its client. And to an extent, this is still true even for the most advanced API development workflows. However, the benefit of a robust API development workflow is not to eliminate all integration errors, but to give you more control and observability of the errors and enable you to fix them more quickly.
The ideal is being able to deliver API integrations that work regardless of the complexity of the API and the integration. This means that both the API client and the API server development teams have a crystal clear idea of how the API works, and that both the API client and the server are implemented and validated against the API design. I assess the maturity of each API documentation methodology and each API development workflow in terms of their ability to help us reach this ideal.
I divide the analysis into two categories: 1) API modeling and documentation strategies and 2) API development workflows. For each category, I’ll discuss the most common strategies and workflows that I’ve found when working with different teams. Before heading over to the analysis, here’s the list of documentation strategies and development workflows that I’ll analyze:
API modeling and documentation strategies:
- JSON Examples
- Data modeling with programming languages
- OpenAPI
- Less standard API description languages
API development workflows:
- Server first, client later
- Code-first
- Bi-directional testing
- Developing the API client against manually crafted mocks
- Developing the API client against the spec using mock servers
- Validating the server against the spec using contract testing tools
I’ll begin by analyzing API modeling and documentation strategies and then I’ll analyze API development workflows. I’ll close the article with a summary and discussion.
API modeling and documentation
API modeling refers to the process of designing the API endpoints and schemas, while API documentation refers to the process of describing the API design. In this section, I analyze the most common strategies that I’ve come across for modeling and documenting APIs.
JSON Examples
With JSON Examples, we design the API endpoints and their associated payloads in the form of JSON examples. So instead of having schemas for the payloads, we have examples of what the payloads look like. This approach works fine for very simple APIs. However, APIs with complex schemas that include optional properties or properties with multiple types aren’t easy to document with JSON Examples - it means having different JSON examples for the same payload. And when the payload design changes, it means updating all the JSON examples. There are three big flaws in this method:
- Incomplete documentation: JSON Examples aren’t usually exhaustive and some cases are typically missing (specially for complex payloads); it’s also very easy to produce examples that contain mistakes.
- Maintenance burden: as the API design evolves, we need to update all the JSON examples. In my experience, it’s common to forget to update some examples, which ends up making the collection of examples unreliable.
- High noise-to-signal ratio: With multiple examples per payload, it’s difficult to get an idea of how the API actually works. You end up having to infer the schemas by collating multiple examples.
Data modeling with programming languages
This is a surprisingly common approach to API documentation. In this approach, developers enumerate the API endpoints and model their payloads as types using programming languages (such as TypeScript). The appeal of this approach is that the defined models are reusable straight away in API client and server code.
When properly done, this approach is quite powerful, as it allows you to define reusable models, optional properties, and properties with multiple types. It also has a nice and clear syntax that everyone (who’s familiar with the chosen language) understands. The downside is that we’re mixing data types with data schemas. An API schema is not exactly a type - it’s far more precise than that. A schema allows you to do things that you can’t do with types, like defining the minimum length of a string or array, describing dependencies between properties, and so on.
The better version of this strategy is when the models can be exported to JSON Schema or at least can be used to generate JSON Examples. Any update to the models can come with an update list of JSON Examples that API client developers can use for their testing.
OpenAPI
The gold standard. The OpenAPI specification was created to satisfy the documentation needs of REST APIs, and for all of its imperfections, it really does a great job of describing how an API works.
Unfortunately, this approach isn’t quite as common among self-claimed API-first developers as one would expect. Why is that? When I ask software developers why they don’t use OpenAPI to document their API designs, the most common answer is that they find OpenAPI difficult to understand and to work with.
Behind that reasoning, there seems to be an assumption here that you must write your OpenAPI specifications by hand. However, there really are many different ways to produce OpenAPI documentation: you can use a framework to generate documentation from code, or your can use one of the many tools that make it easier to produce OpenAPI documentation, like Postman’s API Builder, Insomnia’s Designer, or Stoplight’s Studio.
Using documentation-generation tools is very effective for those less familiar with OpenAPI. What I’d say is that you must check out the output of those tools to make sure it really aligns with the API design you want to achieve, and make any necessary amendments.
OpenAPI has a large community and a vast ecosystem of tools and frameworks that make it easier to design, build, test, and work with APIs. Once you have your OpenAPI documentation at hand, you can leverage this ecosystem and turbocharge your API development process.
Less standard API description languages
A common complaint about OpenAPI is that it’s difficult to learn and to read. Consequently, over the years we’ve seen many alternatives to OpenAPI, such as RAML, WADL, API Blueprint, and others. The problem with many of these alternatives is that in most cases they aren’t really more readable or easier to learn.
Simpler description languages also tend to support less capabilities for documenting API features. Finally, being less standard, those alternatives have smaller communities, and smaller ecosystems of tools and frameworks than OpenAPI, which makes for a poor development experience.
API development workflows
API development workflow refers to the process followed to build an API. When it comes to API development workflows, the biggest differentiator is whether you have an API specification in place before you begin implementing, or whether the API specification comes after you implemented the API. As I mentioned earlier, this analysis is for API-first strategies, so in all cases I assume that there has been an element of API design, with the exception of the “Server first, client later” strategy.
Server first, client later
In this approach, you build the API server first, and then you build the API client. In the early days of API development, this was a very common approach. Surprisingly, this approach still exists. The idea is that you don’t know what the API is going to look like or how it’s going to work until it’s built, and therefore it doesn’t make sense to build the client before the server is implemented.
This approach often goes hand in hand with a lack of API design and documentation practices. I remember following this approach many years ago, and it was most frustrating. Documentation was an afterthought of the API development process, and you’d often have to hack around the endpoints or dive into the server code to figure out how the API worked.
Sadly, I’ve also encountered this approach among teams that design and document their APIs first. The reason why client developers decide to wait for the server implementation even if the documentation is available is because they don’t trust that server developers will comply with the API specification, which means that the API server isn’t validated against the specification. The obvious risk of this approach is ending up with an API server implementation that doesn’t reflect the intended API design.
Code-First
Code-first generally fits within the concept of self-documenting APIs. This approach is very popular among backend developers. The idea is to use API development frameworks that generate API documentation from our code, and to publish that documentation.
Advocates of this approach claim that this is the only way to offer reliable API documentation and to keep it up to date, since it automatically updates itself when you change the code. The problem with this approach is that it presumes that the API implementation must be the source of truth. In practice, this approach means that the server isn’t checked for compliance against the API design.
This approach makes the lives of API client developers miserable, since new server releases can easily break the existing API integration. As an API client developer, it’s very frustrating to spend hours testing your code against the server, only for a new server release to introduce changes in the API. A recent post in Reddit illustrated well the resentment this approach creates among client developers. You can argue that better team communication can prevent this kind of problem. However, if you’ve been in software long enough, you know that communication isn't enough. You need to have automatic validation in place.
Another disadvantage of this approach is that API documentation generation tools are usually limited and sometimes can be wrong. For example, most frameworks don’t allow you to create OpenAPI links. Documenting security schemes is also challenging unless you use the framework’s specific security implementation, which may not satisfy your needs.
Bi-directional testing
This approach is popular among the contract-testing community and it was popularized by Pactflow. The idea is to implement the server with a self-documenting framework and to build the API client using manually crafted mocks for testing. Then you generate the API specification for the backend and the API specification for the client. Then you compare both specifications and see whether they agree. The nice thing about this approach is that it forces conciliation between the server and the client before you release the code.
The downside of this approach is that neither the server nor the client are directly implemented against the API design. Instead of using the API specification as the single source of truth, this approach lets the server and the client generate their own specifications, and it’s up to the developers to resolve the differences between them. This can cause unnecessary friction and delays in the API development process and can make discussions more difficult, involving specific implementation details of each side of the API.
Developing the API client against manually crafted mocks
In this approach, you manually create mocks to build the API client. The most traditional version of this approach involves creating mock responses in the form of JSON examples, while recent frameworks allow you to run mock servers based on those JSON examples and some additional custom configuration.
This is the most traditional approach to building API clients, but also the most common to this date, and sadly also the most error-prone. As we discussed earlier, JSON Examples tend to be incomplete and often wrong. They’re also a maintenance burden and become deprecated as soon as the API changes.
The biggest limitation of this approach is that it doesn’t take advantage of a full API specification, and as the API design changes, there’s a potentially growing drift between JSON examples and the actual API.
Developing the API client against the spec using mock servers
In this approach, you first produce an API specification, and then you run mock servers directly against the specification. Then you build the client against those mock servers. A mock server is a fake server that replicates the behavior of the real server based on the API specification. This is the best approach for building API clients, since it gives you the closest experience to interacting with and testing against the real API server. It’s a big advantage over traditional approaches like using manually crafted mock responses, since you don't need to create and maintain JSON Examples anymore.
In my experience, running API mock servers against the specification is very effective, although not all API designs are easily mockable. For example, APIs that return time series data in which the dates must be sequential and within certain bounds are difficult to mock. APIs with badly designed data models can be very difficult to mock as well (see examples in this article). In those cases, it’s still possible to leverage mock servers as long as you can provide examples or additional configuration.
There’re many different ways to launch API mock servers and this is a list of the most commonly used tools:
- Stoplight’s prism: the easiest way to launch a simple mock server locally.
- Microcks: an increasingly popular framework for running mock servers locally and in the cloud, with nice visualization dashboards and many other features.
- Postman: a popular choice for those who’re already familiar with Postman, allowing integration with existing collections.
- Wiremock/MockLab: a classic in the API mocking space, MockLab allows you to customize your mock server with your own stubs.
- (upcoming) microapis.io: in my previous work, I’ve always come across annoying limitations in the previously mentioned tools, like the inability to save results, add examples outside of the API documentation, leverage OpenAPI links, or obtain randomly successful and unsuccessful responses with varying payloads and response latencies. I'm currently building microapis.io’s mocker to address these limitations.
Validating the server against the spec using contract testing tools
In this approach, you produce an API specification first, then you build the API against the specification, and then you validate your implementation against the specification using automated API testing tools. This is the most reliable approach for building API servers, since it’s the only one that holds the server accountable and validates the implementation against the source of truth.
Unfortunately, this approach isn’t as common as it should be. One of the reasons why it isn’t so common is because it requires you to produce the API specification first, which, as we saw earlier, puts off many developers who don’t know how to work with OpenAPI. However, like I said before, generating OpenAPI specifications doesn’t need to be painful since you can use tools for that.
In this approach, you use automated API testing tools to validate your implementation. Tools like Dredd and schemathesis. These tools work by parsing your API specification and automatically generating tests that ensure your implementation complies with the specification. They look at every aspect of your API implementation, including use of headers, status codes, compliance with schemas, and so on. The most advanced of these tools at the moment is schemathesis, which I highly encourage you to check out.
Summary tables
The below tables summarize the analysis from the previous sections and scores the maturity of each strategy and workflow on a scale of 0 to 3, with 0 being the most immature and 3 being the most mature.
API documentation methods
Like I said before, OpenAPI is the gold standard for documenting REST APIs - it’s the most widely supported documentation format and it offers the highest degree of granularity for documenting API features, hence it gets the highest maturity score.
JSON Examples are the most basic form of API documentation. They tend to be incomplete and inaccurate, and they’re difficult to maintain, hence they represent the most immature type of API modeling and documentation.
Modeling with programming languages is better since it forces you to think about data models as opposed to specific examples, hence the score of 1.
Finally, using a proper API description language, even if it isn’t widely supported, is still better than using JSON Examples or programming language models, hence the score of 2.
| API doc method | Benefits | Challenges | Documentation format | Maturity |
|---|---|---|---|---|
| OpenAPI | Highly detailed docs | Learning OpenAPI | JSON or YAML | 3 |
| Less standard description languages | Very customizable docs | Smaller ecosystem | Custom file formats | 2 |
| Programming languages | Reusable code | Inability to represent schema constraints | Language-specific files | 1 |
| JSON Examples | Real payload examples | Maintenance burden | JSON | 0 |
API development workflows
Developing and validating against a unique source of truth is the most effective way for delivering reliable API integrations, hence why “API server tested against the API spec” and “API client tested against the API spec” get the highest rating.
On the other side of the spectrum, the “Server first, client later” strategy is the most ineffective and unreliable way to deliver API integrations since it doesn’t validate the server implementation and it rarely exposes reliable documentation, hence the score of 0.
Code-first and API client against manual mocks are sub-optimal since they don’t enforce a single source of truth, but at least they have a concept of API documentation, hence the score of 1.
Bi-directional testing forces us to conciliate the API client and server implementations, which is a neat feature to ensure reliable integrations. However the lack of a single source of truth for reference and validation causes friction and earns it the score of 2.
| API Workflow | Benefits | Challenges | Maturity |
|---|---|---|---|
| API server tested against the API spec | Ensures API server accurately reflects the spec | Producing the OpenAPI documentation | 3 |
| API client tested against the API spec | Accurate reflection of how the API server works | Some APIs are not easily mocked | 3 |
| Bi-directional testing | Forces conciliation between client and server | Doesn’t rely on a single source of truth | 2 |
| API client against manual mocks | Testing against payload examples | Non-comprehensive and quickly deprecated examples | 1 |
| Code-first | Accurately documented API implementation | Unaccounted/untested API changes | 1 |
| Server first, client later | You know how the API server works | Undocumented API changes | 0 |
Conclusions
APIs are fundamental components of modern platforms, and they give us access to many of services across the Internet. The sustainability of many businesses depends on having robust and reliable APIs. That’s why it’s important to invest in solid API modeling and documentation processes and development workflows. Hopefully, the discussion in this article has given you some insight into the different ways you can approach these tasks, and how they fit within the whole spectrum of choices.
Although I do have a preference towards the most mature API documentation and development workflows, I reckon it isn’t always easy or possible to get started straight away with them. Embracing API design, using the OpenAPI specification, and leveraging contract-testing tools require skills and knowledge that sometimes aren’t there. They also require an additional investment of time and resources that sometimes cannot be afforded. And in fairness, there are situations, when your APIs are very simple and not business-critical, in which you may be able to get away with plain JSON examples and simple development workflows.
The goal of the API-first development maturity framework is to make you aware of the benefits and constraints of each approach, so that given your specific circumstances, you can assess the risk of each approach and choose the strategy that best fits your needs.
If you enjoyed reading this article and found it useful, you’ll like my book Microservice APIs. It teaches you everything you need to know to build APIs from the ground up, including design, documentation, implementation, testing, security, and deployments!
You can download two chapters of the book for free from this link.
You can also use the following code to obtain a 40% discount when buying the book: slperalta.
I also run a very popular series of workshops, both free and paid ones, in which I teach best practices and advanced patterns for building microservices and APIs. I’d love to see you around in one of my workshops! An updated list of upcoming workshops is always available here: https://microapis.io/workshops and here https://www.eventbrite.co.uk/o/jose-haro-peralta-43771267993.
Top comments (0)