Guidelines to build good RESTful APIs - Part 1: Introduction and Design Phase

Introduction

I have been working on RESTful APIs for several years at Rakuten in Tokyo using various languages and web frameworks. Their size ranged from tiny single-endpoint micro-services to big and complex ones. And most of them had pretty decent quality.

But I also met some APIs that were much more difficult to work with. Not because the problem they tried to solve was complex, but because they all had accumulation of issues during their development. Issues that could and should have been avoided because they weren't proper thoughtful trade-offs.

Software development in a company has its own challenges, namely resources allocation, schedules and deadlines, business requirements etc... The key to build good quality APIs even with those constraints is to properly choose your battles.

For that reason I would like to share a list of practices our team tries to apply every time we build a RESTful Web API. We will go through the following development lifecycle and point the particular areas that require focus.

Note: for readability reasons, the article will be split into several parts and the links to each of them will be updated once they are published.

• Part 1: Introduction and Design Phase - You are reading this 😀

• Part 2: Implementation Phase

• Part 3: Testing Phase

• Part 4: Ops, Ops, Ops! and Final Word

🗺 "Does anyone have a plan?"

First, a reminder about typical properties of most APIs:

• An API solely exists to deliver a SERVICE by being consumed by "clients". It is similar to services providers in real life, be it a shop, a restaurant, the postal services etc... You want to deliver a good service and the clients to be "happy". If you ever had to use and consume a third-party APIs in your projects, maybe you have experienced the following:
  • I wonder if there is an endpoint to request multiple results in a single request instead of sending a lot of single result requests?
  • Is it possible to filter the results? I don't need all these.
  • Do I really need to provide all those hundred parameters that looks like they could be optional?
  • I followed the example but I keep getting an error with no explanation.
  • Where is the documentation?
  • Why the naming is so inconsistent? It's confusing.

When you are the one building the API, you also want to ask yourself those same questions. You are not building an API for the sake of building it, someone will use it! It can be yourself, your teammates, another team in the company or even clients who are paying for your product.

• Modifying an API Contract is HARD, especially if your API is public. Any change on your API behavior can break the clients and their applications (it can cause crashes and affect millions of users). In practice, an API will always have to evolve to a certain degree. There are techniques to manage these changes like API versioning or backwards-compatibility support, but it will always be a tricky exercise.

That's why you WANT to spend time on designing your API. It's the phase where you can create early mistakes that may have a huge impact on your API usability. Those mistakes are also often the most difficult to fix at a later stage. The time spent can vary depending on the size of the API, if it's public facing with a lot of clients etc... You don't need 10 hours of design for a throw-away API only used for some prototype but keep that in mind.

⚠ Make sure you understand REST

If your answer to the question "What is REST for you?"

is just "it's about using GET, POST and DELETE".

It's probably a good idea to refresh your memory a bit by reading the following article: https://restfulapi.net/.

It's well written and concise. You also have more details if you want to dig a bit deeper.

I insist on having solid fundamentals because REST has a lot of misconceptions about what it does and doesn't enforce.

For example and contrary to the popular belief, REST doesn't tell you which HTTP method (GET, POST, PUT, DELETE) you should use for operations as REST is not even technically tied to HTTP.

It is therefore more productive to know exactly the constraints of the REST architecture, work with them and know which parts of the design require you to take more complex decisions.

Key-points

• It's bold but you NEED to know and understand REST to build good RESTful APIs.

• Having solid foundations will let you focus on the more custom (and more interesting) parts of your design.

💦 Naming and endpoint structure is hard but there are good practices out there

When designing your API contract, the first difficulty is to come up with a proper hierarchy for your endpoints and NAME them.

REST asks you to organize your data around Resources but doesn't enforce any specific way to create a proper hierarchy.

A large number of people wrongly relate resource methods to HTTP GET/PUT/POST/DELETE methods. Roy Fielding has never mentioned any recommendation around which method to be used in which condition. All he emphasizes is that it should be uniform interface. If you decide HTTP POST will be used for updating a resource – rather than most people recommend HTTP PUT – it’s alright and application interface will be RESTful.

In practice, most of the public APIs follow some conventions considered as "reasonable". By following the same conventions, you help other developers to understand your API faster as they are used to see those patterns. Here is a very good (and short) list of some popular naming conventions:

• REST Resource Naming Guide.

Among some of those naming guidelines:

• Use nouns to represent resources: api/my-resource/, api/my-other-resource, api/my-resource/{id}/sub-resource
• Avoid verbs especially if they are mirroring what the HTTP methods could perform (in an HTTP context): Avoid api/delete_my-resource and api/get_my-resource when you can simple have HTTP DELETE api/my-resource and HTTP GET api/my-resource
• BE CONSISTENT! You don't want the same parameters to have totally different names in 2 similar endpoints.

I would lie if I say that every time we design APIs in our team everything goes well. We sometimes have "passionate" discussions and 2 developers may propose different designs that are technically correct and elegant. As mentioned earlier, there are a lot of things REST doesn't enforce. At times like this, it's not worth wasting too much time on it. Pick one, and stick with this design philosophy for your entire API. The key is consistency.

One of my personal tip is to read the documentation and the API contracts of popular open APIs accessible on the Internet. It gives good examples and can be source of inspiration. Here are some of my favorites:

• Github API
• Auth0 API
• Marvel API

Key-points

• Using common naming conventions makes your API more predictable and easier to understand by other developers.

• You don't have to start from scratch and reinvent the wheel, a lot of people came up with those battle-tested naming and structure recommendations.

• REST is flexible enough to give you freedom but always plan wisely.

🔎 Always have your design proposition be reviewed by your peers

The "passionate" discussions I have mentioned earlier are not considered a bad thing in our team. Quite the opposite, we encourage challenges when designing new systems. Having various people and their own knowledge, vision and skills can help to find mistakes, typos but also fundamental design issues. These issues can be costly at a later stage of the development so you want to detect them as early as possible.

We started recently to involve not only developers in that review process but also our QA engineers.

The QA Team will be the first real client using your API. As such, it is in a formidable position to detect validation mistakes, ask more details about unclear specifications and make sure the product scenarios are covered.

• "What's the exact format of that parameter representing a date? A timestamp? An ISO 8601?"
• "What supposed to happen if no result is found?"
• "Are we okay with an error or do we prefer an empty response?"

Everyone's opinion is valuable as long as it's communicated in a constructive way. When you give a comment, have an objection or wish to give a proposition, always explain properly your reasoning.

Key-points

• More people and more point of views: Less blind spots.

• It's an excellent knowledge-transfer practice of the specs of the product when performed with a constructive mindset.

📃 Document your API

I know a lot of developers don't like writing it and find it boring. But if there is something you should not skip when building a Web API, it's the DOCUMENTATION. Again, the main purpose of an API is to be used. You don't want your clients to spend hours or days trying to figure out your endpoints location, the format of the parameters or the required headers.

The minimum should be the list of all endpoints with their:

• HTTP methods
• List of headers/path/query parameters with their validation
• The format of the body
• Response and error codes and response payloads schemas.

You can write your documentation with the OpenAPI specifications (formerly known as the Swagger Specification). It quite popular and there is an entire ecosystem of tools built on top of them. But as long as your documentation is accessible and you give enough information for your API to be used, it's the most important.

It's also easier to write the documentation as you design and review your API contract. It will serve as a proper reference for the implementation and the testing.

I suggest you to look at the documentation of the following APIs. You will notice that they go further than a simple endpoints list. They also describe the global concepts and the entities they are using and also have interactive parts:

• Stripe API
• Twilio

Key-points

• An API without documentation is almost useless.

• The documentation serves as a reference for the different team members and roles. This can help avoid misunderstandings.

• Having a documentation early will be useful on ALL the following steps of the development cycle.

⏩ Next part

Now that we have designed our API, it's time to write some code and make it real.

Part 2: Implementation Phase