Guidelines to build good RESTful APIs - Part 2: Implementation Phase

This current post is a 2nd part of a 4 articles series about our Guidelines to build good RESTful APIs.

You can access the other articles from the following links:

• Part 1: Introduction and Design Phase

• Part 2: Implementation Phase - You are reading this 😀

• Part 3: Testing Phase

• Part 4: Ops, Ops, Ops! and Final Word

✅ "Just Do It!"

Now that you have designed our API, it's coding time! I have selected a couple of good practices that can be applied independently of the language or web framework you may be using.

🍰 Layer your API

Except for Proofs of Concept or prototypes, layer your implementation from the start! You won't gain anything by not doing it.

Layering is a way to organize your code for separation of concerns and follows those rules:

• Each layer depends on the layers beneath it.

• A layer should have no knowledge about any layer above it.

Here is a layering we often use in our APIs:

• The Edge layer (alternative names: Controller/Presentation layer) hosting the API Controllers, the various helpers and middlewares. It's the layer in contact with the requests from the clients.
• The Logic layer (alternative names: Business/Application/Domain layer) usually implementing the business logic.
• The Data layer (alternative name: Persistence layer) that handle the data persistence.

If we look closer, we realize that the nature of the code of each layer is different:

The Edge layer for example is usually tightly coupled to the web framework you are using. The way controllers, middlewares, validators are implemented can be vastly different between 2 frameworks even if they are written in the same language. It also handles a lot things related to HTTP requests and responses.

On the other hand, the Logic Layer is often made of plain objects representing your business entities and code taking care of manipulating them.

As for the Data Layer, you are likely to import and use clients that "talk" with specific databases, external APIs or some type of storage. And each usually come with their own specific and complex objects (the way MongoDB and MySQL represent and expose queries and results objects to you are totally different for example).

The benefits of layering are then more obvious:

• Clearer scope for each layer in terms of role but also in terms of dependencies. It's unlikely that your Logic Layer will have to deal with HTTP Requests objects or HTTP codes for example.

• Easier to expand or modify one specific part of your code without impacting the rest.

• Improved reusability. For example, you can have multiple endpoints on the Edge Layer that use the same logic in the Logic Layer.

• Clearer testability. A layer like the Data Layer is usually more difficult to unit test than the Logic Layer. But because they are clearly separated, you have a more comprehensive view of the testing status of each layer and may decide to rely on unit tests for the Logic Layer and more on integration tests for the Data Layer.

One of the reason given by developers that use the "kitchen sink" approach is that their API is not complex enough to require layering and it's a waste of time. I don't buy that, it takes literally a few of seconds/minutes to separate your layers. And naming is the longest part of it. Of course just creating a bunch of folders is not enoughs. You have to think where each piece of code goes and make sure it stays organized thorough the development but it can still be a lightweight process. But it's nothing compared to the hours of refactoring a "simple" API with all the codebase in the controllers can cause once it has evolved into a "spaghetti mess".

Key-points

• Layering your codebase brings tons of benefits.

• Layer from the beginning, it costs nothing.

👻 The power of abstraction

Even if a layer creates some degree of isolation, it still needs to communicate with components located in lower layers. This communication can become messy if not controlled properly.

A solution we use in our team is proper abstraction and clear contracts between layers.

The goal is to avoid leaking implementation details.

I'll take a simplified example of a Data Layer having to interact with a MySQL database with the following properties:

• The Data Layer is using a third-party MySQL client MySqlClient to connect and perform queries to a remote MySQL database.

• MySqlClient uses its own objects to represent results MySqlResult or errors MySqlError.

In a typical API, the Logic Layer needs to fetch data from the database in order to execute some business logic. And it instructs the Data Layer to do so.

A mistake would be for the Logic Layer to directly communicate with MySqlClient. Because it would then become tightly coupled to the underlying database: MySQL. The code in the Logic Layer would have to extract results from MySqlResultand handle MySqlErrors.

What happens if one day MySQL doesn't suit the needs anymore and we want to to change it to let's say MongoDB? We would have to change the Data Layer and all those MySQL related objects that leaked in the Logic Layer.

A solution would be to abstract your MySqlClient by something like DatabaseClient and have your custom and more generic DatabaseResult and DatabaseError. The Logic Layer would then interact with this abstract client instead of MySqlClient. That way, your Logic Layer becomes unaware of the real database you are using. And it's good as it doesn't care! It just wants the results and knows that errors can happen with the "database".

The other advantage is that the communication "contract" between the layers is less sensitive to changes. You better understand how the data flows between your layers.

I concede that you don't change your database everyday but this concept can be generalized to a multitude of other situations.

In the recent years, we had to migrate several of our databases hosted on Azure Cosmos DB to MongoDB and we were able to do it by just adding a new implementation class and making a small change of configuration in all the impacted APIs. Abstracting the Data Layer only took us a couple of minutes in the early stages of the project and saved us hours of refactoring of the Logic Layer code and its unit tests.

Key-points

• Having clear communication contracts helps to understand the data flow between layers.

• Don't leak implementation details especially between layers.

• Abstraction makes the code more flexible to changes.

• Abstraction reinforce the isolation.

• Abstract from the beginning, it's not expensive.

🎣 Beware of your exception catching

I won't get into the details because exception handling is a rich topic but you have to know that unmanaged and non-consistent exception handling can become a huge source of issues in your API.

Keep the following rules in mind instead, they work quite well for us:

• If you encounter an exception, catch it only if you can handle it locally. If not, it's better to let it resurface to the upper levels.

  • Badly handled exceptions in the bottom layers can silence real problems.
  • Don't catch and re-throw a more generic exception, you may lose important information.
  • It's fine to wrap an exception with one of your own exception (but make sure you don't remove information in the process).
  • In most cases don't catch and re-throw the same exception if you just want to log (see next point).

• Have a general exception catcher on the Edge Layer that will catch any non-handled exception that managed to bubble to the surface. This is where you will be able to handle the logging in a centralized way and standardize error messages that are returned with HTTP 500 for example.

⏩ Next part

You have written all the code needed for your API to run. We now need to check it works properly.

Part 3: Testing Phase