This post was originally posted on my blog
REST is built around the concept of resources, represented as URIs. Making an HTTP call specifying an HTTP verb and a resource URI takes an action on the specified resource. Most REST frameworks provide generators where you specify a resource name, and the framework generates scaffolding around it. Unfortunately, many of these generators use the CRUD model (Create, Read, Update, Delete) as the default starting point. The resource is defined as a bag of attributes, using something like JSONSchema or a language-specific data object definition and then method stubs are generated to create, read, update and delete that resource.
While it's good to give developers a starting point to work from, I have a big problem with using CRUD as that starting point for an API. The U in CRUD is my least favorite, although I'm not fond of the other letters, depending on the use case. But let's talk about U. Generic update methods allow the client to update any field of the resource and then overwrite the existing version with the new version. However, if you allow clients to do this, your service API provides little value on top of whatever underlying datastore it's using. One of the key value-adds of a service layer is enforcing business constraints on top of the underlying data, and resources always end up having business constraints.
But can't we add business constraints to our update method? Let's use a simple bank account resource as an example and see what happens. First, clients shouldn't be able to call an API and just update their account balance to whatever they want. There might be a minimum balance for the account. Ok, so you add some checks to the update method such that if the account balance value is changed, it must be within a specified range. Problem solved? Well, no, not really. Any balance adjustment should be recorded as some kind of transaction, right? Was this a credit? A debit? A transfer? What if the client tries to change the account number? Is this allowed at all? Could that break other data relationships? It's not hard to see how your update method implementation can quickly turn to spaghetti code the more questions we ask. I've seen teams go down this path where their code tries to infer what the intent of the client is from which fields are changed, and the code ends up a total mess.
So what's the alternative? Personally, I'm a big fan of Domain-Driven Design (DDD) for designing any kind of API. DDD is based around the idea that software should be modeled after the real-world problem being solved. It creates a language for describing software in terms of key business objects called Entities or Aggregates. It also defines terms such as Services, Value Objects, and Repositories, which work together to solve problems in a particular business domain, or Bounded Context in DDD terms. You don't have to use REST to use DDD, however, I find it works especially naturally with REST APIs, because REST resources map very well to DDD entities.
So what does all this mean? It means your API should be centered around domain objects and the business operations they provide. Business operations are the key alternative to the generic update method and all of its pitfalls. Let's use the banking example from earlier to illustrate.
For a banking API, an obvious domain object (or entity in DDD terms) is an account, which models a bank account. Rather than following the CRUD model for accounts, we should define specific business operations that make sense to perform on a bank account. Here's a good starter set of write operations:
- Open - open a new account.
- Close - close an existing account.
- Debit - remove money from an account.
- Credit - add money to an account.
These operations are specific and can enforce certain business constraints. For example, we may not want to allow crediting a closed account, and we can enforce our minimum balance check as part of the debit operation. On the read side, we can also provide specific queries that match our client use cases:
- Load - load single account by its account id.
- Transaction history - list transaction history for an account.
- Customer accounts - list accounts for the given customer id.
Now that we know what our business operations are, here's an example of mapping them to a REST API:
- POST /account - open a new account.
- PUT /account/<accountId>/close - close an existing account.
- PUT /account/<accountId>/debit - remove money from an account.
- PUT /account/<accountId>/credit - add money to an account.
- GET /account/<acountId> - load single account by its account id
- GET /account/<accountId>/transactions - list transaction history for an account.
- GET /accounts/query/customerId/<customerId> - list accounts for the given customer id.
This looks a lot different than a basic CRUD API, but the key is that the operations allowed are specific and well-defined. This results in a better experience, both for the service implementor, as well as the client. The service implementation no longer has to guess what business operation is implied based on which attributes are updated. Instead, the business operation is explicit, which leads to simpler, more maintainable code. On the client side, it's much clearer exactly what operations can and cannot be performed. If the API is documented well, for example, using a Swagger definition, it will also be very clear what the constraints of each API are.
Defining your APIs this way requires more up front thinking than a simple CRUD generator, but I think this is a very good thing. If you're planning on exposing your API as a public endpoint, you are going to have to support that API for a very, very long time. Basically think of it as forever by software standards. I always encourage teams to take time up front on the things that are hard to change later, and APIs are the first example I give.
So resist the urge to follow the CRUD model for service APIs (REST or otherwise). Instead, use DDD to define your API in terms of domain objects and the business operations that can be performed against them.
If you want to see more examples of defining APIs in terms of domain objects, I recommend checking out the Amazon Web Services APIs. Look up the developer guide for any service and they should start with a section labeled "Key Concepts" or something similar. There, they describe the conceptual domain objects of the service. For example, S3 defines objects like Buckets, Objects, and Permissions. Kinesis has streams and shards. Once you understand the domain objects of a service, look at the API reference and skim the list of APIs for that service. What you'll notice is the API is built around these domain objects, making it more intuitive to understand and use.
Hope this helps!
Top comments (31)
Hi James, nice article, thank you!
I absolutely agree that REST APIs should not be mere wrappers for database tables. Instead, they should indeed represent business concepts. Kind of like in DDD, that's a good idea.
I don't agree with your suggestion for alternative endpoints, though. In particular, this one is a bad idea in my opinion:
That's because HTTP has clearly defined semantics for the various request methods. And, in the specification we can read that PUT is idempotent - in stark contrast to POST. This means, if we are talking about a PUT request for removing money from an account, I should be able to send this request again, and it should not have any effect if the transaction occurred already. That's probably not what's going to happen in your scenario. A POST request, e.g. to a /transactions endoint, would probably be the right way to go.
(P.S.: Just because PUT/PATCH theoretically allows me to update certain fields of a resource, does not mean the server has to accept the requests. It can still enforce its business rules, and respond with appropriate status codes to signal failure.)
Hi Franz. Good catch! There is a lot of information that is not shown by just listing the method and resource path for that API. You are correct that PUT must be an idempotent operation. This API is actually idempotent, but you can't tell just from the method + resource. In the request body definition, the client is required to pass in a referenceId that is unique. The behavior of the API is you can call it multiple times with the same refId and it will only apply the debit operation once. This makes it idempotent and safe to retry, which is critical in a distributed architecture.
I didn't want to detract from the main point of my article by explaining this in the post.
Hope this helps clarify my choice of HTTP verb.
Thanks for clarifying!
One more thing about the choice of PUT, though. The spec states:
...which your suggestion does not match. At that point, this might just be nit-picking, though, as I am not aware of any practical benefits of this principle, other than semantics (whereas the idempotency thing may very well be relevant in terms of intermediaries etc.)
Hmm, I'm failing to see how I'm not meeting that requirement...
For all of the PUT requests, I am specifying a specific resource to take the action on by passing the accountId. Are you saying I'm breaking the REST spec by following that with the operation in the URI (debit, credit, etc)?
If so, that was a pragmatic choice I made. If we want to be pedantic, I should have a single PUT URI that just ends with the accountId. But then how do you support multiple business operations? There are only so many verbs available. One option is to add an action query parameter and use that as a switch inside your resource method implementation. However that code gets messy, especially in a strongly typed language. You're basically hand-writing custom routing code. At that point, we decided it was cleaner to put the action in the URI and let the framework's routing layer take care of this for us. Plus, it made it easier to document the different operations in a swagger definition.
So I think what I'm doing is consistent with the REST spec, although as you alluded, we're probably in nit-picking territory at this point. I would be cautious about treating the spec as gospel. Sometimes bending the rules should win out if the pros outweigh the cons.
By expressing them in terms of nouns, not verbs (such as POST /transactions). The verbs are fixed, and you apply them to nouns whatever form they take (i.e. they are polymorphic).
I think we'd cut through a lot of back and forth if you propose an alternative set of REST resource methods to the ones I propose in my post and explain how they're better. If they're not better, but just different, that's fine. The reason I chose my resource Uris the way I did was because of DDD. I think it's cleaner to have actions that affect a single entity reference the entity you're performing the action on in the URI, rather than turning the action into a noun and making that the URI. However I do use that strategy for actions that happen across entities. For example, to do a transfer between accounts, I promote the transfer itself to a first-class entity in the system. At that point, transfer is its own URI.
Sorry, took me a while to get back. Busy week...
Your suggestion:
My suggestion, without putting too much thought into it.
We basically have three sets of resources now: customers, (customers') accounts and (accounts') transactions. The given operations map very nicely to the existing HTTP verbs. :)
Thanks so much for taking the time to present this alternative endpoint design! First off, I want to say I'm a firm believer that there's no single "right" solution, so keep that in mind as I share my thoughts.
I think the main strength here is that your design sticks with resources as nouns, unlike mine where the URI can contain explicit business operations. I can definitely see how this is more in-line with the original REST concept.
Some potential weaknesses I see with it are
Again, there's no single "right" solution, so just sharing my opinions. Thanks again for the follow up!
Hi James, allow me to get back once more. ;)
You write:
The API is not the business layer. In fact, my experience tells me it is quite useful when the API hides/encapsulates as much of the business logic as possible. And I don't see how changing regulations would affect the interface of these operations in such a way that a single endpoint for both operations would make you less flexible. The operations should absolutely be modeled separately in your domain, but again: that's not the API.
Yep, that's always tricky. The way I do it at work is to provide one base endpoint (for /accounts in this case) that offers all of the filters and in addition have these named subresources (/customer/:id/accounts) - basically as an alias. The idea here is to split between real "filters" (e.g. accounts started after a certain date or having a certain balance) and "lenses" (for lack of a better word) - different use cases that might be mutually exclusive or warrant naming as their own resource.
Thanks for the interesting debate! I'm off for my vacation now. :)
Likewise! Great discussion! š
Agreed that the API interface and internal implementation should be distinct concepts. However, I find when using microservices architecture and DDD, generally you at least start out with an implementation that closely mirrors the API model. The implementation may evolve over time, but I like to start out simple.
The alias idea is great. Again, I'm coming at this from a microservices architecture, which might be where some of the disconnects are coming from. There's the API at the microservices layer and then you front all of your microservices with an API Gateway layer. So I would keep my /account centric stuff at the microservices layer, then add the customer/:id/accounts at the API Gateway layer. So I think we're in agreement here.
Have a good vacation!
Hello James and Franz,
This thread is an awesome complement to this post, I'll follow with just one thing:
I completely agree with James on this one, keeping one noun as transactions might be more compliant to REST, in the other hand both client and server are going to (possibly) disrespect the single responsibility principle, the server will have to split the business logic internally while the client depending on its usage might have also to do the same due to having multiple operations in a single endpoint.
That's why I agree "bending the rules" are important as well since the advantage of the transactions endpoint is almost purely to follow the standards without giving much back to clients and servers.
Business rules need to be enforced on the server side. With James' example, I see nothing in there that requires the client to know the details of these rules. Hence, it should not have to deal with two separate endpoints because to it, the operations are the same. That, in my eyes, is the biggest benefit of a combined endpoint in this case. Depends on the concrete example, though.
Hi James, thanks for this good article in explain why we should avoid the CRUD trap ;)
When coding I prefer to be explicit over implicit once I also prefer to use the domain level language to define each action for the resource being requested.
I split my folders structure as
resource/action, so for routeaccount/{id}/debitI would end up with a controller insrc/Resources/Accounts/Debit/AccountsDebitController.extension...This is what I call the Resource Design Pattern, that allows us to achieve more easily SOLID and Clean Code and at same time acts as documentation for the project.
Once the folder structure reflects all actions available for each resource this will be the type of documentation that never goes out of sync with the project, therefore never lies ;).
Cool. That's a nice structure. I'm generally using JAX-RS so I create a controller for the overall entity type, then have a method for each action. This works ok because my domain logic is delegated to a separate layer so the REST controller is generally just translating between the REST protocol and the domain logic's interfaces. However if I did one controller per action, your directory structure would work well.
My Controller don't have any logic related with the Domain, it also delegate that details to another layer.
Any Domain Logic is handled by a Handler dedicated to each Resource action.
This allows me to handle the request for each Resource action in the same way, no matter is source.
So the Handler can receive requests from a Controller, Console or Socket and will always deal with them without knowing the the source of the request, neither the type of response it has to return, because the input and output will be always an Immutable Value Object or scalar values.
I stumbled onto this via Twitter, and I agree wholeheartedly with your suggestions.
Two things leap out at me however:
Your API definition for accounts is absolutely not REST.
This is OK.
Most "REST" APIs are not REST in the sense that Roy Fielding defined Representational State Transfer in his PhD thesis. Most miss out at least the discovery part - you're really not meant to ascribe meaning to a URL, instead just discover them via hypermedia. The fact I even said "hypermedia" in a description of APIs is probably enough for people to look blank.
As a result, most REST APIs are built around only the network side of REST, rather than the hypermedia bits. This means that GET is always idempotent and so on. And URIs represent actual resources - if you PUT something to a resource, you can later GET the same thing (modulo other operations). It's this that represents a broader departure from REST in your design.
A more traditional REST API would have transactions processed by POSTing them to the account (perhaps), and result in a new URI for the transaction record itself. In your outline, you're PUTting continuously to the same URI. Performing a GET won't give you back what you PUT there before.
That's not, however, to say that your design is therefore "wrong". PUTting transactions you can no longer GET will work just fine - individually, the verbs all operate properly. It's a perfectly good design, and conforms to how HTTP itself works.
REST is often treated as the one true way of HTTP API design - it's held in almost religious fervour, despite, as I say, most REST APIs actually not following REST. But as you demonstrate, there are other ways of designing HTTP-based APIs, and they may well be a better fit for the business problem at hand.
I think it is nearly impossible to avoid a "U" api in reality and it is not conflict with the business operation APIs.
Use the bank account example, there must be some information that user could change, like name, alias and etc. They could either be changed separately or together. So logically there could be 3 or more operations to do some update,
Instead of model them as 3 different business operations, I think it will be modeled as single "UpdateInfo" operation.
In an information management domain, it is common that there is a "Update" operation that could change any fields. I could not think of a reason not naming the API as "update".
I don't like the generic "Update" API but it seems inevitable for any entity that contains mutable information input directly by user.
I really hope to see your thoughts on this. Thanks
As I understand REST, URIs point to resources on server so that I can perform CRUD operations (GET/POST/PUT/DELETE operations if HTTP) on them using these URIs.
So when I perform GET operations on
/account/<accountId>/debitand/account/<accountId>/credit, would I be getting back DebitableAccount and CreditableAccount representations of account back respectively?Or is it that they are not pointing to any resource and are just for performing some particular action?
Very interesting, thank you very much.
From what I understand, this architecture still calls for models and collections, but now we change the interaction with those by adding a list of actions and behaviours, preferably replacing the conventional commands that come "out of the box" with most frameworks.
And another by-the-way understanding - there is no project too small for this architecture, because they ALWAYS grow.
Exactly! š
REST APIs never work well for these kind of things. HTTP was designed for dealing with static HTML pages where it is sensible to update a whole resource in one go. It also works well for content systems like blogs but REST is less suitable for verb heavy APIs.
In this case we should be replacing the HTTP verb to give us:
CLOSE /account/<accountId>
Instead of:
PUT /account/<accountId>/close
But HTTP does not allow that. Maybe the solution set out in the article is a good one, but it isn't really in the spirit of REST.
PS: That last api should be
GET /accounts/?customerId=<customerId>
instead of
GET /accounts/query/customerId/<customerId>
Disagree. I've seen it work well many times.
Totally see where you're coming from, and I started out following this pattern as well. However, I've found this to be an antipattern in practice. The more queries you have to support, the hairier your query params become. Your API starts to become really complex where one query requires params A, B, and C to be set, another requires A and C to be set, another requires B xor C to be set, etc. I prefer to make each supported query explicit by separating them into a separate URI. This might also have to do with the fact that I'm implementing this API as a distributed system, rather than a monolithic service attached to an RDBMS. A lot of these nuances are less important with a simpler stack.
We can agree to disagree on whether this is "pure REST" or what Roy Fielding originally intended. I prefer applying well-reasoned pragmatism as opposed to trying to adhere to a spec, just because it's the spec.
I'm not saying that you haven't seen many successful APIs or even that the one presented in this article won't be successful. All I'm saying is that I don't think it is the way REST APIs are supposed to be done.
This really is the biggest issue with so called REST APIs. The fact that no two developers quite agree on how they should be implemented makes the term REST rather useless. If you called it 'JSON over HTTP' it would tell me more.
In 20 years developing web systems, I've yet to see two REST APIs with a consistent usage pattern. Telling me it is a REST API gives me no information about how to use it at all, which means I always have to create a client library pretty much from scratch each time.
I don't think the term is useless at all. It is very well defined. It annoys me as well that many people mean many different things when using that term. It should just be used less often. Other things are simply not REST. Doesn't mean they are bad, but they are something else.
(I do wish Fielding's work would contain more practical examples, as that would help in understanding. But hey, it's a thesis that describes an architecture style on a very abstract level.)
Hello James.
Very interesting article.
We went through the same questions and concluded that trying to stick with REST is only working with CRUD and therefore simple applications. What we finally end up with is an RPC api.
How would you design the operation allowing to update a bunch of account attributes that are not bound to any specific business validation? Do you use DTO?
Maybe the best solution is CQRS?
Thanks for reading and great points! For me, I really consider REST and RPC two different protocols, which can be used to achieve similar ends. The point I was trying to make with the article is not that REST is the best, but rather DDD is a powerful tool for modeling a public API and CRUD contradicts that. I used REST as an example to make it concrete how the entity operations can be mapped to an existing protocol. Also, REST is the de facto standard for microservices currently, like it or not. However, I've used the same paradigm with RPC and had it work well too.
Regarding updating multiple attributes of an account without business validation, I have yet to see a case where there's zero validation. You at least want to enforce a max length on a string. But I think what you're getting at is certain user-controlled metadata that is only used by the user, e.g., a friendly name to display in their console. In those cases, the business operation is something like "UpdateMetadata" and the request body is a DTO structure that contains only the attributes that the user is allowed to update.
If you're instead talking about some kind of operational override where you can force update fields of an entity, bypassing validation, I have created an override operation in the past. However, it's a risky operation and should never be surfaced as a public API. It's something I would only surface as a CLI tool that requires dev or support privileges to run.
Hope this helps!
Nice post. There had always been some nagging sense that the typical "API = HTTP-exposed database" pattern was lacking usefulness - particularly if the back-end store was also a proper database that utilized HTTP calls. It's almost like, why even add "my" layer in the middle?
I'm working a side project now, and this will definitely make me reevaluate what I thought my endpoints were going to be. Thanks!
Glad you found it helpful!
Really good article explaining how to go about designing rest endpoints. Because the efforts you put when you design anything will help you in a longer run and avoid spaghetti mess.
Nice job on this post. What makes it even better are the comments. I'm working on an API for a large client server app making its way to a web app. This is exactly what I was thinking of doing.
Nice. Good luck!