Good error message, bad error message
Error messages are like letters from the tax authorities. You’d rather not get them, but when you do, you’d prefer them to be clear about what they want you to do next.
When integrating a new API it is inevitable that you’ll encounter an error at some point in your journey. Even if you’re following the docs to the letter and copy & paste code samples, there’s always something that will break – especially if you’ve moved beyond the examples and are now adapting them to fit your use case.
Good error messages are an underrated and underappreciated part of APIs. I would argue that they are just as important a learning path as documentation or examples in teaching developers how to use your API.
As an example, there are many people out there who prefer kinesthetic learning, or learning by doing. They forgo the official docs and prefer to just hack away at their integration armed with an IDE and an API reference.
Let’s start by showing an example of a real error message I’ve seen in the wild:
{
status: 200,
body: {
message: "Error"
}
}
If it seems underwhelming, that’s because it is. There are many things that make this error message absolutely unhelpful; let’s go through them one by one.
Send the right code
The above is an error, or is it? The body message says it is, however the status code is 200, which would indicate that everything’s fine. This is not only confusing, but outright dangerous. Most error monitoring systems first filter based on status code and then try to parse the body. This error would likely be put in the “everything’s fine” bucket and get completely missed. Only if you add some natural language processing could you automatically detect that this is in fact an error, which is a ridiculously overengineered solution to a simple problem.
Status codes are for machines, error messages are for humans. While it’s always a good idea to have a solid understanding of status codes, you don’t need to know all of them, especially since some are a bit esoteric. In practise this table is all a user of your API should need to know:
| Code | Message |
|---|---|
| 200 - 299 | All good |
| 400 - 499 | You messed up |
| 500 - 599 | We messed up |
You of course can and should get more specific with the error codes (like a 429 should be sent when you are rate limiting someone for sending too many requests in a short period of time).
The point is that HTTP response status codes are part of the spec for a reason, and you should always make sure you’re sending back the correct code.
This might seem obvious, but it’s easy to accidentally forget status codes, like in this Node example using Express.js:
// ❌ Don't forget the error status code
app.post('/your-api-route', async (req, res) => {
try {
// ... your server logic
} catch (error) {
return res.send({ error: { message: error.message } });
}
return res.send('ok');
});
// ✅ Do set the status correctly
app.post('/your-api-route', async (req, res) => {
try {
// ... your server logic
} catch (error) {
return res.status(400).send({ error: { message: error.message } });
}
return res.send('ok');
});
In the top snippet we send a 200 status code, regardless of whether an error occurred or not. In the bottom we fix this by simply making sure that we send the appropriate status along with the error message. Note that in production code we’d want to differentiate between a 400 and 500 error, not just a blanket 400 for all errors.
Be descriptive
Next up is the error message itself. I think most people can agree that “Error” is just about as useful as not having a message at all. The status code of the response should already tell you if an error happened or not, the message needs to elaborate so you can actually fix the problem.
It might be tempting to have deliberately obtuse messages as a way of obscuring any details of your inner systems from the end user; however, remember who your audience is. APIs are for developers and they will want to know exactly what went wrong. It’s up to these developers to display an error message, if any, to the end user. Getting an “An error occurred” message can be acceptable if you’re the end user yourself since you’re not the one expected to debug the problem (although it’s still frustrating). As a developer there’s nothing more frustrating than something breaking and the API not having the common decency to tell you what broke.
Let’s take that earlier example of a bad error message and make it better:
{
status: 404,
body: {
error: {
message: "Customer not found"
}
}
}
Already we can see:
- We have a relevant status code: 404, resource not found
- The message is clear: this was a request that tried to retrieve a customer, and it failed because the customer could not be found
- The error message is wrapped in an error object, making working with the error a little easier. If not relying on status codes, you could simply check for the existence of
body.errorto see if an error occurred.
That’s better, but there’s room for improvement here. The error is functional but not helpful.
Be helpful
This is where I think great APIs distinguish themselves from simply “okay” APIs. Letting you know what the error was is the bare minimum, but what a developer really wants to know is how to fix it. A “helpful” API wants to work with the developer by removing any barriers or obstacles to solving the problem.
The message “Customer not found” gives us some clues as to what went wrong, but as API designers we know that we could be giving so much more information here. For starters, let’s be explicit about which customer was not found:
{
status: 404,
body: {
error: {
message: "Customer cus_Jop8JpEFz1lsCL not found"
}
}
}
Now not only do we know that there’s an error, but we get the incorrect ID thrown back at us. This is particularly useful when looking through a series of error logs as it tells us whether the problem was with one specific ID or with multiples. This provides clues on whether it’s a problem with a singular customer or with the code that makes the request. Furthermore, the ID has a prefix, so we can immediately tell if it was a case of using the wrong ID type.
We can go further with being helpful. On the API side we have access to information that could be beneficial in solving the error. We could wait for the developer to try and figure it out themselves, or we could just provide them with additional information that we know will be useful.
For instance, in our “Customer not found” example, it’s possible that the reason the customer was not found is because the customer ID provided exists in live mode, but we’re using test mode keys. Using the wrong API keys is an easy mistake to make and is trivial to solve once you know that’s the problem. If on the API side we did a quick lookup to see if the customer object the ID refers to exists in live mode, we could immediately provide that information:
{
status: 404,
body: {
error: {
message: "Customer cus_Jop8JpEFz1lsCL not found; a similar object exists in live mode, but a test mode key was used to make this request."
}
}
}
This is much more helpful than what we had before. It immediately identifies the problem and gives you a clue on how to solve it. Other examples of this technique are:
- In the case of a type mismatch, state what was expected and what was received (“Expected a string, got an integer”)
- Is the request missing permissions? Tell them how to get them (“Activate this payment method on your dashboard with this URL”)
- Is the request missing a field? State exactly which one is missing, perhaps linking to the relevant page in your docs or API reference
Note: Be careful with what information you provide in situations like that last bullet point, as it’s possible to leak information that could be a security risk. In the case of an authentication API where you provide a username and password in your request, returning an “incorrect password” error lets a would-be attacker know that while the password isn’t correct, the username is.
Provide more pieces of the puzzle
We can and should strive to be as helpful as possible, but sometimes it isn’t enough. You’ve likely encountered the situation where you thought you were passing in the right fields in your API request, but the API disagrees with you. The easiest way to get to a solution is to look back at the original request and what exactly you passed in. If a developer doesn’t have some sort of logging setup then this is tricky to do, however an API service should always have logs of requests and responses, so why not share that with the developer?
At Stripe we provide a request ID with every response, which can easily be identified as it always starts with req_. Taking this ID and looking it up on the Dashboard gets you a page that details both the request and the response, with extra details to help you debug.
Note how the Dashboard also provides the timestamp, API version and even the source (in this case version 8.165 of stripe-node).
As an extra bonus, providing a request ID makes it extremely easy for Stripe engineers in our Discord server to look up your request and help you debug by looking up the request on Stripe’s end.
Be empathetic
The most frustrating error is the 500 error. It means that something went wrong on the API side and therefore wasn’t the developer’s fault. These types of errors could be a momentary glitch or a potential outage on the API provider’s end, which you have no real way of knowing at the time. If the end user relies on your API for a business critical path, then getting these types of errors are very worrying, particularly if you start to get them in rapid succession.
Unlike with other errors, full transparency isn’t as desired here. You don’t want to just dump whatever internal error caused the 500 into the response, as that would reveal sensitive information about the inner workings of your systems. You should be fully transparent about what the user did to cause an error, but you need to be careful what you share when you cause an error.
Like with the first example way up top, a lacklustre “500: error” message is just as useful as not having a message at all. Instead you can put developers at ease by being empathetic and making sure they know that the error has been acknowledged and that someone is looking at it. Some examples:
- “An error occurred, the team has been informed. If this keeps happening please contact us at
{URL}” - “Something went wrong, please check our status page at
{URL}if this keeps happening” - “Something goofed, our engineers have been informed. Please try again in a few moments”
It doesn’t solve the underlying problem, but it does help to soften the blow by letting your user know that you’re on it and that they have options to follow up if the error persists.
Putting it all together
In conclusion, a valuable error message should:
- Use the correct status codes
- Be descriptive
- Be helpful
- Provide elaborative information
- Be empathetic
Here’s an example of a Stripe API error response after trying to retrieve a customer with the wrong API keys:
{
status: 404,
body: {
error: {
code: "resource_missing",
doc_url: "https://stripe.com/docs/error-codes/resource-missing",
message: "No such customer: 'cus_Jop8JpEFz1lsCL'; a similar object exists in live mode, but a test mode key was used to make this request.",
param: "id",
type: "invalid_request_error"
}
},
headers: {
'request-id': 'req_su1OkwzKIeEoCy',
'stripe-version': '2020-08-27',
}
}
(some headers omitted for brevity)
Here we are:
- Using the correct HTTP status code
- Wrapping the error in an “error” object
- Being helpful by providing:
- The error code
- The error type
- A link to the relevant docs
- The API version used in this request
- A suggestion on how to fix the issue
- Providing the request ID to look up the request and response pairing
The result is an error message so overflowing with useful information that even the most junior of developers will be able to fix the issue and discover how to use the available tools to debug their code themselves.
Designing APIs for humans
By putting all these pieces together we not only provide a way for developers to correct mistakes, but also ensure a powerful way of teaching developers how to use our API. Designing APIs with the human developer in mind means we take steps to make sure that our API isn’t just intuitive, but easy to work with as well.
We covered a lot here and it might seem overwhelming to implement some of these mitigations, however luckily there are some resources out there that can help you make your API human-friendly:
- The excellent APIs you won’t hate community (co-run by my colleague Mike Bifulco) has some great articles on the subject:
- Tools like Spectral can be set up to provide useful linting for APIs - catching things like “200 OK - Error” and making sure best practices are adhered to
Got any examples of error messages you thought were excellent (or terrible, because those are more fun)? I’d love to see them! Drop a comment below or reach out on Twitter.
About the author
Paul Asjes is a Developer Advocate at Stripe where he writes, codes and hosts a monthly Q&A series talking to developers. Outside of work he enjoys brewing beer, making biltong and losing to his son in Mario Kart.
Top comments (16)
Great blog! We are big fans of RFC7808 - application/problem+json If your making new APIs and are unsure on an error schema, we would recommend checking that out. Here is an example of it in action where we reference in the responses app.swaggerhub.com/apis/AdvancedCo...
I can see where you're coming from, but I think you're making a few assumptions here that can muddle the mixture.
For starters, I think it's a mistake to guess the intent of the end user. In the case of the "Customer not found" error, in 99% of cases it's caused by using the wrong API key. However we can't know that for sure. What if the user did mean to use their test keys, but accidentally used the ID of a live mode customer instead of the intended test mode one? In their case a
403: Test mode key was usedis confusing and doesn't help them solve the problem. Returning a404and a message stating why it is a 404 is useful for 100% of cases, rather than 99%.Secondly, while having good documentation is always a good idea and having terse headers rather than error messages might feel a little cleaner, we should take a step back and think about what we're trying to achieve. Is it more important that people read our docs or that they use the API?
I think it's the latter, and having valuable error messages helps push that goal forward.
That was a nice read! Liked, bookmarked and followed, keep the good work! 🙌
Nice article!
Thank you for sharing! I've been facing this problem and not entirely sure what's the next step do. I'm glad that you show step by step. I'll surely implement it. thanks again! :D
Great advice! However, returning something like "Customer cus_Jop8JpEFz1lsCL not found" may be dangerous if the customer ID is an input parameter. It could be an attack. It’s worth to note that some kind of checks on the parameters must be carried out first.
Putting http status code in a message body is ugly. If the server responds with 404 but the status is 200 who do u believe? What youve defined is a message envelope, which is what we tried to kill when moving away from soap.
Http proxies exist in both client and reverse. It is impossible for you to guarantee as a service that you can return a message with this format. Nginx faulting in front of your service probably will return xml. That means any http client has to look at the http info and deal with that or theyll have critical failures. So every http client already has to look at http response codes and youve just made it so they have to look in 2 places, how inconvenient.
Even worse, this means u are assuming the response is being sent over http. I send way more resources over amqp than http. What does 404 mean in amqp? It means nothing!
The practice that DOES make sense is correlating responses with requests, this is even more important in amqp. But you should
Immediately abandon your status code envelope idea its been proven a poor idea going on 20 years
Coming from cybersecurity, I am inclined to agree with Tamas. As much as I appreciate a helpful error message, too much help is a security risk, and API security is generally pretty lax as it is.
I think you have a point but I also think the first question from your reply defeats the purpose of what you're trying to point out. "...if my HTTP API communicates with non-humans..." is an assumption in the context of the purpose of the article which is already titled "Designing APIs for humans". So if the post is about "humans", I don't think making it about "non-humans" is a fair argument.
Sorry to confront you this way but isn't API meant for applications? It's abbreviation of Application Programming Interface in the end. It's an interface to another application - a contract.
Why invest in making API human readable when it's meant to be consumed by machines/programs? The only case when you read these messages is in early prototyping stage or debugging. Even then a good documentation (e.g. SWAGGER/OpenAPI) is better suited.
You don't want your consumers to implement business/retry logic based on full-text english based error messages, do you?
How would you proceed in case of PROTOBUFF contracts? They are plain binary? What about human readabality in this case?
Good points! I would argue that in their base form APIs are meant to be consumed by applications, but in the end it's humans who implement and design these APIs. APIs tend to change over time, adding and removing features, which can lead to unexpected results. Having good error messages in this case means you have a better chance of fixing something if it breaks after your prototyping and debugging stage. After all, it's humans that do the fixing (for now).
I completely agree that good documentation is required, I would argue why not have good documentation and good error messages? Good error messages don't mean replacing essential information like status codes with full-text English, it means providing the extra information additionally to be helpful.
The beauty of designing with humans in mind is that all the extra information can be completely ignored by machines (or you) if you're confident that it all works. The status code (and hopefully an error type) will still be there, the rest is just gravy.
API should not change to break stuff. You're supposed to issue a new version, mark your endpoints deprecated and let your consumers know about changes, replacement endpoints and new contracts (by pointing them to your documentation). If you keep only single version of your API and break it with incremental development changes you're doing it wrong.
You assume development/prototyping in form of "let's call this enpoint with any possible data in any possible way and let's see what error it throws to implement our business logic to handle all the cases". That's weird thinking in programming.
You invest time and effort to implement meaningful multi-language error messages for developers across all the world (happy translating your error messages to kanji, cantonese or hebrew and figuring out how to deliver them to their respective nationalities) just to help users debug/prototype solutions quickly in 0.00001% of your API call cases wasting bandwidth during production for the rest 99.99999% (applications neither don't care about meaningful error messages nor are willing to parse them to understand the source of the error) where a single error code and a Swagger documentation page would suffice.
I wouldn't advocate explicitly using 404 or 500. Leave them for the server to handle or they lose value. Not finding a resource isn't a 4xx, imagine if Google returned a 404 for no results when you type in a search. Be careful to not leak information with your messaging too. Secure messaging shouldn't be seen as an inconvenience.
It's awesome to see that you're recommending a Catalog of errors with metadata about the error.
Good information
Good article. I just don't understand why you return 'invalid_request_error' as the error type in the last sample. It's clearly a 404 error so the type should be 'not_found'.