Having built many Product APIs in my experience for multiple companies, there are a number of Myths we've come to learn about APIs in general. Here are some of the more interesting ones we've learned through building Authress, which is an AuthN + AuthZ solution. For reference here is the Authress API.
Myths
1. Clients should ignore properties in an HTTP response that they don’t understand.
In reality they will start to depend on every undocumented property in every conceivable way Hyrum's Law. Our APIs are frequently used in unexpected ways. We go back through our logs every time before we make a change to see every call that was made (in a reasonable timeframe) to make sure we don't break any previously expected behavior. "How could our customers break if we change this" is not a game we play, it is a frequent topic of conversation.
2. Clients should follow redirects if presented with them.
But they never will. Clients will assume getting back anything other than 200 is an error. Yes that is even 200, a 201 will cause an error. So we have to be very careful when changing the status code we return, or even the errorCode in the error response. Changing a 400 to a 422 will cause someone to break because they definitely wrote code that says:
if (statuscode === 400) {
doSomething();
}
3. Clients should will follow the recommended HTTP status codes and headers to handle async requests.
Not even remotely. Client usually assume all API calls are synchronous and instantaneous. That means the default behavior always has to be make sense, and it can never change. When you want to offer clients better functionality, it either has to be the default or put behind a documented RFC header flag like prefer: async. Clients will call these endpoints way more than they should.
4. Clients will use your published API specification and SDKs.
Instead, they'll make every kind of call to your API, they don't care if you can't handle it. If there is an API rate-limit or size limit, you are guaranteed that they will attempt to overcome it. They'll use every API software client known to humans to call your API not just Postman. And the bugs with those clients will become your problems, they'll often not work for some reason, and then you get a nice introduction to how yet-another-api-client has some weird defaults.
5. Clients can use your published API specification to make HTTP calls.
Clients expect that your API is available in every programming language and every framework for every language. If it's not available in their preferred language...To bad, they'll just use a different service. And also they will find every bug in your SDK, not just actual bugs, issues with how a library dependency of a dependency of a dependency didn't interpret the RFC correctly and so now you need to make a bug fix to 3 other open source projects in order to get your customer working.
6. You can make a secure Login/Signup portal for your users.
No matter what you do, these unprotected endpoints will be spammed until you run out of money in your wallet. We know because we run Authress which provides this for our customers. And to no end do we get requests there. Don't have these endpoints, use a federated provider, or even better use the right identity aggregator for your use case by reviewing this Auth situation report
7. You will only get requests that make sense.
Welcome to the world of bots, where every day your APIs will be spammed with an arbitrary list of fuzzers attempting to find exposed and unsecured MongoDBs, PHP WordPress instances, and any number of other requests with random Authorization headers. Be prepared to handle the errors in your application that you didn't intend on because no human would have thought to make that API call. Everything that can go wrong, will. And those errors will always be the first to show up in your alert tracker.
8. Following the standard for what you need will work out--RFC, IETF, W3C.
It would be nice if there was actually a standard for what you are working on, but at best there are just some documents that sort of look like what you want. Here's an awesome curated list of falsehoods you might believe
Your clients will always request something that doesn't match the spec. Worse still they'll require it in order to use your API. The spec doesn't support it, not even remotely, leaving you in a really bad place. Worst of all, you know that they are right.
9. Supporting an integration with a third party for your customer is easy.
Every API today needs to support a deluge of third party products to integrate with their solution. When we built Authrss, we had to integrate with just a couple of OAuth/SAML/OIDC/AD solutions. Now, the list goes on and on--Google, Azure, Steam, Zoho, Slack, Discord, MagicLinks, etc...-- And that's just the AuthN part, then we integrate with every cloud provider (AWS, GCP, Azure) to send our partner SIEM logs (SumoLogic, Elastic, DataDog), and still more for handling other types of data integrations.
10. Clients will cache reasonably and use your Cache-Control header to do so.
Clients will never cache their responses unless they are somehow forced. Even when the data almost never changes, the idea of caching the data for even one or two seconds never crosses their mind. They will happily call your API for the same data over and over again. Rate-limiting will force them, at some point to make a change.
11. Your rate limits will make sense for you users.
The only thing rate limiting does is create an expensive piece of technology you now need to maintain. If you have multiple services/applications, then you've created a very dangerous piece of code/library/reverse-proxy that one small bug will cause downtime for everyone.
Further, rate-limits are surprises for your clients. Your clients will accidentally hit them one day and start causing huge problems, and that's because rate-limits aren't for them, they are for you. And so, if you make the mistake of adding rate-limits, now you need to monitor all the traffic in/out of your service to make sure non of your clients ever hit the rate-limit. Because if they do, it's the same as your service being down.
Likely you don't actually even need the rate-limit, but one time, one person with too much authority said you needed them, and now you are stuck. Or worse still, you are handing out api keys to your customers and never thought about how to track and secure them. (This by the way is why Authress offers API keys as a service)
12. We can release new versions of our API when something changes.
This is probably the worst lie. No, you can never release a new version of your API. Sure, you can publish it, but no one will use it, or maybe only a few people will. Realistically, integrations don't get updated. If you release an endpoint today, assume you need to maintain it indefinitely, unless you want to have the conversation with your CEO about why you lost 90% of your customer base because you dropped a necessary endpoint and their product stopped working.
No one wants to change anything that is working, don't try to make them. It's also worth mentioning here that there are ZERO consistent ways to actually change a REST api to support multiple versions.
- Different versions of the spec is a nightmare to maintain, and that's just the documentation part
-
v1.service.com- Using a subdomain is the worst possible idea, now instead of maintaining one service, you've got two, that do almost the exact same thing, 99% of the endpoints are the same, and one of them is only slightly different. -
/v1/resources- Different versions in the resource path of the url, actually means these are different resources. If they are don't use a version here, just change the name of the resource -
X-Header: v1A header is difficult to discover and worst still, there's no way to default it a new version -
resources?version=v1- Query parameters seam like the best possible solution, but have all the same problems as the headers--difficult to document, difficult to discover, difficult to know what the right one is.
In the end of the day, don't release breaking changes to your API.
Conclusion
Don't get stuck in the Myths of building an API. And if you find yourself wanting a more secure API in the process, you know where to go.
Come join our Community and discuss this and other security related topics!
Top comments (0)