For the last few years, there has been an explosion of API powered businesses. There are revenue generating APIs, developer platforms, partner marketplaces, and even internal APIs powering single page apps.
With this explosion, there has also been a large increase in API tooling to help these companies go to market with their API platforms as quick as possible and out innovate any competition. Much of this increase in tooling mirrors what we saw in the mobile era. However, with this explosion, there is now an increase in number tools and solutions to build and grow APIs and platforms.
In addition, tools like API gateways can be used both as a proxy in front of public facing APIs, but can also behave like a service mesh orchestrating between various internal services. This guide aims to provide an overview of various API tools.
API Gateways
An API Gateway is a HTTP server/proxy server that sits in front of your API and provides a central place to administrate, route, and secure your APIs and services. API gateways should be deployed in clusters so you can scale the system horizontally by adding more machines and ensure high availability. API gateways are usually on-premises appliances, but modern ones are usually based on an open-source or open-core model. For example, Kong is based on NGINX and Express Gateway is based on Node.js Express.
There are also proprietary cloud-based solutions from cloud vendors like AWS API Gateway and Azure API Management. Usually, an API gateway will do the following:
Authentication & Rate Limiting
A primary objective of deploying an API gateway is to provide a secure way to access to your APIs and prevent malicious activity. If your API requires authentication, an API gateway provides an easy way to generate and also manage API keys for each consumer. Clients accessing your API without the correct credentials will get a 401 Unauthorized. To ensure fairness, security policies such as rate limiting can be enforced such as a limit of 100 requests per minute per API key. Rate limiting can be enforced based on API keys, via IP address if no authentication layer is added, or other custom policy. You can even include additional security policies such as bot detection and prevent the HTTP payload from being too large.
Caching & CORS Policies
Caching enables your API platform to handle a higher number of clients and absorb peak traffic. Certain types of content such as e-commerce and travel can benefit greatly from caching. Others such as banking and financial services may not benefit from caching so you need to perform a cost-benefit analysis whether caching makes sense for your applications. Other policies such as CORS (Cross Origin Resource Sharing) can be enforced to allow the API to be accessed from a web browser.
Data Validation & Transformation
An API gateway provides a central mechanism to add data transform rules to your API. Data transformation can be as simple as URL rewrite such as pi.example.com/search to search.example.com to more complicated transform rules such as transforming XML to JSON. This could be handy if you already have many legacy internal services that you want to expose as an API even if the legacy services use an older content format such as XML instead of JSON.
In addition to data transformation, some API gateways can also combine endto avoid API chaining. This enables your customers to leverage a single public end point like GET /user/me even though it has to fetch from various internal services like your authentication service, user service, and billing service. The responses from those internal services are combined into a single response to be
returned to the client. This makes working with your API easier for customers since they don't have to perform API chaining. An extreme example of this would be Apollo GraphQL which can fetch many entities from many services and combine them into a single endpoint POST /graphql
API Canary Release/Versioning
Similar to data transformation, with an API gateway, you can leverage an API gateway to route users to specific versions of your services. This can be done via a header field, a path segment such as /v1 or /v2 or even handle
sophisticated canary releases where a percentage of user traffic is routed to a new version. The ratio of users hitting the updated API version can slowly be increased as your confidence in the release increases. By performing a gradual ramp, you minimize the risk that a new API release becomes a system wide outage due to functional or performance problems.
API Analytics
Once you invested in your API platform, you'll want to ensure you can get a return on your investment by building an API platform that customers actually adopt and is bug free. An API analytics solutions like Moesif complements your API gateway by providing you with the necessary visibility and reporting needed to create product lead growth. This includes how your customers are using your APIs, who they are, and which marketing channels lead them to integrate. In addition to reporting, API analytics usually includes real-time monitoring capabilities so you can stay alerted when issues do occur.
While API management and API gateway solutions are designed for infrastructure and site reliability engineering in mind, API analytics usually is designed with a broader audience of both technical and non-technical users including product, engineering, and support. Because of this, API analytics vendors like Moesif are more likely to be managed SaaS solutions rather than on-premises, but can be homegrown also. Checkout our guide on when to build vs buy an API analytics solution.
API Product Analytics
Product analytics enables you to track and understand key API metrics like your API DAU (Daily Active Users), the endpoints your top customers are using, and 90th percentile latency for key endpoints. Some API analytics tools provide
more sophisticated analysis like cohort retention analysis and funnel analysis so you can track KPIs like average Time To First Hello World (TTFHW) and conversion rates so you can to understand how engaging and sticky your product is.
TTFHW measures how long it takes from first visit to your landing page to an MVP integration that makes the first transaction through your API platform. This is a cross-functional metric tracking marketing, documentation and tutorials, to the API itself.
API Logging/Debugging
Besides measuring product metrics like retention and engagement, API analytics also provides an easy way to speed up investigating and debugging API functional and performance API issues. High-cardinality log analysis enables you to
plot trends using heatmaps, time series, waterfall diagrams, and other charts. Some API analytics also log the full API calls in real-time so you can inspect the request and response headers and body and do a diff or even replay and generate tests from failing transactions.
API Monitoring
Many API analytics tools also provide API monitoring and alerting capabilities so you can get notified when an API is down or acting incorrectly. Unlike synthetic monitoring which are simple health probes, API Real User Monitoring , detects anomalous behavior from your actual API traffic made by your customers based on historical trends. API issues could be business related such as unexpected drop in API activity from key customers or it could be engineering related such as functional or performance problems. Specifically, API analytics focuses on a few key pillars:
Team Dashboards
Some API analytics solutions provide a way to share and collaborate on dashboards with both technical and non-technical team members or automatically sandbox the usage data so you can share dashboards directly to your external partners in a secure way. By embedding an API debug log and plan usage information in your customer facing portal, customers can resolve issues without overloading your already stretched support team.
User and/or Company personas
API analytics tools pull data from your CRM, BI, and support tools. This enables your customer success and support teams to look up a customer's API activity by their email or name to resolve integration issues quickly. User and company level information also enables your company to centrally track how various prospects and customers are integrating and sticking with your APIs.
API Management
While API Gateways and API management can be used interchangeably, strictly speaking an API gateway refers to the individual proxy server, while API management refers to the overall solution of managing APIs in production
which includes a set of API gateways acting in a cluster, an administrative UI, and may even include additional items such as a developer portal for customers to sign up and generate new API keys.
The admin panel enables a few things:
- The ability to add and remove plugins such as for security
- A way to edit and visualize data transforms such as XML to JSON
- Upload and administrate SSL certificates
Closing thoughts
There is a lot of tooling revamp happening in the API space due to the recent growth in new business models and API strategies. More info can be found in our guide to building APIs and our guide to API analytics.
This article was written by Moesif founder and CEO Derric Gilling for the Moesif blog.
Looking for a complete API analytics solution with plugins for popular API gateways? Check out Moesif
Top comments (2)
While the article touches in security it does it lightly, and would be nice to see more focus on security in this type of solutions. Remember that APIs are the new focus of attackers and they are breached to often nowadays. An API Gateway is a good starting point for implementing API security defences, but it must not be the only place where you secure your API, and developers should be aware of the OWASP API Security Top 10 risks and have them always mitigated in their APIs.
Couldn't agree more. As APIs become the primary way of business transctions are done, APIs are the new attack surface.
API Management/Gateway is often used as a security solution as well.
We'll write a more in-depth article on how to leverage API gateways and management solution to really secure your APIs.