Slow or problematic backend? - Bulkhead is here for the rescue

webMethods API Gateway version: 10.15 and above

Problem Statement

Ever been in a situation where one backend service starts to malfunction and eventually, its queuing in the API Gateway exhausting all the available threads, having a knock-on effect with serving requests for the other APIs?

Solution

Restricting the number of concurrent API requests processed for a more robust and fault tolerant API deployment

Microservice patterns can be extended to the API Gateway world to make the APIs resilient. One such pattern is Bulkhead, wherein elements of an application are isolated into pools so that if one fails, the others continue to function. API Gateway supports the implementation of Bulkhead pattern through a policy in the Transport section of the API policy configuration.

What is Bulkhead?

The term bulkhead comes from the structure reliability engineering concept that is used in ships to create partitions in the hull. In the event of a hull breach, these isolated partitions prevent water from filling up the entire hull and sinking the ship.The aim is to limit and contain the damage to the isolated partitions.

The bulkhead pattern in software systems works similarly by isolating services to prevent them from starving the system of resources.

A ship without bulkheads sinking as the water fills the entire hull

A ship with bulkheads isolates and contains the damage

Enable Bulkhead Policy

Our approach uses a thread pool to separate the service from the caller and contain it to a subset of system resources.

Bulkhead configuration allows you to specify the maximum number of concurrent requests processed by an API. You can configure this setting individually for an API or globally for all APIs.

When the number of concurrent requests to an API exceeds the specified limit, the excess requests are rejected. In such scenarios, the policy violation events are generated to report the violations occurred for an API. If there are 100 violations, then 100 policy violation events are generated.

You can control the response status code and the status message using the extended settings.

For detailed documentation, please refer Bulkhead Policy.

Want to try? Get started with our Free Forever Edition: Sign up - Software AG Cloud

_{Credit: openliberty.io}

Read full topic