DEV Community: Anaïs Schlienger

Avoiding API Gateway’s integrations hard limit: scaling serverless architectures efficiently

Anaïs Schlienger — Tue, 26 Nov 2024 07:00:00 +0000

Avoid AWS API Gateway’s integration limit with efficient multi-API Gateway setups. Learn how to scale serverless projects!

Last week, I was coding a new feature. It was looking good, until I had to deploy. AWS cli yelled at me and refused to deploy my stack. I got the following error : CREATE_FAILED: HttpApiIntegration (AWS::ApiGatewayV2::Integration) Maximum number of Integrations for this API has been reached. Did this ever happen to you? If it did not, and you are using AWS, then it might be worth to run a quick sanity check on your architecture.

AWS combined with API Gateway is widely used for scale-up apps. If API Gateway can handle a lot of routes (there is a 300 quota that can be increased), it has however a lesser-known but critical limitation of 300 integrations per API Gateway instance.

An API Gateway integration is a connection between an API Gateway and a backend service, such as a Lambda function, an HTTP endpoint, or an AWS service, allowing the API Gateway to route incoming requests to the appropriate service for processing and return responses to the client. An endpoint can have only one integration. Here is a great article about integrations.

In a non-monolithic serverless architecture, where every Lambda function represents a different endpoint, it’s easy to hit this limit once your app grows. This was exactly what happened to me in a large-scale serverless project using the Serverless Framework.

The challenge: reaching API Gateway’s 300 integration limit

Let’s represent my project as an online library, where users can read, buy, lend and comment on books. The backend is split into two services, users and books, representing the core features.

Our architecture, typical of many non-monolithic setups, follows a one Lambda per endpoint model. When you have multiple services each exposing dozens of endpoints, it doesn’t take long to bump against the 300 integration per API limit (in this simplified example with only 2 services, it might take a long time I agree). This bottleneck emerged (I was not aware of it before!) when our API maxed out its available integrations and simply refused to deploy.

Evaluating the options: lambda-lith vs micro-APIs

With an architecture with all services living behind the same API Gateway, we faced two clear paths forward:

Split the API into smaller chunks (micro-APIs).
Shift to a more monolithic Lambda setup, where multiple endpoints share a Lambda.

Going down the monolith route would have required significant changes to our architecture, a large refactor, and a fair amount of coordination with the front-end. This wasn't ideal for us since we wanted minimal disruption to our existing setup and quick implementation. So we opted for the second option: splitting the API into smaller chunks by creating a new API Gateway for each service.

This approach allowed us to break our system into manageable micro-services while keeping our frontend mostly unchanged.

The solution: multi-API Gateways behind CloudFront

The architecture change was made easier by the fact that we had CloudFront in front of our API Gateway. Our frontend communicates with CloudFront, and CloudFront handles routing requests to our API.

This setup meant that adding new API Gateways could be done without any drastic changes to the frontend. All we needed to do was redirect calls to the appropriate API Gateway, based on the service prefix.

Here’s how we did it step-by-step:

Step 1: create a new API Gateway for the `user` service

We began by creating a new API Gateway for the user service. Since this was essentially a new API, we also had to configure a new Cognito authorizer to handle authentication for this service.

const serverlessConfiguration: AwsConfig.Serverless  = {
    servcice: 'user',
    ...
    providers: {
        httpApi: {
          cors: {
            allowedOrigins: ['http://localhost:3000',],
          allowedHeaders: ['Content-Type', 'Authorization', 'Origin'],
          allowedMethods: ['GET', 'POST', 'PATCH', 'DELETE', 'PUT', 'OPTIONS'],
          exposedResponseHeaders: ['apigw-requestid'],
        },
            authorizers: {
              httpUserApiAuthorizer: {
              identitySource: '$request.header.Authorization',
            audience: ['${self:custom.cognitoUserPoolClientId}'],
            issuerUrl: '${self:custom.issuerUrl}',
          },
          },
    },
    },
    custom: {
    cognitoUserPoolClientId: 'CognitoUserPoolClientId-${sls:stage}',
    issuerUrl: 'CognitoUserPoolIssuerUrl-${sls:stage}',
  },
};

The goal here is not to explain how to set-up a Cognito authorization, if you want to learn about it you can read this article.

Step 2: Link service lambdas to the new API gateway

The next step involved linking all of the Lambdas in the user service to this new API Gateway. Serverless Framework does it for us when we declare the API in the configuration (step 1). However, we still need to set the new authorizer on our lambdas.

export const createUser = {
  handler: 'handler.main',
  events: [
    {
      httpApi: {
        method: 'post',
        path: '/api/user/create',
        // authorizer: { name: 'httpApiAuthorizer' }, < previous authorizer
        **authorizer: { name: 'httpUserApiAuthorizer' },**
      },
    },
  ],
};

Step 3: Update API routes

We then prefixed all API routes for the user service with a specific route : /user. This allowed our CloudFront distribution to easily differentiate between services based on the URL.

export const createUser = {
  handler: 'handler.main',
  events: [
    {
      httpApi: {
        method: 'post',
        **path: '/api/user/create'**, // make sure all user routes are prefixed
        authorizer: { name: 'httpUserApiAuthorizer' },
      },
    },
  ],
};

Step 4: Update frontend API calls

On the frontend, we updated all API calls related to the users service by adding the /users prefix. This ensured that requests were routed to the correct API Gateway without any confusion.

const userList = await client.request('get', '/users/list');

Step 5: Redirect Service Requests via CloudFront

Finally, we set up a redirection rule in CloudFront. Any requests starting with /service2 were routed to the new API Gateway.

export const CloudFrontApiDistribution: AwsConfig.CloudFormationResource = {
  Type: 'AWS::CloudFront::Distribution',
  Properties: {
    DistributionConfig: {
      Enabled: true,
      HttpVersion: 'http2',
      Origins: [
        {
          DomainName: {
            'Fn::Join': [
              '',
              [
                '${self:custom.httpBooksApiId}',
                '.execute-api.',
                '${self:provider.region}',
                '.amazonaws.com',
              ],
            ],
          },
          // Id: 'BackendAPI', <- previous global API name
          **Id: 'BooksAPI',**
          OriginPath: '/api',
          CustomOriginConfig: {
            OriginProtocolPolicy: 'https-only',
          },
        },
        **{
          DomainName: {
            'Fn::Join': [
              '',
              [
                '${self:custom.httpUsersApiId}',
                '.execute-api.',
                '${self:provider.region}',
                '.amazonaws.com',
              ],
            ],
          },
          Id: 'UsersAPI',
          OriginPath: '/api',
          CustomOriginConfig: {
            OriginProtocolPolicy: 'https-only',
          },
        },**
      ],
      PriceClass: 'PriceClass_All',
      ViewerCertificate: {
        CloudFrontDefaultCertificate: true
      },
      DefaultCacheBehavior: {
        // TargetOriginId: 'BooksAPI', <- previous global API name
        TargetOriginId: 'BooksAPI',
      },
      **CacheBehaviors: [
        {
          PathPattern: '/users/*',
          TargetOriginId: 'UsersAPI',
        },
      ],**
    },
  },
};

The default cache behaviors redirects all other routes to the original API, which is a keep-safe for all our other routes, because we don’t need to take care of them as they are already routed.

The benefits of this approach

By splitting the API and allocating different services to their own API Gateways, we gained several benefits:

Modular growth: Each service can now grow independently, with its own API Gateway, reducing the risk of hitting the 300 integration limit again.
Minimal frontend changes: Thanks to CloudFront, the frontend barely needed any modifications. The redirection was handled transparently.
Service isolation: Each service can now be managed and deployed independently, allowing for greater flexibility and scalability.

Conclusion

The AWS API Gateway limit of 300 integrations per API can be a hidden bottleneck, especially in fast-growing projects when you are unaware of its existence. In order to spot in before it arrives, I am adding a new rule on the sls-mentor tool, in order to be warned when you reach 250 integrations.

While initially daunting, addressing it through a multi-API Gateway approach can keep your architecture flexible, scalable, and modular. This experience reinforced the importance of periodically reviewing architectural decisions and splitting services when they become too large. From now on, when I have an architecture with micro services, I will be creating a micro API for each service.

For teams scaling on AWS, this approach ensures that when your services grow to the point of hitting such limits, you have a clear path forward without drastic changes to the frontend or overall application flow.

References

sls-mentor and how to use it
Cover photo : by Arthur Mazi on Unsplash

Override the 500 resource limit of AWS CloudFormation templates with Serverless Framework

Anaïs Schlienger — Fri, 08 Dec 2023 14:47:45 +0000

AWS provides a robust infrastructure for deploying serverless applications using AWS CloudFormation. However, it's not uncommon to encounter resource limits when working on large serverless projects. One way to tackle this challenge is by splitting your CloudFormation stacks into smaller, more manageable units. AWS suggests themselves to use nested stacks to overcome the 500-resource limit. In this article, we'll explore how to achieve this using the Serverless Framework and a plugin called serverless-plugin-split-stacks. We'll also discuss some tips and tricks to make the process smoother.

Nested stacks are a safe way to address the resource limit wall whilst keeping the benefits of a single stack. The main benefit of a single stack is that you can deploy and rollback all resources at once. As nested stacks have a dependency to their parent stack, so if one of the nested stacks fails to deploy, the parent stack will roll back to its previous state. The nested stacks also share the output variables (such as ids or arn). We keep an atomic behavior of our stack.

Prerequisites

Before we dive into the details, make sure you have the following in place:

AWS Account
Node.js and npm installed
Serverless Framework installed (npm install -g serverless)
A Serverless Framework project set up

Splitting Stacks with serverless-plugin-split-stacks

Installation

First, you need to install the serverless-plugin-split-stacks plugin. Navigate to your Serverless project directory and run the following command:

npm install serverless-plugin-split-stacks --save-dev

Configuration

In your serverless.ts (or serverless.yml) file, add the plugin to the plugins section:

 plugins: ['serverless-plugin-split-stacks'],

and configure it under the custom section:

custom: {
  // ...
  splitStacks: {
    perFunction: false,
    perType: false,
    perGroupFunction: true,
    nestedStackCount: 10,
  },
}

perFunction: Setting this to true would split the stack for each AWS Lambda function. Setting it to false keeps your structure manageable.
perType: If you want to split stacks based on resource types (e.g., DynamoDB tables, S3 buckets), set this to true.
perGroupFunction: This option is set to true, which splits stacks equally based groupings of functions (the lambda and all its associated ressources, eg. IAM roles).
nestedStackCount: disabled if not specified ; it controls how many resources are deployed in parallel.
stackConcurrency: number: disabled if not specified ; it controls how many stacks are deployed in parallel.

Renaming Existing Lambdas

Here is the tricky part. The plugin doesn't work with existing resources. You can force the migration of an existing resource to a new stack if you use a custom migration with force: true but it will delete the resource and recreate it. It might create conflicts in CloudFormation or issues with IAM. This is not a good idea for production environments.

No worries, there is a workaround.

If you have existing Lambdas, you need to rename them to follow the new structure. The easiest way is to suffix their names with an underscore _. For example, if you had a Lambda named myFunction, you can rename it to myFunction_. You use any other suffix you like honestly, but the underscore is a good convention as it is easy to read and less bulky visually.

Just like that, without being deleted, your lambdas are moved to the new stack.

Deploying the Stacks

When deploying your Serverless project for the first time with these changes, ensure you set the disableRollback parameter to false. This way, if something goes wrong during deployment, AWS will not automatically roll back the changes. Here's how you can set it:

sls deploy --disableRollback false

After the initial deployment, it's recommended to set disableRollback back to true for safety reasons. This ensures that the stack rolls back to its previous state in case of deployment failures.

Example

Let's illustrate these steps with a simple example. Consider a Serverless service with three functions: userFunction, orderFunction, and paymentFunction.

Install the serverless-plugin-split-stacks plugin.
Configure your serverless.ts file:

service: my-serverless-app

frameworkVersion: '>=2.50.0'
plugins:
  - serverless-plugin-split-stacks

provider:
  name: aws
  runtime: nodejs18.x

custom:
  splitStacks:
    perFunction: false
    perType: false
    perGroupFunction: true
    nestedStackCount: 10

functions:
  userFunction:
    handler: userFunction.handler
  orderFunction:
    handler: orderFunction.handler
  paymentFunction:
    handler: paymentFunction.handler

Rename existing Lambdas if necessary.
Deploy the stacks as explained above.

💭 Limitations

Hitting the resource limit may be a sign that your application is becoming too complex. It's a good idea to review your architecture and see if you can simplify it or re-organize it. Having smaller services is easier to maintain and manage.

🧠 Conclusion

By splitting your AWS CloudFormation stacks using the serverless-plugin-split-stacks, you can overcome the 500-resource limit and manage your serverless applications more effectively. This approach not only makes your infrastructure more scalable but also eases the maintenance of your serverless projects as they grow.

However, sub-stacks have their limits too and it does not mean you can grow them out indefinitely. They have the same resource limit as the main stack, and the more nested stacks you have, the longer your stack takes to deploy.

Having to split your stacks is a sign that your application is becoming too complex and you should review your architecture. One way to change the way you split your stacks is to have a stack for each decoupled business entity. Or maybe a micro-service architecture is maybe not the good fit for your use case: you can look into hexagonal architectures for example.

References

Understand the AWS SSO login configuration

Anaïs Schlienger — Wed, 12 Jul 2023 22:33:11 +0000

TL;DR

AWS SSO makes it easy for you to switch between profiles. In this article, you'll learn how to easily set up your AWS profiles, to switch between them, and use the automatic SSO login! Moreover, we will focus on the new SSO sessions parameters.

What is SSO?

AWS Single Sign-On (SSO) is a service provided by AWS that simplifies the management of user access to multiple AWS accounts and applications.

By providing a single set of credentials for accessing multiple AWS accounts and services. Users can log in once using their organization's identity provider (IdP) credentials and then access multiple AWS accounts without the need to enter separate credentials each time.

How did we do before SSO?

Before using SSO, you could log into AWS by using credentials composed of either user/password (in the Management Console) or access_key/secret_key (with the CLI or different SDKs and APIs).

These methods required users to manage separate sets of credentials for different AWS accounts and regions. Each time they accessed a different account or region, they had to provide the appropriate credentials.

AWS Single Sign-On (SSO) simplifies this process by enabling centralized user management. It was made available in the aws-cli with the V2.

Why do we use SSO?

The main advantages are:

Simplified Access Management: administrators can define user permissions centrally and apply them across multiple accounts, making it easier to grant or revoke access as needed.
Centralized User Management: administrators can manage user identities, roles, and permissions from a single location, making it efficient to handle user onboarding, offboarding, and role changes.
Integration with Identity Providers (IdP): it allows organizations to leverage their existing identity systems and enable users to log in using their familiar corporate credentials. (IdP: Microsoft Active Directory, Okta, Azure Active Directory, …).
Single Sign-On Experience: once authenticated with their organization's IdP, users can access various AWS accounts and applications without needing to re-enter credentials, improving productivity and reducing authentication fatigue.
Increased Security: reduces the risk of password-related issues. Users don't need to remember multiple passwords, reducing the likelihood of weak or reused passwords. Centralized access management also ensures the consistent application of security policies and allows for easier auditing and monitoring of user activity.

Overall, AWS SSO simplifies the management of user access to AWS accounts and applications, enhances security, and provides a better user experience by enabling single sign-on functionality.

How to configure a profile?

You need to have a user account within an organization. Go to the start url of the access portal and sign in. (It looks like this: https://sso-portal.awsapps.com/start). Once you're signed in with your unique identifier, you will have access to all the accounts the organization has set for you.

Now you can configure your profile locally. The most common way to do so is with the aws-cli command

aws configure sso

It will save your profile configuration in the ~/.aws/config file.

What are the different parameters?

AWS added support for SSO with CLI v2. A configuration file looks like this:

[profile dev]
sso_account_id = 123456789011
sso_role_name = FullAccess
sso_region = eu-west-2
sso_start_url = https://sso-portal.awsapps.com/start
region = west-1
output = json

You can have different profiles that point to the same account. However, the parameters give the profiles different use cases:

sso_account_id: Your account id with the organization. (1 account id = 1 line on the access portal)
sso_role_name: The IAM role that defines the permissions a user with this profile will have.
sso_region: The region where the access portal is hosted.
sso_start_url: The URL to your organization's access portal (you can get it in the invitation email or the IAM Identity Center service in the console).
region: All the commands made through this profile will be sent to this region.
output: The default output format for the commands.

You may already be used to this if you used the previous legacy configuration. Let's see what changed.

New configuration: SSO introduces sso-sessions

In the 2.90 release of the aws-cli, AWS introduced sessions to manage our SSO profiles. Now, we only have one set of credentials for our session (instead of one set per profile) and we do not need to refresh our tokens periodically when they expire. The refreshed tokens are automatically fetched by our SDK or tool (ie AWS SDK).

What changed in the new configuration: sso-sessions

We can now add sessions in our SSO configuration file. A session is linked to a start url and the region where the start url is hosted (different from the url where your account is hosted).

Then for one session, you can have multiple profiles. I can have a single session for my project start url, and different profiles linked to this session (so I have to configure only the session details once).

The credentials are now stored at the session level, instead of having one set of credentials per profile.

Another feature of the session is that you can restrict the access scope of the profiles. For the same start url, you could have a session allowing the profiles default access to your accounts, and one session restricting its profiles to only Read Access (e.g.). The default config is to grant the profile the access set to your listed accounts on the start url.

What is in the new configuration file

Now, in your config file, in addition to having the regular properties listed above, you can have a session linked to a profile. The start url, region, and registration scope are linked to the session, to better reflect what is going on in aws.

[profile dev-profile]
sso_session = devto-article
sso_account_id = 123456789011
sso_role_name = FullAccess
region = eu-west-1
output = json

[sso-session devto-article]
sso_region = eu-west-2
sso_start_url = https://sso-portal.awsapps.com/start
sso_registration_scopes = sso:account:access

Let's break it down:

In the profile, we now have sso_session points to the session linked to this profile
The fields sso_region and sso_start_url are moved to the session's configuration, as they do not depend on the profile but on the access portal.
There is a new parameter, sso_registration_scopes that grants the scopes allowed to an application. It is the minimum access the accounts need to have to retrieve the access tokens. There is still very little documentation about it, here is what AWS provides.

Let's add a new profile with a session

This time, we are going to add a profile that only has read access to the account.

$ aws configure sso

SSO session name (Recommended): devto-article
SSO start URL [None]: https://sso-portal.awsapps.com/start
SSO region [None]: eu-west-2
SSO registration scopes [sso:account:access]:

Using the account ID: 123456789011
There are 2 roles available to you.
> ReadOnly
  FullAccess
Using the role name "ReadOnly"

CLI default client region [eu-west-1]: eu-west-1
CLI default output format [json]: json
CLI profile name [123456789011_ReadOnly]: test-profile

Our configuration file now looks like this:

[profile dev-profile]
sso_session = devto-article
sso_account_id = 123456789011
sso_role_name = FullAccess
region = eu-west-1
output = json

[profile test-profile]
sso_session = devto-article
sso_account_id = 123456789012
sso_role_name = ReadOnly
region = eu-west-1
output = json

[sso-session devto-article]
sso_region = eu-west-2
sso_start_url = https://sso-portal.awsapps.com/start
sso_registration_scopes = sso:account:access

Tips

You can you this profile by running. The command will automatically open an authorization page in your browser and fill in the authorization code.

aws sso login --profile test-profile

To log out:

aws sso logout

To know which profile you are using, you can run:

aws sts get-caller-identity

Take-aways

AWS SSO makes it easy for you to switch between profiles and to make the best use of the start url features. You don't have to deal with access keys anymore!
SSO sessions add one level of abstraction to have different access patterns with the same organization.
Overall, the commands to set a profile and session do not change a lot, but if you decide to use sso sessions, make sure that the non-native AWS tools you use are maintained and have integrated the changes.
An easy tool that supports SSO to manage your profiles is Granted, I will write a follow-up article on it!

DEV Community: Anaïs Schlienger

Avoiding API Gateway’s integrations hard limit: scaling serverless architectures efficiently

The challenge: reaching API Gateway’s 300 integration limit

Evaluating the options: lambda-lith vs micro-APIs

The solution: multi-API Gateways behind CloudFront

Step 1: create a new API Gateway for the user service

Step 2: Link service lambdas to the new API gateway

Step 3: Update API routes

Step 4: Update frontend API calls

Step 5: Redirect Service Requests via CloudFront

The benefits of this approach

Conclusion

References

Override the 500 resource limit of AWS CloudFormation templates with Serverless Framework

Prerequisites

Splitting Stacks with serverless-plugin-split-stacks

Installation

Configuration

Renaming Existing Lambdas

Deploying the Stacks

Example

💭 Limitations

🧠 Conclusion

References

Understand the AWS SSO login configuration

TL;DR

What is SSO?

How did we do before SSO?

Why do we use SSO?

How to configure a profile?

What are the different parameters?

New configuration: SSO introduces sso-sessions

What changed in the new configuration: sso-sessions

What is in the new configuration file

Let's add a new profile with a session

Tips

Take-aways

Step 1: create a new API Gateway for the `user` service