Configuring Serverless Framework for multiple stages

AWS CloudFormation lets you define "variables" in your templates by specifying Parameters along with a Default value. However, they're limited to static values. That is, you can't provide dynamic values based on stage values or other inputs

You can use dynamic values stored in other AWS services within your CloudFormation template, but that's a little different.

One of my favorite Serverless Framework features is its custom section. Not only does it let you specify static values and values based on inputs, but it also enables you to compose variables and even create maps so that we can have different values based on stage/environment.

Here's a simplified serverless.yaml showing how I configure and use environment-specific values:

provider:
  profile: ${self:custom.stages.${self:provider.stage}.profile}

custom:
  stages:
    dev:
      profile: halfstack_software_dev
      domainEnabled: false
    staging:
      profile: halfstack_software_staging
      domainEnabled: true
      domain: staging.halfstack.software
    prod:
      profile: halfstack_software_prod
      domainEnabled: true
      domain: halfstack.software
  domainName: ${self:custom.stages.${self:provider.stage}.domain}
  domainEnabled: ${self:custom.stages.${self:provider.stage}.domainEnabled}

resources:
  Conditions:
    UseDomainName:
      !Equals
        - ${self:custom.domainEnabled}
        - true

To access a stage-specific value, I use ${self:custom.stages.${self:provider.stage}.domainName} -- quite wordy. To make this more accessible throughout the template, I often duplicate the value into a second variable so that I can access it via ${self:custom.domainName}. Annoying, but manageable.

You can see the example above also includes a profile config for each stage. These all have an accompanying profile entry in ~/.aws/credentials that lets me easily deploy to different AWS accounts based on stage. Keep in mind that for any production system that's receiving traffic, it's dangerous to have production profiles on your local development machines lest you accidentally deploy to production (plus, I hear it's a security risk? 🤷‍♂️).

# ~/.aws/credentials
[halfstack_software_dev]
aws_access_key_id = ABC123DEF456GHI789JK
aws_secret_access_key = aB1Cd2aB1Cd2aB1Cd2aB1Cd2+73f8+nWFQ

One alternative for stage-specific values is the serverless-dotenv-plugin package. This plugin even allows you to create files like .env.development and .env.production and it automatically uses the appropriate file when running NODE_ENV=production sls deploy. 👍

My personal preference is to store my non-secret values in serverless.yaml and resort to .env for secrets only (even better, Use AWS Secrets Manager 🔒). This approach makes it easier for developers to set up the project on their machine (they don't need to create a .env file), and grants you version control over these values since it's checked into your source code repository.

Have any other tips for multi-stage deployments? Comment below or @ me on Twitter.

Comments

[Mohammed Tantawy]

Hi! I found this interesting. Just one question. Is there a way to achieve a deployment of a single stack with multiple stages and using the stage variables viewable from the API Gateway dashboard?

[Alaa-Mahmoud]

i have the same issue i want to deploy multi stages on the same api gateway like we can do from AWS dashboard without creating a new stack , have u found a solution for this ?

[Ken Choong]

If you want to deploy multiple stage with same API,

You can:

• Make multiple stage : dev and prod
• State stageVariable for both stage
• Create a new version for your Lambda, for example it is Version:1
• Create 2 alias of Lambda: dev and prod
• Then make dev alias point to $Latest version
• Make prod alias point to Version:1 (Created at step above)
• Then in your method -> Put/Get/Post-> Integration Request -> Lamba Function-> set the value to: your-lambda-arn:${stageVariables.lambdaAlias} (Note that lambaAlias is what you set at stageVariable above

Then you done. The idea here is,

• prod api stage with stage variable prod point to prod lambda alias which will point to version 1
• dev api stage with stage variable dev point to dev lambda alias which will always point to $Latest

So by this, prod api endpoint will always go to Version 1, which will never changed.
Then you continue the development that will always changing and test. By this, any changes only will happen in api dev stage and wont fucked up in production stage. Production will stay unchanged.

All the stuff will inside 1 stack

I done this all using CDK. You can try in console. Just my 2 cents. hope you find helpful

[kotireddy555]

Really useful, Trying to implement the same however I am struggling to pass the below configuration using serverless yaml.

Put/Get/Post-> Integration Request -> Lamba Function-> set the value to: your-lambda-arn:${stageVariables.lambdaAlias}

Can you please share your thoughts? any snippet?

[Ken Choong]

I using CDK. so I share some of the snippet, then you will get the idea, not sure about Serverless yaml, never used serverless framework.

First define a API uri like this in a CDK contrust

 `   
 class ApiIntegration(core.Construct):
   @property
   def integration(self):
       return self._integration

   def __init__(self, scope: core.Construct, id: str, function: _lambda,  ** kwargs):
      super().__init__(scope, id, **kwargs)

      api_uri = (  # this API uri
          f"arn:aws:apigateway:"
          f"{core.Aws.REGION}"
          f":lambda:path/2015-03-31/functions/"
          f"{function.function_arn}"
          f":"
          f"${{stageVariables.lambdaAlias}}"
          f"/invocations"
       )

      self._integration = apigateway.Integration(
          type=apigateway.IntegrationType.AWS_PROXY,
          integration_http_method="POST", ## here is always POST. 
          uri=api_uri
       )`

The api_uri will make a endpoint to your lambda. The idea is make a URL for your lambda, then APIgateway will call to this URL for integration.

For the URI I define above, you can look at this example (The URI part in Open 3.0) :
docs.aws.amazon.com/apigateway/lat...

And this one, read at the 6th item
docs.aws.amazon.com/apigateway/lat...

If you using CDK, then you integrate your Apigateway like this:

          api = apigateway.LambdaRestApi(
                      self, "YourApiName",
                       #.. all other stuff 
                  )

          # The contruct define above. 
          my_api_integration = ApiIntegration(self, "MyApiIntegration", function="MyFunctionLambda") 

          # pass the constuct to the method 
         my_api_method = api.add_method(
           "POST", integration=my_api_integration.integration)

So I think in your serverless.yaml, you need to define the API uri like I done above. So the process look like this

User make request -> hit your apigateway endpoint -> apigateway hit your lambda using the "API uri"

Why api_uri?

This is the only way you can pass the {stageVariable.lambdaAlias} value to the lambda. When you get this value inside the lambda, then you can do other crazy shit depends to this lambdaAlias value.

So you get the idea. And remember, grant the apigateway endpoint permission to invoke your lambda.

[Ken Choong]

Recently I answer a question about this topic in SO. If interested you can look it here. Cheers

[Ken Choong]

I have create a whole tutorial series in here using CDK about this topic as well, feel free to take a look