A different way of retrieving secrets in Lambda functions

We can now retrieve secrets with the new Parameter Store and Secrets Manager extension. We don't need the SDK for this operation, which means a smaller package size and faster deployment for our Lambda functions.

1. The new extension

AWS announced the new AWS Parameters and Secrets Lambda Extension a few days ago.

This extension is great because

• we can retrieve secrets without installing the SDK SSM client or Secrets Manager client, which means smaller deployment packages
• it can cache the secrets, and we can configure the TTL, which reduces the number of API calls to Parameter Store or Secrets Manager
• we can configure the extension relatively easily
• it's free.

Let's see how we can incorporate the Parameter Store and Secrets Manager extension to our Lambda functions.

2. Sample implementation

We can relatively easily add the extension to the code.

2.1. Code sample

It's common to use axios for making HTTP requests, so let's do so here.

In this example, we are fetching an API key called /app/environment/api-key from Parameter Store. We have stored the key as a SecureString, and the extension will return the decrypted value.

const axios = require('axios');

const client = axios.create({
  baseURL: `http://localhost:2773`,
  timeout: 5000,
  headers: {
    'x-aws-parameters-secrets-token': process.env.AWS_SESSION_TOKEN
  },
});

exports.handler = async (event) => {
  // code here
  try {
    const ssmResponse = await client.get(
      `/systemsmanager/parameters/get?name=%2Fapp%2Fenvironment%2Fapi-key&version=1&withDecryption=true`
    );
    const secret = ssmResponse.data.Parameter.Value;
    // do something with the secret
  } catch (error) {
    // handle error
  }

  // more code here
}

As the code sample indicates, we use axios to make the HTTP request.

In the example, the API key is called /app/environment/api-key (replace app and environment with the relevant values). The secret is encrypted, so we should specify the withDecryption=true query parameter in the URL. If we omit it, the extension will know that the secret will be in an unencrypted format in Parameter Store.

2.2. Mandatory elements

HTTP request - The extension runs in the Lambda execution environment isolated from the runtime, and we call a localhost endpoint to connect. It means that the URL starts with http and not https.

Port - The default port is 2773, but we can change it by setting a different value to the PARAMETERS_SECRETS_EXTENSION_HTTP_PORT environment variable. I'm not sure if we ever need to do that, but it's good to know that we have this option.

Secret header - We must specify the x-aws-parameters-secrets-token header with the value of the session token. The AWS_SESSION_TOKEN environment variable contains this value. Lambda automatically provides this variable (along with the other credential variables), which is readily available inside the function.

HTTP encoding - If our secret is in a parameter hierarchy format, we must HTTP URL encode the / characters. It means we should write %2F instead of /.

Query parameters - The name and version (or label) query parameters are mandatory in the URL. We should only specify withDecryption when we want to retrieve secrets we store in a SecureString format.

Permissions - The extension will use the Lambda function's execution role. We must give the role ssm:GetParameter permission for the given secret. If the secret is a SecureString, we should also add the kms:Decrypt permission to the KMS key that encrypts the secret.

2.3. Caching

The extension can cache the secret. The SSM_PARAMETER_STORE_TTL environment variable contains the TTL value in seconds. The default caching time is 5 minutes (300 seconds), and we can change it by adding a different value to the environment variable.

If we change the variable's value, the extension won't check if the previous and the current values are the same. It will return the cached value, so in this case, we can either wait for the TTL to expire or turn off caching by setting the value to 0.

With this feature, the extension won't make additional API calls to Parameter Store within the TTL period. In theory, it will lead to faster execution time and protects us from intermittent downtimes in Parameter Store.

3. Errors

We might encounter some errors when we try to set up the extension in the Lambda function.

The most frequent error messages with their potential solutions are the following.

Request failed with status code 404 - The HTTP URL path or the port is incorrect. We should call http://localhost:2773.

HTTP 400 error - There is a chance that we'll see this error message:

...ValidationException: Parameter name: can't be prefixed with "ssm"
(case-insensitive). If formed as a path, it can consist of sub-paths
divided by slash symbol; each sub-path can be formed as a mix of letters,
numbers and the following 3 symbols .-_.

The error message is misleading. Even if the parameter's name doesn't contain ssm, we can still get this error. It usually means that something is wrong with the request URL.

It could be an invalid query string character (? instead of &) or incorrect encoding. The correct encoded version of / is %2F.

HTTP 401 error - The x-aws-parameters-secrets-token is probably not or incorrectly specified in the request.

4. Conclusion

The Parameter Store and Secrets Manager Lambda extension makes it easier for the functions to retrieve secrets from Parameter Store and Secrets Manager. The extension has benefits, like smaller deployment package size and potentially faster execution time.

The extension runs in the Lambda execution environment and is available via an HTTP call from the Lambda function.

5. Further reading

Using Parameter Store parameters in AWS Lambda functions - (Almost) everything about the extension

Lambda Extensions API - How extensions work (hardcore Lambda fans only)

Working with parameter hierarchies - Parameter hierarchy and paths in parameters

Comments

[Chris Cook]

I would actually prefer that they include the AWS SDK v3 on all Node.js runtimes the same way they did it for AWS SDK v2. I know they moved away from this practice in order to let developers pin the exact version they need in package.json.
However, the modular architecture of SDK v3 would allow to do both options: make all SDKs available by default and let developers bundle specific versions with their deployment.