Securely Using AWS Secrets Manager with Lambda in a VPC using Secrets Lambda Extension

Introduction

This guide serves as a comprehensive resource on how to securely use AWS Secrets Manager within a Lambda function in a Virtual Private Cloud (VPC), utilizing the AWS Cloud Development Kit (AWS CDK) and the AWS Parameters and Secrets Lambda Extension. Designed to be both a personal reminder and a helpful tool for others, it delves into the critical aspects of integrating Secrets Manager with Lambda functions, ensuring enhanced security and efficient management of sensitive data.

Whether you are refining your AWS skills or are new to this domain, this guide aims to provide clear, actionable steps and insights for setting up a robust and secure environment in AWS, covering everything from initial configuration to advanced integration techniques.

Steps to Implement

1. Create or Reference a Secret

   • This involves either creating a new secret in AWS Secrets Manager or referencing an existing one.

2. Create a Security Group

   • This security group should allow outbound traffic, necessary for the Lambda function's operations.

3. Create Parameters and Secrets Instance

   • Set up the instance that will manage the parameters and secrets for the Lambda function.

4. Create a Lambda Function

   • Add VPC: Ensure the Lambda function is added to the correct VPC.
   • Add Security Group: Attach the previously created security group to the Lambda function.
   • Add Parameters and Secrets Instance: Incorporate the parameters and secrets instance into the Lambda function.
   • Use Node 18 or Later: Ensure that the Lambda function is using Node 18 or later, to make use of fetch.
   • Pass the secret name to the Lambda function: Pass the secret name to the Lambda function as an environment variable.

5. Grant Read Access to the Lambda Role

   • Modify the IAM role associated with the Lambda function to grant it read access to the secret.

6. Add Interface Endpoint for Secrets Manager to the VPC

   • Attach the Security Group to the Endpoint: Link the security group to the Secrets Manager endpoint.
   • Enable Private DNS for the Endpoint: Ensure that the endpoint can be resolved within the VPC using private DNS.
   • Optionally Set Subnet Selection: If needed, specify which subnets the endpoint should be associated with.

7. Lambda function example

   • set the X-Aws-Parameters-Secrets-Token header to the value of the AWS_SESSION_TOKEN environment variable
   • get the secret name from the SECRET_NAME environment variable
   • make a request to the local endpoint at port 2773
   • log the response

Example

Create or Reference a Secret

// Create a new secret

const newSecret = new Secret(this, "NewSecret", {
    generateSecretString: {
        passwordLength: 32,
    },
});

Create a Security Group

// Create a security group that allows outbound traffic

declare const vpc: IVpc;

const securityGroup = new SecurityGroup(this, "SecurityGroup", {
    vpc: vpc,
    allowAllOutbound: true,
});

Create Parameters and Secrets Instance

// Create an instance that will manage the parameters and secrets for the Lambda function

const paramsAndSecrets = ParamsAndSecretsLayerVersion.fromVersion(
    ParamsAndSecretsVersions.V1_0_103,
    {
        // The maximum number of secrets and parameters to cache.
        // Must be a value from 0 to 1000. A value of 0 means there
        // is no caching. The default is 1000.
        // See https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_lambda.ParamsAndSecretsOptions.html
        cacheSize: 500,
        logLevel: ParamsAndSecretsLogLevel.DEBUG,
    },
);

Create a Lambda Function

// Create a Lambda function

declare const vpc: IVpc;
declare const securityGroup: ISecurityGroup;
declare const paramsAndSecrets: ParamsAndSecretsLayerVersion;
declare const newSecret: ISecret;

const lambdaFunction = new Function(this, "LambdaFunction", {
    code: Code.fromAsset("lambda"),
    handler: "index.handler",
    runtime: Runtime.NODEJS_20_X,
    vpc: vpc,
    securityGroups: [securityGroup],
    layers: [paramsAndSecrets],
    environment: {
        SECRET_NAME: newSecret.secretName,
    },
});

Grant Read Access to the Lambda Role

// Grant read access to the Lambda role

declare const newSecret: ISecret;

newSecret.grantRead(lambdaFunction);

Add Interface Endpoint for Secrets Manager to the VPC

// Add an interface endpoint for Secrets Manager to the VPC

declare const vpc: IVpc;
declare const securityGroup: ISecurityGroup;

const Secrets ManagerEndpoint = new InterfaceVpcEndpoint(
    this,
    "Secrets ManagerEndpoint",
    {
        vpc: vpc,
        service: InterfaceVpcEndpointAwsService.SECRETS_MANAGER,
        securityGroups: [securityGroup],
        privateDnsEnabled: true,
    },
);

Lambda function example

import { APIGatewayProxyHandler } from "aws-lambda";
import { GetSecretValueResponse } from "@aws-sdk/client-secrets-manager";

const handler: APIGatewayProxyHandler = async (event) => {
    // fetch secret
    const result: GetSecretValueResponse = await fetch(
        `http://localhost:2773/secretsmanager/get?secretId=${process.env.SECRET_NAME}`,
        {
            headers: {
                "X-Aws-Parameters-Secrets-Token": process.env
                    .AWS_SESSION_TOKEN as string,
            },
        },
    )
    .then((response) => response.json());

    // don't log secret, just the length
    if(result.SecretString) {
        console.log("Got a secret string with length", result.SecretString.length);
    }

    return {
        statusCode: 200,
        body: JSON.stringify({
            message: "Successfully executed",
            input: event,
        }),
    };
};

export { handler };

References

• https://docs.aws.amazon.com/secretsmanager/latest/userguide/retrieving-secrets_lambda.html
• https://docs.aws.amazon.com/secretsmanager/latest/userguide/vpc-endpoint-overview.html