Connect lambda and RDS with RDS proxy using AWS CDK(ver.2) python

shinya — Tue, 13 Jun 2023 06:56:21 +0000

Since I had the opportunity to develop with AWS CDK for a project, I would share the knowledge I gained from the development.
In this article, I will explain how to use CDK (ver. 2) Python to connect from an AWS Lambda to RDS in a private subnet via RDS Proxy.

See Github for full code.
Github here

Prerequisites.

AWS CDK v.2 must be installed
Python 3.x must be installed
AWS account and AWS CLI must be set up.

Cloud9 can be used to pass the above, so the content of this article was created using Cloud9.

Build procedure

Initialize CDK application

First, initialize the CDK application.

$ mkdir cdk-lambda-rds-proxy
$ cd cdk-lambda-rds-proxy
$ cdk init --language python

2. install necessary packages

Here we activate the .venv created when we initialized the CDK app.

$ source .venv/bin/activate
$ pip install -r requirements.txt

3. create the stack

from constructs import Construct
from aws_cdk import (
    Stack,
    RemovalPolicy,
    aws_ec2 as ec2,
    aws_lambda as _lambda,
    aws_rds as rds,
    aws_secretsmanager as sm,
)


class CdkLambdaRdsProxyStack(Stack):
    def __init__(self, scope: Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        # Create VPC
        vpc = ec2.Vpc(self, "MyVPC",
            max_azs=2
        )

        # Create security group for RDS
        rds_security_group = ec2.SecurityGroup(
            self, 
            "RdsSecurityGroup",
            vpc=vpc,
            description="Allow Lambda access to RDS"
        )

        # Create RDS instance
        rds_instance = rds.DatabaseInstance(
            self, 
            "RDS",
            engine=rds.DatabaseInstanceEngine.postgres(
                version=rds.PostgresEngineVersion.VER_14_5
            ),
            vpc=vpc,
            security_groups=[rds_security_group],
            instance_type=ec2.InstanceType.of(
                ec2.InstanceClass.BURSTABLE3, ec2.InstanceSize.SMALL
            ),
            allocated_storage=20,
            removal_policy=RemovalPolicy.DESTROY,
            deletion_protection=False,
            database_name="MyDatabase",
            auto_minor_version_upgrade=True,
            multi_az=False
        )

        # Create security group for lambda
        lambda_security_group = ec2.SecurityGroup(
            self, 
            "LambdaSecurityGroup",
            vpc=vpc,
            description="Allow Lambda to access RDS Proxy"
        )

        # Create RDS Proxy
        rds_proxy = rds.DatabaseProxy(
            self, 
            "RDSProxy",
            proxy_target=rds.ProxyTarget.from_instance(rds_instance),
            vpc=vpc,
            security_groups=[lambda_security_group],
            db_proxy_name="MyRDSProxy",
            debug_logging=False,
            secrets=[rds_instance.secret],
            require_tls=True
        )

        # lambda layer of psycopg2
        psycopg2_layer = _lambda.LayerVersion.from_layer_version_arn(
            self, 
            "Psycoog2Layer",
            'arn:aws:lambda:ap-northeast-1:898466741470:layer:psycopg2-py38:1'
        )

        # Create lambda function
        lambda_function = _lambda.Function(
            self, 
            "LambdaFunction",
            runtime=_lambda.Runtime.PYTHON_3_8,
            handler="lambda_function.handler",
            code=_lambda.Code.from_asset("cdk_lambda_rds_proxy/lambda"),
            layers=[psycopg2_layer],
            vpc=vpc,
            security_groups=[lambda_security_group],
            vpc_subnets=ec2.SubnetSelection(subnets=vpc.private_subnets),
            environment={
                "DB_SECRET_ARN": rds_instance.secret.secret_arn,
                "DB_PROXY_ENDPOINT": rds_proxy.endpoint
            }
        )

        # Policies
        rds_instance.connections.allow_from(
            lambda_security_group,
            port_range=ec2.Port.tcp(5432)
        )
        rds_proxy.connections.allow_from(
            lambda_security_group,
            port_range=ec2.Port.tcp(5432)
        )

        rds_instance.secret.grant_read(lambda_function)

4. Create a Lambda function

Create a lambda folder, create a lambda_function.py file in it, and write the following

import os
import json
import psycopg2
import boto3
from botocore.exceptions import ClientError

def get_db_credentials(secret_arn):.
    secrets_manager = boto3.client("secretsmanager")
    try:: secrets_manager = boto3.client("secretsmanager")
        response = secrets_manager.get_secret_value(SecretId=secret_arn)
        return json.loads(response["SecretString"])
    except ClientError as e: print(e)
        print(e)
        return None

def handler(event, context):
    secret_arn = os.environ["DB_SECRET_ARN"]]
    db_proxy_endpoint = os.environ["DB_PROXY_ENDPOINT"]]

    credentials = get_db_credentials(secret_arn)

    if not credentials: return { if not credentials
        return {
            "statusCode": 500, "body": "Error retrieving
            "body": "Error retrieving database credentials"
        }

    connection_string = f "dbname={credentials['dbname']} user={credentials['username']} password={credentials['password']} host={db_proxy_ endpoint} port={credentials['port']} sslmode=require"

    try:.
        conn = psycopg2.connect(connection_string)
        cursor = conn.cursor()
        cursor.execute("SELECT version();")
        result = cursor.fetchone()

        return {
            "statusCode": 200,.
            "body": f "Database version: {result[0]}"
        }

    except Exception as e:.
        print(e)
        return {
            "statusCode": 500, "body": "Error connecting to the database" }
            "body": "Error connecting to the database"
        }

5. Deploy the CDK app

Now that the Stack is ready, deploy it.

$ cdk deploy

Once deployed, the Lambda function will connect to the RDS database to retrieve version information.

↓↓ Actual connection confirmed.

Summary

This article described how to use CDK(ver.2) Python to connect from a Lambda function to an RDS instance in a private subnet via RDS Proxy.
It seems to be an anti-pattern to connect Lambda and RDS directly, so please refer to this when configuring Lambda + RDS.

Reference

AWS CDK API Reference

DEV Community: shinya