Introduction
In this article we'd like to explore how to connect Lamdba function to the Aurora database through the RDS Proxy using Java JDBC capabilities. You can explore RDS Proxy capabilities in the Using Amazon RDS Proxy article. We will use Aurora Serverless v2 PostgreSQL database in this article but the same will also work Amazon Aurora PostgreSQL-Compatible Edition, Amazon Aurora MySQL-Compatible Edition, Amazon RDS for PostgreSQL, Amazon RDS for MySQL, Amazon RDS for MariaDB, and Amazon RDS for SQL Server as well. And it's easy to re-write Lambda function using different programming language as the basic concepts remain the same.
Solution
What we'd like to achieve is described in the following architecture diagram:
We'll focus on the part where Lambda connects through RDS Proxy to Aurora database. As the code sample we will use what we have created as part of the Data API for Amazon Aurora Serverless v2 with AWS SDK for Java series, and the project can be found on my GitHub repository. The relevant part of the Infrastructure as a Code can be found in this AWS SAM template.
First of all, we need to create VPC to put Lambda, RDS Proxy and Aurora Cluster to, so that Lambda is able to talk to RDS Proxy which in turn connects to Aurora Cluster. For that, please define your own VPC Id and Subnet list in the "Parameter" section of the SAM template like this :
VpcId:
Type: String
Default: vpc-950cd6fd
Description: VpcId of your existing Virtual Private Cloud (VPC)
Subnets:
Type: CommaDelimitedList
Default: subnet-0787be4d, subnet-88dc46e0
Description: The list of SubnetIds, for at least two Availability Zones in the
region in your Virtual Private Cloud (VPC)
Then we need to define Lambda Security Group which references the VPC Id:
LambdaSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: SecurityGroup for Serverless Functions
VpcId:
Ref: VpcId
and then VPC Security Group itself:
VPCSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Security group for RDS DB Instance.
VpcId:
Ref: VpcId
SecurityGroupEgress:
- CidrIp: '0.0.0.0/0'
Description: lambda RDS access over 5432
FromPort: 5432
IpProtocol: TCP
ToPort: 5432
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: '5432'
ToPort: '5432'
SourceSecurityGroupId:
Ref: LambdaSecurityGroup
VPC Security Group defines Security Group engress and igress roules which enable Lambda to talk to RDS Proxy via the port number 5432 (default port for PostgreSQL) and references the defined Lambda Security Group.
Now let's create Aurora Serverless v2 PostgreSQL database cluster. But before the let's create SecretsManager to store the database user and password:
DBSecret:
Type: AWS::SecretsManager::Secret
Properties:
Name: !Ref UserSecret
Description: RDS database auto-generated user password
GenerateSecretString:
SecretStringTemplate: !Sub '{"username": "${DBMasterUserName}"}'
GenerateStringKey: "password"
PasswordLength: 30
ExcludeCharacters: '"@/\'
Then we need to define DB Subnet Group:
DBSubnetGroup:
Type: AWS::RDS::DBSubnetGroup
Properties:
DBSubnetGroupDescription: Subnets available for the RDS DB Instance
SubnetIds:
Ref: Subnets
and then create the Aurora Cluster with the engine "aurora-postgresql" itself:
AuroraServerlessV2Cluster:
Type: 'AWS::RDS::DBCluster'
DeletionPolicy: Delete
Properties:
DBClusterIdentifier: !Ref DBClusterName
Engine: aurora-postgresql
Port: 5432
EnableHttpEndpoint: true
MasterUsername: !Join ['', ['{{resolve:secretsmanager:', !Ref DBSecret, ':SecretString:username}}' ]]
MasterUserPassword: !Join ['', ['{{resolve:secretsmanager:', !Ref DBSecret, ':SecretString:password}}' ]]
DatabaseName: !Ref DatabaseName
ServerlessV2ScalingConfiguration:
MinCapacity: 0.5
MaxCapacity: 1
DBSubnetGroupName:
Ref: DBSubnetGroup
VpcSecurityGroupIds:
- !Ref VPCSecurityGroup
In the last lines we reference already created DB Subnet Group and Vpc Security Group Ids. With ServerlessV2ScalingConfiguration we define the scaling behaviour of our Aurora Serverless V2 Cluster.
After it we need to create database instance of the Aurora (Serverless v2) cluster :
AuroraServerlessV2Instance:
Type: 'AWS::RDS::DBInstance'
Properties:
Engine: aurora-postgresql
DBInstanceClass: db.serverless
DBClusterIdentifier: !Ref AuroraServerlessV2Cluster
MonitoringInterval: 1
MonitoringRoleArn: !GetAtt EnhancedMonitoringRole.Arn
PubliclyAccessible: false
EnablePerformanceInsights: true
PerformanceInsightsRetentionPeriod: 7
Now let's take care of the creation of the RDS Proxy itself :
RDSProxy:
Type: AWS::RDS::DBProxy
Properties:
Auth:
- { AuthScheme: SECRETS, SecretArn: !Ref DBSecret}
DBProxyName: 'rds-proxy'
RoleArn: !GetAtt RDSProxyRole.Arn
EngineFamily: 'POSTGRESQL'
IdleClientTimeout: 120
RequireTLS: true
DebugLogging: false
VpcSecurityGroupIds:
- !Ref VPCSecurityGroup
VpcSubnetIds: !Ref Subnets
We use EngineFamily 'POSTGRESQL' and AuthScheme connected to the created SecretsManager secret for authentication in our scenario. There are also other authentication possibilities like IAM authentication offered. We also connect RDS Proxy to the already created Vpc Security Group and Vpc Subnet Id.
We also need to create RDS Proxy Role to allow RDS Proxy to fetch secrets from the SecretsManager:
RDSProxyRole:
Type: AWS::IAM::Role
Properties:
Path: /
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Action: [ 'sts:AssumeRole' ]
Effect: Allow
Principal:
Service: [ rds.amazonaws.com ]
Policies:
- PolicyName: DBProxyPolicy
PolicyDocument:
Version: '2012-10-17'
Statement:
- Action:
- secretsmanager:GetSecretValue
Effect: Allow
Resource:
- !Ref DBSecret
As the final step we need to create DB Proxy Target Group:
ProxyTargetGroup:
Type: AWS::RDS::DBProxyTargetGroup
Properties:
DBProxyName: !Ref RDSProxy
DBClusterIdentifiers: [ !Ref AuroraServerlessV2Cluster]
TargetGroupName: default
ConnectionPoolConfigurationInfo:
MaxConnectionsPercent: 5
MaxIdleConnectionsPercent: 4
ConnectionBorrowTimeout: 120
DB Proxy Target Group connects RDS Proxy with the Aurora V2 Cluster and defines Connection Pool Configuration. Please refer to Configure Connection Settings documentation for the extended explanation of this configuration.
Now let's connect Lambda function "GetProductByIdViaAuroraServerlessV2WithRDSProxy" to the RDS Proxy.
The relevant part is this one:
Policies:
- Statement:
- Sid: AllowDbConnect
Effect: Allow
Action:
- rds-db:connect
Resource:
- !Sub arn:aws:rds-db:${AWS::Region}:${AWS::AccountId}:dbuser:!Select [6, !Split [":", !GetAtt RDSProxy.DBProxyArn]]/*
- VPCAccessPolicy: {}
VpcConfig:
SecurityGroupIds:
- Fn::GetAtt: LambdaSecurityGroup.GroupId
SubnetIds: !Ref Subnets
In the VPCAccessPolicy part we reference already created Security Groups and Subnets. In the Policies part we allow Lambda to connect to our RDS Proxy instance by defining the exact resource ARN by knowing how the resource schema name works but extracting the appropriate RDSProxy ARN part from it. For the exact explanation please visit Creating and using an IAM policy for IAM database access.
The word of caution: only for the demonstration purpose I passed the database name and password as the Lambda environment variables to connect to RDS Proxy which introduces the security risk.
Environment:
Variables:
DB_USER_PASSWORD: !Join ['', ['{{resolve:secretsmanager:', !Ref DBSecret, ':SecretString:password}}' ]]
DB_USER_NAME: !Join ['', ['{{resolve:secretsmanager:', !Ref DBSecret, ':SecretString:username}}' ]]
RDS_PROXY_ENDPOINT: !GetAtt RDSProxy.Endpoint
The proper solution is to use the stored database name and password in Amazon Secret Manager and then retrieve the in the Lambda function itself.
That it on the Infrastructure as a Code level.
Everything else will be done in the Lambda handler itself to connect to the RDS Proxy. In my case I wrote GetProductByIdViaAuroraServerlessV2RDSProxyHandler in Java and used JDBC for the connection with the PostgreSQL database driver in the pom.xml like this.
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>42.5.4</version>
</dependency>
Each programming languge in which Lambda function can be implemented offers its own capabilities to connect to the database.
Conclusion
In this article we explored how to connect Lamdba function to the Aurora database through the RDS Proxy and demonstrated the example of the corresponding Infrastructure as a Code part and the Lambda function written in Java which uses JDBC to connect to RDS Proxy.
Top comments (1)
nice