DEV Community

AWS Community Builders profile image Vadym Kazulkin
Vadym Kazulkin for AWS Community Builders

Posted on • Edited on

How to set up Amazon RDS Proxy for Amazon Aurora (Serverless) database cluster and connect AWS Lambda function to it

#aws #serverless #postgres #java

Introduction

In this article, we'd like to explore how to connect Lamdba function to the Aurora database through the RDS Proxy using Java JDBC capabilities. You can explore RDS Proxy capabilities in the Using Amazon RDS Proxy article. We will use Aurora Serverless v2 PostgreSQL database in this article, but the same will also work with Amazon Aurora PostgreSQL-Compatible Edition, Amazon Aurora MySQL-Compatible Edition, Amazon RDS for PostgreSQL, Amazon RDS for MySQL, Amazon RDS for MariaDB, and Amazon RDS for SQL Server as well. And it's easy to re-write Lambda function using a different programming language, as the basic concepts remain the same.

Solution

What we'd like to achieve is described in the following architecture diagram:

We'll focus on the part where Lambda connects through RDS Proxy to the Aurora database. As the code sample we will use what we have created as part of the Data API for Amazon Aurora Serverless v2 with AWS SDK for Java series, and the project can be found on my GitHub repository. The relevant part of the Infrastructure as a Code can be found in this AWS SAM template.

First of all, we need to create a VPC to put Lambda, RDS Proxy and Aurora Cluster in, so that Lambda is able to talk to RDS Proxy, which in turn connects to Aurora Cluster. For that, please define your own VPC Id and Subnet list in the "Parameter" section of the SAM template like this :

  VpcId:
    Type: String
    Default: vpc-950cd6fd
    Description: VpcId of your existing Virtual Private Cloud (VPC)
  Subnets:
    Type: CommaDelimitedList  
    Default: subnet-0787be4d, subnet-88dc46e0
    Description: The list of SubnetIds, for at least two Availability Zones in the
      region in your Virtual Private Cloud (VPC)
Enter fullscreen mode Exit fullscreen mode

Then we need to define Lambda Security Group, which references the VPC Id:

  LambdaSecurityGroup:
      Type: AWS::EC2::SecurityGroup
      Properties:
        GroupDescription: SecurityGroup for Serverless Functions
        VpcId: 
          Ref: VpcId
Enter fullscreen mode Exit fullscreen mode

And then define the VPC Security Group itself:

  VPCSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Security group for RDS DB Instance.
      VpcId: 
         Ref: VpcId 
      SecurityGroupEgress: 
        - CidrIp: '0.0.0.0/0'
          Description: lambda RDS access over 5432
          FromPort: 5432
          IpProtocol: TCP
          ToPort: 5432
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: '5432'
          ToPort: '5432'
          SourceSecurityGroupId:
               Ref: LambdaSecurityGroup
Enter fullscreen mode Exit fullscreen mode

VPC Security Group defines Security Group egress and ingress rules, which enable Lambda to talk to RDS Proxy via the port number 5432 (default port for PostgreSQL) and references the defined Lambda Security Group.

Now let's create an Aurora Serverless v2 PostgreSQL database cluster. But before, let's create a SecretsManager to store the database user and password:

  DBSecret:
    Type: AWS::SecretsManager::Secret
    Properties:
      Name: !Ref UserSecret
      Description: RDS database auto-generated user password
      GenerateSecretString:
        SecretStringTemplate: !Sub '{"username": "${DBMasterUserName}"}'
        GenerateStringKey: "password"
        PasswordLength: 30
        ExcludeCharacters: '"@/\'
Enter fullscreen mode Exit fullscreen mode

Then we need to define the DB Subnet Group:

  DBSubnetGroup:
    Type: AWS::RDS::DBSubnetGroup
    Properties:
      DBSubnetGroupDescription: Subnets available for the RDS DB Instance
      SubnetIds:
       Ref: Subnets
Enter fullscreen mode Exit fullscreen mode

Then we create the Aurora Cluster with the engine "aurora-postgresql" itself:

 AuroraServerlessV2Cluster:
    Type: 'AWS::RDS::DBCluster'
    DeletionPolicy: Delete
    Properties:
      DBClusterIdentifier: !Ref DBClusterName
      Engine: aurora-postgresql
      Port: 5432
      EnableHttpEndpoint: true
      MasterUsername: !Join ['', ['{{resolve:secretsmanager:', !Ref DBSecret, ':SecretString:username}}' ]]
      MasterUserPassword: !Join ['', ['{{resolve:secretsmanager:', !Ref DBSecret, ':SecretString:password}}' ]]
      DatabaseName: !Ref DatabaseName
      ServerlessV2ScalingConfiguration:
        MinCapacity: 0.5
        MaxCapacity: 1
      DBSubnetGroupName:
        Ref: DBSubnetGroup
      VpcSecurityGroupIds:
      - !Ref VPCSecurityGroup
Enter fullscreen mode Exit fullscreen mode

In the last lines, we reference the already created DB Subnet Group and Vpc Security Group Ids. With ServerlessV2ScalingConfiguration, we define the scaling behaviour of our Aurora Serverless V2 Cluster.

After it, we need to createa database instance of the Aurora (Serverless v2) cluster :

  AuroraServerlessV2Instance:
    Type: 'AWS::RDS::DBInstance'
    Properties:
      Engine: aurora-postgresql
      DBInstanceClass: db.serverless
      DBClusterIdentifier: !Ref AuroraServerlessV2Cluster
      MonitoringInterval: 1
      MonitoringRoleArn: !GetAtt EnhancedMonitoringRole.Arn
      PubliclyAccessible: false
      EnablePerformanceInsights: true
      PerformanceInsightsRetentionPeriod: 7
Enter fullscreen mode Exit fullscreen mode

Now let's take care of the creation of the RDS Proxy itself :

  RDSProxy:
    Type: AWS::RDS::DBProxy
    Properties:
      Auth:
        - { AuthScheme: SECRETS, SecretArn: !Ref DBSecret}
      DBProxyName: 'rds-proxy'
      RoleArn: !GetAtt RDSProxyRole.Arn
      EngineFamily: 'POSTGRESQL'
      IdleClientTimeout: 120
      RequireTLS: true
      DebugLogging: false
      VpcSecurityGroupIds:
      - !Ref VPCSecurityGroup
      VpcSubnetIds: !Ref  Subnets
Enter fullscreen mode Exit fullscreen mode

We use EngineFamily 'POSTGRESQL' and AuthScheme connected to the created SecretsManager secret for authentication in our scenario. There are also other authentication possibilities like IAM authentication offered. We also connect RDS Proxy to the already created Vpc Security Group and Vpc Subnet Id.

We also need to create an RDS Proxy Role to allow the RDS Proxy to fetch secrets from the SecretsManager:

  RDSProxyRole:
    Type: AWS::IAM::Role
    Properties:
      Path: /
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Action: [ 'sts:AssumeRole' ]
            Effect: Allow
            Principal:
              Service: [ rds.amazonaws.com ]
      Policies:
        - PolicyName: DBProxyPolicy
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Action:
                  - secretsmanager:GetSecretValue
                Effect: Allow
                Resource:
                  - !Ref DBSecret
Enter fullscreen mode Exit fullscreen mode

As the final step, we need to create a DB Proxy Target Group:

  ProxyTargetGroup:
    Type: AWS::RDS::DBProxyTargetGroup
    Properties:
      DBProxyName: !Ref RDSProxy
      DBClusterIdentifiers: [ !Ref AuroraServerlessV2Cluster]
      TargetGroupName: default
      ConnectionPoolConfigurationInfo:
        MaxConnectionsPercent: 5
        MaxIdleConnectionsPercent: 4
        ConnectionBorrowTimeout: 120
Enter fullscreen mode Exit fullscreen mode

DB Proxy Target Group connects RDS Proxy with the Aurora V2 Cluster and defines Connection Pool Configuration. Please refer to Configure Connection Settings documentation for the extended explanation of this configuration.

Now let's connect the Lambda function "GetProductByIdViaAuroraServerlessV2WithRDSProxy" to the RDS Proxy.
The relevant part is this one:

      Policies:
        - Statement:
            - Sid: AllowDbConnect
              Effect: Allow
              Action:
                - rds-db:connect
              Resource:
                - !Sub arn:aws:rds-db:${AWS::Region}:${AWS::AccountId}:dbuser:!Select [6, !Split [":", !GetAtt RDSProxy.DBProxyArn]]/*

        - VPCAccessPolicy: {}
      VpcConfig:
        SecurityGroupIds:
          - Fn::GetAtt: LambdaSecurityGroup.GroupId
        SubnetIds: !Ref  Subnets
Enter fullscreen mode Exit fullscreen mode

In the VPCAccessPolicy part, we reference already created Security Groups and Subnets. In the Policies part, we allow Lambda to connect to our RDS Proxy instance by defining the exact resource ARN and knowing how the resource schema name works, and by extracting the appropriate RDSProxy ARN part from it. For the exact explanation, please visit Creating and using an IAM policy for IAM database access.

The word of caution: Only for demonstration purposes, I passed the database name and password as the Lambda environment variables to connect to RDS Proxy, which introduces a security risk.

      Environment:
        Variables:
          DB_USER_PASSWORD: !Join ['', ['{{resolve:secretsmanager:', !Ref DBSecret, ':SecretString:password}}' ]]
          DB_USER_NAME: !Join ['', ['{{resolve:secretsmanager:', !Ref DBSecret, ':SecretString:username}}' ]]
          RDS_PROXY_ENDPOINT: !GetAtt RDSProxy.Endpoint
Enter fullscreen mode Exit fullscreen mode

The proper solution is to use the stored database name and password in Amazon Secret Manager and then retrieve them in the Lambda function itself.

That's it on the Infrastructure as a Code level.

Everything else will be done in the Lambda handler itself to connect to the RDS Proxy. In my case, I wrote GetProductByIdViaAuroraServerlessV2RDSProxyHandler in Java and used JDBC for the connection with the PostgreSQL database driver in the pom.xml like this.

<dependency>
   <groupId>org.postgresql</groupId>
   <artifactId>postgresql</artifactId>
   <version>42.5.4</version>
</dependency>
Enter fullscreen mode Exit fullscreen mode

Each programming language in whicha Lambda function can be implemented offers its own capabilities to connect to the database.

Conclusion

In this article, we explored how to connect Lamdba function to the Aurora database through the RDS Proxy and demonstrated the example of the corresponding Infrastructure as a Code part and the Lambda function written in Java, which uses JDBC to connect to the RDS Proxy.

Top comments (1)

Collapse
 
sundus profile image
Sundus Hussain •

nice