Monitoring AWS RDS Postgres Parameter Changes

When working with AWS RDS (Relational Database Service) instances, it’s important to monitor and manage the parameters configured for your databases. These parameters can be adjusted for performance optimization, security, or any other specific use case. AWS provides an easy way to manage and view these parameters using the AWS Management Console, but there’s a powerful alternative: you can use the AWS CLI or SDKs like boto3 to automate the process.

In this article, we’ll explore how to track changes in AWS RDS PostgreSQL parameters, focusing on retrieving parameters that have been modified from their default values, using both AWS CLI and Python with boto3.

Understanding AWS RDS Parameters

AWS RDS uses parameter groups to manage the settings for your databases. These parameter groups define the configuration options for a specific database engine (in this case, PostgreSQL). Parameters can be adjusted based on your workload’s needs. The changes you make can impact the performance, security, or behavior of the database.

Parameters are typically categorized into two types:

• User-modified parameters: These are the parameters that have been manually modified by the user.
• Default parameters: These are the values that are set by AWS for the RDS instance when it is created, before any customization.

Fetching Modified Parameters Using AWS CLI

The easiest way to see parameters that have been modified from their defaults is by using the AWS CLI. You can list the parameters in a specific RDS parameter group and filter the ones that have been altered by the user.

For example, the following command queries parameters from a specified PostgreSQL parameter group and filters out the ones that are either modified by the user (Source=='user') or set to apply immediately (ApplyMethod=='immediate'):

aws rds describe-db-parameters \
  --db-parameter-group-name postgres-16-dima \
  --region us-east-1 \
  --query "Parameters[?Source=='user' || ApplyMethod=='immediate'].[ParameterName]" \
  --output text

• --db-parameter-group-name: This is the name of your PostgreSQL parameter group.
• --region: The AWS region where your RDS instance resides.
• --query: This is used to filter parameters based on Source or ApplyMethod.
• --output text: Outputs the result as plain text.

This query will give you a list of parameter names that are either modified by the user or set to apply immediately.

Automating the Process Using Python and Boto3

Now let’s take this a step further by automating the process using Python with the boto3 library. Below is a Python script that connects to AWS RDS, retrieves parameters for a given parameter group, and filters those that have been modified by the user or require immediate application.

Python Script:

import boto3

def get_changed_parameters(db_parameter_group_name, region):
    # Create a session using your AWS credentials and region
    session = boto3.Session(region_name=region)
    rds_client = session.client('rds')

    try:
        # List to hold filtered parameters
        filtered_params = []

        # Start pagination
        marker = None
        while True:
            # Describe DB parameters for the specified parameter group with pagination support
            if marker:
                # If there is a marker, use it to get the next set of results
                response = rds_client.describe_db_parameters(
                    DBParameterGroupName=db_parameter_group_name,
                    Marker=marker
                )
            else:
                # Initial request, no marker required
                response = rds_client.describe_db_parameters(
                    DBParameterGroupName=db_parameter_group_name
                )

            # Filter parameters based on Source and ApplyMethod conditions
            for param in response['Parameters']:
                # Check if the parameter has been modified by the user or has an immediate apply method
                if param.get('Source') == 'user' or param.get('ApplyMethod') == 'immediate':
                    # Append parameter details, including default and changed values
                    filtered_params.append({
                        'ParameterName': param['ParameterName'],
                        'ParameterValue': param.get('ParameterValue'),
                        'DefaultValue': param.get('DefaultValue')
                    })

            # Check if there are more pages
            marker = response.get('Marker')
            if not marker:  # No more pages, exit the loop
                break

        if filtered_params:
            print("Changed Parameters:")
            for param in filtered_params:
                print(f"Parameter: {param['ParameterName']}")
                print(f"  Default Value: {param['DefaultValue']}")
                print(f"  Current Value: {param['ParameterValue']}")
                print("-----")
        else:
            print("No parameters match the specified conditions.")

    except Exception as e:
        print(f"Error: {e}")

# Example usage
if __name__ == '__main__':
    db_parameter_group_name = 'postgres-16-dima'  # Replace with your DB parameter group name
    region = 'us-east-1'  # Replace with your region
    get_changed_parameters(db_parameter_group_name, region)

Breakdown of the Code:

1. Boto3 Session: We create a boto3 session with the specified AWS region.
2. Describe DB Parameters: The describe_db_parameters function is used to fetch parameters from the specified parameter group.
3. Pagination: If there are more than 50 parameters (the AWS default maximum per response), pagination is supported with the Marker parameter to fetch the next set of results.
4. Filtering: We filter parameters based on whether they have been modified by the user (Source == 'user') or have an immediate apply method (ApplyMethod == 'immediate').
5. Displaying Results: The script outputs the parameter name along with both its current value (ParameterValue) and default value (DefaultValue).

Example Output:

Changed Parameters:
Parameter: max_standby_archive_delay
  Default Value: None
  Current Value: 900000
-----
Parameter: max_standby_streaming_delay
  Default Value: None
  Current Value: 900000
-----

Conclusion:

Monitoring and managing AWS RDS PostgreSQL parameters can be made more efficient by automating the process using AWS CLI or the boto3 library in Python. The approach we covered in this article allows you to easily retrieve the parameters that have been changed from their defaults, and even view both the default and modified values for a better understanding of your database configuration.

With this knowledge, you can optimize your RDS PostgreSQL instances more effectively, ensuring that they align with your application's performance and reliability requirements.