Modern organizations often begin with self-managed databases running on virtual machines because they provide complete control and flexibility. As applications grow, however, managing backups, patching, scaling, and high availability becomes an operational burden. A managed database service can significantly reduce that overhead, but moving production data without lengthy downtime is often the biggest challenge.
In this project, I built an end-to-end migration solution that moves data from a self-managed MariaDB database running on Amazon EC2 to Amazon RDS for PostgreSQL using AWS Database Migration Service (AWS DMS). Rather than relying on manual console configuration, the infrastructure was provisioned with Terraform, making the deployment repeatable and easier to manage.
The source database runs on an Amazon EC2 instance inside a VPC, while Amazon RDS for PostgreSQL serves as the managed destination. AWS DMS sits between the two databases, performing an initial full load before switching to Change Data Capture (CDC) so that new inserts and updates continue to replicate with minimal downtime.
This architecture mirrors a common migration scenario where organizations want to modernize their database platform without interrupting application availability.
Deploying the infrastructure with Terraform allowed the networking components, security groups, IAM roles, database instance, DMS replication instance, and supporting resources to be created consistently from code.
After provisioning completed successfully, the EC2 instance was configured with MariaDB and prepared as the migration source. Binary logging was enabled using the ROW format, which is a prerequisite for AWS DMS to capture ongoing database changes.
A sample retail database containing customer records was then created to simulate production data.
The target environment consisted of an Amazon RDS for PostgreSQL instance deployed in Multi-AZ mode. AWS DMS endpoints were then configured for both the MariaDB source and PostgreSQL destination.
One interesting challenge during this stage involved authentication to the PostgreSQL target. Although the database credentials themselves were correct, AWS DMS expected a Secrets Manager secret containing additional connection metadata such as the database name, engine, host, and port. After creating a custom secret with the complete connection details, endpoint validation completed successfully.
With both endpoints validated, the replication task was configured to perform a Full Load followed by Change Data Capture.
The initial migration copied all existing customer records from MariaDB into PostgreSQL. To verify that CDC was functioning correctly, an additional customer record was inserted into MariaDB after the migration had completed.
Without restarting the task or manually copying data, the new record appeared automatically in PostgreSQL, confirming that ongoing replication was working as expected.
Although the overall architecture appears straightforward, building the solution highlighted several practical lessons that are difficult to appreciate from documentation alone. Configuring binary logging correctly on the source database was essential before CDC could function. AWS DMS also relies heavily on IAM permissions and correctly structured Secrets Manager credentials, making those components just as important as the databases themselves. Finally, network connectivity and security group rules played a critical role in allowing each service to communicate securely.
Working through these issues reinforced an important lesson: successful cloud migrations depend not only on choosing the right AWS services, but also on understanding how those services interact.
This project demonstrates how Infrastructure as Code and AWS Database Migration Service can be combined to automate heterogeneous database migrations while minimizing downtime. Using Terraform ensured that the environment could be recreated consistently, while AWS DMS handled both the initial migration and continuous replication with minimal manual intervention.
The complete Terraform code, architecture diagram, SQL scripts, and supporting files are available in the GitHub repository below.
GitHub Repository: https://github.com/Naomiansah/terraform-aws-dms-mariadb-postgresql-migration
Top comments (0)