DEV Community

StackOverflowWarrior
StackOverflowWarrior

Posted on • Updated on

Day 18 of 100 Days of Cloud: Migrating Data from a PostgreSQL Instance to a Docker Container

Welcome to Day 18 of our 100 Days of Cloud series! Today’s focus is on migrating PostgreSQL data from a remote server to a Docker container running PostgreSQL. This guide will walk you through system updates, PostgreSQL installation, data backup, Docker setup, and finally, restoring the database inside a Docker container.


Table of Contents

  1. Update and Upgrade Your System
  2. Install PostgreSQL
  3. Backup PostgreSQL Database
  4. Prepare for Docker Installation
  5. Install Docker
  6. Clone Git Repository and Set Up Docker Containers
  7. Move Environment Configuration
  8. Copy Backup File to Docker Container
  9. Restore Database in Docker Container
  10. Access Docker Container

1. Update and Upgrade Your System

Begin by updating your system to ensure that all existing packages are current. This helps avoid conflicts and ensures compatibility.

sudo apt update
sudo apt upgrade
Enter fullscreen mode Exit fullscreen mode
  • sudo apt update: Updates the list of available packages and their versions.
  • sudo apt upgrade: Upgrades all installed packages to their latest versions.

2. Install PostgreSQL

Install PostgreSQL 16 to manage your databases.

sudo apt-get install -y postgresql-16
Enter fullscreen mode Exit fullscreen mode
  • sudo apt-get install -y postgresql-16: Installs PostgreSQL 16, automatically confirming any prompts.

To verify available PostgreSQL packages, use:

apt search postgres
Enter fullscreen mode Exit fullscreen mode
  • apt search postgres: Lists PostgreSQL packages and versions available in the repository.

3. Backup PostgreSQL Database

Create a backup of your PostgreSQL database from the remote server. Adjust the command according to your database details.

pg_dump -h verisafe.xxxxxxxxxxxxpx.us-east-1.rds.amazonaws.com -U verisafe -F c -b -v -f /home/ec2-user/verisafe/backup_file.dump postgres
Enter fullscreen mode Exit fullscreen mode
  • -h: Hostname of the PostgreSQL server.(The RDS endpoint in my case)
  • -U: Username for database access.
  • -F c: Specifies the custom format for the backup.
  • -b: Includes large objects (blobs).
  • -v: Enables verbose output.
  • -f: Path where the backup file will be saved.

4. Prepare for Docker Installation

Before installing Docker, ensure you have the necessary tools and dependencies.

sudo apt update
sudo apt install apt-transport-https ca-certificates curl software-properties-common
Enter fullscreen mode Exit fullscreen mode
  • apt-transport-https: Allows apt to handle HTTPS.
  • ca-certificates: Manages SSL certificates.
  • curl: Command-line tool for transferring data from or to a server.
  • software-properties-common: Provides the add-apt-repository command for managing PPAs.

5. Install Docker

Add Docker’s official GPG key and repository, then install Docker Community Edition (CE).

First, add Docker’s GPG key:

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
Enter fullscreen mode Exit fullscreen mode
  • curl -fsSL ... | sudo apt-key add -: Adds Docker’s GPG key to verify package integrity.

Add Docker’s repository:

sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable"
Enter fullscreen mode Exit fullscreen mode
  • sudo add-apt-repository ...: Adds Docker’s repository to your package sources.

Update the package list and install Docker CE:

sudo apt update
sudo apt install docker-ce
Enter fullscreen mode Exit fullscreen mode
  • sudo apt install docker-ce: Installs Docker Community Edition.

Verify the Docker installation:

apt-cache policy docker-ce
Enter fullscreen mode Exit fullscreen mode
  • apt-cache policy docker-ce: Displays the Docker CE version and installation status.

6. Clone Git Repository and Set Up Docker Containers

Clone the repository that contains your Docker setup and bring up the Docker containers.

Check your Git installation:

git --version
Enter fullscreen mode Exit fullscreen mode
  • git --version: Displays the installed Git version.

Clone the repository:

git clone https://github.com/dita-daystaruni/verisafe.git
Enter fullscreen mode Exit fullscreen mode
  • git clone ...: Downloads the repository to your local machine.

Navigate to the repository:

cd verisafe
Enter fullscreen mode Exit fullscreen mode
  • cd verisafe: Changes directory to the cloned repository.

Start the Docker containers:

sudo docker compose up
Enter fullscreen mode Exit fullscreen mode
  • sudo docker compose up: Launches the Docker containers defined in docker-compose.yml.

7. Move Environment Configuration

If your project includes an example.env file, rename it to .env to configure your environment.

Edit the example.env file if necessary:

nano example.env
Enter fullscreen mode Exit fullscreen mode
  • nano example.env: Opens the file in the nano text editor.

Rename the file:

mv example.env .env
Enter fullscreen mode Exit fullscreen mode
  • mv example.env .env: Renames example.env to .env to be used by Docker Compose.

Restart Docker containers to apply the configuration changes:

sudo docker compose up
Enter fullscreen mode Exit fullscreen mode

8. Copy Backup File to Docker Container

Transfer the backup file into the Docker container where PostgreSQL is running.

docker cp /home/ubuntu/verisafe/backup_file.dump verisafe-postgres-1:/backup_file.dump
Enter fullscreen mode Exit fullscreen mode
  • docker cp ...: Copies the backup file to the Docker container.

If you encounter permissions issues, use sudo:

sudo docker cp /home/ubuntu/verisafe/backup_file.dump verisafe-postgres-1:/backup_file.dump
Enter fullscreen mode Exit fullscreen mode

9. Restore Database in Docker Container

Access the Docker container and restore the database using pg_restore.

First, access the container:

docker exec -it verisafe-postgres-1 bash
Enter fullscreen mode Exit fullscreen mode
  • docker exec -it ... bash: Opens an interactive bash shell inside the container.

Restore the database:

pg_restore -U verisafe -d verisafe /backup_file.dump
Enter fullscreen mode Exit fullscreen mode
  • -U postgres: PostgreSQL user to perform the restoration.
  • -d your_database_name: The target database where the data will be restored.
  • /backup_file.dump: Path to the backup file inside the container.

10. Access Docker Container

To perform operations within the Docker container, you might need to access its shell.

docker exec -it verisafe-postgres-1 bash
Enter fullscreen mode Exit fullscreen mode
  • docker exec -it ... bash: Opens a bash shell in the specified container.

If necessary, use sudo:

sudo docker exec -it verisafe-postgres-1 bash
Enter fullscreen mode Exit fullscreen mode

Conclusion

In today’s tutorial, we covered the entire process of migrating PostgreSQL data from a remote server to a Docker container. This included updating your system, installing PostgreSQL, creating and restoring backups, setting up Docker, and managing Docker containers.

Feel free to revisit any steps as needed and ensure everything is correctly configured. Stay tuned for more cloud management tips and techniques in our ongoing series. Happy migrating!

Top comments (0)