Automating a Multi-Tier Web App on AWS with Terraform and Ansible: Triumphs and Challenges

As part of my journey to master cloud computing and DevOps, I recently tackled an advanced project: deploying a fully automated multi-tier web application on AWS using Terraform for infrastructure provisioning and Ansible for server configuration. This project built on my previous experience with AWS, pushing me to embrace infrastructure as code (IaC) and automation. Despite numerous challenges, including a critical misconfiguration in my Terraform variables, I successfully deployed a secure, monitored web application. In this blog post, I’ll share what I accomplished, the hurdles I faced, how I overcame them, and the lessons that will shape my future projects.

Project Objectives

The goal was to create a scalable, secure multi-tier web application on AWS with the following components:

• Terraform: Provision a Virtual Private Cloud (VPC), public and private subnets, EC2 instances, security groups, and IAM roles.
• Ansible: Configure a web server with Nginx and PHP, and a database server with MariaDB, deploying a PHP application that connects to the database.
• Enhancements:
  • Set up AWS CloudWatch to monitor CPU usage.
  • Store Terraform state remotely in an S3 bucket for consistency.
  • Implement a unit test to verify the web application’s functionality.

The final deliverable was a working web application accessible via the web server’s public IP, with the database securely isolated in a private subnet, all provisioned and configured automatically.

Step-by-Step Accomplishments

Here’s what I achieved, broken down into key phases, along with the challenges I encountered and how I resolved them.

1. Setting Up the Project Environment

I started by organizing my project directory (project4/) with subfolders for Terraform (terraform/), Ansible (ansible/), and scripts (scripts/). I installed Terraform, Ansible, and the AWS CLI on my local machine, ensuring my AWS credentials were configured correctly.

Accomplishment: Established a clean, structured project environment to support IaC and automation workflows.

2. Provisioning Infrastructure with Terraform

Using Terraform, I provisioned a robust AWS infrastructure:

• VPC: Created multi-tier-vpc with a CIDR block of 10.0.0.0/16.
• Subnets: Set up a public subnet (10.0.1.0/24) for the web server and a private subnet (10.0.2.0/24) for the database server.
• Networking: Configured an Internet Gateway for public subnet access, a NAT Gateway for private subnet outbound traffic, and route tables to manage traffic flow.
• Security Groups:
  • web-sg: Allowed HTTP (port 80) from anywhere and SSH (port 22) from my IP.
  • db-sg: Permitted MariaDB (port 3306) and SSH (port 22) from the web server’s security group.
• IAM Role: Granted EC2 instances S3 read-only access via an instance profile.
• EC2 Instances: Launched a web server in the public subnet and a database server (IP: 10.0.2.219) in the private subnet, both using Amazon Linux 2023.
• Outputs: Exposed the web server’s public IP and database server’s private IP for Ansible.

Challenge: When creating the variables.tf file, I mistakenly defined the web_sg_ingress variable with port 80 twice instead of including port 22 for SSH:

web_sg_ingress = [
  { from_port = 80, to_port = 80, protocol = "tcp", cidr_blocks = ["0.0.0.0/0"] },
  { from_port = 80, to_port = 80, protocol = "tcp", cidr_blocks = ["MY_IP/32"] }  # Error: Should be port 22
]

This caused SSH connection failures to the web server, as port 22 was blocked. I spent hours troubleshooting, expecting a key pair or network issue, only to realize the security group misconfiguration.

Solution: I updated variables.tf to:

web_sg_ingress = [
  { from_port = 80, to_port = 80, protocol = "tcp", cidr_blocks = ["0.0.0.0/0"] },
  { from_port = 22, to_port = 22, protocol = "tcp", cidr_blocks = ["MY_IP/32"] }
]

Reapplied Terraform (terraform apply), and SSH access was restored.

Accomplishment: Successfully provisioned a secure, multi-tier AWS infrastructure using Terraform, learning the importance of double-checking security group configurations.

3. Configuring Servers with Ansible

I used Ansible to automate server setup:

• Web Server:

  • Installed Nginx, PHP, and PHP-FPM.
  • Deployed index.php, a PHP script that connects to the database and displays "Connected successfully to the database!".
  • Configured Nginx to serve PHP files.

• Database Server:

  • Installed MariaDB, created a database (my_database), and set up a user (web_user) with access from the web server.
  • Secured MariaDB with a root password.

Challenge: The initial db_playbook.yml failed because mariadb-server wasn’t available on Amazon Linux 2023:

No package mariadb-server available.

AL2023 doesn’t use amazon-linux-extras (unlike Amazon Linux 2), and the correct package was mariadb105-server. Additionally, I used yum instead of dnf, AL2023’s default package manager.

Solution: Updated the playbook to use dnf and install mariadb105-server:

- name: Install MariaDB server
  dnf:
    name: mariadb105-server
    state: present

Challenge: An earlier playbook run failed on the mysql_user task to set the MariaDB root password, likely due to AL2023’s default unix_socket authentication for the root user.

Solution: Modified the playbook to handle initial root access without a password and ignore errors for fresh installs:

- name: Set MariaDB root password for the first time
  mysql_user:
    name: root
    password: "{{ mysql_root_password }}"
    host: localhost
    state: present
    login_user: root
    login_password: ''
  no_log: true
  ignore_errors: yes

Accomplishment: Automated server configuration with Ansible, overcoming package and authentication issues specific to AL2023.

4. Adding CloudWatch Monitoring

I configured AWS CloudWatch to monitor CPU utilization for both EC2 instances:

• Created alarms (web-cpu-alarm and db-cpu-alarm) to trigger if CPU usage exceeds 80% for 4 minutes.
• Integrated this into main.tf with Terraform.

Accomplishment: Implemented basic monitoring to ensure infrastructure health, enhancing the project’s production readiness.

5. Configuring S3 Backend for Terraform State

To ensure state consistency, I stored Terraform’s state file in an S3 bucket (terraform-state-bucket):

• Created the bucket manually in the AWS Console.
• Updated main.tf with:

terraform {
  backend "s3" {
    bucket = "terraform-state-bucket"
    key    = "project4/terraform.tfstate"
    region = "us-east-1"
  }
}

• Reinitialized Terraform (terraform init).

Accomplishment: Enabled remote state management, a best practice for collaborative IaC projects.

6. Implementing a Unit Test

I wrote a Bash script (test.sh) to verify the web application:

• Fetched the web server’s public IP from Terraform output.
• Used curl to check if http://<web_public_ip>/index.php returned "Connected successfully to the database!".
• Output "Test passed" or "Test failed" based on the response.

Accomplishment: Added automated testing to validate application functionality, ensuring reliability.

7. Cleaning Up Resources

Learning from my previous project’s $27 bill due to lingering resources, I destroyed all AWS resources after completion:

terraform destroy

This removed the VPC, EC2 instances, NAT Gateway, and other components, avoiding unexpected charges.

Accomplishment: Practiced responsible cloud management, ensuring no cost overruns.

Lessons Learned

This project was a significant step forward in my DevOps journey, but it wasn’t without challenges. Key takeaways include:

• Attention to Detail in IaC: The variables.tf error (port 80 instead of 22) taught me to meticulously review configurations, especially for security groups, as small mistakes can cause significant issues.
• OS-Specific Considerations: AL2023’s differences (e.g., dnf vs. yum, mariadb105-server vs. mariadb-server) highlighted the importance of understanding the target operating system.
• Automation Saves Time: Ansible and Terraform reduced manual setup, but debugging automation scripts requires patience and log analysis.
• Cost Awareness: Destroying resources immediately after use is critical to avoid charges, a lesson reinforced by my previous project’s mistake.
• Persistence Pays Off: Each challenge, from SSH failures to package errors, was a chance to deepen my understanding of AWS and automation tools.

Next Steps

With this project complete, I’m excited to explore more advanced DevOps concepts, such as:

• Implementing auto-scaling groups with Terraform.
• Using Ansible roles for more modular playbooks.
• Adding CI/CD pipelines to automate deployments.

This project has equipped me with the skills to tackle these challenges with confidence. Stay tuned for my next adventure in cloud computing!

Conclusion

Deploying a fully automated multi-tier web app on AWS with Terraform and Ansible was both challenging and rewarding. From misconfiguring security groups to navigating AL2023’s package quirks, I faced obstacles that tested my problem-solving skills. Yet, the result—a secure, monitored, and tested web application—was worth the effort. This project not only solidified my understanding of IaC and automation but also taught me the importance of precision and cost management in the cloud. If you’re diving into a similar project, embrace the challenges—they’re your best teachers. Happy automating!