DEV Community: Louretta

Automating Aws Cost Optimization Using Iac(Terraform)

Louretta — Sat, 11 Jan 2025 01:17:35 +0000

This project focuses on cost optimization while also improving resource management, monitoring, and cost analysis. It’s a comprehensive approach to managing cloud resources efficiently and cost-effectively.
Incorporating these strategies can make a significant difference for small businesses, helping them thrive in a competitive landscape while maintaining financial health and operational efficiency.
Resources used for this project includes: AWS Resource Groups,
Amazon CloudWatch Dashboards, Amazon CloudWatch Alarms, AWS Budgets with email/SNS alerts, Cost and Usage Report (CUR),Aws Lambda ,S3 Bucket for CUR storage and Amazon QuickSight for visualization
*Architectural View *

***PEREQUISITES *
Basic Knowledge of AWS Services
AWS Management Console
Scripting and Automation
Understanding of Cost Management Tools

Terraform setup
Terraform was used to ensure consistency and repeatability in resource provisioning and also automating infrastructure deployment reduces errors , manual effort and Tracks infrastructure changes using version control.

Add resource groups for cost monitoring, budgets and cost and usage report: This Provides detailed billing and usage data for all AWS services

Add S3 bucket resource group: This stores Cost and Usage Reports (CUR) data in a specified folder structure. , s3 is cost effective and acts as a data source for QuickSight

Add resource groups for cloud watch dashboard alarms and Lambda archive_file: This Monitors cost-related metrics like Estimated Charges and triggers alerts for anomalies to SNS when threshold set is exceeded
Terraform archive_file was used to automate the ZIP process eliminating the need for external dependencies.

Add a python script for your Lambda function. This Automate Resource Scheduling using Lambda-based start/stop automation

Configure your Lambda function to trigger start/stop resources based on a schedules via Event Bridge (make reference to the script above)
and for best security practices, add Iam role for Lambda. This Securely grants permissions to CUR, QuickSight, Lambda, and other services.

Add Lambda Iam policy-This ensure resources only have the permissions they need and provides fine-grained control for secure infrastructure operations

run terraform init, plan and apply to deploy

Some Tips :you can use commands to output the result on console should look like this.

some of the outs via console

Always remember to clean after yourself -Run terraform destroy to clean resources
PS: S3 bucket has to be emptied before it can be deleted, best to do the console

Feel free to pull codes from my GitHub. Happy Coding
https://github.com/Louretta/Aws-cost-optimization-Automation.git

Configuring a Secure Nginx Server Using Let's Encrypt with Certbot

Louretta — Mon, 22 Jul 2024 16:02:48 +0000

Let's Encrypt is a free service that provides SSL certificates, which are used to secure websites by enabling HTTPS. This means that the data sent between your web browser and the website is encrypted, making it private and secure. The process of getting and renewing these certificates is automated and straightforward, often done using a tool called Certbot

*Perequisite *
Resgistered domain
Ubuntu server

1. Update Package List

SSH ito your ubuntu instance

Update and install package list to ensure you have the latest information about avalaible packages

Install Ngnix using the package manager

Run this commands below to start the Ngnix service and check the status

Expected output

sample of Ngnix webpage without domain name

2.Configure Firewall

Allow http and https traffic through the firewall

3.Create a hosted zone in route 53 (if not created already)
-open the amazon route 53 console(see image below)

-Enter your domain name

once hosted zone is created,get name servers information(note the four nameservers provivded by route 53. see image below)

Update name server in your domain registrar's website (for this project I used GoDaddy). Find the DNS settings or nameserver settings for your domain and replace the existing server with the foure nameservers provided by route 53

-Add DNS records in Route 53
on Route 53 go to "create record button"see image below(make sure to replace with your domain and Ip address)

4.Configure Ngnix for Your Domain
create a new configuration file for your domain in Ngnix

Add the following content below, replace with your domain name
Create a symbolic link to enable your configuration
use command below to test configuration to ensure there are no syntax errors
Create a root document. This ensures the webser knows exactly where to find the files requested by users, providing a consistent and reliable way to serve web content- See command below
Change ownership of the directory and its contents. This ensures that the user has full control over the files and directories and that the webserver can serve content correctly
Run this command to create an index.html file to test your configuration
Run this command to enable your configuration(ensure to replace with your own domain name )
Run this command below to to test configuration and relod Ngnix

-open browser to verify setup using your domain (http://your_domain). you should be able to see if your website is configured correctly.
The image below displayed a "Not secure" warning on the website when we accessed over HTTP, meaning that any data exchanged between the users browser and the server can be intercepted and read by third parties.

5.Secure Your Website with HTTPS
we will be using lets encrypt for this tutorial

-Before using Snap to install Certbot, check if Certbot is already installed with apt and remove it if it is. This ensures that the Snap version of Certbot will be used instead of the one from your operating system's package manager

Run this command to instal Certbot

Run this command sudo ln -s /snap/bin/certbot /usr/bin/certbot to make sure you can execute the Certbot command. This links the path to the Certbot binary file.
Run the command sudo certbot --nginx to obtain a certificate and automatically configure Nginx with Certbot. During the process, you will be prompted to:

Enter your email address.
Read and accept the terms and conditions of Let’s Encrypt by typing y to proceed.
Type y to allow notifications.
Choose your domain name or enter it manually.

The output should look like the image below.

The Certbot packages on your system include a scheduled task that automatically renews your certificates before they expire. You don't need to run Certbot again unless you change your settings. To test the automatic renewal, run this command
To check if your site is set up correctly, go to https://yourwebsite.com/ in your browser and look for the lock icon in the address bar.

*Conclusion *
By securing your website with HTTPS, you not only protect your user's data but also improve your sites credibility, SEO, and overall user experience.

Run A NodeJs Application In a Docker Container

Louretta — Tue, 09 Jul 2024 18:27:33 +0000

we will cover the following:
1.Installation of docker on ubuntu(AWS)
2.Containerisation of Nodejs application
3.update the application
4.share the application

Perequisites

AWS Account with EC2 access
Basic knowledge of Docker concepts and commands
Steps to Setup Docker Project on Ubuntu (AWS EC2 Instance)
Create an Ubuntu EC2 Instance
Sign in to AWS Management Console
Go to the AWS Management Console and log in with your credentials.
Launch an EC2 Instance
Navigate to EC2 Dashboard:
Click on Services and select EC2 under Compute.
Click Launch Instance.
Choose an Ubuntu Server AMI (e.g., Ubuntu Server 20.04 LTS).
Select an Instance Type (e.g., t2.micro).
Configure Instance Details and add storage as needed.(see image below)
Configure Security Group:
Add rules to open ports 22 (SSH), 80 (HTTP), and 3000 (Application port).
Review and Launch the instance, then SSH into the terminal.

2.Install Docker and Docker Compose
Run this command to install the necessary tools and packages to handle secure software installations and manage additional software repositories effectively.

Add the Docker GPG key to your system's list of trusted keys using command below

Add the Docker repository on your Ubuntu system which contains Docker packages including its dependencies, for that execute the below command:

Update the package index on your Ubuntu system and install Docker Community Edition suing commands below(Ps: To Check the Latest Version: Visit the Docker Compose Releases page and find the latest version)

Run the following command to download the latest version of docker compose (ensure you have the latest: To Check the Latest Version: Visit https://github.com/docker/compose/releases)

Set the correct permissions so that doxker-compose is executable

Run this command to verify sucessfule installation

3.Containerization of Node.js Application
Start docker engine and check status

Clone docker compose application:Cloning a Docker Compose application this is important because it ensures consistency, ease of setup, and portability across different environments.Run the command below to clone the application

Run ls to see cloned files

To see content in directory run comand below

Build the app container image:To build the container image, you need to create a Dockerfile. A Dockerfile is a plain text file that contains a set of instructions which Docker uses to create a container image.
In the app directory of the cloned repository, where the package.json file is located, create a file named Dockerfile
Open the Dockerfile and paste the code provided below into it.
Save and close the Dockerfile.
Run the following command to create a Dockerfile

Then, paste the following code into the Dockerfile

Change directory to the app directory

Build the container image using the following command

Start application container by running the command below.This command ensures that the Docker container runs correctly, and the application is accessible on port 3000 of your host machine

After a few seconds, open your web browser and paste the following url http://18.169.10.56:3000/ replacing your-docker-server-ip with your Docker server's public IP to view the application
you can add one or two items as seen in the image below to ensure the application is working as expected

Update The Application To modify the app.js file follow these steps Run the command to open the file in vi:sudo vi ~/getting-started/app/src//static/js/app.js-expected output, see image below -search for text "NO items yet! Add one above!" press '/' to initiate a search and then type 'No' and press 'Enter' on your keyboard, the curson will move the first occureence of the text starting with "No"

Press 'a' to enter insert mode, allowing you to edit the text starting from the next character
Change 'No items yet! Add one above! to you have no todo items yet! Add one above!
Press esc to exit insert mode, type :wq! and press enter to save chnages and close file

Use the same docker build command you used before to create an updated version of the image.

After building start a new container

The previous step failed because an existing container is already using the host’s port 3000. To resolve this issue, we need to stop and remove the old container before running the updated image:To do this run command below
Run sudo docker ps command to see the container ID of the running container
Find the container ID from the output of the previous command and stop the container by running

Remove the stopped container to free up the port

Now that the old container is stopped and removed, run the updated image

After running the updated Docker container, you might notice that the data added previously has disappeared. This is because the data stored in the container does not persist when the container is removed.

5. Share the Application
Go to Docker Hub and sign in with your Docker ID or create a new Docker ID if you don't have one
After signing in, click on your profile icon at the top right corner of the page and select Account Settings from the dropdown menu.
In the Account Settings page, click on Repositories in the left sidebar.
Click on the green Create Repository button.

Your repository is now created and ready to use. You will see the repository details and instructions for pushing your Docker images to this repository.

Run the command below to login to Docker Hub, make you use you replace the username with your username . When prompted for password, type your Docker Hub password

Push image to dockerhub using the command below and expected output

After pushing the image to docker hub pull image and run it as a container:Pulling Docker images and running them as containers ensures that your applications are portable, scalable, and secure. It simplifies deployment processes and enhances consistency across different environments, making Docker a powerful tool for modern software development and deployment practices

SUMMARY
This project involves setting up a Docker environment on an Ubuntu EC2 instance in AWS, containerizing a Node.js application, updating it, and sharing it on Docker Hub. The Node.js application runs in a Docker container, accessible via port 3000, and can be accessed and modified through commands and scripts.
Regarding data persistence, while the Docker container provides a portable and scalable environment for applications, data persistence strategies such as volume mounts or database containerization are recommended for ensuring that data remains intact across container restarts or updates. This aspect will be covered in subsequent posts or steps, focusing on maintaining data integrity and application reliability in Dockerized environments