DEV Community: Adesokan Israel

Automating AWS Resource Management with Shell Scripting

Adesokan Israel — Fri, 16 Aug 2024 22:33:06 +0000

In the dynamic world of cloud computing, managing resources across an AWS environment can be a daunting task. With numerous services spread across different regions, keeping track of everything manually is not only time-consuming but also prone to error. To address this challenge, I’ve developed a simple yet powerful shell script that automates the process of listing all resources in an AWS account.

This script provides a quick and efficient way to gather detailed information about your AWS infrastructure, covering a wide range of services like EC2, RDS, S3, and more. Whether you’re a cloud architect responsible for multiple environments or a developer managing a single account, this tool can significantly simplify your workflow, ensuring that you always have an up-to-date inventory of your cloud resources.

In this article, I’ll walk you through the development and functionality of this script, showcasing how automation can enhance your cloud management strategy.
The focus of this project is to provide a solution for listing resources of a user
on AWS in an automated way, and basically could be used to perform cron job as the case may suit the user and the need in the line of automation.

Supported Services

The script currently supports listing resources for the following AWS services:

EC2
RDS
S3
CloudFront
VPC
IAM
Route53
CloudWatch
CloudFormation
Lambda
SNS
SQS
DynamoDB
EBS

Features

Multi-Region Support: The script can query resources across multiple AWS regions.
Service Coverage: Lists resources from major AWS services such as EC2, S3, RDS, Lambda, and more.
Output Formats: Results can be outputted in plain text or saved to a file for easy integration with other tools.
Error Handling: Includes basic error handling to manage API rate limits and service-specific issues.

Prerequisites

Before running the script, ensure you have the following:

AWS CLI: Installed and configured with the necessary permissions to query AWS resources.

Bash Shell: The script is written for Unix-like environments (Linux, macOS, WSL on Windows).

IAM Permissions: The IAM user or role running the script should have read-only permissions to all AWS services being queried.

Setup

STEPS

clone the Repository:

https://github.com/Aymogul/Aws-resource-script.git

Install AWS CLI on the instance

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

Configure AWS CLI: Ensure your AWS CLI is configured with the necessary credentials and default region

aws configure

Input your:
Access key
Secret Key
Regon
Output as JSON

Make script Executable: Give execution permission to the script

chmod +x aws_resource_list.sh

chmod 771 aws_resource_list.sh

Usage

Run the script This can be executed directly from the command line after you have logged into an EC2 via ssh or puTTY

./aws_resource_list.sh <aws_region> <aws_service>

example usage

./aws_resource_list.sh us-east-1 ec2

Output: This script will run and the output of the AWS resource list will be outputed to aws_resources.txt.

./_aws_resource_list.sh > aws_resources.txt

Customization of Regions and other Variables: By default, the script will query resources in all regions. You can edit the regions variable in the script to specify particular regions:

regions="us-east-1"

This command will list all EC2 instances in the us-east-1 region.

Valid AWS Services:

The script supports various services like ec2, rds, s3, cloudfront, vpc, iam, route53, cloudwatch, cloudformation, lambda, sns, sqs, dynamodb, and ebs.

Error Handling

AWS CLI Not Installed: The script checks if AWS CLI is installed and configured. If not, it will prompt the user to install and configure it.
Invalid Service: If an unsupported service is specified, the script will return an error message.

Customization

Additional Services: You can modify the script to include additional AWS services by adding new cases to the case block.
Output Format: The script can be enhanced to format output in different ways (e.g., JSON, CSV) depending on your requirements.

Contribution

Contributions are welcome! If you have improvements or additional features, feel free to fork the repository and submit a pull request.

Super-Mario deployment on K8 with Terraform

Adesokan Israel — Wed, 14 Aug 2024 16:11:26 +0000

*Introduction*
In the ever-evolving world of DevOps and cloud infrastructure, automating deployments has become crucial for managing complex systems efficiently. Kubernetes, with its robust orchestration capabilities, has emerged as the go-to solution for containerized applications, while Terraform allows for infrastructure as code, enabling repeatable and consistent deployments.

Imagine combining these powerful tools to deploy something as iconic as Super Mario! In this guide, we’ll walk you through how to deploy the classic Super Mario game on a Kubernetes cluster using Terraform. This tutorial not only showcases the power of Kubernetes and Terraform but also adds a touch of nostalgia by bringing a beloved game into the world of cloud-native applications. Whether you’re a DevOps enthusiast, a cloud engineer, or a gaming aficionado, this deployment will take you on a fun and educational journey into the realms of modern infrastructure management. Let's dive in and bring Mario to life in the cloud!

Prerequisites:

AWS account
An Ubuntu Instance
IAM role
Terraform should be installed on instance
AWS CLI and KUBECTL on Instance

General project overview

login and basic setup on AWS
Setup Docker,Terraform,AWS CLI and kubectl
Provision IAM role for the EC2
Attach IAM role with the EC2
Build the infrastucture using Terraform
Create service and deployment for EKS
Destroy the infrastructure

STEP 1: Launch Ubuntu instance

a.Sign in to AWS Console: Log in to your AWS Management Console.

b.Navigate to EC2 Dashboard: Go to the EC2 Dashboard by selecting “Services” in the top menu and then choosing “EC2” under the Compute section.

c. Launch Instance: Click on the “Launch Instance” button to start the instance creation process.

d. Choose an Amazon Machine Image (AMI): Select an appropriate AMI for your instance. For example, you can choose Ubuntu image.

e. Choose an Instance Type: In the “Choose Instance Type” step, select t2.micro as your instance type. Proceed by clicking “Next: Configure Instance Details.”

f. Configure Instance Details:

For “Number of Instances,” set it to 1 (unless you need multiple instances).
Configure additional settings like network, subnets, IAM role, etc., if necessary.
For “Storage,” click “Add New Volume” and set the size to 8GB (or modify the existing storage to 8GB).
Click “Next: Add Tags” when you’re done.

g. Add Tags (Optional): Add any desired tags to your instance. This step is optional, but it helps in organizing instances.

h. Configure Security Group:
Choose an existing security group or create a new one.
Ensure the security group has the necessary inbound/outbound rules to allow access as required.

i. Review and Launch: Review the configuration details. Ensure everything is set as desired.

j.Select Key Pair:

Select “Choose an existing key pair” and choose the key pair from the dropdown.
Acknowledge that you have access to the selected private key file.
Click “Launch Instances” to create the instance.

k. Access the EC2 Instance: Once the instance is launched, you can access it using the key pair and the instance’s public IP or DNS.
Ensure you have necessary permissions and follow best practices while configuring security groups and key pairs to maintain security for your EC2 instance.

STEP 2: Create IAM role

Search for IAM in the search bar of AWS and click on roles.

Create Role

Select entity type as AWS service

Use case as EC2 and click on Next.

Select Administrator Access for permision policy

Name the Role

Role creation successful

Then attach this role to Ec2 instance that we created earlier, so we can provision cluster from that instance.

Go to EC2 Dashboard and select the instance.

Click on Actions –> Security –> Modify IAM role.

Select the Role that created earlier and click on Update IAM role.

From this point, you can connect to the server via any means you prefer either: AWS instance connect or putty

STEP 3: Cluster provision
Now clone this Repo.

git clone https://github.com/Aymogul/k8-super-mario-deployment-with-terraform.git

change directory

cd k8-super-mario-deployment-with-terraform

Update the server and Provide the executable permission to script.sh file, and run it.

sudo su
sudo apt update -y

sudo chmod +x script.sh
./script.sh

The execution of this script will install AWS CLI, Terraform, Kubectl and Docker.

check the versions of installed tools

docker --version
aws --version
kubectl version --client
terraform --version

Now change directory into the EKS-TF

Run Terraform init

NOTE: Don’t forgot to change the s3 bucket name in the backend.tf file

cd EKS-TF
terraform init

The next step is to run validate and plan the terraform deployment

terraform validate
terraform plan

Provision the cluster by running this command

terraform apply --auto-approve

Update the Kubernetes configuration

Make sure change your desired region

aws eks update-kubeconfig --name EKS_CLOUD --region us-west-1

Now change directory back to k8-super-mario-deployment-with-terraform

cd ..

let's now move to the most exciting part of the project as we deploy the game on EKS cluster and enjoy the thrill of super-mario.

Deployment

kubectl apply -f deployment.yaml
#to check the deployment 
kubectl get all

Now let’s apply the service

Service

kubectl apply -f service.yaml
kubectl get all

Now let’s describe the service and copy the LoadBalancer Ingress

kubectl describe service mario-service

Paste the ingress link in a browser and you will see the Mario game.

EUREKA

Enjoy the classic thrill of super-mario...

Destruction :

Let’s remove the service and deployment first

kubectl get all
kubectl delete service mario-service
kubectl delete deployment mario-deployment

Destruction of the infrastructure

terraform destroy --auto-approve

Deploying Super Mario on Kubernetes using Terraform demonstrates the power of modern infrastructure tools like Kubernetes for container orchestration and Terraform for infrastructure as code. This project combines nostalgia with practical skills, showcasing how these technologies enable consistent and scalable deployments. By following the steps in this guide, you've not only deployed a classic game but also gained valuable insights into managing complex systems in the cloud. The knowledge gained here can be applied to various projects, setting you up for success in the evolving world of DevOps and cloud infrastructure.
Thank you for following

Linux users creation with Bash script

Adesokan Israel — Thu, 04 Jul 2024 16:38:58 +0000

Introduction

In a growing organization, managing user accounts efficiently is crucial. Automating the process can save significant time and reduce errors. This article explains a Bash script designed to read a list of users and groups from a text file, create the users and groups, set up home directories, generate passwords, and log all actions. This script is particularly useful for SysOps engineers responsible for maintaining system user accounts.

The purpose of this blog is to provide solution for the creation of Linux users using bash script in an automated and reliable way. Below is the scenario for the problem statement.
Your company has employed many new developers. As a SysOps engineer, write a bash script called create_users.sh that reads a text file containing the employee’s usernames and group names, where each line is formatted as user;groups.
The script creates users and groups as specified, set up home directories with appropriate permissions and ownership, generate random passwords for the users, and log all actions to /var/log/user_management.log. Additionally, store the generated passwords securely in /var/secure/user_passwords.txt.

Script Overview

The create_users.sh script reads a text file where each line is formatted as user;groups, creates the users and groups, sets up home directories, and generates random passwords. Actions are logged to /var/log/user_management.log, and passwords are securely stored in /var/secure/user_passwords.csv.

Detailed Breakdown

Log and Secure Password File Setup:
The script begins by setting up the log file and secure password file. It ensures that the directories exist and sets appropriate permissions for the password file.

LOG_FILE="/var/log/user_management.log"
PASSWORD_FILE="/var/secure/user_passwords.csv"

mkdir -p /var/log
touch $LOG_FILE

mkdir -p /var/secure
touch $PASSWORD_FILE
chmod 600 $PASSWORD_FILE

Logging Function:
A function is defined to log messages with timestamps.

log_message() {
    echo "$(date +'%Y-%m-%d %H:%M:%S') - $1" >> $LOG_FILE
}

Password Generation

A helper function generates random passwords.

generate_password() {
    tr -dc A-Za-z0-9 </dev/urandom | head -c 12
}

Input File Check

The script checks if an input file is provided and exits if not.

if [ -z "$1" ]; then
    echo "Usage: $0 <user-file>"
    exit 1
fi

USER_FILE="$1"

Processing the Input File

The script reads the input file line by line, ignoring whitespace and empty lines, and processes each user.

while IFS=';' read -r username groups; do
    username=$(echo $username | xargs)
    groups=$(echo $groups | xargs)
    [ -z "$username" ] && continue

Creating Users and Groups

For each user, the script checks if the user already exists, creates the primary group (same as the username), creates the user, sets up home directory permissions, and generates a password.

if id -u "$username" >/dev/null 2>&1; then
    log_message "User $username already exists"
else
    groupadd "$username"
    log_message "Group $username created"

    useradd -m -g "$username" -s /bin/bash "$username"
    log_message "User $username created with home directory /home/$username"

    chmod 700 "/home/$username"
    log_message "Set permissions for /home/$username"

    password=$(generate_password)
    echo "$username:$password" | chpasswd
    log_message "Password set for user $username"

    echo "$username,$password" >> $PASSWORD_FILE
fi

Adding Users to Additional Groups

The script then adds the user to any additional groups specified in the input file.

if [ -n "$groups" ]; then
    IFS=',' read -ra GROUP_ARRAY <<< "$groups"
    for group in "${GROUP_ARRAY[@]}"; do
        group=$(echo $group | xargs)
        if ! getent group "$group" >/dev/null; then
            groupadd "$group"
            log_message "Group $group created"
        fi
        usermod -aG "$group" "$username"
        log_message "User $username added to group $group"
    done
fi

Completion

Finally, the script logs the completion of the user creation process.

"log_message "User creation process completed"

echo "User creation process completed. Check the log file at $LOG_FILE for details."

To run and test the create_users.sh script, follow these steps:

Step 1:

Prepare Your Environment

Ensure you have the necessary permissions to create users, groups, and modify system files. Running the script might require superuser privileges.

Step 2:

Create the Input File

Create a text file with the usernames and groups. For example, create a file named users.txt with the following content:

isreal;sudo,dev,www-data
isreal2;sudo
isreal3;dev,www-data

Step 3:

Ensure Necessary Directories Exist

Ensure that the directories /var/log and /var/secure exist and have the appropriate permissions. You might need to create them if they don't exist:

sudo mkdir -p /var/log /var/secure
sudo touch /var/log/user_management.log /var/secure/user_passwords.csv
sudo chmod 600 /var/secure/user_passwords.csv

Step 4:

Run the Script

To execute the script, use the following command, passing the name of the input file as an argument:

sudo bash create_users.sh users.txt

Step 5:

Verify the Script's Actions

Check the Log File: Verify the actions logged in /var/log/user_management.log.

sudo cat /var/log/user_management.log

Check the Passwords File: Verify the securely stored passwords in /var/secure/user_passwords.csv.

sudo cat /var/secure/user_passwords.csv

Verify User and Group Creation: Check if the users and groups were created correctly.

List users and groups

getent passwd | grep -E 'isreal|isreal2|isreal3'
getent group | grep -E 'isreal|sudo|dev|www-data'

Check Home Directory Permissions:

Ensure the home directories were created with the correct permissions.

ls -ld /home/isreal /home/isreal2 /home/isreal3

Conclusion

With this, we have successfully automated user creation with a Bash script which could help to reduce errorand ensure reliability, from defining user details in users.txt to execution, the project has transitioned from execution to completion.
To be part of the program that provided this task scenario, visit their official websites to gain more insights

https://hng.tech/internship

https://hng.tech/hire

Thanks for reading