DEV Community: Pratik Nalawade

Setting Up a Monitoring Stack with Nginx Logging Using Grafana, cAdvisor, Promtail, Prometheus, and Loki

Pratik Nalawade — Thu, 05 Sep 2024 17:59:56 +0000

In this guide, I’ll walk you through setting up a comprehensive monitoring and logging stack using Docker and Docker Compose. We'll integrate Nginx logging into Grafana, using Prometheus, Loki, Promtail, and cAdvisor. These tools help you track container resource usage and visualize metrics and logs in real-time.

Why This Stack?

Grafana: Dashboarding tool for visualizing data from various sources.
Loki: Lightweight log aggregation system.
Promtail: Agent to ship logs to Loki.
Prometheus: Monitoring system that collects and processes metrics.
cAdvisor: Analyzes and exposes resource usage for running containers.

Step-by-Step Setup

1. Install Grafana on Debian/Ubuntu

Install Grafana using the following commands:

sudo apt-get install -y apt-transport-https software-properties-common wget
sudo wget -q -O /usr/share/keyrings/grafana.key https://apt.grafana.com/gpg.key
echo "deb [signed-by=/usr/share/keyrings/grafana.key] https://apt.grafana.com stable main" | sudo tee /etc/apt/sources.list.d/grafana.list
sudo apt-get update
sudo apt-get install grafana
sudo systemctl start grafana-server

Verify Grafana by opening http://localhost:3000 in your browser.

2. Install Loki and Promtail Using Docker

First, download the Loki and Promtail configuration files:

wget https://raw.githubusercontent.com/grafana/loki/v2.8.0/cmd/loki/loki-local-config.yaml -O loki-config.yaml
wget https://raw.githubusercontent.com/grafana/loki/v2.8.0/clients/cmd/promtail/promtail-docker-config.yaml -O promtail-config.yaml

Next, run the containers for Loki and Promtail:

docker run -d --name loki -v $(pwd):/mnt/config -p 3100:3100 grafana/loki:2.8.0 --config.file=/mnt/config/loki-config.yaml
docker run -d --name promtail -v $(pwd):/mnt/config -v /var/log:/var/log --link loki grafana/promtail:2.8.0 --config.file=/mnt/config/promtail-config.yaml

3. Install Prometheus and cAdvisor

To monitor containers with Prometheus and cAdvisor, follow these steps:

wget https://raw.githubusercontent.com/prometheus/prometheus/main/documentation/examples/prometheus.yml -O prometheus.yml
docker run -d --name=prometheus -p 9090:9090 -v $(pwd)/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus --config.file=/etc/prometheus/prometheus.yml
docker run -d --name=cadvisor -p 8080:8080 gcr.io/cadvisor/cadvisor:latest

4. Configure Nginx Logging

I set up Nginx to forward its logs to Promtail and visualize them in Grafana. Here's how you can do the same:

Install and run Nginx:

sudo apt install nginx
sudo systemctl start nginx

Add Nginx logs to Promtail:
Update the Promtail configuration (promtail-config.yaml) to include the Nginx logs directory, typically /var/log/nginx/*.log.
Restart Promtail to apply the new configuration:
```
docker restart promtail
```

5. Set Up Grafana Dashboards

Add Prometheus as a data source in Grafana.
Add Loki as a data source for log aggregation.
Create custom dashboards to visualize Nginx access and error logs, along with container metrics from cAdvisor and Prometheus.

Conclusion

This stack gives you a comprehensive monitoring setup with real-time metrics and logs for your Nginx server and Docker containers. You can monitor resource usage, analyze logs, and even set alerts for performance issues.

By leveraging Grafana's powerful visualization capabilities, you get full insights into your infrastructure, ensuring everything runs smoothly.

Automating Security Audits and Server Hardening on Linux Servers with Bash Scripting

Pratik Nalawade — Thu, 29 Aug 2024 14:39:41 +0000

Introduction

In the world of IT and DevOps, security is paramount. Ensuring that Linux servers are secure from vulnerabilities, properly configured, and compliant with industry standards is crucial. This blog explores a Bash script I developed to automate security audits and server hardening on Linux servers, providing a modular, reusable solution that can be easily deployed across multiple environments.

Project Overview

The task was to create a comprehensive Bash script that not only audits security settings on a Linux server but also implements necessary hardening measures. This script aims to identify and address common security vulnerabilities, manage user and group permissions, enforce strict file and directory permissions, monitor services, and ensure that the server’s network configuration adheres to best practices.

Key Components

1. User and Group Audits

Objective: The goal here is to ensure that only authorized users have access to the server, and that their permissions are appropriate.

Code:

# List all users and groups on the server
echo "Listing all users:"
cut -d: -f1 /etc/passwd

echo "Listing all groups:"
cut -d: -f1 /etc/group

# Check for users with UID 0 (root privileges)
echo "Checking for non-standard users with UID 0:"
awk -F: '($3 == "0") {print}' /etc/passwd

# Check for users without passwords
echo "Checking for users without passwords:"
awk -F: '($2 == "") {print $1}' /etc/shadow

Explanation:

The script lists all users and groups by extracting them from /etc/passwd and /etc/group.
It identifies users with UID 0 (root privileges) to ensure no unauthorized users have root access.
The script also checks for users without passwords by inspecting the /etc/shadow file.

2. File and Directory Permissions

Objective: Ensure that sensitive files and directories are not accessible by unauthorized users.

Code:

# Scan for world-writable files and directories
echo "Scanning for world-writable files and directories:"
find / -xdev -type d -perm -0002 -exec ls -ld {} \;

# Check for .ssh directories with secure permissions
echo "Checking .ssh directories permissions:"
find /home -type d -name ".ssh" -exec chmod 700 {} \;

# Report SUID/SGID files
echo "Checking for SUID/SGID files:"
find / -xdev \( -perm -4000 -o -perm -2000 \) -type f -exec ls -ld {} \;

Explanation:

The script scans the entire file system for world-writable files and directories using the find command, which could pose a security risk.
It ensures .ssh directories are secure by setting their permissions to 700.
The script also identifies files with SUID/SGID bits set, which could allow unauthorized users to execute files with elevated privileges.

3. Service Audits

Objective: Ensure that only necessary and authorized services are running, and that they are configured securely.

Code:

# List all running services
echo "Listing all running services:"
systemctl list-units --type=service --state=running

# Check for unnecessary services
echo "Checking for unnecessary services:"
UNNECESSARY_SERVICES=(avahi-daemon cups)
for SERVICE in "${UNNECESSARY_SERVICES[@]}"; do
  systemctl is-active --quiet $SERVICE && echo "$SERVICE is running, consider disabling."
done

# Ensure critical services are running
echo "Checking critical services:"
CRITICAL_SERVICES=(sshd iptables)
for SERVICE in "${CRITICAL_SERVICES[@]}"; do
  systemctl is-active --quiet $SERVICE || echo "$SERVICE is not running!"
done

Explanation:

The script lists all running services using systemctl and checks for any unnecessary ones.
It verifies that critical services like sshd and iptables are running, ensuring that the server’s basic security mechanisms are in place.

4. Firewall and Network Security

Objective: Verify that the server’s firewall is configured correctly and that there are no insecure network settings.

Code:

# Verify if the firewall is active
echo "Checking if firewall is active:"
ufw status | grep -qw "active" || echo "Firewall is not active!"

# Report open ports and services
echo "Listing open ports:"
netstat -tuln

# Check for IP forwarding
echo "Checking for IP forwarding:"
sysctl net.ipv4.ip_forward

Explanation:

The script checks if the firewall (ufw) is active and configured properly.
It lists all open ports and their associated services using netstat.
The script also checks if IP forwarding is enabled, which could expose the server to security risks if not required.

5. IP and Network Configuration Checks

Objective: Ensure that the server’s IP addresses are properly configured and identified as public or private.

Code:

# List all IP addresses and classify them
echo "Listing all IP addresses:"
ip -o addr show | awk '{print $2, $4}'

# Identify public vs private IPs
echo "Identifying public vs private IP addresses:"
ip -o -4 addr list | awk '{print $4}' | while read IP; do
  if [[ $IP =~ ^10\.|^172\.16|^192\.168 ]]; then
    echo "$IP is private."
  else
    echo "$IP is public."
  fi
done

Explanation:

The script lists all IP addresses assigned to the server using the ip command.
It then classifies each IP address as public or private, providing a clear understanding of the server’s network exposure.

6. Security Updates and Patching

Objective: Ensure the server is up-to-date with the latest security patches.

Code:

# Check for available updates
echo "Checking for available updates:"
apt-get update -qq
apt-get upgrade -s | grep -i "security"

# Ensure automatic security updates are enabled
echo "Ensuring automatic security updates are enabled:"
dpkg-reconfigure --priority=low unattended-upgrades

Explanation:

The script checks for available security updates using apt-get.
It ensures that automatic security updates are enabled, reducing the risk of the server being compromised due to outdated software.

7. Log Monitoring

Objective: Detect any suspicious activity that could indicate a security breach.

Code:

# Check for suspicious log entries
echo "Checking for suspicious log entries:"
grep "Failed password" /var/log/auth.log | tail -10

Explanation:

The script searches the authentication log (/var/log/auth.log) for failed login attempts, which could indicate brute-force attacks or unauthorized access attempts.

8. Server Hardening Steps

Objective: Implement security best practices to harden the server.

Code:

# Disable root login via SSH
echo "Disabling root login via SSH:"
sed -i 's/PermitRootLogin yes/PermitRootLogin no/' /etc/ssh/sshd_config
systemctl reload sshd

# Disable IPv6 if not required
echo "Disabling IPv6:"
echo "net.ipv6.conf.all.disable_ipv6 = 1" >> /etc/sysctl.conf
sysctl -p

# Secure GRUB with a password
echo "Securing GRUB with a password:"
grub2-mkpasswd-pbkdf2

Explanation:

The script disables root login via SSH to prevent unauthorized access.
It disables IPv6 if not required, reducing the server’s attack surface.
The script also secures the GRUB bootloader with a password to prevent unauthorized changes to boot parameters.

9. Custom Security Checks

Objective: Allow customization of the script to meet specific security policies.

Code:

# Example custom check: Ensure no files have '777' permissions
echo "Checking for files with 777 permissions:"
find / -type f -perm 0777 -exec ls -l {} \;

Explanation:

The script includes an example of a custom security check, ensuring that no files have overly permissive 777 permissions.
This modular design allows users to easily add or modify security checks according to their organization’s policies.

10. Reporting and Alerting

Objective: Provide a comprehensive summary report and optional alerts for critical issues.

Code:

# Generate summary report
REPORT="/var/log/security_audit_report.txt"
echo "Generating security audit report..."
{
  echo "Security Audit Report"
  date
  echo "---------------------------------"
  echo "User and Group Audits:"
  # Include results of user and group audits
  echo "File and Directory Permissions:"
  # Include results of permission checks
  echo "Service Audits:"
  # Include results of service audits
  echo "Firewall and Network Security:"
  # Include results of firewall and network checks
  echo "IP and Network Configuration:"
  # Include results of IP checks
} > $REPORT

# Send email alert if critical issues found
echo "Sending email alert if necessary..."
# Example: Send an alert if a critical issue is found
grep "CRITICAL" $REPORT && mail -s "Security Audit Alert" admin@example.com < $REPORT

Explanation**:

The script generates a structured summary report that includes the results of all security checks and hardening steps.
If any critical issues are detected, the script can send an email alert to the administrator.

Challenges and Solutions

Dependency Management

One of the initial challenges was ensuring that all necessary tools and dependencies were installed on the server. For example, tools like netstat for network monitoring and chkrootkit for rootkit detection had to be installed manually if not already present. To resolve this, the script checks for required packages and installs them as needed.

Permission Issues

Running the script without appropriate permissions led to errors, particularly when modifying system files or changing configurations. This was resolved by running the script with sudo, ensuring that it had the necessary privileges to execute all commands.

Modular Design

Creating a modular script that could be easily extended or customized was another challenge. To achieve this, each audit or hardening step was encapsulated in its function, allowing for independent execution and easy updates. This approach made the script more maintainable and adaptable to different server environments.

Public vs. Private IP Identification

Identifying public versus private IP addresses required careful handling of the server’s network interfaces. By leveraging ip commands and regular expressions, the script accurately categorized each IP address and provided a clear summary in the final report.

Reporting and Alerting

Generating a comprehensive and readable report was essential for ensuring that the security audit results were actionable. The script formats the output into a structured report, with critical findings highlighted. Additionally, integrating email alerts for critical issues ensures that administrators are promptly notified of any urgent vulnerabilities.

How to Use the Script

Prerequisites

Ensure that your server is running a Linux distribution (e.g., Ubuntu, CentOS).
Install necessary packages like net-tools, chkrootkit, ufw, and unattended-upgrades.
Clone the script from the GitHub repository and give it executable permissions using chmod +x.

Running the Script

Full Audit and Hardening: To run the full audit and hardening process, simply execute the script with sudo privileges:

   sudo ./security_audit_hardening.sh

Custom Checks: If you wish to run specific checks or hardening steps, modify the script’s configuration file to enable or disable particular modules.
Reviewing Reports: After execution, the script will generate a summary report in the specified output directory. Review this report for any vulnerabilities or issues that require attention.
Email Alerts: If configured, email alerts will be sent automatically for any critical findings.

Conclusion

This Bash script offers a powerful, automated solution for conducting security audits and hardening Linux servers. By addressing common vulnerabilities, enforcing strict security measures, and providing detailed reporting, it helps ensure that your servers remain secure and compliant with industry standards.

Check out the complete script and documentation on my GitHub![https://github.com/PRATIKNALAWADE/Audit-ServerHardening]

Building a Real-Time System Monitoring Dashboard with Bash

Pratik Nalawade — Tue, 27 Aug 2024 09:55:11 +0000

In today’s fast-paced IT environments, monitoring system resources in real time is crucial for maintaining the health and performance of servers. Whether you’re managing a proxy server or any other critical infrastructure, having a lightweight, customizable monitoring solution can make a big difference. In this blog post, I’ll walk you through how to create a real-time system monitoring dashboard using a Bash script. This script will allow you to keep an eye on CPU usage, network activity, disk space, and more—all from your terminal.

Why Bash?

Bash is a powerful and flexible tool that comes pre-installed on most Linux systems. While there are many sophisticated monitoring tools available, Bash offers a simple, no-frills approach that’s easy to customize and deploy. Plus, it’s a great way to deepen your understanding of system resources and shell scripting.

Key Features of the Script

The monitoring automation script we’ll build includes the following features:

Top 10 Most Used Applications: Displays the top 10 processes consuming the most CPU and memory.
Network Monitoring: Tracks the number of concurrent connections, packet drops, and data transfer rates.
Disk Usage: Monitors disk space usage, highlighting partitions using more than 80% of their capacity.
System Load: Shows the current load average and a detailed breakdown of CPU usage.
Memory Usage: Provides an overview of total, used, and free memory, including swap usage.
Process Monitoring: Lists the number of active processes and the top 5 resource-intensive ones.
Service Monitoring: Checks the status of essential services like sshd, nginx/apache, and iptables.
Custom Dashboard: Allows users to view specific sections of the dashboard using command-line switches.

Building the Script

Let’s dive into how we can build this script. Below is a breakdown of each section.

1. Monitoring Top Applications

To display the top 10 applications consuming the most CPU and memory, we use the ps command combined with sort:

ps aux --sort=-%cpu | head -n 11
ps aux --sort=-%mem | head -n 11

This provides a quick overview of the most resource-hungry processes.

2. Network Monitoring

For network monitoring, we need to track concurrent connections, packet drops, and data transfer rates. We use netstat and ifconfig for this:

netstat -an | grep ESTABLISHED | wc -l
ifconfig eth0 | grep 'RX packets'
ifconfig eth0 | grep 'TX packets'

These commands give us insights into the network activity in real-time.

3. Disk Usage

Monitoring disk usage is crucial for preventing storage issues. Here’s how we can highlight partitions using more than 80% of disk space:

df -h | awk '$5 > 80 {print $0}'

This snippet checks the disk usage and flags any partitions that exceed the 80% threshold.

4. System Load

To monitor the system load, we use the uptime command:

uptime

This command gives a snapshot of the load average, which indicates how many processes are demanding CPU time.

5. Memory Usage

Memory usage is tracked using free:

free -h

This provides a detailed look at total, used, and free memory, including swap space.

6. Process Monitoring

We can easily monitor active processes and identify the top 5 resource-intensive ones:

ps -e | wc -l
ps aux --sort=-%cpu | head -n 6

This combination helps in understanding the process load on the system.

7. Service Monitoring

Ensuring essential services are running is vital. We can check the status of critical services like sshd and iptables using systemctl:

systemctl is-active sshd
systemctl is-active iptables

These checks ensure that critical services are operational and correctly configured.

8. Custom Dashboard

Finally, to make the script more flexible, we add command-line switches to allow users to view specific parts of the dashboard:

case "$1" in
    -cpu) monitor_cpu ;;
    -memory) monitor_memory ;;
    -network) monitor_network ;;
    -disk) monitor_disk ;;
    *) echo "Usage: $0 {-cpu|-memory|-network|-disk}" ;;
esac

This modular approach makes it easy to focus on the metrics that matter most at any given time.

Lessons Learned

While building this script, I encountered a few challenges. For example, I initially forgot to install net-tools, which includes the netstat command necessary for network monitoring. This was quickly resolved by installing the package via sudo apt-get install net-tools. Additionally, ensuring that the script was executable required setting the correct permissions with chmod +x.

Conclusion

This Bash script provides a powerful, lightweight way to monitor system resources in real-time. It’s easy to customize and deploy, making it a valuable tool for anyone managing Linux servers. Whether you’re a seasoned sysadmin or just getting started with shell scripting, this project offers a hands-on way to deepen your understanding of system monitoring.

You can find the full script and documentation in my GitHub repository. Feel free to clone, modify, and use it as a starting point for your own monitoring solutions.

DevOps Monitoring Project

Pratik Nalawade — Sat, 24 Aug 2024 14:39:48 +0000

Learning DevOps Monitoring with DevOps Shack: A Hands-On Journey

In the ever-evolving world of DevOps, monitoring is a critical aspect of maintaining the health and performance of your infrastructure. Recently, I embarked on a hands-on learning journey, following along with a YouTuber named DevOps Shack. Through their comprehensive tutorials, I implemented a full-fledged monitoring solution using Prometheus, Node Exporter, Alertmanager, and Blackbox Exporter. This blog post shares my experience and the key takeaways from the project.

Project Overview

The project is designed to provide an end-to-end monitoring solution for your infrastructure. By following the guidance of DevOps Shack, I was able to set up a robust system that not only monitors the health of virtual machines but also sends alerts for critical issues and probes service availability.

Tools and Technologies

Prometheus: Used for collecting and storing metrics.
Node Exporter: Used to expose hardware and OS metrics to Prometheus.
Alertmanager: Manages alerts generated by Prometheus.
Blackbox Exporter: Probes endpoints to check their availability.

Prerequisites

Before getting started, I ensured that:

Two virtual machines (VMs) were prepared.
wget and tar were installed on both VMs.
I had the necessary permissions to download, extract, and run the binaries.

Step-by-Step Setup

VM-1: Setting Up Node Exporter

1. Download Node Exporter:

wget https://github.com/prometheus/node_exporter/releases/download/v1.8.1/node_exporter-1.8.1.linux-amd64.tar.gz

2. Extract Node Exporter:

tar xvfz node_exporter-1.8.1.linux-amd64.tar.gz

3. Start Node Exporter:

cd node_exporter-1.8.1.linux-amd64
./node_exporter &

VM-2: Setting Up Prometheus, Alertmanager, and Blackbox Exporter

Prometheus Setup

1. Download Prometheus:

wget https://github.com/prometheus/prometheus/releases/download/v2.52.0/prometheus-2.52.0.linux-amd64.tar.gz

2. Extract Prometheus:

tar xvfz prometheus-2.52.0.linux-amd64.tar.gz

3. Start Prometheus:

cd prometheus-2.52.0.linux-amd64
./prometheus --config.file=prometheus.yml &

Alertmanager Setup

1. Download Alertmanager:

wget https://github.com/prometheus/alertmanager/releases/download/v0.27.0/alertmanager-0.27.0.linux-amd64.tar.gz

2. Extract Alertmanager:

tar xvfz alertmanager-0.27.0.linux-amd64.tar.gz

3. Start Alertmanager:

cd alertmanager-0.27.0.linux-amd64
./alertmanager --config.file=alertmanager.yml &

Blackbox Exporter Setup

1. Download Blackbox Exporter:

wget https://github.com/prometheus/blackbox_exporter/releases/download/v0.25.0/blackbox_exporter-0.25.0.linux-amd64.tar.gz

2. Extract Blackbox Exporter:

tar xvfz blackbox_exporter-0.25.0.linux-amd64.tar.gz

3. Start Blackbox Exporter:

cd blackbox_exporter-0.25.0.linux-amd64
./blackbox_exporter &

Configuration Details

Prometheus Configuration (prometheus.yml)

Global Configuration:
- Scrape interval: 15s
- Evaluation interval: 15s
Scrape Configurations:
- Prometheus itself:
- Job name: prometheus
- Target: localhost:9090
- Node Exporter:
- Job name: node_exporter
- Target: 3.110.195.114:9100
- Blackbox Exporter:
- Job name: blackbox
- Targets:
  - http://prometheus.io
  - https://prometheus.io
  - http://3.110.195.114:8080/

Alertmanager Configuration (alertmanager.yml)

Routing Configuration:
- Group alerts by: alertname
- Group wait: 30s
- Group interval: 5m
- Repeat interval: 1h
- Default receiver: email-notifications
Receiver Configuration:
- Receiver name: email-notifications
- Email recipient: email@gmail.com
- SMTP server: smtp.gmail.com:587
- Auth username and password (to be configured).
Inhibition Rules:
- Source match: severity: critical
- Target match: severity: warning
- Equal fields: alertname, dev, instance

Alert Rules Configuration (alert_rules.yml)

Some of the key alert rules I configured include:

InstanceDown: Alerts if an instance is down for more than 1 minute.
WebsiteDown: Alerts if a website probe fails.
HostOutOfMemory: Alerts if memory availability drops below 25%.
HostOutOfDiskSpace: Alerts if disk space is less than 50%.
HostHighCpuLoad: Alerts if CPU load exceeds 80%.
ServiceUnavailable: Alerts if a service is unavailable.
HighMemoryUsage: Alerts if memory usage exceeds 90%.
FileSystemFull: Alerts if file system free space drops below 10%.

Firewall and Security Settings

I had to configure the firewall to allow traffic on the necessary ports:

Prometheus: 9090
Alertmanager: 9093
Blackbox Exporter: 9115
Node Exporter: 9100

Key Features and Functionalities

Monitoring: Using Node Exporter and Prometheus, I was able to monitor crucial system metrics such as CPU usage, memory availability, and disk space. These metrics are scraped at regular intervals, providing real-time insights into the system's performance.

Alerting: With Alertmanager, I was able to set up notifications for critical events, ensuring that I was immediately informed of any issues, such as an instance going down or high CPU load. The flexibility of Alertmanager's configuration allowed me to tailor alerts to meet specific needs.

Probing: The Blackbox Exporter allowed me to monitor the availability and response times of various endpoints, including web services, ensuring that they remained accessible and responsive.

Challenges and Solutions

One of the challenges I faced during this project was configuring the firewall and ensuring proper communication between the services across the two VMs. By carefully adjusting firewall rules and thoroughly reviewing the configuration files, I was able to overcome these hurdles and achieve a seamless setup.

Future Enhancements

Moving forward, I plan to enhance this setup by integrating Grafana for more sophisticated visualization of the metrics collected by Prometheus. Additionally, I am considering automating the entire deployment process using Ansible, making it easier to replicate the setup across different environments.

Conclusion

Following along with DevOps Shack's tutorials provided me with a solid foundation in setting up a comprehensive monitoring and alerting system using open-source tools. This project, DevOps Shack, serves as a testament to the power of hands-on learning in mastering DevOps concepts. I encourage anyone interested in DevOps to explore these tools, experiment with configurations, and discover the power of effective monitoring in maintaining a healthy infrastructure.

You can explore more about the project on DevOps Shack’s YouTube channel (insert the actual link). If you have any questions or feedback, feel free to reach out!

This revised version emphasizes your learning experience and the value of following DevOps Shack's guidance. Feel free to personalize it further before publishing!

Shell scripting Interview Questions

Pratik Nalawade — Mon, 19 Aug 2024 10:31:23 +0000

Most asked shell scripting and Linux interview questions along with their answers:

What is a shell script?
— Answer: A shell script is a text file containing a sequence of commands that are executed by the shell interpreter. It allows automation of repetitive tasks and execution of multiple commands in sequence.
Differentiate between a shell and a terminal.
— Answer: A shell is a command-line interpreter that interprets user commands and executes them. A terminal is a user interface that provides access to the shell. Multiple terminals can run concurrently, each running its own instance of the shell.
Explain the difference between absolute and relative paths.
— Answer: An absolute path specifies the location of a file or directory from the root directory (/) of the file system. A relative path specifies the location of a file or directory relative to the current working directory.
What is the shebang (#!) line in a shell script?
— Answer: The shebang line (#!) is a special line at the beginning of a shell script that indicates the path to the shell interpreter that should be used to execute the script. For example, #!/bin/bash specifies that the Bash shell should be used.
How do you comment in a shell script?
— Answer: Comments in shell scripts are preceded by the # symbol. Anything after the # symbol on a line is considered a comment and is ignored by the shell.
What is the difference between $ and $$ in shell scripting?
— Answer: In shell scripting, $ is used to reference the value of a variable, whereas $$ represents the process ID (PID) of the current shell.
Explain the difference between the && and || operators in shell scripting.
— Answer: The && operator is used to execute the command following it only if the preceding command succeeds (returns a zero exit status). The || operator is used to execute the command following it only if the preceding command fails (returns a non-zero exit status).
How do you pass arguments to a shell script?
— Answer: Arguments can be passed to a shell script as command-line arguments. These arguments can be accessed within the script using special variables such as $1, $2, etc., representing the first, second, etc., arguments respectively.
What is process substitution in shell scripting?
— Answer: Process substitution is a feature of some shells (such as Bash) that allows the output of a command or commands to be used as input to another command or commands. It is represented by the <() and >() syntax.
Explain the grep command in Linux.
— Answer: The grep command is used to search for patterns in text files or streams. It outputs lines that match a specified pattern or regular expression.
What is the purpose of the awk command in Linux?
— Answer: The awk command is a powerful text processing tool used for pattern scanning and processing. It processes input lines based on patterns and performs actions defined by the user.
Explain the purpose of the tar command in Linux.
— Answer: The tar command is used to create, view, and extract archives (tarballs) containing multiple files. It is often used for packaging files and directories for distribution or backup purposes.
What is SSH and how is it used in Linux?
— Answer: SSH (Secure Shell) is a cryptographic network protocol used for secure remote access to Linux systems. It provides encrypted communication between the client and the server, allowing users to log in and execute commands on remote machines securely.
How do you check system resource usage in Linux?
— Answer: System resource usage can be checked using commands such as top, htop, free, and df. These commands provide information about CPU usage, memory usage, and disk space usage respectively.

These questions cover a range of fundamental concepts and practical skills related to shell scripting and Linux system administration, which are commonly assessed in interviews for DevOps and system administration roles.

Shell script count no. of s in mississippi

x="mississippi"
grep -o "s" <<<"$x" | wc -l

explanation:
grep -o “s” <<<”$x”:

grep: This command is used to search for patterns in text.
-o: This option tells grep to only output the parts of the text that match the pattern.

“s”: This is the pattern we’re searching for, in this case, the letter “s”.
<<<”$x”: This feeds the value of the variable $x as input to grep. In simple terms, it’s like saying “search for the letter ‘s’ in the text stored in the variable $x”.

| (pipe symbol): This symbol is used to pass the output of one command as the input to another command. It’s like connecting two commands together in a pipeline.

wc -l: wc: Stands for word count. It’s used to count lines, words, and characters in text.
-l: This option tells wc to count only the lines in the input text. In this case, it counts the number of lines.

Putting it all together, the command grep -o “s” <<<”$x” | wc -l first searches for the letter “s” in the text stored in the variable $x, then it counts the number of lines in the output, which corresponds to the number of times the letter “s” appears in the text.

crontab and job scheduling

crontab is a command used in Unix-like operating systems to schedule jobs or commands to run periodically at fixed times, dates, or intervals. Here’s an example of how to use crontab:

Let’s say you want to schedule a script to run every day at 3:00 AM.

Open your terminal.
Type crontab -e and press Enter. This command opens the crontab editor.
If it’s your first time running crontab -e, it might prompt you to choose an editor. Select your preferred editor (e.g., nano, vim).
Once the editor opens, add a new line at the end of the file with the following format:

m h * * * /path/to/your/script.sh
Replace /path/to/your/script.sh with the actual path to your script.

m stands for the minute (0–59). — h stands for the hour (0–23). — * represents all possible values for that field. — So * * means every minute of every hour. — Finally, the full line means “run the script at every hour (0–23) and minute (0–59) of every day of every month and every day of the week”.

So, if you want your script to run every day at 3:00 AM, you would set it up like this:

0 3 * * * /path/to/your/script.sh

Save and close the crontab editor. In nano, you can do this by pressing Ctrl + O to write the file and then Ctrl + X to exit.
You’ve now set up your cron job. It will run your script at 3:00 AM every day.

Remember, cron uses 24-hour time format, so 3:00 AM is represented as 3. If you wanted to run the script every Sunday at 3:00 AM, you would modify the cron job like this:

0 3 * * 0 /path/to/your/script.sh
In this case, 0 in the fifth field represents Sunday.

That’s a basic example of how to use crontab to schedule tasks in Unix-like systems.

Loops and conditionals in shell

Below are examples of a shell script demonstrating the usage of if, else-if, and for loop constructs:

Using if, else-if, and else statements:

!/bin/bash

1. Prompt user to enter a number

echo "Enter a number:"
read num

Check if the number is positive, negative, or zero

if [ $num -gt 0 ]; then
echo "The number is positive."
elif [ $num -lt 0 ]; then
echo "The number is negative."
else
echo "The number is zero."
fi

2. Using a `for` loop to iterate over a list of items:**

!/bin/bash

Define a list of fruits

fruits=("Apple" "Banana" "Orange" "Grapes" "Watermelon")

Iterate over the list of fruits using a for loop

echo "List of fruits:"
for fruit in "${fruits[@]}"
do
echo "$fruit"
done
In the first script, the user is prompted to enter a number, and then the script checks whether the number is positive, negative, or zero using if, elif, and else statements.

In the second script, a list of fruits is defined, and a for loop is used to iterate over each item in the list and print it to the console.

These examples demonstrate the basic usage of if, else-if, else, and for loop constructs in shell scripting.

Shell script to print only process ids of all processes

ps -ef | awk -F " " '{print $2}'

Hard link vs soft link in linux

In Linux, “hard” and “soft” links are two types of links used to point to files. Here’s a brief explanation of each along with their respective syntax:

Hard Link:
— A hard link is a direct pointer to the inode (metadata) of a file. It essentially creates a new directory entry that refers to the same underlying data as the original file.
— Changes made to the original file will affect all hard links pointing to it, as they all reference the same data blocks.
— Hard links cannot be created for directories.
— Hard links cannot span across different filesystems.
Syntax to create a hard link:

Example:



 ln myfile.txt myhardlink.txt
2. Soft Link (Symbolic Link or Symlink):
— A soft link, also known as a symbolic link or symlink, is a special file that points to another file or directory by its pathname.
— Unlike a hard link, a soft link simply contains the path of the target file or directory.
— Soft links can span across different filesystems.
— Deleting the original file or directory won’t affect the soft link; it will become a “dangling” link pointing to nothing.

Syntax to create a soft link:

 ln -s <target_file> <soft_link_name>


 Example:

 ln -s /path/to/original_file.txt softlink.txt
Remember to replace `<original_file>`, `<hard_link_name>`, `<target_file>`, and `<soft_link_name>` with the appropriate file names and paths in the commands.

20. Shell scripting disadvantages
While shell scripting is powerful and widely used for automating tasks in Unix-like operating systems, it also comes with its own set of disadvantages:

1. Portability: Shell scripts are typically written for specific Unix-like operating systems such as Linux or macOS. Porting shell scripts to other platforms, such as Windows, can be challenging due to differences in shell syntax and commands.

2. Performance: Shell scripts are interpreted rather than compiled, which can lead to slower execution compared to compiled languages like C or Java, especially for complex tasks or large data processing.

3. Error Handling: Error handling in shell scripts can be more challenging compared to compiled languages. Shell scripts often rely on return codes or exit statuses of commands, which may not provide detailed information about the cause of errors.

4. Limited Functionality: While shell scripting is suitable for many system administration tasks and simple automation, it may lack the robustness and advanced features available in higher-level programming languages.

5. Security Risks: Writing secure shell scripts requires careful consideration of potential vulnerabilities, such as command injection and improper handling of user input. Insecure shell scripts can pose significant security risks to systems and data.

6. Debugging: Debugging shell scripts can be more difficult compared to compiled languages. Shell scripts may produce cryptic error messages, and debugging tools are often limited, requiring manual inspection of code and output.

7. Complexity: As shell scripts grow in size and complexity, they can become difficult to maintain and understand, especially for developers unfamiliar with shell scripting conventions and best practices.

8. Limited Support for Data Structures: Shell scripting languages like Bash have limited support for complex data structures such as arrays and associative arrays, which can make certain programming tasks more challenging.

Despite these disadvantages, shell scripting remains a valuable tool for system administration, automation, and quick prototyping of tasks in Unix-like environments. It’s essential to understand the limitations and choose the appropriate tool for the task at hand.

Deploying a Scalable Web Application with Terraform on AWS

Pratik Nalawade — Sat, 17 Aug 2024 12:09:36 +0000

In today's cloud-centric world, infrastructure management has shifted from manual configurations to automation with Infrastructure as Code (IaC). Terraform, developed by HashiCorp, is a powerful IaC tool that allows us to define and provision data center infrastructure using a declarative configuration language. In this post, I'll walk you through a project where I used Terraform to deploy a scalable web application on AWS, leveraging a range of services including VPCs, EC2 instances, S3 buckets, and an Application Load Balancer (ALB).

Project Overview

The goal of this project is to set up a simple yet scalable web application hosted on AWS. The infrastructure includes:

A Virtual Private Cloud (VPC) with custom subnets.
An Internet Gateway and Route Table for connectivity.
Security Groups to manage access.
EC2 instances running Apache servers.
An S3 bucket for storage.
An Application Load Balancer to distribute traffic across the instances.

1. Setting Up Terraform and AWS Provider

To start with Terraform, we need to configure the provider, which in this case is AWS. The provider.tf file specifies the AWS provider and the region where the infrastructure will be deployed.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "5.11.0"
    }
  }
}

provider "aws" {
  region = "us-east-1"
}

This configuration ensures that Terraform uses the correct provider and region. It's crucial to keep the provider version updated to take advantage of the latest features and security patches.

2. VPC and Subnet Configuration

Next, we define the Virtual Private Cloud (VPC) where all our resources will reside. The VPC is isolated and helps secure our instances and other resources.

resource "aws_vpc" "myvpc" {
  cidr_block = var.cidr
}

We also create two subnets in different availability zones to ensure high availability. These subnets will host our EC2 instances.

hcl
resource "aws_subnet" "sub1" {
  vpc_id = aws_vpc.myvpc.id
  cidr_block = "10.0.0.0/24"
  availability_zone = "us-east-1a"
  map_public_ip_on_launch = true
}

resource "aws_subnet" "sub2" {
  vpc_id = aws_vpc.myvpc.id
  cidr_block = "10.0.1.0/24"
  availability_zone = "us-east-1b"
  map_public_ip_on_launch = true
}

These subnets are public, allowing us to access the instances directly over the internet.

3. Internet Gateway and Route Table

To connect our VPC to the internet, we need an Internet Gateway. The gateway is associated with the VPC, and a route table is configured to direct internet-bound traffic through the gateway.

resource "aws_internet_gateway" "igw" {
  vpc_id = aws_vpc.myvpc.id
}

resource "aws_route_table" "RT" {
  vpc_id = aws_vpc.myvpc.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.igw.id
  }
}

resource "aws_route_table_association" "rta1" {
  subnet_id      = aws_subnet.sub1.id
  route_table_id = aws_route_table.RT.id
}

resource "aws_route_table_association" "rta2" {
  subnet_id      = aws_subnet.sub2.id
  route_table_id = aws_route_table.RT.id
}

This setup ensures that any traffic from our instances can reach the internet, and vice versa.

4. Security Groups

Security groups act as virtual firewalls, controlling inbound and outbound traffic to our instances. In this project, I created a security group that allows HTTP (port 80) and SSH (port 22) traffic from any IP address.

resource "aws_security_group" "webSg" {
  name_prefix = "web-sg"
  vpc_id = aws_vpc.myvpc.id

  ingress {
    description = "HTTP from VPC"
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  ingress {
    description = "SSH"
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name = "Web-sg"
  }
}

While allowing SSH access from anywhere isn't a best practice, it's done here for simplicity. In a production environment, restrict SSH access to specific IP addresses or use a bastion host.

5. Deploying EC2 Instances

Two EC2 instances are deployed in different subnets, ensuring redundancy. Each instance runs an Apache server with a custom user data script that installs Apache, creates a simple HTML file, and starts the service.

resource "aws_instance" "webserver1" {
  ami                    = "ami-0261755bbcb8c4a84"
  instance_type          = "t2.micro"
  vpc_security_group_ids = [aws_security_group.webSg.id]
  subnet_id              = aws_subnet.sub1.id
  user_data              = base64encode(file("userdata.sh"))
}

resource "aws_instance" "webserver2" {
  ami                    = "ami-0261755bbcb8c4a84"
  instance_type          = "t2.micro"
  vpc_security_group_ids = [aws_security_group.webSg.id]
  subnet_id              = aws_subnet.sub2.id
  user_data              = base64encode(file("userdata1.sh"))
}

The user data scripts (userdata.sh and userdata1.sh) are executed when the instance launches, automating the setup process. Here's an example of one of the scripts:

#!/bin/bash
apt update
apt install -y apache2

INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)

cat <<EOF > /var/www/html/index.html
<!DOCTYPE html>
<html>
<head>
  <title>My Portfolio</title>
</head>
<body>
  <h1>Terraform Project Server 1</h1>
  <h2>Instance ID: <span style="color:green">$INSTANCE_ID</span></h2>
</body>
</html>
EOF

systemctl start apache2
systemctl enable apache2

This script updates the package lists, installs Apache, and creates a simple HTML file that displays the instance ID.

6. Setting Up the S3 Bucket

An S3 bucket is also created for storage needs, which can be useful for storing logs, backups, or static assets.

resource "aws_s3_bucket" "example" {
  bucket = "aUnique-name-for-s3-terraform"
}

In this project, the S3 bucket is created with a unique name and can be used for various purposes depending on the application's needs.

7. Configuring the Application Load Balancer (ALB)

The Application Load Balancer (ALB) is set up to distribute traffic across the two EC2 instances. This ensures that the application remains available even if one instance fails.

resource "aws_lb" "myalb" {
  name               = "myalb"
  internal           = false
  load_balancer_type = "application"
  security_groups = [aws_security_group.webSg.id]
  subnets         = [aws_subnet.sub1.id, aws_subnet.sub2.id]

  tags = {
    Name = "web"
  }
}

resource "aws_lb_target_group" "tg" {
  name     = "myTG"
  port     = 80
  protocol = "HTTP"
  vpc_id   = aws_vpc.myvpc.id

  health_check {
    path = "/"
    port = "traffic-port"
  }
}

resource "aws_lb_target_group_attachment" "attach1" {
  target_group_arn = aws_lb_target_group.tg.arn
  target_id        = aws_instance.webserver1.id
  port             = 80
}

resource "aws_lb_target_group_attachment" "attach2" {
  target_group_arn = aws_lb_target_group.tg.arn
  target_id        = aws_instance.webserver2.id
  port             = 80
}

resource "aws_lb_listener" "listener" {
  load_balancer_arn = aws_lb.myalb.arn
  port              = 80
  protocol          = "HTTP"

  default_action {
    target_group_arn = aws_lb_target_group.tg.arn
    type             = "forward"
  }
}

The ALB is configured to listen on port 80 and forward traffic to the target group, which contains the two EC2 instances. By distributing the load between the instances, the ALB ensures higher availability and better fault tolerance.

Monitoring and Scaling Monitoring the health of our infrastructure is crucial for maintaining the reliability and performance of the application. In this project, AWS CloudWatch is used for monitoring metrics like CPU utilization, network traffic, and instance health. By setting up CloudWatch Alarms, we can trigger actions such as auto-scaling when specific thresholds are reached.

For example, if the CPU utilization of an EC2 instance exceeds a certain percentage, an auto-scaling policy can be triggered to launch additional instances. This ensures that the application can handle increased traffic without degradation in performance.

Automating with Terraform Modules As projects grow, so does the complexity of managing infrastructure. Terraform modules allow us to break down our configurations into reusable components. In this project, I created modules for the VPC, EC2 instances, and ALB, making the infrastructure more modular and easier to manage.

Here's an example of how the VPC module might look:

hcl
Copy code
module "vpc" {
  source = "./modules/vpc"

  cidr = var.cidr
}

By using modules, we can easily replicate environments (e.g., staging, production) with consistent configurations, reducing the risk of errors.

Conclusion
This Terraform project demonstrates how to automate the deployment of a scalable web application on AWS. By leveraging Terraform's capabilities, we can quickly spin up infrastructure that is consistent, reliable, and easily scalable. The use of modules, monitoring tools like CloudWatch, and automated scaling policies further enhances the robustness of the infrastructure.

Whether you're building a simple web application or a complex multi-tier architecture, Terraform offers the flexibility and power to manage your infrastructure as code. As cloud environments continue to evolve, tools like Terraform will remain essential for managing infrastructure efficiently.

This project has been a valuable learning experience, reinforcing the importance of automation, scalability, and monitoring in cloud environments. I encourage anyone interested in cloud infrastructure to explore Terraform and experiment with deploying their own projects on AWS.

If you have any questions or suggestions, feel free to reach out or leave a comment. Happy coding!

git repo:https://github.com/PRATIKNALAWADE/Terraform-AWS/

Building and Dockerizing a MERN Stack Application

Pratik Nalawade — Thu, 08 Aug 2024 14:23:54 +0000

The MERN stack (MongoDB, Express.js, React, Node.js) is a powerful technology stack for building modern web applications. In this blog post, we’ll walk through the process of creating a simple Todo application using the MERN stack and then dockerizing it for easy deployment and scalability.

A Closer Look at MERN Stack Components
MongoDB: A cross-platform document database
MongoDB is a NoSQL (non-relational) document-oriented database. Data is stored in flexible documents with a JSON (JavaScript Object Notation)-based query language. MongoDB is known for being flexible and easy to scale.

Express: A back-end web application framework
Express is a web application framework for Node.js, another MERN component. Instead of writing full web server code by hand on Node.js directly, developers use Express to simplify the task of writing server code.

React: A JavaScript library for building user interfaces
React was originally created by a software engineer at Facebook, and was later open-sourced. The React library can be used for creating views rendered in HTML. In a way, it’s the defining feature of the stack.

Node.js: A cross-platform JavaScript runtime environment
Node.js is constructed on Chrome’s V8 JavaScript engine. It’s designed to build scalable network applications and can execute JavaScript code outside of a browser.

Prerequisites
Before we start, ensure you have the following installed on your machine:

Node.js and npm
MongoDB
Docker
Step 1: Setting Up the Project
First, create a directory for your project and initialize it:

mkdir merntodo
cd merntodo

Step 2: Creating the Backend with Node.js and Express
Initialize the backend:

mkdir server
cd server
npm init -y
npm install express mongoose cors body-parser

Create a basic Express server (server/server.js):

const express = require('express');
const mongoose = require('mongoose');
const cors = require('cors');
const bodyParser = require('body-parser');
const app = express();
const PORT = process.env.PORT || 5000;
app.use(cors());
app.use(bodyParser.json());
const dbURI = process.env.MONGO_URI || 'mongodb://localhost:27017/mern-todo';
mongoose.connect(dbURI, {
  useNewUrlParser: true,
  useUnifiedTopology: true
}).then(() => console.log('MongoDB connected'))
  .catch(err => console.log(err));
const todoRoutes = require('./routes/todo');
app.use('/api', todoRoutes);
app.listen(PORT, () => {
  console.log(`Server running on port ${PORT}`);
});

use your db URL in const dbURI = process.env.MONGO_URI ||‘mongodb://localhost:27017/mern-todo’;

Define the Todo model (server/models/Todo.js):

const mongoose = require('mongoose');
const todoSchema = new mongoose.Schema({
  text: {
    type: String,
    required: true
  },
  completed: {
    type: Boolean,
    default: false
  }
});
module.exports = mongoose.model('Todo', todoSchema);

Create routes for the Todo API (server/routes/todo.js):

const express = require('express');
const router = express.Router();
const Todo = require('../models/Todo');
// Get all todos
router.get('/todos', async (req, res) => {
  try {
    const todos = await Todo.find();
    res.json(todos);
  } catch (err) {
    res.status(500).json({ message: err.message });
  }
});

// Create a new todo
router.post('/todos', async (req, res) => {
  const todo = new Todo({
    text: req.body.text
  });
  try {
    const newTodo = await todo.save();
    res.status(201).json(newTodo);
  } catch (err) {
    res.status(400).json({ message: err.message });
  }
});
// Delete a todo
router.delete('/todos/:id', async (req, res) => {
  try {
    const todo = await Todo.findById(req.params.id);
    if (!todo) {
      return res.status(404).json({ message: 'Todo not found' });
    }
    await todo.remove();
    res.json({ message: 'Todo deleted' });
  } catch (err) {
    res.status(500).json({ message: err.message });
  }
});
module.exports = router;

Step 3: Creating the Frontend with React
Initialize the frontend:

npx create-react-app client
cd client

Create a Todo List component (client/src/TodoList.js):

import React, { useState, useEffect } from 'react';
import axios from 'axios';
const TodoList = () => {
  const [todos, setTodos] = useState([]);
  const [newTodo, setNewTodo] = useState('');
  useEffect(() => {
    fetchTodos();
  }, []);
  const fetchTodos = async () => {
    try {
      const res = await axios.get('http://localhost:5000/api/todos');
      setTodos(res.data);
    } catch (err) {
      console.error(err);
    }
  };
  const createTodo = async () => {
    try {
      const res = await axios.post('http://localhost:5000/api/todos', { text: newTodo });
      setTodos([...todos, res.data]);
      setNewTodo('');
    } catch (err) {
      console.error(err);
    }
  };
  const deleteTodo = async (id) => {
    try {
      await axios.delete(`http://localhost:5000/api/todos/${id}`);
      setTodos(todos.filter(todo => todo._id !== id));
    } catch (err) {
      console.error(err);
    }
  };
  return (
    <div>
      <h1>Todo List</h1>
      <input 
        type="text" 
        value={newTodo} 
        onChange={(e) => setNewTodo(e.target.value)} 
      />
      <button onClick={createTodo}>Add Todo</button>
      <ul>
        {todos.map(todo => (
          <li key={todo._id}>
            {todo.text}
            <button onClick={() => deleteTodo(todo._id)}>Delete</button>
          </li>
        ))}
      </ul>
    </div>
  );
};
export default TodoList;

Update client/src/App.js to include the Todo List component:

import React from 'react';
import TodoList from './TodoList';
function App() {
  return (
    <div className="App">
      <TodoList />
    </div>
  );
}
export default App;

Step 4: Dockerizing the Application
Create Dockerfiles
Backend Dockerfile (server/Dockerfile):

# Use the official Node.js image as the base image
FROM node:14
# Create and set the working directory
WORKDIR /usr/src/app
# Copy package.json and package-lock.json to the working directory
COPY package*.json ./
# Install dependencies
RUN npm install
# Copy the rest of the application code
COPY . .
# Expose the port the app runs on
EXPOSE 5000
# Start the application
CMD ["node", "server.js"]

Frontend Dockerfile (client/Dockerfile):

# Use the official Node.js image as the base image
FROM node:14
# Create and set the working directory
WORKDIR /usr/src/app
# Copy package.json and package-lock.json to the working directory
COPY package*.json ./
# Install dependencies
RUN npm install
# Copy the rest of the application code
COPY . .
# Build the React app
RUN npm run build
# Install serve to serve the production build
RUN npm install -g serve
# Expose the port the app runs on
EXPOSE 3000
# Start the application
CMD ["serve", "-s", "build"]
Create Docker Compose File (docker-compose.yml)
version: '3.8'
services:
  backend:
    build:
      context: ./server
      dockerfile: Dockerfile
    ports:
      - "5000:5000"
    environment:
      - NODE_ENV=development
      - MONGO_URI=mongodb://mongo:27017/mern-todo
    depends_on:
      - mongo
  frontend:
    build:
      context: ./client
      dockerfile: Dockerfile
    ports:
      - "3000:3000"
    depends_on:
      - backend
  mongo:
    image: mongo:4.2
    ports:
      - "27017:27017"
    volumes:
      - mongo-data:/data/db
volumes:
  mongo-data:

Step 5: Build and Run Docker Containers
From the root directory of your project, run the following commands to build and start your Docker containers:

docker-compose build
docker-compose up

Step 6: Access Your Application
Frontend: Open your browser and navigate to http://localhost:3000
Backend: The backend API will be running at http://localhost:5000
Conclusion
By following these steps, you have successfully built a MERN stack Todo application and dockerized it for easy deployment. Dockerizing your application ensures that it runs consistently across different environments and simplifies the deployment process. Happy coding!

code repo:github.com/PRATIKNALAWADE/Docker-mern

How to Setup Passwordless SSH on Linux

Pratik Nalawade — Mon, 05 Aug 2024 12:23:40 +0000

Linux fundamentals: How to Setup Passwordless SSH on Linux
In the realm of server management, secure access is paramount. SSH, or Secure Shell, stands as the cornerstone protocol for remote server administration, offering a robust and encrypted communication channel. However, traditional password-based authentication methods present challenges in terms of security and convenience.

Enter passwordless SSH — a paradigm shift in authentication that streamlines access while fortifying security. In this comprehensive guide, we’ll explore the intricacies of setting up passwordless SSH on Linux-based systems like Ubuntu and CentOS, empowering you to elevate your server management practices.

Advantages of Passwordless SSH:
Before delving into implementation, let’s examine why passwordless SSH is gaining traction among sysadmins and DevOps professionals alike. Passwordless SSH, also known as public key-based authentication, offers several compelling advantages:

Enhanced Security: By leveraging public-private key cryptography, passwordless SSH eliminates the vulnerabilities associated with password-based authentication, fortifying your server against brute-force attacks and unauthorized access attempts.
Streamlined Access: With passwordless SSH, users enjoy seamless and non-interactive login experiences, eliminating the need to repeatedly enter passwords for each session. This enhances productivity and simplifies remote server management tasks.
Robust Authentication: Public key-based authentication provides a more robust and reliable means of verifying user identities, fostering a foundation for stringent authentication and authorization policies.

Now, let’s embark on the journey of setting up passwordless SSH on your Linux server, exploring three distinct methods for implementation.

Method 1: Leveraging ssh-copy-id for Effortless Key Distribution
The ssh-copy-id command simplifies the process of distributing your public key to remote servers, automating the appending of the key to the authorized_keys file. Follow these steps to utilize ssh-copy-id:

Step 1: Generate a Public-Private Key Pair: Use the ssh-keygen command to generate a key pair on your local machine.
Step 2: Copy the Public Key: Execute ssh-copy-id remote_username@remote_IP_Address, and authenticate with the remote server’s password when prompted.
Step 3: Verify Connectivity: Attempt to SSH into the remote server — if successful, you’ve configured passwordless SSH using ssh-copy-id.

Method 2: SSH-Based Key Distribution for Flexibility and Control
For scenarios where ssh-copy-id isn’t available, SSH-based key distribution offers a viable alternative. Follow these steps to distribute your public key using SSH:

Step 1: Generate a Public-Private Key Pair: Utilize ssh-keygen to generate a key pair on your local machine.
Step 2: Copy the Public Key: Execute the command cat ~/.ssh/id_rsa.pub | ssh remote_username@remote_ip_address “mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys”.
Step 3: Validate Configuration: Attempt to SSH into the remote server — successful login indicates successful passwordless SSH setup using SSH-based key distribution.

Method 3: Manual Key Insertion for Unyielding Situations
In scenarios where automated methods fail, manual key insertion provides a fail-safe approach. Follow these steps to manually insert your public key:

Step 1: Generate a Public-Private Key Pair: Create a key pair using ssh-keygen on your local machine.
Step 2: Copy the Public Key: Display the contents of the id_rsa.pub file using cat ~/.ssh/id_rsa.pub, and manually append the key to the remote server’s authorized_keys file.
Step 3: Secure Permissions: Ensure proper permissions are set for the .ssh directory and authorized_keys file using chmod, and validate the configuration by attempting to SSH into the remote server.

Conclusion:
In conclusion, passwordless SSH represents a paradigm shift in secure server access, offering enhanced security, streamlined access, and robust authentication mechanisms. By mastering the methods outlined in this guide, you can elevate your server management practices and fortify your infrastructure against potential security threats. Embrace the power of passwordless SSH and unlock a new realm of efficiency and security in your server management endeavors.

Mastering Git for DevOps: A Comprehensive Guide

Pratik Nalawade — Sat, 03 Aug 2024 06:08:36 +0000

Welcome to our tech blog focused on mastering Git for DevOps interviews. In this guide, I’ll cover everything from the basics of Git to advanced topics, ensuring you’re well-prepared to ace any Git-related questions during your DevOps interview.

Understanding Version Control Systems

Difference between CVCS and DVCS

CVCS (Centralized Version Control System): In CVCS, there’s a central server that stores all versions of a project’s files. Users check out files from this central repository to work on them and then check them back in when done.
DVCS (Distributed Version Control System): Unlike CVCS, DVCS does not necessarily rely on a central server. Each user has a complete copy (clone) of the repository, including its full history. This allows for offline work and more flexibility in collaboration.
Importance of Git
Git has become the de facto standard for version control in modern software development due to its numerous advantages:

Distributed nature: Allows for flexible and decentralized workflows.
Branching and merging: Facilitates parallel development and easy integration of features.
Speed: Git is incredibly fast, making it ideal for both small and large projects.
Data integrity: Uses cryptographic hashing to ensure the integrity of your data.
Large community and ecosystem: There’s extensive documentation, support, and a plethora of third-party tools and integrations available.

Git Three-stage Architecture

Git has three main stages:

1.Working Directory: The directory where you do all your work.
2.Staging Area (Index): Acts as a buffer between the working directory and the repository. Files in the staging area are ready to be committed.
3.Repository (HEAD): The repository stores all committed changes and their history.
Understanding Key Git Concepts
Repository, Commit, Tags, Snapshots
Repository: A Git repository is a directory that contains all the files and folders of a project, along with metadata and version history.
Commit: A commit represents a snapshot of changes to the repository at a particular point in time.
Tags: Tags are pointers to specific commits, often used to mark release versions.
Snapshots: Git uses snapshots to store changes to files over time, rather than storing the changes themselves.
Push-Pull Mechanism and Branching Strategy
Push-Pull Mechanism: Git uses the push and pull commands to synchronize changes between your local repository and a remote repository.
push => code changes uploaded into remote repository
pull => code changes downloaded into local repository

Branching Strategy: Git’s branching model allows for parallel development by creating separate branches for features, bug fixes, etc. Common strategies include GitFlow and GitHub Flow.

Advanced Git Operations

Working with Git Stash and Git Pop
Git Stash: Stashing allows you to temporarily shelve changes in your working directory, allowing you to switch to a different branch or work on a different task.
Git Pop: Pop applies the most recently stashed changes back to your working directory.
Use Case: Suppose you’re working on a feature branch and need to quickly switch to another branch to address an urgent bug. However, your changes are not ready for commit. Here, Git stash comes in handy to temporarily store your changes.

example:


# Stash changes
git stash

# Switch to another branch
git checkout bug-fix

# Make necessary changes and commit

# Switch back to the feature branch
git checkout feature-branch

# Apply stashed changes
git stash pop

Resolving Merge Conflicts in Git
Merge conflicts occur when Git is unable to automatically merge changes from different branches. To resolve conflicts, you need to manually edit the conflicting files to incorporate the desired changes.

Use Case: When you merge branches and Git encounters conflicting changes, it pauses the merge process and prompts you to resolve the conflicts manually.

example:

# Switch to the branch you want to merge into
git checkout main

Merge the feature branch into main

git merge feature-branch
When conflicts occur, Git will display the conflicted files. Open the conflicted files in your code editor and resolve the conflicts. After resolving conflicts

# Add the resolved files
git add

Commit the merge

git commit -m "Merge feature-branch into main, resolved conflicts"
Git Revert and Reset (Reset vs Revert)
Git Revert: Revert undoes a previous commit by creating a new commit that undoes the changes introduced by the original commit.
Git Reset: Reset moves the HEAD and current branch pointer to a specified commit, optionally modifying the working directory and staging area.
Use Case:

Git Revert: Let’s say you’ve pushed a commit containing a bug, and you want to undo the changes introduced by that commit without altering the commit history. Revert creates a new commit that undoes the changes.
Git Reset: Suppose you’ve made local commits that you want to discard entirely, resetting the branch to a previous state. Reset moves the HEAD and branch pointers to a specified commit.
Syntax:

Revert a commit

git revert

Reset to a previous commit

git reset [--soft | --mixed | --hard]
Working with Git Squash
Squashing commits combines multiple commits into a single commit. This is often done to clean up the commit history before merging a feature branch into the main branch.

Use Case: Before merging a feature branch into the main branch, you want to clean up the commit history by combining multiple small, related commits into a single, meaningful commit.

example:

Start an interactive rebase

git rebase -i HEAD~3 # Squash last 3 commits

In the interactive rebase editor, change 'pick' to 'squash' for commits you want to squash

What is Git Fork?
A Git fork is a copy of a repository that allows you to freely experiment with changes without affecting the original repository. Forks are commonly used in open-source collaboration on platforms like GitHub.

git rebase, git merge, and git cherry-pick:

Git Merge Use Case:

Merge is a straightforward way to integrate changes from one branch into another.
It creates a new commit that combines the commit histories of the merged branches, preserving the history of each branch.
Syntax:

Merge feature-branch into main

git checkout main
git merge feature-branch
Pros:

Preserves the history of both branches.
Easy to understand and use.
Cons:

Can result in a cluttered commit history, especially in long-running projects with many branches.

Git Rebase Use Case:

Rebase is used to incorporate changes from one branch into another by reapplying commits on top of another branch.
It creates a linear history, making the commit history cleaner and easier to follow.
Syntax:

Rebase feature-branch onto main

git checkout feature-branch
git rebase main
Pros:

Creates a cleaner, linear commit history.
Simplifies the process of integrating changes from feature branches into the main branch.
Cons:

Rewrites commit history, which can lead to conflicts and potential loss of work if not used carefully.
Should not be used on shared branches as it alters commit history.

Git Cherry-pick Use Case:

Cherry-pick allows you to select specific commits from one branch and apply them to another branch.
Useful for applying individual commits, such as bug fixes or specific features, to another branch without merging the entire branch.
Syntax:

Cherry-pick a commit from feature-branch into main

git checkout main
git cherry-pick
Pros:

Allows for selective integration of commits.
Useful for applying critical bug fixes or specific features to release branches.
Cons:

Can lead to disjointed commit histories if used excessively.
May introduce conflicts that need to be resolved manually.
Choosing the Right Approach
Merge: Use when you want to preserve the commit history of both branches and don’t mind a potentially cluttered history.
Rebase: Ideal for creating a clean, linear commit history and integrating changes from feature branches into the main branch.
Cherry-pick: Useful for selectively applying individual commits to another branch without merging the entire branch.
Each approach has its strengths and weaknesses, so it’s essential to understand when to use each one based on your project’s needs and collaboration workflow.

Git Integration on VScode, Git Authentication with GitHub via SSH and HTTPS Protocol
Visual Studio Code (VS Code) has built-in Git integration, allowing you to perform most Git operations directly within the editor. You can authenticate with GitHub using SSH keys or HTTPS credentials for secure access to repositories.

GitHub Integration and Collaboration
GitHub Introduction, Creating Repositories, PR’s
GitHub is a popular platform for hosting Git repositories and collaborating on projects. You can create repositories, open pull requests (PRs) to propose changes, review code, and merge changes into the main branch.

Thus, with this comprehensive guide, you should now have a solid understanding of Git, from basic version control concepts to advanced operations and collaboration on platforms like GitHub.

Networking fundamental concepts for DevOps

Pratik Nalawade — Fri, 02 Aug 2024 19:48:57 +0000

Relevance of Networking to DevOps
DevOps champions collaboration, automation, and efficient delivery of software. Networking plays a big role in achieving these objectives by enabling various aspects of a DevOps environment:

Infrastructure Provisioning: Networking knowledge is essential for configuring and managing virtualized environments, such as cloud platforms or on-premises data centers. Understanding networking concepts helps in setting up and connecting different resources effectively.
Scalability and Performance: DevOps teams strive to create scalable and high-performance systems. Networking skills enable engineers to design and optimize network architecture, load balancing, and traffic management to meet these requirements.
Security and Monitoring: Networking forms the foundation for implementing robust security measures and monitoring solutions. Concepts like firewalls, VPNs, and network monitoring tools are essential for ensuring the safety and reliability of systems.
Continuous Integration and Deployment: Networking expertise allows DevOps professionals to design efficient continuous integration and deployment (CI/CD) pipelines. This involves configuring network connectivity between various components involved in the delivery process.
IPv4 addressing subnetting, and CIDR + subnet masking
CIDR (Classless Inter-Domain Routing) is a way to efficiently manage and allocate IP addresses in an IPv4 network. It combines the concept of IP addresses with a subnet mask into a single notation, making it easier to understand how many usable IP addresses a specific network segment can hold.

Here’s a breakdown of CIDR subnetting:

IP Addresses:

An IPv4 address consists of 32 bits, typically represented in four decimal octets separated by dots (e.g., 192.168.1.0).
Each octet can have a value between 0 and 255.
Subnet Mask:

Traditionally, subnetting involved using a subnet mask (also represented in four octets) to divide the IP address into two parts:
Network address: Identifies the network itself and cannot be assigned to a device.
Host address: Identifies individual devices within the network.
The subnet mask defines the boundary between the network and host portions using ones (1) for the network part and zeros (0) for the host part. For example, a subnet mask of 255.255.255.0 allocates the first three octets (24 bits) for the network and the last octet (8 bits) for the host addresses.
CIDR Notation:

CIDR simplifies subnet masking by expressing the number of network bits directly in the IP address notation. It uses a forward slash (/) followed by the number of bits dedicated to the network portion.
For example, 192.168.1.0/24 is equivalent to the combination of the IP address 192.168.1.0 and a subnet mask of 255.255.255.0 (both indicate the first 24 bits define the network).
Benefits of CIDR Subnetting:

Efficient IP Address Allocation: CIDR allows for creating subnets with varying sizes depending on the number of devices needed. This avoids wasting a large pool of addresses in a small network or cramming too many devices into a limited address space.
Hierarchical Routing: CIDR simplifies inter-domain routing by allowing routers to identify the network portion of an IP address quickly, enabling efficient routing of traffic.
Flexibility: CIDR offers more flexibility for network administrators to design subnets that match specific needs.
Calculating Usable IP Addresses:

Check out cidr.xyz for interactive session
With CIDR notation, you can easily calculate the number of usable IP addresses within a subnet. Here’s the formula:

Usable IP addresses = 2^(number of host bits) — 2 (exclude network and broadcast addresses)
For example, in the subnet 10.0.1.0/24 (24 network bits), there are 8 host bits (32 total bits — 24 network bits). So, the usable IP addresses would be 2⁸ — 2 = 254 (excluding the network address 10.0.1.0 and the broadcast address 10.0.1.255).

Summary:

CIDR subnetting is a fundamental concept in network design and management. By understanding how CIDR notation works, you can effectively allocate IP addresses within your network, ensuring efficient resource utilization and proper network communication.

video explaining the same:

https://www.youtube.com/watch?v=aPW-ZAo09Pg

OSI Layer

The OSI (Open Systems Interconnection) model is a conceptual framework that defines network communication into seven distinct layers. Each layer has specific functionalities and protocols that DevOps engineers should be familiar with to troubleshoot network issues, manage infrastructure, and ensure efficient communication within their systems. Here’s a breakdown of the 7 layers and relevant protocols from a DevOps perspective:

Layer 1: Physical Layer

Function: Deals with the physical transmission of raw data bits across a physical medium like cables or wireless signals.
DevOps Relevance: Understanding physical connectivity issues like faulty cables or incorrect port configurations is crucial for troubleshooting basic network connectivity problems.
Layer 2: Data Link Layer

Function: Handles error detection and correction at the frame level (groups of data bits).
Important Protocol: Ethernet (Most common LAN protocol for error-free data transmission on wired networks)
DevOps Relevance: Knowledge of Ethernet and troubleshooting tools like ping can help identify issues with network devices like switches and network interface cards (NICs).
Layer 3: Network Layer

Function: Responsible for routing packets (datagrams) across networks based on IP addresses.
Important Protocol: IP (Internet Protocol) — Defines the addressing scheme (IPv4 or IPv6) for identifying devices on a network.
DevOps Relevance: Understanding IP addressing and subnetting is essential for configuring network devices, managing cloud resources (VPCs), and troubleshooting routing issues.
Layer 4: Transport Layer

Function: Provides reliable or unreliable data transfer between applications on different devices.
Important Protocols:
TCP (Transmission Control Protocol): Ensures reliable, in-order delivery of data with error checking and retransmission.
UDP (User Datagram Protocol): Provides connectionless, best-effort data delivery suitable for real-time applications like video streaming.
DevOps Relevance: Understanding TCP and UDP is crucial for troubleshooting application communication issues and choosing the appropriate protocol for different DevOps tools and deployments.
Layer 5: Session Layer

Function: Establishes, manages, and terminates sessions between communicating applications.
Important Protocol: SSH (Secure Shell) — Provides secure remote access and communication for DevOps engineers to manage servers and infrastructure.
DevOps Relevance: SSH is a fundamental tool for DevOps engineers to access and manage remote systems securely.
Layer 6: Presentation Layer

Function: Deals with data format and encryption/decryption before transmission and after reception.
Important Protocol: HTTPS (Hypertext Transfer Protocol Secure) — Encrypts communication between web servers and clients, protecting data integrity.
DevOps Relevance: Understanding protocols like HTTPS is essential for securing application communication and ensuring data privacy.
Layer 7: Application Layer

Function: Provides network services directly to user applications.
Important Protocols:
HTTP (Hypertext Transfer Protocol): The foundation of web communication, used for data exchange between web browsers and servers.
DNS (Domain Name System): Translates human-readable domain names (like [invalid URL removed]) into machine-readable IP addresses.
FTP (File Transfer Protocol): Used for transferring files between computers on a network.
DevOps Relevance: Familiarity with application layer protocols is essential for DevOps engineers to deploy and manage web applications, configure load balancers, and troubleshoot application communication issues.
In summary:

The OSI model provides a framework for understanding network communication. While not a strict implementation in modern networks, each layer plays a role, and the associated protocols are crucial for DevOps engineers to manage infrastructure, troubleshoot network issues, and ensure efficient communication within their systems.

TCP is a connection-oriented protocol. This means that it first establishes a link between the source and destination before it sends data. Once the connection has been made, then TCP breaks down large data sets into smaller packets, sends them along the connection, and ensures data integrity throughout the entire process. TCP is a preferred protocol when data integrity is critical, such as in any transactional system.

UDP in turn is not connection-oriented. UDP starts transmitting data immediately, without waiting for connection confirmation from the receiving side. Even though some data loss can happen, UDP is most often used in cases where speed is more important than perfect transmission, such as in voice or video streaming

Domain Name System (DNS)

Client initiates a query to a Recursive Resolver.

The Recursive Resolver connects to a Root Server.
The Root Nameserver then responds to the resolver with the address of a Top Level Domain (TLD) Server (such as .com or .net)
The Root Resolver makes a request to the TLD Server.
The TLD Server returns the IP address of the Domain Nameserver, which stores the information about the requested domain.
The Recursive Resolver sends a query to the Domain Nameserver.
The IP address for the requested domain is returned to the Recursive Resolver from the Domain Nameserver.
The Recursive Resolver provides the client with the IP address of the requested domain. DNS record types DNS records, also known as zone files, provide information about a domain. This includes the IP address that is associated with this domain and how to handle queries for it. Each DNS record has a time-to-live setting (TTL) which indicates how often a DNS server will refresh it.

Below are the most commonly used types of DNS records and their meaning.

HTTP
HTTP Methods
You’ve probably heard about Hypertext Transfer Protocol, also known as HTTP. HTTP allows you to interact with Web pages, HTML documents, and APIs. It is the foundation of any data exchange on the Internet

There are 7 main HTTP request methods:

There are 4 categories of HTTP responses:

200s: Successful responses
300s: Redirects
400s: Client errors
500s: Server errors
HTTP Headers
HTTP headers allow the client to add additional information to a request for purposes such as authentication, caching, and specifying the type of client device sending the request.
Headers fall into 4 general contexts:

General Header: A header that works for both response and requests messages.
Request Header: A header that only applies to request messages from a client.
Response Header: A header that only applies to responses from a server.
Entity Header: A header that gives information about the entity itself or the resource requested.
Network Troubleshooting Tools
DevOps engineers rely on various tools to diagnose and troubleshoot network connectivity issues, ensuring optimal performance and reliability within their systems. Here are some essential network troubleshooting tools commonly used by DevOps professionals:

Basic Command-Line Tools:

ping: The most fundamental tool used to test basic network connectivity between devices by sending echo requests and waiting for responses. It helps identify if a specific device is reachable on the network.
traceroute/tracert: Reveals the path that data packets take to reach a destination. This helps pinpoint where potential network delays or outages might be occurring along the route.
netstat: Provides information about network connections, routing tables, and network interface statistics. Useful for identifying active connections, listening ports, and potential bottlenecks.
nslookup: Looks up information about domain names, including translating them into corresponding IP addresses. This helps verify DNS resolution functionality.
Advanced Command-Line Tools:

nmap: A powerful network scanner that can identify devices on a network, operating systems, open ports, and services running on those ports. This aids in comprehensive network security assessments and vulnerability discovery.
tcpdump/Wireshark: Packet capture and analysis tools. They capture network traffic on a specific interface, allowing you to inspect individual data packets and analyze their contents to diagnose communication issues or identify security threats.
curl: A command-line tool for transferring data from or to servers. It can be used to test web server functionality and diagnose HTTP communication problems.

Python for devops

Pratik Nalawade — Wed, 31 Jul 2024 14:27:37 +0000

Here are some important Python modules used for DevOps automation:

os module: The os module provides a way to interact with the operating system, including file operations, process management, and system information.

Requests and urllib3 modules: The Requests and urllib3 modules are used to send HTTP requests and handle HTTP responses.

Logging module: The logging module provides a way to log messages from Python applications.

boto3 module: The boto3 module provides an interface to the Amazon Web Services (AWS) SDK for Python.

paramiko module : The paramiko module is a Python implementation of SSH protocol, used for secure remote connections.

JSON module : The JSON module is used to encode and decode JSON data.

PyYAML module : The PyYAML module provides a way to parse and generate YAML data.

pandas module: The pandas module provides data analysis tools, including data manipulation and data visualization.

smtplib module: The smtplib module provides a way to send email messages from Python applications.

Python Use Cases in DevOps

1.Automation of Infrastructure Provisioning

Tooling: AWS Boto3, Azure SDK, Terraform, Ansible
Example: Automating the creation and management of cloud resources such as EC2 instances, S3 buckets, and RDS databases. Python scripts can use the AWS Boto3 library to manage AWS resources programmatically.

example code:

import boto3

def lambda_handler(event, context):
    ec2 = boto3.client('ec2')

    # Get all EBS snapshots
    response = ec2.describe_snapshots(OwnerIds=['self'])

    # Get all active EC2 instance IDs
    instances_response = ec2.describe_instances(Filters=[{'Name': 'instance-state-name', 'Values': ['running']}])
    active_instance_ids = set()

    for reservation in instances_response['Reservations']:
        for instance in reservation['Instances']:
            active_instance_ids.add(instance['InstanceId'])

    # Iterate through each snapshot and delete if it's not attached to any volume or the volume is not attached to a running instance
    for snapshot in response['Snapshots']:
        snapshot_id = snapshot['SnapshotId']
        volume_id = snapshot.get('VolumeId')

        if not volume_id:
            # Delete the snapshot if it's not attached to any volume
            ec2.delete_snapshot(SnapshotId=snapshot_id)
            print(f"Deleted EBS snapshot {snapshot_id} as it was not attached to any volume.")
        else:
            # Check if the volume still exists
            try:
                volume_response = ec2.describe_volumes(VolumeIds=[volume_id])
                if not volume_response['Volumes'][0]['Attachments']:
                    ec2.delete_snapshot(SnapshotId=snapshot_id)
                    print(f"Deleted EBS snapshot {snapshot_id} as it was taken from a volume not attached to any running instance.")
            except ec2.exceptions.ClientError as e:
                if e.response['Error']['Code'] == 'InvalidVolume.NotFound':
                    # The volume associated with the snapshot is not found (it might have been deleted)
                    ec2.delete_snapshot(SnapshotId=snapshot_id)
                    print(f"Deleted EBS snapshot {snapshot_id} as its associated volume was not found.")

Lambda function is designed to clean up unused EBS snapshots. Here's a brief rundown of what it does:

Describe Snapshots: Retrieves all EBS snapshots owned by the account.
Describe Instances: Retrieves all running EC2 instances.
Iterate through Snapshots: For each snapshot, checks if it's attached to a volume.
If the snapshot is not attached to any volume, it deletes the snapshot.
If the snapshot is attached to a volume, it checks if the volume is attached to any running instance.
If the volume is not attached to any running instance or the volume does not exist, it deletes the snapshot.
repo:https://github.com/PRATIKNALAWADE/AWS-Cost-Optimization/blob/main/ebs_snapshots.py

2.Use Case: Automating CI/CD Pipelines with Python

In a CI/CD pipeline, automation is key to ensuring that code changes are built, tested, and deployed consistently and reliably. Python can be used to interact with CI/CD tools like Jenkins, GitLab CI, or CircleCI, either by triggering jobs, handling webhook events, or interacting with various APIs to deploy applications.

Below is an example of how you can use Python to automate certain aspects of a CI/CD pipeline using Jenkins.

Example: Triggering Jenkins Jobs with Python

Scenario:
You have a Python script that needs to trigger a Jenkins job whenever a new commit is pushed to the main branch of a GitHub repository. The script will also pass some parameters to the Jenkins job, such as the Git commit ID and the branch name.

Step 1: Set Up Jenkins Job

First, ensure that you have a Jenkins job configured to accept parameters. You will need the job name, Jenkins URL, and an API token for authentication.

Step 2: Writing the Python Script

Below is a Python script that triggers the Jenkins job with specific parameters:

import requests
import json

# Jenkins server details
jenkins_url = 'http://your-jenkins-server.com'
job_name = 'your-job-name'
username = 'your-username'
api_token = 'your-api-token'

# Parameters to pass to the Jenkins job
branch_name = 'main'
commit_id = 'abc1234def5678'

# Construct the job URL
job_url = f'{jenkins_url}/job/{job_name}/buildWithParameters'

# Define the parameters to pass
params = {
    'BRANCH_NAME': branch_name,
    'COMMIT_ID': commit_id
}

# Trigger the Jenkins job
response = requests.post(job_url, auth=(username, api_token), params=params)

# Check the response
if response.status_code == 201:
    print('Jenkins job triggered successfully.')
else:
    print(f'Failed to trigger Jenkins job: {response.status_code}, {response.text}')

Step 3: Explanation

Jenkins Details:
- jenkins_url: URL of your Jenkins server.
- job_name: The name of the Jenkins job you want to trigger.
- username and api_token: Your Jenkins credentials for authentication.
Parameters:
- branch_name and commit_id are examples of parameters that the Jenkins job will use. These could be passed dynamically based on your CI/CD workflow.
Requests Library:
- The script uses Python's requests library to make a POST request to the Jenkins server to trigger the job.
- auth=(username, api_token) is used to authenticate with the Jenkins API.
Response Handling:
- If the job is triggered successfully, Jenkins responds with a 201 status code, which the script checks to confirm success.

Step 4: Integrate with GitHub Webhooks

To trigger this Python script automatically whenever a new commit is pushed to the main branch, you can configure a GitHub webhook that sends a POST request to your server (where this Python script is running) whenever a push event occurs.

GitHub Webhook Configuration:
1. Go to your GitHub repository settings.
2. Under "Webhooks," click "Add webhook."
3. Set the "Payload URL" to the URL of your server that runs the Python script.
4. Choose application/json as the content type.
5. Set the events to listen for (e.g., push events).
6. Save the webhook.
Handling the Webhook:
- You may need to set up a simple HTTP server using Flask, FastAPI, or a similar framework to handle the incoming webhook requests from GitHub and trigger the Jenkins job accordingly.

from flask import Flask, request, jsonify
import requests

app = Flask(__name__)

# Jenkins server details
jenkins_url = 'http://your-jenkins-server.com'
job_name = 'your-job-name'
username = 'your-username'
api_token = 'your-api-token'

@app.route('/webhook', methods=['POST'])
def github_webhook():
    payload = request.json

    # Extract branch name and commit ID from the payload
    branch_name = payload['ref'].split('/')[-1]  # Get the branch name
    commit_id = payload['after']

    # Only trigger the job if it's the main branch
    if branch_name == 'main':
        job_url = f'{jenkins_url}/job/{job_name}/buildWithParameters'
        params = {
            'BRANCH_NAME': branch_name,
            'COMMIT_ID': commit_id
        }

        response = requests.post(job_url, auth=(username, api_token), params=params)

        if response.status_code == 201:
            return jsonify({'message': 'Jenkins job triggered successfully.'}), 201
        else:
            return jsonify({'message': 'Failed to trigger Jenkins job.'}), response.status_code

    return jsonify({'message': 'No action taken.'}), 200

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)

Step 5: Deploying the Flask App

Deploy this Flask app on a server and ensure it is accessible via the public internet, so GitHub's webhook can send data to it.

Conclusion

This example illustrates how Python can be integrated into a CI/CD pipeline, interacting with tools like Jenkins to automate essential tasks.

3.Configuration Management and Orchestration

Tooling: Ansible, Chef, Puppet
Example: Using Python scripts with Ansible to manage the configuration of servers. Scripts can be used to ensure that all servers are configured consistently and to manage complex deployments that require orchestration of multiple services.

In this example, we'll use Python to manage server configurations with Ansible. The script will run Ansible playbooks to ensure servers are configured consistently and orchestrate the deployment of multiple services.

Example: Automating Server Configuration with Ansible and Python

Scenario:
You need to configure a set of servers to ensure they have the latest version of a web application, along with necessary dependencies and configurations. You want to use Ansible for configuration management and Python to trigger and manage Ansible playbooks.

Step 1: Create Ansible Playbooks

playbooks/setup.yml:
This Ansible playbook installs necessary packages and configures the web server.

---
- name: Configure web servers
  hosts: web_servers
  become: yes
  tasks:
    - name: Install nginx
      apt:
        name: nginx
        state: present

    - name: Deploy web application
      copy:
        src: /path/to/local/webapp
        dest: /var/www/html/webapp
        owner: www-data
        group: www-data
        mode: '0644'

    - name: Ensure nginx is running
      service:
        name: nginx
        state: started
        enabled: yes

inventory/hosts:
Define your servers in the Ansible inventory file.

[web_servers]
server1.example.com
server2.example.com

Step 2: Write the Python Script

The Python script will use the subprocess module to run Ansible commands and manage playbook execution.

import subprocess

def run_ansible_playbook(playbook_path, inventory_path):
    """
    Run an Ansible playbook using the subprocess module.

    :param playbook_path: Path to the Ansible playbook file.
    :param inventory_path: Path to the Ansible inventory file.
    :return: None
    """
    try:
        result = subprocess.run(
            ['ansible-playbook', '-i', inventory_path, playbook_path],
            check=True,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE,
            text=True
        )
        print('Ansible playbook executed successfully.')
        print(result.stdout)
    except subprocess.CalledProcessError as e:
        print('Ansible playbook execution failed.')
        print(e.stderr)

if __name__ == '__main__':
    # Paths to the playbook and inventory files
    playbook_path = 'playbooks/setup.yml'
    inventory_path = 'inventory/hosts'

    # Run the Ansible playbook
    run_ansible_playbook(playbook_path, inventory_path)

Step 3: Explanation

Ansible Playbook (setup.yml):
- Tasks: This playbook installs Nginx, deploys the web application, and ensures Nginx is running.
- Hosts: web_servers is a group defined in the inventory file.
Inventory File (hosts):
- Groups: Defines which servers are part of the web_servers group.
Python Script (run_ansible_playbook function):
- subprocess.run: Executes the ansible-playbook command to apply configurations defined in the playbook.
- Error Handling: Catches and prints errors if the playbook execution fails.

Step 4: Running the Script

Make sure Ansible is installed on the system where the Python script is running.
Ensure the ansible-playbook command is accessible in the system PATH.
Execute the Python script to apply the Ansible configurations:

python3 your_script_name.py

Step 5: Advanced Use Cases

Dynamic Inventory: Use Python to generate dynamic inventory files based on real-time data from a database or an API.
Role-based Configurations: Define more complex configurations using Ansible roles and use Python to manage role-based deployments.
Notifications and Logging: Extend the Python script to send notifications (e.g., via email or Slack) or log detailed information about the playbook execution.

Conclusion

By integrating Python with Ansible, you can automate server configuration and orchestration tasks efficiently. Python scripts can manage and trigger Ansible playbooks, ensuring that server configurations are consistent and deployments are orchestrated seamlessly.

4 Monitoring and Alerting with Python

In a modern monitoring setup, you often need to collect metrics and logs from various services, analyze them, and push them to monitoring systems like Prometheus or Elasticsearch. Python can be used to gather and process this data, and set up automated alerts based on specific conditions.

Example: Collecting Metrics and Logs, and Setting Up Alerts

1. Collecting Metrics and Logs

Scenario:
You want to collect custom metrics and logs from your application and push them to Prometheus and Elasticsearch. Additionally, you'll set up automated alerts based on specific conditions.

Step 1: Collecting Metrics with Python and Prometheus

To collect and expose custom metrics from your application, you can use the prometheus_client library in Python.

Install prometheus_client:

pip install prometheus_client

Python Script to Expose Metrics (metrics_server.py):

from prometheus_client import start_http_server, Gauge
import random
import time

# Create a metric to track the number of requests
REQUESTS = Gauge('app_requests_total', 'Total number of requests processed by the application')

def process_request():
    """Simulate processing a request."""
    REQUESTS.inc()  # Increment the request count

if __name__ == '__main__':
    # Start up the server to expose metrics
    start_http_server(8000)  # Metrics will be available at http://localhost:8000/metrics

    # Simulate processing requests
    while True:
        process_request()
        time.sleep(random.uniform(0.5, 1.5))  # Simulate random request intervals

Step 2: Collecting Logs with Python and Elasticsearch

To push logs to Elasticsearch, you can use the elasticsearch Python client.

Install elasticsearch:

pip install elasticsearch

Python Script to Send Logs (log_collector.py):

from elasticsearch import Elasticsearch
import logging
import time

# Elasticsearch client setup
es = Elasticsearch([{'host': 'localhost', 'port': 9200}])
index_name = 'application-logs'

# Configure Python logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger('log_collector')

def log_message(message):
    """Log a message and send it to Elasticsearch."""
    logger.info(message)
    es.index(index=index_name, body={'message': message, 'timestamp': time.time()})

if __name__ == '__main__':
    while True:
        log_message('This is a sample log message.')
        time.sleep(5)  # Log every 5 seconds

Step 3: Setting Up Alerts

To set up alerts, you need to define alerting rules based on the metrics and logs collected. Here’s an example of how you can configure alerts with Prometheus.

Prometheus Alerting Rules (prometheus_rules.yml):

groups:
- name: example_alerts
  rules:
  - alert: HighRequestRate
    expr: rate(app_requests_total[1m]) > 5
    for: 2m
    labels:
      severity: critical
    annotations:
      summary: "High request rate detected"
      description: "Request rate is above 5 requests per minute for the last 2 minutes."

Deploying Alerts:

Update Prometheus Configuration: Ensure that your Prometheus server is configured to load the alerting rules file. Update your prometheus.yml configuration file:

   rule_files:
     - 'prometheus_rules.yml'

Reload Prometheus Configuration: After updating the configuration, reload Prometheus to apply the new rules.

   kill -HUP $(pgrep prometheus)

Grafana Setup:

Add Prometheus as a Data Source:
Go to Grafana's data source settings and add Prometheus.
Create Dashboards:
Create dashboards in Grafana to visualize the metrics exposed by your application. You can set up alerts in Grafana as well, based on the metrics from Prometheus.

Elasticsearch Alerting:

Install Elastic Stack Alerting Plugin:
If you're using Elasticsearch with Kibana, you can use Kibana's alerting features to create alerts based on log data. You can set thresholds and get notifications via email, Slack, or other channels.
Define Alert Conditions:
Use Kibana to define alert conditions based on your log data indices.

Conclusion

By using Python scripts to collect and process metrics and logs, and integrating them with tools like Prometheus and Elasticsearch, you can create a robust monitoring and alerting system. The examples provided show how to expose custom metrics, push logs, and set up alerts for various conditions. This setup ensures you can proactively monitor your application, respond to issues quickly, and maintain system reliability.

5. Use Case: Scripting for Routine Tasks and Maintenance

Routine maintenance tasks like backups, system updates, and log rotation are essential for keeping your infrastructure healthy. You can automate these tasks using Python scripts and schedule them with cron jobs. Below are examples of Python scripts for common routine maintenance tasks and how to set them up with cron.

Example: Python Scripts for Routine Tasks

1. Backup Script

Scenario:
Create a Python script to back up a directory to a backup location. This script will be scheduled to run daily to ensure that your data is regularly backed up.

Backup Script (backup_script.py):

import shutil
import os
from datetime import datetime

# Define source and backup directories
source_dir = '/path/to/source_directory'
backup_dir = '/path/to/backup_directory'

# Create a timestamped backup file name
timestamp = datetime.now().strftime('%Y%m%d-%H%M%S')
backup_file = f'{backup_dir}/backup_{timestamp}.tar.gz'

def create_backup():
    """Create a backup of the source directory."""
    shutil.make_archive(backup_file.replace('.tar.gz', ''), 'gztar', source_dir)
    print(f'Backup created at {backup_file}')

if __name__ == '__main__':
    create_backup()

2. System Update Script

Scenario:
Create a Python script to update the system packages. This script will ensure that the system is kept up-to-date with the latest security patches and updates.

System Update Script (system_update.py):

import subprocess

def update_system():
    """Update the system packages."""
    try:
        subprocess.run(['sudo', 'apt-get', 'update'], check=True)
        subprocess.run(['sudo', 'apt-get', 'upgrade', '-y'], check=True)
        print('System updated successfully.')
    except subprocess.CalledProcessError as e:
        print(f'Failed to update the system: {e}')

if __name__ == '__main__':
    update_system()

3. Log Rotation Script

Scenario:
Create a Python script to rotate log files, moving old logs to an archive directory and compressing them.

Log Rotation Script (log_rotation.py):

import os
import shutil
from datetime import datetime

# Define log directory and archive directory
log_dir = '/path/to/log_directory'
archive_dir = '/path/to/archive_directory'

def rotate_logs():
    """Rotate log files by moving and compressing them."""
    for log_file in os.listdir(log_dir):
        log_path = os.path.join(log_dir, log_file)
        if os.path.isfile(log_path):
            timestamp = datetime.now().strftime('%Y%m%d-%H%M%S')
            archive_file = f'{archive_dir}/{log_file}_{timestamp}.gz'
            shutil.copy(log_path, archive_file)
            shutil.make_archive(archive_file.replace('.gz', ''), 'gztar', root_dir=archive_dir, base_dir=log_file)
            os.remove(log_path)
            print(f'Log rotated: {archive_file}')

if __name__ == '__main__':
    rotate_logs()

Setting Up Cron Jobs

You need to set up cron jobs to schedule these scripts to run at specific intervals. Use the crontab command to edit the cron schedule.

Open the Crontab File:

   crontab -e

Add Cron Job Entries:

Daily Backup at 2 AM:

 0 2 * * * /usr/bin/python3 /path/to/backup_script.py

Weekly System Update on Sunday at 3 AM:

 0 3 * * 0 /usr/bin/python3 /path/to/system_update.py

Log Rotation Every Day at Midnight:

 0 0 * * * /usr/bin/python3 /path/to/log_rotation.py

Explanation:

0 2 * * *: Runs the script at 2:00 AM every day.
0 3 * * 0: Runs the script at 3:00 AM every Sunday.
0 0 * * *: Runs the script at midnight every day.

Conclusion

Using Python scripts for routine tasks and maintenance helps automate critical processes such as backups, system updates, and log rotation. By scheduling these scripts with cron jobs, you ensure that these tasks are performed consistently and without manual intervention. This approach enhances the reliability and stability of your infrastructure, keeping it healthy and up-to-date.

SDLC , Agile vs DevOps

Pratik Nalawade — Tue, 23 Jul 2024 05:13:48 +0000

Software Development Life Cycle (SDLC) and DevOps are both methodologies used in the software development process, but they focus on different aspects and have distinct goals. Here's a comparison:

SDLC (Software Development Life Cycle)

Definition:
- SDLC is a structured process that defines the stages involved in the development of software applications.
- It ensures that the software meets business requirements and is of high quality.
Phases:
- Planning: Define objectives, scope, resources, and schedule.
- Requirement Analysis: Gather and analyze business and user requirements.
- Design: Create the architecture and design of the software.
- Implementation: Write the code according to the design.
- Testing: Test the software for defects and issues.
- Deployment: Release the software to production.
- Maintenance: Provide ongoing support and bug fixes.
Focus:
- Emphasizes detailed planning and linear progression through distinct phases.
- Ensures that each phase is completed before moving to the next.
Approach:
- Traditional and often follows a waterfall or sequential model.
- Can also follow iterative models like Agile, where each iteration includes all phases of the SDLC.

DevOps (Development and Operations)

Definition:
- DevOps is a set of practices that combines software development (Dev) and IT operations (Ops).
- Aims to shorten the development lifecycle and provide continuous delivery with high software quality.
Principles:
- Collaboration: Close collaboration between development and operations teams.
- Automation: Automate repetitive tasks to increase efficiency.
- Continuous Integration (CI): Frequently integrate code changes into a shared repository.
- Continuous Delivery (CD): Continuously deliver code to production.
- Monitoring and Feedback: Continuously monitor applications and provide feedback for improvements.
Focus:
- Emphasizes a culture of collaboration and shared responsibility.
- Aims to deliver features, fixes, and updates frequently and reliably.
Approach:
- Uses tools and practices such as CI/CD pipelines, infrastructure as code, automated testing, and configuration management.
- Promotes iterative improvements and rapid deployment cycles.

Key Differences

Objective:
- SDLC: Focuses on the systematic development and maintenance of software.
- DevOps: Focuses on speed, collaboration, and continuous improvement in both development and operations.
Phases vs. Practices:
- SDLC: Structured phases with a clear sequence.
- DevOps: Set of practices and tools integrated into the workflow.
Team Structure:
- SDLC: Distinct roles for development and operations.
- DevOps: Blurred lines between development and operations, promoting cross-functional teams.
Delivery:
- SDLC: Often results in longer release cycles.
- DevOps: Enables faster and more frequent releases.

The Impact of DevOps on SDLC

Increased Collaboration: DevOps fosters collaboration between development, operations, and other cross-functional teams. By breaking down silos and promoting shared responsibilities, DevOps encourages teams to work together towards common goals.
Faster Delivery: Automation and CI/CD pipelines enable faster and more frequent delivery of software updates. This accelerates the pace of innovation and allows organizations to respond quickly to changing market demands.

Improved Quality: DevOps practices such as automated testing, continuous monitoring, and feedback loops enhance the overall quality of software. By detecting and addressing issues early in the development process, teams can deliver more reliable and resilient applications.

Enhanced Scalability: DevOps principles like IaC and containerization facilitate the scalability of infrastructure and applications. This allows organizations to efficiently manage growth and handle fluctuations in demand without compromising performance or reliability.

Greater Agility: DevOps promotes agility by enabling rapid iterations, experimentation, and adaptation. Teams can quickly pivot in response to customer feedback, market trends, or competitive pressures, ensuring continued relevance and competitiveness.

Agile and DevOps are both methodologies that aim to improve software development processes, but they focus on different aspects and principles. Here’s a detailed comparison:

Agile

Definition:
- Agile is a set of principles and practices for software development under which requirements and solutions evolve through collaborative effort.
- It promotes flexible responses to change and iterative progress through short development cycles called sprints.
Principles:
- Customer Collaboration: Engage customers throughout the development process.
- Iterative Development: Deliver small, incremental updates frequently.
- Flexible and Adaptive Planning: Respond to changes rather than following a rigid plan.
- Self-Organizing Teams: Empower teams to make decisions and manage their work.
Focus:
- Emphasizes adaptability, customer feedback, and small, frequent releases.
- Aims to improve customer satisfaction through continuous delivery of valuable software.
Methodologies:
- Common frameworks include Scrum, Kanban, Lean, and Extreme Programming (XP).
- Iterations typically last 1-4 weeks, with regular retrospectives and reviews.
Team Structure:
- Cross-functional teams that include developers, testers, and business analysts.
- Roles such as Scrum Master and Product Owner are key in Agile frameworks like Scrum.

DevOps

Definition:
- DevOps is a set of practices that combines software development (Dev) and IT operations (Ops) to shorten the development lifecycle and deliver high-quality software continuously.
- It aims to create a culture of collaboration between development and operations teams.
Principles:
- Collaboration and Communication: Foster close collaboration between development, operations, and other stakeholders.
- Automation: Automate repetitive tasks to increase efficiency and reduce errors.
- Continuous Integration (CI): Frequently merge code changes into a central repository.
- Continuous Delivery (CD): Continuously deploy code to production environments.
- Monitoring and Feedback: Continuously monitor applications and provide feedback for improvements.
Focus:
- Emphasizes the automation of processes, continuous delivery, and the integration of development and operations.
- Aims to improve the speed, efficiency, and reliability of software delivery.
Practices and Tools:
- Uses tools for CI/CD (e.g., Jenkins, GitLab CI), configuration management (e.g., Ansible, Puppet), and infrastructure as code (e.g., Terraform).
- Promotes practices like version control, automated testing, and continuous monitoring.
Team Structure:
- Blends development and operations roles to create cross-functional teams.
- Encourages shared responsibilities for deployment and maintenance tasks.

Key Differences

Scope:
- Agile: Primarily focuses on the development process and managing changes in requirements.
- DevOps: Encompasses the entire software lifecycle, from development to operations and maintenance.
Objective:
- Agile: Aims to deliver small, incremental changes quickly and efficiently.
- DevOps: Aims to automate and integrate the processes between software development and IT operations to enable continuous delivery.
Practices and Tools:
- Agile: Focuses on methodologies and frameworks (Scrum, Kanban) and practices like daily stand-ups, sprints, and retrospectives.
- DevOps: Focuses on automation and tools for CI/CD, configuration management, and monitoring.
Team Dynamics:
- Agile: Teams are usually small and focused on development, with roles like Scrum Master and Product Owner.
- DevOps: Teams are cross-functional, including both development and operations, with a focus on collaboration and shared responsibilities.

Integration of Agile and DevOps

Agile and DevOps can be integrated to complement each other:
- Agile can be used to manage and deliver small, incremental changes efficiently.
- DevOps can ensure that these changes are deployed quickly and reliably, with a focus on automation and continuous delivery.

In summary, while Agile focuses on improving the development process through iterative progress and customer collaboration, DevOps extends this improvement to the entire software lifecycle by promoting automation, continuous delivery, and close collaboration between development and operations teams.

DEV Community: Pratik Nalawade

Setting Up a Monitoring Stack with Nginx Logging Using Grafana, cAdvisor, Promtail, Prometheus, and Loki

Why This Stack?

Step-by-Step Setup

1. Install Grafana on Debian/Ubuntu

2. Install Loki and Promtail Using Docker

3. Install Prometheus and cAdvisor

4. Configure Nginx Logging

5. Set Up Grafana Dashboards

Conclusion

Automating Security Audits and Server Hardening on Linux Servers with Bash Scripting

Introduction

Project Overview

Key Components

1. User and Group Audits

2. File and Directory Permissions

3. Service Audits

4. Firewall and Network Security

5. IP and Network Configuration Checks

6. Security Updates and Patching

7. Log Monitoring

8. Server Hardening Steps

9. Custom Security Checks

10. Reporting and Alerting

Challenges and Solutions

Dependency Management

Permission Issues

Modular Design

Public vs. Private IP Identification

Reporting and Alerting

How to Use the Script

Prerequisites

Running the Script

Conclusion

Building a Real-Time System Monitoring Dashboard with Bash

Why Bash?

Key Features of the Script

Building the Script

1. Monitoring Top Applications

2. Network Monitoring

3. Disk Usage

4. System Load

5. Memory Usage

6. Process Monitoring

7. Service Monitoring

8. Custom Dashboard

Lessons Learned

Conclusion

DevOps Monitoring Project

Learning DevOps Monitoring with DevOps Shack: A Hands-On Journey

Project Overview

Tools and Technologies

Prerequisites

Step-by-Step Setup

VM-1: Setting Up Node Exporter

VM-2: Setting Up Prometheus, Alertmanager, and Blackbox Exporter

Configuration Details

Firewall and Security Settings

Key Features and Functionalities

Challenges and Solutions

Future Enhancements

Conclusion

Shell scripting Interview Questions

!/bin/bash

1. Prompt user to enter a number

Check if the number is positive, negative, or zero

2. Using a for loop to iterate over a list of items:**

!/bin/bash

Define a list of fruits

Iterate over the list of fruits using a for loop

Deploying a Scalable Web Application with Terraform on AWS

Project Overview

1. Setting Up Terraform and AWS Provider

2. VPC and Subnet Configuration

3. Internet Gateway and Route Table

4. Security Groups

5. Deploying EC2 Instances

6. Setting Up the S3 Bucket

7. Configuring the Application Load Balancer (ALB)

Building and Dockerizing a MERN Stack Application

2. Using a `for` loop to iterate over a list of items:**