DEV Community: Samson

Interacting with AWS

Samson — Mon, 03 Mar 2025 11:58:04 +0000

1. AWS Management Console (Protected by Password + MFA)

What it is: The AWS Management Console is a web-based interface where you can manage AWS services. It requires a username, password, and Multi-Factor Authentication (MFA) for added security.
Real-time Example:
- Imagine you are a DevOps engineer responsible for managing an EC2 instance (a virtual server) running a web application.
- You log in to the AWS Management Console using your username and password. Additionally, you are prompted to enter a one-time code from your MFA device (e.g., Google Authenticator or a hardware token).
- Once logged in, you navigate to the EC2 dashboard, where you can start, stop, or configure your virtual server. You can also monitor metrics like CPU usage and network traffic.

2. AWS Command Line Interface (CLI) (Protected by Access Keys)

What it is: The AWS CLI is a command-line tool that allows you to interact with AWS services using text-based commands. It uses access keys (Access Key ID and Secret Access Key) for authentication.
Real-time Example:
- Suppose you are a developer who needs to automate the deployment of a new version of your application to an S3 bucket (a storage service).
- Instead of using the AWS Management Console, you use the AWS CLI on your local machine. You configure the CLI with your access keys:
```
aws configure
```
You enter your Access Key ID, Secret Access Key, default region, and output format.
- Then, you upload your application files to the S3 bucket using the following command:
```
aws s3 cp my-app.zip s3://my-bucket/
```
- This method is faster and more efficient for repetitive tasks or automation.

3. AWS Software Developer Kit (SDK) - For Code (Protected by Access Key)

What it is: The AWS SDK is a set of libraries that allow developers to interact with AWS services programmatically in their preferred programming language (e.g., Python, Java, JavaScript). Like the CLI, it uses access keys for authentication.
Real-time Example:
- Imagine you are a backend developer building a Python application that processes user-uploaded images. You want to store these images in an S3 bucket and trigger a Lambda function to resize them.
- You use the AWS SDK for Python (Boto3) to write code that uploads the image to S3:
```
import boto3

# Initialize the S3 client
s3 = boto3.client('s3', aws_access_key_id='YOUR_ACCESS_KEY', aws_secret_access_key='YOUR_SECRET_KEY')

# Upload a file to S3
s3.upload_file('user-image.jpg', 'my-bucket', 'user-image.jpg')
```
- After the image is uploaded, you can use the SDK to invoke a Lambda function that processes the image:
```
lambda_client = boto3.client('lambda')
response = lambda_client.invoke(
    FunctionName='resize-image-function',
    Payload='{"bucket": "my-bucket", "key": "user-image.jpg"}'
)
```
- This approach is ideal for integrating AWS services directly into your applications.

Summary of Real-Time Use Cases:

AWS Management Console: Used for manual, visual management of AWS resources (e.g., starting an EC2 instance).
AWS CLI: Used for scripting and automation (e.g., uploading files to S3).
AWS SDK: Used for programmatic integration of AWS services into applications (e.g., uploading files to S3 and triggering a Lambda function via code).

Each method has its own use case, and the choice depends on whether you need manual control, automation, or programmatic integration.

Choosing the Right DevOps Monitoring Tool: AWS CloudWatch vs. Prometheus

Samson — Fri, 07 Feb 2025 16:36:40 +0000

When it comes to monitoring your DevOps infrastructure, selecting the right tool can make all the difference in maintaining system reliability and performance. Two popular options are AWS CloudWatch and Prometheus, each with its own strengths and use cases.

AWS CloudWatch

Scope: Monitors the entire infrastructure, including servers.
Metrics: Tracks up to 21 metrics.
Scrape Time: Checks metrics every 60 seconds, with the ability to extend to longer intervals.
Use Case: Ideal for general monitoring and checking system health over longer periods.

Prometheus

Scope: Focuses on server and infrastructure monitoring.
Metrics: Capable of monitoring over 1,000 metrics.
Scrape Time: Checks metrics every 15 seconds to 1 minute, allowing for faster detection of issues.
Evaluation Time: Evaluates metrics within 15 to 60 seconds, enabling quicker response to anomalies.
Default Timeout: If no response is received within 10 seconds, it triggers a timeout, ensuring timely error detection.
Time Series Database: Stores initial metrics and only updates changes every 15 seconds, avoiding duplication and optimizing storage.

Key Takeaways

Faster Detection: Prometheus, with its 15-second scrape time, can detect errors more quickly than CloudWatch, which operates on a 60-second interval.
Comprehensive Monitoring: Prometheus supports over 1,000 metrics, making it more detailed and versatile for complex infrastructures.
Efficient Storage: Prometheus uses a time series database to store and update only the changes in metrics, reducing duplication and saving storage space.

Conclusion

While AWS CloudWatch is a robust tool for general monitoring, Prometheus stands out for its faster scrape times, detailed metrics, and efficient storage mechanisms. For DevOps teams looking to detect and resolve issues swiftly, Prometheus offers a significant advantage.

Choosing between these tools depends on your specific needs—whether you prioritize broader, less frequent monitoring or detailed, real-time insights. Both tools have their place in a DevOps toolkit, but Prometheus is particularly well-suited for environments where rapid detection and response are critical.

Mastering Terraform: A Comprehensive Guide to Infrastructure as Code (IaC) from Basics to Advanced Concepts

Samson — Fri, 07 Feb 2025 16:16:00 +0000

Terraform Summary: Key Concepts and Workflow

Day 1: Introduction to Terraform

Terraform Files (.tf): Used to define infrastructure as code.
On-Demand vs. Reserved Resources: On-demand for unplanned needs, reserved for planned future use.
File Transfer: Use scp to copy files from a local machine to a Linux server using a PEM key for authentication.
Terraform Init: Initializes the working directory, creating .terraform (environment setup) and a lock file (plugin version control).
Terraform Blocks:
1. Terraform Block: Specifies the provider (e.g., AWS, GCP).
2. Provider Block: Defines the region (e.g., us-east-1).
3. Resource Block: Defines the resources to be created.
Multi-Region VPC: Multiple VPCs in different regions can be created using aliases.

Day 2: Immutability and Resource Management

Terraform Apply vs. Plan: You can run terraform apply before terraform plan, but it’s not recommended.
Resource Modification: Small changes (e.g., instance type) are modified in-place, while major changes (e.g., OS change) result in resource recreation.
Lifecycle Rules:
1. Create Before Destroy: Ensures new resources are created before old ones are destroyed.
2. Prevent Destroy: Prevents accidental deletion of resources.
Count: Used to create multiple instances of a resource (e.g., count = 5 creates 5 EC2 instances).
For Each: Used for creating resources like S3 buckets where count is not applicable.

Day 3: Variables and State Management

State File: Stores the current state of the infrastructure. Changes made manually in the cloud console are not reflected in the state file unless terraform refresh is run.
Lifecycle Ignore Changes: Prevents Terraform from overwriting manual changes (e.g., tags).
Depends On: Ensures resources are created in a specific order.
Variables: Declared in var.tf to avoid hardcoding values. Can be overridden via CLI or environment variables.
Variable Precedence: Terraform looks for variables in the following order:
1. auto.tfvars
2. terraform.tfvars
3. Environment variables
4. variable.tf
5. main.tf

Day 4: Advanced Variables and Outputs

Lists and Maps: Used to define multiple values for variables (e.g., instance types).
File Variables: Use file() to include scripts or configuration files in resources.
Sensitive Data: Mark variables as sensitive = true to hide sensitive information.
Outputs: Use output.tf to display resource attributes (e.g., public IP of an EC2 instance).
Local Variables: Use locals to define reusable values (e.g., tags).

Day 5: State Locking, Provisioners, and Workspaces

State Locking: Prevents concurrent operations on the same state file using DynamoDB and S3.
Terraform Commands:
- terraform show: Displays the state file content.
- terraform state list: Lists resources in the state file.
- terraform taint/untaint: Marks a resource for recreation or removes the mark.
- terraform apply -target: Applies changes to a specific resource.
Provisioners:
1. Remote: Executes commands on a remote machine.
2. Local: Executes commands locally.
3. File: Transfers files to or from a remote machine.
Workspaces: Used to manage multiple environments (e.g., prod, dev, staging) within the same configuration.

Key Takeaways

Terraform is a powerful tool for managing infrastructure as code, offering flexibility through variables, lifecycle rules, and state management.
Use count and for_each to manage multiple resources efficiently.
Leverage output.tf to extract and display resource attributes.
State locking and workspaces help manage complex environments and prevent conflicts.
Provisioners allow for additional configuration and file transfers during resource creation.

This summary provides a high-level overview of Terraform's core concepts and workflows, making it easier to understand and implement infrastructure as code in real-world scenarios.

Leetcode : 1280 (Counting Student Exam Attendances)

Samson — Thu, 06 Feb 2025 12:53:09 +0000

Problem Statement:
We need to find out how many times each student attended each exam. The data is spread across three tables: Students, Subjects, and Examinations. The result should include all students and all subjects, even if a student did not attend any exams for a particular subject.

Key Steps in the Solution:

Combine All Students and Subjects:
- Use a CROSS JOIN between the Students and Subjects tables to create all possible combinations of students and subjects. This ensures that every student is paired with every subject, even if they didn’t attend any exams.
Count Exam Attendances:
- Use a LEFT JOIN with the Examinations table to count how many times each student attended each exam. If a student did not attend any exams for a subject, the count will be 0.
Group and Order the Results:
- Group the results by student_id, student_name, and subject_name to calculate the count of attended exams.
- Order the results by student_id and subject_name for a clean and organized output.

SQL Query:

SELECT 
    s.student_id, 
    s.student_name, 
    sub.subject_name, 
    COUNT(e.student_id) AS attended_exams
FROM 
    Students s
CROSS JOIN 
    Subjects sub
LEFT JOIN 
    Examinations e 
ON 
    s.student_id = e.student_id 
    AND sub.subject_name = e.subject_name
GROUP BY 
    s.student_id, 
    s.student_name, 
    sub.subject_name
ORDER BY 
    s.student_id, 
    sub.subject_name;

Example Input Tables:

`Students` Table:

student_id	student_name
1	Alice
2	Bob
13	John
6	Alex

`Subjects` Table:

subject_name
Math
Physics
Programming

`Examinations` Table:

student_id	subject_name
1	Math
1	Physics
1	Programming
2	Programming
1	Physics
1	Math
13	Math
13	Programming
13	Physics
2	Math
1	Math

Example Output:

student_id	student_name	subject_name	attended_exams
1	Alice	Math	3
1	Alice	Physics	2
1	Alice	Programming	1
2	Bob	Math	1
2	Bob	Physics	0
2	Bob	Programming	1
6	Alex	Math	0
6	Alex	Physics	0
6	Alex	Programming	0
13	John	Math	1
13	John	Physics	1
13	John	Programming	1

Explanation of the Output:

Alice attended the Math exam 3 times, the Physics exam 2 times, and the Programming exam 1 time.
Bob attended the Math exam 1 time, the Programming exam 1 time, and did not attend the Physics exam.
Alex did not attend any exams.
John attended the Math, Physics, and Programming exams 1 time each.

Why This Query Works:

The CROSS JOIN ensures all student-subject combinations are included.
The LEFT JOIN ensures that students who did not attend any exams are still included with a count of 0.
The COUNT function calculates the number of times each student attended each exam.
The GROUP BY and ORDER BY clauses organize the results for clarity.

Find Duplicate in a list

Samson — Wed, 05 Feb 2025 16:18:49 +0000

The findDuplicate function is designed to find the first duplicate number in a list. Here's a concise breakdown:

Code Explanation

class Solution(object):
    def findDuplicate(self, nums):
        a = {}  # Dictionary to store encountered numbers
        i = 0
        while i < len(nums):  # Loop through the list
            if nums[i] in a:  # If the number is already in the dictionary, it's a duplicate
                return nums[i]  # Return the duplicate number
            else:
                a[nums[i]] = 1  # Mark the number as seen
            i += 1  # Move to the next number

How It Works

It initializes an empty dictionary a to store numbers encountered so far.
It loops through the list nums:
- If the number is already in a, it returns that number as the first duplicate.
- Otherwise, it adds the number to a and continues.
The function returns the first duplicate found in the list.

Example

Input:

nums = [1, 3, 4, 2, 2]
solution = Solution()
print(solution.findDuplicate(nums))

Output:

Explanation:

The number 2 appears twice in the list, and the function detects it as the first duplicate.

Split Words By Separator

Samson — Wed, 05 Feb 2025 16:04:05 +0000

Here’s a Python function to split words using a separator and remove empty results:

class Solution(object):
    def splitWordsBySeparator(self, words, separator):
      ans = []
      for i in range(len(words)):  # Loop through each word inthe list
          ans.append(words[i].split(separator))  # Split wordby the separator

      new = sum(ans, [])  # Flatten the list (converts list oflists into a single list)
      res = list(filter(None, new))  # Remove empty stringsfrom the list

      return res  # Return the cleaned list of words

How It Works

It takes two inputs:

words: A list of strings.
separator: A character used to split the words.

It loops through each word and splits it using the separator.
The split words are stored in a list (ans), which may contain sublists.
It flattens ans using sum(ans, []) to merge all sublists into a single list.
It removes empty strings using filter(None, new) to get rid of unnecessary empty elements.
Finally, it returns the cleaned list of words.

Example

solution = Solution()
words = ["hello.world", "python.is.awesome", "split,this"]
separator = "."

result = solution.splitWordsBySeparator(words, separator)
print(result)

Output

['hello', 'world', 'python', 'is', 'awesome', 'split', 'this']

DEV Community: Samson

Interacting with AWS

1. AWS Management Console (Protected by Password + MFA)

2. AWS Command Line Interface (CLI) (Protected by Access Keys)

3. AWS Software Developer Kit (SDK) - For Code (Protected by Access Key)

Summary of Real-Time Use Cases:

Choosing the Right DevOps Monitoring Tool: AWS CloudWatch vs. Prometheus

AWS CloudWatch

Prometheus

Key Takeaways

Conclusion

Mastering Terraform: A Comprehensive Guide to Infrastructure as Code (IaC) from Basics to Advanced Concepts

Terraform Summary: Key Concepts and Workflow

Day 1: Introduction to Terraform

Day 2: Immutability and Resource Management

Day 3: Variables and State Management

Day 4: Advanced Variables and Outputs

Day 5: State Locking, Provisioners, and Workspaces

Key Takeaways

Leetcode : 1280 (Counting Student Exam Attendances)

Key Steps in the Solution:

SQL Query:

Example Input Tables:

Students Table:

Subjects Table:

Examinations Table:

Example Output:

Explanation of the Output:

Why This Query Works:

Find Duplicate in a list

Code Explanation

How It Works

Example

Input:

Output:

Explanation:

Split Words By Separator

Here’s a Python function to split words using a separator and remove empty results:

How It Works

Example

Output

`Students` Table:

`Subjects` Table:

`Examinations` Table: