DEV Community: Muhammad

Creating a Simple Pastebin Service in Python and Flask

Muhammad — Mon, 08 Jul 2024 04:40:02 +0000

In this blog post, we will be building a simple Pastebin service using Python and Flask. Pastebin is a popular web application used to store plain text or code snippets for a certain period of time. We'll create a basic version that allows users to paste text, select the programming language, and get a URL to share the paste. I have also created a YouTube video about this, which you can view here.

Getting Starting

Before begin creating our application lets setup our environment and in order to setup your environment follow these steps:

First, Let's create a virtual environment in the project directory.

python -m venv venv

Now, once we have created the virtual environment, let's activate it and install all the required libraries that are going to be used by this project.

pip install Flask shortuuid pygments

We'll also use shortuuid for generating unique IDs for each paste and pygments for syntax highlighting.

Now that we have installed all the required libraries, let's create the necessary files and folders.

mkdir -p pastes templates static && touch index.py templates/index.html static/styles.css

This is how your folder structure should look:

pastebin/
│
├── app.py
├── pastes/
├── templates/
│   └── index.html
└── static/
    └── styles.css

The pastes directory will store the text files for each paste. The templates directory contains our HTML templates, and the static directory contains CSS for styling.

Now that we have set up the environment, it's time to code.

Writing Code

Let's dive into the code. Create a file named index.py and add the following code:

from flask import Flask, request, render_template, abort
import shortuuid
import os
from pygments import highlight
from pygments.lexers import get_lexer_by_name, get_all_lexers
from pygments.formatters import HtmlFormatter

app = Flask(__name__)

# Directory to store paste files
PASTE_DIR = 'pastes'
if not os.path.exists(PASTE_DIR):
    os.makedirs(PASTE_DIR)

# Function to get available programming languages for syntax highlighting
def get_language_options():
    return sorted([(lexer[1][0], lexer[0]) for lexer in get_all_lexers() if lexer[1]])

# Route for the main page
@app.route('/', methods=['GET', 'POST'])
def index():
    if request.method == 'POST':
        # Get content and language from the form
        content = request.form['content']
        language = request.form['language']
        # Generate a unique ID for the paste
        paste_id = shortuuid.uuid()
        # Create the file path for the paste
        file_path = os.path.join(PASTE_DIR, paste_id)

        # Save the paste content to a file
        with open(file_path, 'w') as f:
            f.write(f"{language}\n{content}")

        # Generate the URL for the new paste
        paste_url = request.url_root + paste_id
        return render_template('index.html', paste_url=paste_url, languages=get_language_options())

    # Render the form with available languages
    return render_template('index.html', languages=get_language_options())

# Route to view a specific paste by its ID
@app.route('/<paste_id>')
def view_paste(paste_id):
    # Create the file path for the paste
    file_path = os.path.join(PASTE_DIR, paste_id)
    if not os.path.exists(file_path):
        abort(404)  # Return a 404 error if the paste does not exist

    # Read the paste file
    with open(file_path, 'r') as f:
        language = f.readline().strip()  # First line is the language
        content = f.read()  # Remaining content is the paste

    # Get the appropriate lexer for syntax highlighting
    lexer = get_lexer_by_name(language, stripall=True)
    # Create a formatter for HTML output
    formatter = HtmlFormatter(linenos=True, cssclass="source")
    # Highlight the content
    highlighted_content = highlight(content, lexer, formatter)
    # Get the CSS for the highlighted content
    highlight_css = formatter.get_style_defs('.source')

    # Render the paste with syntax highlighting
    return render_template('index.html', paste_content=highlighted_content, highlight_css=highlight_css)

if __name__ == '__main__':
    app.run(debug=True)

Once you have created the flask now let's create html template in templates/index.html and style.css in static/style.css

templates/index.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Pastebin Service</title>
    <link rel="stylesheet" href="{{ url_for(\'static\', filename=\'styles.css\') }}">
    {% if highlight_css %}
    <style>
        {{ highlight_css|safe }}
    </style>
    {% endif %}
</head>
<body>
    <h1>Pastebin Service</h1>
    {% if paste_url %}
        <p>Your paste URL: <a href="{{ paste_url }}">{{ paste_url }}</a></p>
    {% endif %}
    {% if paste_content %}
        <div class="highlight">
            {{ paste_content|safe }}
        </div>
    {% endif %}
    <form method="post">
        <textarea name="content" rows="10" cols="50" placeholder="Paste your text here..."></textarea><br>
        <select name="language">
            {% for code, name in languages %}
                <option value="{{ code }}">{{ name }}</option>
            {% endfor %}
        </select><br>
        <button type="submit">Submit</button>
    </form>
</body>
</html>

static/style.css

body {
    font-family: Arial, sans-serif;
    margin: 20px;
}

h1 {
    color: #333;
}

textarea {
    width: 100%;
    margin-top: 10px;
}

select, button {
    margin-top: 10px;
}

.highlight {
    background-color: #f5f5f5;
    padding: 10px;
    border: 1px solid #ccc;
    margin-top: 20px;
}

Now that we have created our application, before we run it, let's try to understand how it works by breaking down the code.

Code Breakdown

First, we import the necessary libraries and modules. Flask is our web framework, shortuuid is used for generating unique IDs, and Pygments is for syntax highlighting. We also set up a directory to store our pastes/.

from flask import Flask, request, render_template, abort
import shortuuid
import os
from pygments import highlight
from pygments.lexers import get_lexer_by_name, get_all_lexers
from pygments.formatters import HtmlFormatter

app = Flask(__name__)

PASTE_DIR = 'pastes'
if not os.path.exists(PASTE_DIR):
    os.makedirs(PASTE_DIR)

Then we write a function that retrieves all available programming languages supported by Pygments for syntax highlighting and returns them as a sorted list of tuples.

def get_language_options():
    return sorted([(lexer[1][0], lexer[0]) for lexer in get_all_lexers() if lexer[1]])

Then we write the main route for our application. If the request method is POST (i.e., when the user submits a form), it saves the content and language to a new file with a unique ID. The URL for the new paste is generated and displayed to the user. If the request method is GET, it simply renders the form.

@app.route('/', methods=['GET', 'POST'])
def index():
    if request.method == 'POST':
        content = request.form['content']
        language = request.form['language']
        paste_id = shortuuid.uuid()
        file_path = os.path.join(PASTE_DIR, paste_id)

        with open(file_path, 'w') as f:
            f.write(f"{language}\n{content}")

        paste_url = request.url_root + paste_id
        return render_template('index.html', paste_url=paste_url, languages=get_language_options())

    return render_template('index.html', languages=get_language_options())

This route handles viewing a specific paste. It reads the paste file, applies syntax highlighting using pygments, and renders the highlighted content.

@app.route('/<paste_id>')
def view_paste(paste_id):
    file_path = os.path.join(PASTE_DIR, paste_id)
    if not os.path.exists(file_path):
        abort(404)

    with open(file_path, 'r') as f:
        language = f.readline().strip()
        content = f.read()

    lexer = get_lexer_by_name(language, stripall=True)
    formatter = HtmlFormatter(linenos=True, cssclass="source")
    highlighted_content = highlight(content, lexer, formatter)
    highlight_css = formatter.get_style_defs('.source')

    return render_template('index.html', paste_content=highlighted_content, highlight_css=highlight_css)

Now once we understand how everything works, now you can simply run the application using this command
python index.py

Conclusion

You've built a simple Pastebin service using Python and Flask! This service allows users to paste text, select a programming language, and share the paste via a unique URL. You can expand this project by adding features like expiration times for pastes, user authentication, or even a database to store pastes more efficiently.

If you have any feedback, please feel free to leave a comment below. If you prefer not to comment publicly, you can always send me an email.

ORIGINALLY POSTED HERE

Understanding Python Variables: Namespaces and Variable Scope

Muhammad — Thu, 05 Oct 2023 07:05:52 +0000

I have been using Python extensively throughout my career. I wanted to write this post to provide an understanding of Namespaces and Variable Scope. Like most programming languages, Python offers a structured way to store and access data through variables. However, understanding where and how these variables exist and interact can sometimes be complicated. This post will help you grasp the fundamental concepts related to Python variables: namespaces and variable scope.

What is a Namespace?

In computer programming, and more explicitly in Python, understanding the concept of a namespace is pivotal to managing variable references and ensuring code clarity. At its core, a namespace serves as a fundamental structure, encapsulating and organizing identifiers to avoid potential naming conflicts.

In the simplest terms, a namespace is a container that holds a collection of identifiers. These identifiers can be variable names, function names, class names, and more. Each of these identifiers is associated with specific objects (values) in memory. Think of it as a dictionary where the keys represent variable names (or other identifiers) and the values correspond to the actual objects or references in memory

Unique Naming System: Namespaces ensure that there is no ambiguity in the naming system. For instance, you can have a function named calculate in one namespace and another function with the same name in a different namespace without any conflict.

Lifetime of a Namespace: The existence of a namespace is dependent on the scope of the objects. If the scope of an object ends, the namespace might also get deleted, and thus all the names defined in that namespace will be made unbound.

Types of Namespaces

Python has various namespaces, created and deleted at different times:

Built-in Namespace: Contains Python's built-in functions and exceptions. Created when the Python interpreter starts up.
Global (Module) Namespace: Specific to a module or script. Created when the module is imported or the script is run.
Enclosing (Function) Namespace: Exists for nested functions. It chains multiple function namespaces from innermost to outermost.
Local Namespace: Created when a function is called. Once the function execution completes, the namespace is discarded.

Variable Scope

Scope defines the region of the code where a variable can be accessed or modified. Python has four primary variable scopes:

Local (L): Inside the current function.
Enclosing (E): Inside enclosing functions.
Global (G): At the top level of the module.
Built-in (B): In the built-in namespace.

These scopes form the LEGB rule, which Python follows when resolving variable names.

Understanding Scope with Examples

x = 10  # global variable

def outer_function():
    y = 5  # enclosing variable

    def inner_function():
        z = 3 # local var

        print(x, y, z)

    inner_function()

outer_function()

When inner_function is called, it accesses:

z from its local scope.
y from the enclosing scope of outer_function.
x from the global scope.

The `global` and `nonlocal` Keywords

To modify global or enclosing variables within a function, Python provides the global and nonlocal keywords:

x = 10

def modify_global():
    global x
    x = 20

def outer_function():
    y = 5

    def modify_enclosing():
        nonlocal y
        y = 15

    modify_enclosing()
    print(y)

modify_global()
outer_function()
print(x)

This code will output

15
20

The global keyword tells Python we're referring to the global x, and the nonlocal keyword indicates we're targeting the y from the enclosing function.

Avoid Variable Shadowing

If a local variable shares the same name as a global variable or a built-in, it shadows the global or built-in variable:

x = 10

def shadow_example():
    x = 5
    print(x)

shadow_example()  # Outputs: 5
print(x)  # Outputs: 10

Shadowing can lead to unexpected behaviors, so it's recommended to avoid using the same names across different scopes.

Conclusion

Namespaces and variable scope form the bedrock of how Python manages and accesses data. By understanding these concepts, you can write clearer, more predictable code and avoid common pitfalls. Remember the LEGB rule, be cautious of shadowing, and use the global and nonlocal keywords judiciously to maintain clean and efficient code.

Happy Coding!

Originally Published HERE

Commandline Productivity Part 1: fzf - The Command-Line Fuzzy Finder

Muhammad — Wed, 20 Sep 2023 15:18:42 +0000

I've been using the command line extensively at my day job. I utilize various command line tools, enhancing my workflow and boosting my productivity. Therefore, I'm launching a new biweekly series where I'll cover the tools I use, dedicating each post to a specific tool. In today's post, we'll explore fzf - The Command Line Fuzzy Finder, and discuss how it can improve daily your workflow.

What's `fzf` and why use it?

Before I dive into fzf, I'd like to take a moment to explain what fuzzy matching is and discuss the algorithms used behind the scenes in fuzzy matching.

What's Fuzzy Matching?

Fuzzy matching is a technique used in computing to find strings that are approximately equal or closely resemble each other. Unlike exact matching, where the aim is to find an exact match or replicate, fuzzy matching identifies matches that may not be perfect but are "close enough" based on a set criteria.

Fuzzy matching is frequently used in data cleaning, where it helps in identifying duplicate records in large databases, even when the entries aren't exactly the same (e.g., "McDonald's" vs. "Mc Donalds").

It's also useful in search engines and autocorrect features, where slight variations or typos in a search term can still yield the desired results.

How it works? In Simplest Terms

Fuzzy matching algorithms evaluate strings based on various metrics, such as the number of changes required to turn one string into another (edit distance) or the number of shared character sequences. The outcome is often a score that represents the similarity between the two strings, with higher scores indicating greater similarity.

Algorithms used within Fuzzy Matching

Edit Distance (Levenshtein Distance):
This algorithm measures the similarity between two strings by determining the minimum number of single-character edits (i.e., insertions, deletions, or substitutions) required to change one string into the other. For example, the Levenshtein distance between "kitten" and "sitting" is 3.
Damerau-Levenshtein Distance: This is an extension of the Levenshtein distance, taking into account transpositions (swapping of two adjacent characters). For instance, the distance between "flaw" and "lawn" considering a transposition would be 1 using Damerau-Levenshtein.
Smith-Waterman Algorithm: Originally developed for bioinformatics, this local sequence alignment algorithm can also be used for text comparisons. It's particularly effective for scoring the similarity of substrings.
Jaro and Jaro-Winkler Distance: These are measures of similarity between two strings. The Jaro-Winkler distance gives more weight to the prefix of the strings and is especially useful for short strings like person names.
n-gram Analysis: In this technique, strings are broken down into overlapping substrings of 'n' characters. These n-grams are then compared to identify similarities. For example, using 2-grams (or bigrams), the word "hello" can be broken down into ["he", "el", "ll", "lo"]. For example, using 2-grams (or bigrams), the word "hello" can be broken down into ["he", "el", "ll", "lo"].
Token-Based Matching: This approach involves breaking strings into tokens (typically words) and comparing these tokens for similarity. Techniques like cosine similarity or Jaccard similarity can then be applied on these tokens.
Tf-idf (Term Frequency-Inverse Document Frequency): While more common in information retrieval systems, it can be applied to fuzzy matching. It measures how important a word is within a document relative to a collection of documents. It can be used in conjunction with cosine similarity for document comparisons
Longest Common Subsequence (LCS): This algorithm identifies the longest sequence of characters that two strings have in common. The LCS of "ABCBDAB" and "BDCAB" is "BCAB".

Different use-cases and applications may demand different algorithms or combinations of them. The choice often depends on the specific requirements of the task , such as the need for speed versus accuracy, the nature of the data, and the context in which the fuzzy matching is being applied.

So Now what's `fzf` and why use it?

fzf is a flexible tool that allows you to search and navigate any list (files, command history, git branches, etc.) using fuzzy matching. In essence, fuzzy matching means that you don't need to type exact search terms; instead, you can make typos or give partial input, and fzf will intelligently suggest matches.

For instance, if you have files named "important_document", "imported_files", and "impromptu_notes", typing "imp doc" in fzf might highlight "important_document" as the top match even though the search isn't an exact substring.

fzf is incredibly fast, enabling swift searches through files and command history. It offers an intuitive interface that lets you search through files in real-time as you type. Additionally, fzf provides numerous integrations with other tools, including Vim, among others.

Setting up and using `fzf`.

In order to use fzf you can simply follow this link which directs you to the installation instructions for fzf tailored to your OS. However, if you're on macOS, you can install fzf with the command brew install fzf, then execute /opt/homebrew/opt/fzf/install to install the shell completions.

`fzf` usage

Here's the basic usage of fzf,

File Search

fzf

Command History Search

Press CTRL + R in your terminal to interactively search through your command history.

Preview Window

$ find dir/ | fzf --preview 'cat {}'

Using fzf with Other Commands

$ ls -l $(fzf)  # List the details of a selected file

Select and Kill Processes

$ kill -9 $(ps aux | fzf | awk '{print $2}')

Filter Git Branches

$ git branch | fzf

Searching through your browser history (FireFox)

To search through your browser history, you can also utilize fzf. The SQLite database, which stores the history, is typically found at the following path on a Mac:

~/Library/Application Support/Firefox/Profiles/*.default-release.

After navigating to that directory, execute:

sqlite3 places.sqlite "SELECT url FROM moz_places" | fzf

Here we've covered the basic usage of fzf, but the tool offers so much more to explore and utilize. I hope this provided insight into how fzf can enhance your daily workflow. If you have any tips related to fzf, please share them in the comments. I'd also love to hear how you've been using fzf.

If you loved this post, you can always support my work by buying me a coffee. Also, if you end up sharing this on Twitter, definitely tag me @muhammad_o7.

Note: If you like to be notified about the upcoming posts you can follow me here or you can leave your email here

ORIGINALLY PUBLISHED HERE

WebScraping in Bash

Muhammad — Tue, 05 Sep 2023 02:42:00 +0000

In the realm of web scraping, Python often takes the spotlight with robust libraries such as BeautifulSoup and Scrapy. But did you know that web scraping can also be accomplished using Bash scripting? In this blog post, we'll delve into a Bash script that extracts links and titles from a webpage and stores them in a CSV file.

Spending most of my workday in the terminal, I've become intimately familiar with writing Bash automation scripts. However, to add a creative twist, I ventured into the world of web scraping using Bash. While Bash excels at scripting, I discovered its hidden talents in web scraping, which I'm excited to share in this blog post.

The Bash Script

#!/bin/bash

# Define the URL to scrape
base_url="https://lite.cnn.com"
url="https://lite.cnn.com/"

# Create a CSV file and add a header
echo "Link,Title" > cnn_links.csv

# Extract links and titles and save them to the CSV file
link_array=($(curl -s "$url" | awk -F 'href="' '/<a/{gsub(/".*/, "", $2); print $2}'))

for link in "${link_array[@]}"; do
    full_link="${base_url}${link}"
    title=$(curl -s "$full_link" | grep -o '<title[^>]*>[^<]*</title>' | sed -e 's/<title>//g' -e 's/<\/title>//g')
    echo "\"$full_link\",\"$title\"" >> cnn_links.csv
done

echo "Scraping and CSV creation complete. Links and titles saved to 'cnn_links.csv'."

How it Works?

This Bash script accomplishes the following:

It defines the base URL and the URL of the webpage you want to scrape.
It creates a CSV file named cnn_links.csv with a header row containing "Link" and "Title" columns.
Using curl, it fetches the HTML content of the specified webpage and extracts all the links found within anchor tags (<a>) using awk.
It then iterates through the array of links and extracts the page titles by making additional curl requests to each link.
Finally, it appends the extracted links and titles to the CSV file in the desired format.

Breaking it down further

grep -o '<title[^>]*>[^<]*</title>' extracts the page title from the HTML content using regular expressions.:
1. -o option tells grep to only output the matched part of the input text.
2. <title[^>]*> matches the opening <title> tag and any attributes (e.g., <title attribute="value">), if present.
3. [^<]* matches any characters that are not < (i.e., the text within the <title> tag). </title> matches the closing </title> tag.
4. </title> matches the closing </title> tag.
sed -e 's/<title>//g' -e 's/<\/title>//g' removes the <title> and </title> tags from the extracted title.`:
1. -e option allows specifying multiple commands to be executed by sed.
2. 's/<title>//g' is a sed command that replaces all occurrences of <title> with an empty string (i.e., removes the opening <title> tag).
3. 's/<\/title>//g' is another sed command that replaces all occurrences of </title> with an empty string (i.e., removes the closing </title> tag).

Combining these commands:

grep extracts the text within the <title> and </title> tags.
sed then removes the tags themselves, leaving only the text content of the title.

This command also uses awk to extract URLs from an HTML document. Let's break it down step by step:

awk -F 'href="':
- awk is a text processing tool that operates on text files or input streams.
- -F 'href="' sets the field separator to 'href="'. This means awk will treat 'href="' as the delimiter for splitting input lines into fields.
'/<a/{gsub(/".*/, "", $2); print $2}':
- /<a/ is a pattern that specifies a condition: lines containing <a>. This ensures that the following actions are only applied to lines containing anchor tags.
- gsub(/".*/, "", $2) is an awk function that globally substitutes (gsub) everything from the first double quote (") to the end of the field ($2) with an empty string. In this case, it effectively removes the opening ", and the result is the URL.
- print $2 prints the modified field (the extracted URL).

So, this awk command looks for lines containing anchor tags (<a>) and extracts the URLs by removing everything before the first double quote (") in the href attribute. The extracted URLs are then printed as output.

Conclusion

So here's a simple web scraper written in bash and uses cli tools such as awk, sed , grep and curl. As bash is available on most Linux system so it can be useful for scraping data from web pages without having to install any additional software. However, it is not as powerful as Python or other programming languages when it comes to web scraping. But it can be useful for simple tasks such as extracting links and titles from a webpage and I would not recommend using it for complex web scraping tasks.

Anyways this was a fun little script I created while learning about bash scripting and using cli tools such as awk , grep , sed and curl. I would still consider myself a beginner at this.

Lastly I hope you enjoyed reading this and got a chance to learn something new from this post and if you have any bash tips or used bash for similar task feel free to comment below as I would love to hear it.

If you loved this post, you can always support my work by buying me a coffee. your support would mean the world to me!. Also you can follow me on Twitter, and definitely tag me @muhammad_o7. when you share this post on twitter

Originally Published here

Using Commandline To Process CSV files

Muhammad — Mon, 04 Sep 2023 03:49:30 +0000

The Command Line is a powerful tool for processing data. With the right combination of commands, you can quickly and easily manipulate data files to extract the information you need. In this blog post, we will explore some of the ways you can use the command line to process data.

One of the key benefits of using the command line to process data is its flexibility. The command line provides a wide variety of tools and utilities that can be used to perform a wide range of data processing tasks. For example, you can use the awk command to extract specific fields from a delimited data file, or you can use the sort command to sort a file based on the values in a particular column.

Another benefit of the command line is its scriptability. Because the command line is a text-based interface, you can easily create scripts that combine multiple commands to perform complex operations on data files. This can be particularly useful for automating repetitive tasks, such as cleaning up data files or performing data transformations.

The command line also offers a high level of control over the data processing process. Because you have direct access to the data files and the tools that are used to process them, you can easily fine-tune the behavior of the commands and customize the output to suit your specific needs.

Overall, the command line is a powerful and flexible tool for processing data. With the right combination of commands and scripts, you can easily manipulate data files to extract the information you need. Whether you are a data scientist, a system administrator, or a developer, the command line offers a wealth of opportunities for working with data.

Here are some oneliners which can help you get started with processing data simply by using commandline

To print the first column of a CSV file:

awk -F, '{print $1}' file.csv

To print the first and third columns of a CSV file:

awk -F, '{print $1 "," $3}' file.csv

To print only the lines of a CSV file that contain a specific string:

grep "string" file.csv

To sort a CSV file based on the values in the second column:

sort -t, -k2 file.csv

To remove the first row of a CSV file (the header row):

tail -n +2 file.csv

To remove duplicates from a CSV file based on the values in the first column:

awk -F, '!seen[$1]++' file.csv

To calculate the sum of the values in the third column of a CSV file:

awk -F, '{sum+=$3} END {print sum}' file.csv

To convert a CSV file to a JSON array:

jq -R -r 'split(",") | {name:.[0],age:.[1]}' file.csv

To convert a CSV file to a SQL INSERT statement:

awk -F, '{printf "INSERT INTO table VALUES (\"%s\", \"%s\", \"%s\");\n", $1, $2, $3}' file.csv

Lastly, these are just a few examples of the many things you can do with these oneliners to process CSV data. With the right combination of commands, you can quickly and easily manipulate CSV files to suit your needs.

I hope you enjoyed reading this post and got a chance to learn something new. If you have any oneliner when it comes to processing data especially CSV files feel free to comment below.

I write occasionally feel free to follow me on twitter

Originally published here

Grep and Log Analysis

Muhammad — Fri, 01 Sep 2023 06:09:11 +0000

I recently began a new role as a Software Engineer, and in my current position, I spend a lot of time in the terminal. Even though I have been a long-time Linux user, I embarked on my Linux journey after becoming frustrated with setting up a Node.js environment on Windows during my college days. It was during that time that I discovered Ubuntu, and it was then that I fell in love with the simplicity and power of the Linux terminal. Despite starting my Linux journey with Ubuntu, my curiosity led me to try other distributions, such as Manjaro Linux, and ultimately Arch Linux. Without a doubt, I have a deep affection for Arch Linux. However, at my day job, I used macOS, and gradually, I also developed a love for macOS. Now, I have transitioned to macOS as my daily driver. Nevertheless, my love for Linux, especially Arch Linux and the extensive customization it offers, remains unchanged.

Anyways, In this post, I will be discussing grep and how I utilize it to analyze logs and uncover insights. Without a doubt, grep has proven to be an exceptionally powerful tool. However, before we delve into grep, let's first grasp what grep is and how it works.

What is grep? and How it Works?

grep is a powerful command-line utility in Unix-like operating systems used for searching text or regular expressions (patterns) within files. The name "grep" stands for "Global Regular Expression Print." It's an essential tool for system administrators, programmers, and anyone working with text files and logs.

How it works?

When you use grep, you provide it with a search pattern and a list of files to search through. The basic syntax is:

grep [options] pattern [file...]

Here's a simple understanding of how it works:

Search Pattern: You provide a search pattern, which can be a simple string or a complex regular expression. This pattern defines what you're searching for within the files.
Files to Search: You can specify one or more files (or even directories) in which grep should search for the pattern. If you don't specify any files, grep reads from the standard input (which allows you to pipe in data from other commands).
Matching Lines: grep scans through each line of the specified files (or standard input) and checks if the search pattern matches the content of the line.
Output: When a line containing a match is found, grep prints that line to the standard output. If you're searching within multiple files, grep also prefixes the matching lines with the file name.
Options: grep offers various options that allow you to control its behavior. For example, you can make the search case-insensitive, display line numbers alongside matches, invert the match to show lines that don't match, and more.

Backstory of Development

grep was created by Ken Thompson, one of the early developers of Unix, and its development dates back to the late 1960s. The context of its creation lies in the evolution of the Unix operating system at Bell Labs. Ken Thompson, along with Dennis Ritchie and others, was involved in developing Unix in the late 1960s. As part of this effort, they were building tools and utilities to make the system more practical and user-friendly. One of the tasks was to develop a way to search for patterns within text files efficiently.

The concept of regular expressions was already established in the field of formal language theory, and Thompson drew inspiration from this. He created a program that utilized a simple form of regular expressions for searching and printing lines that matched the provided pattern. This program eventually became grep. The initial version of grep used a simple and efficient algorithm to perform the search, which is based on the use of finite automata. This approach allowed for fast pattern matching, making grep a highly useful tool, especially in the early days of Unix when computing resources were limited.

Over the years, grep has become an integral part of Unix-like systems, and its functionality and capabilities have been extended. The basic concept of searching for patterns in text using regular expressions, however, remains at the core of grep's functionality.

`grep` and Log Analysis

So you might be wondering how grep can be used for log analysis. Well, grep is a powerful tool that can be used to analyze logs and uncover insights. In this section, I will be discussing how I use grep to analyze logs and find insights.

Isolating Errors

Debugging often starts with identifying errors in logs. To isolate errors using grep, I use the following techniques:

Search for Error Keywords: Start by searching for common error keywords such as "error", "exception", "fail" or "invalid" . Use case-insensitive searches with the -i flag to ensure you capture variations in case.
Multiple Pattern Search: Use the -e flag to search for multiple patterns simultaneously. For instance, you could search for both "error" and "warning" messages to cover a wider range of potential issues. 3.Contextual Search: Use the -C flag to display a certain number of lines of context around each match. This helps you understand the context in which an error occurred.

Tracking Down Issues

Once you've isolated errors, it's time to dig deeper and trace the source of the issue:

Timestamp-Based Search: If your logs include timestamps, use them to track down the sequence of events leading to an issue. You can use grep along with regular expressions to match specific time ranges.
Unique Identifiers: If your application generates unique identifiers for events, use these to track the flow of events across log entries. Search for these identifiers using grep.
Combining with Other Tools: Combine grep with other command-line tools like sort, uniq, and awk to aggregate and analyze log entries based on various criteria.

Identifying Patterns

Log analysis is not just about finding errors; it's also about identifying patterns that might provide insights into performance or user behavior:

Frequency Analysis: Use grep to count the occurrence of specific patterns. This can help you identify frequently occurring events or errors.
Custom Pattern Matching: Leverage regular expressions to define custom patterns based on your application's unique log formats.
Anomaly Detection: Regular expressions can also help you detect anomalies by defining what "normal" log entries look like and searching for deviations from that pattern.

Conclusion

In the world of debugging and log analysis, grep is a tool that can make a significant difference. Its powerful pattern matching capabilities, combined with its versatility in handling regular expressions, allow you to efficiently isolate errors, track down issues, and identify meaningful patterns in your log files. With these techniques in your toolkit, you'll be better equipped to unravel the mysteries hidden within your logs and ensure the smooth operation of your systems and applications. Happy log hunting!

Remember, practice is key. The more you experiment with grep and apply these techniques to your real-world scenarios, the more proficient you'll become at navigating through log files and gaining insights from them.

Examples

Isolating Errors:

Search for lines containing the word "error" in a log file:

grep -i "error" application.log

Search for lines containing either "error" or "warning" in a log file:

grep -i -e "error" -e "warning" application.log

Display lines containing the word "error" along with 2 lines of context before and after:

grep -C 2 "error" application.log

Tracking Down Issues:

Search for log entries within a specific time range (using regular expressions for timestamp matching):

grep "^\[2023-08-31 10:..:..]" application.log

Search for entries associated with a specific transaction ID:

grep "TransactionID: 12345" application.log

Count the occurrences of a specific error:

grep -c "Connection refused" application.log

Identifying Patterns:

Count the occurrences of each type of error in a log file:

grep -i -o "error" application.log | sort | uniq -c

Search for log entries containing IP addresses:

grep -E "[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+" application.log

Detect unusual patterns using negative lookaheads in regular expressions:

grep -E "^(?!.*normal).*error" application.log

Lastly I hope you enjoyed reading this and got a chance to learn something new from this post and if you have any grep tips or how you started your linux journey feel free to comment below as I would love to hear it.

If you loved this post, you can always support my work by buying me a coffee. our support would mean the world to me! Also, if you end up sharing this on Twitter, definitely tag me @muhammad_o7.

Exploring Linux File System

Muhammad — Mon, 29 Mar 2021 17:58:43 +0000

What is File System and How they work?

In this post I will mostly explore linux file system and its directory structure but In order to explore linux file system first we need to understand what's a file system and how do they work?. In simple words, A filesystem is way in which files are named and logically placed in the computer for storage and retrieval.
Without a filesystem, information placed in a storage medium would be one large body of data with no way to tell where it begins or stops and this is one of the reasons why having a file system is important as it keeps the data organized and makes it easier for the computer to retrieve the data.

A file system consists of two or three layers, sometimes these layers are explicitly separated, and sometimes the functions are combined.

Logical File System: This layer is responsible for interaction with user applications such as providing with an api that allows functions such as
OPEN,CLOSE and READ.These functions passes through this layer for processing.
The logical file system also contains metadata of the file and directory for instance you can run ls -la -h to see file metadata such as
name,size,file permissions and etc.
if the program doesn't have access to a file or directory then this layer will throw an error. This layer provides file access,
directory operations and security.
VFS(Virtual File System): This is an optional layer but you can think of VFS as a layer on the top of real file system or an interface between the kernel and a concrete file system. This allows you to access files across different filesystems. For example Windows can access Linux filesystem and vice versa without having to know which filesystem is being accessed.
Physical File System: This layer is concerned with how physical blocks are being written or read. It handles memory management and buffering. It handles physical placement of the blocks in specific location of the storage medium. In simple words this layer decides where on the hard drive your files are supposed to be saved and how space should be saved by efficiently placing the files in the right location. Last but not least this layer mostly interacts with device drivers which allows saving on the storage medium such as HDD or SSD.

Linux File System.

After understanding how a filesystem works now I will be covering specifically linux file system and its directory structure, explaining how the data is stored and kept in linux. Linux supports many other filesystems such as ext3,
ext4 ,btrfs and many more. It supports around 100 types of filesytems and even the old ones but the most common filesystem among linux distributions is
ext4.

The Directory Structure.

In Linux and its many other distributions, A directory is structured in a tree like hiearchy system. Linux directory structure is well defined and documented in the FHS (File Hiearchy Standard). FHS is maintained by the Linux Foundation and its followed by major linux distributions.

The root / is the top of the filesystem (basically it contains everything that is required or used by the OS).

/
├── bin -> usr/bin
├── boot
├── dev
├── etc
├── home
├── lib -> usr/lib
├── lib64 -> usr/lib
├── lost+found
├── mnt
├── opt
├── proc
├── root
├── run
├── sbin -> usr/bin
├── snap -> /var/lib/snapd/snap
├── srv
├── sys
├── tmp
├── usr
└── var

20 directories, 0 files

As you can see how root directory represented using (/) contains everything.

Note: You can also think of directory as file that holds bunch of addresses to other files.

I will be covering each directory separately explaining what each directory holds and represents.

`bin/`

This directory is called binaries as this holds all the programs that live in our machine. For example here's what a bin/ directory might look like and it mostly contains executables.

➜  ~ tree -L 1 /bin -C | tail -n 20
├── znew
├── zonetab2pot.py
├── zoom -> /opt/zoom/ZoomLauncher
├── zramctl
├── zresample
├── zretune
├── zsh
├── zsh-5.8
├── zsoelim -> soelim
├── zstd
├── zstdcat -> zstd
├── zstdgrep
├── zstdless
├── zstdmt -> /usr/bin/zstd
├── zvbi-atsc-cc
├── zvbi-chains
├── zvbid
└── zvbi-ntsc-cc

`boot/`

This directory contains all the files required by the kernel at the time of boot and our bootloader also resides in this directory.

/boot
├── grub
├── initramfs-linux-fallback.img
├── initramfs-linux.img
├── initramfs-linux-lts-fallback.img
├── initramfs-linux-lts.img
├── intel-ucode.img
├── lost+found
├── vmlinuz-linux
└── vmlinuz-linux-lts

As you can see grub is also present in the boot/ directory

`sbin/`

This directory contains the system binaries that are required for the system adminstration.

`dev/`

This directory contains all the devices and each device is represented with a file. This only represents devices attached to the system. For example your disk might show up as dev/sda and partition might show up as dev/sda1. This folder is usually accesed by the drivers and applications.

`etc/`

This directory is known as edit to configure but it was also known as
et cetera as you can read here more about the history of
etc/ directory name.

etc/ directory is where all your configs are stored for software used by the system. For example ,package manager such as pacman or apt as etc/ holds the config of your package manager.

➜  ~ tree -L 1 /etc | grep "pacman"
├── pacman.conf

`lib/`

This directory is also known as libraries and it contains all the libraries that are required to boot the system and used by different applications to perform different functions.

`media & mnt/`

This directory contains the mounted drives. For example this is where you will find an another mounted drive such as External HDD or SSD. If you are mounting things manually use the mnt/ directory and media/ is where your
OS will automatically mount.

`opt/`

This drive contains all the manually installed software usually from the vendor.

Note: Some software installed from package manager might also live here

➜  ~ tree -L 1 /opt
/opt
├── Simplenote
├── sublime_text_3
└── zoom

For example you can see zoom and sublime text.

`proc/`

In this directory you will mostly find files which contains the information of the hardware and even the running processes in the system. Each process is represented by a directory in proc/.

For example spotifyd (spotify daemon) represented as a process in proc/

➜  ~ ps aux | grep "spotifyd"
hackerm+   98147  0.0  0.1 365184 13324 ?        Ssl  07:15   0:00 /usr/bin/spotifyd --no-daemon

➜  ~ tree -L 1 /proc | grep "98147"
├── 98147

`root/`

This folder is basically considered as the home/ folder of the root user and it is only accessed by the user with root permission.

`run/`

This is a fairly new folder and different linux distributions use this in different ways. This folder is basically mounted as temporary filesystem (tmpfs) as this folder is wiped when the system is rebooted or shutdown and it contains programs required early in the boot procedure.

➜  ~ tree -L 1 /run
/run
├── credentials
├── cups
├── dbus
├── dhcpcd
├── dmeventd-client
├── media
├── mount
├── mysqld
├── named
├── NetworkManager
├── nscd
├── openvpn-client
├── user
├── utmp
└── wpa_supplicant

`srv/`

This is also known as service directory and this is where all the files stored that are accessed by external users when using ftp server.

`sys/`.

Its also known as the system folder and this folder contains files that interact with the kernel. This directory is created when the system boots up.

➜  ~ tree -L 1 /sys
/sys
├── block
├── bus
├── class
├── dev
├── devices
├── firmware
├── fs
├── hypervisor
├── kernel
├── module
└── power

`tmp/`

This directory is known as temporary directory where files are stored by the applications that can be stored during a session. For example a word processor like libreoffice might store temporarily if the program crashes or system reboots.

`usr/`

This directory contains shareable, read-only files, including executable binaries and libraries, man files, and other types of documentation.

`var/`

This directory contains files or directories that are expected to grow and that's why its know as variable folder. For example you can find logs of
databases,webservers emailboxes and etc.

`home/`

Home directory is the storage for user files. Each user has a subdirectory in /home. This is where you will find application settings as hidden directories for example browser cache .cache/ and this is where your dotfiles live.

I have been using linux for more than 4 years now and as a linux user I have always been intrigued to explore the linux file system and its directory structure. This posts aims towards high level explanation of how filesystem works,linux file system and its directory structure. I do not intend to go over in detail about a specific filesystem such as ext4 but that's a topic for my another blog post.

I hope you enjoyed reading this post and got to learn something new. If you think I missed anything you can always DM on twitter or EMAIL ME. Last but not least I would love to hear your thoughts.

Originally Posted Here

Everything you need to know for SAA Exam

Muhammad — Fri, 18 Sep 2020 16:45:02 +0000

Solutions Architect Associate Exam
Exam Domains
IAM (Identity Access Management)
- IAM Authentication Methods.
- MFA (MultiFactor Authentication)
- STS (AWS Security Token)
AWS Global Infrastructure Overview
VPC (Virtual Private Cloud)
EC2 (Elastic Compute Cloud)
- Security Groups.
- Instance Metadata
- Instance Userdata
- Status Checks and Monitoring
- Public,Private and Elastic IP addresses.
- Private Subnets and Bastion Hosts
- NAT Instances and NAT Gateways
- EC2 Placement Groups
- Few Notes
Elastic Load Balancing and Auto Scaling.
- Elastic Load Balancing
- Application Load Balancer
- Network Load Balancer
- Classic Load Balancer
- Internet Facing VS Internal
- Elastic Load Balancing
- ELB Security Groups
- ELB Monitoring
- EC2 Auto Scaling
- EC2 Autoscaling - Scaling Types
- EC2 Autoscaling - Termination Policy
Virtual Private Cloud
- Amazon VPC Components.
- Amazon VPC - Routing
- Amazon VPC - Subnets
- Amazon VPC - Internet Gateways.
- Amazon VPC - Secuirty Groups
- Amazon VPC - Network ACLs
- Amazon VPC - Connectivity
Route 53
- Route 53 Hosted Zones
- Route 53 Health Checks
- CNAME vs Alias
- Route 53 Routing Policies
- Route 53 Traffic Flow
- Route 53 Resolver
- AWS Global Accelarator
Amazon S3
- Amazon S3 Buckets
- Amazon S3 Objects:
- Amazon S3 Sub-resources
- Amazon S3 Storage Classes
- Amazon S3 Multipart upload
- Amazon S3 Copy
- Amazon S3 Transfer Accelaration
- Amazon S3 Encryption
- Amazon S3 Performance
Amazon CloudFront
- Amazon CloudFront Origins
- Amazon CloudFront Distributions
- Amazon CloudFront Charges
Amazon EBS (Elastic Block Store)

Solutions Architect Associate Exam

Its multiple choice and multiple response questions.
130 mins to complete the exam.
It contains 65 questions and costs $150 dollars.
It requires 720 in order to pass the exam.

Exam Domains

The Exam consists of following domains.
Design Resilient Architectures
- Design a multi-tier architecture solution.
- Design highly available or/ fault-tolerant architectures.
- Design decoupling mechanisms using AWS services.
- Choosing appropiate resilient storage.
Design High-Performing Architectures
- Identify elastic and scalable compute solutions for the workload.
- Selecting high performance and scalable storage solution for a workload.
- Selecting high performance networking solutions for a workload.
- Choosing high performance database solutions for the workload.
Design Secure Applications and Architectures
- Designing secure access to AWS resources.
- Designing secure applications tiers.
- Selecting appropiate data security options
Design Cost-Optimized Architectures
- Identify cost-effective storage solutions
- Identify cost-effective compute and database services.
- Design cost-optimized network architectures

IAM (Identity Access Management)

Its a service that provides users,groups,IAM policies.
IAM USER: Its an entity that represents a person or a service and you associate IAM Policy directly with the user and it defines its permissions and what the user is allowed to do within AWS environment. Furthermore, a user can be assigned the following.
- An access key-pair that allows user programmatic access to the AWS API,CLI,SDK and other development tools.
- A password for access to the management console.
- By default users can't do anything within their accounts.
- the account user crendentials are usually the email address used to create the account and a password.
- Root account has full admin priviledges and you can think of it as sudo.
- The best practice is to not use the root crendentials instead create an IAM user assign admin priviledges
- Never share root crendentials.
- Make sure you enable MFA.
- IAM users can be created to represent applications and these are known as service accounts
- You can have upto 5000 users per AWS account.
- Each user account has a friendly name and an ARN(Amazon Resource Name) which uniquely identifies the user across AWS.
- You should always create individual IAM accounts for the users(Not to share them).
- A password policy can be defined for users enforcing them to have stronger passwords (applies to all users)
IAM GROUP: Its a collection of users that have policies attached to them such as group for developers,sys-admins .
- Its not an identity and cannot be identified as principal in an IAM policy.
- use groups to assigns permissions to the users.
- always assign the least priviledges when assigning permissions.
IAM ROLES: Think of it as assigning access to the AWS services such as you might set a role for DynamoDB to readonly.
- They are created and then assumed by trusted entities and define a set of permissions for making AWS requests.
- with roles you can delegate permissions to resources for users and services without using a permanent credentials.
- AWS users or services can assume a role to obtain temporary security crendentials that can used to make aws api calls.
IAM Policies: They are documents that defines the permissions and can be applied to users,groups and roles.
- Policy documents are written in json.
- All permissions are implicitly denied by default.
- the most restrictive policy is applied.
- The IAM policy simulator is a tool that helps you understand,test and validate the effects of access controls policies.

IAM Authentication Methods.

You can use key-pair access keys and its used for programmatic access especially CLI (You can't add MFA to this).
- A combination of access key ID and secret key access.
- this is used to make programmatic calls to aws when using the api. For example boto. Its also used to access AWS using CLI.
- you can create,modify,view or rotate access keys.
- When created IAM returns the access key ID and secret access key.
- The secret access key is returned only at the creation time and if lost new key must be created.
- Make sure access keys and secret access keys are stored securely.
- Users can be given access to change their own keys through IAM policy(Not from console).
- You can disable user's access key which prevents it from being used for API calls.
Simple username and password method to access the management console.
- The password that user uses to sign in into aws web console
Some AWS services uses signing certificate.
- SSL/TLS certificates that can be used to authenticate with some AWS services.
- AWS recommends that you use ACM(AWS Certificate Manager) to provision manage and deploy your server certificates.
- You can also use IAM only when you need to support HTTPS connections in a region that is not supported by ACM.

MFA (MultiFactor Authentication)

Having physical token or soft token that will allow you to access the AWS
- Soft token can be Google Authenticator
- Hard Token can be YuBi Key.
- AWS also provides soft token and physical access keys for MFA.
- By having two factors of authentication it makes it very secure.

STS (AWS Security Token)

STS is a web service that enables you to request temporary, limited-priviledge crendentials for IAM users or for the users that you authenticate (federated users).
By default AWS STS is available as global service and all AWS STS requests go to a single endpoint at https://sts.amazonaws.com
All regions are enabled by default for STS but can be disabled.
The region in which temporary crendentials are requested must be enabled.

AWS Global Infrastructure Overview

Region: A geographical area with 2 or more AZs, isolated from other AWS regions. There are 23 regions around the world at the moment.
Availability Zone: One of more data centers that are physically separate and isolated from other AZs.
Edge Location: A location with Cache Content that can be delivered at low latency to the users. Mostly used by CloudFront for delivering content such as static files or video content.
Regional Edge Cache: Also part of the CloudFront network.These are larger caches that sit between AWS services and Edge Locations.
Global Network: Highly Available, low latency private global network interconnecting every data center, AZ and AWS region.

VPC (Virtual Private Cloud)

Its an isolated section of AWS where you can launch your own resources.
Within VPC you can create your own networks with your own IP ranges.
A VPC sits within a region and you create an VPC within that region and then you create subnets within that regions that sits within AZs. The subnets can public or private and then we launch resources within those subnets. You can have 5 VPCs within the region by default.
A VPC Router is used to communicate within subnets and availablity zones and it has a route table that we can configure and it has an IP address range. Basically every VPC has a CIDR(Classless Inter-Domain Routing) block.
You define the IP range for your VPC.
You can also attach internet gateway to your VPC that sends requests to the outside internet and for that we need igw-id and IP-ADDR as destination.
Internet Gateways allows you to make requests to the public internet and for that we have to add the entry to the route table.
Each VPC has its own CIDR block.

EC2 (`Elastic Compute Cloud`)

Its an elastic service that allows you to launch compute resources on the AWS cloud. In AWS context we call EC2 instances but you can think of them as virual machines.
Each instance has an operating system,storage and virtual hard drive.
When launching instances you can choose from AWS MarketPlace and Community AMIs.
You can connect to EC2 instances using ssh.

Security Groups.

These are firewalls that are applied at the instance level.
They monitor traffic going in and out of EC2 instances.
You can have multiple instances in a security group and you can have multipe security groups applied to the instances.
Security groups are stateful.
For example having a security group with port 22 access applied to the ec2 instances will allow you to launch secure shell.

Instance Metadata

Instance metadata is data about your instance that can be used to configure or manage the running instance. Its divided into categories and gives you the information about your instance such as hostname,ami-id and etc.

You can run this command within your ec2 in commandline to get the meta data

curl http://169.254.169.254/latest/meta-data/

Instance Userdata

Its basically the information that you can pass into instance when it boots up.

Think of it as a bash script contains all the commands that you need to run when booting up the ec2 instance.
This can be your regular ubuntu apt commands. For example


#!bin/bash
sudo apt-get upgrade
sudo apt install xyz

You can basically paste this into userdata located in advance details when launching an instance.

you can also make a request to userdata by following command.

curl http://169.254.169.254/latest/meta-data/

Status Checks and Monitoring

You can setup cloudwatch alarms on ec2 instances to help monitor the instances effectively and futhermore you can also install stress tool to perform stress testing on the instances.

Public,Private and Elastic IP addresses.

Public IP Address:
- Lost when the instance is stopped.
- Used in public subnets.
- No Charge
- Associated with private IP address on the instance.
- Cannot be moved between instances.
Private IP Address:
- Retained when the instance is stopped.
- Used in public and private subnets.
Elastic IP Address:
- Static Public IP Address.
- You are charged if not used.
- Associated with a private IP address on the instance.
- Note: Elastic Fabric Adapter is a network device that you can attach to reduce latency and increase throughput for distributed HPC(High Performance Computing)

Private Subnets and Bastion Hosts

Public Subnets are easily accessible through public internet
Private Subnet route table doesn't have the igw route and its not configured to provide public ip addresses to the instances launched into this.
There's no way to directly manage this instance through the internet.
A bastian host is a public instance that you used to jump to private instance and it is also known as jump host and through bastian host you will be able to ssh into your private instance.
We use agent forwarding and use the bastian host to connect to the private subnet instance.

NAT Instances and NAT Gateways

NAT Instance: Network Address Translation(NAT).Its basically a process of taking the private ip address and translating it to public ip address so it allows you to connect to the public internet.
- It's managed by you
- The only way to scale this is to do it manually which means using a powerful and bigger instance with more resources and enhanced networking with additional bandwith.
- There's no high availability and it has to be done manually.
- Need to assign security groups.
- Can be used as bastion host.
- You can use an Elastic IP address or a public address with a NAT instance.
- Can implement port forwarding manually
NAT Gateway: A better version of NAT Instance.
- Its managed by AWS
- Its elastically scaled upto 45 GBps
- Provides automatic high availability within an AZ and can be placed in multiple AZs.
- No security groups
- Cannot be accessed through ssh
- Choose the Elastic IP address to associate with a NAT instance gateway at creation.
- Doesn't support port forwarding.
Lastly Private Subnet contains nat-gateway-id instead of igw-id and anything that isn't defined within ip cidr block of private subnet route table is handled by the nat-gateway-id.

EC2 Placement Groups

Cluster: packs instances close together inside an Availability Zone. This strategy enables workloads to achieve the low latency network performance necessary for tightly coupled node to node communication that is typical of HPC applications.
- WHAT: Instances are placed into a low latency group within a single AZ
- WHEN: Need a low network latency or high network throughput.
- Pros: Get most out of enhanced networking instances.
- Cons: Finite capacity recommends launching all you might need upfront.
Partition: spreads your instances across logical partitions such that groups of instances in once partition do not share the underlying hardware with groups of instances in different partitions.This strategy is typically used by large distributed and replicated workloads such as Hadoop,Cassandra and Kafka.
- WHAT: Instances are grouped into logical segments called partitions which use distinct hardware.
- WHEN: Need control and visibility into instance placement.
- Pros: Reduces likelihood of correlated failures for large workloads.
- Cons: Partition placement groups are not supported for dedicated hosts.
Spread: strictly places a small group of instances across distinct underlying hardward to reduce correlated failures.
- WHAT: Instances are spread across underlying hardware.
- WHEN: Reduce the risk of simultaneous instance failure if underlying hardware fails.
- Pros: Can span multiple AZs
- Cons: Maximum upto 7 instances running per group,per AZ.

Few Notes

Amazon EC2:
- Its a compute cloud basically a web service that provides resizeable compute capacity in the cloud.
- With EC2 you have full control of the operating system layer.
- You use key-pair to securely connect to ec2 instances using ssh.
- A keypair consists of public key that AWS stores and private key that we store on our local machine.
- user-data is a script that you provide when starting an instance.
- meta-data is the data of your instance that you can use to configure the instance.
EC2 Pricing Models:
- On Demand:
  - No upfront fee.
  - Charged per hour or second
  - No Commitment
  - Ideal for short term needs or unpredictable workloads
- Reserved:
  - Options: No Upfront,Partial Upfront or all Upfront.
  - Charged by hour or second.
  - 1year or 3year commitment.
  - Ideal for steady-state workloads and predictable usage.
- Spot:
  - No upfront fee
  - Charged by hour or second
  - No commitment
  - Ideal for cost sensitive, compute sensitive use cases that can withstand interruption. (Batch Processing)
- Dedicated Hosts:
  - Good for enterprise customers who are looking for isolated environments
Amazon EC2 AMIs:
- An Amazon Machine Image provides the information required to launch an instance
- An AMI includes the following
  - A template for the root volume for the instance (OS,system,an application server and applications).
  - Launch permissions that control which AWS accounts can use the AMI to launch instances.
  - A block device mapping that specifies the volumes to attach to the instance when its launched (which EBS to attach to the instance).
- Volumes attached to the instances are either EBS or Instance store.
  - Amazon EBS provided persistent store.EBS snapshots resides on Amazon S3 are used to create the volume.
  - Instance store volumes are ephermeral(non-persistent).this means data is lost when the instance is shut down/
- AMIs are regional.You can only launch an AMI from the region in which it is stored. However you can copy AMI's to other regions using console,CLS or API.
Elastic Network Interface (ENI):
- A logical networking component in a VPC that represents a virtual network card.
- Can include attributes such as IP addresses,security groups,MAC addresses, source/destination check flag,description.
- You can create and configure network interfaces in your account and attach them to your instances in VPC.
- eth0 is the primary network interface and cannot be moved or detached.
- An ENI is bound to an availability zone and you can specify which subnet/AZ you want the ENI to be loaded in.
Elastic Fabric Adapter (EFA):
- An AWS Elastic Network Adapter (ENA) with added capabilities.
- Enables customers to run applications requiring high levels of internode communications at scale on AWS.
- With EFA, High Performance Computing (HPC) applications using the Message Passing Interface (MPI) and Machine Learning (ML) applications using NVIDIA Collective Communications Library (NCCL) can scale upto thousands of CPUs or GPUs.
ENI vs ENA vs EFA:
- When to use ENI: This is the basic adapter type for when you don't have any high performance requirements. Can use with all instance types.
- When to use ENA: Good for use cases that require higher bandwidth and lower inter-instance latency. Supported for limited instance types (HVM only).

Elastic Load Balancing and Auto Scaling.

Elastic Load Balancing

Load Balancing refers to efficiently distributing incoming network traffic across a group of backend servers.

Application Load Balancer

Operates at the request level.
Routes based on the content of request (Layer 7)
Supports path based routing,host based routing,query string parameter based routing and source IP address based routing.
Supports IP addresses, Lambda Functions and containers as targets.

- `HTTPS`,`HTTP`

Network Load Balancer

Operates at the connection level.
Routes connections based on IP protocol Data (layer 4)
Offers ultra high performance,low latency,and TLS offloading at scale.
Can have static IP / Elastic IP
Supports UDP and static IP addresses as targets.

Classic Load Balancer

Old generation; not recommended for new applications.
Performs routing at Layer 4 and Layer 7
Use for existing applications running on EC2-Classic instance

Internet Facing VS Internal

Internet Facing:
- ELB nodes have public IPs.
- Routes traffic to the private IP addresses of the EC2 instances.
- Need one public subnet in each AZ where ELB is defined.
- ELB dns name format <name>-<id-number>.<region>.elb.amazonaws.com
Internal only ELB:
- ELB nodes have private IPs.
- Routes traffic to the private IPs of the EC2 instances.
- ELB dns name format internal-<name>-<id-number>.<region>.elb.amazonaws.com

Elastic Load Balancing

EC2 instances and containers can be registered against an ELB
ELB nodes use IP addresses within your subnets, ensure at least a /27 subnet and make sure there are at least 8 IP addresses available in that order for the ELB to scale.
An ELB forwards traffic to eth0 (primary IP address).
An ELB listener is the process that checks for the connection requests:
- Listeners for CLB provide options for TCP and HTTP/HTTPS.
- Listeners for ALB only provide options for HTTPS and HTTP
- Listeners for NLB only provide only TCP as an option ### ELB Security Groups
Security Groups control the ports and protocols that can reach the front-end listener.
You must assign a security group for the ports and the protocols on the front end listener.

ELB Monitoring

CloudWatch every 1 min.
ELB service sends information when requests are active.
Access Logs:
- Disabled by Default.
- Includes information about the client(not included in the Cloud Watch Metrics)
- Can identify requester IP,request type etc.
- Can be optionally stored and retained in S3. ### EC2 Auto Scaling
You can attach one or more classic ELBs to your existing Auto Scaling Groups.
You can attach one or more Target Groups to your ASG to include instances behind an ALB.
The ELBs must be in same region.
Launch configuration is the template used to create new EC2 instances and includes parameters such as instance family, instance type,AMI,keypair and security groups.

Scaling Option	What is it?	When to use?
Maintain	Ensures the required number of instances are running	Use when you always need a known number of instances running at all times
Manual	Manually change the desired capacity via console or CLI	Use when your needs change rarely enough that you are Ok! to make manual changes.
Schedule	Adjust Min/Max instances on specific dates/times or recurring time periods	Use when you know you are busy and quiet times are. Useful for ensuring enough instances are available before busy times
Dynamic	Scale in response to a system load or other triggers using metrics	Useful for changing capacity based on system usage eg if cpu hits 80%

EC2 Autoscaling - Scaling Types

Scaling	What is it?	When to use?
Target Tracking Policy	The scaling adds or removes capacity as required to keep the metric at or close to the specified target value	A use case,when you want to keep the aggregate CPU usage of your ASG at 70%
Simple Scaling Policy	Waits until health check and cool down period expires before revaluating	This is more conservative way to add/remove instances.Useful when load is eratic.AWS recommend step scaling instead of simple in most cases
Step Scaling Policy	Increase or decrease the current capacity of your Auto Scaling group based on a set of scaling adjustments known as step adjustments	Useful when you want to vary adjustments based on the size of the alarm breach

Can also scale based on AWS SQS.
Uses a custom metric that's sent to Amazon Cloud Watch that measures the number of messages in the queue per EC2 instance in the auto scaling group.
Then use a target tracking policy that configures your ASG to scale based on the custom metric and a set target value. Cloud watch alarms invoke the scaling policy.
Use a custom backlog per instance metric to track not just the number of messages in the queue but the number available for retrieval.

EC2 Autoscaling - Termination Policy

Termination policies control which instances are terminated first when scale in event occurs.
There is default termination policy and options for configuring your own customized termination policies.
The default termination policy is designed to help ensure that instances span AZs evenly for High Availability Zones.
The default policy is kept generic and flexible to cover a range of scenarios.

Virtual Private Cloud

A Virtual Private Cloud (VPC) is logically isolated from other VPCs on AWS.
VPCs are regions specific
A default VPC is created in each region with a subnet in each AZ.
You can define dedicated tenancy for a VPC to ensure instances are launched on a dedicated hardware.
The default VPC has all public subnets.
Public Subnets:
- Auto Assign public IPv4 address set to Yes
- The subnet route table has an attached Internet Gateway.
Instances in the default VPC always have both a public and private IP addresses.
AZ names are mapped to different zones for different users.

Amazon VPC Components.

VPC: A logically isolated virtual network in the AWS cloud. You define VPC's IP address space from ranges you select.
Subnet: A segment of a VPC's IP address range where you can place groups of isolated resources (maps to sinlge AZ).
Internet Gateway: The Amazon VPC side of a connection to the public internet.
NAT Gateway: A highly available,managed Network Address Translation (NAT) service for your resources in a private subnet to access the internet.
Hardware VPN Connection: A hardware based VPN connection between your AWS VPC and your data center,home network, or co location facility.
Virtual Private Gateway: The VPC side of an VPN Connection.
Customer Gateway: Our side of a VPN Connection.
Router: Routers interconnect subnets and direct traffic between Internet gateways,virtual private gateways, NAT gateways and subnets.
Peering Connection: A peering connection allows you to route traffic via private IP addresses between two peered VPCs
VPC Endpoints: Enables private connectivity to services hosted in AWS from within VPC without using an Internet Gateway,VPN,NAT Devices or firewall proxies.
Egress-only Internet Gateway: A stateful gateway to provide egress only access for IPv6 traffic from the VPC to the internet.

Amazon VPC - Routing

The VPC router performs routing between AZs within a region.
The VPC router connects different AZs together and connects the VPC to the internet Gateway.
Each subnet has a route table the router uses to forward traffic withing the VPC.
Route tables also have entries to external destinations.

Amazon VPC - Subnets

Types of subnets:
- If a subnet's traffic is routed to an internet gateway the subnet is known as a public subnet.
- If a subnet doesn't have a route to the internet gateway the subnet is known as private subnet.
- The VPC is created with a master address range(CIDR block,can be anywhere from 16-28 bits) and subnet ranges are created within that range.
- New subnets are always associated with the default route table
- Once VPC is created you cannot change the CIDR block.
- You cannot created additional CIDR blocks that overlap with existing CIDR blocks
- You cannot create additional CIDR blocks in a different RFC 1918 range.
- Subnets with overlapping IP address ranges cannot be created
- The first 4 and last 1 IP address in a subnet are reserved.
- Subnets are created within AZs
- Subnets map 1:1 to AZs and cannot span AZs.

Amazon VPC - Internet Gateways.

An Internet Gateway serves two purposes:
- To provide a target in your vpc route tables for internet routable traffic.
- To perform network address translation(NAT) for instances that have been assigned public IPv4 addresses.
Internet Gateways must be created and then attached to a VPC, be added to a route table and then associated with the relevant subnets.
No availability risk or bandwidth constraints.
You cannot have multiple Internet Gateways in a VPC
Egress-Only internet Gateway provides outbound Internet Access for IPv6 addressed instances.

Amazon VPC - Secuirty Groups

Security Group act like a firewall at the instance level (network interface) level.
Can only assign permit rules in a security group, cannot assign deny rules.
All rules are evaluated until a permit is encountered or continues until the implicit deny.
Can control ingress and egress traffic.
Security groups are stateful
By default, custom security groups do not have inbound allow rules (all inbound traffic is denied by default).
By defualt, default security groups do have inbound allow rules (allowing traffic from within the group).
All outbound traffic is allowed by default in custom abd default security groups.
You cannot delete the security group that's created by default within a VPC.
You can use security group names as the source of destination in other security groups.
You can use the security group name as a source in its own inbound rules.
Secuirty group membership can be changed whilst instances are running.
Any changes made will take effect immediately.
You cannot block specific ip addresses using the security groups use NACLs instead.

Amazon VPC - Network ACLs

Network ACLs function at the subnet level.
With NACLs you can have permit and deny rules.
Network ACLs contain a numbered list of rules that are evaluated in order from the lowest number until the explicit deny.
Network ACLs have separate inbound and outbound rules and each rule can allow or deny traffic.
Network ACLs are stateless so responses are subject to the rules for the direction of traffic.
NACLs only apply to traffic that is ingress or egress to the subnet not to traffic within the subnet.
A VPC automatically comes with a default network ACL which allows all inbound/outbound traffic.
A custom NACL denies all traffic both inbound and outbound by default.
All subnets must be associated with a network ACL.
You can create custom network ACLs. By default each custom network ACL denies all the inbound and outbound traffic until you add rules.
You can associate a network ACL with multiple subnets; however a subnet can only be associated with one network ACL at a time.
Network ACLs do not filter traffic between instances in the same subnet.
NACLs are preffered option when it comes to blocking specific IPs or ranges.
Security groups cannot be used to block specific ranges of IPs.
NACL is the first line of defence, the secuirty group is the second.
Changes to NACL take immediately.

Security Group	Network ACL
Operates at the instance level	Operates at the subnet level
Support allow rules only	Supports allow and deby rules only
Stateful	Stateless
Evaluates all rules	Processes rules in order
Applies to an instance only if associated with a group	Automatically applies to all instances in the subnets its associated with

Amazon VPC - Connectivity

There are several methods of connecting to a vpc and these include
AWS Managed VPN
- What: AWS Managed IPSec VPN Connection over your existing internet.
- When: Quick and usually simple way to establish a secure tunnelled connection to a vpc; Redundant link for direct connect or other VPC VPN
- Pros: Supports static routes or BGP peering and routing
- Cons: Dependant on your internet connection.
- How: Create a virtual private gateway on AWS and Customer gateway on the on premise.
AWS Direct Connect
- What: Dedicated network connection over private lines straight into AWS backbone.
- When: Requires a large network link into AWS; lots of resources and services being provided on AWS to your coporate users
- Pros: More predictable network performance; potential bandwidth cost reduction; upto 10 GBps provisioned connections; supports BGP peering and routing.
- Cons: May require additional telecom and hosting provider relationships and /or network circuits; costly;takes time to provision.
- How: Work with your existing data networking provider;create virtual interfaces (VIFs) to connect to VPCs (Private VIFs) or other AWS services like s3 or glacier (public VIFs).
AWS Direct Connect plus a VPN
- What: IPSec VPN connection over private lines (DirectConnect).
- When: Need the added security of encrypted tunnels over direct connect.
- Pros: More secure (in-theory) than Direct Connect Alone.
- Cons: More complexity introduced by VPN Layer
AWS VPN CloudHub
- What: Connect location in the hub and spoke manner using AWSs VPC.
- When: Link remote offices for backup or primary WAN access to AWS resources.
- Pros: Reuses existing Internet Connections;supports BGP routes to direct traffic
- Cons: Dependant on Internet Connection; No inherent redundacny
- How: Assign multiple Customer Gateways to Virtual Private Gateway, each with their own BGP ASN and unique IP ranges.
Software VPN
- What: You provide your own VPN endpoint and software
- When: You must manage both ends of the vpn connection for compliance reasons or you want to use a VPN option not supported by AWS
- Pros: Ultimate flexibility and manageability
- Cons: You must design for an needed redundancy across the whole chain
- How: Install VPN software via marketplace appliance of on an EC2 instance.
Transit VPC
- What: Common strategy for connecting geographically dispersed VPCs and locations in order to create a global network transit center.
- When: Locations and VPC-deployed assests across multiple regions that need to communicate with one another.
- Pros: Ultimate flexibility and manageability but also AWS-managed VPN hub-and-spoke between VPCs
- Cons: You must design for any needed redundancy across the whole chain
- How: Providers like cisco,juniper networks and riverbed have offerings that work with their equipment and AWS VPC
VPC Peering:
- What: AWS provided network connectivity between two VPCs
- When: Multiple VPCs need to communicate or access each others resources.
- Pros: Uses AWS Backbone without traversing the internet.
- Cons: Transitive peering is not supported
- How: VPC peering request made; accepter accepts the request (either within or across the accounts)
AWS Private Link:
- What: AWS provided network connectivity between VPCs and or AWS services using interface endpoints.
- When: Keep private Subnets truly private by using AWS backbone to reach other AWS or Marketplace services rather than the public internet.
- Pros: Redundant;uses AWS backbone
- How : Create endpoint for required AWS or Marketplace service in all required subnets;access via provided DNS hostname.

VPC Endpoints:

	Interface Endpoint	Gateway Endpoint
What	Elastic Network Interface with a private IP	A gateway that is a target for a specific route
How	Uses DNS entries to redirect traffic	Uses prefix lists in the route table to redirect traffic
Which services	API Gateway,CloudFormation,CloudWatch	Amazon S3,DynamoDB
Security	Security Groups	VPC Endpoint Policies

Route 53

According to wikipedia, Amazon Route 53 is a scalable and highly available Domain Name System. It was lauched in December 2010 and has been part AWS since then.
Route 53 allows you register domain name for your service and it offers following functions:

Domain Name Registry.
DNS Resolution
Health Checks of the resources

Route 53 is located alongside of all edge locations and when you register a domain with Route 53 it becomes the authoritative DNS server for that domain and creates a public hosted zone.

Route 53 also you to transfer your domains to it as long as it supports TLD(Top Level Domain) is supported and you can even transfer to another registrar by contacting aws support.

You can also transfer a domain to another account in AWS however it does not migrate the hosted by default(optional) and its also possible to have domain registered in one aws account and hosted zone in the other aws account

Route 53 also allows you have to private DNS which lets you have authoritative DNS within your VPCs without exposing DNS records to the public internet.

Lastly you can use AWS Management Console or API to register new domain names with route 53

Route 53 Hosted Zones

A hosted zone is a collection of records for a specified domain.
A hosted zone is analogous to a traditional DNS zone file; it represents a collection of records that can be managed together.
There are two types of zones
- Public Hosted Zone: Determines how much traffic is routed on the internet
- Private Hosted Zone for VPC: Determines how traffic is routed within VPC (Not accessible outside of VPC).
For Private hosted zones you must set the following VPC settings to true
- enableDnsHostname
- enableDnsSupport You also need to create a DHCP options set.

Route 53 Health Checks

Health checks ensures the instance health by connecting to it.
Health can be pointed at:
- Endpoints (IP addresses or Domain Names)
- Status of other health checks
- Status of cloudwatch alarm

Route 53 supports most of the DNS record types; Alias record is specific to Route 53 and its pointed to DNS name of the service.

CNAME vs Alias

CNAME	ALIAS
Route 53 charges for CNAME queries	Route 53 doesn't charge for alias queries to AWS resources
You cannot create a CNAME record at the top of a DNS namespace (zone apex)	You can create ALIAS record at the zone apex (You cannot route to a CNAME at the zone apex)
A CNAME can point to any DNS record that is hosted anywhere	An alias record can only point to a CLoudFront distribution,Elastic BeanStalk,ELB,S3 bucket as a static site or to another record in the same hosted zone that you are creating the alias record in

Route 53 Routing Policies

Policy	What it does
Simple	Simple DNS response by providing the IP address associated with a name
Failover	If primary is down(based on health checks) routes to secondary destination
Geolocation	Uses geographic location you are in (eg US) routes to closest location
Geoproximity	Routes you to the closest region within geographic area
Latency	Directs you based on the lowest latency routes
Multivalue answer	Returns several IP addresses and functions as a basic load balancer
Weighted	Uses the relative weights assigned to resources to determine which route to

Simple
- An A record is mapped to one or more IP addresses.
- Uses round robin
- Does not support health checks.
Failover:
- Failover to a secondary IP address.
- Associated with a health check
- Used for active-passive
- Can be used with an ELB.
Geolocation:
- Caters to different users in different countries and different languages.
- Contains users within a specific geography and offers them a customized version of the workloads based on their specific needs.
- Geolocation can be used for localizing the content and presenting some or all of your website in the language of the users.
- Can also protect distribution rights.
- Can be used for spreading load evenly between regions.
- If you have multiple records for overlapping regions,Route 53 will route to the smallest geographic region.
Geoproximity:
- Use for routing the traffic based on the location of resources and optionally shift traffic from resources in one location to resources in another.
Latency Based Routing:
- AWS maintains a database of latency from different parts of the world.
- Focused on improving performance by routing to the region with the lowest latency.
Multi-value answer:
- Use for responding to the DNS queries with upto 8 healthy records selected at random.
Weighted:
- Similar to simple but you can specify a weight per IP address.
  - You create records that have same name and type and assign each record a relative weight.

Route 53 Traffic Flow

Route 53 traffic flow provides Global Traffic Management services.
Traffic flow policies allow you to create routing configurations for resources using routing types such as failover and geolocation.
Create policies that route traffic based on specific constraints,including latency,endpoint health,load,geo-proximity and geography.
Scenarios:
- A backup page in Amazon S3 for a website.
- Building routing policies that consider an end users geographic location,proximity to an AWS region,and the health of each of your endpoints.

Route 53 Resolver

It's a set of features that enable bi-directional querying between on-premise and AWS other private connections.
Used for enabling DNS resolution for hybrid clouds.
Route 53 Resolver Endpoints.
- Inbound query capability is provided by Route 53 Resolver Endpoints,allowing DNS queries that originate on-premises to resolve AWS hosted domains.
- Connectivity needs to be established between your on-premise DNS infrastructure and AWS through a DirectConnect or a VPN.
- Endpoints are configured through IP address assignment in each subnet for you would like to provide a resolver.
Conditional Forwarding Rules:
- Outbound DNS queries are enabled through the use of Conditional Forwarding Rules.
- Domains hosted within your on-premise DNS infrastructure can be configured as forwarding rules in Route 53 Resolver.
- Rules will trigger when a query is made to one of those domains and will attempt to forward DNS requests to your DNS servers that were configured along with the rules.
- Like the inbound queries,this requires a private connection over DX or VPN.

AWS Global Accelarator

AWS Global Accelarator is a service that improves the availability and performance of applications with local or global users.
It provides static IP addresses that act as a fixed entry point to application endpoints in a single or multiple AWS Regions, such as ALB,NLB or EC2 instances.
Uses AWS Global Network to optimize the path from users to applications,improving the performance of TCP and UDP traffic.
AWS Global Accelarator continually monitors the health of the application endpoints and will detect an unhealthy endpoint and redirect traffic to healthy endpoints in less than 1 minute.
Uses Redundant (two) static anycast IP addresses in different network Zones (A & B).
The redundant pair are globally advertized.
Uses AWS Edge Locations - addresses are announced from multiple edge locations at the same time.
Addresses are associated to regional AWS resources or endpoints.
AWS Global Accelarator IP addresses serve as the frontend interface of the applications.
Intelligent traffic distribution: Routes connections to the closest point of presence for applications.

Amazon S3

Amazon Simple Storage Service is a object storage service built to store and retrieve any amount of data from anywhere in the internet.

Amazon S3 is a distributed architecture and objects are redundantly stored on multiple devices across multiple facilities (AZs) in an Amazon S3 region.
Amazon S3 is a simple key-based object store.
Amazon S3 provides a simple ,standard-based REST web services interface that is designed to work with any Internet-Development toolkit.
Files can be from 0TB to 5TB.
The largest object that can be uploaded in a single PUT is 5 gigabytes.
For objects larger than 100 MegaBytes use the Multipart Upload capability.
Event notifications for specific actions, can send alerts or trigger actions.
Notifications can be sent to:
- SNS Topics
- SQS Queues
- Lambda functions
- Need to configure SNS/SQS/Lambda before S3
- No extra charges from S3 but you pay for SNS,SQS and Lambda.
Provides read after write consistency for PUTS for new objects.
Provides eventual consistency for overwrite PUTS and DELETES(Takes time to propogate).
S3 is made up of the following:
- Key (name)
- Value (data)
- Version ID
- MetaData
- Access Control Lists

S3 Capability	How it works?
Transfer Accelaration	Speed up data uploads using CloudFront in reverse
Requester Pays	The requester rather than the bucket owner pays for the requests and data transfer
Tags	Assign tags to objects to use in costing,billing,security and etc
Events	Trigger notifications to SNS,SQS,or lambda when certain events happen in your bucket
Static Web Hosting	Simple and massively scalable website hosting
BitTorrent	Use the BitTorrent protocol to retrieve any publicly available object by automatically generating a .torrent file

You can use S3 for following:
- Backup and Storage: Providing data backup and storage services for others
- Application Hosting: Provides services that deploy,install,and manage web applications.
- Media Hosting: Building a redudant,scalable,and highly available insfrastrucute that hosts video,photo,or music uploads and downloads.
- Software Delivery: Hosting software applications that customers can download.
- Static Website: Hosting static sites such as html pages or blogsite.

Amazon S3 Buckets

Files are stored in the bucket:
- A bucket can be viewed as a container for objects.
- A bucket is a flat container of objects.
- It doesn't provide a hiearchy of objects.
- You can use an object key name(prefix) to mimic folders.
100 buckets per account by default.
You can store unlimited objects in your buckets.
You can create folders in your buckets
You cannot create nested buckets.
An S3 Bucket is region specific.

Amazon S3 Objects:

Each object is stored and retrieved by a unique key (ID or name).
An object in S3 is uniquely identified and addressed through:
- Service end point.
- Bucket Name
- Object Key
- Optionally an object version
Objects stored in a bucket will never leave the region in which they are stored unless you move them to another region or enable cross-region replication.
You can define permissions on objects when uploading and at any time afterwards using the AWS management console.

Amazon S3 Sub-resources

Sub resources (configuration containers) associated with the buckets include:
- Lifecycle - define an object's lifecyle.
- Website - configuration for hosting static sites.
- Versioning - retain multiple versions of objects as they are changed
- Access Control Lists (ACLs) - control permissions access to the bucket.
- Bucket Policies - control access to the bucket.
- CORs (Cross Origin Sharing Resources).
- Logging

Amazon S3 Storage Classes

Storage classes include:
- S3 Standard (durable,immediately available,frequent access)
- S3 Intelligent-Tiering (automatically moves data to the most cost effective tiering)
- S3 Standard-IA (durable,immediately-available,infrequent access)
- S3 One Zone-IA (lower cost for infrequently accessed data with less resilience)
- S3 Glacier (archieved data,longer retrieval times)
- S3 Glacier Deep Archive (lowest cost storage class for long term retention).

Amazon S3 Multipart upload

Multipart upload uploads objects in parts independently, in a parallel and in any order.
Performed using the S3 Multipart upload API.
It is recommended for objects larger than 100MB or 100MB
Can be used for objects from 5MB upto 5TB.
Must be used for objects larger than 5GB.

Amazon S3 Copy

You can create a copy of objects upto 5GB in size in a single atomic operation.
For files larger than 5GB you must use the multipart upload API.
Can be performed using the AWS SDKs or REST API.
The copy operation can be used to:
- Generate additional copies of the objects.
- Renaming the objects
- Changing the copy's storage class or encryption at rest status
- Move objects across AWS locations/regions
- Change object metadata.

Amazon S3 Transfer Accelaration

Amazon S3 Transfer Accelaration enables fast,easy,and secure transfers of files over long distances between your client and your S3 bucket.
S3 Transfer Accelaration leverages Amazon CloudFront's globally distributed AWS Edge Locations.
Used to accelarate object uploads to S3 over long distances (latency)
Transfer accelaration is as secure as a direct upload to S3
You are charged only if there was a benefit in the transfer times
Need to enable transfer accelaration on S3 bucket.
Cannot be disabled, can only be suspended

Amazon S3 Encryption

Option	How it works
SSE-S3	Use S3's existing encryption key for AES-256
SSE-C	Upload your own AES-256 encryption key which uses S3 uses when it writes objects
SSE-KMS	Use a key generated and managed by AWS KMS
Client-Side	Encrypt objects using your own local encryption process before uploading to S3

Amazon S3 Performance

Measure Performance
Scale Storage Connections Horizontally
Use byte-range fetches
Retry Requests for Latency-Sensitive Applications
Combine Amazon S3 (Storage) and Amazon EC2 (Compute) in the Same AWS Region.
Use Amazon S3 Transfer accelaration to minimize Latency Caused by the distance.

Amazon CloudFront

Cloud front is web service that distributes content with low latency and high data transfer speeds. Its usually used for dynamic,static,streaming and interactive content.
For instance Netflix might use this service to deliver their content globally.

CloudFront is Global Service:
- Ingress to upload objects.
- Egress to distribute the content
You can use a zone apex DNS name on cloudfront
CloudFront supports wildcard CNAME.
Supports SSL certificates, Dedicated IP, Custom SSL and SNI Custom SSL (Cheaper)
You can restrict access to the content using the following methods:
- Restrict access to content using the signed cookies or signed URLs.
- Restrict access to objects in your S3 bucket.
A special type of user called an Origin Access Identity (OAI) can be used to restrict access to content in an Amazon S3 bucket.
By using an OAI you can restrict users so they cannot access the content directly using the S3 url,they must connect via CloudFront.

Amazon CloudFront Edge Locations and Regional Edge Caches

An edge location is the location where the content is cached.
Requests are automatically routed to the nearest edge location
Edge locations are not tied to Availability Zones or Regions
Regional Edge caches are located between origin web servers and global edge locations and have a larger cache.
Regional Edge Caches have a larger cache-width than any individual edge location so your objects remain in cache longer at these locations.
Regional Edge caches aim to get content closer to users.

Amazon CloudFront Origins

An origin is the origin of the files that CDN will distribute.
Origins can be either an S3 bucket,an EC2 instance,an ELB,or Route 53 - can also be external.
A custom origin server is a HTTP server which can be an EC2 instance or an on-premise/non AWS web servers.
Amazon EC2 instances are considered custom origins.
Static sites on Amazon S3 are also considered custom origins.

Amazon CloudFront Distributions

There are two types of distribution.
Web:
- Static and Dynamic content including .html,.js or .css.
- Distributes files over HTTPS or HTTP
- Add,update,or delete objects and data from submit forms.
- Use live streaming to stream an event in realtime.
RMTP:
- Distribute streaming media files using Adobe FLash Media Server's RTMP protocol.
- Allows an end user to begin playing a media file before the file has finished downloading from a CloudFront edge location
- Files must be stored in an S3 bucket.

Amazon CloudFront Charges

You pay for:
- Data Transfer out to Internet
- Data Transfer out to Origin
- Number of HTTP/HTTPS Requests
- Invalidation Requests
- Dedicated IP Custom SSL
- Field Level encryption requests.
You don't pay for:
- Data transfer between AWS regions and Cloudfront
- Regional Edge Cache
- AWS ACM SSL/TLS Certificates
- Shared Cloudfront certificates

Amazon EBS (Elastic Block Store)

EBS volumes are network attached storage that can be attached to EC2 instances.
EBS volume data persists independently of the life of the instance.
EBS Volumes do not need to be attached to an instance.
You can attach multiple EBS volumes to an instance.
You cannot attach an EBS volume to multiple instances (Use EFS instead).
EBS volume data is replicated across multiple servers in AZ
EBS volumes must be in same AZ as the instances they are attached to
Root EBS volumes are deleted on termination by default.
Extra non-boot volumes are not deleted on termination by default.
The behavior can be changed by altering the DeleteOnTermination attribute.

Note: These are comprehensive and covers almost everything but if you like to add more to it feel free fork this and create a PR. I would be happy to add your new changes

layout: post
title: Everything you need to know for SAA Exam
tags : [aws]
published: true

description: notes compiled for people studying for aws solutions architect associate certification.

Solutions Architect Associate Exam
Exam Domains
IAM (Identity Access Management)
- IAM Authentication Methods.
- MFA (MultiFactor Authentication)
- STS (AWS Security Token)
AWS Global Infrastructure Overview
VPC (Virtual Private Cloud)
EC2 (Elastic Compute Cloud)
- Security Groups.
- Instance Metadata
- Instance Userdata
- Status Checks and Monitoring
- Public,Private and Elastic IP addresses.
- Private Subnets and Bastion Hosts
- NAT Instances and NAT Gateways
- EC2 Placement Groups
- Few Notes
Elastic Load Balancing and Auto Scaling.
- Elastic Load Balancing
- Application Load Balancer
- Network Load Balancer
- Classic Load Balancer
- Internet Facing VS Internal
- Elastic Load Balancing
- ELB Security Groups
- ELB Monitoring
- EC2 Auto Scaling
- EC2 Autoscaling - Scaling Types
- EC2 Autoscaling - Termination Policy
Virtual Private Cloud
- Amazon VPC Components.
- Amazon VPC - Routing
- Amazon VPC - Subnets
- Amazon VPC - Internet Gateways.
- Amazon VPC - Secuirty Groups
- Amazon VPC - Network ACLs
- Amazon VPC - Connectivity
Route 53
- Route 53 Hosted Zones
- Route 53 Health Checks
- CNAME vs Alias
- Route 53 Routing Policies
- Route 53 Traffic Flow
- Route 53 Resolver
- AWS Global Accelarator
Amazon S3
- Amazon S3 Buckets
- Amazon S3 Objects:
- Amazon S3 Sub-resources
- Amazon S3 Storage Classes
- Amazon S3 Multipart upload
- Amazon S3 Copy
- Amazon S3 Transfer Accelaration
- Amazon S3 Encryption
- Amazon S3 Performance
Amazon CloudFront
- Amazon CloudFront Origins
- Amazon CloudFront Distributions
- Amazon CloudFront Charges
Amazon EBS (Elastic Block Store)

Solutions Architect Associate Exam

Its multiple choice and multiple response questions.
130 mins to complete the exam.
It contains 65 questions and costs $150 dollars.
It requires 720 in order to pass the exam.

Exam Domains

The Exam consists of following domains.
Design Resilient Architectures
- Design a multi-tier architecture solution.
- Design highly available or/ fault-tolerant architectures.
- Design decoupling mechanisms using AWS services.
- Choosing appropiate resilient storage.
Design High-Performing Architectures
- Identify elastic and scalable compute solutions for the workload.
- Selecting high performance and scalable storage solution for a workload.
- Selecting high performance networking solutions for a workload.
- Choosing high performance database solutions for the workload.
Design Secure Applications and Architectures
- Designing secure access to AWS resources.
- Designing secure applications tiers.
- Selecting appropiate data security options
Design Cost-Optimized Architectures
- Identify cost-effective storage solutions
- Identify cost-effective compute and database services.
- Design cost-optimized network architectures

IAM (Identity Access Management)

Its a service that provides users,groups,IAM policies.
IAM USER: Its an entity that represents a person or a service and you associate IAM Policy directly with the user and it defines its permissions and what the user is allowed to do within AWS environment. Furthermore, a user can be assigned the following.
- An access key-pair that allows user programmatic access to the AWS API,CLI,SDK and other development tools.
- A password for access to the management console.
- By default users can't do anything within their accounts.
- the account user crendentials are usually the email address used to create the account and a password.
- Root account has full admin priviledges and you can think of it as sudo.
- The best practice is to not use the root crendentials instead create an IAM user assign admin priviledges
- Never share root crendentials.
- Make sure you enable MFA.
- IAM users can be created to represent applications and these are known as service accounts
- You can have upto 5000 users per AWS account.
- Each user account has a friendly name and an ARN(Amazon Resource Name) which uniquely identifies the user across AWS.
- You should always create individual IAM accounts for the users(Not to share them).
- A password policy can be defined for users enforcing them to have stronger passwords (applies to all users)
IAM GROUP: Its a collection of users that have policies attached to them such as group for developers,sys-admins .
- Its not an identity and cannot be identified as principal in an IAM policy.
- use groups to assigns permissions to the users.
- always assign the least priviledges when assigning permissions.
IAM ROLES: Think of it as assigning access to the AWS services such as you might set a role for DynamoDB to readonly.
- They are created and then assumed by trusted entities and define a set of permissions for making AWS requests.
- with roles you can delegate permissions to resources for users and services without using a permanent credentials.
- AWS users or services can assume a role to obtain temporary security crendentials that can used to make aws api calls.
IAM Policies: They are documents that defines the permissions and can be applied to users,groups and roles.
- Policy documents are written in json.
- All permissions are implicitly denied by default.
- the most restrictive policy is applied.
- The IAM policy simulator is a tool that helps you understand,test and validate the effects of access controls policies.

IAM Authentication Methods.

You can use key-pair access keys and its used for programmatic access especially CLI (You can't add MFA to this).
- A combination of access key ID and secret key access.
- this is used to make programmatic calls to aws when using the api. For example boto. Its also used to access AWS using CLI.
- you can create,modify,view or rotate access keys.
- When created IAM returns the access key ID and secret access key.
- The secret access key is returned only at the creation time and if lost new key must be created.
- Make sure access keys and secret access keys are stored securely.
- Users can be given access to change their own keys through IAM policy(Not from console).
- You can disable user's access key which prevents it from being used for API calls.
Simple username and password method to access the management console.
- The password that user uses to sign in into aws web console
Some AWS services uses signing certificate.
- SSL/TLS certificates that can be used to authenticate with some AWS services.
- AWS recommends that you use ACM(AWS Certificate Manager) to provision manage and deploy your server certificates.
- You can also use IAM only when you need to support HTTPS connections in a region that is not supported by ACM.

MFA (MultiFactor Authentication)

Having physical token or soft token that will allow you to access the AWS
- Soft token can be Google Authenticator
- Hard Token can be YuBi Key.
- AWS also provides soft token and physical access keys for MFA.
- By having two factors of authentication it makes it very secure.

STS (AWS Security Token)

STS is a web service that enables you to request temporary, limited-priviledge crendentials for IAM users or for the users that you authenticate (federated users).
By default AWS STS is available as global service and all AWS STS requests go to a single endpoint at https://sts.amazonaws.com
All regions are enabled by default for STS but can be disabled.
The region in which temporary crendentials are requested must be enabled.

AWS Global Infrastructure Overview

Region: A geographical area with 2 or more AZs, isolated from other AWS regions. There are 23 regions around the world at the moment.
Availability Zone: One of more data centers that are physically separate and isolated from other AZs.
Edge Location: A location with Cache Content that can be delivered at low latency to the users. Mostly used by CloudFront for delivering content such as static files or video content.
Regional Edge Cache: Also part of the CloudFront network.These are larger caches that sit between AWS services and Edge Locations.
Global Network: Highly Available, low latency private global network interconnecting every data center, AZ and AWS region.

VPC (Virtual Private Cloud)

Its an isolated section of AWS where you can launch your own resources.
Within VPC you can create your own networks with your own IP ranges.
A VPC sits within a region and you create an VPC within that region and then you create subnets within that regions that sits within AZs. The subnets can public or private and then we launch resources within those subnets. You can have 5 VPCs within the region by default.
A VPC Router is used to communicate within subnets and availablity zones and it has a route table that we can configure and it has an IP address range. Basically every VPC has a CIDR(Classless Inter-Domain Routing) block.
You define the IP range for your VPC.
You can also attach internet gateway to your VPC that sends requests to the outside internet and for that we need igw-id and IP-ADDR as destination.
Internet Gateways allows you to make requests to the public internet and for that we have to add the entry to the route table.
Each VPC has its own CIDR block.

EC2 (`Elastic Compute Cloud`)

Its an elastic service that allows you to launch compute resources on the AWS cloud. In AWS context we call EC2 instances but you can think of them as virual machines.
Each instance has an operating system,storage and virtual hard drive.
When launching instances you can choose from AWS MarketPlace and Community AMIs.
You can connect to EC2 instances using ssh.

Security Groups.

These are firewalls that are applied at the instance level.
They monitor traffic going in and out of EC2 instances.
You can have multiple instances in a security group and you can have multipe security groups applied to the instances.
Security groups are stateful.
For example having a security group with port 22 access applied to the ec2 instances will allow you to launch secure shell.

Instance Metadata

You can run this command within your ec2 in commandline to get the meta data

curl http://169.254.169.254/latest/meta-data/

Instance Userdata

Its basically the information that you can pass into instance when it boots up.

Think of it as a bash script contains all the commands that you need to run when booting up the ec2 instance.
This can be your regular ubuntu apt commands. For example


#!bin/bash
sudo apt-get upgrade
sudo apt install xyz

You can basically paste this into userdata located in advance details when launching an instance.

you can also make a request to userdata by following command.

curl http://169.254.169.254/latest/meta-data/

Status Checks and Monitoring

You can setup cloudwatch alarms on ec2 instances to help monitor the instances effectively and futhermore you can also install stress tool to perform stress testing on the instances.

Public,Private and Elastic IP addresses.

Public IP Address:
- Lost when the instance is stopped.
- Used in public subnets.
- No Charge
- Associated with private IP address on the instance.
- Cannot be moved between instances.
Private IP Address:
- Retained when the instance is stopped.
- Used in public and private subnets.
Elastic IP Address:
- Static Public IP Address.
- You are charged if not used.
- Associated with a private IP address on the instance.
- Note: Elastic Fabric Adapter is a network device that you can attach to reduce latency and increase throughput for distributed HPC(High Performance Computing)

Private Subnets and Bastion Hosts

Public Subnets are easily accessible through public internet
Private Subnet route table doesn't have the igw route and its not configured to provide public ip addresses to the instances launched into this.
There's no way to directly manage this instance through the internet.
A bastian host is a public instance that you used to jump to private instance and it is also known as jump host and through bastian host you will be able to ssh into your private instance.
We use agent forwarding and use the bastian host to connect to the private subnet instance.

NAT Instances and NAT Gateways

NAT Instance: Network Address Translation(NAT).Its basically a process of taking the private ip address and translating it to public ip address so it allows you to connect to the public internet.
- It's managed by you
- The only way to scale this is to do it manually which means using a powerful and bigger instance with more resources and enhanced networking with additional bandwith.
- There's no high availability and it has to be done manually.
- Need to assign security groups.
- Can be used as bastion host.
- You can use an Elastic IP address or a public address with a NAT instance.
- Can implement port forwarding manually
NAT Gateway: A better version of NAT Instance.
- Its managed by AWS
- Its elastically scaled upto 45 GBps
- Provides automatic high availability within an AZ and can be placed in multiple AZs.
- No security groups
- Cannot be accessed through ssh
- Choose the Elastic IP address to associate with a NAT instance gateway at creation.
- Doesn't support port forwarding.
Lastly Private Subnet contains nat-gateway-id instead of igw-id and anything that isn't defined within ip cidr block of private subnet route table is handled by the nat-gateway-id.

EC2 Placement Groups

Cluster: packs instances close together inside an Availability Zone. This strategy enables workloads to achieve the low latency network performance necessary for tightly coupled node to node communication that is typical of HPC applications.
- WHAT: Instances are placed into a low latency group within a single AZ
- WHEN: Need a low network latency or high network throughput.
- Pros: Get most out of enhanced networking instances.
- Cons: Finite capacity recommends launching all you might need upfront.
Partition: spreads your instances across logical partitions such that groups of instances in once partition do not share the underlying hardware with groups of instances in different partitions.This strategy is typically used by large distributed and replicated workloads such as Hadoop,Cassandra and Kafka.
- WHAT: Instances are grouped into logical segments called partitions which use distinct hardware.
- WHEN: Need control and visibility into instance placement.
- Pros: Reduces likelihood of correlated failures for large workloads.
- Cons: Partition placement groups are not supported for dedicated hosts.
Spread: strictly places a small group of instances across distinct underlying hardward to reduce correlated failures.
- WHAT: Instances are spread across underlying hardware.
- WHEN: Reduce the risk of simultaneous instance failure if underlying hardware fails.
- Pros: Can span multiple AZs
- Cons: Maximum upto 7 instances running per group,per AZ.

Few Notes

Amazon EC2:
- Its a compute cloud basically a web service that provides resizeable compute capacity in the cloud.
- With EC2 you have full control of the operating system layer.
- You use key-pair to securely connect to ec2 instances using ssh.
- A keypair consists of public key that AWS stores and private key that we store on our local machine.
- user-data is a script that you provide when starting an instance.
- meta-data is the data of your instance that you can use to configure the instance.
EC2 Pricing Models:
- On Demand:
  - No upfront fee.
  - Charged per hour or second
  - No Commitment
  - Ideal for short term needs or unpredictable workloads
- Reserved:
  - Options: No Upfront,Partial Upfront or all Upfront.
  - Charged by hour or second.
  - 1year or 3year commitment.
  - Ideal for steady-state workloads and predictable usage.
- Spot:
  - No upfront fee
  - Charged by hour or second
  - No commitment
  - Ideal for cost sensitive, compute sensitive use cases that can withstand interruption. (Batch Processing)
- Dedicated Hosts:
  - Good for enterprise customers who are looking for isolated environments
Amazon EC2 AMIs:
- An Amazon Machine Image provides the information required to launch an instance
- An AMI includes the following
  - A template for the root volume for the instance (OS,system,an application server and applications).
  - Launch permissions that control which AWS accounts can use the AMI to launch instances.
  - A block device mapping that specifies the volumes to attach to the instance when its launched (which EBS to attach to the instance).
- Volumes attached to the instances are either EBS or Instance store.
  - Amazon EBS provided persistent store.EBS snapshots resides on Amazon S3 are used to create the volume.
  - Instance store volumes are ephermeral(non-persistent).this means data is lost when the instance is shut down/
- AMIs are regional.You can only launch an AMI from the region in which it is stored. However you can copy AMI's to other regions using console,CLS or API.
Elastic Network Interface (ENI):
- A logical networking component in a VPC that represents a virtual network card.
- Can include attributes such as IP addresses,security groups,MAC addresses, source/destination check flag,description.
- You can create and configure network interfaces in your account and attach them to your instances in VPC.
- eth0 is the primary network interface and cannot be moved or detached.
- An ENI is bound to an availability zone and you can specify which subnet/AZ you want the ENI to be loaded in.
Elastic Fabric Adapter (EFA):
- An AWS Elastic Network Adapter (ENA) with added capabilities.
- Enables customers to run applications requiring high levels of internode communications at scale on AWS.
- With EFA, High Performance Computing (HPC) applications using the Message Passing Interface (MPI) and Machine Learning (ML) applications using NVIDIA Collective Communications Library (NCCL) can scale upto thousands of CPUs or GPUs.
ENI vs ENA vs EFA:
- When to use ENI: This is the basic adapter type for when you don't have any high performance requirements. Can use with all instance types.
- When to use ENA: Good for use cases that require higher bandwidth and lower inter-instance latency. Supported for limited instance types (HVM only).

Elastic Load Balancing and Auto Scaling.

Elastic Load Balancing

Load Balancing refers to efficiently distributing incoming network traffic across a group of backend servers.

Application Load Balancer

Operates at the request level.
Routes based on the content of request (Layer 7)
Supports path based routing,host based routing,query string parameter based routing and source IP address based routing.
Supports IP addresses, Lambda Functions and containers as targets.

- `HTTPS`,`HTTP`

Network Load Balancer

Operates at the connection level.
Routes connections based on IP protocol Data (layer 4)
Offers ultra high performance,low latency,and TLS offloading at scale.
Can have static IP / Elastic IP
Supports UDP and static IP addresses as targets.

Classic Load Balancer

Old generation; not recommended for new applications.
Performs routing at Layer 4 and Layer 7
Use for existing applications running on EC2-Classic instance

Internet Facing VS Internal

Internet Facing:
- ELB nodes have public IPs.
- Routes traffic to the private IP addresses of the EC2 instances.
- Need one public subnet in each AZ where ELB is defined.
- ELB dns name format <name>-<id-number>.<region>.elb.amazonaws.com
Internal only ELB:
- ELB nodes have private IPs.
- Routes traffic to the private IPs of the EC2 instances.
- ELB dns name format internal-<name>-<id-number>.<region>.elb.amazonaws.com

Elastic Load Balancing

EC2 instances and containers can be registered against an ELB
ELB nodes use IP addresses within your subnets, ensure at least a /27 subnet and make sure there are at least 8 IP addresses available in that order for the ELB to scale.
An ELB forwards traffic to eth0 (primary IP address).
An ELB listener is the process that checks for the connection requests:
- Listeners for CLB provide options for TCP and HTTP/HTTPS.
- Listeners for ALB only provide options for HTTPS and HTTP
- Listeners for NLB only provide only TCP as an option ### ELB Security Groups
Security Groups control the ports and protocols that can reach the front-end listener.
You must assign a security group for the ports and the protocols on the front end listener.

ELB Monitoring

CloudWatch every 1 min.
ELB service sends information when requests are active.
Access Logs:
- Disabled by Default.
- Includes information about the client(not included in the Cloud Watch Metrics)
- Can identify requester IP,request type etc.
- Can be optionally stored and retained in S3. ### EC2 Auto Scaling
You can attach one or more classic ELBs to your existing Auto Scaling Groups.
You can attach one or more Target Groups to your ASG to include instances behind an ALB.
The ELBs must be in same region.
Launch configuration is the template used to create new EC2 instances and includes parameters such as instance family, instance type,AMI,keypair and security groups.

Scaling Option	What is it?	When to use?
Maintain	Ensures the required number of instances are running	Use when you always need a known number of instances running at all times
Manual	Manually change the desired capacity via console or CLI	Use when your needs change rarely enough that you are Ok! to make manual changes.
Schedule	Adjust Min/Max instances on specific dates/times or recurring time periods	Use when you know you are busy and quiet times are. Useful for ensuring enough instances are available before busy times
Dynamic	Scale in response to a system load or other triggers using metrics	Useful for changing capacity based on system usage eg if cpu hits 80%

EC2 Autoscaling - Scaling Types

Scaling	What is it?	When to use?
Target Tracking Policy	The scaling adds or removes capacity as required to keep the metric at or close to the specified target value	A use case,when you want to keep the aggregate CPU usage of your ASG at 70%
Simple Scaling Policy	Waits until health check and cool down period expires before revaluating	This is more conservative way to add/remove instances.Useful when load is eratic.AWS recommend step scaling instead of simple in most cases
Step Scaling Policy	Increase or decrease the current capacity of your Auto Scaling group based on a set of scaling adjustments known as step adjustments	Useful when you want to vary adjustments based on the size of the alarm breach

Can also scale based on AWS SQS.
Uses a custom metric that's sent to Amazon Cloud Watch that measures the number of messages in the queue per EC2 instance in the auto scaling group.
Then use a target tracking policy that configures your ASG to scale based on the custom metric and a set target value. Cloud watch alarms invoke the scaling policy.
Use a custom backlog per instance metric to track not just the number of messages in the queue but the number available for retrieval.

EC2 Autoscaling - Termination Policy

Termination policies control which instances are terminated first when scale in event occurs.
There is default termination policy and options for configuring your own customized termination policies.
The default termination policy is designed to help ensure that instances span AZs evenly for High Availability Zones.
The default policy is kept generic and flexible to cover a range of scenarios.

Virtual Private Cloud

A Virtual Private Cloud (VPC) is logically isolated from other VPCs on AWS.
VPCs are regions specific
A default VPC is created in each region with a subnet in each AZ.
You can define dedicated tenancy for a VPC to ensure instances are launched on a dedicated hardware.
The default VPC has all public subnets.
Public Subnets:
- Auto Assign public IPv4 address set to Yes
- The subnet route table has an attached Internet Gateway.
Instances in the default VPC always have both a public and private IP addresses.
AZ names are mapped to different zones for different users.

Amazon VPC Components.

VPC: A logically isolated virtual network in the AWS cloud. You define VPC's IP address space from ranges you select.
Subnet: A segment of a VPC's IP address range where you can place groups of isolated resources (maps to sinlge AZ).
Internet Gateway: The Amazon VPC side of a connection to the public internet.
NAT Gateway: A highly available,managed Network Address Translation (NAT) service for your resources in a private subnet to access the internet.
Hardware VPN Connection: A hardware based VPN connection between your AWS VPC and your data center,home network, or co location facility.
Virtual Private Gateway: The VPC side of an VPN Connection.
Customer Gateway: Our side of a VPN Connection.
Router: Routers interconnect subnets and direct traffic between Internet gateways,virtual private gateways, NAT gateways and subnets.
Peering Connection: A peering connection allows you to route traffic via private IP addresses between two peered VPCs
VPC Endpoints: Enables private connectivity to services hosted in AWS from within VPC without using an Internet Gateway,VPN,NAT Devices or firewall proxies.
Egress-only Internet Gateway: A stateful gateway to provide egress only access for IPv6 traffic from the VPC to the internet.

Amazon VPC - Routing

The VPC router performs routing between AZs within a region.
The VPC router connects different AZs together and connects the VPC to the internet Gateway.
Each subnet has a route table the router uses to forward traffic withing the VPC.
Route tables also have entries to external destinations.

Amazon VPC - Subnets

Types of subnets:
- If a subnet's traffic is routed to an internet gateway the subnet is known as a public subnet.
- If a subnet doesn't have a route to the internet gateway the subnet is known as private subnet.
- The VPC is created with a master address range(CIDR block,can be anywhere from 16-28 bits) and subnet ranges are created within that range.
- New subnets are always associated with the default route table
- Once VPC is created you cannot change the CIDR block.
- You cannot created additional CIDR blocks that overlap with existing CIDR blocks
- You cannot create additional CIDR blocks in a different RFC 1918 range.
- Subnets with overlapping IP address ranges cannot be created
- The first 4 and last 1 IP address in a subnet are reserved.
- Subnets are created within AZs
- Subnets map 1:1 to AZs and cannot span AZs.

Amazon VPC - Internet Gateways.

An Internet Gateway serves two purposes:
- To provide a target in your vpc route tables for internet routable traffic.
- To perform network address translation(NAT) for instances that have been assigned public IPv4 addresses.
Internet Gateways must be created and then attached to a VPC, be added to a route table and then associated with the relevant subnets.
No availability risk or bandwidth constraints.
You cannot have multiple Internet Gateways in a VPC
Egress-Only internet Gateway provides outbound Internet Access for IPv6 addressed instances.

Amazon VPC - Secuirty Groups

Security Group act like a firewall at the instance level (network interface) level.
Can only assign permit rules in a security group, cannot assign deny rules.
All rules are evaluated until a permit is encountered or continues until the implicit deny.
Can control ingress and egress traffic.
Security groups are stateful
By default, custom security groups do not have inbound allow rules (all inbound traffic is denied by default).
By defualt, default security groups do have inbound allow rules (allowing traffic from within the group).
All outbound traffic is allowed by default in custom abd default security groups.
You cannot delete the security group that's created by default within a VPC.
You can use security group names as the source of destination in other security groups.
You can use the security group name as a source in its own inbound rules.
Secuirty group membership can be changed whilst instances are running.
Any changes made will take effect immediately.
You cannot block specific ip addresses using the security groups use NACLs instead.

Amazon VPC - Network ACLs

Network ACLs function at the subnet level.
With NACLs you can have permit and deny rules.
Network ACLs contain a numbered list of rules that are evaluated in order from the lowest number until the explicit deny.
Network ACLs have separate inbound and outbound rules and each rule can allow or deny traffic.
Network ACLs are stateless so responses are subject to the rules for the direction of traffic.
NACLs only apply to traffic that is ingress or egress to the subnet not to traffic within the subnet.
A VPC automatically comes with a default network ACL which allows all inbound/outbound traffic.
A custom NACL denies all traffic both inbound and outbound by default.
All subnets must be associated with a network ACL.
You can create custom network ACLs. By default each custom network ACL denies all the inbound and outbound traffic until you add rules.
You can associate a network ACL with multiple subnets; however a subnet can only be associated with one network ACL at a time.
Network ACLs do not filter traffic between instances in the same subnet.
NACLs are preffered option when it comes to blocking specific IPs or ranges.
Security groups cannot be used to block specific ranges of IPs.
NACL is the first line of defence, the secuirty group is the second.
Changes to NACL take immediately.

Security Group	Network ACL
Operates at the instance level	Operates at the subnet level
Support allow rules only	Supports allow and deby rules only
Stateful	Stateless
Evaluates all rules	Processes rules in order
Applies to an instance only if associated with a group	Automatically applies to all instances in the subnets its associated with

Amazon VPC - Connectivity

There are several methods of connecting to a vpc and these include
AWS Managed VPN
- What: AWS Managed IPSec VPN Connection over your existing internet.
- When: Quick and usually simple way to establish a secure tunnelled connection to a vpc; Redundant link for direct connect or other VPC VPN
- Pros: Supports static routes or BGP peering and routing
- Cons: Dependant on your internet connection.
- How: Create a virtual private gateway on AWS and Customer gateway on the on premise.
AWS Direct Connect
- What: Dedicated network connection over private lines straight into AWS backbone.
- When: Requires a large network link into AWS; lots of resources and services being provided on AWS to your coporate users
- Pros: More predictable network performance; potential bandwidth cost reduction; upto 10 GBps provisioned connections; supports BGP peering and routing.
- Cons: May require additional telecom and hosting provider relationships and /or network circuits; costly;takes time to provision.
- How: Work with your existing data networking provider;create virtual interfaces (VIFs) to connect to VPCs (Private VIFs) or other AWS services like s3 or glacier (public VIFs).
AWS Direct Connect plus a VPN
- What: IPSec VPN connection over private lines (DirectConnect).
- When: Need the added security of encrypted tunnels over direct connect.
- Pros: More secure (in-theory) than Direct Connect Alone.
- Cons: More complexity introduced by VPN Layer
AWS VPN CloudHub
- What: Connect location in the hub and spoke manner using AWSs VPC.
- When: Link remote offices for backup or primary WAN access to AWS resources.
- Pros: Reuses existing Internet Connections;supports BGP routes to direct traffic
- Cons: Dependant on Internet Connection; No inherent redundacny
- How: Assign multiple Customer Gateways to Virtual Private Gateway, each with their own BGP ASN and unique IP ranges.
Software VPN
- What: You provide your own VPN endpoint and software
- When: You must manage both ends of the vpn connection for compliance reasons or you want to use a VPN option not supported by AWS
- Pros: Ultimate flexibility and manageability
- Cons: You must design for an needed redundancy across the whole chain
- How: Install VPN software via marketplace appliance of on an EC2 instance.
Transit VPC
- What: Common strategy for connecting geographically dispersed VPCs and locations in order to create a global network transit center.
- When: Locations and VPC-deployed assests across multiple regions that need to communicate with one another.
- Pros: Ultimate flexibility and manageability but also AWS-managed VPN hub-and-spoke between VPCs
- Cons: You must design for any needed redundancy across the whole chain
- How: Providers like cisco,juniper networks and riverbed have offerings that work with their equipment and AWS VPC
VPC Peering:
- What: AWS provided network connectivity between two VPCs
- When: Multiple VPCs need to communicate or access each others resources.
- Pros: Uses AWS Backbone without traversing the internet.
- Cons: Transitive peering is not supported
- How: VPC peering request made; accepter accepts the request (either within or across the accounts)
AWS Private Link:
- What: AWS provided network connectivity between VPCs and or AWS services using interface endpoints.
- When: Keep private Subnets truly private by using AWS backbone to reach other AWS or Marketplace services rather than the public internet.
- Pros: Redundant;uses AWS backbone
- How : Create endpoint for required AWS or Marketplace service in all required subnets;access via provided DNS hostname.

VPC Endpoints:

	Interface Endpoint	Gateway Endpoint
What	Elastic Network Interface with a private IP	A gateway that is a target for a specific route
How	Uses DNS entries to redirect traffic	Uses prefix lists in the route table to redirect traffic
Which services	API Gateway,CloudFormation,CloudWatch	Amazon S3,DynamoDB
Security	Security Groups	VPC Endpoint Policies

Route 53

Domain Name Registry.
DNS Resolution
Health Checks of the resources

Route 53 is located alongside of all edge locations and when you register a domain with Route 53 it becomes the authoritative DNS server for that domain and creates a public hosted zone.

Route 53 also you to transfer your domains to it as long as it supports TLD(Top Level Domain) is supported and you can even transfer to another registrar by contacting aws support.

Route 53 also allows you have to private DNS which lets you have authoritative DNS within your VPCs without exposing DNS records to the public internet.

Lastly you can use AWS Management Console or API to register new domain names with route 53

Route 53 Hosted Zones

A hosted zone is a collection of records for a specified domain.
A hosted zone is analogous to a traditional DNS zone file; it represents a collection of records that can be managed together.
There are two types of zones
- Public Hosted Zone: Determines how much traffic is routed on the internet
- Private Hosted Zone for VPC: Determines how traffic is routed within VPC (Not accessible outside of VPC).
For Private hosted zones you must set the following VPC settings to true
- enableDnsHostname
- enableDnsSupport You also need to create a DHCP options set.

Route 53 Health Checks

Health checks ensures the instance health by connecting to it.
Health can be pointed at:
- Endpoints (IP addresses or Domain Names)
- Status of other health checks
- Status of cloudwatch alarm

Route 53 supports most of the DNS record types; Alias record is specific to Route 53 and its pointed to DNS name of the service.

CNAME vs Alias

CNAME	ALIAS
Route 53 charges for CNAME queries	Route 53 doesn't charge for alias queries to AWS resources
You cannot create a CNAME record at the top of a DNS namespace (zone apex)	You can create ALIAS record at the zone apex (You cannot route to a CNAME at the zone apex)
A CNAME can point to any DNS record that is hosted anywhere	An alias record can only point to a CLoudFront distribution,Elastic BeanStalk,ELB,S3 bucket as a static site or to another record in the same hosted zone that you are creating the alias record in

Route 53 Routing Policies

Policy	What it does
Simple	Simple DNS response by providing the IP address associated with a name
Failover	If primary is down(based on health checks) routes to secondary destination
Geolocation	Uses geographic location you are in (eg US) routes to closest location
Geoproximity	Routes you to the closest region within geographic area
Latency	Directs you based on the lowest latency routes
Multivalue answer	Returns several IP addresses and functions as a basic load balancer
Weighted	Uses the relative weights assigned to resources to determine which route to

Simple
- An A record is mapped to one or more IP addresses.
- Uses round robin
- Does not support health checks.
Failover:
- Failover to a secondary IP address.
- Associated with a health check
- Used for active-passive
- Can be used with an ELB.
Geolocation:
- Caters to different users in different countries and different languages.
- Contains users within a specific geography and offers them a customized version of the workloads based on their specific needs.
- Geolocation can be used for localizing the content and presenting some or all of your website in the language of the users.
- Can also protect distribution rights.
- Can be used for spreading load evenly between regions.
- If you have multiple records for overlapping regions,Route 53 will route to the smallest geographic region.
Geoproximity:
- Use for routing the traffic based on the location of resources and optionally shift traffic from resources in one location to resources in another.
Latency Based Routing:
- AWS maintains a database of latency from different parts of the world.
- Focused on improving performance by routing to the region with the lowest latency.
Multi-value answer:
- Use for responding to the DNS queries with upto 8 healthy records selected at random.
Weighted:
- Similar to simple but you can specify a weight per IP address.
  - You create records that have same name and type and assign each record a relative weight.

Route 53 Traffic Flow

Route 53 traffic flow provides Global Traffic Management services.
Traffic flow policies allow you to create routing configurations for resources using routing types such as failover and geolocation.
Create policies that route traffic based on specific constraints,including latency,endpoint health,load,geo-proximity and geography.
Scenarios:
- A backup page in Amazon S3 for a website.
- Building routing policies that consider an end users geographic location,proximity to an AWS region,and the health of each of your endpoints.

Route 53 Resolver

It's a set of features that enable bi-directional querying between on-premise and AWS other private connections.
Used for enabling DNS resolution for hybrid clouds.
Route 53 Resolver Endpoints.
- Inbound query capability is provided by Route 53 Resolver Endpoints,allowing DNS queries that originate on-premises to resolve AWS hosted domains.
- Connectivity needs to be established between your on-premise DNS infrastructure and AWS through a DirectConnect or a VPN.
- Endpoints are configured through IP address assignment in each subnet for you would like to provide a resolver.
Conditional Forwarding Rules:
- Outbound DNS queries are enabled through the use of Conditional Forwarding Rules.
- Domains hosted within your on-premise DNS infrastructure can be configured as forwarding rules in Route 53 Resolver.
- Rules will trigger when a query is made to one of those domains and will attempt to forward DNS requests to your DNS servers that were configured along with the rules.
- Like the inbound queries,this requires a private connection over DX or VPN.

AWS Global Accelarator

AWS Global Accelarator is a service that improves the availability and performance of applications with local or global users.
It provides static IP addresses that act as a fixed entry point to application endpoints in a single or multiple AWS Regions, such as ALB,NLB or EC2 instances.
Uses AWS Global Network to optimize the path from users to applications,improving the performance of TCP and UDP traffic.
AWS Global Accelarator continually monitors the health of the application endpoints and will detect an unhealthy endpoint and redirect traffic to healthy endpoints in less than 1 minute.
Uses Redundant (two) static anycast IP addresses in different network Zones (A & B).
The redundant pair are globally advertized.
Uses AWS Edge Locations - addresses are announced from multiple edge locations at the same time.
Addresses are associated to regional AWS resources or endpoints.
AWS Global Accelarator IP addresses serve as the frontend interface of the applications.
Intelligent traffic distribution: Routes connections to the closest point of presence for applications.

Amazon S3

Amazon Simple Storage Service is a object storage service built to store and retrieve any amount of data from anywhere in the internet.

Amazon S3 is a distributed architecture and objects are redundantly stored on multiple devices across multiple facilities (AZs) in an Amazon S3 region.
Amazon S3 is a simple key-based object store.
Amazon S3 provides a simple ,standard-based REST web services interface that is designed to work with any Internet-Development toolkit.
Files can be from 0TB to 5TB.
The largest object that can be uploaded in a single PUT is 5 gigabytes.
For objects larger than 100 MegaBytes use the Multipart Upload capability.
Event notifications for specific actions, can send alerts or trigger actions.
Notifications can be sent to:
- SNS Topics
- SQS Queues
- Lambda functions
- Need to configure SNS/SQS/Lambda before S3
- No extra charges from S3 but you pay for SNS,SQS and Lambda.
Provides read after write consistency for PUTS for new objects.
Provides eventual consistency for overwrite PUTS and DELETES(Takes time to propogate).
S3 is made up of the following:
- Key (name)
- Value (data)
- Version ID
- MetaData
- Access Control Lists

S3 Capability	How it works?
Transfer Accelaration	Speed up data uploads using CloudFront in reverse
Requester Pays	The requester rather than the bucket owner pays for the requests and data transfer
Tags	Assign tags to objects to use in costing,billing,security and etc
Events	Trigger notifications to SNS,SQS,or lambda when certain events happen in your bucket
Static Web Hosting	Simple and massively scalable website hosting
BitTorrent	Use the BitTorrent protocol to retrieve any publicly available object by automatically generating a .torrent file

You can use S3 for following:
- Backup and Storage: Providing data backup and storage services for others
- Application Hosting: Provides services that deploy,install,and manage web applications.
- Media Hosting: Building a redudant,scalable,and highly available insfrastrucute that hosts video,photo,or music uploads and downloads.
- Software Delivery: Hosting software applications that customers can download.
- Static Website: Hosting static sites such as html pages or blogsite.

Amazon S3 Buckets

Files are stored in the bucket:
- A bucket can be viewed as a container for objects.
- A bucket is a flat container of objects.
- It doesn't provide a hiearchy of objects.
- You can use an object key name(prefix) to mimic folders.
100 buckets per account by default.
You can store unlimited objects in your buckets.
You can create folders in your buckets
You cannot create nested buckets.
An S3 Bucket is region specific.

Amazon S3 Objects:

Each object is stored and retrieved by a unique key (ID or name).
An object in S3 is uniquely identified and addressed through:
- Service end point.
- Bucket Name
- Object Key
- Optionally an object version
Objects stored in a bucket will never leave the region in which they are stored unless you move them to another region or enable cross-region replication.
You can define permissions on objects when uploading and at any time afterwards using the AWS management console.

Amazon S3 Sub-resources

Sub resources (configuration containers) associated with the buckets include:
- Lifecycle - define an object's lifecyle.
- Website - configuration for hosting static sites.
- Versioning - retain multiple versions of objects as they are changed
- Access Control Lists (ACLs) - control permissions access to the bucket.
- Bucket Policies - control access to the bucket.
- CORs (Cross Origin Sharing Resources).
- Logging

Amazon S3 Storage Classes

Storage classes include:
- S3 Standard (durable,immediately available,frequent access)
- S3 Intelligent-Tiering (automatically moves data to the most cost effective tiering)
- S3 Standard-IA (durable,immediately-available,infrequent access)
- S3 One Zone-IA (lower cost for infrequently accessed data with less resilience)
- S3 Glacier (archieved data,longer retrieval times)
- S3 Glacier Deep Archive (lowest cost storage class for long term retention).

Amazon S3 Multipart upload

Multipart upload uploads objects in parts independently, in a parallel and in any order.
Performed using the S3 Multipart upload API.
It is recommended for objects larger than 100MB or 100MB
Can be used for objects from 5MB upto 5TB.
Must be used for objects larger than 5GB.

Amazon S3 Copy

You can create a copy of objects upto 5GB in size in a single atomic operation.
For files larger than 5GB you must use the multipart upload API.
Can be performed using the AWS SDKs or REST API.
The copy operation can be used to:
- Generate additional copies of the objects.
- Renaming the objects
- Changing the copy's storage class or encryption at rest status
- Move objects across AWS locations/regions
- Change object metadata.

Amazon S3 Transfer Accelaration

Amazon S3 Transfer Accelaration enables fast,easy,and secure transfers of files over long distances between your client and your S3 bucket.
S3 Transfer Accelaration leverages Amazon CloudFront's globally distributed AWS Edge Locations.
Used to accelarate object uploads to S3 over long distances (latency)
Transfer accelaration is as secure as a direct upload to S3
You are charged only if there was a benefit in the transfer times
Need to enable transfer accelaration on S3 bucket.
Cannot be disabled, can only be suspended

Amazon S3 Encryption

Option	How it works
SSE-S3	Use S3's existing encryption key for AES-256
SSE-C	Upload your own AES-256 encryption key which uses S3 uses when it writes objects
SSE-KMS	Use a key generated and managed by AWS KMS
Client-Side	Encrypt objects using your own local encryption process before uploading to S3

Amazon S3 Performance

Measure Performance
Scale Storage Connections Horizontally
Use byte-range fetches
Retry Requests for Latency-Sensitive Applications
Combine Amazon S3 (Storage) and Amazon EC2 (Compute) in the Same AWS Region.
Use Amazon S3 Transfer accelaration to minimize Latency Caused by the distance.

Amazon CloudFront

CloudFront is Global Service:
- Ingress to upload objects.
- Egress to distribute the content
You can use a zone apex DNS name on cloudfront
CloudFront supports wildcard CNAME.
Supports SSL certificates, Dedicated IP, Custom SSL and SNI Custom SSL (Cheaper)
You can restrict access to the content using the following methods:
- Restrict access to content using the signed cookies or signed URLs.
- Restrict access to objects in your S3 bucket.
A special type of user called an Origin Access Identity (OAI) can be used to restrict access to content in an Amazon S3 bucket.
By using an OAI you can restrict users so they cannot access the content directly using the S3 url,they must connect via CloudFront.

Amazon CloudFront Edge Locations and Regional Edge Caches

An edge location is the location where the content is cached.
Requests are automatically routed to the nearest edge location
Edge locations are not tied to Availability Zones or Regions
Regional Edge caches are located between origin web servers and global edge locations and have a larger cache.
Regional Edge Caches have a larger cache-width than any individual edge location so your objects remain in cache longer at these locations.
Regional Edge caches aim to get content closer to users.

Amazon CloudFront Origins

An origin is the origin of the files that CDN will distribute.
Origins can be either an S3 bucket,an EC2 instance,an ELB,or Route 53 - can also be external.
A custom origin server is a HTTP server which can be an EC2 instance or an on-premise/non AWS web servers.
Amazon EC2 instances are considered custom origins.
Static sites on Amazon S3 are also considered custom origins.

Amazon CloudFront Distributions

There are two types of distribution.
Web:
- Static and Dynamic content including .html,.js or .css.
- Distributes files over HTTPS or HTTP
- Add,update,or delete objects and data from submit forms.
- Use live streaming to stream an event in realtime.
RMTP:
- Distribute streaming media files using Adobe FLash Media Server's RTMP protocol.
- Allows an end user to begin playing a media file before the file has finished downloading from a CloudFront edge location
- Files must be stored in an S3 bucket.

Amazon CloudFront Charges

You pay for:
- Data Transfer out to Internet
- Data Transfer out to Origin
- Number of HTTP/HTTPS Requests
- Invalidation Requests
- Dedicated IP Custom SSL
- Field Level encryption requests.
You don't pay for:
- Data transfer between AWS regions and Cloudfront
- Regional Edge Cache
- AWS ACM SSL/TLS Certificates
- Shared Cloudfront certificates

Amazon EBS (Elastic Block Store)

EBS volumes are network attached storage that can be attached to EC2 instances.
EBS volume data persists independently of the life of the instance.
EBS Volumes do not need to be attached to an instance.
You can attach multiple EBS volumes to an instance.
You cannot attach an EBS volume to multiple instances (Use EFS instead).
EBS volume data is replicated across multiple servers in AZ
EBS volumes must be in same AZ as the instances they are attached to
Root EBS volumes are deleted on termination by default.
Extra non-boot volumes are not deleted on termination by default.
The behavior can be changed by altering the DeleteOnTermination attribute.

Note: These are comprehensive and covers almost everything but if you like to add more to it feel free fork this and create a PR. I would be happy to add your new changes

layout: post
title: Everything you need to know for SAA Exam
tags : [aws]
published: true

description: notes compiled for people studying for aws solutions architect associate certification.

Solutions Architect Associate Exam
Exam Domains
IAM (Identity Access Management)
- IAM Authentication Methods.
- MFA (MultiFactor Authentication)
- STS (AWS Security Token)
AWS Global Infrastructure Overview
VPC (Virtual Private Cloud)
EC2 (Elastic Compute Cloud)
- Security Groups.
- Instance Metadata
- Instance Userdata
- Status Checks and Monitoring
- Public,Private and Elastic IP addresses.
- Private Subnets and Bastion Hosts
- NAT Instances and NAT Gateways
- EC2 Placement Groups
- Few Notes
Elastic Load Balancing and Auto Scaling.
- Elastic Load Balancing
- Application Load Balancer
- Network Load Balancer
- Classic Load Balancer
- Internet Facing VS Internal
- Elastic Load Balancing
- ELB Security Groups
- ELB Monitoring
- EC2 Auto Scaling
- EC2 Autoscaling - Scaling Types
- EC2 Autoscaling - Termination Policy
Virtual Private Cloud
- Amazon VPC Components.
- Amazon VPC - Routing
- Amazon VPC - Subnets
- Amazon VPC - Internet Gateways.
- Amazon VPC - Secuirty Groups
- Amazon VPC - Network ACLs
- Amazon VPC - Connectivity
Route 53
- Route 53 Hosted Zones
- Route 53 Health Checks
- CNAME vs Alias
- Route 53 Routing Policies
- Route 53 Traffic Flow
- Route 53 Resolver
- AWS Global Accelarator
Amazon S3
- Amazon S3 Buckets
- Amazon S3 Objects:
- Amazon S3 Sub-resources
- Amazon S3 Storage Classes
- Amazon S3 Multipart upload
- Amazon S3 Copy
- Amazon S3 Transfer Accelaration
- Amazon S3 Encryption
- Amazon S3 Performance
Amazon CloudFront
- Amazon CloudFront Origins
- Amazon CloudFront Distributions
- Amazon CloudFront Charges
Amazon EBS (Elastic Block Store)

Solutions Architect Associate Exam

Its multiple choice and multiple response questions.
130 mins to complete the exam.
It contains 65 questions and costs $150 dollars.
It requires 720 in order to pass the exam.

Exam Domains

The Exam consists of following domains.
Design Resilient Architectures
- Design a multi-tier architecture solution.
- Design highly available or/ fault-tolerant architectures.
- Design decoupling mechanisms using AWS services.
- Choosing appropiate resilient storage.
Design High-Performing Architectures
- Identify elastic and scalable compute solutions for the workload.
- Selecting high performance and scalable storage solution for a workload.
- Selecting high performance networking solutions for a workload.
- Choosing high performance database solutions for the workload.
Design Secure Applications and Architectures
- Designing secure access to AWS resources.
- Designing secure applications tiers.
- Selecting appropiate data security options
Design Cost-Optimized Architectures
- Identify cost-effective storage solutions
- Identify cost-effective compute and database services.
- Design cost-optimized network architectures

IAM (Identity Access Management)

Its a service that provides users,groups,IAM policies.
IAM USER: Its an entity that represents a person or a service and you associate IAM Policy directly with the user and it defines its permissions and what the user is allowed to do within AWS environment. Furthermore, a user can be assigned the following.
- An access key-pair that allows user programmatic access to the AWS API,CLI,SDK and other development tools.
- A password for access to the management console.
- By default users can't do anything within their accounts.
- the account user crendentials are usually the email address used to create the account and a password.
- Root account has full admin priviledges and you can think of it as sudo.
- The best practice is to not use the root crendentials instead create an IAM user assign admin priviledges
- Never share root crendentials.
- Make sure you enable MFA.
- IAM users can be created to represent applications and these are known as service accounts
- You can have upto 5000 users per AWS account.
- Each user account has a friendly name and an ARN(Amazon Resource Name) which uniquely identifies the user across AWS.
- You should always create individual IAM accounts for the users(Not to share them).
- A password policy can be defined for users enforcing them to have stronger passwords (applies to all users)
IAM GROUP: Its a collection of users that have policies attached to them such as group for developers,sys-admins .
- Its not an identity and cannot be identified as principal in an IAM policy.
- use groups to assigns permissions to the users.
- always assign the least priviledges when assigning permissions.
IAM ROLES: Think of it as assigning access to the AWS services such as you might set a role for DynamoDB to readonly.
- They are created and then assumed by trusted entities and define a set of permissions for making AWS requests.
- with roles you can delegate permissions to resources for users and services without using a permanent credentials.
- AWS users or services can assume a role to obtain temporary security crendentials that can used to make aws api calls.
IAM Policies: They are documents that defines the permissions and can be applied to users,groups and roles.
- Policy documents are written in json.
- All permissions are implicitly denied by default.
- the most restrictive policy is applied.
- The IAM policy simulator is a tool that helps you understand,test and validate the effects of access controls policies.

IAM Authentication Methods.

You can use key-pair access keys and its used for programmatic access especially CLI (You can't add MFA to this).
- A combination of access key ID and secret key access.
- this is used to make programmatic calls to aws when using the api. For example boto. Its also used to access AWS using CLI.
- you can create,modify,view or rotate access keys.
- When created IAM returns the access key ID and secret access key.
- The secret access key is returned only at the creation time and if lost new key must be created.
- Make sure access keys and secret access keys are stored securely.
- Users can be given access to change their own keys through IAM policy(Not from console).
- You can disable user's access key which prevents it from being used for API calls.
Simple username and password method to access the management console.
- The password that user uses to sign in into aws web console
Some AWS services uses signing certificate.
- SSL/TLS certificates that can be used to authenticate with some AWS services.
- AWS recommends that you use ACM(AWS Certificate Manager) to provision manage and deploy your server certificates.
- You can also use IAM only when you need to support HTTPS connections in a region that is not supported by ACM.

MFA (MultiFactor Authentication)

Having physical token or soft token that will allow you to access the AWS
- Soft token can be Google Authenticator
- Hard Token can be YuBi Key.
- AWS also provides soft token and physical access keys for MFA.
- By having two factors of authentication it makes it very secure.

STS (AWS Security Token)

STS is a web service that enables you to request temporary, limited-priviledge crendentials for IAM users or for the users that you authenticate (federated users).
By default AWS STS is available as global service and all AWS STS requests go to a single endpoint at https://sts.amazonaws.com
All regions are enabled by default for STS but can be disabled.
The region in which temporary crendentials are requested must be enabled.

AWS Global Infrastructure Overview

Region: A geographical area with 2 or more AZs, isolated from other AWS regions. There are 23 regions around the world at the moment.
Availability Zone: One of more data centers that are physically separate and isolated from other AZs.
Edge Location: A location with Cache Content that can be delivered at low latency to the users. Mostly used by CloudFront for delivering content such as static files or video content.
Regional Edge Cache: Also part of the CloudFront network.These are larger caches that sit between AWS services and Edge Locations.
Global Network: Highly Available, low latency private global network interconnecting every data center, AZ and AWS region.

VPC (Virtual Private Cloud)

Its an isolated section of AWS where you can launch your own resources.
Within VPC you can create your own networks with your own IP ranges.
A VPC sits within a region and you create an VPC within that region and then you create subnets within that regions that sits within AZs. The subnets can public or private and then we launch resources within those subnets. You can have 5 VPCs within the region by default.
A VPC Router is used to communicate within subnets and availablity zones and it has a route table that we can configure and it has an IP address range. Basically every VPC has a CIDR(Classless Inter-Domain Routing) block.
You define the IP range for your VPC.
You can also attach internet gateway to your VPC that sends requests to the outside internet and for that we need igw-id and IP-ADDR as destination.
Internet Gateways allows you to make requests to the public internet and for that we have to add the entry to the route table.
Each VPC has its own CIDR block.

EC2 (`Elastic Compute Cloud`)

Its an elastic service that allows you to launch compute resources on the AWS cloud. In AWS context we call EC2 instances but you can think of them as virual machines.
Each instance has an operating system,storage and virtual hard drive.
When launching instances you can choose from AWS MarketPlace and Community AMIs.
You can connect to EC2 instances using ssh.

Security Groups.

These are firewalls that are applied at the instance level.
They monitor traffic going in and out of EC2 instances.
You can have multiple instances in a security group and you can have multipe security groups applied to the instances.
Security groups are stateful.
For example having a security group with port 22 access applied to the ec2 instances will allow you to launch secure shell.

Instance Metadata

You can run this command within your ec2 in commandline to get the meta data

curl http://169.254.169.254/latest/meta-data/

Instance Userdata

Its basically the information that you can pass into instance when it boots up.

Think of it as a bash script contains all the commands that you need to run when booting up the ec2 instance.
This can be your regular ubuntu apt commands. For example


#!bin/bash
sudo apt-get upgrade
sudo apt install xyz

You can basically paste this into userdata located in advance details when launching an instance.

you can also make a request to userdata by following command.

curl http://169.254.169.254/latest/meta-data/

Status Checks and Monitoring

You can setup cloudwatch alarms on ec2 instances to help monitor the instances effectively and futhermore you can also install stress tool to perform stress testing on the instances.

Public,Private and Elastic IP addresses.

Public IP Address:
- Lost when the instance is stopped.
- Used in public subnets.
- No Charge
- Associated with private IP address on the instance.
- Cannot be moved between instances.
Private IP Address:
- Retained when the instance is stopped.
- Used in public and private subnets.
Elastic IP Address:
- Static Public IP Address.
- You are charged if not used.
- Associated with a private IP address on the instance.
- Note: Elastic Fabric Adapter is a network device that you can attach to reduce latency and increase throughput for distributed HPC(High Performance Computing)

Private Subnets and Bastion Hosts

Public Subnets are easily accessible through public internet
Private Subnet route table doesn't have the igw route and its not configured to provide public ip addresses to the instances launched into this.
There's no way to directly manage this instance through the internet.
A bastian host is a public instance that you used to jump to private instance and it is also known as jump host and through bastian host you will be able to ssh into your private instance.
We use agent forwarding and use the bastian host to connect to the private subnet instance.

NAT Instances and NAT Gateways

NAT Instance: Network Address Translation(NAT).Its basically a process of taking the private ip address and translating it to public ip address so it allows you to connect to the public internet.
- It's managed by you
- The only way to scale this is to do it manually which means using a powerful and bigger instance with more resources and enhanced networking with additional bandwith.
- There's no high availability and it has to be done manually.
- Need to assign security groups.
- Can be used as bastion host.
- You can use an Elastic IP address or a public address with a NAT instance.
- Can implement port forwarding manually
NAT Gateway: A better version of NAT Instance.
- Its managed by AWS
- Its elastically scaled upto 45 GBps
- Provides automatic high availability within an AZ and can be placed in multiple AZs.
- No security groups
- Cannot be accessed through ssh
- Choose the Elastic IP address to associate with a NAT instance gateway at creation.
- Doesn't support port forwarding.
Lastly Private Subnet contains nat-gateway-id instead of igw-id and anything that isn't defined within ip cidr block of private subnet route table is handled by the nat-gateway-id.

EC2 Placement Groups

Cluster: packs instances close together inside an Availability Zone. This strategy enables workloads to achieve the low latency network performance necessary for tightly coupled node to node communication that is typical of HPC applications.
- WHAT: Instances are placed into a low latency group within a single AZ
- WHEN: Need a low network latency or high network throughput.
- Pros: Get most out of enhanced networking instances.
- Cons: Finite capacity recommends launching all you might need upfront.
Partition: spreads your instances across logical partitions such that groups of instances in once partition do not share the underlying hardware with groups of instances in different partitions.This strategy is typically used by large distributed and replicated workloads such as Hadoop,Cassandra and Kafka.
- WHAT: Instances are grouped into logical segments called partitions which use distinct hardware.
- WHEN: Need control and visibility into instance placement.
- Pros: Reduces likelihood of correlated failures for large workloads.
- Cons: Partition placement groups are not supported for dedicated hosts.
Spread: strictly places a small group of instances across distinct underlying hardward to reduce correlated failures.
- WHAT: Instances are spread across underlying hardware.
- WHEN: Reduce the risk of simultaneous instance failure if underlying hardware fails.
- Pros: Can span multiple AZs
- Cons: Maximum upto 7 instances running per group,per AZ.

Few Notes

Amazon EC2:
- Its a compute cloud basically a web service that provides resizeable compute capacity in the cloud.
- With EC2 you have full control of the operating system layer.
- You use key-pair to securely connect to ec2 instances using ssh.
- A keypair consists of public key that AWS stores and private key that we store on our local machine.
- user-data is a script that you provide when starting an instance.
- meta-data is the data of your instance that you can use to configure the instance.
EC2 Pricing Models:
- On Demand:
  - No upfront fee.
  - Charged per hour or second
  - No Commitment
  - Ideal for short term needs or unpredictable workloads
- Reserved:
  - Options: No Upfront,Partial Upfront or all Upfront.
  - Charged by hour or second.
  - 1year or 3year commitment.
  - Ideal for steady-state workloads and predictable usage.
- Spot:
  - No upfront fee
  - Charged by hour or second
  - No commitment
  - Ideal for cost sensitive, compute sensitive use cases that can withstand interruption. (Batch Processing)
- Dedicated Hosts:
  - Good for enterprise customers who are looking for isolated environments
Amazon EC2 AMIs:
- An Amazon Machine Image provides the information required to launch an instance
- An AMI includes the following
  - A template for the root volume for the instance (OS,system,an application server and applications).
  - Launch permissions that control which AWS accounts can use the AMI to launch instances.
  - A block device mapping that specifies the volumes to attach to the instance when its launched (which EBS to attach to the instance).
- Volumes attached to the instances are either EBS or Instance store.
  - Amazon EBS provided persistent store.EBS snapshots resides on Amazon S3 are used to create the volume.
  - Instance store volumes are ephermeral(non-persistent).this means data is lost when the instance is shut down/
- AMIs are regional.You can only launch an AMI from the region in which it is stored. However you can copy AMI's to other regions using console,CLS or API.
Elastic Network Interface (ENI):
- A logical networking component in a VPC that represents a virtual network card.
- Can include attributes such as IP addresses,security groups,MAC addresses, source/destination check flag,description.
- You can create and configure network interfaces in your account and attach them to your instances in VPC.
- eth0 is the primary network interface and cannot be moved or detached.
- An ENI is bound to an availability zone and you can specify which subnet/AZ you want the ENI to be loaded in.
Elastic Fabric Adapter (EFA):
- An AWS Elastic Network Adapter (ENA) with added capabilities.
- Enables customers to run applications requiring high levels of internode communications at scale on AWS.
- With EFA, High Performance Computing (HPC) applications using the Message Passing Interface (MPI) and Machine Learning (ML) applications using NVIDIA Collective Communications Library (NCCL) can scale upto thousands of CPUs or GPUs.
ENI vs ENA vs EFA:
- When to use ENI: This is the basic adapter type for when you don't have any high performance requirements. Can use with all instance types.
- When to use ENA: Good for use cases that require higher bandwidth and lower inter-instance latency. Supported for limited instance types (HVM only).

Elastic Load Balancing and Auto Scaling.

Elastic Load Balancing

Load Balancing refers to efficiently distributing incoming network traffic across a group of backend servers.

Application Load Balancer

Operates at the request level.
Routes based on the content of request (Layer 7)
Supports path based routing,host based routing,query string parameter based routing and source IP address based routing.
Supports IP addresses, Lambda Functions and containers as targets.

- `HTTPS`,`HTTP`

Network Load Balancer

Operates at the connection level.
Routes connections based on IP protocol Data (layer 4)
Offers ultra high performance,low latency,and TLS offloading at scale.
Can have static IP / Elastic IP
Supports UDP and static IP addresses as targets.

Classic Load Balancer

Old generation; not recommended for new applications.
Performs routing at Layer 4 and Layer 7
Use for existing applications running on EC2-Classic instance

Internet Facing VS Internal

Internet Facing:
- ELB nodes have public IPs.
- Routes traffic to the private IP addresses of the EC2 instances.
- Need one public subnet in each AZ where ELB is defined.
- ELB dns name format <name>-<id-number>.<region>.elb.amazonaws.com
Internal only ELB:
- ELB nodes have private IPs.
- Routes traffic to the private IPs of the EC2 instances.
- ELB dns name format internal-<name>-<id-number>.<region>.elb.amazonaws.com

Elastic Load Balancing

EC2 instances and containers can be registered against an ELB
ELB nodes use IP addresses within your subnets, ensure at least a /27 subnet and make sure there are at least 8 IP addresses available in that order for the ELB to scale.
An ELB forwards traffic to eth0 (primary IP address).
An ELB listener is the process that checks for the connection requests:
- Listeners for CLB provide options for TCP and HTTP/HTTPS.
- Listeners for ALB only provide options for HTTPS and HTTP
- Listeners for NLB only provide only TCP as an option ### ELB Security Groups
Security Groups control the ports and protocols that can reach the front-end listener.
You must assign a security group for the ports and the protocols on the front end listener.

ELB Monitoring

CloudWatch every 1 min.
ELB service sends information when requests are active.
Access Logs:
- Disabled by Default.
- Includes information about the client(not included in the Cloud Watch Metrics)
- Can identify requester IP,request type etc.
- Can be optionally stored and retained in S3. ### EC2 Auto Scaling
You can attach one or more classic ELBs to your existing Auto Scaling Groups.
You can attach one or more Target Groups to your ASG to include instances behind an ALB.
The ELBs must be in same region.
Launch configuration is the template used to create new EC2 instances and includes parameters such as instance family, instance type,AMI,keypair and security groups.

Scaling Option	What is it?	When to use?
Maintain	Ensures the required number of instances are running	Use when you always need a known number of instances running at all times
Manual	Manually change the desired capacity via console or CLI	Use when your needs change rarely enough that you are Ok! to make manual changes.
Schedule	Adjust Min/Max instances on specific dates/times or recurring time periods	Use when you know you are busy and quiet times are. Useful for ensuring enough instances are available before busy times
Dynamic	Scale in response to a system load or other triggers using metrics	Useful for changing capacity based on system usage eg if cpu hits 80%

EC2 Autoscaling - Scaling Types

Scaling	What is it?	When to use?
Target Tracking Policy	The scaling adds or removes capacity as required to keep the metric at or close to the specified target value	A use case,when you want to keep the aggregate CPU usage of your ASG at 70%
Simple Scaling Policy	Waits until health check and cool down period expires before revaluating	This is more conservative way to add/remove instances.Useful when load is eratic.AWS recommend step scaling instead of simple in most cases
Step Scaling Policy	Increase or decrease the current capacity of your Auto Scaling group based on a set of scaling adjustments known as step adjustments	Useful when you want to vary adjustments based on the size of the alarm breach

Can also scale based on AWS SQS.
Uses a custom metric that's sent to Amazon Cloud Watch that measures the number of messages in the queue per EC2 instance in the auto scaling group.
Then use a target tracking policy that configures your ASG to scale based on the custom metric and a set target value. Cloud watch alarms invoke the scaling policy.
Use a custom backlog per instance metric to track not just the number of messages in the queue but the number available for retrieval.

EC2 Autoscaling - Termination Policy

Termination policies control which instances are terminated first when scale in event occurs.
There is default termination policy and options for configuring your own customized termination policies.
The default termination policy is designed to help ensure that instances span AZs evenly for High Availability Zones.
The default policy is kept generic and flexible to cover a range of scenarios.

Virtual Private Cloud

A Virtual Private Cloud (VPC) is logically isolated from other VPCs on AWS.
VPCs are regions specific
A default VPC is created in each region with a subnet in each AZ.
You can define dedicated tenancy for a VPC to ensure instances are launched on a dedicated hardware.
The default VPC has all public subnets.
Public Subnets:
- Auto Assign public IPv4 address set to Yes
- The subnet route table has an attached Internet Gateway.
Instances in the default VPC always have both a public and private IP addresses.
AZ names are mapped to different zones for different users.

Amazon VPC Components.

VPC: A logically isolated virtual network in the AWS cloud. You define VPC's IP address space from ranges you select.
Subnet: A segment of a VPC's IP address range where you can place groups of isolated resources (maps to sinlge AZ).
Internet Gateway: The Amazon VPC side of a connection to the public internet.
NAT Gateway: A highly available,managed Network Address Translation (NAT) service for your resources in a private subnet to access the internet.
Hardware VPN Connection: A hardware based VPN connection between your AWS VPC and your data center,home network, or co location facility.
Virtual Private Gateway: The VPC side of an VPN Connection.
Customer Gateway: Our side of a VPN Connection.
Router: Routers interconnect subnets and direct traffic between Internet gateways,virtual private gateways, NAT gateways and subnets.
Peering Connection: A peering connection allows you to route traffic via private IP addresses between two peered VPCs
VPC Endpoints: Enables private connectivity to services hosted in AWS from within VPC without using an Internet Gateway,VPN,NAT Devices or firewall proxies.
Egress-only Internet Gateway: A stateful gateway to provide egress only access for IPv6 traffic from the VPC to the internet.

Amazon VPC - Routing

The VPC router performs routing between AZs within a region.
The VPC router connects different AZs together and connects the VPC to the internet Gateway.
Each subnet has a route table the router uses to forward traffic withing the VPC.
Route tables also have entries to external destinations.

Amazon VPC - Subnets

Types of subnets:
- If a subnet's traffic is routed to an internet gateway the subnet is known as a public subnet.
- If a subnet doesn't have a route to the internet gateway the subnet is known as private subnet.
- The VPC is created with a master address range(CIDR block,can be anywhere from 16-28 bits) and subnet ranges are created within that range.
- New subnets are always associated with the default route table
- Once VPC is created you cannot change the CIDR block.
- You cannot created additional CIDR blocks that overlap with existing CIDR blocks
- You cannot create additional CIDR blocks in a different RFC 1918 range.
- Subnets with overlapping IP address ranges cannot be created
- The first 4 and last 1 IP address in a subnet are reserved.
- Subnets are created within AZs
- Subnets map 1:1 to AZs and cannot span AZs.

Amazon VPC - Internet Gateways.

An Internet Gateway serves two purposes:
- To provide a target in your vpc route tables for internet routable traffic.
- To perform network address translation(NAT) for instances that have been assigned public IPv4 addresses.
Internet Gateways must be created and then attached to a VPC, be added to a route table and then associated with the relevant subnets.
No availability risk or bandwidth constraints.
You cannot have multiple Internet Gateways in a VPC
Egress-Only internet Gateway provides outbound Internet Access for IPv6 addressed instances.

Amazon VPC - Secuirty Groups

Security Group act like a firewall at the instance level (network interface) level.
Can only assign permit rules in a security group, cannot assign deny rules.
All rules are evaluated until a permit is encountered or continues until the implicit deny.
Can control ingress and egress traffic.
Security groups are stateful
By default, custom security groups do not have inbound allow rules (all inbound traffic is denied by default).
By defualt, default security groups do have inbound allow rules (allowing traffic from within the group).
All outbound traffic is allowed by default in custom abd default security groups.
You cannot delete the security group that's created by default within a VPC.
You can use security group names as the source of destination in other security groups.
You can use the security group name as a source in its own inbound rules.
Secuirty group membership can be changed whilst instances are running.
Any changes made will take effect immediately.
You cannot block specific ip addresses using the security groups use NACLs instead.

Amazon VPC - Network ACLs

Network ACLs function at the subnet level.
With NACLs you can have permit and deny rules.
Network ACLs contain a numbered list of rules that are evaluated in order from the lowest number until the explicit deny.
Network ACLs have separate inbound and outbound rules and each rule can allow or deny traffic.
Network ACLs are stateless so responses are subject to the rules for the direction of traffic.
NACLs only apply to traffic that is ingress or egress to the subnet not to traffic within the subnet.
A VPC automatically comes with a default network ACL which allows all inbound/outbound traffic.
A custom NACL denies all traffic both inbound and outbound by default.
All subnets must be associated with a network ACL.
You can create custom network ACLs. By default each custom network ACL denies all the inbound and outbound traffic until you add rules.
You can associate a network ACL with multiple subnets; however a subnet can only be associated with one network ACL at a time.
Network ACLs do not filter traffic between instances in the same subnet.
NACLs are preffered option when it comes to blocking specific IPs or ranges.
Security groups cannot be used to block specific ranges of IPs.
NACL is the first line of defence, the secuirty group is the second.
Changes to NACL take immediately.

Security Group	Network ACL
Operates at the instance level	Operates at the subnet level
Support allow rules only	Supports allow and deby rules only
Stateful	Stateless
Evaluates all rules	Processes rules in order
Applies to an instance only if associated with a group	Automatically applies to all instances in the subnets its associated with

Amazon VPC - Connectivity

There are several methods of connecting to a vpc and these include
AWS Managed VPN
- What: AWS Managed IPSec VPN Connection over your existing internet.
- When: Quick and usually simple way to establish a secure tunnelled connection to a vpc; Redundant link for direct connect or other VPC VPN
- Pros: Supports static routes or BGP peering and routing
- Cons: Dependant on your internet connection.
- How: Create a virtual private gateway on AWS and Customer gateway on the on premise.
AWS Direct Connect
- What: Dedicated network connection over private lines straight into AWS backbone.
- When: Requires a large network link into AWS; lots of resources and services being provided on AWS to your coporate users
- Pros: More predictable network performance; potential bandwidth cost reduction; upto 10 GBps provisioned connections; supports BGP peering and routing.
- Cons: May require additional telecom and hosting provider relationships and /or network circuits; costly;takes time to provision.
- How: Work with your existing data networking provider;create virtual interfaces (VIFs) to connect to VPCs (Private VIFs) or other AWS services like s3 or glacier (public VIFs).
AWS Direct Connect plus a VPN
- What: IPSec VPN connection over private lines (DirectConnect).
- When: Need the added security of encrypted tunnels over direct connect.
- Pros: More secure (in-theory) than Direct Connect Alone.
- Cons: More complexity introduced by VPN Layer
AWS VPN CloudHub
- What: Connect location in the hub and spoke manner using AWSs VPC.
- When: Link remote offices for backup or primary WAN access to AWS resources.
- Pros: Reuses existing Internet Connections;supports BGP routes to direct traffic
- Cons: Dependant on Internet Connection; No inherent redundacny
- How: Assign multiple Customer Gateways to Virtual Private Gateway, each with their own BGP ASN and unique IP ranges.
Software VPN
- What: You provide your own VPN endpoint and software
- When: You must manage both ends of the vpn connection for compliance reasons or you want to use a VPN option not supported by AWS
- Pros: Ultimate flexibility and manageability
- Cons: You must design for an needed redundancy across the whole chain
- How: Install VPN software via marketplace appliance of on an EC2 instance.
Transit VPC
- What: Common strategy for connecting geographically dispersed VPCs and locations in order to create a global network transit center.
- When: Locations and VPC-deployed assests across multiple regions that need to communicate with one another.
- Pros: Ultimate flexibility and manageability but also AWS-managed VPN hub-and-spoke between VPCs
- Cons: You must design for any needed redundancy across the whole chain
- How: Providers like cisco,juniper networks and riverbed have offerings that work with their equipment and AWS VPC
VPC Peering:
- What: AWS provided network connectivity between two VPCs
- When: Multiple VPCs need to communicate or access each others resources.
- Pros: Uses AWS Backbone without traversing the internet.
- Cons: Transitive peering is not supported
- How: VPC peering request made; accepter accepts the request (either within or across the accounts)
AWS Private Link:
- What: AWS provided network connectivity between VPCs and or AWS services using interface endpoints.
- When: Keep private Subnets truly private by using AWS backbone to reach other AWS or Marketplace services rather than the public internet.
- Pros: Redundant;uses AWS backbone
- How : Create endpoint for required AWS or Marketplace service in all required subnets;access via provided DNS hostname.

VPC Endpoints:

	Interface Endpoint	Gateway Endpoint
What	Elastic Network Interface with a private IP	A gateway that is a target for a specific route
How	Uses DNS entries to redirect traffic	Uses prefix lists in the route table to redirect traffic
Which services	API Gateway,CloudFormation,CloudWatch	Amazon S3,DynamoDB
Security	Security Groups	VPC Endpoint Policies

Route 53

Domain Name Registry.
DNS Resolution
Health Checks of the resources

Route 53 is located alongside of all edge locations and when you register a domain with Route 53 it becomes the authoritative DNS server for that domain and creates a public hosted zone.

Route 53 also you to transfer your domains to it as long as it supports TLD(Top Level Domain) is supported and you can even transfer to another registrar by contacting aws support.

Route 53 also allows you have to private DNS which lets you have authoritative DNS within your VPCs without exposing DNS records to the public internet.

Lastly you can use AWS Management Console or API to register new domain names with route 53

Route 53 Hosted Zones

A hosted zone is a collection of records for a specified domain.
A hosted zone is analogous to a traditional DNS zone file; it represents a collection of records that can be managed together.
There are two types of zones
- Public Hosted Zone: Determines how much traffic is routed on the internet
- Private Hosted Zone for VPC: Determines how traffic is routed within VPC (Not accessible outside of VPC).
For Private hosted zones you must set the following VPC settings to true
- enableDnsHostname
- enableDnsSupport You also need to create a DHCP options set.

Route 53 Health Checks

Health checks ensures the instance health by connecting to it.
Health can be pointed at:
- Endpoints (IP addresses or Domain Names)
- Status of other health checks
- Status of cloudwatch alarm

Route 53 supports most of the DNS record types; Alias record is specific to Route 53 and its pointed to DNS name of the service.

CNAME vs Alias

CNAME	ALIAS
Route 53 charges for CNAME queries	Route 53 doesn't charge for alias queries to AWS resources
You cannot create a CNAME record at the top of a DNS namespace (zone apex)	You can create ALIAS record at the zone apex (You cannot route to a CNAME at the zone apex)
A CNAME can point to any DNS record that is hosted anywhere	An alias record can only point to a CLoudFront distribution,Elastic BeanStalk,ELB,S3 bucket as a static site or to another record in the same hosted zone that you are creating the alias record in

Route 53 Routing Policies

Policy	What it does
Simple	Simple DNS response by providing the IP address associated with a name
Failover	If primary is down(based on health checks) routes to secondary destination
Geolocation	Uses geographic location you are in (eg US) routes to closest location
Geoproximity	Routes you to the closest region within geographic area
Latency	Directs you based on the lowest latency routes
Multivalue answer	Returns several IP addresses and functions as a basic load balancer
Weighted	Uses the relative weights assigned to resources to determine which route to

Simple
- An A record is mapped to one or more IP addresses.
- Uses round robin
- Does not support health checks.
Failover:
- Failover to a secondary IP address.
- Associated with a health check
- Used for active-passive
- Can be used with an ELB.
Geolocation:
- Caters to different users in different countries and different languages.
- Contains users within a specific geography and offers them a customized version of the workloads based on their specific needs.
- Geolocation can be used for localizing the content and presenting some or all of your website in the language of the users.
- Can also protect distribution rights.
- Can be used for spreading load evenly between regions.
- If you have multiple records for overlapping regions,Route 53 will route to the smallest geographic region.
Geoproximity:
- Use for routing the traffic based on the location of resources and optionally shift traffic from resources in one location to resources in another.
Latency Based Routing:
- AWS maintains a database of latency from different parts of the world.
- Focused on improving performance by routing to the region with the lowest latency.
Multi-value answer:
- Use for responding to the DNS queries with upto 8 healthy records selected at random.
Weighted:
- Similar to simple but you can specify a weight per IP address.
  - You create records that have same name and type and assign each record a relative weight.

Route 53 Traffic Flow

Route 53 traffic flow provides Global Traffic Management services.
Traffic flow policies allow you to create routing configurations for resources using routing types such as failover and geolocation.
Create policies that route traffic based on specific constraints,including latency,endpoint health,load,geo-proximity and geography.
Scenarios:
- A backup page in Amazon S3 for a website.
- Building routing policies that consider an end users geographic location,proximity to an AWS region,and the health of each of your endpoints.

Route 53 Resolver

It's a set of features that enable bi-directional querying between on-premise and AWS other private connections.
Used for enabling DNS resolution for hybrid clouds.
Route 53 Resolver Endpoints.
- Inbound query capability is provided by Route 53 Resolver Endpoints,allowing DNS queries that originate on-premises to resolve AWS hosted domains.
- Connectivity needs to be established between your on-premise DNS infrastructure and AWS through a DirectConnect or a VPN.
- Endpoints are configured through IP address assignment in each subnet for you would like to provide a resolver.
Conditional Forwarding Rules:
- Outbound DNS queries are enabled through the use of Conditional Forwarding Rules.
- Domains hosted within your on-premise DNS infrastructure can be configured as forwarding rules in Route 53 Resolver.
- Rules will trigger when a query is made to one of those domains and will attempt to forward DNS requests to your DNS servers that were configured along with the rules.
- Like the inbound queries,this requires a private connection over DX or VPN.

AWS Global Accelarator

AWS Global Accelarator is a service that improves the availability and performance of applications with local or global users.
It provides static IP addresses that act as a fixed entry point to application endpoints in a single or multiple AWS Regions, such as ALB,NLB or EC2 instances.
Uses AWS Global Network to optimize the path from users to applications,improving the performance of TCP and UDP traffic.
AWS Global Accelarator continually monitors the health of the application endpoints and will detect an unhealthy endpoint and redirect traffic to healthy endpoints in less than 1 minute.
Uses Redundant (two) static anycast IP addresses in different network Zones (A & B).
The redundant pair are globally advertized.
Uses AWS Edge Locations - addresses are announced from multiple edge locations at the same time.
Addresses are associated to regional AWS resources or endpoints.
AWS Global Accelarator IP addresses serve as the frontend interface of the applications.
Intelligent traffic distribution: Routes connections to the closest point of presence for applications.

Amazon S3

Amazon Simple Storage Service is a object storage service built to store and retrieve any amount of data from anywhere in the internet.

Amazon S3 is a distributed architecture and objects are redundantly stored on multiple devices across multiple facilities (AZs) in an Amazon S3 region.
Amazon S3 is a simple key-based object store.
Amazon S3 provides a simple ,standard-based REST web services interface that is designed to work with any Internet-Development toolkit.
Files can be from 0TB to 5TB.
The largest object that can be uploaded in a single PUT is 5 gigabytes.
For objects larger than 100 MegaBytes use the Multipart Upload capability.
Event notifications for specific actions, can send alerts or trigger actions.
Notifications can be sent to:
- SNS Topics
- SQS Queues
- Lambda functions
- Need to configure SNS/SQS/Lambda before S3
- No extra charges from S3 but you pay for SNS,SQS and Lambda.
Provides read after write consistency for PUTS for new objects.
Provides eventual consistency for overwrite PUTS and DELETES(Takes time to propogate).
S3 is made up of the following:
- Key (name)
- Value (data)
- Version ID
- MetaData
- Access Control Lists

S3 Capability	How it works?
Transfer Accelaration	Speed up data uploads using CloudFront in reverse
Requester Pays	The requester rather than the bucket owner pays for the requests and data transfer
Tags	Assign tags to objects to use in costing,billing,security and etc
Events	Trigger notifications to SNS,SQS,or lambda when certain events happen in your bucket
Static Web Hosting	Simple and massively scalable website hosting
BitTorrent	Use the BitTorrent protocol to retrieve any publicly available object by automatically generating a .torrent file

You can use S3 for following:
- Backup and Storage: Providing data backup and storage services for others
- Application Hosting: Provides services that deploy,install,and manage web applications.
- Media Hosting: Building a redudant,scalable,and highly available insfrastrucute that hosts video,photo,or music uploads and downloads.
- Software Delivery: Hosting software applications that customers can download.
- Static Website: Hosting static sites such as html pages or blogsite.

Amazon S3 Buckets

Files are stored in the bucket:
- A bucket can be viewed as a container for objects.
- A bucket is a flat container of objects.
- It doesn't provide a hiearchy of objects.
- You can use an object key name(prefix) to mimic folders.
100 buckets per account by default.
You can store unlimited objects in your buckets.
You can create folders in your buckets
You cannot create nested buckets.
An S3 Bucket is region specific.

Amazon S3 Objects:

Each object is stored and retrieved by a unique key (ID or name).
An object in S3 is uniquely identified and addressed through:
- Service end point.
- Bucket Name
- Object Key
- Optionally an object version
Objects stored in a bucket will never leave the region in which they are stored unless you move them to another region or enable cross-region replication.
You can define permissions on objects when uploading and at any time afterwards using the AWS management console.

Amazon S3 Sub-resources

Sub resources (configuration containers) associated with the buckets include:
- Lifecycle - define an object's lifecyle.
- Website - configuration for hosting static sites.
- Versioning - retain multiple versions of objects as they are changed
- Access Control Lists (ACLs) - control permissions access to the bucket.
- Bucket Policies - control access to the bucket.
- CORs (Cross Origin Sharing Resources).
- Logging

Amazon S3 Storage Classes

Storage classes include:
- S3 Standard (durable,immediately available,frequent access)
- S3 Intelligent-Tiering (automatically moves data to the most cost effective tiering)
- S3 Standard-IA (durable,immediately-available,infrequent access)
- S3 One Zone-IA (lower cost for infrequently accessed data with less resilience)
- S3 Glacier (archieved data,longer retrieval times)
- S3 Glacier Deep Archive (lowest cost storage class for long term retention).

Amazon S3 Multipart upload

Multipart upload uploads objects in parts independently, in a parallel and in any order.
Performed using the S3 Multipart upload API.
It is recommended for objects larger than 100MB or 100MB
Can be used for objects from 5MB upto 5TB.
Must be used for objects larger than 5GB.

Amazon S3 Copy

You can create a copy of objects upto 5GB in size in a single atomic operation.
For files larger than 5GB you must use the multipart upload API.
Can be performed using the AWS SDKs or REST API.
The copy operation can be used to:
- Generate additional copies of the objects.
- Renaming the objects
- Changing the copy's storage class or encryption at rest status
- Move objects across AWS locations/regions
- Change object metadata.

Amazon S3 Transfer Accelaration

Amazon S3 Transfer Accelaration enables fast,easy,and secure transfers of files over long distances between your client and your S3 bucket.
S3 Transfer Accelaration leverages Amazon CloudFront's globally distributed AWS Edge Locations.
Used to accelarate object uploads to S3 over long distances (latency)
Transfer accelaration is as secure as a direct upload to S3
You are charged only if there was a benefit in the transfer times
Need to enable transfer accelaration on S3 bucket.
Cannot be disabled, can only be suspended

Amazon S3 Encryption

Option	How it works
SSE-S3	Use S3's existing encryption key for AES-256
SSE-C	Upload your own AES-256 encryption key which uses S3 uses when it writes objects
SSE-KMS	Use a key generated and managed by AWS KMS
Client-Side	Encrypt objects using your own local encryption process before uploading to S3

Amazon S3 Performance

Measure Performance
Scale Storage Connections Horizontally
Use byte-range fetches
Retry Requests for Latency-Sensitive Applications
Combine Amazon S3 (Storage) and Amazon EC2 (Compute) in the Same AWS Region.
Use Amazon S3 Transfer accelaration to minimize Latency Caused by the distance.

Amazon CloudFront

CloudFront is Global Service:
- Ingress to upload objects.
- Egress to distribute the content
You can use a zone apex DNS name on cloudfront
CloudFront supports wildcard CNAME.
Supports SSL certificates, Dedicated IP, Custom SSL and SNI Custom SSL (Cheaper)
You can restrict access to the content using the following methods:
- Restrict access to content using the signed cookies or signed URLs.
- Restrict access to objects in your S3 bucket.
A special type of user called an Origin Access Identity (OAI) can be used to restrict access to content in an Amazon S3 bucket.
By using an OAI you can restrict users so they cannot access the content directly using the S3 url,they must connect via CloudFront.

Amazon CloudFront Edge Locations and Regional Edge Caches

An edge location is the location where the content is cached.
Requests are automatically routed to the nearest edge location
Edge locations are not tied to Availability Zones or Regions
Regional Edge caches are located between origin web servers and global edge locations and have a larger cache.
Regional Edge Caches have a larger cache-width than any individual edge location so your objects remain in cache longer at these locations.
Regional Edge caches aim to get content closer to users.

Amazon CloudFront Origins

An origin is the origin of the files that CDN will distribute.
Origins can be either an S3 bucket,an EC2 instance,an ELB,or Route 53 - can also be external.
A custom origin server is a HTTP server which can be an EC2 instance or an on-premise/non AWS web servers.
Amazon EC2 instances are considered custom origins.
Static sites on Amazon S3 are also considered custom origins.

Amazon CloudFront Distributions

There are two types of distribution.
Web:
- Static and Dynamic content including .html,.js or .css.
- Distributes files over HTTPS or HTTP
- Add,update,or delete objects and data from submit forms.
- Use live streaming to stream an event in realtime.
RMTP:
- Distribute streaming media files using Adobe FLash Media Server's RTMP protocol.
- Allows an end user to begin playing a media file before the file has finished downloading from a CloudFront edge location
- Files must be stored in an S3 bucket.

Amazon CloudFront Charges

You pay for:
- Data Transfer out to Internet
- Data Transfer out to Origin
- Number of HTTP/HTTPS Requests
- Invalidation Requests
- Dedicated IP Custom SSL
- Field Level encryption requests.
You don't pay for:
- Data transfer between AWS regions and Cloudfront
- Regional Edge Cache
- AWS ACM SSL/TLS Certificates
- Shared Cloudfront certificates

Amazon EBS (Elastic Block Store)

EBS volumes are network attached storage that can be attached to EC2 instances.
EBS volume data persists independently of the life of the instance.
EBS Volumes do not need to be attached to an instance.
You can attach multiple EBS volumes to an instance.
You cannot attach an EBS volume to multiple instances (Use EFS instead).
EBS volume data is replicated across multiple servers in AZ
EBS volumes must be in same AZ as the instances they are attached to
Root EBS volumes are deleted on termination by default.
Extra non-boot volumes are not deleted on termination by default.
The behavior can be changed by altering the DeleteOnTermination attribute.

Note: These are comprehensive and covers almost everything but if you like to add more to it feel free fork this and create a PR. I would be happy to add your new changes

A guide to AWS Cloud Practitioner Exam

Muhammad — Tue, 16 Jun 2020 04:19:39 +0000

AWS Cloud Practioner
Where you can take this exam ?
Exam Guide
What is Cloud Computing
Benefits of the cloud computing
Types of Cloud Computing
Cloud Computing Models
AWS Global Infrastructure
Regions AWS
AZs(Availability Zones)
Edge Locations (Get Data Fast or Upload Data Fast)
Gov Cloud
EC2 Instances
AMI (Amazon Machine Image)
AutoScaling
Elastic Load Balancer (ELBs)
S3 Buckets
CloudFront (CDN)
RDS (Relational Database Service)
AWS Lambda
EC2 Pricing Models
- On Demand
- Reserved Instances
- Spot instances
- Dedicated
Billing and Pricing
- AWS Free Services
- AWS Support Plan
- AWS MarketPlace
- AWS Trusted Advisor Check
- Consolidated Billing
- AWS Cost Explorer
- AWS Budgets
- TCO Calculator
- AWS Landing Zone
- AWS Resource Groups & Tagging
- AWS QuickStart
- AWS Cost and Usage Report
AWS Networking
AWS Database Services
AWS Provisioning Services
AWS Compute Services
AWS Storage Services
AWS Business Centric Services
AWS Enterprise Integration
AWS Logging Services
Most Common AWS Initials
AWS Security
- Things Customer should take care of
- Things AWS takes care of
- AWS Compliance Programs
- AWS Artifact
- AWS Inspector
- AWS WAF (Web Application Firewall)
- AWS Shield
- AWS Penetration Testing
- Permitted Services
- Prohibited Services
Amazon Guard Duty Service
KMS (Key Management System)
- Envelope Encryption
Amazon Macie
Security Groups VS NACLs(Network Access Control Lists)
AWS VPN(Virtual Private Network)
Same Name But Different Services (Don't Get Confused)
AWS Connect Services
Elastic Transcoder VS AWS Elemental MediaConvert
SNS vs SQS
- SNS
- SQS (Examples RabbitMQ) - (think of it as message broker service)
- NLB vs ALB vs CLB

AWS Cloud Practioner

This certification is mostly used by people to get the understanding of the AWS services.

Where you can take this exam ?

You can take this exam on Pearson VUE online.

This cost $100
90 mins
65 Questions
70% passing score
Valid for 3 years.

Exam Guide

This is what exam mostly comprises of

Cloud Concepts 28%.
Security 24%
Technology 36%
Billing & Pricing 12%

We have total of 65 questions and most of them are mutiple choice or multiple response questions.

What is Cloud Computing

Its the practice of using a network of remote servers hosted on the internet to store,manage and process data rather using a local server or personal computer.

Benefits of the cloud computing

No upfront costs such as paying for server. You can Pay On Demand.
Economies of sale you can simply save alot of money since there are so many people using the cloud.
You can scale up or scale down based on your need.
With few clicks of a button your service is deployed.
No more maintenance costs.
Go global in few minutes since there are global regions where cloud servers are hosted.

Types of Cloud Computing

SaaS --> Software as a Service. Basically a complete product that is ran and managed by the service provider.(Examples: Salesforce,Gmail,GoogleDocs)
PaaS --> Platform as a Service. Focusing on deploying applications without worrying about managing the infrastructure. (Examples: Heroku,Netlify and etc)
IaaS --> Infrastructure as a Service. The building blocks of the IT. Providing computers access and storage needs and etc. (Examples: AWS,GCP and Azure).

Cloud Computing Models

Cloud (Fully hosted on Cloud such as startups)
Hybrid (On-Premise and Public Cloud such as Banks)
On-Premise (On Private Cloud where sensitive data is being stored).

`AWS` Global Infrastructure

There over a million active customers using aws.
there are total 69 Availability Zones 22 Geographic Regions around the world.
Regions --> physical location with multiple availabilty zones.
Availability Zones(AZ) --> one or more discrete data locations.(owned by aws)
Edge Locations --> data center owned by trusted partner of aws

Regions `AWS`

A geographically distinct location with multiple data centers.
Each region has two AZs.
US-EAST(North Virginia) is the largest AWS region and services almost always become available first in this region.
Not all services are available in the all regions.
US-EAST-1 is where you see your billing information.

AZs(Availability Zones)

Each Region has two AZs.
An AZ is a data center ran and owned by AWS.
less than 10ms latency between AZs

Edge Locations (Get Data Fast or Upload Data Fast)

A data center owned by trusted partner of AWS and has direct connection to the aws network.
These location serve requests for CloudFront and Route53. Requests going to either of these services will be routed to the nearest edge location automatically.
S3 Transfer accelaration and API Gateway endpoint also use the AWS Edge Network.
This allows for low latency no matter where ever you are located.

Gov Cloud

An AWS service that allows customers to host sensitive Controlled
Unclassified Information or other types of workloads

Only operated by US Citizens or on the US Soil.
Should follow several compliance guidelines.
GovCloud

EC2 Instances

In order to create a EC2 instance head over to AWS Console.
Choose EC2 and follow along. Make sure you select Amazon Linux 2 AMI and select the type as t2.micro since that is offered with free tier.
Now follow along and make sure to set IAM role.
lastly make sure you have billing alerts turned on.
You can either use ssh or sessions manager to get into ec2 instance.
You can also get to sessions-manager by going to systems manager.
sessions-manager opens a simple bash shell that can help you access your ec2 instance.

AMI (Amazon Machine Image)

You can create an image by going into ec2 management console and clicking on actions and selecting image.
Basically this creates a copy that allows you launch multiple servers.

AutoScaling

This allows you ensure that multiple instances and multiple servers are running.
This also allows you meet the demand of web traffic.
In order to configure this its located in ec2 management console.

Elastic Load Balancer `(ELBs)`

This allows reroute the traffic. especially when doing updates to the application.

S3 Buckets

Its usually global and the buckets are usually region specific.
its a block storage used to store the files.

CloudFront `(CDN)`

CloudFront is basically Cotent Delivery Network.
It makes easier for companies to distribute there content.
Hook it up to S3 Bucket and deliver your content around the world.

RDS `(Relational Database Service)`

It's used to setup a relational database (Examples: SQL,PSQL).
Amazon aurora would be default when setting up this service
It has auto scaling.

AWS Lambda

Serverless framework.
Allows you to run simple functions you can think of it as cronjobs.

EC2 Pricing Models

E2 has four different pricing models

On Demand

Low Cost and Flexible.
only charges per hr
short term
good for first time apps or prototypes.
No upfront payment.

Reserved Instances

Good for committed applications.
Standard savings 75% (Cannot change the RI attributes.)
Best for long term
You can schedule to reserve the instances.
there's a commitment like 1 to 3 year with AWS.
RI's can be shared between multiple accounts.
You can even sell your unused instances.

Spot instances

You can think of it has a hotel who offers discounts to fill there spots.
Just like hotel aws uses similar approach to maximize the usage of there idle servers.
There are conditions such as,
- Instances can be terminated anytime.
- If you instance gets terminated you dont get charged for partial hour of usuage.
- If you terminate an instance you will be charged for any hour that it ran.
Good for applications feasible for very low usage.
It provides you 90% savings.

Dedicated

Its the most expensive EC2 instance.
Its built for tenant customers. Its more useful for large enterprises.
Its offered both in demand and reserved.

Billing and Pricing

AWS Free Services

IAM (Identity Access Management) --> used for creating user roles.
Auto Scaling
CloudFormation
Elastic Bean
Opswork
Amplify
AppSync
CodeStar
AWS Cost Explorer

NOTE: Services in bold are free but they can provision AWS Services to cost money.

AWS Support Plan

$0/month - Basic (Support Only by Email).
$20/month - Developer (Tech Support Via Email reply within 24hrs)
- No Third Party Support.
- General Guidance only.
$100/month - Business (Tech Support Via Chat/Phone 24/7)
- Does support third party.
- Production system down less than 1hr response time (business downtime)
$15000/month - Enterprise (Tech Support Via Chat/Phone 24/7)
- Personal Concierge.
- TAM (Technical Account Manager)
- Response time less than 15min. (business downtime)

AWS MarketPlace

A place where you will thousands of software listings from independant software vendors.
The product is free to use or can have a charge which becomes part of the
AWS bill.

AWS Trusted Advisor Check

This advises customers on security,saving money,performance , service limits and fault tolerance.
FREE has 7 trusted Advisor Checks
Enterprise and Business - All trusted advisor checks.

Consolidated Billing

One Master account for all member accounts.
Cost Explorer tool for visualizing the usage.
It also offers volume discounts (The more you use the cheaper it gets.)

AWS Cost Explorer

Allows you to visualize the usage of the multiple accounts.

AWS Budgets

First two budgets are free of charge.
Allows you setup alerts when you exceed your limits.
You can set three types of alerts Budget,Usage and instance reservation
Alerts supports EC2,RDS,RedShift and Elastic Cache.
You can manage budgets from AWS Budget Dashboard or Budgets API.
Get notified through email or ChatBot.

TCO Calculator

The Total Cost of Ownership calculator allows you show how much can save by shifting to aws.

A tool that allows you build reports for execs to show how much you can save.
Only for approximation purposes.

AWS Landing Zone

Meant for enterprises.
Automatically provisions and configure new accounts via Service Catalog template.
uses SSO.

AWS Resource Groups & Tagging

Tags are words or phrases that act as metadata for organizing AWS resources.
Resource Groups are collection of resources that share one or more tag.
Resource Group can display following details of about a group of resource based on.
- Metrics
- Alarms
- Configuration Settings.

AWS QuickStart

Prebuilt templates offered by AWS or AWS partners that helps you deploy popular stacks on AWS. This allows to reduce the manual effort.

It's divided into three steps.

A reference architecture for deployment.
AWS CloudFormation templates that automate and configure the deployment.
A guide explaining the architecture and implementation in detail.

AWS Cost and Usage Report

This allows you to generate a detailed report of your AWS costs.
You'll get a spreadsheet highlighting the costs.

Reports are stored in S3.
You can use ATHENA to turn this into queryable database.
QuickSight for analyzing.

AWS Networking

Region -> The geographic location of the network.
VPC -> An isolated space of aws where you can launch aws resources.
AZ -> the data center of the aws resources.
Security Groups -> Acts as a firewall at the instance level.
Internet Gateway -> Enables access to the internet.
NACLs -> acts as a firewall at the subnet level.
Route Tables -> determine where network traffic from your subnets are directed.
Subnets -> A logical partition of an IP network into multiple,smaller network segments.

AWS `Database` Services

DynamoDB -> NoSQL key/value database. (Examples:Cassandra).This is really fast for read and write access.
DocumentDB -> NoSQL Document database that is compatible to MongoDB.
RDS -> Relational DataBase Service that supports multiple engines MySQL,Postgres,MariaDB. (Most Popular DB)
AuroraDB -> MySQL (5x fast) and PSQL (3x Fast) fully managed database. (Runs 6 copies of the Database when used and more expensive DB)
Aurora Serverless -> Only runs when needed.
Neptune -> Graph DataBase.
RedShift -> Columnar Database petabyte warehouse.
Elastic Cache -> Redis or Memecached database.

AWS `Provisioning` Services

Elastic BeanStalk -> Think of it as Heroku. Its a service used for deploying and scaling the web applications and services deployed with Java,python,c++ and etc (Perfect for deploying WebApps)
OpsWork -> Configuration management service that provides managed instances of CHEF and Puppet.(It has layers like tier 2 or tier 3)
CloudFormation -> IaaS , Infrastructure as Code JSON or YAML.
AWS QuickStart -> ready made templates that can launch and configure your aws compute, network, and other services.
AWS MarketPlaces A place where you can buy or sell software or services for AWS Cloud.

AWS Compute Services

EC2 -> Elastic Compute Cloud highly configurable server.
ECS -> Elastic Container Service Docker As Service highly scalable,high performance and good for microservices.
Fargate -> You don't chose the ec2 like you might chose in ECS. You define and AWS will run the service. (Like Lambda since you dont pay for EC2)
EKS -> Kuberenetes as services makes it easy to deploy ,manage and scale.
Lambda Serverless. Just upload code as function and AWS will run the code for you.
Elastic BeanStalk -> upload the code and it will do the rest for you. Good for developers who want to just upload there apps.
AWS Batch -> Its for Batch Processing where you can schedule Batch jobs.

AWS Storage Services

S3 -> A simple storage service - Object Store (Simply Upload Files).
S3 Glacier -> low cost for storage and good for archiving the data for long term backup.
Storage Gateway -> A hybrid solution from on premisis to cloud for storage.
EBS (Elastic Block Storage) -> A hard drive in cloud you attach to ec2 instance such as SSDs,HDDS.
EFS (Elastic File Storage) -> file storage moutable to multiple EC2 instances at the same time.
Snowball -> A way of moving data from on premise to aws.
- Snowball Edge -> 100 TB (better version and additional features).
- SnowMobile -> Allows to move petabytes of data (DataCenter on Wheels).

AWS Business Centric Services

Amazon Connect -> Cloud Based call center service you can setup in few minutes and later you can save the calls in s3 for furhter analysis.You can even route calls based on defined rules
WorkSpaces -> Secured managed virutal desktops.
WorkDocs -> aws version of sharepoint where you can collaborate and share documents.
Chime -> Think of it as skype where you can do business calls and meetings.
WorkMail -> Managed aws email service just like Microsoft Outlook Exchange uses IMAP Protocol
PinPoint -> For marketing campaigns for targetted sending emails and sms notifications.
SES (Simple Email Service) A cloud based email sending service used to send emails and notifications (Good for webapps that supports sending email notifications and has HTML format email option)
QuickSight. Think of it as QlikSense or Tableau as this allows you to visualize the data.

AWS Enterprise Integration

Direct Connect -> A dedicated Gigabit connection from on premise to aws.
VPN
- Site to Site -> Connecting to on premise to aws.
- Client Vpn -> Connecting a client to AWS network.
Storage Gateway A hybrid storate service that enables on premise applications to use AWS cloud storage.
- Good for backup.
- Archiving
- Disaster Recovery.
- migration
- data processing.
AD (Active Directory) An AWS directory service for Microsoft Active Directory also known as AWS Managed Microsoft AD - Enables your workloads and AWS related resources to use managed AD in aws cloud.

AWS Logging Services

CloudTrail -> a logging service that logs all the api calls (SDK,CLIs) between AWS services.
- Who created the service.
- Who spun up the EC2 instance.
- Who launched sagemaker notebook
- Detects developer misconfigurations.
- Detects Malicious Activity.
- Automates responses.
CloudWatch A collection of multiple services. Its more like a storage solution for all the logs.
- Stores all types of the logs.
- CloudWatch Metrics -> timeseries data of logs.
- CloudWatch Events -> trigger event based on a condition.(Taking the snapshot of the server)
- CloudWatch Alarms -> trigger notifications based on metrics.
- CloudWatch Dashboard -> create visualizations based on metrics.

Most Common `AWS` Initials

IAM:Identity Access Management.
S3: Simple Storage
SWF: Simple Workflow Service.
SNS: Simple Notification System.
SQS: Simple Queue Service.
SES: Simple Email Service.
SSM: Simple Systems Manager.
RDS: Relations DataBase Service.
VPC: Virtual Private Cloud.
VPN: Virtual Private Network.
CFN: Cloud Formation
WAF: Web Application Firewall.
MQ: Amazon ActiveMQ.
ASG: AutoScaling Groups.
TAM: Technical Account Manager.
ELB: Elastic Load Balancer.
ALB: Application Load Balancer.
NLB: Network Load Balancer.
EC2: Elastic Cloud Compute .
ECS: Elastic Container Service.
ECR: Elastic Container Repository.
EBS: Elastic Block Storage.
ELF: Elastic File Storage.
EMR: Elastic MapReduce.
EB: Elastic Beanstalk.
ES: Elastic Search.
EKS: Elastic Kubernetes Service.
MKS: Managed Kafka Service.
IoT: Internet of Things.
RI: Reserved Instances.

AWS Security

In the Shared Responsibility Model the customer is responsible for the security of the cloud such as securing the data and using the right configuration.
and anything the customer can't touch or get access is secured by aws. Such as Hardward,Operation of Managed Services and Global Infrastructure.

Things Customer should take care of

IAM
Customer Data
OS,Network and Firewall Configs.
Maintaining Encryption Protocols.

Things AWS takes care of

Software
Hardware
Services that aws provides.

AWS Compliance Programs

A set of internal policies and procedures of a company to comply with rules and regulations or to uphold reputation.

two most popular ones

HIPPA
PCI (Payment Card Data/You can readmore by googling it)
ReadMore Here

AWS Artifact

A no cost, self service portal for on demand access to AWS compliance reports.

AWS Inspector

AWS inspector is a tool that runs a security benchmarks against specific EC2 instances. The most popular one is run by CIS (Center for Internet Security) which has 699 benchmarks.It can even inspect the network to check if there are any ports are open and running.

AWS WAF (Web Application Firewall)

It allows you protect your web application against the most common exploits.
You can write your own rules that will allow the traffic based on the contents of HTTP requests. You can use ruleset from AWS trusted secuirty partner. It can be either attached to CloudFront or Application Load Balance (ALB).

Most Common Attacks Include

Injection
Broken Authentication
Sensitive Data Exposure
XML External Entities XXE
Broken Access Control
Security Misconfigurations
XSS Cross Site Scripting
Insecure Deserialization
Using Components with known Vulnerabilities
Insufficient logging and Monitoring

AWS Shield

A managed DDOS(Distributed Denial of Service) protection service that safeguards applications running on aws.

All AWS customers benefit from the automatic protections of AWS shield standard at no charge.

When you route your traffic through ROUTE53 or CloudFront you are using AWS Shield.

Protects you against Layer3,Layer4, and Layer7 attacks

7 Application
4 Transport
3 Network

There's also a paid tier known as Shield Advance and that costs $3000/Year(upfront or Commitment).
It gives you extra protection with 24/7 support and its available on

Amazon Route 53
Amazon CloudFront
ELB
AWS Global Accelarator
Elastic IP(Amazon Elastic Compute Cloud and Netword and Load Balancer).

AWS Penetration Testing

An authorized service that allows you simulate a cyber attack on a computer system, performed to evaluate the security of the system.

Permitted Services

EC2 Instances,NAT Gateways, and ELB.
RDS
CloudFront
Aurora
API Gateways
AWS Lambda and Lambda@Edge function
LightSail resources.

Prohibited Services

DNS zone walking via Amazon Route 53 Hosted Zones.
DoS(Denial Of Service),DDoS,Simulated DoS,Simulated DDoS.
Port flooding
Protocol flooding
Request flooding

Amazon Guard Duty Service

IDS: Intrustion Detection System

IPS: Intrustion Protection System

A device or software that monnitors a network or systems for malicious activity or policy violations.

Guard Duty is a threat detection service that continuously monitors for the malicious,suspicious activity and unauthorized behavior. It uses machine learning ty8111o analyze following AWS logs.

CloudTrail Logs.
VPC Flow Logs
DNS logs.

It will alert you of the findings which you can automate an incident response via CloudWatch Events or 3rd Party Software.

KMS (Key Management System)

A managed service that makes it easy for you to create or control the encryption keys used to encrypt that data.

KMS is a multi-tenant HSM (Hardware Security Model).
Many AWS services are intergrated to use KMS to encrypt your data with simple checkbox.
KMS uses Envelope Encryption.

Envelope Encryption

When you encrypt your data, your data is protect but you have to protect your encryption key. When you encrypt your data key with master key as an additional layer of security. READMORE

Amazon Macie

Macie is fully managed service that continuously monitors S3 data access activity for anomalies and generates detailed alerts when it detects risk of unqthorized access or inadvertent data leaks.

It uses machine learning to analyze CloudTrail logs.

It provides you with following alerts.

Anonymized Access.
Config Compliance
Credential loss
Data Compliance
File Hosting
Identity Enumeration
Information Loss
Location Anomaly
Open Permisson
Privilege Escalation
Ransomware
Service Disruption
Suspiscious Activity

Security Groups VS NACLs(Network Access Control Lists)

Security Groups	Network Access Control Lists (NACLs)
Acts as a firewall at the instance level.	Acts as a firewall at the subnet level
Implicitly denies all traffic. You create Allow rules (For Example allow an `EC2` Instance to access `port 22`)	You create `allow` and `deny` rules

AWS VPN(Virtual Private Network)

It allows you create a secure and private tunnel from your network or device to the aws global network.

AWS Site-to-Site VPN : Securely connect to on premises network or branch office site to VPC.
AWS Client VPN: Securely connect users to AWS or on premises networks.

Same Name But Different Services (Don't Get Confused)

CloudFormation: IaaS (Infrastructure as a Service) used to setup template scripting (YAML,JSON).
CloudTrail: logs all the api calls between aws-services (who to blame).
CloudFront: CDN(Content Delivery Network),It is used to distribute the content (Such as videos,static assets and etc).
CloudWatch: a collection of multiple services
CloudSearch: search engine for your site (Ecommerce).

AWS Connect Services

Direct Connect: A dedicated fiber optics connections from data center to AWS.
Lets say an enterprise want a direct connection from there on premise datacenter to aws they might use this service to connect to AWS.
If you want to add extra layer of security you might need a vpn connection.
Amazon Connect: Call Center Service.
Media Connect: A new version of Elastic Transcoder, Converts videos to different formats.

Lets say you have 1000 videos and you need to transcode them into different formats then this might be a useful service. You can even add watermarks and insert intro infront of every video.

Elastic Transcoder VS AWS Elemental MediaConvert

Elastic Transcoder `OldWay`	AWS Elemental MediaConvert `NewWay`
Transcodes videos to streaming formats	Transcodes videos to streaming formats
	Overlay images
	Insert Video Clips
	Extract captions data
	Better and Robust UI

`SNS` vs `SQS`

They Both Connect Apps via Messages.

SNS

It uses PubSub model which is also known as publisher subscriber model.
It sends notifications to subscribers via protocols such as HTTP,Email SQS and SMS.
It is generally used for sending plaintext emails which is triggered via other aws services. The best example can be billing alerts.
Can retry sending in case of failure for HTTPS.
Its good for webhooks,internal emails and triggering lambda functions.

SQS (Examples `RabbitMQ`) - (think of it as message broker service)

Queue Up Messages, Guaranteed Delivery

It places messages into a queue and applications pull queue using AWS SDK.
It can retain a message for up to 14 days.
Can send them in sequential order or parallel.
Can ensure only one message is sent.
Can ensure messages are delivered at least once.
Really good for delayed tasks,queueing up emails.

You can readmore about this here

NLB vs ALB vs CLB

Application	Network	Classic
`Layer 7` Requests	`Layer 4` IP protocol data	`Layer 4` and `Layer 7`
`HTTPS` and `HTTP` traffic	`TCP` and `TLS` traffic where extreme performance is required (Example: Netflix)	Intended for applications that were built within the `EC2 Classic network`
Routing Rules,more usability from one load balancer	Capable of handling millions of requests per second while maintaining `ultra-low-latencies`	Doesn't use Target Groups
Can attach WAF(WebApp Firewall)	Optimized for `sudden and volatile` traffic patterns while using a single static IP address per AZ

Can Attach ACM (Amazon Certification Manager) SSL Manager

Anyways this all you need to know for your AWS Cloud Practitioner Exam and I hope you found these helpful.

I am always available through twitter
and email

Originally Posted here

Creating Simple Web App Using Django

Muhammad — Fri, 15 Nov 2019 18:41:42 +0000

In this post I will be walking you through the process of creating a simple web app using django web framework.

What is `Django` ?

It is a web framework that allows us to build web apps rapidly without inventing everything from scratch. It uses MVC architecture that is known as Model View and Controller. Basically Model deals with data, View deals with web logic and Controller is basically url dispatcher. Furthermore, Django uses ORM (Object Relational Mapping) that allows us to interact with application data from Relational Database.

Getting Started with `Django`

In order to get started with django make sure we have python and pip installed.
Once we have python and pip now we can install pipenv that will allow us to manage project dependencies. So in order to install pipenv you can refer to the documentation. Pipenv.
Now we have pipenv , we can now create project folder where we will be installing django. In order to install django, first we need to activate our virtual enviroment using this command pipenv shell once we have shell activated now we can run this command pipenv install django to install django.
Finally we have django installed.

Starting Project

Now we have django up and running, it's time start our first django project. In our case we will name it as newsapp. Once you run this command django-admin.py startproject newsapp a folder will be generated with bunch of files.

__init__.py file is present to tell python interpreter that this directory is package.
setting.py file contains settings for the project and thats where we add our apps and middleware. This file contains the list of apps and middleware used by the django project.
urls.py file contains the project level url information and connects our apps with the project.
wsgi.py file is important if you want to deploy your application to apache server. as we know django is based on python and python uses wsgi server.
manage.py file is usually outside the project and it provides us with useful commands such as runserver,makemigrations,migrate and etc.

Now you can run this command to run the server python manage.py runserver and view it on http://127.0.0.1:8000/

Performing system checks...

System check identified no issues (0 silenced).

You have 17 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): admin, auth, contenttypes, sessions.
Run 'python manage.py migrate' to apply them.

November 07, 2019 - 16:30:33
Django version 2.2.6, using settings 'newsapp.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

Note: Don't worry about the unapplied migrations.

Creating Django App

Difference between projects and app.

In simple words app is something that does something such as a social media network,a blog,news app and project is collection of configurations and apps for a particular website.

Creating app

Once we have our django project created now we can create our financial news app by runnning this command python manage.py startapp financialnewsapp.

Now before we begin onto next step let me give an overview of each file in the app we just created.

admin.py file is a configuration file for built in django DJANGO ADMIN.
apps.py file is configuration file for the app.
tests.py file is where we write unit tests for the app.
views.py file handles our request and response.
models.py file is where we define our data using python classes that is turned in tables using ORM.
migrations/ folder keep tracks of every database migration we make.

Before we go to next step lets add our newly created app to SETTINGS.PY file in newsapp/ project.

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'financialnewsapp'
]

As you can see it already has some pre installed apps. So these pre installed apps offer us prebuilt solution to deal with things like authentication,staticfiles and etc.

Writing our view

Now we will write our view that queries top 10 bitcoin news articles and display them on html page. In order to do accomplish that first we need to install newsapi and get apikey. To install newsapi we can use this command pipenv install newsapi-python.
Before you can access the api you need the KEY

Once we have newsapi we can write our view that will request news articles and display it on HTML pages.

In our views.py we need to add the following.

from django.shortcuts import render
from django.shortcuts import render
from newsapi import NewsApiClient
import json
# Initializing API KEY
newsapi = NewsApiClient(api_key='xxxxxxx')
all_articles = newsapi.get_everything(q='bitcoin',sources='yahoo,the-verge,cnbc,bloomberg',
language='en',sort_by='relevancy')

# loading all_articles as json
new = json.dumps(all_articles)
# This will allow us to create creat dictionary from the json which will make easier for us to use the data
data = json.loads(new)
def index(request):
    return render(request,'news/index.html',{'data':data})

Once we have created our view that has all our logic. Now we will create url that will request this view but first we need to add our app urls in project urls.py file

"""newsapp URL Configuration

The `urlpatterns` list routes URLs to views. For more information please see:
    https://docs.djangoproject.com/en/2.2/topics/http/urls/
Examples:
Function views
    1. Add an import:  from my_app import views
    2. Add a URL to urlpatterns:  path('', views.home, name='home')
Class-based views
    1. Add an import:  from other_app.views import Home
    2. Add a URL to urlpatterns:  path('', Home.as_view(), name='home')
Including another URLconf
    1. Import the include() function: from django.urls import include, path
    2. Add a URL to urlpatterns:  path('blog/', include('blog.urls'))
"""
from django.contrib import admin
from django.urls import path ,include

urlpatterns = [
    path('admin/', admin.site.urls),
    # This line will look for urls in app
    path('',include('financialnewsapp.urls'))
]

Now we need to create new urls.py file in our financialnewsapp/ folder and add the following to the file.

from django.urls import path
# Import our views
from . import views
# When we request home/index url it use index view logic
urlpatterns = [
    path('',views.index,name='index')
    ]

Lastly now we need to create templates/ folder inside of our app directory where django would look for our html files.

Once we have created templates/ we need to create another folder within that directory where we will save our html file. In our case it would be something like this templates/news/index.html

After creating the directory and file now paste the following code in the file.

So I am using Bulma to make it look clean. Furthermore as you have noticed that django uses jinja templating engine to render dynamic data from the database or api.

As you can see I am using a for loop to iterate through articles and using key of dictonary access the values and display it using these double curly braces tags.

Note:

Single curly brace are used for conditions , for loops or extending a template in jinja templating engine
Double curly brace is used to evaluate something which in our case we are getting the value of the each key.

So this was a simple django app that displays news related to bitcoin. I hope you enjoyed this post if you think I missed any thing feel free to dm me on twitter and I will be covering about
database and models.py in an another post soon.

End Result

Links

This article was published here

Natural Language Processing

Muhammad — Tue, 08 Oct 2019 18:42:10 +0000

NLP is a branch of computer science that allows computers to understand Human Language.Using NLP we are able to derive meaningful insights and use them in practical applications such as ChatBots,Spam filtering,spell check ,making google search better and the list goes on ….

NLP has few steps such as

Tokenization
Stemmation.
Lemmetization.
POS Tags(Parts of Speech).
Named Entity Recognition.
Chunking.

Tokenization

Tokenization is the first step of the NLP process. It’s process of splitting text into minimal meaningful units so our machine can understand.Furthermore Read

Stemmation

In simplest terms Stemmation is a process of getting a root word. For instance if there are words such as Plays,Played,Playing for this example the root word is Play. Stemming is usually done by stripping the prefixes and suffixes from the words.

Further Read

Link 1.
Link 2.

Lemmetization

Lemmetization is more sophisticated technique compared stemmation. As we know stemmation just gets the root word for instance words like car wont be matched with automobile when doing stemmation. But in case of lemmmetization car will be matched with automobile. In lemmatization, the part of speech of a word should be first determined and the normalisation rules will be different for different part of speech.

Note : Stemmation only strips down words to root words while stripping prefixes and suffixes. While lemmatization will put together words by the use of correct vocabulary. For instance , car will be matched with automobile. or truck will be matched with lorry.

Further Read

POS

Tagging words with correct parts of speech.

Named Entities.

This process is related to defining the named entities in the text. For instance Mark Zuckerburg is the CEO of Facebook. In this examples Mark Zuckerburg and Facebook is a named entity.

Chunking

Chunking is a process of extracting phrases from unstructured text. Instead of just simple tokens which may not represent the actual meaning of the text, its advisable to use phrases such as “South Africa” as a single word instead of ‘South’ and ‘Africa’ separate words.

** Further Read **

Link

Using Regex To Extract Links.

Muhammad — Tue, 08 Oct 2019 00:00:00 +0000

Did you know we can use this regular expression to extract links

(?:(?:https?|ftp):\/\/)?[\w/\-?=%.]+\.[\w/\-?=%.]+

This will match all the urls in the file and we can write a python script to extract the urls.

text = "<CONTAINING URLS>"
urls = re.findall('(?:(?:https?|ftp):\/\/)?[\w/\-?=%.]+\.[\w/\-?=%.]+', text)
print(urls)

DEV Community: Muhammad

Creating a Simple Pastebin Service in Python and Flask

Getting Starting

Writing Code

Code Breakdown

Conclusion

Understanding Python Variables: Namespaces and Variable Scope

What is a Namespace?

Types of Namespaces

Variable Scope

Understanding Scope with Examples

The global and nonlocal Keywords

Avoid Variable Shadowing

Conclusion

Commandline Productivity Part 1: fzf - The Command-Line Fuzzy Finder

What's fzf and why use it?

What's Fuzzy Matching?

How it works? In Simplest Terms

Algorithms used within Fuzzy Matching

So Now what's fzf and why use it?

Setting up and using fzf.

fzf usage

WebScraping in Bash

The Bash Script

How it Works?

Breaking it down further

Conclusion

Using Commandline To Process CSV files

Grep and Log Analysis

What is grep? and How it Works?

How it works?

Backstory of Development

grep and Log Analysis

Isolating Errors

Tracking Down Issues

Identifying Patterns

Conclusion

Examples

Isolating Errors:

Tracking Down Issues:

Identifying Patterns:

Exploring Linux File System

What is File System and How they work?

Linux File System.

The Directory Structure.

bin/

boot/

sbin/

dev/

etc/

lib/

media & mnt/

opt/

proc/

root/

run/

srv/

sys/.

tmp/

usr/

var/

home/

Originally Posted Here

Everything you need to know for SAA Exam

Solutions Architect Associate Exam

Exam Domains

IAM (Identity Access Management)

IAM Authentication Methods.

MFA (MultiFactor Authentication)

STS (AWS Security Token)

AWS Global Infrastructure Overview

VPC (Virtual Private Cloud)

EC2 (Elastic Compute Cloud)

Security Groups.

Instance Metadata

Instance Userdata

Status Checks and Monitoring

Public,Private and Elastic IP addresses.

Private Subnets and Bastion Hosts

NAT Instances and NAT Gateways

The `global` and `nonlocal` Keywords

What's `fzf` and why use it?

So Now what's `fzf` and why use it?

Setting up and using `fzf`.

`fzf` usage

`grep` and Log Analysis

`bin/`

`boot/`

`sbin/`

`dev/`

`etc/`

`lib/`

`media & mnt/`

`opt/`

`proc/`

`root/`

`run/`

`srv/`

`sys/`.

`tmp/`

`usr/`

`var/`

`home/`

EC2 (`Elastic Compute Cloud`)

- `HTTPS`,`HTTP`

EC2 (`Elastic Compute Cloud`)

- `HTTPS`,`HTTP`