DEV Community: Joseph Ibeh

Terraform Commands Every Cloud Engineer Should Know

Joseph Ibeh — Tue, 22 Apr 2025 07:18:10 +0000

Infrastructure as Code (IaC) has revolutionized the way cloud environments are managed, and Terraform stands out as one of the most popular IaC tools today. Whether you're provisioning compute resources, managing storage, or automating networking setups, Terraform provides a consistent and powerful CLI experience.

In this article, I'll walk you through essential Terraform commands every Cloud Engineer should know—especially if you're just getting started or want a solid foundation for managing infrastructure efficiently.

1. `terraform init`

terraform init

This is the first command you run when starting a new Terraform configuration. It initializes the working directory, downloads the required provider plugins, and prepares your project for further actions.

Use it when:

You clone a new Terraform repo
You add/change providers
You're setting up a new module

2. `terraform validate`

terraform validate

This command checks your configuration files for syntax errors and ensures they're internally consistent.

Think of it as your syntax checker.

3. `terraform plan`

terraform plan

This is your preview command. It shows what actions Terraform will take to reach the desired state, without actually making any changes.

Why it's important:

Avoid unintended resource changes
Review planned changes with your team

4. `terraform apply`

terraform apply

This command executes the changes outlined in the plan. You’ll be prompted to approve before any infrastructure is provisioned or updated.

You can auto-approve using:

terraform apply -auto-approve

Pro tip: Avoid using -auto-approve in production unless you really know what you're doing.

5. `terraform destroy`

terraform destroy

Need to tear down everything? This command removes all infrastructure created by Terraform. Use with caution!

6. `terraform output`

terraform output

Displays the values of output variables from your configuration. This is useful for retrieving endpoints, resource IDs, and other important data post-deployment.

7. `terraform fmt`

terraform fmt

Automatically formats your Terraform code according to standard conventions. This keeps your code clean, readable, and consistent across teams.

8. `terraform show`

terraform show

Want to inspect your Terraform state? This command gives you a human-readable view of what's currently managed in your state file.

9. `terraform state`

terraform state list

Manage the current state of your resources. You can list, pull, move, or remove resources directly from the state file—but this command is powerful and risky, so use with understanding.

10. `terraform taint`

terraform taint <resource_name>

Marks a specific resource for destruction and recreation during the next apply. Useful when a resource is in a bad state and you want to force a rebuild.

Bonus: `terraform import`

terraform import <resource_type.resource_name> <resource_id>

This command lets you bring existing infrastructure under Terraform management. It doesn’t generate the code for you, but it links the resource to your state file.

Final Thoughts

Mastering these Terraform commands is a game-changer for any cloud engineer. They form the core toolkit for managing, updating, and troubleshooting infrastructure as code. Whether you're deploying your first EC2 instance or managing multi-cloud environments, these commands will make your work efficient and reliable.

Got more Terraform tips or favorite commands?

Drop them in the comments—I’d love to hear from fellow DevOps and Cloud Engineering enthusiasts!

Building and Automating the Deployment of a Flask Application with Docker, GitHub, and CI/CD

Joseph Ibeh — Fri, 11 Apr 2025 13:40:24 +0000

Introduction

This project demonstrates how to build, containerize, and deploy a simple Flask application with Docker, using GitHub for version control and GitHub Actions for Continuous Integration/Continuous Deployment (CI/CD). You will learn how to:

Create a simple Flask web application.
Containerize the application using Docker.
Set up version control with Git and GitHub.
Implement CI/CD pipelines using GitHub Actions to build and deploy the Docker image.

⚙️ Prerequisites

Before you begin, ensure that you have the following tools installed:

Python: A programming language for building web applications.
Flask: A lightweight Python web framework.
Docker: A platform for developing, shipping, and running applications in containers.
Git: A version control system for tracking changes in code.
GitHub: A platform for hosting and sharing code repositories.
Docker Hub: A cloud-based registry to share Docker images.

If you don't have these installed, refer to their respective official websites for installation instructions.

✅ Step 1: Create a Simple Flask Application

Create a new folder for the project:

mkdir flask-devops-app
cd flask-devops-app

Set up a Python virtual environment:
```
python -m venv venv
```

Activate the virtual environment:

source venv/Scripts/activate  # On Windows use: venv\Scripts\activate

Install Flask:
```
pip install flask
```

Create the app.py file:

# app.py
from flask import Flask

app = Flask(__name__)

@app.route('/')
def hello():
    return "Hello DevOps!"

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)

Run the app locally:
```
python app.py
```
Visit http://localhost:5000 in your browser. You should see "Hello DevOps!".

✅ Step 2: Containerize the Application

Create a Dockerfile in the same folder:

# Dockerfile
FROM python:3.9-slim

WORKDIR /app

COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt

COPY . .

CMD ["python", "app.py"]

Generate requirements.txt:
```
pip freeze > requirements.txt
```
Create a .dockerignore file:
```
__pycache__/
*.pyc
venv/
```
Build the Docker image:
```
docker build -t flask-devops-app .
```

Run the Docker container:

docker run -p 5000:5000 flask-devops-app

Open http://localhost:5000 again and see your app running inside a Docker container.

✅ Step 3: Set Up Version Control

Tools: Git + GitHub

Initialize a Git repository:
```
git init
```
Create a .gitignore file:
```
venv/
__pycache__/
*.pyc
```

Commit your files:

git add .
git commit -m "Initial commit"

Create a GitHub repository and link it to your local repo:

git remote add origin https://github.com/YOUR_USERNAME/flask-devops-app.git
git branch -M main
git push -u origin main

✅ Step 4: Implement CI/CD with GitHub Actions

Tools: GitHub Actions, Docker Hub

Create a .github/workflows/docker-publish.yml file:

mkdir -p .github/workflows
touch .github/workflows/docker-publish.yml

Edit the docker-publish.yml file and add the following content:

# .github/workflows/docker-publish.yml
name: Build and Push Docker Image

on:
  push:
    branches: [ main ]

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2

      - name: Log in to Docker Hub
        uses: docker/login-action@v2
        with:
          username: ${{ secrets.DOCKER_USERNAME }}
          password: ${{ secrets.DOCKER_PASSWORD }}

      - name: Build and push image
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: YOUR_DOCKERHUB_USERNAME/flask-devops-app:latest

Add Docker Hub credentials as GitHub Secrets:
- Go to your GitHub repository → Settings → Secrets and variables → Actions
- Add the following secrets:
  - DOCKER_USERNAME → your Docker Hub username
  - DOCKER_PASSWORD → your Docker Hub password or access token

Push your code again:

git add .
git commit -m "Add CI/CD workflow"
git push

GitHub Actions will now automatically build your Docker image and push it to Docker Hub whenever you push to the main branch.

Final Output

✅ A live Flask app running inside a Docker container.
✅ The source code is hosted on GitHub.
✅ The Docker image is automatically built and pushed to Docker Hub on every commit.

🎯 Conclusion

Congratulations! You've successfully created a simple Flask app, containerized it with Docker, set up version control with Git and GitHub, and implemented CI/CD with GitHub Actions. This project is a foundational step in building scalable and automated systems in DevOps.

Troubleshooting

If Docker is not running: Ensure Docker Desktop is open and running.
If you encounter permission issues with GitHub Actions: Double-check your Docker Hub credentials in GitHub Secrets.
If the Flask app does not load: Ensure you are using the correct port and that the Docker container is running. needs!

From Dockerfile to Deployment: A Simple Guide for New Developers

Joseph Ibeh — Fri, 21 Feb 2025 13:05:57 +0000

Containerization has become a game-changer in modern software development, making application deployment more efficient and scalable. If you're a new developer looking to understand Docker and how to containerize your applications, this guide will walk you through the essentials—from writing your first Dockerfile to running multi-container setups using Docker Compose.

What Is Docker and Why Should You Use It?

Docker is an open-source platform that enables developers to build, deploy, and run applications inside containers. Containers package all the dependencies an application needs to run, ensuring consistency across development, testing, and production environments.

Why Use Docker?

Consistent development and production environments.
Simplifies application deployment.
Enhances scalability and resource efficiency.
Easier collaboration across teams.

Step-by-Step Guide to Writing a Dockerfile

A Dockerfile is a text file that contains a series of instructions Docker uses to build an image. Here’s a simple example to help you get started:

Example: Dockerfile for a Simple Node.js Application

Explanation of Each Command:

FROM: Specifies the base image for your application.
WORKDIR: Sets the working directory inside the container.
COPY: Copies files from your local machine into the container.
RUN: Executes commands (e.g., installing dependencies).
EXPOSE: Informs Docker which port the app will use.
CMD: Specifies the command that will run when the container starts.

Best Practices for Managing Docker Images and Containers

To ensure efficient and secure containerization, follow these best practices:

Use Small Base Images: Smaller images reduce build time and improve security.
Minimize Layers: Combine commands using && to minimize layers and optimize the image size.
Leverage .dockerignore: Exclude unnecessary files from being copied into your container.
Tag Images Properly: Use meaningful tags (e.g., v1.0, latest) for easier tracking.
Clean Up After Installation: Remove unnecessary dependencies to keep the image lightweight.

Basic Docker Commands Every Developer Should Know

Here are some essential Docker commands to help you get started:

Command	Description
`docker build -t app-name .`	Build an image from a Dockerfile
`docker images`	List all Docker images
`docker run -d -p 3000:3000 app-name`	Run a container from an image
`docker ps`	List running containers
`docker stop <container_id>`	Stop a running container
`docker rm <container_id>`	Remove a stopped container
`docker rmi <image_id>`	Remove an image
`docker logs <container_id>`	View logs from a running container

Using Docker Compose for Multi-Container Applications

When your application relies on multiple services (like a database and a web server), Docker Compose simplifies the process of running multi-container setups.

Example: docker-compose.yml for a Node.js App with MongoDB

Explanation:

version: Specifies the Docker Compose version.
services: Defines the containers to run.
web: Builds the app from your Dockerfile and maps port 3000.
mongo: Pulls a MongoDB image and maps port 27017.

Running Your Docker Compose Application:

Save the file as docker-compose.yml.
Run the following command:

docker-compose up

To stop the containers:

docker-compose down

Conclusion

Containerization with Docker is a powerful tool for modern developers, simplifying deployment and enhancing scalability. By mastering Dockerfiles, managing images and containers efficiently, and using Docker Compose for multi-container setups, you'll be well on your way to building robust applications.

Building a Scalable Data Lake for Sports Analytics with AWS

Joseph Ibeh — Mon, 17 Feb 2025 14:15:41 +0000

Sports Analytics Data Lake

This project sets up a data lake to store and process NBA sports data using AWS services such as S3, AWS Glue, and Athena. It uses the SportsData.io API to fetch NBA player data and process it for analytics purposes. There are two main scripts in the project:

setup_nba_data_lake.py: This script sets up the data lake by creating the necessary AWS resources and fetching NBA data from the SportsData.io API.
delete.py: This script deletes the resources created during the setup, including S3 buckets, Glue database, and Athena query results.

Overview of AWS Services Used

S3 Buckets: Used to store data in different formats (JSON files).
AWS Glue: A serverless integration service to catalog the data stored in S3 and load it into Athena.
Amazon Athena: Allows querying of the data stored in S3 using SQL queries.

Interpretation of the Project

This project is designed to automate the setup of a sports analytics data lake for NBA data. By using AWS services, it allows users to:

Collect NBA player data from the SportsData.io API.
Store the data in an S3 bucket.
Catalog and structure the data using AWS Glue.
Set up Athena to allow querying of the data stored in S3.

This approach provides a scalable, cost-efficient solution for analyzing large amounts of sports data and performing advanced analytics using SQL-based queries.

Steps to Set Up

Prerequisites

Before running the script, ensure you have the following:

Go to Sportsdata.io and create a free account.
At the top left, you should see "Developers"; hover over it to see "API Resources".
Click on Introduction & Testing, then select "SportsDataIO API Free Trial" and fill out the information. Ensure to select NBA for this tutorial.
You will receive an email with the subject "Launch Developer Portal". Follow the link provided.
By default, it takes you to the NFL section. On the left, click on NBA.
Scroll down until you see "Standings".
Under "Query String Parameters", the value in the drop-down box is your API key.
Copy this string as you will need to paste it into the script.

IAM Role/Permissions:

Ensure the user or role running the script has the following permissions:

S3: s3:CreateBucket, s3:PutObject, s3:DeleteBucket, s3:ListBucket
Glue: glue:CreateDatabase, glue:CreateTable, glue:DeleteDatabase, glue:DeleteTable
Athena: athena:StartQueryExecution, athena:GetQueryResults

Step 1: Open CloudShell Console

Go to aws.amazon.com and sign into your account.
In the top right, next to the search bar, click the square with a >_ inside to open the CloudShell.

Step 2: Create the `setup_nba_data_lake.py` File

In the CLI (Command Line Interface), type:

nano setup_nba_data_lake.py

Paste the following script into the file:

import boto3
import json
import time
import requests
from dotenv import load_dotenv
import os

# Load environment variables from .env file
load_dotenv()

# AWS configurations
region = "us-east-1"  # Replace with your preferred AWS region
bucket_name = "Jose-sports-analytics-data-lake"  # Change to a unique S3 bucket name
glue_database_name = "glue_nba_data_lake"
athena_output_location = f"s3://{bucket_name}/athena-results/"

# Sportsdata.io configurations (loaded from .env)
api_key = "SPORTS_DATA_API_KEY"  # Replace with your API key
nba_endpoint = "NBA_ENDPOINT"  # Get NBA endpoint from .env

# Use the hardcoded values from the script
api_key = SPORTS_DATA_API_KEY
nba_endpoint = NBA_ENDPOINT

# Create AWS clients
s3_client = boto3.client("s3", region_name=region)
glue_client = boto3.client("glue", region_name=region)
athena_client = boto3.client("athena", region_name=region)

def create_s3_bucket():
    """Create an S3 bucket for storing sports data."""
    try:
        if region == "us-east-1":
            s3_client.create_bucket(Bucket=bucket_name)
        else:
            s3_client.create_bucket(
                Bucket=bucket_name,
                CreateBucketConfiguration={"LocationConstraint": region},
            )
        print(f"S3 bucket '{bucket_name}' created successfully.")
    except Exception as e:
        print(f"Error creating S3 bucket: {e}")

def create_glue_database():
    """Create a Glue database for the data lake."""
    try:
        glue_client.create_database(
            DatabaseInput={
                "Name": glue_database_name,
                "Description": "Glue database for NBA sports analytics.",
            }
        )
        print(f"Glue database '{glue_database_name}' created successfully.")
    except Exception as e:
        print(f"Error creating Glue database: {e}")

def fetch_nba_data():
    """Fetch NBA player data from sportsdata.io."""
    try:
        headers = {"Ocp-Apim-Subscription-Key": api_key}
        response = requests.get(nba_endpoint, headers=headers)
        response.raise_for_status()  # Raise an error for bad status codes
        print("Fetched NBA data successfully.")
        return response.json()  # Return JSON response
    except Exception as e:
        print(f"Error fetching NBA data: {e}")
        return []

def convert_to_line_delimited_json(data):
    """Convert data to line-delimited JSON format."""
    print("Converting data to line-delimited JSON format...")
    return "\n".join([json.dumps(record) for record in data])

def upload_data_to_s3(data):
    """Upload NBA data to the S3 bucket."""
    try:
        # Convert data to line-delimited JSON
        line_delimited_data = convert_to_line_delimited_json(data)

        # Define S3 object key
        file_key = "raw-data/nba_player_data.jsonl"

        # Upload JSON data to S3
        s3_client.put_object(
            Bucket=bucket_name,
            Key=file_key,
            Body=line_delimited_data
        )
        print(f"Uploaded data to S3: {file_key}")
    except Exception as e:
        print(f"Error uploading data to S3: {e}")

def create_glue_table():
    """Create a Glue table for the data."""
    try:
        glue_client.create_table(
            DatabaseName=glue_database_name,
            TableInput={
                "Name": "nba_players",
                "StorageDescriptor": {
                    "Columns": [
                        {"Name": "PlayerID", "Type": "int"},
                        {"Name": "FirstName", "Type": "string"},
                        {"Name": "LastName", "Type": "string"},
                        {"Name": "Team", "Type": "string"},
                        {"Name": "Position", "Type": "string"},
                        {"Name": "Points", "Type": "int"}
                    ],
                    "Location": f"s3://{bucket_name}/raw-data/",
                    "InputFormat": "org.apache.hadoop.mapred.TextInputFormat",
                    "OutputFormat": "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat",
                    "SerdeInfo": {
                        "SerializationLibrary": "org.openx.data.jsonserde.JsonSerDe"
                    },
                },
                "TableType": "EXTERNAL_TABLE",
            },
        )
        print(f"Glue table 'nba_players' created successfully.")
    except Exception as e:
        print(f"Error creating Glue table: {e}")

def configure_athena():
    """Set up Athena output location."""
    try:
        athena_client.start_query_execution(
            QueryString="CREATE DATABASE IF NOT EXISTS nba_analytics",
            QueryExecutionContext={"Database": glue_database_name},
            ResultConfiguration={"OutputLocation": athena_output_location},
        )
        print("Athena output location configured successfully.")
    except Exception as e:
        print(f"Error configuring Athena: {e}")

# Main workflow
def main():
    print("Setting up data lake for NBA sports analytics...")
    create_s3_bucket()
    time.sleep(5)  # Ensure bucket creation propagates
    create_glue_database()
    nba_data = fetch_nba_data()
    if nba_data:  # Only proceed if data was fetched successfully
        upload_data_to_s3(nba_data)
    create_glue_table()
    configure_athena()
    print("Data lake setup complete.")

if __name__ == "__main__":
    main()

Step 3: Create the delete.py Script

import boto3
from botocore.exceptions import ClientError

# Define the names of resources to delete
BUCKET_NAME = "Jose-sports-analytics-data-lake"
GLUE_DATABASE_NAME = "glue_nba_data_lake"

def delete_athena_query_results(bucket_name):
    """Delete Athena query results stored in the specified S3 bucket."""
    s3 = boto3.client("s3")
    try:
        print(f"Deleting Athena query results in bucket: {bucket_name}")
        objects = s3.list_objects_v2(Bucket=bucket_name, Prefix="athena-results/")
        if "Contents" in objects:
            for obj in objects["Contents"]:
                s3.delete_object(Bucket=bucket_name, Key=obj["Key"])
                print(f"Deleted Athena query result: {obj['Key']}")
    except ClientError as e:
        print(f"Error deleting Athena query results in bucket {bucket_name}: {e}")

def delete_s3_bucket(bucket_name):
    """Delete a specific S3 bucket and its contents."""
    s3 = boto3.client("s3")
    try:
        print(f"Deleting bucket: {bucket_name}")
        # Delete all objects in the bucket
        objects = s3.list_objects_v2(Bucket=bucket_name)
        if "Contents" in objects:
            for obj in objects["Contents"]:
                s3.delete_object(Bucket=bucket_name, Key=obj["Key"])
                print(f"Deleted object: {obj['Key']}")
        # Delete the bucket
        s3.delete_bucket(Bucket=bucket_name)
        print(f"Deleted bucket: {bucket_name}")
    except ClientError as e:
        print(f"Error deleting bucket {bucket_name}: {e}")

def delete_glue_resources(database_name):
    """Delete Glue database and associated tables."""
    glue = boto3.client("glue")
    try:
        print(f"Deleting Glue database: {database_name}")
        # Get tables in the database
        tables = glue.get_tables(DatabaseName=database_name)["TableList"]
        for table in tables:
            table_name = table["Name"]
            print(f"Deleting Glue table: {table_name} in database {database_name}")
            glue.delete_table(DatabaseName=database_name, Name=table_name)
        # Delete the database
        glue.delete_database(Name=database_name)
        print(f"Deleted Glue database: {database_name}")
    except ClientError as e:
        print(f"Error deleting Glue resources for database {database_name}: {e}")

def main():
    print("Deleting resources created during data lake setup...")
    # Delete the S3 bucket
    delete_s3_bucket(BUCKET_NAME)
    # Delete Glue resources
    delete_glue_resources(GLUE_DATABASE_NAME)
    # Delete Athena query results
    delete_athena_query_results(BUCKET_NAME)
    print("All specified resources deleted successfully.")

if __name__ == "__main__":
    main()

ls to see see the files and directories that are present in the current directory or the directory

Running the Scripts

python3 setup_nba_data_lake.py

N.b ignore the errors, it was because I ran the script twice

Go to s3 bucket and open the Json file to get the data

To delete the resources, run:

python3 delete.py

Conclusion
This project demonstrates how to set up a data lake using AWS services and integrate data from an external API for sports analytics.

Building an NBA Game Day Notification System with AWS Lambda, SNS & EventBridge

Joseph Ibeh — Sat, 08 Feb 2025 06:46:50 +0000

🏀 NBA Game Day Notification System

This project is an AWS Lambda-powered notification system that sends NBA game updates via Amazon SNS (Simple Notification Service). It fetches game details from SportsData.io and notifies subscribed users via email. The function runs on a scheduled Amazon EventBridge rule.

Features

Fetches NBA game data from SportsData.io API
Sends real-time updates on game status, scores, and channels
Uses AWS Lambda, SNS, and EventBridge for automation
Supports multiple game statuses: Final, In Progress, and Scheduled

Prerequisites

Before deploying, ensure you have:

AWS Account with Lambda, SNS, and EventBridge permissions
SportsData.io API Key (Create a free API key here)
Email Subscription to the SNS topic

Setup Instructions

1️⃣ Create an SNS Topic

Open AWS SNS and create a new topic named gd_topic.
Subscribe an email address to this topic and confirm the email subscription.

2️⃣ Create an IAM Role for Lambda

Open IAM → Roles → Create Role.
Select AWS Service → Lambda.
Attach the AmazonSNSFullAccess policy.
Create a custom JSON policy with the following:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "sns:Publish",
      "Resource": "arn:aws:sns:us-east-1:XXXXXXXXXXXX:gd_topic"
    }
  ]
}

Replace XXXXXXXXXXXX with your AWS Account ID.

Name the role gd_lambda_role and attach it to your Lambda function.

3️⃣ Deploy the AWS Lambda Function

Navigate to AWS Lambda → Create Function → Author from Scratch.
Function Name: gd_notification
Runtime: Python 3.13
Attach gd_lambda_role to the function.
Paste the following Python script into the function editor:

import os
import json
import urllib.request
import boto3
from datetime import datetime, timedelta, timezone

# Function to format NBA game data
def format_game_data(game):
    status = game.get("Status", "Unknown")
    away_team = game.get("AwayTeam", "Unknown")
    home_team = game.get("HomeTeam", "Unknown")
    final_score = f"{game.get('AwayTeamScore', 'N/A')}-{game.get('HomeTeamScore', 'N/A')}"
    start_time = game.get("DateTime", "Unknown")
    channel = game.get("Channel", "Unknown")

    if status == "Final":
        return f"Game Status: {status}\n{away_team} vs {home_team}\nFinal Score: {final_score}\nStart Time: {start_time}\nChannel: {channel}\n"
    elif status == "InProgress":
        return f"Game Status: {status}\n{away_team} vs {home_team}\nCurrent Score: {final_score}\nChannel: {channel}\n"
    elif status == "Scheduled":
        return f"Game Status: {status}\n{away_team} vs {home_team}\nStart Time: {start_time}\nChannel: {channel}\n"
    else:
        return f"Game Status: {status}\n{away_team} vs {home_team}\nDetails are unavailable.\n"

# Lambda function handler
def lambda_handler(event, context):
    api_key = os.getenv("NBA_API_KEY")
    sns_topic_arn = os.getenv("SNS_TOPIC_ARN")
    sns_client = boto3.client("sns")

    utc_now = datetime.now(timezone.utc)
    today_date = (utc_now - timedelta(hours=6)).strftime("%Y-%m-%d")
    api_url = f"https://api.sportsdata.io/v3/nba/scores/json/GamesByDate/{today_date}?key={api_key}"

    try:
        with urllib.request.urlopen(api_url) as response:
            data = json.loads(response.read().decode())
    except Exception as e:
        print(f"Error fetching data: {e}")
        return {"statusCode": 500, "body": "Error fetching data"}

    messages = [format_game_data(game) for game in data]
    final_message = "\n---\n".join(messages) if messages else "No games available for today."

    try:
        sns_client.publish(TopicArn=sns_topic_arn, Message=final_message, Subject="NBA Game Updates")
        print("Message published to SNS successfully.")
    except Exception as e:
        print(f"Error publishing to SNS: {e}")
        return {"statusCode": 500, "body": "Error publishing to SNS"}

    return {"statusCode": 200, "body": "Data processed and sent to SNS"}

Click Deploy.

Add Environment Variables:
- NBA_API_KEY: Your SportsData.io API Key
- SNS_TOPIC_ARN: ARN of your SNS Topic (e.g., arn:aws:sns:us-east-1:XXXXXXXXXXXX:gd_topic)

Increase timeout under General Configuration to 5 seconds or more.

4️⃣ Test the Lambda Function

Click Test, create a test event, and run it.
Check your email inbox for an NBA game update notification!

5️⃣ Automate with AWS EventBridge

Open Amazon EventBridge → Create Rule
Rule Name: gd_rule
Schedule Pattern:

  0 9-23/2,0-2/2 * * ? *

(Runs every 2 hours from 9 AM - 11 PM UTC and 12 AM - 2 AM UTC)

Target: AWS Lambda → NBA-GameDay-Notifier
Click Create Rule.

Email Notification

Summary

AWS SNS sends game notifications to subscribed emails.
AWS Lambda fetches NBA data and publishes updates.
Amazon EventBridge schedules function execution.

Building a Scalable Real-Time Weather Dashboard with Python, OpenWeather API, and AWS S3

Joseph Ibeh — Sat, 18 Jan 2025 11:41:39 +0000

Weather Dashboard Project

The Weather Dashboard is a Python-based project that pulls weather data from the OpenWeather API and uploads it to an AWS S3 bucket. It is designed to provide a simple interface for displaying weather information for different cities and saving the results to the cloud.

This project leverages AWS S3 to store weather data, making it highly scalable, and uses Python to interact with the OpenWeather API.

Prerequisites
Project Overview
Core Features
Tools Used
Project Setup
Environment Setup
Running the Project

Prerequisites

Before starting this project, ensure you have the following:

Python 3.x: Install Python from the official website.
AWS Account: Create an AWS account to access AWS S3.
OpenWeather API Key: Obtain an API key from the OpenWeather website.
AWS CLI Installed: Download and install the AWS CLI tool from the official website.
Basic Python Knowledge: Familiarity with Python scripting, working with APIs, and environment variables.
Text Editor or IDE: Use a development environment like VS Code, PyCharm, or any text editor of your choice.
Git Installed: Ensure Git is installed for version control. Download it from the Git website.

Project Overview

This weather dashboard project fetches weather information for a given location using the OpenWeather API. The data is then uploaded to an AWS S3 bucket where it can be accessed remotely. The system is designed to be flexible, allowing users to specify different cities and view weather updates in real-time.

Core Features

Fetches weather data from OpenWeather API.
Uploads weather data to an AWS S3 bucket.
Allows you to configure API keys and AWS credentials securely via environment variables.

Tools Used

This project utilizes several tools and libraries to build the dashboard:

Python 3.x: The main programming language for the project.
boto3: AWS SDK for Python used to interact with AWS S3.
python-dotenv: A library to read environment variables from a .env file to keep sensitive data safe.
requests: A simple HTTP library for making API requests to OpenWeather.
AWS CLI: Command-line tool to interact with AWS services, such as configuring access keys and managing S3 buckets.

Project Setup

Follow these steps to set up the project on your local machine.

1. Create Project Structure

weather-dashboard/
├── src/
│ ├── init.py
│ └── weather_dashboard.py
├── .env
├── tests/
├── data/
├── .gitignore
└── README.md

Run the following commands to create the necessary directories and files:

mkdir weather_dashboard_demo
cd weather_dashboard_demo
mkdir src tests data

2. Create Files

Create the required Python files and configuration files:

touch src/__init__.py src/weather_dashboard.py
touch requirements.txt README.md .env

3. Initialize Git Repository

Initialize a new Git repository and set up the main branch:

git init
git branch -M main

4. Configure .gitignore

Create a .gitignore file to ignore unnecessary files:

echo ".env" >> .gitignore
echo "__pycache__/" >> .gitignore
echo "*.zip" >> .gitignore

5. Add Dependencies

Add the required dependencies to requirements.txt:

echo "boto3==1.26.137" >> requirements.txt
echo "python-dotenv==1.0.0" >> requirements.txt
echo "requests==2.28.2" >> requirements.txt

6. Install Dependencies

Run the following command to install the dependencies from requirements.txt:

pip install -r requirements.txt

Environment Setup

1. AWS CLI Configuration

Before interacting with AWS services like S3, you need to configure your AWS CLI with your access keys. This allows the Python script to communicate with AWS securely.
To configure AWS CLI, run the following command:

Run:

aws configure

This will prompt you to enter your AWS credentials. You'll need your AWS Access Key ID and AWS Secret Access Key, which you can get from your AWS Management Console under IAM > Users > Your User > Security Credentials.

You will also be prompted for the default region and default output format:

AWS Access Key ID: Your unique key that identifies your AWS account.
AWS Secret Access Key: A secret key that corresponds to your AWS Access Key ID.
Default region name: This is typically something like us-east-1, us-west-2, etc. You can check the AWS S3 bucket region to match this.
Default output format: Choose json for the output format, or leave it as the default (None).

Example:

$ aws configure
AWS Access Key ID [None]: your-access-key-id
AWS Secret Access Key [None]: your-secret-access-key
Default region name [None]: us-east-1
Default output format [None]: json

Check if AWS CLI is installed correctly by running:

aws --version

2. Configure .env

Create a .env file to store sensitive information like your API keys and AWS bucket name. Add the following entries:

echo "OPENWEATHER_API_KEY=your key here" >> .env
echo "AWS_BUCKET_NAME=weather-dashboard-joseph" >> .env

Make sure to replace your key here with your actual OpenWeather API key and weather-dashboard-joseph with your S3 bucket name.

Running the Project

Here is the script that fetches weather data and uploads it to AWS S3:

import os
import json
import boto3
import requests
from datetime import datetime
from dotenv import load_dotenv

# Load environment variables
load_dotenv()

class WeatherDashboard:
    def __init__(self):
        self.api_key = os.getenv('OPENWEATHER_API_KEY')
        self.bucket_name = os.getenv('AWS_BUCKET_NAME')
        self.s3_client = boto3.client('s3')

    def create_bucket_if_not_exists(self):
        """Create S3 bucket if it doesn't exist"""
        try:
            self.s3_client.head_bucket(Bucket=self.bucket_name)
            print(f"Bucket {self.bucket_name} exists")
        except:
            print(f"Creating bucket {self.bucket_name}")
        try:
            # Simpler creation for us-east-1
            self.s3_client.create_bucket(Bucket=self.bucket_name)
            print(f"Successfully created bucket {self.bucket_name}")
        except Exception as e:
            print(f"Error creating bucket: {e}")

    def fetch_weather(self, city):
        """Fetch weather data from OpenWeather API"""
        base_url = "http://api.openweathermap.org/data/2.5/weather"
        params = {
            "q": city,
            "appid": self.api_key,
            "units": "imperial"
        }

        try:
            response = requests.get(base_url, params=params)
            response.raise_for_status()
            return response.json()
        except requests.exceptions.RequestException as e:
            print(f"Error fetching weather data: {e}")
            return None

    def save_to_s3(self, weather_data, city):
        """Save weather data to S3 bucket"""
        if not weather_data:
            return False

        timestamp = datetime.now().strftime('%Y%m%d-%H%M%S')
        file_name = f"weather-data/{city}-{timestamp}.json"

        try:
            weather_data['timestamp'] = timestamp
            self.s3_client.put_object(
                Bucket=self.bucket_name,
                Key=file_name,
                Body=json.dumps(weather_data),
                ContentType='application/json'
            )
            print(f"Successfully saved data for {city} to S3")
            return True
        except Exception as e:
            print(f"Error saving to S3: {e}")
            return False

def main():
    dashboard = WeatherDashboard()

    # Create bucket if needed
    dashboard.create_bucket_if_not_exists()

    cities = ["Philadelphia", "Seattle", "New York"]

    for city in cities:
        print(f"\nFetching weather for {city}...")
        weather_data = dashboard.fetch_weather(city)
        if weather_data:
            temp = weather_data['main']['temp']
            feels_like = weather_data['main']['feels_like']
            humidity = weather_data['main']['humidity']
            description = weather_data['weather'][0]['description']

            print(f"Temperature: {temp}°F")
            print(f"Feels like: {feels_like}°F")
            print(f"Humidity: {humidity}%")
            print(f"Conditions: {description}")

            # Save to S3
            success = dashboard.save_to_s3(weather_data, city)
            if success:
                print(f"Weather data for {city} saved to S3!")
        else:
            print(f"Failed to fetch weather data for {city}")

if __name__ == "__main__":
    main()

1. Run the Script

Once the setup is complete, you can run the weather_dashboard.py script to fetch weather data and upload it to the AWS S3 bucket:

python src/weather_dashboard.py

This will pull the latest weather data from the OpenWeather API and upload it to your AWS S3 bucket.

2. Verify Your S3 Bucket

Log in to your AWS account and navigate to your S3 bucket to verify that the weather data has been successfully uploaded. If you don't want to accumulate unnecessary costs, be sure to empty and delete the contents of your bucket after verification.

Building a Scalable Web Server on AWS: A Hands-On Guide to Linux, Apache, and SSL Configuration

Joseph Ibeh — Wed, 08 Jan 2025 12:08:57 +0000

Introduction

Building a web server from scratch is an essential skill for any cloud or DevOps engineer. This project walks you through provisioning a Linux server, installing Apache, deploying a custom HTML landing page, and securing the server with SSL encryption. Whether you're a beginner or looking to solidify your skills, this guide provides practical steps to help you deploy a scalable and secure web server.

Objective

The main goals of this project are:

Provision a Linux server on AWS.
Install and configure Apache as a web server.
Create and deploy a custom HTML landing page.
Configure networking to allow HTTP (port 80) and optionally HTTPS (port 443) traffic.
Secure the server with SSL encryption using Certbot.

Tools and Technologies Used

Cloud Platform: AWS (Amazon Web Services)
Linux Distribution: Ubuntu
Web Server: Apache
SSH: For server connection and management
HTML: To design the landing page
Certbot: To manage SSL certificates for HTTPS

Steps to Build the Project

1. Provisioning the Server on AWS

Step 1: Sign In to AWS

Go to the AWS Management Console.

Step 2: Launch a New Instance

Navigate to EC2 and click "Launch Instance."
Choose an Ubuntu Linux AMI (e.g., Ubuntu 22.04 LTS).
Configure instance settings:
- Assign a public IP for accessibility.
- Choose an instance type (e.g., t2.micro for free tier users).

Step 3: Configure Key Pair

Create a new key pair and download the private key file (.pem).

Step 4: Configure Security Group

Allow the following inbound rules:
- HTTP (port 80)
- HTTPS (port 443)
- SSH (port 22)

Step 5: Launch the Instance

Click "Launch Instance" and wait for it to start running.

2. Connecting to the Server

Open a terminal or Git Bash.
Use the private key to SSH into the server:

   ssh -i path_to_private_key.pem ubuntu@public_ip_address

3. Setting Up the Web Server

-Switch to root user sudo -i

Step 1: Update the System

sudo apt update && sudo apt upgrade -y

Step 2: Install Apache

sudo apt install apache2 -y  
sudo systemctl start apache2  
sudo systemctl enable apache2

Step 3: Verify Apache Installation

Open a browser and navigate to the server's public IP address. You should see the Apache default page.

4. Deploying a Custom HTML Landing Page

Step 1: Create the HTML File

Open the Apache default directory:

  sudo vim /var/www/html/index.html

Add your custom HTML content:

Step 2: Save and Exit

Step 3: Test the Landing Page

Open a browser and visit the server’s public IP address.

5. Configuring SSL for HTTPS

Prerequisites:

Purchase a domain name (e.g., yourname.com).
Update your domain's DNS settings to point to your server's public IP address.

Step 1: Install Certbot

sudo apt install certbot python3-certbot-apache -y

Step 2: Obtain an SSL Certificate

sudo certbot --apache -d yourname.com -d www.yourname.com

Step 3: Verify HTTPS

Open a browser and navigate to https://yourname.com.

Conclusion

This project highlights the essential steps to provision a server, deploy a web server, and secure it with HTTPS. By following this guide, you’ll gain hands-on experience with server management, web hosting, and cloud technologies.

Feel free to customize this project and share your version with the tech community. Let’s keep building and learning together!

Cloud Computing Simplified: Models, Services, and the Future of Technology

Joseph Ibeh — Mon, 09 Dec 2024 12:33:55 +0000

Cloud computing has changed how we store, access, and manage data. It offers flexible, affordable, and powerful solutions that help businesses and individuals focus on what matters most. As more organizations adopt cloud technologies, understanding the basics is becoming essential.

In this guide, I’ll explain cloud computing, break down its key models, introduce its services, and highlight how it’s shaping industries and innovation.

Table of Contents

Introduction
Why Use Cloud Computing?
Cloud Deployment Models
- On-Premise Computing
- Public Cloud
- Private Cloud
- Hybrid Cloud
- Community Cloud
- Multi-Cloud
Cloud Services
- IaaS
- PaaS
- SaaS
- DaaS
- STaaS
- FaaS
Conclusion

Introduction

Cloud computing lets you access computing power over the internet instead of relying on expensive, on-site hardware. Whether you’re a business owner or a tech enthusiast, the cloud helps reduce costs, increase flexibility, and speed up innovation.

Let’s explore how it works, step by step.

Why Use Cloud Computing?

Here are five reasons why businesses are embracing cloud computing:

Cost Savings: You only pay for what you use—no need to buy expensive hardware upfront.
Scalability: Easily adjust resources as your needs grow or shrink.
Accessibility: Work from anywhere with an internet connection.
Security: Top providers offer advanced protections against cyber threats.
Speed: Quickly develop, test, and deploy projects to stay ahead.

Cloud computing simplifies IT so you can focus on building ideas instead of managing hardware.

Cloud Deployment Models

Cloud computing can be set up in different ways, depending on your needs:

1. On-Premise Computing
This is the traditional approach where everything is managed in-house.

Pros: Full control over data; suitable for strict security needs.
Cons: Expensive and hard to scale.

2. Public Cloud

Public clouds are shared by multiple users and managed by providers like AWS or Google Cloud.

Pros: Cost-effective and scalable.
Cons: Less customization; potential security concerns.

3. Private Cloud

A private cloud is dedicated to one organization, offering more control.

Pros: Highly secure and customizable.
Cons: Expensive and complex to maintain.

4. Hybrid Cloud

Mixing private and public clouds for flexibility.

Pros: Combines local security with public cloud scalability.

5. Community Cloud
A community cloud is a shared cloud infrastructure tailored for a specific group of users or organizations with shared concerns, such as compliance, security, or mission-critical goals.

Pros Costs are shared among participating organizations, enabling secure collaboration within the community, with access limited to members, making it more secure than public clouds.
Cons: Managing shared resources requires coordination among all parties and community clouds offer less scalability compared to public cloud solutions. Use Cases include healthcare with hospitals sharing patient data while ensuring HIPAA compliance, government agencies collaborating on secure joint initiatives, and education with universities sharing research infrastructure and tools.

6. Multi-Cloud

Using services from multiple cloud providers.

Pros: Using services from multiple cloud providers avoids reliance on a single provider while offering increased reliability through fault tolerance, improved disaster recovery, and higher system availability.

Cons: Managing multiple cloud providers can introduce complexity, lead to higher operational costs, and increase security risks due to varying security protocols across providers.

Cloud Services

Cloud computing isn’t just about storing data. Here are its main types of services:

1. Infrastructure as a Service (IaaS)

Think of IaaS as renting virtual servers and storage online.

Examples: Amazon EC2, Azure Virtual Machines.
Uses: Hosting websites, running applications.

2. Platform as a Service (PaaS)

PaaS provides tools to build and launch apps without managing the hardware.

Examples: Heroku, Google App Engine.
Uses: Streamlining app development.

3. Software as a Service (SaaS)

SaaS delivers ready-to-use apps over the internet, often subscription-based.

Examples: Google Workspace, Salesforce.
Uses: Productivity tools, CRM software.

4. Desktop as a Service (DaaS)

DaaS provides virtual desktops that can be accessed from any device.

Examples: Amazon WorkSpaces, Citrix.
Uses: Remote work setups.

5. Storage as a Service (STaaS)

STaaS offers flexible cloud storage for businesses and individuals.

Examples: Amazon S3, Google Cloud Storage.
Uses: Backups, disaster recovery.

6. Function as a Service (FaaS)

FaaS is about running code without managing servers, also known as serverless computing.

Examples: AWS Lambda, Azure Functions.
Uses: Event-driven apps like chatbots.

Conclusion

Cloud computing is transforming industries by making technology accessible, affordable, and efficient. From IaaS to SaaS, these services empower businesses to innovate faster and smarter.

The future of computing is in the cloud. Whether you’re just starting or looking to dive deeper, there’s no better time to explore its potential.

The Language of the Internet: A Simple Guide to Communication Protocols

Joseph Ibeh — Wed, 04 Dec 2024 16:32:19 +0000

Table of Contents

Introduction
What Are Communication Protocols?
- 2.1 Why Are Protocols Important?
The OSI Model
- 3.1 Layers of the OSI Model
- 3.2 Why the OSI Model Matters
Common Types of Protocols
- 4.1 Common Everyday Protocols
Conclusion

1. Introduction

Imagine you're mailing a birthday gift to a friend. For the gift to reach its destination, you’ll need the correct address, proper packaging, and a delivery service. In networking, communication protocols act like these rules—they guide how data is packaged, sent, and delivered so devices can communicate.

When you stream a video on YouTube, send an email, or browse Instagram, these rules ensure that everything works smoothly. This article will explain what communication protocols are, why they matter, and how they work in networking.

2. What Are Communication Protocols?

Communication protocols are like a universal language for devices, allowing them to understand and exchange data. These rules control how data is broken into smaller parts, sent, and reassembled at its destination.

2.1 Why Are Protocols Important?

Interoperability:

Example: Imagine trying to charge your phone with a charger from another country. Without universal USB standards, it wouldn’t work. Similarly, protocols ensure devices from different manufacturers can connect and share information.

Efficiency:

Example: When you stream a Netflix movie, protocols manage the flow of data so you don’t experience buffering.

Security:

Example: When you shop online, HTTPS ensures your credit card details are encrypted and protected.

3. The OSI Model

The OSI model is like a team of delivery workers who handle a package in steps, from pickup to delivery. It breaks networking into seven layers, with each layer performing a specific role to ensure successful communication.

3.1 Layers of the OSI Model

Physical Layer:
- What it does: Transmits raw data as electrical signals.
- Example: The cables connecting your Wi-Fi router to the internet.
Data Link Layer:
- What it does: Ensures data isn’t lost or damaged.
- Example: When you print a document over Wi-Fi, this layer ensures the connection between your computer and printer is reliable.
Network Layer:
- What it does: Finds the best route for data to travel.
- Example: Like GPS guiding a car, this layer helps emails travel through different networks to reach the correct inbox.
Transport Layer:
- What it does: Ensures data is delivered without errors.
- Example: When you download a file, this layer checks all parts of the file are received in order.
Session Layer:
- What it does: Manages conversations between devices.
- Example: Keeps your video call stable and ensures it doesn’t drop midway.
Presentation Layer:
- What it does: Converts data into a readable format.
- Example: Translates a YouTube video’s file format so it can play on your browser.
Application Layer:
- What it does: Interacts with you directly.
- Example: Loading Instagram photos or sending WhatsApp messages.

3.2 Why the OSI Model Matters

Easy Troubleshooting: If your Zoom call fails, the OSI model helps engineers check whether the issue is in the network connection, session management, or video decoding.
Integration: New tech, like 5G, can fit into specific OSI layers without affecting the rest.

4. Common Types of Protocols

Protocols exist for every layer of the OSI model. Here are some examples you’ve probably encountered:

4.1 Everyday Protocols You Use

HTTP/HTTPS (Application Layer):
- Example: When you type a URL in your browser, HTTP fetches the webpage, while HTTPS encrypts it for safety.
FTP (File Transfer Protocol):
- Example: Uploading photos to Google Drive or downloading documents from a website.
TCP (Transport Layer):
- Example: Ensures your WhatsApp messages arrive in order, even if the network is unstable.
UDP (User Datagram Protocol):
- Example: Used in live gaming or video streaming, where speed matters more than reliability.
IP (Network Layer):
- Example: Works like a mailing address, ensuring your data gets to the right recipient.

5. Conclusion

Communication protocols and the OSI model are like the invisible engines of the internet. They ensure smooth, efficient, and secure communication between devices. From browsing the web to sending messages, these systems work tirelessly to keep everything running.
So next time you stream a video or send an email, you’ll know the magic that makes it happen!

Mastering IP Addresses: A Comprehensive Guide to Understanding, Calculating, and Classifying Network Addresses

Joseph Ibeh — Tue, 26 Nov 2024 06:38:36 +0000

Table of Contents

What is an IP Address?
Types of IP Addresses
- Public IP Address
- Private IP Address
- Static IP Address
- Dynamic IP Address
Classes of IP Addresses
- Class A
- Class B
- Class C
- Class D
- Class E
Netmask and Subnetting
- What is a Netmask?
- Subnetting Explained
How to Convert an Octet to Binary
How to Calculate IP Address and Netmask
- Example Calculation
CIDR Notation
Conclusion.

Introduction
In the world of computer networking, an IP address (Internet Protocol address) is essential for identifying and communicating with devices over a network. Every device that connects to the internet or any other network must have a unique IP address. These addresses are crucial for routing data packets between devices across different networks, ensuring that information reaches its intended destination.
In this article, we'll explore the following:
What IP addresses are
The different types of IP addresses
The concept of netmasks
How to calculate them

1. What is an IP Address?
An IP address is a unique identifier assigned to a device or network interface on a network. It helps route traffic across the internet or local networks, allowing devices to locate and communicate with each other.
There are two main versions of IP addresses:
IPv4 (Internet Protocol version 4): The most widely used format, consisting of four 8-bit octets (32 bits), written as four decimal numbers separated by periods (e.g., 192.168.1.1).
IPv6 (Internet Protocol version 6): Introduced to overcome the limitations of IPv4 (mainly address exhaustion), it uses 128-bit addresses, represented as eight 16-bit groups separated by colons (e.g., 2001:0db8:85a3:0000:0000:8a2e:0370:7334).

2. Types of IP Addresses

IP addresses are categorized based on their usage and purpose. The main types include:

Public IP Address
A public IP address is routable on the internet. These addresses are assigned to devices or networks by an Internet Service Provider (ISP) and are used for identifying devices on the internet, enabling communication with others over the web. Examples of public IP addresses include:8.8.8.8 (Google's public DNS server) 208.67.222.222 (OpenDNS server)
Private IP Address
A private IP address is used within local networks, such as in homes, offices, or enterprise environments. These addresses are not routable over the internet but are used for internal communication. The private IP address ranges for IPv4 are:
10.0.0.0 - 10.255.255.255
172.16.0.0 - 172.31.255.255
192.168.0.0 - 192.168.255.255
Static IP Address: A static IP address is fixed and manually configured, such as a server with a permanent IP like 198.51.100.10.
Dynamic IP Address: A dynamic IP address is automatically assigned by a Dynamic Host Configuration Protocol (DHCP) server, such as a home computer with an IP like 192.168.1.5 that changes periodically.

3. Classes of IP Addresses

IP addresses are categorized into five classes, each serving different purposes based on the size of the network and the range of addresses required. The classes are as follows:

Class A (0.0.0.0 to 127.255.255.255)
- Range: 1.0.0.0 - 127.255.255.255
- Purpose: Designed for large networks, typically used by very large organizations, government bodies, and ISPs.
- Network Size: Up to 16 million hosts per network.
- Default Subnet Mask: 255.0.0.0
Class B (128.0.0.0 to 191.255.255.255)
- Range: 128.0.0.0 - 191.255.255.255
- Purpose: Suitable for medium-sized networks, such as universities or large businesses.
- Network Size: Up to 65,000 hosts per network.
- Default Subnet Mask: 255.255.0.0
Class C (192.0.0.0 to 223.255.255.255)
- Range: 192.0.0.0 - 223.255.255.255
- Purpose: Designed for smaller networks, typically used by small businesses and home networks.
- Network Size: Up to 254 hosts per network.
- Default Subnet Mask: 255.255.255.0
Class D (224.0.0.0 to 239.255.255.255)
- Range: 224.0.0.0 - 239.255.255.255
- Purpose: Reserved for multicast addresses. These are used for one-to-many communication, like streaming media.
- Network Size: N/A (for multicast communication)
Class E (240.0.0.0 to 255.255.255.255)
- Range: 240.0.0.0 - 255.255.255.255
- Purpose: Reserved for experimental use and future applications. These addresses are not used in general networking.
- Network Size: N/A

4. Netmask and Subnetting

What is a Netmask?
A netmask is a 32-bit number that defines the network portion of an IP address. It works by masking (or blocking out) the host part of the IP address, leaving only the network portion visible. A netmask helps devices determine whether a destination IP address is within the same local network or needs to be routed to another network.
Example:
IP Address: 192.168.1.1
Netmask: 255.255.255.0
In this example, the first 24 bits (192.168.1) represent the network, and the remaining 8 bits (.1) represent the host.

Subnetting
Subnetting is the process of dividing a larger network into smaller subnetworks (subnets). By applying different netmasks, subnetting allows network administrators to efficiently allocate IP addresses and improve network performance. Subnetting is especially useful in large networks where it’s necessary to organize devices into logical groups for efficient routing.

5. How to Convert an Octet to Binary

Understand Binary Numbers
Binary numbers use only 1s and 0s, and each position represents a power of 2, from right to left:
128 | 64 | 32 | 16 | 8 | 4 | 2 | 1
Convert the Decimal Number to Binary
Start with the decimal number and determine how many times each power of 2 fits into it. Write 1 if the power of 2 fits, and 0 if it doesn’t.

Example: Convert 192 to Binary

For 192: 128 fits, so write 1. Subtract 128 from 192 → 192 - 128 = 64 remains.

64 fits, so write 1. Subtract 64 from 64 → 64 - 64 = 0 remains.

Thus, 192 in binary is 11000000.

6. How to Calculate IP Address and Netmask
To calculate the network address from an IP address and netmask, follow these steps:
Convert both the IP address and netmask to binary.
Perform a binary AND operation between the IP address and netmask.
Convert the resulting binary network address back to decimal.

Example Calculation: Converting IP Address and Netmask

We will work with the IP address 192.168.1.10 and the netmask 255.255.255.0.

Step 1: Understand Binary Number.
In binary, each digit represents a power of 2. The place values (from right to left) are:
128 | 64 | 32 | 16 | 8 | 4 | 2 | 1
Step 2: Convert the Decimal Number to Binary
Convert 192 to Binary:
The decimal number is 192. Let's figure out which powers of 2 fit into 192:
128 fits into 192, so we write 1.
Subtract 128 from 192 → 192 - 128 = 64 remains.
64 fits into 64, so we write 1.
Subtract 64 from 64 → 64 - 64 = 0 remains.
So, 192 in binary is: 11000000
Convert 168 to Binary:
The decimal number is 168. Let's figure out which powers of 2 fit into 168:
128 fits into 168, so we write 1.
Subtract 128 from 168 → 168 - 128 = 40 remains.
32 fits into 40, so we write 1.
Subtract 32 from 40 → 40 - 32 = 8 remains.
8 fits into 8, so we write 1.
Subtract 8 from 8 → 8 - 8 = 0 remains.
So, 168 in binary is: 10101000
Convert 1 to Binary:
The decimal number is 1. Let's figure out which powers of 2 fit into 1:
1 fits into 1, so we write 1.
No remainder, so the binary representation is just 00000001 (we need 8 bits for each octet).
So, 1 in binary is: 00000001
Convert 10 to Binary:
The decimal number is 10. Let's figure out which powers of 2 fit into 10:
8 fits into 10, so we write 1.
Subtract 8 from 10 → 10 - 8 = 2 remains.
2 fits into 2, so we write 1.
Subtract 2 from 2 → 2 - 2 = 0 remains.
So, 10 in binary is: 00001010
Step 3: Convert the Netmask to Binary
Convert 255 to Binary:
The decimal number is 255. Let's figure out which powers of 2 fit into 255:
128 fits into 255, so we write 1.
Subtract 128 from 255 → 255 - 128 = 127 remains.
64 fits into 127, so we write 1.
Subtract 64 from 127 → 127 - 64 = 63 remains.
32 fits into 63, so we write 1.
Subtract 32 from 63 → 63 - 32 = 31 remains.
16 fits into 31, so we write 1.
Subtract 16 from 31 → 31 - 16 = 15 remains.
8 fits into 15, so we write 1.
Subtract 8 from 15 → 15 - 8 = 7 remains.
4 fits into 7, so we write 1.
Subtract 4 from 7 → 7 - 4 = 3 remains.
2 fits into 3, so we write 1.
Subtract 2 from 3 → 3 - 2 = 1 remains.
1 fits into 1, so we write 1.
Subtract 1 from 1 → 1 - 1 = 0 remains.
So, 255 in binary is: 11111111
Convert 0 to Binary:
The decimal number is 0. Let's figure out which powers of 2 fit into 0:
0 does not fit into any powers of 2, so we write 0 for all bits.
So, 0 in binary is: 00000000
Step 4: Perform the AND Operation
Now we perform the bitwise AND operation (using the IP address and the netmask):
IP Address in Binary:
11000000.10101000.00000001.00001010
Netmask in Binary:
11111111.11111111.11111111.00000000
Perform the AND operation on each bit:
First octet (192) AND 255: 11000000 (IP) AND 11111111 (Netmask) = 11000000 Result: 192
Second octet (168) AND 255: 10101000 (IP) AND 11111111 (Netmask) = 10101000 Result: 168
Third octet (1) AND 255: 00000001 (IP) AND 11111111 (Netmask) = 00000001 Result: 1
Fourth octet (10) AND 0: 00001010 (IP) AND 00000000 (Netmask) = 00000000 Result: 0
Final Network Address:
The result of the AND operation is the network address, which is:
192.168.1.0

Step 5: Calculate the Broadcast Address
To calculate the broadcast address, we take the network address and set all the host bits (the bits corresponding to 0s in the netmask) to 1s.
Network Address in Binary: 11000000.10101000.00000001.00000000
Change the host bits (the last octet, where the netmask is 0) to 1s: 11000000.10101000.00000001.11111111
Now convert the broadcast address back to decimal:
11000000 → 192
10101000 → 168
00000001 → 1
11111111 → 255
Thus, the broadcast address is:
192.168.1.255

7. CIDR Notation
CIDR (Classless Inter-Domain Routing) notation is a shorthand way to specify IP addresses and netmasks. Instead of writing the netmask as four octets, CIDR uses a slash (/) followed by the number of bits used for the network part.
Example:

192.168.1.0/24 means an IP address with a 24-bit network portion (i.e., the netmask is 255.255.255.0)

8. Conclusion
Understanding how IP addresses work, how to calculate network addresses, and how to convert between decimal and binary is crucial for anyone working with networking. Whether you're setting up a local network or managing larger enterprise systems, these fundamental concepts will help you better understand and manage IP addresses and subnets.

Understanding Networking and Internet Infrastructure: A Global Overview with Insights from Nigeria

Joseph Ibeh — Sat, 23 Nov 2024 12:54:25 +0000

In today’s digital world, networking is the backbone of our online interactions, enabling everything from browsing websites to sending emails. While the mechanics of the internet are similar across the globe, the infrastructure that supports it varies by region. In this article, we’ll explore the basics of networking, how the internet works, and examine key components of internet infrastructure, with a special focus on emerging markets like Nigeria.

How Does the Internet Work?
The internet is a vast network of interconnected computers and servers that communicate via the Internet Protocol (IP). This system allows data to travel from one device to another across various networks, regardless of location. Here’s a simplified breakdown of how it works:

1.Data Request: When you access a website, your device sends a request to the server hosting that website.

2.Routing: The request is routed through multiple routers, cables, and switches across several networks until it reaches the server.

3.Response: The server processes the request and sends the requested data (e.g., a web page) back.

4.Display: Your browser receives and interprets the data to display the content.
This seamless communication relies on critical infrastructure, such as cables, data centers, and regulatory bodies, which ensure efficient global connectivity.

Cables: Connecting the World and Emerging Markets Like Nigeria
Submarine cables are the lifeblood of global internet connectivity. These undersea cables transfer massive amounts of data between continents, forming the foundation for all internet traffic.

SEA-ME-WE 3: Connecting Asia, the Middle East, and Europe, this cable is a significant global artery for internet traffic.

TEA-2: Operating between the Americas and Europe, this cable helps serve a large part of transatlantic data needs.
In Nigeria, submarine cables like MainOne and Glo-1are improving connectivity, linking West Africa to Europe and other parts of the world. These cables enable faster internet speeds and improve access to global services, fueling Nigeria’s digital economy.

The Role of Internet Governance: IANA and Regional Authorities

The Internet Assigned Numbers Authority (IANA) plays a crucial role in global internet governance. IANA manages:
IP Addresses: Unique identifiers assigned to devices on a network.
Domain Names: Human-readable names that direct to specific IP addresses.

IANA works alongside regional bodies like the Africa Network Information Centre (AFRINIC), which manages IP address allocation across Africa, including Nigeria. Similarly, organizations such as ICANN and RIPE NCC ensure global coordination and allocation of internet resources.

Internet Exchange Points (IXPs): Improving Local Connectivity
IXPs are key locations where different internet networks exchange traffic. By allowing local traffic to stay within the region, IXPs reduce latency and cut down on the costs associated with sending data across borders.
Globally recognized IXPs like LINX (London Internet Exchange) and DE-CIX (Frankfurt) handle enormous volumes of traffic, enhancing the efficiency of internet services. In Nigeria, the Internet Exchange Point of Nigeria (IXPN) plays a critical role, interconnecting local Internet Service Providers (ISPs) and data centers. This reduces international traffic and improves the internet experience for Nigerian users.

Data Centers: Powering the Digital World, from Global Giants to Local Players
Data centers are the heart of the internet, housing the servers and systems that power websites, cloud services, and apps. Globally, companies like Amazon Web Services (AWS), Google, and Microsoftoperate massive data centers across multiple continents.
However, as the demand for local content and services grows, regions like West Africa are developing their own data centers to support businesses. In Nigeria, data centers such as Rack Centre, MainOne Data Center, and MDXi are playing a key role in Nigeria’s digital transformation. These facilities support cloud services, e-commerce platforms, fintech, and more, driving the country’s digital economy forward.

Why Networking Matters: A Global Perspective
Networking is essential for the seamless functioning of the internet. It ensures efficient communication between devices and allows us to access global content instantly. Key takeaways include:
-Global Connectivity: Submarine cables and IXPs ensure smooth international data exchange, benefiting users worldwide.
-Local Optimization: IXPs and regional data centers improve internet speeds and reliability for local markets, such as Nigeria.
-Support for Digital Services: Data centers are the backbone of cloud services and digital platforms, supporting everything from e-commerce to online banking.

Conclusion

The internet is a global network that connects us all, but the infrastructure supporting it is diverse and region-specific. While developed markets have advanced connectivity systems, emerging markets like Nigeria are rapidly catching up, with submarine cables, local IXPs, and data centers driving digital growth.

As cloud engineers, developers, and tech enthusiasts, it's crucial to understand the foundational elements that make the internet work. From the cables that connect continents to the data centers that power local services, every part of this ecosystem contributes to seamless digital experience.

Ansible Demystified: Your Ultimate Guide to Simplifying IT Automation and Boosting DevOps Efficiency

Joseph Ibeh — Wed, 20 Nov 2024 12:02:25 +0000

As a cloud engineer or someone venturing into DevOps, you've probably heard of Ansible. But what exactly is it, and why should you care? Let’s break it down step by step, so even if you’re a beginner, by the end of this article, you'll have a good grasp of what Ansible is and how it works.

What is Ansible?

Ansible is an open-source IT automation tool. It simplifies tasks like configuring servers, deploying applications, and managing networks. Think of Ansible as your personal assistant in IT, helping you automate repetitive tasks so you can focus on what truly matters.

Here’s why Ansible is awesome:

Agentless: No software needs to be installed on the managed systems; Ansible uses SSH.
Simple YAML Syntax: Ansible uses YAML (Yet Another Markup Language) for its configuration files, making it easy to learn and write.
Idempotent: Running the same Ansible command multiple times won’t change anything if the desired state is already achieved.

Why Use Ansible

Imagine you need to configure 10 servers with identical settings. Doing this manually can be error-prone and time-consuming. With Ansible, you can define the configuration once and apply it to all servers in a matter of seconds.

Key Benefits:

Consistency: Ensures all servers are configured the same way.
Scalability: Manage hundreds or thousands of servers effortlessly.
Efficiency: Automates mundane tasks, saving time and reducing errors.

How Does Ansible Work?

Ansible works by connecting to your managed nodes (servers, devices, etc.) via SSH, applying the instructions you define in Playbooks. Here’s the basic flow:

1. Control Node: The machine where Ansible is installed. It sends instructions.
2. Managed Nodes: Servers or devices Ansible configures.
3. Playbooks: Files written in YAML that describe what Ansible should do.
4. Modules: Pre-built scripts that perform specific tasks like installing software, creating users, or restarting services.

Getting Started with Ansible

1. Install Ansible

To get started, install Ansible on your control node (e.g., your laptop or a dedicated server). Here's how to install it on Linux:

sudo apt update
sudo apt install ansible

For Mac users, use Homebrew:

brew install ansible

Windows users can install Ansible via WSL or use a virtual machine.

Do You Need Python?

Yes, you need to have Python installed to use Ansible. Ansible relies on Python to execute tasks, both on the control node (where you run Ansible) and the managed nodes (target servers or devices). Ansible supports Python 2.7 or Python 3.5+.

You can check if Python is installed by running:

python --version

python3 --version

To install Python, visit the Python website or use a package manager like apt for Linux or brew for Mac. Once Python is installed, you can then install Ansible.

2. Set Up Your SSH Key Authentication

Ansible uses SSH to connect to remote servers. To make the process seamless and secure, it’s recommended to use SSH key authentication rather than passwords.

Generate SSH Key Pair:
On your control node (where Ansible is running), generate an SSH key pair if you don’t have one already. You can do this by running:

ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

This will create two files:

A private key (typically located at ~/.ssh/id_rsa).
A public key (located at ~/.ssh/id_rsa.pub).

Copy the Public Key to Managed Nodes:
Now, copy the public key to your managed nodes (the servers you want to manage). You can do this using the ssh-copy-id command:

ssh-copy-id username@your_server_ip

This will enable password-less SSH access from your control node to the managed nodes.

Verify SSH Connectivity:
Once the key is copied, test the SSH connection by running:

ssh username@your_server_ip

If everything is set up correctly, you should be able to log in without entering a password.

Now your Ansible setup is ready to communicate with the managed nodes using SSH keys, providing a secure and efficient way to manage your infrastructure.

3. Set Up Your Inventory File

Ansible needs to know which servers to manage. This is done using an inventory file. Create a file called inventory:

[webservers]
192.168.1.101
192.168.1.102

Here, [webservers] is a group name, and the IPs are the servers you want to manage.

4. Write Your First Playbook

A Playbook is where you define the tasks you want Ansible to perform. Here's an example playbook to install Nginx on your servers:

- name: Install Nginx on web servers
  hosts: webservers
  become: true 
  tasks:
    - name: Install Nginx
      apt:
        name: nginx
        state: present
    - name: Ensure Nginx is running
      service:
        name: nginx
        state: started

Save this file as install_nginx.yaml.

5. Run Your Playbook

Run your playbook with the following command:

ansible-playbook -i inventory install_nginx.yaml

Modularizing Playbooks with Roles

Ansible roles help you organize playbooks into reusable, modular components. Each role contains specific tasks, variables, and files needed for a part of your infrastructure.

Creating a Role

To create a role, use the following command:

ansible-galaxy init my_role

This creates a directory structure like this:

my_role/
├── tasks/
│   └── main.yml           
├── handlers/
├── templates/
├── files/
├── vars/
├── defaults/
└── meta/

tasks/: This directory contains the main task file, main.yml, where you define the tasks that the role will execute. -handlers/: Define actions triggered by tasks, such as restarting a service. -templates/: Store Jinja2 templates for configuration files that can be dynamically generated. -files/: Contains static files that can be copied to managed systems. -vars/: Used to define role-specific variables that can be referenced in tasks. -defaults/: Contains default variables for the role, which can be overridden by higher-priority variables. -meta/: Contains metadata about the role, such as dependencies or role requirements.

Using Roles in Playbooks

Once you’ve created a role or installed one from Ansible Galaxy, you can use it in your playbook. Here’s how to reference and use a role:

- name: Apply my role
  hosts: webservers
  roles:
    - my_role

Ansible Galaxy

Ansible Galaxy is a central hub where you can find and share pre-built Ansible roles. Instead of building everything from scratch, you can leverage these roles to save time and effort. For example, roles like geerlingguy.nginx help you quickly install and configure Nginx without needing to write all the tasks yourself.

How to Use Roles from Ansible Galaxy:

To use a role from Ansible Galaxy, follow these steps:

Step 1: Install the Role from Galaxy

You can easily install a role using the ansible-galaxy install command. For instance, to install the geerlingguy.nginx role, run the following command:

ansible-galaxy install geerlingguy.nginx

This command downloads the role and places it in your local roles directory (usually /etc/ansible/roles/ or a directory you specify).

Step 2: Reference the Role in Your Playbook

Once the role is installed, you can reference it in your playbook. In the playbook, you’ll define which hosts should execute the tasks in the role.

Example playbook (my_playbook.yaml):

- name: Install Nginx using downloaded role
  hosts: webservers
  roles:
    - geerlingguy.nginx

Here, geerlingguy.nginx is the role you installed, and webservers is the group of hosts that will run the tasks defined in that role.

Step 3: Run the Playbook

Finally, you can run your playbook with the ansible-playbook command, which will execute the tasks on the specified hosts.

Run the playbook like this:

ansible-playbook -i inventory my_playbook.yaml

Ansible will apply the geerlingguy.nginx role on all the hosts defined under webservers in your inventory file.

Handlers

Handlers are tasks triggered only when another task notifies them. For example:

tasks:
  - name: Update the configuration file
    copy:
      src: my_config.conf
      dest: /etc/my_app/
    notify: Restart the application

handlers:
  - name: Restart the application
    service:
      name: my_app
      state: restarted

Here, the handler Restart the application runs only if the copy task changes the configuration file.

Conditionals

Ansible supports conditionals to execute tasks only when specific conditions are met. For example:

- name: Install Nginx on Debian-based systems
  apt:
    name: nginx
    state: present
  when: ansible_os_family == "Debian"

Here, the task runs only if the operating system family is Debian.

Best Practices for Ansible

Use Variables: Keep your playbooks flexible by using variables.
Organize Playbooks: Use roles to structure your playbooks for reusability.
Test Locally: Use Vagrant or Docker to test playbooks before applying them to production.
Keep Secrets Secure: Use Ansible Vault to encrypt sensitive data like passwords.

Conclusion

Ansible simplifies IT automation, making it easier for both beginners and experienced engineers to manage infrastructure efficiently. With its easy-to-understand YAML syntax, agentless architecture, and powerful features like roles, handlers, and conditionals, Ansible can save you time and effort while ensuring consistency and scalability.

DEV Community: Joseph Ibeh

Terraform Commands Every Cloud Engineer Should Know

1. terraform init

2. terraform validate

3. terraform plan

4. terraform apply

5. terraform destroy

6. terraform output

7. terraform fmt

8. terraform show

9. terraform state

10. terraform taint

Bonus: terraform import

Final Thoughts

Building and Automating the Deployment of a Flask Application with Docker, GitHub, and CI/CD

Introduction

⚙️ Prerequisites

✅ Step 1: Create a Simple Flask Application

✅ Step 2: Containerize the Application

✅ Step 3: Set Up Version Control

Tools: Git + GitHub

✅ Step 4: Implement CI/CD with GitHub Actions

Tools: GitHub Actions, Docker Hub

Final Output

🎯 Conclusion

Troubleshooting

From Dockerfile to Deployment: A Simple Guide for New Developers

What Is Docker and Why Should You Use It?

Why Use Docker?

Step-by-Step Guide to Writing a Dockerfile

Example: Dockerfile for a Simple Node.js Application

Explanation of Each Command:

Best Practices for Managing Docker Images and Containers

Basic Docker Commands Every Developer Should Know

Using Docker Compose for Multi-Container Applications

Example: docker-compose.yml for a Node.js App with MongoDB

Explanation:

Running Your Docker Compose Application:

Conclusion

Building a Scalable Data Lake for Sports Analytics with AWS

Sports Analytics Data Lake

Overview of AWS Services Used

Interpretation of the Project

Steps to Set Up

Prerequisites

IAM Role/Permissions:

Step 1: Open CloudShell Console

Step 2: Create the setup_nba_data_lake.py File

Step 3: Create the delete.py Script

Running the Scripts

Go to s3 bucket and open the Json file to get the data

Building an NBA Game Day Notification System with AWS Lambda, SNS & EventBridge

🏀 NBA Game Day Notification System

Features

Prerequisites

Setup Instructions

1️⃣ Create an SNS Topic

2️⃣ Create an IAM Role for Lambda

3️⃣ Deploy the AWS Lambda Function

4️⃣ Test the Lambda Function

5️⃣ Automate with AWS EventBridge

Email Notification

Summary

Building a Scalable Real-Time Weather Dashboard with Python, OpenWeather API, and AWS S3

Weather Dashboard Project

Table of Contents

Prerequisites

Project Overview

Core Features

Tools Used

Project Setup

1. Create Project Structure

2. Create Files

3. Initialize Git Repository

4. Configure .gitignore

5. Add Dependencies

6. Install Dependencies

Environment Setup

1. AWS CLI Configuration

Example:

1. `terraform init`

2. `terraform validate`

3. `terraform plan`

4. `terraform apply`

5. `terraform destroy`

6. `terraform output`

7. `terraform fmt`

8. `terraform show`

9. `terraform state`

10. `terraform taint`

Bonus: `terraform import`

Step 2: Create the `setup_nba_data_lake.py` File