DEV Community: Mubbashir Mustafa

How to Release Unused Elastic IPs on AWS with One Prompt

Mubbashir Mustafa — Mon, 21 Jul 2025 21:10:10 +0000

👉🏽 Try Rebase for free, no credit card needed

If you have used AWS, you know about Elastic IPs. These are public IPs you can stick to your EC2 instances, load balancers, or NAT gateways. They let you keep the same public IP, even if you stop and start your resources. This comes in handy if you want to keep your DNS or firewall rules simple.

The thing is, Elastic IPs are not free if you leave them unused. AWS will charge you for any Elastic IP that is not attached to a running resource. If you forget to clean up, your bill goes up for no good reason. Removing unused IPs is one of those things people put off, especially if you have a bunch of them across different regions.

With Rebase, you do not have to mess around in the console or write a script for it. You just give it a prompt and it handles everything for you. Here is how it looks in action.

First, just ask the agent to show you the VPCs and Elastic IPs you have.

Now, ask the agent to release all the unassociated Elastic IPs.

That is all it takes.

You can check in the AWS Console to confirm they are gone.

How to Set Up AWS VPC Peering for Private Connectivity - Effortlessly

Mubbashir Mustafa — Fri, 18 Jul 2025 13:44:18 +0000

👉🏽 Try Rebase for free, no credit card needed

If you work with multiple VPCs in AWS, sooner or later you’ll want them to talk to each other. Sometimes you split workloads into different VPCs to keep things separate, or your team is working across different AWS accounts. AWS lets you connect those VPCs with something called VPC peering. That way, the traffic stays private, never hitting the public internet. But if you’ve ever tried to set this up by hand, you know it can be a hassle. You have to make sure the CIDR blocks do not overlap, the route tables are right, and both sides accept the connection. Missing one step means your setup just doesn’t work.

With Rebase, you can do all of this just by telling the agent what you want. Here’s how it works from start to finish.

We’ll start by spinning up two VPCs, both with private subnets and non-overlapping CIDR blocks. This is important, because VPC peering won’t work if the address ranges clash.

The AI agent will put together a plan and show you exactly what it’s about to do. If it needs more info, it’ll ask before moving ahead.

Once you answer, the agent finalizes the plan for you.

The agent asks for a quick confirmation to make sure everything looks right.

After you confirm, the agent goes ahead and provisions everything.

When it’s done, you’ll get a summary of everything it created. You can see all the details here.

Now grab those VPC and subnet IDs. Let’s try connecting to an EC2 instance in one of these VPCs using AWS SSM (Session Manager).

When you tell the agent you want to access EC2 via SSM, it will ask for confirmation and take care of the setup.

After it’s done, it will let you know that everything is set up and ready.

You’ll see the exact commands you need to connect through AWS SSM.

If you hit any errors, just tell the agent and it will troubleshoot and fix things for you.

It’ll walk you through any follow-up steps if you need more help.

Now, grab the private IPs of both instances in each VPC. You’ll need them for the next step.

If you’re not sure how to test the setup, just ask the agent and it’ll help out.

When you log in with SSM and try to ping the other instance from the first one, it won’t work yet. That’s because VPC peering isn’t enabled.

Now, let the agent know you want to enable VPC peering. It will handle everything needed for that.

The agent figures out all the steps, including updating route tables, so traffic can flow between the VPCs.

If there’s anything else needed, the agent will suggest it and ask for your go-ahead.

When it’s all done, you’ll get a summary of what was changed.

If you want to double-check, you can confirm everything from the AWS console.

Here’s the VPCs:

And here’s the EC2 instances:

Now, if you try pinging from one EC2 instance to the other over their private IPs, it works. You’re all set.

That’s it. VPC peering, start to finish, without having to mess with the AWS console or memorize all the steps.

How to Set Up a VPC with Public and Private Subnets, Multi-AZ, IGW, and NAT Gateway on AWS in Seconds

Mubbashir Mustafa — Mon, 14 Jul 2025 22:21:50 +0000

👉🏽 Try Rebase for free, no credit card needed

Every app on AWS needs a network setup that just works. You want your web servers out in the open, but your databases safe and locked away. That’s why people set up a VPC with both public and private subnets, spread across a few zones, with the right gateways so traffic moves how it should.

Setting this up in AWS is harder than it sounds. You miss one step, pick the wrong subnet, or forget a route, and things break. Your servers might not talk to the internet, or worse, you end up exposing your database.

Rebase makes this part easy. Instead of clicking through AWS or messing with templates, you just say what you want, and Rebase handles the rest. Here’s what it looks like.

Once you’re logged in and done with onboarding, you’ll land on the homepage.

We have a few prompts ready to go, and we’re adding more. If you pick “VPC with public and private subnets”, the prompt fills in for you.

The agent will check with you before doing anything.

If you want something specific, just say so. You can mention a CIDR block or any detail you care about. If you want Rebase to figure it out, just leave it to the agent.

Rebase will share what it plans to do and will ask you to approve it.

Once you give the green light, it gets started.

For every action that changes something, Rebase will always ask you first. You get three options:

Yes: Proceed
No: Cancel
Feedback: Tell the agent to change something or do it differently. For example, you might say, actually use just one AZ.

You can turn off these permission checks anytime from the workspace settings.

Rebase will build out everything, and if it hits something new, it will check with you first.

When it’s done, you’ll see a summary of what the agent did, step by step.

That’s it. The stuff that usually eats up hours or days, Rebase does it for you in a few seconds.

You can always double check in AWS to see everything is there.

The tags are there too, just like you asked for.

You can also ask Rebase what this setup is going to cost you every month. The agent will break down all the main parts that AWS charges for, show you what’s free, and give you a ballpark number. You can even get a detailed estimate based on how much traffic or data you expect.

This is useful if you’re trying to keep your AWS bill in check or just want to know what to expect before you hit deploy.

Python Essentials: A Speedy Introduction

Mubbashir Mustafa — Tue, 25 Jun 2024 14:51:02 +0000

Are you ready to dive into the exciting world of Artificial Intelligence and Machine Learning but need a quick introduction to Python first? This crash course is here to help! In this article, we'll cover the basics of Python programming to get you up to speed quickly. Whether you're new to programming or just need a refresher, this guide will provide the essential knowledge you need to start coding confidently. Let's get started!

Chapter 1: Python Syntax and Basics

In this chapter, we'll quickly cover Python's syntax and foundational concepts. This will serve as a concise refresher to familiarize you with Python's unique features.

Code Structure

Python uses indentation to define blocks of code, unlike languages like JavaScript which use braces. Consistent indentation is crucial as it directly impacts the program's flow. Here's a quick example:

def greet(name):
    if name:
        print(f"Hello, {name}!")
    else:
        print("Hello, world!")

Variables and Data Types

Python is dynamically typed, meaning you don't need to declare the type of a variable when you create one. The basic data types in Python include:

Strings: Defined with single ('...') or double ("...") quotes.
Integers: No special syntax required, e.g., 5.
Floats: Use a point to denote decimal, e.g., 5.0.
Booleans: Written as True or False.

Example

name = "Alice"
age = 30
is_registered = True #capitalized boolean in python

Collections

Lists: Ordered and mutable collection, e.g., numbers = [1, 2, 3].
Dictionaries: Key-value pairs, e.g., person = {"name": "Alice", "age": 30}.
Sets: Unordered collection of unique elements, e.g., unique_numbers = {1, 2, 3}.
Tuples: Ordered and immutable collection, e.g., point = (1, 2).

List Comprehensions

List comprehensions provide a concise way to create lists:

squares = [x**2 for x in range(10)]
print(squares)

[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

Functions

Defining functions in Python uses the def keyword. You can specify default values for parameters, which makes them optional during calls:

def greet(name, greeting="Hello"):
    print(f"{greeting}, {name}!")

greet("Alice")  # Uses the default greeting
greet("Bob", "Howdy")  # Uses a custom greeting

This chapter sets the foundation for understanding how Python code is structured and executed. Up next, we will explore control structures to manage the flow of your Python programs.

Chapter 2: Control Structures

Python's control structures allow you to direct the flow of your program's execution through conditional statements and loops. This section will refresh your understanding of these constructs.

If, Elif, and Else

Conditional statements in Python are straightforward. The if statement evaluates a condition and executes a block of code if the condition is true. You can extend this logic with elif (else if) and else:

age = 25
if age < 18:
    print("You are a minor.")
elif age < 65:
    print("You are an adult.")
else:
    print("You are a senior.")

Python evaluates conditions until one is true, then skips the rest, known as short-circuiting.

Loops

Python provides for and while loops for iterating over sequences or executing a block of code repeatedly under a condition.

For Loops: Used for iterating over a sequence (like a list, tuple, or string).

for number in range(5):  # range(5) generates numbers from 0 to 4
    print(number)

While Loops: Execute as long as a condition is true.

count = 0
while count < 5:
    print(count)
    count += 1

Control within loops can be managed with break (to exit the loop) and continue (to skip the current iteration and continue with the next one):

for number in range(10):
    if number == 3:
        continue  # Skip the print statement for number 3
    if number == 8:
        break  # Exit the loop when number is 8
    print(number)

Iterators and Generators

Python uses iterators for looping. You can create your own iterator by implementing iter() and next() methods or using generator functions.

Generators: Use yield to generate a sequence of values. Generators are useful when you want to iterate over a sequence without storing the entire sequence in memory.

def countdown(num):
    while num > 0:
        yield num
        num -= 1

for i in countdown(5):
    print(i)

This chapter covers the basic control structures in Python. You'll find these essential for creating conditional logic and managing looping in your Python programs. Next, we'll dive into more Pythonic techniques and features to help you write cleaner and more efficient code.

Chapter 3: Pythonic Techniques

This chapter delves into more sophisticated Pythonic techniques, enabling you to write cleaner, more efficient, and more Python-specific code. These techniques leverage Python's unique capabilities and idiomatic practices.

Unpacking

Unpacking allows you to assign values from a list or tuple to variables in a single statement, improving readability and conciseness.

a, b, c = [1, 2, 3]
print(a, b, c)  # Outputs: 1 2 3

# Star expressions for capturing excess items
first, *middle, last = [1, 2, 3, 4, 5]
print(first, middle, last)  # Outputs: 1 [2, 3, 4] 5

Lambda Functions

Lambda functions are small anonymous functions defined with the lambda keyword. They are best used for short, simple functions.

# A simple lambda to add two numbers
add = lambda x, y: x + y
print(add(5, 3))  # Outputs: 8

# Often used in functions like map() and filter()
squares = list(map(lambda x: x**2, range(10)))
print(squares)  # Outputs: [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

Map and Filter

The map and filter functions are functional programming tools that apply a function to each item in an iterable (like a list) and return an iterable.

map(): Applies a function to each item in an iterable.
filter(): Extracts elements from an iterable for which a function returns True.

# Using map to square numbers
nums = [1, 2, 3, 4, 5]
squared = list(map(lambda x: x**2, nums))

# Using filter to find even numbers
evens = list(filter(lambda x: x % 2 == 0, nums))

When compared to list comprehensions, map() and filter() can be preferable for their readability and expressive power with small lambda functions. However, list comprehensions are often clearer and more Pythonic when the operation is straightforward.

This chapter introduces you to writing Python in a way that is not just correct but also stylistically Pythonic, embracing the language's philosophy of readability and simplicity. By using these advanced features, you can make your code more modular, reusable, and expressive. Up next, we'll cover file handling and exception management to ensure your programs are robust and professional.

Chapter 4: File Handling and Exception Management

In Python, managing files and handling exceptions are critical for creating robust applications. This chapter covers the essentials of these areas, focusing on best practices and Pythonic approaches.

File Handling

Python simplifies file operations with built-in functions and methods. The with statement ensures that files are properly closed after their suite finishes, even if an exception is raised.

# Reading from a file
with open('example.txt', 'r') as file:
    content = file.read()
    print(content)

# Writing to a file
with open('output.txt', 'w') as file:
    file.write("Hello, Python!\n")

For reading and writing files, Python offers methods like read(), readline(), readlines(), write(), and writelines(), allowing you to handle different file sizes and content types efficiently.

Exception Handling

Proper error and exception handling is essential for writing reliable and user-friendly programs. Python uses try, except, else, and finally blocks to handle exceptions.

try:
    # Try to do something that might raise an exception
    result = 10 / 0
except ZeroDivisionError:
    # Handle specific exceptions
    print("Divided by zero!")
else:
    # Execute if no exceptions
    print("Division successful!", result)
finally:
    # Always executed
    print("Cleaning up, if needed.")

Use specific exception types in except blocks to catch and handle different error conditions appropriately. This not only helps in debugging but also allows the program to continue or fail gracefully.

Context Managers

For more complex resource management, Python provides context managers that allow you to allocate and release resources precisely when you want. The with statement is commonly used with file handling, as shown above, but can also be used with other resources like network connections or locking mechanisms.

from contextlib import contextmanager

@contextmanager
def managed_file(name):
    try:
        f = open(name, 'w')
        yield f
    finally:
        f.close()

# Usage
with managed_file('hello.txt') as f:
    f.write('hello, world!')

This chapter ensures that you are equipped to handle file I/O operations and exceptions in your Python applications effectively, contributing to their reliability and maintainability. Next, we will explore modules, packages, and Python's environment management tools.

Chapter 5: Modules and Packages

This chapter focuses on the organization and reusability of code in Python through modules and packages. Understanding how to create, import, and manage modules and packages is key for developing maintainable and scalable Python applications.

Importing Modules

Python's modules are simply Python files with .py extensions containing Python code that can be reused in other Python scripts.

# Importing a whole module
import math
print(math.sqrt(16))  # Outputs: 4.0

# Importing specific functions
from math import sqrt
print(sqrt(16))  # Outputs: 4.0

# Using alias for modules
import numpy as np
array = np.array([1, 2, 3])

When you import a module, Python executes all of the code in the module file.

Creating Modules

Any Python file can be a module. To create a module, simply save your Python code in a .py file. Other Python scripts can then import this file as a module.

# Example of a simple module, saved as calculator.py

def add(a, b):
    return a + b

def subtract(a, b):
    return a - b

You can then use this module in other Python scripts:

from calculator import add, subtract
print(add(5, 3))  # Outputs: 8

Packages

A package is a collection of Python modules under a common directory. For Python to recognize a directory as a package, it must contain a file named init.py. The file can be empty but it signifies that the directory contains Python modules.

# Assume the following directory structure:
# mypackage/
#   __init__.py
#   subpackage1/
#       __init__.py
#       module1.py
#   subpackage2/
#       __init__.py
#       module2.py

# Importing from a package
from mypackage.subpackage1 import module1
module1.some_function()

Virtual Environments

Using virtual environments in Python helps manage dependencies for different projects. By creating a virtual environment, you can keep dependencies required by different projects separate from each other.

# Creating a virtual environment
python -m venv myprojectenv

# Activating the virtual environment
# On Windows
myprojectenv\Scripts\activate

# On MacOS/Linux
source myprojectenv/bin/activate

This setup ensures that any Python packages installed while the virtual environment is active will only affect this particular environment, helping avoid conflicts between project dependencies.

This chapter provides the tools you need to effectively manage and modularize your Python code, essential for professional, clean, and efficient Python programming. Coming up next, we'll discuss best practices in logging and debugging to enhance the observability and maintainability of your Python applications.

Chapter 6: Logging and Debugging

Effective logging and debugging are crucial for developing, maintaining, and troubleshooting Python applications. This chapter covers how to utilize Python's built-in logging module and debugging tools to keep your applications running smoothly and efficiently.

Logging

Python's logging module provides a flexible framework for emitting log messages from Python programs. It is preferable to using print statements because it offers different severity levels and allows you to direct the logs to different outputs.

import logging

# Basic configuration to log to file
logging.basicConfig(filename='app.log', level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')

# Different levels of logs
logging.debug('This is a debug message')
logging.info('This is an info message')
logging.warning('This is a warning message')
logging.error('This is an error message')
logging.critical('This is a critical message')

Using Log Levels

Log levels provide a way to categorize the importance of the messages logged by the application:

DEBUG: Detailed information, typically of interest only when diagnosing problems.
**INFO: Confirmation that things are working as expected.
WARNING: An indication that something unexpected happened, or indicative of some problem in the near future.
ERROR: Due to a more serious problem, the software has not been able to perform some function.
CRITICAL: A serious error, indicating that the program itself may be unable to continue running.

Debugging Tools
When it comes to debugging, Python provides several tools to help identify issues in the code. The most commonly used tool is the Python Debugger (pdb), which allows interactive debugging.

import pdb

# Example usage
def div(a, b):
    pdb.set_trace()
    return a / b

print(div(4, 2))

Using pdb, you can step through your code, inspect variables, and evaluate expressions to diagnose and fix issues more effectively.

Commands: list (shows current position in the code), next (executes the next line), continue (continues execution until the next breakpoint), break (adds breakpoints), print (prints a variable), and quit (exits the debugger).

Tips for Effective Debugging

Start Small: Test small parts of your code as you write them.
Use Logs: Insert logging statements to report the occurrence of particular events.
Isolate Problems: When you encounter a bug, narrow down where it could be by using unit tests or dividing the code.

This chapter equips you with essential tools for logging and debugging, helping ensure your Python applications perform as intended and making it easier to maintain and troubleshoot them. In the next chapter, we will explore the broader Python ecosystem, including toolchains and additional libraries that can help streamline your Python development process.

Chapter 7: Toolchain and Additional Libraries

In this final chapter, we'll explore the broader Python ecosystem, focusing on toolchains for code style enforcement, testing, and some additional libraries that extend Python's capabilities. These tools and libraries can significantly enhance productivity and ensure high-quality software development.

PEP 8 and Linters

PEP 8 is the style guide for Python code. Adhering to PEP 8 helps ensure that your Python code is readable and consistent. Python's linters like flake8 or pylint help enforce coding standards and catch potential errors.

# Installing flake8
pip install flake8

# Using flake8
flake8 my_script.py

These tools provide feedback on style issues, complexity, and programmatic errors, making them indispensable for maintaining code quality.

Testing
Testing is critical in the development process to ensure your code behaves as expected.

unittest: Python’s built-in library that allows you to test small units of code independently.

import unittest

class TestMathOperations(unittest.TestCase):
    def test_add(self):
        self.assertEqual(1 + 1, 2)

    def test_subtract(self):
        self.assertEqual(2 - 1, 1)

if __name__ == '__main__':
    unittest.main()

pytest: A more powerful, third-party testing framework with a simpler syntax and more features than unittest.

# Installing pytest
pip install pytest

# Using pytest
pytest

Popular Python Libraries

Several Python libraries can help simplify complex tasks across different domains:

Requests: Simplifies making HTTP requests for web clients.

import requests
response = requests.get('https://api.example.com/data')

Pandas: Essential for data analysis and manipulation.

import pandas as pd
data = pd.read_csv('data.csv')

NumPy: Provides support for large, multi-dimensional arrays and matrices.

import numpy as np
array = np.array([1, 2, 3])

Scikit-Learn: Ideal for implementing machine learning algorithms.

from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier()
model.fit(train_data, train_labels)

Matplotlib: A plotting library for creating static, interactive, and animated visualizations in Python

import matplotlib.pyplot as plt
plt.plot([1, 2, 3], [4, 5, 6])
plt.show()

Extra Chapter: Python and Object-Oriented Programming

Introduction to Classes and Objects

In Python, classes are created using the class keyword, and objects are instances of these classes. Here’s a simple example:

class Dog:
    def __init__(self, name, age):
        self.name = name
        self.age = age

    def speak(self, sound):
        return f"{self.name} says {sound}"

# Creating an instance of Dog
my_dog = Dog("Rex", 4)
print(my_dog.speak("Woof"))

Attributes and Methods

Attributes are variables associated with a class, and methods are functions defined within a class that operate on its attributes. The init method is a special method called a constructor, used for initializing an instance of the class.

Inheritance

Python supports inheritance, allowing multiple base classes and thus facilitating complex relationships between objects.

class Cat(Dog):  # Inherits from Dog
    def purr(self):
        return f"{self.name} purrs."

# Usage
my_cat = Cat("Whiskers", 3)
print(my_cat.speak("Meow"))  # Inherited method
print(my_cat.purr())         # New method

Encapsulation

Encapsulation is the bundling of data with the methods that operate on these data. It restricts direct access to some of the object’s components, which can prevent the accidental modification of data:

class Person:
    def __init__(self, name, age):
        self.name = name
        self._age = age  # Leading underscore suggests protected (by convention)

    def get_age(self):
        return self._age

# Direct access to _age is discouraged
john = Person("John", 28)
print(john.get_age())

Polymorphism

Polymorphism allows for the interchangeability of components through a common interface. In Python, it’s more loosely applied, allowing different classes to be used interchangeably if they implement the same methods.

def animal_sound(animal):
    print(animal.speak("Hi there"))

animal_sound(my_dog)
animal_sound(my_cat)

Congratulations on completing this quick crash course on Python basics! I hope this speedy overview has given you a solid foundation in Python programming, equipping you with the essential skills needed to embark on your AI and ML journey. While this guide covered the fundamentals, there's so much more to learn and explore when implementing practical solutions in the world of Artificial Intelligence and Machine Learning. As you continue to build on these basics, you'll discover the power and versatility of Python in solving complex problems and creating innovative solutions. Stay curious, keep coding, and get ready to transform your ideas into reality with Python!

Notebook version: https://github.com/mubbashir10/applied_ai/blob/main/Chapter%200%20-%20Python%20basics.ipynb

Enable Slack Notifications for AWS Amplify Deployments

Mubbashir Mustafa — Sun, 20 Mar 2022 08:09:39 +0000

AWS Amplify is a set of purpose-built tools and features that lets front-end web and mobile developers quickly and easily build full-stack applications on AWS, with the flexibility to leverage the breadth of AWS services as your use cases evolve.

At our company, we use AWS Amplify for hosting our frontend (React) apps. Although it's an amazing service and we love it, it lacks one important feature: integration with AWS Chat Bot (for sending pipeline notification to slack). It however allows you to send notifications to email addresses (which is not ideal, at least for us).

But there's a robust workaround for this problem, let me share that with you.

From AWS web console, go to AWS Amplify

Select the app against which you want to enable slack notifications

Select "Notifications" from the left sidebar

Enter an email address that you use and then click save

Now you need to create a lambda function that will receive messages from the AWS SNS topic (created by AWS Amplify) and forward that to slack

From AWS Console, go to AWS Lambda

Select "Create function"

Give a name to your function, choose runtime as node 14.x, and click create

Add the following code to your function

// deps in lambda are added as layers
const fetch = require('node-fetch')

// this should be your webhook URL (doc: https://api.slack.com/messaging/webhooks)
const integrationURL = ''

exports.handler = async (event) => {

  const message = event.Records[0].Sns.Message

  return await fetch(integrationURL, {
      method: 'POST',
      body: JSON.stringify({
        attachments: [{
          title: `AWS Amplifyy Notification!`,
          text: message,
        }]
      }),
      headers: { 'Content-Type': 'application/json' },
    })
    .then((data) => console.log('sent!'))
    .catch((e) => console.error(e.response.data))
}

Or grab it from github gist

The code uses the node-fetch package to make HTTP post request, you will need to add a dependency layer to your lambda function for it to work

To get integration URL you will need to create the slack app and enable incoming webhooks, check out the official tutorial.

Save your changes and click the "Deploy" button

Next, add a trigger to your AWS Lambda function

Select "SNS" as the trigger, and then select the SNS Topic that would be created by AWS Amplify (when you added email earlier to enable notifications)

To test things out, go back to AWS Amplify and from being within the project, click "Redeploy this version"

You should receive a notification in your slack

The email you added earlier is no longer needed (we only did that to expose SNS Topic) and you could remove it by visiting AWS Amplify->notifications.

Let's connect:

Linkedin: https://www.linkedin.com/in/mubbashir10/

Twitter: https://twitter.com/mubbashir100

AWS Copilot - launch and manage containerized applications quickly 🐳

Mubbashir Mustafa — Mon, 31 Jan 2022 01:10:33 +0000

AWS Copilot is an open-source CLI-based tool by the AWS team which lets you deploy containerized services easily by running a few simple commands. Behind the scenes, AWS takes care of all the infrastructure provisioning and configuration.

Before we begin, it's assumed that you have already containerized your application and it's ready to be deployed. If not, you can clone this sample repo and follow along.

Step 1

Currently, AWS Copilot doesn't work with root users and you need to create an IAM user with programmatic access. Go to IAM and create a new IAM account with programmatic access

Note: ℹ️ It's not mentioned in the documentation which policies are needed for AWS Copilot to work, see this github issue. For the sake of this tutorial you can give Administrator Access to your role.

Step 2

Install the AWS Copilot, if you are on a mac and use homebrew, you can simply run

brew install aws/tap/copilot-cli

For other OS, grab the relevant binary and install that

Verify the installation by running copilot -v

Step 3

Inside your app folder, run

copilot init

It will ask you to name your application, provide any name

Next, select the application type (for this tutorial I am selecting Load Balanced Web Service

Next, it will ask for the name of the service, provide any suitable name

If there are multiple Dockerfiles present, it will ask you to pick the desired one

It will begin the build process now

Once the build is ready, it will ask you if you would like to deploy your service to a test environment, select Yes

Wait for it to finish the process, once finished it will give you a URL to the service

Grab the URL and test it in a browser

Step 4

Now set up a pipeline for automated workflows (CI/CD)

Inside your app run

copilot pipeline init

git add copilot/pipeline.yml copilot/buildspec.yml copilot/.workspace && git commit -m "Adding pipeline artifacts" && git push

copilot pipeline update

Once the pipeline finishes, you can verify the changes by visiting the same URL again

Cleanup

To delete the app just run copilot app delete

AWS Copilot takes the hassle of provisioning and managing infrastructure and lets you deploy your containerized applications quickly. Behind the scenes, it creates everything for you (Cloud formation template, AWS ECR repo, AWS Code Pipeline, AWS Code build project, AWS ECS task definition, AWS ECS Cluster, AWS ECS Service, AWS Load balancer, Security groups, etc.
If you need more granular control and are interested in deploying your dockerized application on AWS ECS manually, you can follow the step-by-step tutorial here.

Let's connect:

Linkedin: https://www.linkedin.com/in/mubbashir10/

Twitter: https://twitter.com/mubbashir100

Deploy containerized functions on AWS Lambda

Mubbashir Mustafa — Sun, 31 Oct 2021 19:21:31 +0000

AWS Lambda lets you deploy containerized functions by packaging your AWS lambda function code and dependencies using Docker up to a size of 10GB. Here's a tutorial to demonstrate how you can containerize and deploy nodejs based lambda functions.

Prepare Container Image for AWS Lambda:

If you prefer you can clone the repo, otherwise follow along.

Create a file called functions.js inside your node project and add the following sample function to it.

// A sample function to demo containers deployment on aws lambda
exports.helloLambda = async (event) => {
  const response = {
    isBase64Encoded: false,
    statusCode: 200,
    headers: {
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      message: "Containers on lambda!🐳",
    }),
  };
  return response;
};

Create a Dockerfile with the following content

FROM amazon/aws-lambda-nodejs:12
COPY functions.js package*.json ./
# RUN npm install // uncomment if your functions has dependencies
CMD [ "functions.helloLambda" ]

Build, tag and push the image to ECR*

aws ecr get-login-password --region <region-name> | docker login --username AWS --password-stdin <ecr-repo-uri-without-tag>

docker build -t node-app .

docker tag node-app:latest <ecr-repo-uri-without-tag>/<repo-name>:latest

docker push <ecr-repo-uri-without-tag>/<repo-name>:latest

*Learn how to publish images on ECR

Deploy The Image on AWS Lambda:

From the AWS Lambda landing page, select "Create function"

Choose "Container image", give any name, add image URI (can be obtained from AWS ECR) and click 'Create function'

To test the function, add a trigger

Choose API Gateway as a trigger and create an HTTP API and leave the security to open (for simplicity)

Once the trigger has been created, copy the endpoint URL and paste it in the browser

It should show you the response content

For containers to work with AWS Lambda, you can either use the open-source base container images that AWS Provides or you can add lambda runtime interface clients to your base images. In the tutorial, we have used a pre-built images.

Let's connect:

Linkedin: https://www.linkedin.com/in/mubbashir10/

Twitter: https://twitter.com/mubbashir100

Run Containers on AWS Lightsail

Mubbashir Mustafa — Mon, 13 Sep 2021 14:22:08 +0000

AWS Lightsail is an easy-to-use service that offers all the features needed to host web applications of any scale. Besides VPS services, the AWS Lightsail also lets you deploy your containers.

From the AWS console, navigate to 'Lightsail'

From the Lightsail homepage, switch to the 'Containers' tab and then click 'Create container service'

Select the region where you want to deploy your containerized application

Select size and number of instances based on your usage

Click 'Set up deployment'

Choose 'Specify a custom deployment'
Enter any meaningful name as Container name
Enter your public image name with tag, example: nginx:latest (at the time of writing AWS Lightsail doesn't support private images for container deployment)
Click 'Add open ports' and add ports on which your application listens to
In the PUBLIC ENDPOINT, select the container name which is open to public

Give a name to your service and click 'Create container service'

Once the deployment is finished successfully, it will change the status to active

To access your service/containerized application, use the link given as a public domain

Let's connect:

Linkedin: https://www.linkedin.com/in/mubbashir10/

Twitter: https://twitter.com/mubbashir100

Deploy Microservices on AWS ECS with Fargate (Serverless)

Mubbashir Mustafa — Sun, 12 Sep 2021 12:16:04 +0000

Amazon ECS (Amazon Elastic Container Service) is a highly scalable and fully managed container management service that makes it a breeze to run, stop, and manage containers on a cluster. You can use it to deploy your containerized applications or your microservices.

Before starting, here's a summary of key concepts in AWS ECS:

Cluster

A logical grouping of your services or tasks

Task Definition

Specification about how your container(s) should be run on AWS ECS

Task

An instance of the Task Definition

Service

A task manager

Section 1: Create an AWS ECS Cluster

From the AWS console, navigate to AWS ECS

Click 'Get started'

Click 'Create cluster'

Select 'Network only' as cluster template, click Next step

Configure your cluster and click 'Create'

Cluster name
Give any meaningful name to your cluster

Create VPC (optional)
Enable to launch your cluster in a VPC (A VPC is an isolated portion of the AWS Cloud)

Tags (optional)
You can use them to help you organize/group your AWS resources

CloudWatch Container Insights
Check to enable container insights (failures, restarts, CPU utilization, memory usage, .etc) and view them inside AWS Cloud Watch

Once the cluster has been created, click 'View Cluster'

Section 2: Create an AWS ECS Task Definition

From the left sidebar (expand it by clicking the hamburger menu from the left top), click 'Task Definitions'

Click 'Create new Task Definition'

Select 'Fargate' and click 'Next step'

Give a meaningful name to your task definition, leave the Task Role field empty (or you can create a new role from IAM and select it here), and leave the 'Network mode' to awsvpc (when launching as the Fargate type, only awsvpc can be selected)

Select 'Create new role' (or select any other existing role that you have created, and configure vCPU and Memory size for this task (this determines what charges you will be paying for this task)

Click 'Add container'

Give a name to your container, in the image field paste the image URI of the image (If you are not familiar with AWS ECR then read this first), add port to which your container listens to (for example if your nodejs app is running on 3000, then you should enter 3000 in this field)

You can skip the Advanced container configuration but if you need to configure other aspects of your container (like health checks, volumes, networking, the environment variable, etc. then you can do that from this section).

ℹ️ Note: You can add multiple containers under a Task definition

Once the container configuration is completed, click 'Add'

You can skip the integrations with AWS Firelens and AWS App Mesh unless you need it. If you need to add volumes that should be accessible to every container within a task, do so from the volumes section. The Tags are optional and let you organize your AWS resources.

Click 'Add'

Once the task definition has been created, it will show you a success message

Section 3: Create an AWS ECS Service

Now go back to your cluster and click 'Deploy' (from the services section)

Leave everything to default

Application Type:
Service

Specify revision manually
Uncheck

Family
Select the Task Definition you created earlier

Revision
Latest

Service Name
Any meaningful name

Desired Tasks
Enter the number of Tasks (instances) you want to run

Load balancing is optional but recommended. Configure a load balancer to distribute incoming traffic across the tasks running in your service.

Click on 'Create a new load balancer'
Give any meaningful name
Enter in the port where you want AWS ELB (elastic load balancer) to listen on (this is not the container's port that you configured earlier in the task definition)
Select HTTP/HTTPS as protocol (for HTTPS you will need to have a valid SSL certificate)
Target group routes traffic from AWS ELB to tasks/instances; give any name and select the protocol as HTTP

In the Networking section:

Select the VPC in which your cluster and task definitions are (by default, it's always already selected)
Select subnets (within VPC) where you want to deploy the task (I have selected all available subnets under my VPC)
Select "Use an existing security group", and select a security group that allows inbound HTTP traffic - learn how to create
Every task can be associated with a Public IP address, disable it if you don't need a public IP

Add tags if you want to and then click Deploy

Once the service has deployed, it will change the status to "Active"

Section 4: Access the AWS ECS Service

To access the service you can go to AWS EC2-> Load Balancers->Your Load Balancer Name and then copy the A-Record

To access an individual task (provided you enabled the public IP earlier), go to AWS ECS->Services->Your Service->Logs->Your Task->Network->Public IP

Let's connect:

Linkedin: https://www.linkedin.com/in/mubbashir10/

Twitter: https://twitter.com/mubbashir100

Control inbound/outbound traffic to an AWS Resource using AWS Security Groups

Mubbashir Mustafa — Sun, 12 Sep 2021 10:40:50 +0000

AWS Security group is like a virtual firewall within a VPC that acts at the instance level and not at a subnet level. Security groups have a set of rules to allow/disallow incoming/outgoing traffic to an instance.

Creating a security group is a fairly simple task. From the EC2 Console, select "Security Groups" under the Network & Security section

Click 'Create security group'

Give any meaningful name and a description, enter in the ID of the VPC in which you want to create the security group

Click 'Add rule' within the Inbound rules section and add HTTP and HTTPS and set the source to 'Anywhere' in both

Leave the Outbound rules section as it is (unless you want to change something) and click 'Create security group'

Let's connect:

Linkedin: https://www.linkedin.com/in/mubbashir10/

Twitter: https://twitter.com/mubbashir100

Deploy containerized services on AWS App Runner

Mubbashir Mustafa — Sun, 05 Sep 2021 11:38:15 +0000

AWS App Runner is the easiest way to deploy your Containerized, Node or Python based services. You don't have to worry about managing infrastructure or scaling up and down the resources when the load increases or decreases.

ℹ️ Note: It's assumed that you are familiar with AWS ECR and your image has already been containerized and pushed. If you are new to AWS ECR then read this first.

To deploy your services on AWS App Runner, navigate to AWS App Runner from the AWS console

From there click on "Create an App Runner service"

Now you need to provide the source image - which AWS App Runner will deploy as a service. For this, select "Container Registry", "Amazon ECR" (select Public, if your image is not private) and paste the Container image URI (you can get it from AWS ECR)

Next configure the deployment settings for your service. Select "Automatic" (it will deploy a new version of your service as soon as the new image has been pushed), "Create new service role" and "Next"

Give any name to your backend service, select vCPU and memory as per the need (expected load/traffic), and enter environment variables (if needed) as key/value pairs

In the "Auto-scaling" settings, you can use the default one or if you need to adjust them then create a "Custom configuration"

Concurrency:
Max. number of concurrent requests/connections after which a new instance will be launched
Minimum size:
It's the minimum number of instances that will always be running regardless of load
Maximum Size:
Maximum number of instances you want AWS App runner to launch when the load increases (in this case more than 100 concurrent requests/connections)

The health check is a mechanism to know whether the service is performing and stable or not. You can leave the default settings or configure them as needed

By default, the Unhealthy threshold is set to 5. This essentially means that whenever 5 health check requests from the load balancer to an instance fails, it will consider it unhealthy and try to replace it.

Next, you can create a custom IAM role from the https://aws.amazon.com/iam/ that will give permissions to your container to communicate with other AWS services or you can leave it. To encrypt the stored image (if you chose ECR image ) or stored source code bundle (if you chose GitHub, earlier) you can use AWS-owned encryption key or customer-managed key CMK (can be created from https://aws.amazon.com/kms/.

By default, if you don't provide one, an AWS-owned encryption key is used.

Tags are optional key/value pairs to help you identify and group your AWS resources. Add tags if needed and then click "Next"

Review your configurations and click "Create & Deploy"

It will take some time to build your app, once the status changes from "Operation in progress" to "Completed", your changes are deployed

Once the service has deployed successfully, get the default domain link and see it in action

Let's make some changes to your code and push it (build the docker image and push to AWS ECR - you will notice that the build process triggers automatically and deploys the new version

Notes:
AWS App Runner offers the easiest and quickest way to run containerized application. If you don't want to use conainerized application(s), then you can also deploy your nodejs or python based services on AWS App Runner from Github.

Let's connect:

Linkedin: https://www.linkedin.com/in/mubbashir10/

Twitter: https://twitter.com/mubbashir100

Push/Publish Docker images to AWS ECR

Mubbashir Mustafa — Fri, 03 Sep 2021 17:44:43 +0000

Amazon Elastic Container Registry (AWS ECR) is a container registry service where you store, manage, share, and deploy your container images (like DockerHub). Consider it as npmjs of containers instead of JS packages. AWS ECR lets you publish private as well as public images.

Before you proceed, it's assumed that you already have a containerized application. If you don't have one, you can clone this sample nodejs application.

From the AWS ECR console, click 'Get Started'

Options

Visibility settings: select Private or Public
Repository name: give any meaningful name
Tag immutability: when enabled, it will prevent tags from being overwritten by new pushes (when pushed with the same tag)

Enable Image scan setting to enable vulnerability scan after the image is pushed

Enable to use AWS KMS for encryption

Once the repository is created, it will take you back to the repositories list. Select the newly created repository and then click on the "View push commands" button.

Now you need to use these commands one by one to build, tag, and push your container image. But before proceeding make sure you have the latest version of AWS CLI installed.

Use the following command to check that AWS CLI is correctly installed on your system.

aws --version

It would output something like this. Your version could be different, it depends when you are reading this article.

Next, you need to create 'Access Keys'. For that, go to "My Security Credentials" from the dropdown with your username.

Click on "Create New Access Key", it will create a new key for you.

Copy "Access Key ID" and "Secret Access Key"

Now go back to your terminal and enter the following command
aws configure

It will ask for "Access Key ID" and "Secret Access Key", "Default region", provide them one by one. You can skip the default output format.

Once the AWS CLI has been configured, you can run the push commands. Open up the terminal and make sure you are inside the containerized app's root directory. Run each command shown in the popup opened up by clicking the "View push commands" button. Please note that the commands in the screenshot could be different than what you see in your popup and it's totally fine.

*If your 'Dockerfile' is named something other than 'Dockerfile', you will have to specify it using -f flag (see below).

Let's connect:

Linkedin: https://www.linkedin.com/in/mubbashir10/

Twitter: https://twitter.com/mubbashir100