DEV Community: Nur Fathiha Tahiat Seeum

What is AWS Lambda - Everything I know about AWS Lambda

Nur Fathiha Tahiat Seeum — Thu, 20 Jul 2023 22:06:11 +0000

In the ever-evolving world of cloud computing, AWS Lambda has emerged as a game-changer, revolutionizing the way developers build and deploy applications. As a serverless compute service, AWS Lambda enables developers to focus solely on writing code, without worrying about server management or infrastructure scaling. In this blog post, we will explore the wonders of AWS Lambda and uncover its potential to transform the way we approach application development.

What is AWS Lambda?

AWS Lambda is a serverless compute service provided by Amazon Web Services (AWS). It allows developers to run code without provisioning or managing servers. With Lambda, developers can effortlessly build scalable and cost-efficient applications by executing code in response to events such as changes to data in Amazon S3 buckets, updates in Amazon DynamoDB tables, or HTTP requests via Amazon API Gateway.

Source: https://images.app.goo.gl/mGKzcoRUZJHZJbxS8

Key Features and Benefits:

Pay-as-you-go Model: AWS Lambda follows a pay-as-you-go pricing model, where you are billed only for the compute time consumed by your code. There are no upfront fees or hidden costs, making it extremely cost-effective.

Automatic Scaling: Lambda automatically scales your applications in response to incoming requests. It dynamically allocates resources based on the number of requests, ensuring your application remains highly available and responsive.

Event-Driven Architecture: Lambda seamlessly integrates with various AWS services, allowing developers to create event-driven architectures. This enables real-time data processing, trigger-based workflows, and more.

Wide Language Support: AWS Lambda supports multiple programming languages, including Node.js, Python, Java, Go, and more, giving developers the flexibility to choose their preferred language.

Easy Integration: Lambda integrates effortlessly with other AWS services, such as Amazon S3, Amazon DynamoDB, Amazon API Gateway, and AWS Step Functions, enabling the creation of sophisticated serverless applications.

Getting Started with AWS Lambda:

To get started with AWS Lambda, follow these steps:

Step 1: Sign in to the AWS Management Console and open the Lambda service.

Step 2: Create a Lambda function by selecting a blueprint or starting from scratch.

Step 3: Write your code in the Lambda function editor or upload a ZIP file containing your code.

Step 4: Configure your Lambda function's trigger, which can be an event from an AWS service or an HTTP request.

Step 5: Test your Lambda function to ensure it works as expected.

Step 6: Deploy your Lambda function, and AWS will take care of the rest, handling all the infrastructure management and scaling.

Real-World Use Cases:

AWS Lambda has found its way into various industries and use cases:

Real-Time Data Processing: Lambda can process streaming data from sources like IoT devices or social media streams in real-time, enabling businesses to respond to events immediately.

Serverless Web Applications: By combining Lambda with Amazon API Gateway and Amazon S3, developers can build serverless web applications with minimal infrastructure overhead.

Image and Video Processing: Lambda can be utilized to resize images, transcode videos, or perform other media processing tasks on-the-fly.

Chatbots: Developers can build intelligent chatbots by integrating Lambda with natural language processing services like Amazon Lex or Amazon Comprehend.

AWS Lambda has revolutionized the world of cloud computing, offering developers an unparalleled platform to build serverless applications effortlessly. By eliminating the burden of managing servers, AWS Lambda empowers developers to focus on writing code and delivering innovative solutions. Whether you're a seasoned developer or just starting your cloud journey, AWS Lambda is a powerful tool that opens up a world of possibilities for building scalable, cost-effective, and highly responsive applications. Embrace the serverless revolution and unleash the full potential of AWS Lambda in your next project!

Exploring the Lifecycle of Amazon EC2 Instances: A Comprehensive Guide

Nur Fathiha Tahiat Seeum — Mon, 10 Jul 2023 08:25:00 +0000

In this blog post, we will dive into the lifecycle of Amazon EC2 instances, discussing the essential configurations required to create an EC2 instance, including instance types, network settings, and storage options. We will explore the concept of instance families and their use cases, the default Amazon Virtual Private Cloud (VPC), and how to architect for high availability. Additionally, we will examine the EC2 instance lifecycle, pricing options, and cost-saving strategies through on-demand, reserved, and spot instances.

Background:

Understanding the lifecycle and pricing options of Amazon EC2 instances is crucial for optimizing resource utilization and cost efficiency. By selecting the appropriate configurations and implementing high availability strategies, users can enhance the performance and reliability of their applications while controlling expenses.

Exploring the EC2 Instance Components:

EC2 instances are composed of virtual processors (vCPUs), memory, network capabilities, and, in some cases, instance storage and graphics processing units (GPUs). By choosing the right combination of these components, users can tailor their instances to meet the specific requirements of their applications.

Instance Families and Use Cases:

EC2 instances are categorized into different instance families based on their optimized workloads. We will discuss the general-purpose, compute optimized, memory optimized, accelerated computing, and storage optimized instance families, exploring their respective use cases and performance characteristics.

Understanding the Default VPC:

The default Amazon Virtual Private Cloud (VPC) simplifies the initial setup of EC2 instances but should not be used for storing sensitive or private data. We will emphasize the importance of creating custom VPCs and implementing appropriate network configurations to ensure security and control over resources.

Architecting for High Availability:

To ensure high availability, it is recommended to distribute EC2 instances across multiple Availability Zones. We will explain the significance of designing architecture with redundancy and fault tolerance in mind, highlighting the advantages of using multiple smaller instances over a few larger ones.

Exploring the EC2 Instance Lifecycle:

From creation to termination, EC2 instances undergo various states. We will discuss the pending and running states, the implications of rebooting, stopping, starting, and terminating instances, as well as the differences between stop and stop-hibernate options. Understanding the lifecycle helps optimize resource management and facilitates efficient application deployment.

Deciphering EC2 Pricing:

We will unravel the pricing structure of EC2 instances, distinguishing instance costs from other associated services. Users will gain insights into per-second billing, exceptions to pricing conventions, and the options to reduce costs through on-demand instances, reserved instances (RIs), and spot instances.

Pricing Options for EC2 Instances:

We will explore the benefits of on-demand instances for continuous utilization, the significant discounts offered by reserved instances, and the potential savings with spot instances. Each option will be discussed, enabling users to select the most suitable pricing model based on their application requirements and cost considerations.

Conclusion:

Understanding the EC2 instance lifecycle and the available pricing options empowers users to make informed decisions when configuring, deploying, and managing their instances. By leveraging the right configurations and pricing models, users can optimize performance, achieve high availability, and control costs effectively within the Amazon EC2 environment.

References:

Amazon Elastic Compute Cloud (EC2)

Nur Fathiha Tahiat Seeum — Mon, 10 Jul 2023 07:23:00 +0000

What Is Amazon EC2?

Amazon EC2 is a web service that provides secure, resizable compute capacity in the cloud. It allows you to provision virtual servers called EC2 instances. Although AWS uses the phrase “web service” to describe it, it doesn’t mean that you are limited to running just web servers on your EC2 instances. You can create and manage these instances through the AWS Management Console, the AWS Command Line Interface (CLI), AWS Software Development Kits (SDKs), or through automation tools and infrastructure orchestration services. In order to create an EC2 instance, you need to define:

Hardware specifications, like CPU, memory, network, and storage.

Logical configurations, like networking location, firewall rules, authentication, and the operating system of your choice.

When launching an EC2 instance, the first setting you configure is which operating system you want by selecting an Amazon Machine Image (AMI).

What Is an AMI?

In the traditional infrastructure world, the process of spinning up a server consists of installing an operating system from installation disks, installation drives, or installation wizards over the network. In the AWS Cloud, this operating system installation is no longer your responsibility, and is instead built into the AMI that you choose. Not only does an AMI let you configure which operating system you want, you can also select storage mappings, the architecture type (such as 32-bit, 64-bit, or 64-bit ARM), and additional software installed.

What Is the Relationship Between AMIs and EC2 Instances?

EC2 instances are live instantiations of what is defined in an AMI, much like a cake is a live instantiation of a cake recipe. If you are familiar with software development, you can also see this kind of relationship between a Class and an Object.

A Class is something you model and define, while an object is something you interact with. In this case, the AMI is how you model and define your instance, while the EC2 instance is the entity you interact with, where you can install your web server, and serve your content to users. When you launch a new instance, AWS allocates a virtual machine that runs on a hypervisor. Then the AMI you selected is copied to the root device volume, which contains the image used to boot the volume. In the end, you get a server you can connect to and install packages and any additional software. In this case, you install a web server along with the properly configured source code of your employee directory app.

One advantage of using AMIs is that they are reusable.

You might choose a Linux-based AMI and configure the HTTP server, application packages, and any additional software you may need to run your application.

If you wanted to create a second EC2 instance with the same configurations, how can you easily do that? One option is to go through the entire instance creation and configuration process and try to match your settings to the first instance. However, this is time consuming and leaves room for human error.

The second, better option, is to create an AMI from your running instance and use this AMI to start a new instance. This way, your new instance will have all the same configurations as your current instance, because the configurations set in the AMIs are the same.

Where Can You Find AMIs?

You can select an AMI from the following categories.

Quick Start AMIs that are premade by AWS and allow you to get started quickly.

AWS Marketplace AMIs that provide popular open source and commercial software from third-party vendors.

My AMIs that are created from your EC2 instances.

Community AMIs that are provided by the AWS user community.

Build your own custom image with EC2 Image Builder.

Each AMI in the AWS Management Console has an AMI ID, which is prefixed by “ami-”, followed by a random hash of numbers and letters. These IDs are unique to each AWS region.

Resources:

Book Synapsis write-up - "The 7 Spiritual Laws of Success"

Nur Fathiha Tahiat Seeum — Mon, 13 Mar 2023 06:11:36 +0000

"The 7 Spiritual Laws of Success" is a self-help book by Deepak Chopra, which explores the spiritual principles that can help individuals to achieve success in their personal and professional lives. The book is based on the idea that success is not just a matter of hard work and determination, but also of aligning oneself with the natural laws of the universe.

It represents a set of principles which can be applied to one's life to achieve success in all aspects, including health, wealth, and happiness. The seven laws outlined in the book are:

Law-1: The Law of Pure Potentiality: This law states that we all have unlimited potential within us. By connecting with the universe and utilizing this potential, we can achieve success. This law encourages individuals to connect with their inner self and to recognize their own unique talents and abilities. To demonstrate this law, imagine a seed that contains everything it needs to become a fully grown plant. We too have everything we need within us to reach our full potential; we just need to nurture and cultivate it.
Ways to increase consciousness to utilize our potential:

Practicing silence: Practicing silence for 5-10 minutes or more to gather the unorganized thoughts of our minds.
Meditation: Practicing any form of meditation for 30 minutes each day to connect to our inner beings.
Spending time with nature: Watching sunsets or listening to the sound of ocean or smelling a flower can help us to develop a sense of harmonious interaction with the nature.
Practicing non-judgement: Spending a day without judging anyone’s act based on the outcomes.

Law-2: The Law of Giving: This law states that the more we give, the more we receive. For example, imagine a river that is constantly flowing. If it blocks its nature of flowing, the water becomes stagnant and stops flowing. But if we allow the river to flow freely, it continues to flow and bring life to everything around it. Overall, this law emphasizes the importance of generosity and encourages individuals to give freely of their time, energy, and resources.

Law-3: The Law of Karma: This law states that every action we take has a consequence, and we reap as we sow. By being mindful of our behavior and intentions, we can create positive outcomes in our own lives. This law teaches us that. To exemplify this law, imagine a ball that we throw out into the world. Whatever we throw out into the world comes back to us, so it's important to be mindful of our actions and intentions.

Law-4: The Law of Least Effort: This law encourages us to work in harmony with the universe rather than against it. By letting go of battle and surrendering to the flow of life, we can achieve success with minimal effort. imagine a sailboat that is traveling down a river. The boat doesn't have to use much effort to move with the current, but if it tries to go against the current, it will require a lot more effort and energy.

Law-5: The Law of Intention and Desire: This law emphasizes the power of intention and the importance of having clear goals and desires. By focusing our energy and attention on what we want, we can manifest it into reality, just like a bow and arrow. Our intentions and desires are like the bow, and the arrow is our focused energy. By aiming our intentions and desires towards our goals, we can use our energy to propel us towards them.

Law-6: The Law of Detachment: This law teaches us to let go of attachment to outcomes and trust in the universe to deliver what is best for us. By releasing our attachment to specific outcomes, we can reduce stress and anxiety and enjoy the journey towards success. To clarify this law, think of a bird that is flying through the sky. The bird doesn't hold onto anything, it simply trusts in its ability to fly and lets the wind guide it.

Law-7: The Law of Dharma: This law encourages us to discover and follow our life's purpose. By aligning with our true nature and pursuing our passions, we can live a fulfilling and purposeful life. Suppose we are solving a puzzle. Each piece represents a different aspect of our life's purpose, and when we put all the pieces together, we can see the bigger picture and understand our unique contribution to the world.

Overall, "The 7 Spiritual Laws of Success" is a concise and practical guide for achieving fulfillment in all aspects of life by aligning with the natural laws of the universe. The book encourages readers to cultivate a deeper understanding of themselves and their place in the world, and to live their lives with greater purpose, meaning, and fulfillment.

Advanced AI: Fuzzy TOPSIS

Nur Fathiha Tahiat Seeum — Tue, 07 Mar 2023 16:23:43 +0000

The Fuzzy TOPSIS (Technique for Order Preference by Similarities to Ideal Solution) is a decision-making approach used to select the best alternative among a set of options by considering multiple criteria simultaneously. It uses fuzzy logic to handle the uncertainty and imprecision of the data and allows for both quantitative and qualitative criteria to be incorporated into the decision-making process. Fuzzy TOPSIS ranks the alternatives based on their similarity to the ideal solution and the distance from the negative ideal solution, ultimately providing a comprehensive ranking of the alternatives. It is commonly used in fields such as engineering in the domains of artificial intelligence, decision-making, and pattern recognition, handling imprecise or uncertain information and modeling human reasoning processes. It is also highly used in the sectors of management, and environmental studies.

As we have seen earlier, Fuzzy TOPSIS is a combination of two concepts which are: Fuzzy set theory and TOPSIS (Technique for Order Preference by Similarities to Ideal Solution).

Now, What is Fuzzy Set?

Fuzzy sets, introduced independently by Lotfi A. Zadeh and Dieter Klaua in 1965 is a mathematical concept used in fuzzy logic to represent a set of elements that have degrees of membership or belongingness to the set.

Unlike classical or crisp sets, which have elements that are either fully in or fully out of the set, fuzzy sets allow for elements to have partial membership in the set based on their degree of similarity to the set's defining characteristics.

In a fuzzy set, the membership function assigns a degree of membership between 0 and 1 to each element of the universe of discourse, indicating the extent to which the element belongs to the fuzzy set.

Defination

the fuzzy set Ã in terms of its membership function μÃ(x) and the universe of discourse X is defined as:

Ã = {x, μÃ(x) | x ⋲ X}

The notation {x, μÃ(x) | x ⋲ X} represents the set of ordered pairs (x, μÃ(x)) for all x in the universe of discourse X, where μÃ(x) is the degree of membership of x in the fuzzy set Ã. In other words, for each element x in the universe of discourse X, the membership function μÃ(x) assigns a degree of membership between 0 and 1, indicating the degree to which x belongs to the fuzzy set Ã

The specific form of the membership function μÃ(x) would depend on the problem being modeled and the characteristics of the elements in the universe of discourse X. Different types of membership functions, such as triangular, trapezoidal, or Gaussian functions, can be used to represent different types of fuzzy sets.

Example code

Here is an example code for implementing Fuzzy TOPSIS algorithm in Python:


import numpy as np

def fuzzy_topsis(X, w):
    # Normalize the decision matrix
    X_norm = X / np.sqrt(np.sum(X**2, axis=0))

    # Compute the weighted normalized decision matrix
    X_weighted = X_norm * w

    # Determine the positive and negative ideal solutions
    X_max = np.max(X_weighted, axis=1)
    X_min = np.min(X_weighted, axis=1)

    # Compute the separation measures from the ideal solutions
    S_max = np.sqrt(np.sum((X_weighted - X_max[:, np.newaxis])**2, axis=1))
    S_min = np.sqrt(np.sum((X_weighted - X_min[:, np.newaxis])**2, axis=1))

    # Compute the relative closeness to the ideal solutions
    C = S_min / (S_max + S_min)

    return C

Chapter-2: Decision making through Random Forest

Nur Fathiha Tahiat Seeum — Mon, 27 Feb 2023 06:10:32 +0000

In the second chapter of our Symbolic Machine Learning series, We will be talking about one of the most popular machine learning algorithm Random Forests.

So, Let's explore the definition first.

Random Forest is a popular decision tree-based non-linear Machine learning algorithm (ensemble learning) used for classification, feature selection and regression tasks. It combines multiple decision trees to improve the accuracy and robustness of the model.

Random Forest

The word 'Random' is for random selection of features/data instances which is known as bootstrapping method in statistics and in Machine Learning as well.
The word 'Forest' means that we will not use only one or two decision trees, we will use several decision trees in order to develop decision models through Bagging method.

Here, each and every decision-tree will be formed from a random subset of data. This is how, from large sets of data, by choosing a subset and forming a decision tree at once recursively, we make a Random Forest. Now, there should be a probability that a randomly chosen sample in node would be incorrectly labelled and here comes the term "GINI Impurity".

Gini impurity is a commonly used measure of impurity in decision tree-based algorithms for classification tasks. It is a measure of impurity or heterogeneity which quantifies how often a randomly chosen data point in a given dataset would be incorrectly classified if it were labeled randomly according to the distribution of labels in the subset. In decision trees, the Gini impurity is used to select the best feature to split the data, by comparing the Gini impurity of each possible split and selecting the one that results in the lowest impurity or highest information gain

Formula
Gini impurity = 1 - (p1^2 + p2^2)

where p1 and p2 are the proportions of the two classes in the dataset. In other words, the Gini impurity measures the probability of misclassifying a randomly chosen element in the dataset if it were labeled according to the distribution of class labels. A lower Gini impurity score indicates a more homogeneous subset in terms of the class labels, and therefore a better split.

In case we have have to split the data based on category, we would work on the data clusters. However, if we have numerical data, we may take the mean and split the data into datasets based on a specific condition. The image below will help to have a clear understanding.

Example-3

Here's an example Python code to calculate the Gini impurity of a binary classification problem:

import numpy as np

def gini_impurity(labels):
    # count the number of instances for each class
    _, counts = np.unique(labels, return_counts=True)
    # compute the probabilities of each class
    probs = counts / len(labels)
    # calculate the Gini impurity
    impurity = 1 - np.sum(probs ** 2)
    return impurity

Let us now review the Steps in Random Forest Classification Method:

Bootstrapping for random data subset generation
Decision tree construction for each of the data subset

Determination of GINI impurity of each of the features.
Determination of GINI impurity of prospective splitting
sub-tree
Construction of Decision tree based on the splitting
GINI impurity (i.e. if sum of the GINI impurity of splitted
sub-tree is lower than the GINI impurity of parent node
then split the parent node)

Bagging for ensemble classification
Majority voting for classification decision making.

Let us now see an example of the whole process:

Get started with Machine Learning (Part-1)

Nur Fathiha Tahiat Seeum — Thu, 23 Feb 2023 06:33:58 +0000

Welcome everyone!

Today, In the first session of our series symbolic Machine Learning, we will be talking about some basic ML stuffs.

So, What is Machine learning?

Machine Learning is an application of Artificial Intelligence (AI) that provides systems with the ability to automatically learn and improve from the learnings without being explicitly programmed.

According to guru ChatGPT,

Machine Learning is a technology that uses mathematical and statistical techniques to enable computer systems to learn from data and improve their performance on a particular task without being explicitly programmed. By analyzing and identifying patterns in data, machine learning algorithms can make predictions, identify anomalies, or classify new data points based on their similarity to previously seen data. The mathematical and statistical methods used in machine learning include linear algebra, calculus, probability theory, and optimization.

Data —> Learning Algorithm —> Understanding [Prediction/hypothesis]

ML works all the way from data to Understanding.

Now, someone might ask ...

If Machine Learning works to infer data to hypothesis/understanding, So what’s the difference between Machine Learning and Data mining?

To be clear, Data mining is the process of discovering patterns, correlations, and insights from large datasets using statistical and computational methods. It involves identifying hidden patterns and relationships in data that can be used to make informed decisions, improve business processes, or gain insights into human behavior. On top of that, data mining techniques include clustering, classification, association rule mining, and anomaly detection, among others. The ultimate goal of data mining is to extract valuable information from data and use it to improve business outcomes or gain new insights.

Confusing, huh?

Well, Although the two of the concepts looks exactly the same but There are differences!

Now look closely, Data mining is primarily concerned with discovering patterns, correlations, and insights from large datasets, whereas Machine Learning is focused on building predictive models and making decisions based on those models.

Secondly, Data mining can be applied to both structured and unstructured data, whereas Machine Learning generally requires structured data that has been preprocessed and labeled for training purposes

Lastly, about the applications, Data mining is commonly used in areas such as marketing, finance, and healthcare, where the goal is to gain insights and improve decision-making. Machine Learning is used in a wide range of applications, including image recognition, natural language processing, speech recognition, and recommendation systems.

Is it clear now?
No worries, If you still have any doubts, let me know!

Now, lets move on!

The most commonly recognized types of Machine Learning:

Supervised Learning: In this type of Machine Learning, the algorithm is trained on labeled data, where the correct outputs are already known, and the goal is to learn a mapping function that can predict the output for new input data.
Unsupervised Learning: In this type of Machine Learning, the algorithm is trained on unlabeled data, and the goal is to discover hidden patterns, structures, and relationships in the data without any predefined labels.
Semi-supervised Learning: In this type of Machine Learning, the algorithm is trained on a combination of labeled and unlabeled data, where the goal is to use the unlabeled data to improve the accuracy of the model on the labeled data.
Reinforcement Learning: In this type of Machine Learning, the algorithm learns by interacting with an environment and receiving feedback in the form of rewards or penalties. The goal is to learn a policy that maximizes the cumulative reward over time.
These types of Machine Learning are not mutually exclusive, and many real-world applications involve a combination of multiple techniques.

Different Machine Learning tasks

1. Regression:
Input- Known features and labeled data, output - a numeric value

2. Classification:
Input- Known features and labeled data, output- categories (Cat/dog, Spam/not spam)

3. Clustering:
Input- Known features but unlabeled data, Output- a number of groups based on similarity

4. Anomaly Detection:
Identifying instances that are significantly different from the majority of the data. (fraud detection, system health monitoring and so on).

That's pretty much about the fundamentals of Machine learning. Stay tuned for the next part!

Goodbye!