DEV Community: Sudhanshu Mukherjee

How to use AWS Rekognition for Object Detection

Sudhanshu Mukherjee — Wed, 12 Jul 2023 01:03:43 +0000

Amazon Web Services (AWS) is the leading cloud computing platform offered by Amazon.com. It provides a comprehensive suite of cloud-based services that enable individuals and organizations to build, deploy, and manage applications and infrastructure with ease and scalability. From computing power to storage, databases to machine learning, AWS offers a vast array of services that cater to a wide range of requirements. In this article, we will explore some of the key services provided by Amazon AWS.

Elastic Compute Cloud (EC2): EC2 is a highly scalable virtual server service that allows users to rent virtual machines in the cloud. It provides a flexible and secure computing environment, allowing users to configure instances with various operating systems, storage options, and networking capabilities.
Simple Storage Service (S3): S3 is a highly durable and scalable object storage service that enables users to store and retrieve vast amounts of data. It offers high availability, data encryption, and easy integration with other AWS services, making it an ideal choice for storing backups, static content for websites, and big data analytics.
Relational Database Service (RDS): RDS simplifies the setup, operation, and scaling of relational databases in the cloud. It supports popular database engines such as MySQL, PostgreSQL, Oracle, and Microsoft SQL Server. RDS automates tasks like backups, software patching, and database scaling, allowing users to focus on their applications rather than database management.
Lambda: AWS Lambda is a serverless computing service that enables users to run code without provisioning or managing servers. It allows developers to build applications using functions that are triggered by events, providing a highly scalable and cost-effective solution for running code in response to various events.
Identity and Access Management (IAM): IAM is a service that helps manage user access and permissions for AWS resources. It allows users to create and manage user accounts, roles, and groups, providing fine-grained control over who can access specific resources and actions within the AWS environment.
Elastic Beanstalk: Elastic Beanstalk is a fully managed service for deploying and scaling web applications. It provides an easy-to-use platform that abstracts the underlying infrastructure, allowing developers to focus on writing code. Elastic Beanstalk supports various programming languages and frameworks, making it a versatile choice for deploying web applications.
Simple Queue Service (SQS): SQS is a fully managed message queuing service that enables decoupling of components in distributed systems. It allows applications to send, store, and receive messages between different software components, helping to build scalable and fault-tolerant systems.
Simple Notification Service (SNS): SNS is a highly flexible and scalable messaging service that enables the pub/sub (publish/subscribe) messaging pattern. It allows applications to send notifications to multiple recipients or subscribers via email, SMS, mobile push, or other custom endpoints.
Machine Learning Services: AWS provides a suite of services for machine learning and artificial intelligence (AI) applications. This includes Amazon SageMaker, which simplifies the process of building, training, and deploying machine learning models, as well as services like Amazon Rekognition for image and video analysis, Amazon Comprehend for natural language processing, and Amazon Polly for text-to-speech conversion.
Internet of Things (IoT): AWS offers a comprehensive set of services for building and managing IoT applications. This includes AWS IoT Core, a managed cloud platform for securely connecting and managing devices, as well as services like AWS IoT Analytics for data analysis, AWS IoT Greengrass for offline and edge computing, and AWS IoT Device Management for device provisioning and management.

These are just a few examples of the wide range of services provided by Amazon AWS. With its vast portfolio of offerings, AWS caters to the needs of startups, enterprises, and developers alike, empowering them to innovate and build scalable, reliable, and cost-effective solutions in the cloud. Whether it's computing power, storage, databases, machine learning, or IoT, AWS provides the building blocks to support a wide variety of applications and workloads in the cloud.

Amazon AWS and It's Services

Sudhanshu Mukherjee — Wed, 12 Jul 2023 01:00:43 +0000

Elastic Compute Cloud (EC2): EC2 is a highly scalable virtual server service that allows users to rent virtual machines in the cloud. It provides a flexible and secure computing environment, allowing users to configure instances with various operating systems, storage options, and networking capabilities.
Simple Storage Service (S3): S3 is a highly durable and scalable object storage service that enables users to store and retrieve vast amounts of data. It offers high availability, data encryption, and easy integration with other AWS services, making it an ideal choice for storing backups, static content for websites, and big data analytics.
Relational Database Service (RDS): RDS simplifies the setup, operation, and scaling of relational databases in the cloud. It supports popular database engines such as MySQL, PostgreSQL, Oracle, and Microsoft SQL Server. RDS automates tasks like backups, software patching, and database scaling, allowing users to focus on their applications rather than database management.
Lambda: AWS Lambda is a serverless computing service that enables users to run code without provisioning or managing servers. It allows developers to build applications using functions that are triggered by events, providing a highly scalable and cost-effective solution for running code in response to various events.
Identity and Access Management (IAM): IAM is a service that helps manage user access and permissions for AWS resources. It allows users to create and manage user accounts, roles, and groups, providing fine-grained control over who can access specific resources and actions within the AWS environment.
Elastic Beanstalk: Elastic Beanstalk is a fully managed service for deploying and scaling web applications. It provides an easy-to-use platform that abstracts the underlying infrastructure, allowing developers to focus on writing code. Elastic Beanstalk supports various programming languages and frameworks, making it a versatile choice for deploying web applications.
Simple Queue Service (SQS): SQS is a fully managed message queuing service that enables decoupling of components in distributed systems. It allows applications to send, store, and receive messages between different software components, helping to build scalable and fault-tolerant systems.
Simple Notification Service (SNS): SNS is a highly flexible and scalable messaging service that enables the pub/sub (publish/subscribe) messaging pattern. It allows applications to send notifications to multiple recipients or subscribers via email, SMS, mobile push, or other custom endpoints.
Machine Learning Services: AWS provides a suite of services for machine learning and artificial intelligence (AI) applications. This includes Amazon SageMaker, which simplifies the process of building, training, and deploying machine learning models, as well as services like Amazon Rekognition for image and video analysis, Amazon Comprehend for natural language processing, and Amazon Polly for text-to-speech conversion.
Internet of Things (IoT): AWS offers a comprehensive set of services for building and managing IoT applications. This includes AWS IoT Core, a managed cloud platform for securely connecting and managing devices, as well as services like AWS IoT Analytics for data analysis, AWS IoT Greengrass for offline and edge computing, and AWS IoT Device Management for device provisioning and management.

Web Scraping Multiple Pages using Python & BeautifulSoup

Sudhanshu Mukherjee — Thu, 27 Jan 2022 06:26:59 +0000

Web Scraping

Web Scraping is used to examine websites for Unstructured Data and store it Structurally for our use. Web Scraping helps us iterate over several web pages and extract the required information, It is then stored in a format suitable for the user.

In this project, we will learn 'How to use Python and Beautiful Soup to scrape Mobile Phone Names and Prices of Brand Mi on a Flipkart E-commerce Website'.

Click here for a code

Inspecting the Element

The objective of this project is to scrape Flipkart's Website and extract the 'Name' and 'Prices' of Mi Mobile Phones.
Get the URL that will be needed to make a request.

The URL which we will be using: https://www.flipkart.com/mobiles/mi~brand/pr?sid=tyy%2C4io&otracker=nmenu_sub_Electronics_0_Mi&page=

Now, It is very necessary to identify the right class of the elements stored which we need to scrape. To inspect elements follow these steps.
Step 1 - Visit the URL
Step 2 - Right on the website and select inspect or press Ctrl + shift + I together.
Step 3 - Hover on the name of the phone and click it. Select the class from the window appearing on the right.
Step 4 - Apply the same process for price.
Step 5 - Copy this class somewhere, we will need it later in our code.

Import Libraries, modules, and Packages

Let's import the required libraries and packages first. To begin with, I will be using Beautiful Soup for parsing HTML documents which can later be used to extract data from HTML. Then, We will import 'requests' which allows us to interact with the web, It contains many useful features and methods to make HTTP requests.
Lastly, we will need pandas to store our scraped data in an organized way.

from bs4 import BeautifulSoup import requests import pandas as pd

Write the program

Create an empty list for Phone Name and Phone Price.
phone_name = [] phone_price = []
Declare a variable 'page_num' which will help us take input from users about the number of pages they want to scrape.
page_num = int(input("Enter number of pages:"))
Now we will use for Loop to iterate over multiple pages looking for relevant information and operations to implement.
Then, Let's store our URL in the variable url and make requests to build a connection. Don't forget to add(+) str(i). This will help our program to iterate over multiple pages while using for loop.
After, making a request, let's use Beautiful Soup for parsing HTML and store all the data in a variable called content.

Declare a variable called name to find and store all div and the relevant class(which we copied using inspect element) of phone name.
Declare a variable called price to find and store all div and the relevant class(which we copied using inspect element) of phone price.

Let's append the data extracted in the empty list which we have created at the start.

Now, let's create a DataFrame to store our data structurally and export DataFrame into CSV file.

This is our Aha moment folks. We have successfully created our web scraper using python and Beautiful Soup. To get a better understanding of this, Use a website that you personally like and try scraping it.