DEV Community: Saima Javaid

What is Cloud Computing and what AWS offers?

Saima Javaid — Tue, 29 Mar 2022 21:36:27 +0000

In this era of science and technology everyone is talking about cloud and cloud providers. In this article we will learn what Cloud Computing is and who provides these services. I will specifically talk about AWS as I have got a chance to use its services.

Cloud Computing

It is an on-demand delivery of technology services over the internet, e.g., servers, storage, databases, networking, software, analytics, and intelligence. One do not need to own the complete resource and spend a lot of money, instead they can pay as per need. Typically known as pay-as-you-go.

Cloud Service Providers

There are renowned providers in the market that offers cloud computing services which are not only economical but easy to use. You just need to provision your infrastructure or configure a service in few clicks and the resource is ready to use. The prominent ones are Amazon Web Services (AWS), Microsoft Azure, and Google Cloud. Other providers in the list are IBM, Oracle, Alibaba, Scala, SalsForce and many more.

Amazon Web Services (AWS)

Among the currently available list of cloud providers, Amazon was the first one who has launched a stable cloud services under the name of Amazon Web Services (AWS). It has still maintaining the leading position and keeps growing the list of services, it offers a wide range of cloud computing services.

Amazon Web Services (AWS)

Let’s talk about what type of services they offer.

AWS offers a wide collection of cloud-based products which fits for different business requirements globally. The products includes compute, development tools, management tools, storage, databases, and networking. It also includes IoT, analytics, mobile, and enterprise applications with a pay-as-you-go pricing model. Following is the handy list of services to have a quick reference of business offers provided by AWS.

Cloud Computing Services

There are three types of Cloud computing services:

Infrastructure as a Service (IaaS): provides the infrastructure i.e. mainly Hardware and Operating System (OS). Therefore, user can install and manage software, workloads, and deployments without having pain to purchase/manage hardware and OS.
Platform as a Service (PaaS): provides platform and organization only need to take care of developing their applications and databases.
Software as a Service (SaaS): provides software as a solution where user only needs to think about how they will use the software, rest will be manage by provider.

Let’s name some of the computing services provided by AWS:

Amazon EC2 (Elastic Compute Cloud): Virtual Servers on the cloud that provides you control on the OS. This virtual machine can be run by the user on demand.
Amazon LightSail: It automatically deploy and manages virtual server, container, storage, databases that are required to run user application.
AWS Elastic Beanstalk: It offers automated deployment and provisioning of resources like a highly scalable production website.
Amazon Elastic Kubernetes Service (EKS): It allows to use Kubernetes on Amazon cloud environment without installation.
AWS Lambda: It allows to run code in the cloud as a function without thinking about servers. It is a big cost saver because you only pay when your function executes.

Migration & Transfer

Migration services used to transfer data physically between datacenters and AWS. Following list provides the overview of AWS migration services:

AWS Database Migration Service (DMS): This service helps to migrate on-premise database to AWS Cloud with minimal downtime. It also helps to migrate data between different types of databases, for example, Oracle to MySQL.
AWS Server Migration Service (SMS): It helps to migrate on-premises servers to AWS easily and quickly.
AWS Snowball: Snowball is a small application which allows you to transfer terabytes of data inside and outside of AWS environment.

Storage

Storage services used to store files and binary data.

Amazon Glacier: It is a low-cost storage service that offers secure and fast storage for data archiving and backup.
Amazon Elastic Block Store (EBS): It provides block-level storage to use with Amazon EC2 instances.
AWS Storage Gateway: It provides secure integration between on-premises software applications and cloud-based storage.

Database Services

Amazon RDS: A service for easy setup, operating, and scaling a relational database in the cloud.
Amazon DynamoDB: A fully managed NoSQL database service which allows cost-effective storage and retrieval of data.
Amazon ElastiCache: A web service to easily deploy, operate, and scale in-memory cache in the cloud.
Amazon Aurora: It’s a MySQL and PostgreSQL-compatible relational database, build for cloud. It is high performance and low cost.
Amazon RedShift: It’s a fast, simple and cost effective data warehousing solution.

Analytics

Amazon Athena: It allows to query data using SQL queries on S3 bucket to search files.
Amazon CloudSearch: It is used to create a fully managed search engine for a website.
Amazon OpenSearch: It is derived from ElasticSearch and performs similar search engine as CloudSearch. However, it also provide real time application monitoring.
Amazon Kinesis: It helps to stream and analyze real-time data at massive scale.
Amazon QuickSight: It’s a business analytics tool which helps to create a visualization in the form of a dashboard by using simple human readable queries.
Amazon EMR: This AWS analytics service mainly used for big data processing and running large-scale distributed data processing jobs using open-source frameworks, for example, Spark, Splunk, Hadoop, etc.
AWS Data Pipeline: Allows to automate movement and transformation of data from one place to another.

Deployment & Management Services

Amazon CloudWatch: It is used to monitor AWS environments such as EC2, RDS instances, and CPU utilization. It also allows to configure metrics and triggers alarms.
AWS CloudFormation: It offers to speed up the cloud provisioning with defining infrastructure by coding templates.
AWS CloudTrail: It provides an easy method to track user activity and API usage.
AWS OpsWorks: It automates operations with Chef and Puppets.
AWS Config: It continuously monitors user environment and audit the changes in configuration against the desired ones. It generates alerts for deviation from the required values.
AWS Service Catalog: It helps large organization to create, organize and govern the catalog of AWS products.
AWS Systems Manager: It helps to gain operational insight and take action when required.
AWS Managed Services: It offers management of AWS infrastructure which releases the responsibility and helps to focus only on applications.

Internet of Things

AWS IoT Core: It’s a managed cloud service that connects devices with the cloud.
AWS IoT Device Management: It offers to organize and manage IoT devices remotely.
AWS IoT Analytics: it provides the ability to analyze data collected by IoT devices.
Amazon FreeRTOS: This real-time operating system for microcontrollers enables user to connect IoT devices locally or with cloud.

Application Services

AWS Step Functions: It is a visual workflow for modern applications that manages failures, retries, parallelization, service integrations, and observability, therefore user can focus on business logic.
Simple Workflow Service (SWF): A service to help in coordinating both automated and human-led tasks.
Simple Notification Service (SNS): A notification generation service in the form of email, SMS, or mobile push notification.
Simple Queue Service (SQS): It manages message queues. Message queues is a mechanism to decouple tightly coupled integration between services.

Developer Tools

AWS CodeStar: A cloud-based service for developing and managing various software development projects on AWS.
AWS CodeCommit: An AWS version control service to upload and manage code privately in the cloud.
AWS CodeBuild: A process automation service to compile and build code.
AWS CodeDeploy: Deploying code in EC2 instances automatically.
AWS CodePipeline: It helps to create a deployment pipeline after passing through different stages such as building, testing, authenticating, and deploying on different environment such as testing, development and production environments.
AWS Cloud9: It‘s an IDS for developing and debugging code in the cloud.

Mobile Services

AWS Device Farm: It improves the quality of mobile and web applications by testing them across desktop browsers and real mobile devices that are hosted in the AWS Cloud.
AWS AppSync: Accelerate application development by GraphQL API that offers real-time data synchronization and offline programming features.

Artificial Intelligence

Amazon Lex: It’s a tool to build voice and text chatbots quickly.
Amazon Polly: Its text-to-speech service that enables you to convert text into life like speech.
Amazon Rekognition: It’s an object detection service which helps to find face and object in the images and videos.
Amazon SageMaker: It is a service to build, train, and deploy ML models at any scale.
Amazon Transcribe: It’s a speech recognition service that help to convert speech-to-text and offers high-quality and affordable transcriptions.
Amazon Translate: It’s a natural and fluent language translator.

Final Words

There are hundreds of services offered by AWS, the purpose of this article is to give a quick overview from where we can start our cloud journey. We can find similar services offered by other providers. Since I am mostly working with AWS, therefore I have listed down and explain them for reader understanding.

Reference Links

Message Queues

Saima Javaid — Tue, 29 Mar 2022 21:16:40 +0000

What is Message Queue?

Message Queue allows application to communicate by sending messages without communicating directly. It stores messages temporarily in a simple format until it has been delivered to the receiving application. This model consists of following components:

Message: It is the data that will be send/receive between the applications.
Queue: Messages are stored in a queue so that they can be processed in the specific sorted order when the other application is ready to receive them.
Producer: Producer is the application that sends the message.
Consumer: Consumer is the receiving application.

Why need Message queue?

Let’s understand it with the most common example, a mail delivery mechanism.

There is a small business which serves for the customer service, in the beginning they have small volume of mail and it’s easy to check every piece at the door step.
As the business increases the customer support increases, hence mail volume grows. Mail delivery will become disruptive so it is a good idea to have an unattended mailbox where the mail can be dropped off.
As the business grows over the time, multiple mailboxes becomes the need to handle the volume of mail.
Sometime in future it may require to setup a Post Office box to receive all of the inbound mail and distribute the accumulated mail to various mailboxes by the company.
So the post office box is the great solution to avoid loosing the mail during the peak time. It will get the mail for the sender and hold it until it is dispatched to the intended Mailbox.
Similarly, the message queues are designed to hold the data temporarily when the consumer is busy or it is peak time of traffic and consumer cannot handle the volume. Message Queues offers following benefits to the application architecture.

Reliability

Use of message queues can make application more reliable. During downtime of consumer, Message Queue will hold the data until the consumer is ready to receive the message again. This will increase the reliability and overcome the chances of losing message.

Decoupling

Another advantage of using message queue is decoupling. Producer can send messages without bothering if consumer is available, therefore applications are decoupled.

Scalability

Adding more Mailbox in Mail Delivery example increases the capacity of holding Mail. Similarly, adding more message queues can increase scalability of application during peak time.

There are three types of popular queues (list is not in any specific order):

Apache Kafka
AWS SQS
RabbitMQ

Apache Kafka

Apache Kafka is a distributed data store that’s ideal for processing real-time data which is known as streaming data. Streaming data is a non-stop generation of data from countless number of sources. Kafka will records the streams using a fault-tolerant approach and it offers high throughput to manage real time data transfers.

The major advantage of Kafka is its ability to parallelize topics using partitions. Each partition can be hosted over different machines. Therefore, consumers can access topics in parallel. You can also allow multiple consumers to access multiple partitions which makes scaling even easier. This unique feature significantly increases Kafka’s throughput for processing message queues.

AWS SQS

Amazon Simple Queue Service (SQS) is said to be a fully managed message queuing service. Therefore, using SQS allows to decouple and scale micro-services, distributed systems, and serverless applications. Being a managed service it eliminates the complexity and overhead of the managing and operating message-oriented middleware. The developers can easily focus on business logic and other development tasks.

Amazon SQS offers two types of queues i.e. standard queues and FIFO queues. Both ensure that message reaches a single consumer and offer similar functionality.

RabbitMQ

RabbitMQ supports many messaging techniques including pub-sub, point-to-point, and request-reply. The service is also portable, reliable, and user-friendly, however it’s not as scalable as the other message brokers. The overhead of mirroring and the latency from the central node significantly decreases the throughput of RabbitMQ.

The major advantage of RabbitMQ is the support for complex routing, because of this it is used by many major companies. It is preferred where the message security is crucial or if more control on messages is required. It offers several built-in plugins and also gives the option to use a custom plugin.

Final Words

Every message queues has its own pros and cons, and its selection totally depend on your use case and requirements. For example, companies that do not want to manage MQs and serve more focus on application development will choose AWS SQS. Whereas, Apache Kafka can be a choice where large stream of data is required to be processed in parallel.

Reference & Further Reading

Getting Started Machine Learning with AWS

Saima Javaid — Sun, 27 Mar 2022 19:01:40 +0000

What is Machine Learning?

Machine learning (ML) is a software development technique that enable computers to solve complex problems by using samples from real-world data. Therefore, this technique lets computer to automatically learn and improve with the experience when not being explicitly programmed to do so.

How it works?

In traditional problem solving with software engineering, engineer will gather data and analyze the real-world problem. They design a solution and write down a code to solve that particular problem. It’s a laborious task to develop a code for many real-world problems.

In Machine Learning, developer writes a code snippet that will be trained by the real-world data in such a way that it will start predicting the outcome of new data which was not supplied before. That code snippet is called Model and it will be trained by a special program called Model Training Algorithm.

Why Machine Learning with AWS?

Yes, that’s the first question arises when you’ll start reading the title. Why AWS?

AWS is offering so many easy to use services and tools, which makes it easy for everyone to dive in the machine learning journey. It abstracted out so many complex implementation from the user and provides easy functionality that any new comer can easily start playing with. The mission of AWS is to put machine learning in the hands of every developer.

AWS provides many pre-trained AI services which can be used as a ready-made intelligence to a wide range of applications. There are many example to such services, for example, personalized recommendations, modernizing your contact center, improving safety and security, and increasing customer engagement.

Amazon SageMaker

It’s a fully managed service which removes complexity from machine learning workflows, therefore every developer and data scientist can deploy machine learning easily.

Getting Started

As I have previously explained that AWS is offering many services that enable every developer to easily start working with ML. However, AWS is also offering many ways of learning for the ML students and developers that are new to this field.

AWS has created an educational portfolio of devices that helps to put new machine learning techniques into the hands of developers in a fun ways with AWS DeepLens, AWS DeepRacer, and AWS DeepComposer.

AWS DeepLens: This is a camera device that helps in learning Computer vision.
AWS DeepRacer: It’s a racing car designed to run on the physical track. It helps to learn Reinforcement Learning technique.
AWS DeepComposer: It’s a composing device that powered by generative AI. It creates melody that transforms into a completely original song.

AWS DeepLens

AWS DeepLens allows you to create and deploy end-to-end computer vision–based applications. This device is integrated with several AWS machine learning services. You can connect with AWS cloud to use deployed models that are trained and test with the new data captured by the device. It enables you to learn and explore the latest artificial intelligence (AI) tools and techniques for developing computer vision applications based on a deep learning model.

AWS DeepRacer

This device is an autonomous racing car designed to test reinforcement learning models by racing on a physical track. Reinforcement learning is a ML technique where a model, normally called as agent here, is trained by feedback from the environment. The agent will be rewarded in numbers for the actions it takes. Actions that help the agent achieve its goal are incentivized with higher numbers. The actions that are not helpful will result in a low reward or no reward.

There are several racing competitions held monthly by AWS DeepRacer League. You can enter in these competitions by sending your model for chances to win prizes while diving into machine learning development skills.

Note that the device is not mandatory during development, AWS offers simulator where you can test your model. There competitions also not required physical device, many competitions are held using simulators.

AWS DeepComposer

AWS DeepComposer is specifically design to provide a creative and easy way to get started with machine learning (ML), specifically generative AI. It consists of a USB keyboard that connects with the computer to input melody. There is an AWS DeepComposer console, which includes a Music Studio to generate music and there is a learning capsules to dive deep into generative AI models.

Here again you do not need to purchase a device to sharpen your skill, you can upload existing music and apply ML techniques. There are several AWS DeepComposer Chartbusters challenges where you can share your music generated by ML techniques and showcase your skills with other.

AWS Account

You will need an account to access AWS console and use ML services. It requires payment information but that is only for verification and it won’t deduct amount until you are using free tier and services that are free. AWS offers a free tier that allows you to use several services free of cost up to specified limits for each service. Details can be find on the AWS website.

DEV Community: Saima Javaid

What is Cloud Computing and what AWS offers?

Cloud Computing

Cloud Service Providers

Amazon Web Services (AWS)

Amazon Web Services (AWS)

Cloud Computing Services

Migration & Transfer

Storage

Database Services

Analytics

Deployment & Management Services

Internet of Things

Application Services

Developer Tools

Mobile Services

Artificial Intelligence

Final Words

Reference Links

Further Reading

Message Queues

What is Message Queue?

Why need Message queue?

Reliability

Decoupling

Scalability

Apache Kafka

AWS SQS

RabbitMQ

Final Words

Reference & Further Reading

Getting Started Machine Learning with AWS

What is Machine Learning?

How it works?

Why Machine Learning with AWS?

Amazon SageMaker

Getting Started

AWS DeepLens

AWS DeepRacer

AWS DeepComposer

AWS Account

Reference & Further Readings