DEV Community: Dorra ELBoukari

Container Engine Vs Container Runtime

Dorra ELBoukari — Sat, 16 Jul 2022 17:23:41 +0000

During the last few days, I have been working on comparing container engines. I wanted to study separately popular container engines in order to highlight the vulnerabilities related to each product. To make an unbiased judgement and to have a very clear perspective, I went through a myriad of articles that are published online. I remarked something strange. Even some well-experienced technical writers can be confused about the difference between a "Container Engine" and a "Container Runtime". I remarked that many use these two terms as synonyms, which is not the case.

Container Runtime

Let's bring it this way:
The container runtime can be considered the core component of a container engine. It is the beating heart that enables and initiates containerization. In other terms, without the container runtime, the container engine cannot communicate with the operating system and the containerization process will never be launched. Thus, the container will be never brought to life. The container runtime is a low-level element that handles all the tasks related to running the containerization process. It mounts the container and clones system calls to communicate with the kernel of the operating system on which you intend to run the containers. Cloning system calls mean creating new processes in a way similar to fork() system call ) that host the containerization mission.
We can specify two types of Container Runtimes:
CRI-Compliant Container Runtime:
are those who support CRI (Container Runtime Interface ) . CRI is the API that Kubernetes uses to manage container runtimes. How Kubernetes should communicate with a container runtime is outlined in the CRI API. Consequently, CRI is an interface that can be used with any supported runtime, whereas containerd and Cri-O are the specialized container runtime.

OCI-Compliant Container Runtime:
Are runtimes that obey the OCI standard. OCI is a framework for specifying how container images are organized.OCI images can be run on any container runtime that supports OCI since they have a standard format such as runC.

Container Engine

On the other side, container engines are software programs that handle user inputs, including those from the command line interfaces (CLI), fetches images, and executes the container. To fulfil some of its functionalities, a container engine uses container runtime. In other words, the architecture container engine contains a container runtime along with other elements for networking, orchestration capabilities , etc.
Some container runtime such as Containerd can be viewed as no more than low-level container engines with the most basic functionalities.

Illustration:

Here is a figure that illustrates how container engines work through a simplified example.

AWS DevOps Monitoring Dashboard | AWS Whitepaper Summary

Dorra ELBoukari — Mon, 20 Dec 2021 14:47:51 +0000

Unlike the other AWS Whitepapers for which I wrote summaries , this content builds on an AWS DevOps Monitoring dashboard Architecture Diagram published on August 09 ,2021. We will go through all the details and we will shed the light on several facts,so I can give you ,dear reader, a complete grasp of the architecture under discussion.

I. Architecture Description:

The suggested approach is built to set up a DevOps reporting tool on the AWS cloud infrastructure by using AWS native tools. It automates the process of ingesting, analyzing, and visualizing continuous integration/continuous delivery (CI/CD) metrics.

I.1 Use Case of the discussed architecture:

If you are engaged in building sophisticated applications on an AWS infrastructure using AWS native DevOps tools ,you must be going through lots of deployments .Thus, you need to consider a solution that helps you to visualize, track and analyze your deployments through their DevOps lifecycle .

I.2 AWS CI/CD pipeline

On the left side of the architecture , you can remark the box that presents Customer AWS CI/CD Pipeline . It is crystal clear that the customer has implemented a full Devops CI/CD pipeline with AWS dedicated DevOps Tools.

I.3 AWS DevOps Tools

AWS DevOps Tools comprise a collection of services that are designed to work together so we can safely store and manage our application's source code, as well as automatically create,test and deploy applications to AWS or on-premises environment. In other words AWS DevOps Tools work in harmony to cover the entire software development lifecycle starting from code reviews to deployment and monitoring.

1. AWS CodePipeline:

Is commonly viewed as the equivalent of Jenkins (a CI/CD tool that can integrate with many cloud providers) but Code Pipeline is specific to AWS.It is a fully managed, PAYGO(pay-as-you-go) , continuous delivery solution that automates your release pipeline's build, test, and deploy phases for a fast and dependable application.

2. AWS CodeCommit:

As it name implies ( From Cambridge Dictionary Commit (verb) :To actively put information in your memory or write it down) , Code Commit is a managed source control service that hosts private Git repositories where contributors write down their code instantly . It is designed to be safe , highly scalable and to enable teams to collaborate on code in a secure manner. The contributions are encrypted in transit and at rest. There is no need to worry about scalability as well as the management of the source control system .

3. AWS CodeBuild:

Is an AWS service designed to build your software packages that are ready to deploy. It is a fully managed and a scalable continuous integration service that compiles source code, runs tests, and then produces software packages. CodeBuild processes multiple builds concurrently, so your builds are not left waiting in a queue.

4. AWS CodeDeploy:

Is a fully managed and scalable deployment service used to automate software deployments while eliminating the need for error-prone manual operations. Deployment can be applied to a variety of AWS compute services (Amazon EC2, AWS Fargate, AWS Lambda,on-premises servers)

II. Walking Through The DevOps Monitoring Dashboard Architecture

In this paragraph ,we will use the same architecture provided by AWS along with the same reference numbers. But since it looks crowded,we will " divide to rule ". In other terms, we will break the architecture down into smaller sections for a clearer understanding and a better visualization .
To have a Monitoring dashboard in general, we need to go through a process that contain four main steps :

1. Tracking Event Sources

2. Gathering Data

3. Analyzing Gathered Information

4. Visualizing Analysis Results

Our architecture, obeys to the same philosophy of the Four-Step process . This is the reason why we will divide it into two main sections :

Section1: Tracking and Gathering Data (Figure a and Figure b )

Section2: Analyzing data and Visualizing Analysis Results ( Figure c )

The Figure below explains the process and mentions which AWS Services contribute to the success of the required task at each step.

PS: You can deploy this solution using the available AWS CloudFormation template (Infrastructure As A Code)

II.1 Section 1 : Tracking and Gathering Data

In this paragraph, we mainly focus on the sections responsible for information gathering.
While tracking an application deployed with AWS DevOps tools, we have two main parts to shed the light on:

1. CODE BUILD :
which announces if the build SUCCEEDED or FAILED. (This will be explained in SECTION 1.0).

2. DEPLOYMENT :

After having a successful code build, we have to know whether deployment is SUCCESSFUL or NOT. In case if failure ,we need to see the logs to figure out the cause of this issue.(This is detailed in SECTION 1.1)

Through those two major parts , we will need an appropriate AWS service that will alert us about success or failure events, and the data related to them.In fact, once a contributor (from the development team) initiates an activity in AWS CI/CD Pipeline,his deeds need to be detected so we can visualize it on our DevOps Monitoring Dashboard. The convenient service for this task is AWS CloudWatch.

What is AWS CloudWatch ?
It is a monitoring and observability AWS service . It is also a metric repository that will provide data and actionable insights in form of events . For more understanding here is an AWS CloudWatch use case:
You want to be notified with an SMS (perform an action) once the CPU Utilization of an instance exceeds 60% (Condition on metric is fullfilled). In this scenario , you need to use AWS CloudWatch to watch for the metric CPU utilization until it exceeds the threshold 60%. Once this happens ,it is called an event that will initiate an action .The event will trigger AWS SNS Service which alerts you immediately with an SMS.

AWS CloudWatch is used to:
-Continously Stream important Data, in real-time, through Amazon Kinesis Data Firehose to Amazon S3 buckets. (Discussed in Section 1.0)
-Send data ,in near real-time,to Amazon EventBridge, through Amazon Kinesis Data Firehose and finally to Amazon S3 buckets.(Discussed in Section 1.1)

II.1.0 Section 1.0

As we have already mentioned, AWS CloudWatch continuously Streams data events related to code source compilations, tests, and software packaging produced by AWS CodeBuild to Amazon Kinesis Data Firehose -View Step 1 and Step 2 in the Figure (a)-. The delivery is in near real-time and with low latency.
At this Point ,Amazon Kinesis Data Firehose will perform an ETL operation (Extract, Transform, Load) where a Lambda function undergoes the Extraction and the Transformation .
In fact, each time AWS CloudFormation detects data in real-time ,The Lambda function is activated for few seconds to extracts the relevant data for each metric and to transform it into the convenient format.
Finally, Amazon Kinesis Data Firehose loads the data in real-time to the Amazon S3 data lake for downstream processing. View Step 4 in the Figure (a).

II.1.1 Section 1.1

In this second portion of our architecture, we can simply see how to detect any action performed by the Developement Team on AWS CodeCommit, AWS CodeDeploy and AWS CodePipeline.View Figure (b).
Once a developer commits a code to AWS CodeCommit and deploys his application with AWS CodeDeploy,the actions on the predefined AWS Event Sources are detected by Amazon CloudWatch as events and then transfered to Amazon EventBridge (Step 1).
Amazon CloudWatch alarms also monitor the status of an Amazon CloudWatch synthetics canary service (Step 3). Canaries are customizable scripts that monitor endpoints and APIs on a regular basis. Even if you don't have any user activity on your applications, they actually conduct the same behaviors as a customer to assist you check your customer experience. You can detect problems before your consumers do by utilizing canaries.
PS: Step 2 and 4 are the same as the previous section.

What is Amazon EventBridge ?
It is a serverless event bus service that is used to capture events from an Amazon CloudWatch alarm .We can assume that Amazon EventBridge is the successor of AWS CloudWatch .In fact, it was formely called Amazon CloudWatch Events because it uses the same CloudWatch Events API . It is similar to Amazon CloudWatch but with additional features.

III.1 Section 2: Analyzing data and Visualizing Analysis Results

In the second paragraph , we will go the final steps in the architecture. At this point , we have all the information gathered in an Amazon S3 bucket ,in the appropriate format for analysis (thanks to AWS Lambda Function).Now ,we have two more steps to go: analysis and visualization.

1. Analysis:

A detailed examination of the gathered information is performed by AWS Athena. It is a serverless AWS tool that enables performing interactive queries and data analysis on the big sets of data stored in Amazon S3 while using standard SQL. Analysis results will be mostly delivered within few seconds.

2. Visualization:

To extract easy-to-understand insights, nothing is more adequat that AWS QuickSight. It provides an interactive and visual dashboard while ensuring scalability. In this perspective, AWS QuickSight is our window to deeper DevOps insights. Furthermore, thanks to Amazon QuickSight Q, Management team members can ask questions about DevOps data in natural language ( plain English) , and receive accurate responses with relevent visualizations which make insights clearer.
Aws QuickSight is an important asset for Management team members. In fact ,it describes DevOps insight in a very simple manner . Every one in management team should be able to understant the insights , even if he lacks data science and DevOps experience.

Serverless Architectures with AWS Lambda Summary | AWS Whitepaper Summary

Dorra ELBoukari — Mon, 15 Nov 2021 20:17:33 +0000

AWS Lambda is a serverless computing service launched in 2014 .It brought to existence a new architecture paradigm that doesn't rely on servers.AWS Lambda has also enabled a faster development speed and experimentation comparing to server-based architectures.
This paper summerizes the AWS Whitepaper entitled 'Serverless Architectures with AWS Lambda' released in 2017 to shed the light on Lambda serverless compute concepts .We focus mainly on the compute layer of the serverless applications where the code is executed, as well as the AWS developer tools and services used for best practices.

What Is Serverless?

Serverless literally means: "without need to provision or manage any servers". A serverless platform is responsible for:
1.Server Management:
Provisioning, installing and patching software,OS patching,etc.
2.Flexible scaling:
Applications are scaled automatically or by adjusting the capacity through the units of consumption(throughput,memory,etc)
3.High Availability:
Serverless applications have built-in availability and fault tolerance.
4.No idle capacity:
There is no charge when your code isn’t running

Here is a list of AWS different services that can used in serverless application:

AWS Lambda: Compute
Amazon API Gateway: APIs
S3 (Amazon Simple Storage Service): Storage
Amazon DynamoDB : Databases
Amazon SNS (Simple Notification Service) and Amazon SQS (Simple Queue Service): Interprocess messaging
AWS Step Functions and Amazon CloudWatch Events: Orchestration
Amazon Kinesis: Analytics

PS: Here is an additional list of AWS Serverless services that were not mentioned in the original document.

-AWS Redshift Spectrum: Analytics and interactive querying on S3
-Amazon Athena: interactive querying on S3 with read-on-schema technology
-Amazon Aurora Serverless: Databases
-AWS Fargate: Containerization
-Amazon QuickSight: Analytics and Visualization
-Amazon Cognito: Authentication ,authorization and user management
-AWS KMS: Key Management
-AWS Glue: ETL tool (Extract Transform and Load)
-Amazon EventBridge: Builds an event-driven architecture.
-Amazon AppSync: Create,publish and monitor secure GraphQL APIs and Subscriptions

AWS Lambda -the Basics

Lambda is a high-scale, provision-free serverless compute service . It is a (FaaS) Function-as-a-Service which scales precisely with the size of the workload .It runs more copies of the function in parallel in case of multiple simultaneous events to scale your code with high availability.AWS Lambda enables building reactive events as it works when triggered by an event .This reduces server's idle time and wasted capacity.
Each Lambda function contains:
-The Function Code that you want to execute
-The Function Configuration that defines how your code is executed.
-(Optional) The Event Sources that detect events and invoke you function as they occur. For example, an API Gateway(Event source) invokes a Lambda function when an API method created with API Gateway receives an HTTPS request.
You don’t need to write any code to integrate an event source with your Lambda function ,to manage infrastructure or scaling.Once you configure an event source for your function, your code is invoked when the event occurs.
Also ,it’s a natural fit to build microservices using Lambda functions thanks to the inherent decoupling that is enforced in serverless applications through integrating Lambda functions and event sources.

AWS Lambda- Diving Deeper

This section provides a further explanation of AWS Lambda's components mentioned above .

Lambda Function Code

It is the code you will run with AWS Lambda.AWS Lambda natively supports Java, Go, PowerShell, Node. js, C#, Python,PHP,SmallTAlk and Ruby code.It also supports libraries,artifacts and compiled native binaries.
AWS SAM Local: A set of tools used to compile and test the components you plan to run inside of Lambda within a matching environment .

The Function Code Package:

The Function Code Package contains all of the assets you want to have available locally upon execution of your code (additional files, classes, and libraries to be imported, binaries to be executed, or configuration files that your code might reference upon invocation.).
While creating the Lambda function (through the AWS Management Console or CreateFunction API) and even while publishing an updated code to existing Lambda functions (through UpdateFunctionCode API),you can upload the code package directly or you can refer to the S3 bucket and object key where the package is uploaded.
At minimum,the Function Code Package includes the code function to be executed when your function is invoked.

The Handler:

The handler is the code method (Java,C#) or the function (Node.js,Python) that is in your function code that processes events.It is the first thing that executes when a Lambda function is invoked.

The Event Object:

The Event Object is one of the parameters provided to the handler function.It includes all of the data and metadata that Lambda function needs.the event object differs in structure and contents, depending on which event source created it.

The Context Object:

The context object allows your function code to interact with the Lambda execution environment. It's contents and structure varies depending on the language runtime used by Lambda function.But at minimum it will contain:

AWS RequestId :Used to track specific invocations of a Lambda function(important for error reporting)
Remaining time :The amount of time in milliseconds that remain before your function timeout occurs (maximum 300 seconds)
Logging :The information about which CloudWatch Logs stream your log statements will be sent to.

Writing Code for AWS Lambda—Statelessness and Reuse

While writing your code for Lambda ,it’s important to understand that your code cannot make assumptions that state will be preserved from one invocation to the next.
However, each time a function container is created and invoked, it remains active and available for subsequent
invocations for at least a few minutes before it is terminated. We can define :
-Warm container:When subsequent
invocations occur on a container that has already been active and invoked at least once before
-Cold start:When an invocation occurs for a Lambda function that requires your function code package to be created and invoked for the first time

Fig(1):Invocations of warm function containers and cold function containers

Lambda Function Event Sources

Event sources are the triggers that invoke your code on AWS Lambda function through the Invoke API.You don't have to write, scale, or maintain any of the software that integrates the triggers with your Lambda function.

Invocation Patterns:

There are two models for invoking a Lambda function:
-Push Model: Lambda function is invoked every time a particular event occurs within another AWS service.
-Pull Model:Lambda polls a data source and invokes your function with it.
A a Lambda function can be executed synchronously or asynchronously through the InvocationType parameter . This parameter has three possible values. RequestResponse to execute Synchronously, Event to execute Asynchronously and DryRun to test that the invocation is permitted for the caller, but don’t
execute the function.

Lambda Function Configuration

This section is about the various configuration options that define how your code is executed within Lambda.

Function Memory

Function Memory Helps to define the resources allocated to your executing Lambda function by increasing/decreasing function resources (memory/RAM).You can optimize the price and performance of Lambda function by selecting the appropriate memory allocation.

Versions and Aliases :

Versioning is possible for AWS Lambda functions.Each and every Lambda function has a default version built in: $LATEST that addresses the most recent uploaded code. Each version has its own Amazon Resource Name (ARN).
PS: When calling the Invoke API or creating an event source for your Lambda function, you can also specify a specific version of the Lambda function .Otherwise, $LATEST is invoked by default.
Each Lambda function container is specific to a particular version of your function. A different set of containers is installed and managed for each function version.

Invoking your Lambda functions by their version numbers is useful during testing activities. However, this is not recommended for real application traffic. This this requires updating all of the triggers and clients invoking your Lambda function with Lambda alias to point at a new function version each time you wanted to update your code .Lambda aliases can be used to represent your Lambda function version (live/prod/active),to enable blue/green deployment pattern ,and for debuging when an alias is integrated with a testing stack for example

last paragraph page 15

IAM Role

Lambda Function Permissions

Network Configuration

Environment Variables

Dead Letter Queues

Timeout

Serverless Best Practices

Serverless Architecture Best Practices

Security Best Practices

Reliability Best Practices

Performance Efficiency Best Practices

Operational Excellence Best Practices

Cost Optimization Best Practices

Serverless Development Best Practices

Infrastructure as Code – the AWS Serverless Application Model (AWS SAM)

Local Testing – AWS SAM Local

Coding and Code Management Best Practices

Testing

Continuous Delivery

Sample Serverless Architectures

Conclusion

AWS Redshift (Part 1)

Dorra ELBoukari — Fri, 12 Nov 2021 15:03:25 +0000

As an AWS solutions architect, you must set up a solution that helps the data analysts in your company to process large historical data for some released products. The data scientists and the developers suggest collecting all the results of the queries for additional analytics with Amazon EMR, Athena and SageMaker. What AWS solution can you use in this context?

To answer this question, you need first to know what type of database you are dealing with.

Generally, we can classify databases into two groups, according to the approach that they use, which affects the type of data we want to extract eventually:

1. On-Line Transactional Processing databases(OLTP) :

Like RDS, it has a high transaction volume of simple and short queries. OLTP databases rely on four main operations: Create, Read, Update and Delete.

For example, with RDS you can CREATE a table containing products and their corresponding prices, you can READ the content of the table, UPDATE the names or the prices of the products and DELETE a product that you will no longer sell for the customers.

2. On-Line Analytical Processing Databases(OLAP):

It has a relatively low transaction volume of sophisticated and long queries that urge aggregations. OLAP DBs are used mainly for analytics.

Through the previous definitions, it became obvious that an OLAP is required in our context. An example of an OLAP database on AWS is Redshift.

Redshift is fully managed by AWS. It is a petabyte-scale data warehouse service.

Unlike RDS and many other OLTP databases which use rows, Redshift uses columns to store data. It also uses advanced compression and Massive parallel processing of data . This makes it ten times faster than SQL databases.

Redshift helps to report visualize and analyze collected data. You can save the results of your queries to an S3 data lake so you can do additional analytics with services provided by AWS like Athena and SageMaker.

Although Redshift is fully managed by AWS, it is set up ONLY in ONE availability zone and can’t take large data ingestion in real-time.

Amazon EKS Distro (EKS-D)

Dorra ELBoukari — Fri, 12 Nov 2021 14:58:30 +0000

Since June 2018, AWS has provided Amazon Elastic Kubernetes Service (EKS) to its customers. It is an upstream and certified conformant version of Kubernetes. This service helped to manage containerized workloads and services in the AWS Cloud and in on-premises. Amazon EKS have always guaranteed scalability, reliability, performance and high availability.

This service was satisfying for many users as they have enjoyed applying it effeciently to their projects.

On the 1st of December 2020, AWS announced their new service Amazon EKS Distro (EKS-D) to the audience interested in Kubernetes, the portable, extensible and open-source platform of orchestration. As everyone was curious about this concept, a myriad of questions immerged: What is EKS-D? Why Amazon created this product? What is the advantage of EKS-D?

To answer those questions, we have to explain first the meaning of "Kubernetes distribution" to avoid any confusion.

What is a Kubernetes Distribution?

The Cloud Native Computing Foundation (CNCF) defined this term a long time ago, as the pieces that an end-user needs to install and run Kubernetes on the public cloud or on the on-premises. Here is a spreadsheet that details Kubernetes Distributions and Platforms: https://docs.google.com/spreadsheets/d/1LxSqBzjOxfGx3cmtZ4EbB_BGCxT_wlxW_xgHVVa23es/edit#gid=0

What is EKS-D?

EKS-D is a Kubernetes distribution that relies basically on Amazon EKS and holds the same benefits that his 'ancestor' has. At this point, I found it useful to use the word 'Ancestor' because it is crystal clear that EKS-D is just an evolution and exploitation of the Amazon EKS service. But it is more sophisticated since it creates reliable and secure clusters to host Kubernetes.

Why EKS Distro?

Amazon EKS is convenient for many users, but not all users can take advantage of it. To explain that, you have to consider the Amazon EKS responsibility model in the figure below.

AWS wants to simplify Kubernetes managing for the customers who may not find the right approach to leverage their applications. Customers must spend a minimal duration on operating Kubernetes. Instead, they need to focus on their business. This is the reason why Amazon EKS takes responsibility for Tactical Operations. This sounds great, but in fact, it deprives a considerable number of customers of using Amazon EKS.

Some users need for example to apply their custom tools on the control plane as their applications require a customization of the control plane flags. Another category of customers may have specific security patches to apply according to their compliance. Others have a wide variety of computing requirements (Hardware, CPU, environment, etc.)

Those considerable requirements urged the appearance of EKS Distro. It aims to help users get consistent Kubernetes builds and have a more reliable and secure distribution for an extended number of versions. Customers can now run Kubernetes on your own self-provisioned hardware infrastructure, on bare-metal or on cloud environment.

For more details about the subject , visit:
https://aws.amazon.com/eks/eks-distro/

Exploring OpenStack through MicroStack : Installing MicroStack

Dorra ELBoukari — Tue, 19 Oct 2021 21:28:40 +0000

Summary

1.What is OpenStack?
1.1 About OpenStack versions
1.2 What is an NFV?

What is MicroStack? Why MicroStack ?
2.1 Why MicroStack for our LAB?
2.2 What MicroStack requires
2.3 Why MicroStack is agile
Installation in Action
3.1.Virtual Machine Characteristics
3.1.1.RAM requirements
3.1.2.Hard disk requirements
3.1.3.Processing requirements
3.2 Installing OS
3.3 Installing MicroStack

1.What is OpenStack?

OpenStack is a cloud framework that provides an agile scale-out infrastructure. It can power standard cloud services like compute ,network, storage resource provisioning and self-service automation.
It is an open source project that helps to build a private or public cloud. It was deployed by thousands in the open source community as a set of software components that provide services for the cloud infrastructure. It is built entirely on open industry standards and APIs. This is why it is highly adaptable. No more worries about vendor lock-in.

1.1. About OpenStack versions:

While writing this briefing, the vast majority of the customers are still using or migrating to version 13. This is because the newest versions of OpenStack do not support all the Network Functions Virtualization features that are already by the older versions (10 and 13)

1.2 What is an NFV?
Network functions virtualization (NFV) with which the virtualization of network devices happens.
Networking devices (such as switches, routers,firewalls ) run ordinarily on hardware devices. But through NVF, you can make the services provided by network devices packaged as virtual machines .Network can be run on standard servers instead of physical hardware devices. This improves the scalability and the agility by allowing service providers to shape their network easily with no need for additional resources .

What is MicroStack? Why MicroStack ?

As it name implies MicroStack is OpenStack for micro clouds. It is a single or multi-node OpenStack deployment . It has small-scale cost-efficient private cloud infrastructure.
“MicroStack enables enterprises to quickly deploy cost-efficient private cloud infrastructure from single-node installations to micro cloud clusters.” [1]
A single node deployment means that it can run on one workstation .The core services are included and are located on a single node.

2.1 Why MicroStack for our LAB?

As one node OpenStack deployment, MicroStack is a good tool to use to deploy OpenStack at a small scale.
MicroStack has a straightforward installation Link : https://microstack.run/?_ga=2.26389432.404340272.1634561755-1799874885.1632764872&_gac=1.250131700.1634069838.Cj0KCQjw5JSLBhCxARIsAHgO2SepSJ03AZTrqTGHC28cEQYEOFOUCa1e7VM_83hB-0T4rUZqfmZgFOoaArN-EALw_wcB

2.2 What MicroStack requires:

A multi-core CPU
8 GB RAM
100 GB storage

2.3 Why MicroStack is agile:
It does NOT require so much resources and it can be installed on :

Workstations
Edge devices
Build clusters.

Installation in Action

As it was mentioned in paragraph 2.2 the requirement of MicroStack in terms of memory ,processing and storage, we apply those on our virtual machine .
3.1 Virtual Machine Characteristics
3.1.1 RAM requirements

3.1.2 Hard disk requirements

3.1.3 Processing requirements

3.2 Installing OS
The OS used is Ubuntu 64-bit release in 04.2021

3.3*Installing MicroStack*
The following command

Gives:

To see the information about the installed snap write:

This gives:

Microstack Ussuri is installed successfully

3.4*Initializing MicroStack*

3.5*Interacting with MicroStack*

In this step , we generate the password of the user admin with through which we will interact with our single node OpenStack.

*Interact with your cloud via the web UI visit http://10.20.20.1/

Your private cloud is ready to manage in terms of Compute,Volume , Network (IaaS)

Bibliography:

[1] Last seen at 05:06 18,October 2021
https://microstack.run/?_ga=2.26389432.404340272.1634561755-1799874885.1632764872&_gac=1.250131700.1634069838.Cj0KCQjw5JSLBhCxARIsAHgO2SepSJ03AZTrqTGHC28cEQYEOFOUCa1e7VM_83hB-0T4rUZqfmZgFOoaArN-EALw_wcB

[2] Last seen at 19:40 18,October 2021
https://ubuntu.com/openstack/install/

Amazon EKS Distro (EKS-D)

Dorra ELBoukari — Sat, 04 Sep 2021 14:24:00 +0000

This service was satisfying for many users as they have enjoyed applying it effeciently to their projects.

To answer those questions, we have to explain first the meaning of "Kubernetes distribution" to avoid any confusion.

What is a Kubernetes Distribution?

What is EKS-D?

Why EKS Distro?

Amazon EKS is convenient for many users, but not all users can take advantage of it. To explain that, you have to consider the Amazon EKS responsibility model in the figure below.

AWS wants to simplify Kubernetes managing for the customers who may not find the right approach to leverage their applications. Customers must spend a minimal duration on operating Kubernetes. Instead, they need to focus on their business. This is the reason why Amazon EKS takes responsibility for Tactical Operations. This sounds great, but in fact, it deprives a considerable number of customers of using Amazon EKS.

For more details about the subject , visit: https://aws.amazon.com/eks/eks-distro/

Automated Archival for Amazon Redshift | AWS White Paper Summary

Dorra ELBoukari — Sun, 08 Aug 2021 16:22:24 +0000

Since its appearance, AWS provided a variety of database services to help users manage their data according to their needs. In AWS, you can run OLAP DBs as well as OLTP DBs .
The paper provides a further explanation of the whitepaper entitled “Automated Archival for Amazon Redshift” published In July 2021.It will shed the light on AWS Redshift service and its specifications especially the automated archival.

OLAP databases VS OLTP databases

Those two types of databases each rely on a different processing system. According to the type of information that you want to extract from your database, you generally select one of those two categories. In fact:

• OLAP( OnLine Analytical Processing) : Used for business intelligence and any other operation that require complex analytical calculations .The data is mostly coming from a data warehouse. This process is ideal for reporting and analyzing historical data. Many businesses rely on this type of database to have a clear visualization about their budgeting, sales forecasting and to track the success rate of released products.

• OLTP(OnLine Transactional Processing) : Used for a high volume of simple transactions and short queries. It relies mainly on four operations that can be performed on the databases (CRUD: CREATE, READ, UPDATE, DELETE). Businesses rely on this category to get detailed and current data from organized tables.

What is a Data Warehouse?

A Data Warehouse is a concept that refers to any aggregation of considerable amounts of data from different sources for the sake of analytics. Those resources could be internal (in your own network like marketing) or external (like customer categories, system partners,etc) .It helps to centralize historical data into one consistent repository of information. The Data warehouse concept aims to run analytics on hundreds of gigabytes to petabytes of data.

OLAP usage in Data Warehouse

OLAP is a system dedicated to perform multi-dimensional analysis on considerable volumes of data. It is ideal for querying and analyzing data warehouses. OLAP undergoes complex queries on data from different perspectives.

What AWS Redshift?

Amazon Redshift is a data warehouse service provided by AWS. It is fully managed and it can analyze up to a petabyte of data or more. It enables AWS users to run complex queries that involve aggregation on historical data rather than real-time data. Those analytics are crucial for business reporting and for visualization purposes . this helps managers to have clear insights into the evolution of the business .

Automated Archival for Amazon Redshift

In this paragraph, we discuss the architecture illustrated in Figure (a). This architecture automates the periodic data archival process for an Amazon Redshift database. We will go through each step and explain the ambiguous ones.

1) Data is ingested into the Amazon Redshift cluster at various frequencies:
The ingestion of data is literally the transportation of data from assorted sources like Simple Storage Service S3,Copy command (EMR, DynamoDB), database migration service or data pipeline to the data warehouse like Amazon Redshift cluster. Each given dataset has its own ‘Frequency Of Ingestion’ which defines how often we ingest it.

2.a) After every ingestion load, the process creates a queue of metadata about tables populated into Amazon Redshift tables in Amazon RDS:
In order to have a clear visualization of the data stored in the data warehouse, the process creates a queue of metadata for every ingested data and store it in an Amazon RDS (Relational Database Service).This archived metadata contains various information about the tables populated into Amazon Redshift such as Table Name, Cluster, Region , Processed Date, Archival Date, etc.

2.b) Data Engineers may also create the archival queue manually, if needed.

3) Using Amazon EventBridge, an AWS Lambda function is triggered periodically to read the queue from the RDS table and create an Amazon SQS message for every period due for archival. The user may choose a single SQS queue or an SQS queue per schema based on the volume of tables.
Amazon EventBridge is a serverless event bus to build event-driven applications. Here EventBridge is used to initiate an AWS Lambda function Daily,Weekly or Monthly to get archival tables from the Amazon RDS mentioned in step(1)
4) A proxy Lambda function de-queues the Amazon SQS messages and for every message invokes AWS Step Functions.
The proxy Lambda function links each Amazon SQS message to the corresponding AWS Step Functions.
AWS Step Functions is a low-code visual workflow service used to orchestrate AWS services. It manages failures, retries, parallelization, service integrations, and observability .
5) AWS Step Functions unloads the data from the Amazon Redshift cluster into an Amazon S3 bucket for the given table and period.
6) Amazon S3 Lifecycle configuration moves data in buckets from S3 Standard storage class to S3 Glacier storage class after 90 days.
The definition of the S3 lifecycle relies on the S3 retention policy. The transition action happens every 90 days to Amazon S3 Glacier (which is the minimum storage duration charge of S3 Glacier).S3 Glacier is a deep , long-term and durable archive storage that will conserve the data until deleted, while S3 is not a long term storage device.
7) Amazon S3 inventory tool generates manifest files from the Amazon S3 bucket dedicated for cold data on daily basis and stores them in an S3 bucket for manifest files.
S3 inventory is a tool provided by AWS to help manage the simple storage service. In this context, S3 inventory lies between the standard S3 storage and the S3 Glacier. It keeps track of the cool data archived in S3 Glacier. It generates manifest files that list all the file inventory lists that are stored in the S3 Glacier every 90 days when the transition takes action.

8) Every time an inventory manifest file is created in a manifest S3 bucket, an AWS Lambda function is triggered through an Amazon S3 Event Notification.
The manifest files created by the S3 inventory tool are stored in Amazon S3 . The acquisition of the data on the S3 bucket produces an S3 event notification which will initiate an AWS Lambda function.
9) A Lambda function normalizes the manifest file for easy consumption in the event of restore.
The triggered Lambda Function starts to normalize the inventory data.
10) The data stored in the S3 bucket used for cold data can be queried using Amazon Redshift Spectrum.
Amazon Redshift Spectrum will be used to query data directly from files on the Amazon S3 where the normalized manifests are stored.

Reference :

Automated Archival for Amazon Redshift Whitepaper published On July ,2021 : https://d1.awsstatic.com/architecture-diagrams/ArchitectureDiagrams/automated-archival-for-amazon-redshift-ra.pdf?did=wp_card&trk=wp_card

Architecting Amazon EKS for PCI DSS Compliance Summary | AWS Whitepaper Summary

Dorra ELBoukari — Mon, 19 Jul 2021 20:55:42 +0000

It was in 2013 when I first heard about PCI DSS compliance after the consecutive and massive credit-card data breaches that happened in the US. I was 16 years old and I was excited to know how the breach happened and what does ‘PCI DSS Compliance’ even mean .Today, with a clearer visual on AWS provided technologies on this subject, I selected this whitepaper to enlighten every curious person about the data security standard for payment cards. This paper, provided by two senior solution architects Arindam Chatterji and Tim Sills, outlines the best practices to configure Amazon Elastic Kubernetes services for AWS Fargate or Amazon Elastic Compute Cloud (Amazon EC2) launch types for Payment Card Industry Data Security Standard (PCI DSS) .It also provides various solutions to mitigate security risks while using the provided AWS services.
This document is dedicated to persons who are involved in projects where AWS is applied for PCI DSS compliance.

What is Payment Card Industry Data Security Standard (PCI DSS)?

• Provides technical and operational guidance on securing payment card processing environments
• Entities that store, process, or transmit cardholder data (CHD) must be PCI DSS certified ,so they have proven that the followed policies ,procedures, guidelines and best practices to build cardholder data environment (CDE)

AWS for PCI DSS Compliance

• AWS provided many services that meet PCI DSS Compliance
• AWS Artifact: a central resource for compliance-related information. It can be accessed by companies, on-demand, to reduce compliance efforts .The services provided are containerized by AWS. So companies take advantage of platform independence, deployment speed and resource efficiency.
PS: A service listed as PCI DSS compliant doesn’t mean that it makes a customer’s compliant by default.

PCI DSS compliance status of AWS Services
• AWS is a Level 1 PCI DSS Service Provider: AWS customers meet easily compliance requirements.

• Any data provided by the customer has :

Primary Account Numbers (PAN)
Sensitive Authentication Data (SAD)

•The annually updated PCI DSS assessment includes physical security requirements for AWS datacenters.
AWS Shared Responsibility model
Security and compliance responsibilities and shared between AWS and the customer.

• AWS : Security , management, control of AWS cloud infrastructure (Hardware ,software, networking and facilities)

• Customer: Security of all the systems components and services provisioned on AWS (included in or connected to the customer’s CDE) like access control ,log settings, encryption , etc

PS: The division of responsibilities depends on the AWS service selected by the customer. For example:

PCI DSS scope determination and validation
The cardholder data flow determines:
• Applicability of PCI DSS
• Scope of PCI DSS; boundaries and components of CDE
-The customer must have a procedure for PCI DSS scope determination to assure its completeness and to detect changes and violations of the scope.
-Steps that comprise the PCI DSS scope identification are:

PS: Customers need to be aware of container configuration parameters through all the phases of a container lifecycle to ensure the satisfaction of the compliance requirements.

Securing an Amazon EKS Deployment

While architecting a container-based environment for PCI DSS compliance, you have to follow the best practices recommendations for those key topics:
• Network segmentation
• Host and container image hardening
• Data protection
• Restricting user access
• Event logging
• Vulnerability scanning and penetration testing

Network Segmentation (Requirement N°1):
PCI DSS doesn’t require network segmentation, but it helps to reduce the scope of the customer’s environment.
• VPC, subnets and security groups provide logical isolation of CDE-related resources.

To enforce your VPC’s network policy you can use Calico is an open-source policy engine from Tigera.It works well with Amazon EKS supports extended network policies and can be integrated with service mesh .
• Security groups act as a virtual firewall and provide stateful inspection, they restrict communications by IP address, port, and protocol.They are used by Amazon EKS to control the traffic between the Kubernetes control plane and the cluster's worker nodes.
PS: It is strongly recommended that you use a dedicated security group for each control plane (one for each cluster).
• Individual AWS accounts for PCI DSS provide the highest level of segmentation boundaries on the AWS platform. Their resources are logically isolated from other accounts.

• To isolate containerized application communications ,you need to :

1) Isolate pods on separate nodes based on the sensitivity of services and isolate CDE workloads in a separate cluster with a dedicated Security group.

2) Use AWS security groups to limit communication between nodes and control plane and external communications.

3) Implement micro-segmentation with Kubernetes network policies and consider the usage of the service mesh, Networking and Cryptography library (NaCI) encryption and Container Network Interfaces (CNIs) to limit and secure communications.

4) Implement a network segmentation and tenant isolation network policy.

Host and image hardening (Requirement N°2)
Host and image hardening help to minimize attack vectors by disabling support for vendors for security parameters.In which:
• Customers should create trusted base container images that have been assessed and confirmed to use patched libraries and applications. Use a trusted registry to secure container images, such as Amazon Elastic Container Registry (Amazon ECR). Amazon ECR provides image scanning based upon the Common Vulnerabilities and Exposures (CVEs) database and can identify common software vulnerabilities.
• Container optimized Amazon Machine Image (AMI) contains only essential libraries for deployments. Non-essential services and libraries should be disabled or removed.
• Container builds should be limited and should adopt a model of microservices where a container provides one primary function.
• It is recommended to use special-purpose operating systems (OS) like Bottlerocket that includes a reduced attack surface , a disk image that is verified on boot, and enforced permission boundaries using SELinux.
• Establish configuration standards under the shadow of the industry-accepted system hardening guidelines.
Data protection (Requirements N°3 and 4)
This is about the PCI DSS requirement to protect sensitive data in rest and in transit. PCI DSS compliant services and features to assist with these compliance efforts.

Protect the data in rest:

Secure all the sensitive stored data of PCI DSS workloads on secure stores or databases NOT on the container host.
• Consider the use of AWS Key Management Service (KMS) to secure encryption key storage, access controls and annual rotation.
• Use AWS Secrets Manager and AWS Systems Manager Parameter Store to secure sensitive data within container build files.

Protect the data in transit:

PCI DSS urges the encryption of sensitive data during transmission over open, public networks. Customers are responsible for configuring strong cryptography and security controls.
• Consider a variety of AWS services like Amazon API Gateway and Application Load Balancer
• Encryption in transit for inter-pod communication can also be implemented with a service mesh like AWS App Mesh with support for mTLS.
• Use envelope encryption of Kubernetes secrets in EKS to add a customer-managed layer of encryption for application secrets or user data that is stored within a Kubernetes cluster.

Other protection measures:

• Restrict access to authorized personnel(Requirement N°7 and 8), grant least privileges and authenticate with strong authentication requirements that align with the PCI DSS.
• Run containers with non-privileged user accounts and restrict all access to container images.
• Consider disabling the use of the secure shell (SSH) and instead leverage AWS Systems Manager’s Run Command
• Urge that users sign into the Amazon EKS cluster with an IAM identity(either an IAM user or IAM role)
• Create the cluster with a dedicated IAM role which should be regularly audited.
• Make the Amazon EKS Cluster endpoint private.

Tracking and monitoring access (Requirement N°10)
This is about the use of event logs to track suspicious activities and even anticipate possible threats. So:
• EKS Cluster audit logs need to be enabled as well as VPC Flow Logs, Amazon CloudWatch and Amazon Kinesis
• CloudWatch dashboard should be configured to monitor and alert on all captured event log activity
• Captured event data have to be stored securely within encrypted Amazon S3 buckets to be analyzed with Amazon Athena and Amazon CloudWatch Logs Insights.
• Amazon GuardDuty provides threat detection .
Network Intrusion detection(Requirement N°11)
• Monitoring of all traffic at the perimeter and critical points of the CDE
• Use network inspection options outside of the container host on AWS like :
Amazon GuardDuty: a managed service that provides threat detection across multiple AWS data sources to identify threats.
Amazon VPC Traffic Mirroring: traditional IDS/IPS solution.
Virtual IDS/IPS device from the AWS Marketplace: helps to inspect in transit traffic.You can use a VPC Gateway to route all traffic to on-premises IDS/IPS infrastructure.
Vulnerability scanning and penetration testing(Requirement N°11.2)
It aims to test systems and processes regularly to identify and fix vulnerabilities.
• Penetration testing is to be performed on an annual basis and after any significant environmental changes.
• Penetration testing of AWS resources is allowed at any time for certain permitted services in the perimeter of the penetration testing policy .
• PCI DSS Compliance provides guidance and methodologies to perform penetration testing .It depends on customer’s environment.
• When deploying Amazon EKS on Amazon EC2 instances, customers must perform vulnerability scanning of the underlying host.
• Amazon Inspector is a security assessment tool that helps identify vulnerabilities and prioritizes findings by level of severity.
• The Center for Internet Security (CIS) Kubernetes Benchmark provides guidance for Amazon EKS node security configurations.

Conclusion

AWS provides a convenient infrastructure for customers to address PCI DSS requirements for their containerized workloads. Various security measures are ready to use in order to reduce management complexities for the users.

Docker On AWS | AWS Whitepaper Summary

Dorra ELBoukari — Sun, 11 Jul 2021 15:54:16 +0000

This content is the summary of the AWS whitepaper entitled “ Docker on AWS “ written by Brandon Chavis and Thomas Jones. It discusses the exploitation of the container’s benefits in AWS. I tried to simplify and to gather the most interesting points from each paragraph, in order to give the readers very brief and effective content.

PS: Although the introduction is always ignored by many readers, I found that the authors provided an excellent set of information as an opening to our subject. This is why I found it fruitful o summarize the introduction as well by an explanative figure.

I. Container Benefits:

The benefits of containers reach all the elements of organizations and they are:
Speed: Helps all the contributors to software development activities to act quickly.
Because:

The architecture of containers allows for full process isolation by using the Linux kernel namespaces and cgroups. Containers are independent and share kernel on host OS(No need for full virtualization or for hypervisor)
Containers can be created quickly thanks to their modular nature and lightweight. This becomes more observable in development lifecycle. The granularity leads an easy versioning of released applications .Also it leads to a reduction in resource sharing between application components which minimizes compatibility issues.

Consistency: The ability to relocate entire development environments by moving a container between systems highlights.
Containers provide predictable, consistent and stable applications in all the stages of their lifecycle (Development, test and production) as it encapsulates the exact dependencies, thus minimizes the risk of bugs.

Density and Resource Efficiency: The enormous support of the community to the Docker project increased density and modularity of computing resources.
• Containers increase the efficiency and the agility of applications thanks to the abstraction from OS and hardware. Multiple containers run on a single system.
• You can make a compromise between what resources containers need and what are the hardware limits of the host to reach a maximum number of containers: Higher density, increasing efficiency of computing resources, saving money of the excessed capacity, changing the number of assigned containers to a host instead of horizontal scaling to reach optimal utilization.

Flexibility: Based on Docker portability, ease of deployment, and small size.

• Unlike other applications that require intensive instructions, Docker provides (just like Git) a simple mechanism to download and install containers and their subsequent applications using this command:
$ docker pull
• Docker provides a standard interface : It is easy to deploy
wherever you like and it’s portable between different versions of Linux.
• Containers make microservice architecture possible where services are isolated to adjacent service’s failure and errant patches or upgrades.
• Docker provides clean ,reproducible and modular environment

II. Containers in AWS:

There are two ways to deploy containers in AWS :
AWS Elastic Beanstalk: It is a management layer for AWS services like Amazon EC2, Amazon RDS and ELB.

• It is used to deploy, manage and scale containerized applications
• It can deploy containerized applications to Amazon ECS
• After you specify your requirements (memory, CPU, ports, etc.),it places your containers across your cluster and monitors their health.
• The command-line utility eb can be used to manage AWS Elastic Beanstalk and Docker containers.
• It is used for deploying a limited number of containers
Amazon EC2 Container Service

• Amazon ECS is a high performant management system for Docker containers on AWS .
• It helps to launch, manage, run distributed applications and orchestrate thousands of Linux containers on a managed cluster of EC2 instances, without having to build your own cluster management backend.
• It offers multiple ways to manage container scheduling, supporting various applications.
• Amazon ECS container agent is open source and free ,it can be built into any AMI to be used with Amazon ECS
• On a cluster , a task definition is required to define each Docker image ( name ,location, allocated resources,etc.).
• The minimum unit of work in Amazon ECS is ‘a task’ which is a running instance of a task definition.

About the clusters in this context:

• Clusters are EC2 instances running the ECS container agent that communicates instance and container state information to the cluster manager and dockerd .
• Instances register with the default or specified cluster.
• A cluster has an Auto Scaling group to satisfy the needs of the container workloads.
• Amazon ECS allows managing a large cluster of instances and containers programmatically.

Container-Enabled AMIs: The Amazon ECS-Optimized Amazon Linux AMI includes the Amazon ECS container agent (running inside a Docker container), dockerd (the Docker daemon), and removes the not required packages.

Container Management:
Amazon ECS provides optimal control and visibility over containers, clusters, and applications with a simple, detailed API. You just need to call the relevant actions to carry out your management tasks.
Here is a list containing examples of available API Operations for Amazon ECS.

Scheduling

• Scheduling ensures that an appropriate number of tasks are constantly running , that tasks are registered against one or more load balancers, and they are rescheduled when a task fails .
• Amazon ECS API actions like StartTask can make appropriate placement decisions based on specific parameters (StartTask decisions are based on business and application requirements).
• Amazon ECS allows the integration with custom or third-party schedulers.
• Amazon ECS includes two built-in schedulers:

The RunTask: randomly distributes tasks across your cluster.
CreateService: ideally suited to long-running stateless services.

Container Repositories

• Amazon ECS is repository-agnostic so customers can use the repositories of their choice.
• Amazon ECS can integrate with private Docker repositories running in AWS or an on-premises data center.

Logging and Monitoring

Amazon ECS supports monitoring of cluster contents with Amazon CloudWatch.

Storage

• Amazon ECS allows to store and share information between multiple containers using data volumes. They can be shared on a host as:
•• Empty, non-persistent scratch space for containers

•• Exported volume from one container to be mounted by other containers on mountpoints called containerPaths.

• ECS task definitions can refer to storage locations (instance storage or EBS volumes) on the host as data volumes. The optional parameter referencing a directory on the underlying host is called sourcePath.If it is not provided, data volume is treated as scratch space.
• volumesFrom parameter : defines the relationship of storage between two containers .It requires sourceContainer argument to specify which container's data volume should be mounted.

Networking

• Amazon ECS allows networking features (port mapping ,container linking ,security groups ,IP addresses and ressources, network interfaces, etc.).

III. Container Security

• AWS Costumers combine software capabalities (of Docker, SElinux, iptables,etc) with AWS security measures( IAM, security groups, NACL,VPC) provided in AWS architecture for EC2 and scaled by clusters
• AWS customers maintain, control and configure of the EC2 instances, OS and Docker daemon through AWS deployment &management services.
• Security measures are scaled through clusters.

IV. Container Use Cases

1.Batch jobs

Packaging containers that can batch, extract, transform, and load jobs and deploy them into clusters. Jobs then start quickly. Better performance is witnessed.

2.Distributed Applications

Containers build:

• Distributed applications, which provide loose coupling ,elastic and scalable design. They are quick to deploy across heterogeneous servers ,as they are characterized by density ,consistency and flexibility.

• microservices into adequate encapsulation units.

• Batch job processes which can run on a large numbers of containers.

3.Continuous Integration and Deployment

Containers are a keystone component of continuous integration (CI) and continuous deployment (CD) workflows. It supports streamlined build, test, and deployment from the same container images. As it leverages CI features in tools like GitHub, Jenkins, and DockerHub

4.Platform As a Service

PaaS is a type of service model that presents a set of software, tools and and an underlying infrastructure where the cloud provider manages networking, storage ,OS ,Middleware and the customer performs resources configuration.

The issue : Users and their resources need to be isolated .This is a challenging task for PaaS providers .

The solution: Containers provide the needed isolation concept .they allows also creating and deploying template resources to simplify isolation process.
Also each product offered by the PaaS provider could be built into its own container and deployed on demand quickly.

V. Architectural Considerations

All the containers defined in a task are placed onto a single instance in the cluster. So a task represents an application with multiple tiers requiring inter-container communication.
Tasks give users the ability to allocate resources to containers, so containers can be evaluated on resource requirements and collocated.
Amazon ECS provides three API actions for placing containers onto hosts:
RunTask : allows a specific cluster instance to be passed as a value in the API call
StartTask: uses Amazon ECS scheduler logic to place a task on an open host
CreateService: allows for the creation of a Service object, which, combination of a TaskDefinition object and an existing Elastic Load Balancing load.

Service discovery: Solves challenges with advertising internal container state, such as current IP address and application status, to other containers running on separate hosts within the cluster. The Amazon ECS describe API actions like describe-service can serve as primitives for service discovery functionality.

VI. Walkthrough

Since the commands used in this Walkthrough can be exploited in other complex projects, I suggest a bash file that can help to solve repetitive and difficult real-world problems:
View: link

1.Create your first cluster named ‘Walkthrough’ with create-cluster command
PS: each AWS account is limited to two clusters.
2.Add instances
If you would like to control which cluster the instances register to(not to default cluster), you need to input UserData to populate the cluster name into the /etc/ecs/ecs.config file.
In this lab, we will launch a web server ,so we configure the correct security group permissions and allow inbound access from anywhere on port 80.

3.Run a quick check with the list-container-instances command:
PS: To dig into the instances more, use the describe-container-instances command

4.Register a task definition before running in on ECS cluster:

a)Create the task definition:
It is created a the JSON file called ‘nginx_task.json’ . This specific task launches a pre-configured NGINX container from the Docker Hub repository.

View: link

b)Register the task definition with Amazon ECS:

5.Run the Task with run-task command:
PS:
• Note of the taskDefinition instance value (Walkthrough:1) returned after task registration in the previous step.
• To obtain the ARN use the aws ecs list-task-definitions command.

6.Test the container: The container port is mapped to the instance port 80, so you can curl the utility to test the public IP address .

Conclusion

This whitepaper summary can be a useful resource for those interested in the cloud native technologies. It sheds the light on containers generally, and Docker on AWS specifically. It details the benefits of those technologies especially while using the EC2 cluster, it gives a step-by-step guide for beginners to deploy their first container on a cluster and also provides a bash Script that helps to automate those tasks in more complex projects.