DEV Community: abhick09

CODEDEPLOY ON AWS PART 1

abhick09 — Fri, 29 Jan 2021 11:41:40 +0000

Setup IAM User with access to resources

What are IAM Users and how to create one?

IAM Users are AWS Users who can access,provision,monitor various AWS Manged or any services provided within the AWS Architecture.There can be fine grain control over the IAM users where users can have only READ access to multiple services or full CRUD control over there assigned services.

We can simply create a IAM User from the console but there are three ways where we can create a IAM user :

AWS CLI : Has access key which is a combination of (access key ID and a secret access key)
AWS API : AWS API type of IAM user also interacts with the AWS services VIA CLI.
AWS Console : Has console password and username which will grant access to the AWS Management Console.

What are the types of IAM users?

The types of IAM Users are Users with :

Programmatic access : The IAM user might need to make API calls, use the AWS CLI, or use the Tools for Windows PowerShell. In that case, create an access key (access key ID and a secret access key) for that user.
AWS Management Console access : If the user needs to access the AWS Management Console, create a password for the user.

What are the ways for IAM users to access/assign resources?

The ways for IAM users to access/assign resources are through directly policy applied to the user or group level policy applied to the users.

IAM User : A IAM user associated with an access key or console password which can be attached policies directly.
IAM Group : A IAM group is a collection of IAM Users with similar IAM Roles required to attain an objective from which multiple tasks can be handles by a big group of people or could be used for various stages of software engineering lifecycle (ie dev,prod,staging).
IAM Roles : IAM Roles are a collection of access where we can attach roles to users who can perform special actions as in deployment or any other.
Instance Profiles : The Instance profiles are profiles created by the IAM user to access and manage policies for EC2 Instance to operate and work properly with CODE DEPLOY.

How to assign the IAM user to use CODE DEPLOY?

The CODE DEPLOY and IAM user are connected via Instance Profiles with IAM Roles,IAM Groups.

Instance Profiles : The Instance profiles are profiles created by the IAM user to access and manage policies for EC2 Instance to operate and work properly with CODE DEPLOY.Instance Profiles are attached with IAM Roles which will grant the profile with access to resources only one IAM Role can be assigned to a Instance Profiles.Where as the same IAM Role can be used on multiple Instance Profile.

AWS S3 DEMYSTIFED

abhick09 — Mon, 14 Dec 2020 09:47:02 +0000

What is S3?

S3 is a object based where individual files can be of from 0bytes to 5TB.There is unlimited storage in S3 and are stored in buckets.Naming of the bucket should be unique globally as it is a universal namespace.S3 as if the data stored in s3 are successfully uploaded it return status code of 200.The objects in the S3 will have key value pair.Amazon S3 is a simple key-based object store. When you store data, you assign a unique object key that can later be used to retrieve the data. Keys can be any string, and they can be constructed to mimic hierarchical attributes. Alternatively, you can use S3 Object Tagging to organize your data across all of your S3 buckets and/or prefixes.

What are S3 objects?

In S3 all the files are stored as objects.The objects will have :
*Key : Value
*Version ID
*Metadata
*Subresources : Access Control Lists

But in S3 we have a condition where if you are writing a new file and you can read it immediately but if you are overwriting a file it might take sometime to propagate and get a new data this is also the case when you delete a file

What does are the benefits of S3?

*S3 guarantees 11*9 for durability for its service.
*It has tiered storage available with seven tires to choose from depending on your needs.
*It has life cycle management you can also set lifecycle expiration policies to automatically remove objects based on the age of the object.
*It has versioning.When analyzing the storage costs of the operations, note that the 4 GB object from Day 1 is not deleted from the bucket when the 5 GB object is written on Day 15. Instead, the 4 GB object is preserved as an older version and the 5 GB object becomes the most recently written version of the object within your bucket.
*It has encryption.You can choose to encrypt data using SSE/S3, SSE/C, SSE/KMS, or a client library such as the Amazon S3 Encryption Client. All four enable you to store sensitive data encrypted at rest in Amazon S3.
It has access control lists(file level) and bucket policy for (bucket level)security.

What are the various storage tiers for S3?

*s3 standard
*s3 IA : Infrequently Accessed
*s3 one zone - IA : Infrequently Accessed One Zone only.
*s3 - intelligent tiering (reduced redundancy storage) : Auto Tier by machine learning our usage
*s3 glacier : for very infrequent accessed information
*s3 glacier deep archive : for archiving datas that are not being changed and are not needed rapidly.
*s3 outposts : S3 Outposts storage class to store your S3 data on-premises.Amazon S3 on Outposts delivers object storage in your on-premises environment, using the S3 APIs and capabilities that you use in AWS today. AWS Outposts is a fully managed service that extends AWS infrastructure, AWS services, APIs, and tools to virtually any datacenter, co- location space, or on-premises facility.

What are the conditions for charges of S3?

They are charged for with the selection of storage tier and follows :
*REQUESTS(transfer in - transfer out)
*STORAGE MANAGEMENT PRICING : you can use a single Amazon S3 bucket to store a mixture of S3 Glacier Deep Archive, S3 Standard, S3 Standard-IA, S3 One Zone-IA, and S3 Glacier data. Also S3 Object Tagging are a part of storage management.
*DATA TRANSFER PRICING(in-out)
*Versioning
*Location
*We measure storage usage in “TimedStorage-ByteHrs,” which are added up at the end of the month to generate your monthly charges.
*Assume you store 100GB (107,374,182,400 bytes) of data in Amazon S3 Standard in your bucket for 15 days in March, and 100TB (109,951,162,777,600 bytes) of data in Amazon S3 Standard for the final 16 days in March.
At the end of March, you would have the following usage in Byte-Hours: Total Byte-Hour usage = (107,374,182,400 bytes x 15 days x 24 hours / day)] + 109,951,162,777,600 bytes x 16 days x 24 hours / day)] = 42,259,901,212,262,400 Byte-Hours.
Let's convert this to GB/Months: 42,259,901,212,262,400 Byte-Hours / 1,073,741,824 bytes per GB / 744 hours per month = 52,900 GB/Months
This usage volume crosses two different volume tiers. The monthly storage price is calculated below assuming the data is stored in the US East Northern Virginia) Region: 50 TB Tier: 51,200 GB x $0.023 = $1,177.60 50 TB to 450 TB Tier: 1,700 GB x $0.022 = $37.40
Total Storage Fee = $1,177.60 + $37.40 = $1,215.00

How is the security being handled in S3?

Customers may use four mechanisms for controlling access to Amazon S3 resources: Identity and Access Management (IAM) policies, bucket policies, Access Control Lists (ACLs), and Query String Authentication. IAM enables organizations with multiple employees to create and manage multiple users under a single AWS account. With IAM policies, customers can grant IAM users fine-grained control to their Amazon S3 bucket or objects while also retaining full control over everything the users do. With bucket policies, customers can define rules which apply broadly across all requests to their Amazon S3 resources, such as granting write privileges to a subset of Amazon S3 resources. Customers can also restrict access based on an aspect of the request, such as HTTP referrer and IP address. With ACLs, customers can grant specific permissions (i.e. READ, WRITE, FULL_CONTROL) to specific users for an individual bucket or object. With Query String Authentication, customers can create a URL to an Amazon S3 object which is only valid for a limited time.
By default all new created s3 are private there is no public access.We can setup access control to our bucket with : bucket policy : applied at bucket level which applies to all objects in the bucket written in json (policy generator tool) access control lists : applied at object level that we can apply to individuals or groups.
we can also configure access logs to see all the requests to the s3 bucket all (crud) operations being performed in our S3.
How does S3 handle Encryption
You can choose to encrypt data using SSE/S3, SSE/C, SSE/KMS, or a client library such as the Amazon S3 Encryption Client. All four enable you to store sensitive data encrypted at rest in Amazon S3.

So in transit while the requests are being generated S3 can enable encryption by forcing (SSL/TLS/HTTPS) to handle its encryption.

But at rest AWS handles encryption with its services using :
SSE/S3 provides an integrated solution where Amazon handles key management and key protection using multiple layers of security. You should choose SSE/S3 if you prefer to have Amazon manage your keys.

SSE/C enables you to leverage Amazon S3 to perform the encryption and decryption of your objects while retaining control of the keys used to encrypt objects. With SSE/C, you don’t need to implement or use a client-side library to perform the encryption and decryption of objects you store in Amazon S3, but you do need to manage the keys that you send to Amazon S3 to encrypt and decrypt objects. Use SSE/C if you want to maintain your own encryption keys, but don’t want to implement or leverage a client-side encryption library.

SSE/KMS enables you to use AWS Key Management Service (AWS KMS) to manage your encryption keys. Using AWS KMS to manage your keys provides several additional benefits. With AWS KMS, there are separate permissions for the use of the master key, providing an additional layer of control as well as protection against unauthorized access to your objects stored in Amazon S3. AWS KMS provides an audit trail so you can see who used your key to access which object and when, as well as view failed attempts to access data from users without permission to decrypt the data. Also, AWS KMS provides additional security controls to support customer efforts to comply with PCIDSS, HIPAA/HITECH, and FedRAMP industry requirements.

Handling CORS in S3?

CORS IN S3 is very simple as we can enable CORS using bucket policy and hence services trying to access the bucket will be allowed without and cross origin issues.

Using CLOUDFRONT for S3?

AWS CLOUDFRONT helps in making the S3 bucket available in all edge locations which will optimize the performance of the S3 where anyone can access the S3 bucket from anywhere in the globe and faster with cloudfront where as using cloudfront in s3 will make the bucket available in all the edge locations of Amazon Web Services and hence all the operations(CRUD) to be handled by S3 will improve as it communicates with the edge locations of AWS and then AWS will use its internal network to get the data if they are not cached.
*EDGE locations will cache your information
*Origin is all the aws services that will be needed to be access by the users origin can be s3,ec2
*Distribution : web for website / rtmp for audio/video
*Cloudfront is also used to accelerate the upload of files to s3.(Transfer Acceleration)
*Objects are cache for the life of the ttl(time to live-default:24hrs) you can set the ttl.
*As with ttl as before you data if new will be accessed ASAP but if you edit the data can be cached and to clear the cache you are allowed but you will be charged.
*You can clear the cache with invalidating objects

How to make the most of S3?

We can make the most of S3 with :
*use cloudfront
*using random prefix for keynames(hex hash)
*We can make 3500 put requests per second 5500 get requests so utilize to increase performace.
*CloudWatch Storage Metrics are enabled by default for all buckets, and reported once per day but you can configure it to conditions you want.
*You can use CloudWatch to set thresholds on any of the storage metrics counts, timers, or rates and trigger an action when the threshold is breached. For example, you can set a threshold on the percentage of 4xx Error Responses and when at least 3 data points are above the threshold trigger a CloudWatch alarm to alert a DevOps engineer.

LAB

Create two buckets
*In your bucket enable CORS to access data with these two buckets.
*In your bucket Add ACL and Bucket Policy.
*In your bucket Add Encryption
*Go to cloudfront and enable Cloudfront and Transfer Acceleration
*Go to cloudfront and add restrictions to restrict countries
*Go to cloudfront and use invalidating objects to clear cache
*In your bucket enable versioning to keep all version of your object
*In your bucket enable access logs
*Update your bucket policy to restrict access to come from only cloud front.
*Create iam user to access bucket from cloudfront
*In your cloudfront choose http methods
*In your cloudfront set minimum ttl (for fast changing objects up to date)
*In your cloudfront you can use signed URLs or Signed Cookies) for restricted content
*Enable AWS/WAF to protect the bucket from common web exploits that may affect availability, compromise security, or consume excessive resources.
**PLEASE GO THROUGH FOR MORE COMPREHENSIVE INFORMATION* https://aws.amazon.com/s3/faqs/

Domain Name Server: Give me the IP address.

abhick09 — Sun, 23 Aug 2020 12:05:07 +0000

In the real world as humans we feel comfortable identifying everything around us with specific names whether it be other humans,animals or any object we associate its identification with various names but while on the internet computers/servers are identified with numbers also know as IP addresses.These IP addresses are the reason we are able to communicate with various computers all over the world with the help of internet.

As we feel comfortable with names but the computers on the network use numbers for their identification there tends to arise a problem where we find it hard to memorize these long numeric addresses to communicate with computers so as a result to resolve this and help people use the internet and communicate with various servers were made easy with the introduction to DNS also know as Domain Name System.
Domain Name System is the bridge which translates IP addresses to domain names.For examples these are various domain names with their IP addresses.

Name: facebook.com Address: 157.240.198.35(IP V4) Name: facebook.com Address: 2a03:2880:f144:82:face:b00c:0:25de(IP V6)
Name: google.com Address: 172.217.160.174(IP V4) Name: google.com Address: 2404:6800:4009:80a::200e(IP V6)
Name: linkedin.com Address: 108.174.10.10(IP V4) Name: linkedin.com Address: 2620:109:c002::6cae:a0a(IP V6)

You could copy and paste the IP V4 addresses which will automatically redirect you to their respective domain names.

THE PROCESS

As our machine could not resolve the domain’s IP address within our machine it sends a query to your ISP’s server where if our ISP’s server has the IP address cached it will provide it to us then our machine could request for the results to the particular server with the help of the IP address provided by our ISP.
If our ISP does not have the IP address we were looking for it sends a request to the ROOT SERVER.There are 13 sets of root servers around the world which are handled by 12 different organizations of the world.The root server then redirects the request to TLD servers also know as TOP LEVEL DOMAIN.These TLD servers are supposed to have addresses information for top level domains such as .com,.net,.org etc.But the TLD will still not have the IP address of our domain so it redirects us to Authoritative Name Server.
Authoritative Name Servers are the ones who are supposed to have all the information regarding the domain.Then as the request reaches the authoritative name server it will have the desired IP address of the domain and then sends it back to our machine so that we can access the information we require via the web using the ip addresses associated with our query.

As technology is just a tool to help us live a better and easier life DNS is also a tool which helps us browse the web in a easy and fast way so we could concentrate more on the information we seek rather than spending time trying to figure out the way get information from the internet with very confusing and long numeric addresses.

Restful CRUD API with Express in NODE.js.

abhick09 — Mon, 11 May 2020 15:30:49 +0000

I hope everyone is safe during theses difficult times here i am to today trying to give you some things i have learned on building basic CRUD operations with Express.

As express and node are very powerful things and can be used to built large scale applications.This CRUD results to building restful endpoints which can be used for all your web apps whether it be on the phone via apps or on a domain for the internet.

As we are following the MVC architecture for clean and easily maintainable code so i suggest you to do the same if you are following along.Its pretty simple its is just separating the code into three parts where

Model : Data Schema
View : Your view files
Controller : Where for handling functions for any changes in your schema.

Apart from these some additional folders are needed as your code will tend to become more organized,easy to read and collaborate on as well.We have other folders such as:

Routes: For navigating via different api's for different requests for the same or different schema.
Upload : For our images.
Middleware : For third party configuration such as authentication in my setup. (JWT will be covered in next post)

Lets get started once when you npm init your project as in my previous post.You now create server.js file to run your node server.Where you require a http module which is a default package in NODE and the file is as follows.

Then we should run node server.js in your terminal which will be reflected on localhost:5000 in your browser.

Now creating you first route.Create a route folder and inside it add your file where you want to store your routes in my case which is devRoutes.js.
After that your need to configure your route file as per following.

Once you have configured it you need to add your route in the app.js file where you need to import the route file and then call it using app.use followed by the url you want where app.use('/dev',devRoutes) will take you to getting all the request in that route.As below.

We will be using a non relational database called mongodb.You can configure any other database with this setup.

Now we configure our database and start with the CRUD operations.I suggest that you to use the online db with mongodb atlas which has free for development purpose(simply google mongodb atlas) as they will be very easy to configure and use and you can even put your projects online when you want very less configuration will be needed to configure as compared to shifting from an local environment.

You should now create a model and controller folder. As for now in this project we are dealing with end points and testing it with postman we will not need view aspects of the project but you can add the view and fetch the end points in your node app as well.

We will be using app.js files for all of our configuration needed including our database.As configuring mongo db atlas in pretty simple and easy or you can setup you local mongo db on your device as well.

For online users once you have logged in to the mongodb atlas you need to create a cluster for your database.You should add your admin user name and password as well.

The UI of mongo atlas is pretty comfortable you can find your way around its will be under heading name database security.

Now once you set the user name password you should click on connect and then connect to your application.

Then there you can find a link which will be required to setup mongo to your application.

Lets start our db setup by installing the mongoose package.Where we install npm install mongoose.

You will also need to setup body-parser to access data using json so npm install body-parser and its setup is also needed.

Once you have connected you need to navigate to the app.js file and create a connection.Once everything is configured your app.js will look like.I recommend you need to put your password in a safe env file but for demo i have directly have this there.

Once you have made a connection you need to create a schema.So as to create a schema you need to add models folder in your setup and add a model.js in that folder which should at last look like this or as per your choice of other many data types i would suggest to the official mongodb documentation.

Once the schema is created we need to work on both our routes and controllers where geeting individual data will require id for posting data will require a body and some other functions are needed to be followed as per the files below.Once you create your schema you need to create a controller and a routes file as follows.

router.js

Controller.js

At last you need to test the end points with postman with its requests.

Thanks for making it through this much hope for a feedback just trying to be the part of a community.Next post will be CRUD with image upload and JWT authentication which will guard the routes.

WHY you should start with NODE.js

abhick09 — Sat, 09 May 2020 14:09:38 +0000

While most of the worlds internet is filled with JavaScript we surely know that JavaScript has been through everything and is a very powerful language.While most of the time it is considered to be a language which is responsible for structuring and rendering dynamic content on the UI along side HTML and CSS BUT for quite sometime now it is being used to built the server side or the database part of the web applications which is NODE.js.
It was written using C, C++, JavaScript.

NODE.js was written by Ryan Dahl and had its first release in 2009 by the NODE.js foundation and now has partnered with the community and is under a joint partnership named OpenJS foundation.

Setup your application with your machine.
https://nodejs.org/en/download/ use this link to follow the documentation choose your system for your respective OS.

Once you have configured NODE and npm on your machine use your terminal or bash to work with node and installing packages(npm).

Node package manager is a package manager in NODE and other JavaScript frameworks as Express,React and many more which helps install various JavaScript package,libraries for helping us build large scale applications where libraries have ready to use services to handle various operations required in our application which are stored in the folder named /node_modules. https://www.npmjs.com/

Now as to build large scale applications we use frameworks where as many other technologies NODE also has many frameworks to choose from some of them are.
1.ExpressJs https://expressjs.com/
2.MeterorJs https://www.meteor.com/
3.NestJs https://nestjs.com/
4.SailsJs https://sailsjs.com/

and many more these are the top 4 frameworks according to the stars they have on GITHUB.

So why use NODE?
Node is very popular and as it uses JavaScript you can master full-stack web development with both front-end and the server side using the same language.The main feature of NODE is that it is asynchronous which as a result wont let it run out of memory and makes it very fast.As node works on single thread which can handle thousands of connections very fast using an event loop where a event is triggered and then it moves on.

Still why use NODE?
It excels with REST API,Microservices,Real Time apps(chat,live updates),CRUD apps these can be built with node and will perform very fast applications like netflix,yahoo,paypal,linkedin,godaddy are some diverse examples which uses NODE to serve there clients.

So what is a basic NODE setup?
At first just type node in your terminal and do some basic arthemtic operations or try creating functions within the terminal.

Every node project is initiated with npm init which creates a package.json file.
We need to navigate to the folder where we want node application to be setup and open the terminal/bash and enter the command npm init which creates a package.json file in your folder.

As in the picture we have a package.json file where as we installed express it creates a dependencies section where we can know which package have been installed.

Now we would want to create ourfunction.js file for our specific reasons.Will further continue with express and setup a MVC pattern CRUD app with token based authentication,file upload,route guard,nesting tables in database with relationships as person with profile tables.Stay tuned.
Meanwhile you should learn about if not JSON,Arrow functions,MVC pattern,HTTP,Promises would help you learn node fast.

Hope for a feedback to improve just wanted to give out some prerequisites if you want to start server side work with JavaScript and its many options.