DEV Community: Venkat

IAM Best Practices - AWS

Venkat — Sun, 03 Apr 2022 15:28:00 +0000

Hey everyone! Hope you're doing well and getting ready to read my yet another tech blog on IAM Best Practices - AWS. Let's discuss here on this.

Step 1 - Login to the Console

Visit
https://aws.amazon.com/console/
Choose Sign in to the console.
Choose Root user. Enter the Root user email address.
Choose Next
Enter the Password for the root user. Choose Sign in.

Step 2 - Enable MFA (optional)

At the top right, choose your account name. Then choose My Security Credentials from the drop down menu.
Expand Multi-factor authentication (MFA). Choose Activate MFA.
On the Manage MFA device pop-up window. Choose Virtual MFA device and choose Continue.

Note: You will need a virtual MFA application installed on your device or computer. You can see a list of applications on step 1 on the Set up virtual MFA device pop-up window. There is a hyperlink which shows a list of compatible applications. Before continuing to the next step make sure you have one of these applications installed on your mobile device or computer.

Choose Show QR code and scan the code using your device.

Note: If you are using a computer you can choose Show secret key and type the secret key into your MFA application.

Type the first MFA code into the MFA code 1 field. Then type the second generated number into the MFA code 2 field. Choose Assign MFA.
You should see a pop-up indicating that you have successfully assigned a virtual MFA device. Choose Close.
Expand Access keys (access key ID and secret access key).

Note: There should be no access keys listed. If an access key exists (for your new account) choose Delete under Actions. Choose Deactivate. Enter in the access key ID in the confirmation field. Choose Delete.

Step 3 - Create an IAM user

In the service search bar, type in Identity and Access Management (IAM) dashboard. On the left side panel, choose Users.
Choose Add user. Paste in Admin for the User name. Next to Access type, choose Programmatic access and AWS Management Console access.
Choose Add user. Paste in Admin for the User name. Next to Access type, choose Programmatic access and AWS Management Console access.
Uncheck Require password reset.
Choose Next: Permissions.
Choose Attach existing policies directly. Next to Filter policies, search for administrator. Under Policy name, choose AdministratorAccess. Choose Next: Tags.
Choose Next: Review. Choose Create user.
You can sign in with the new IAM user by clicking the hyperlink at the bottom of the Success window.

Note: It should look similar to the following: https://000000000000.signin.aws.amazon.com/console. Your account number will be different :)

Step 4 - Set up an IAM role for EC2 instance

Now that you are logged in as the Admin user, search for IAM again in the service search bar. On the left side panel, choose Roles. Then, choose Create role.
Choose AWS service. Choose EC2. Choose Next: Permissions.
Next to Filter policies, search for amazons3full and choose AmazonS3FullAccess.
Next to Filter policies search for amazondynamodb and choose AmazonDynamoDBFullAccess.
Choose Next: Tags. Choose Next: Review.
For Role name paste in S3DynamoDBFullAccessRole. Choose Create role. Note: Using full access policies are not something recommended you should do in a production environment. We are using these policies as a proof of concept to get your demo up and running quickly. Once your Amazon S3 bucket and Amazon DynamoDB table are created, you can come back and modify this IAM Role to have more specific and restrictive permissions. More on this later.

Congratulations you successfully completed the exercise...🎉🎉🎉

🚀 If you read something interesting from this article, please like and follow me for more posts.

GCP Part -II (How to set up Kubernetes Engine)

Venkat — Fri, 22 Oct 2021 17:20:51 +0000

Hello there everyone!! Hope you're doing well and getting ready to read my yet another tech blog on Google Cloud Platform. This is continuation of part - I GCP Part -I (Regions, Zones and Compute Engine) and welcome to this blog and here you will get to know how to set up the Kubernetes Engine.

Kubernetes Engine

Let's quickly look into how to set up Kubernetes Engine in Google Cloud console as follows :

Activate Cloud Shell

Cloud Shell is a virtual machine that is loaded with development tools. It offers a persistent 5GB home directory and runs on the Google Cloud. Cloud Shell provides command-line access to your Google Cloud resources.

In the Cloud Console, in the top right toolbar, click the Activate Cloud Shell button.

Click Continue.

It takes a few moments to provision and connect to the environment. When you are connected, you are already authenticated, and the project is set to your PROJECT_ID.

gcloud is the command-line tool for Google Cloud. It comes pre-installed on Cloud Shell and supports tab-completion.

You can list the active account name with this command:

gcloud auth list

You can list the project ID with this command:

gcloud config list project

Set a default compute zone

Your compute zone is an approximate regional location in which your clusters and their resources live. For example, us-central1-a is a zone in the us-central1 region.

To set your default compute zone to us-central1-a, start a new session in Cloud Shell, and run the following command:

gcloud config set compute/zone us-central1-a

Create a GKE cluster

A cluster consists of at least one cluster master machine and multiple worker machines called nodes. Nodes are Compute Engine virtual machine (VM) instances that run the Kubernetes processes necessary to make them part of the cluster.

Note: Cluster names must start with a letter and end with an alphanumeric, and cannot be longer than 40 characters.

To create a cluster, run the following command, replacing [CLUSTER-NAME] with the name you choose for the cluster (for example:my-cluster).

gcloud container clusters create [CLUSTER-NAME]

You can ignore any warnings in the output. It might take several minutes to finish creating the cluster.

Expected output :

NAME        LOCATION       ...   NODE_VERSION  NUM_NODES  STATUS
my-cluster  us-central1-a  ...   1.16.13-gke.401  3        RUNNING

Get authentication credentials for the cluster

After creating your cluster, you need authentication credentials to interact with it.

To authenticate the cluster, run the following command, replacing [CLUSTER-NAME] with the name of your cluster:

gcloud container clusters get-credentials [CLUSTER-NAME]

Expected output :

Fetching cluster endpoint and auth data.
kubeconfig entry generated for my-cluster.

Deploy an application to the cluster

You can now deploy a containerized application to the cluster.

GKE uses Kubernetes objects to create and manage your cluster's resources. Kubernetes provides the Deployment object for deploying stateless applications like web servers. Service objects define rules and load balancing for accessing your application from the internet.

To create a new Deployment hello-server from the hello-app container image, run the following kubectl create command:

kubectl create deployment hello-server --image=gcr.io/google-samples/hello-app:1.0

Expected output :

deployment.apps/hello-server created

This Kubernetes command creates a Deployment object that represents hello-server. In this case, --image specifies a container image to deploy. The command pulls the example image from a Container Registry bucket. gcr.io/google-samples/hello-app:1.0 indicates the specific image version to pull. If a version is not specified, the latest version is used.

Note: You need to create hello-app image and should keep in the gcp bucket.

To create a Kubernetes Service, which is a Kubernetes resource that lets you expose your application to external traffic, run the following kubectl expose command:

kubectl expose deployment hello-server --type=LoadBalancer --port 8080

In this command:

1. --port specifies the port that the container exposes.
2. type="LoadBalancer" creates a Compute Engine load balancer for your container.
Expected output :

service/hello-server exposed

To inspect the hello-server Service, run kubectl get:

kubectl get service

Expected output :

NAME              TYPE              CLUSTER-IP        EXTERNAL-IP      PORT(S)           AGE
hello-server      loadBalancer      10.39.244.36      35.202.234.26    8080:31991/TCP    65s
kubernetes        ClusterIP         10.39.240.1       <none>           433/TCP           5m13s

Note: It might take a minute for an external IP address to be generated. Run the previous command again if the EXTERNAL-IP column status is pending.

To view the application from your web browser, open a new tab and enter the following address, replacing [EXTERNAL IP] with the EXTERNAL-IP for hello-server.

http://[EXTERNAL-IP]:8080

Deleting the cluster

To delete the cluster, run the following command:

gcloud container clusters delete [CLUSTER-NAME]

Congratulations!

You have just deployed 🚀 a containerized application to Kubernetes Engine! 🎉🎉🎉

GCP Part -I (Regions, Zones and Compute Engine)

Venkat — Sat, 09 Oct 2021 11:31:50 +0000

Hello my dear readers!!! Hope you're doing well and getting ready to read my yet another tech blog and this time we are into the cloud technologies...

I am going to take you through this Google Cloud journey as a step by step and I will also be learning along with you, so let's kick start our learning journey together.. :)

Regions & Zones

Basically GCP provides 200+ services. And it is the cleanest cloud in the industry ever. Google Cloud protects your data, applications, infrastructure, and customers from fraudulent activity, spam, and abuse with the same infrastructure and security services Google uses.

Google Cloud’s networking, data storage, and compute services provide data encryption at rest, in transit, and in use. Advanced security tools support compliance and data confidentiality.

Below are the statistical information about the regions, zones,

and it's network as of now.

Google Compute Engine

In corporate data centers, applications are deployed to
physical servers.
Where do you deploy applications in the cloud?
1. Rent virtual servers
2.Virtual Machines - Virtual servers in GCP
3.Google Compute Engine (GCE) - Provision & Manage Virtual Machines

Compute Engine

Secure and customizable compute service that lets you create and run virtual machines on Google’s infrastructure.

Based on our requirements we can choose which type of compute engines from below types :

Predefined machine types: Start running quickly with pre-built and ready-to-go configurations.

Custom machine types: Create VMs with optimal amounts of vCPU and memory, while balancing cost.

Preemptible machines: Reduce computing costs by up to 80% with affordable short-term instances.

Confidential computing: Encrypt your most sensitive data while it’s being processed.

Rightsizing recommendations: Optimize resource utilization with automatic recommendations.

Let's look the below features provided by the compute engine based on OS, memory, costing, security, pricing etc,.

1. VM Manager
VM Manager is a suite of tools that can be used to manage operating systems for large virtual machine (VM) fleets running Windows and Linux on Compute Engine.

Fig : VM Manager architecture overview

2. Confidential VMs
Confidential VMs are a breakthrough technology that allows you to encrypt data in use—while it’s being processed. It is a simple, easy-to-use deployment that doesn't compromise on performance. You can collaborate with anyone, all while preserving the confidentiality of your data.

3. Live migration for VMs
Compute Engine virtual machines can live-migrate between host systems without rebooting, which keeps your applications running even when host systems require maintenance.

4. Sole-tenant nodes
Sole-tenant nodes are physical Compute Engine servers dedicated exclusively for your use. Sole-tenant nodes simplify deployment for bring-your-own-license (BYOL) applications. Sole-tenant nodes give you access to the same machine types and VM configuration options as regular compute instances.

5. Custom machine types
Create a virtual machine with a custom machine type that best fits your workloads. By tailoring a custom machine type to your specific needs, you can realize significant savings.

6. Predefined machine types
Compute Engine offers predefined virtual machine configurations for every need from small general purpose instances to large memory-optimized instances with up to 11.5 TB of RAM or fast compute-optimized instances with up to 60 vCPUs.

7. Preemptible VMs
Low-cost, short-term instances designed to run batch jobs and fault-tolerant workloads. Preemptible VMs provide significant savings of up to 80% while still getting the same performance and capabilities as regular VMs.

8. Instance groups
An instance group is a collection of virtual machines running a single application. It automatically creates and deletes virtual machines to meet the demand, repairs workload from failures, and runs updates.

9. Persistent disks
Durable, high-performance block storage for your VM instances. You can create persistent disks in HDD or SSD formats. You can also take snapshots and create new persistent disks from that snapshot. If a VM instance is terminated, its persistent disk retains data and can be attached to another instance.

10. Local SSD
Compute Engine offers always-encrypted local solid-state drive (SSD) block storage. Local SSDs are physically attached to the server that hosts the virtual machine instance for very high input/output operations per second (IOPS) and very low latency compared to persistent disks.

11. GPU accelerators
GPUs can be added to accelerate computationally intensive workloads like machine learning, simulation, and virtual workstation applications. Add or remove GPUs to a VM when your workload changes and pay for GPU resources only while you are using them. Our new A2 VM family is based on the NVIDIA Ampere A100 GPU. You can learn more about the A2 VM family by requesting access to our alpha program.

12. Global load balancing
Global load-balancing technology helps you distribute incoming requests across pools of instances across multiple regions, so you can achieve maximum performance, throughput, and availability at low cost.

13. Linux and Windows support
Run your choice of OS, including Debian, CentOS, CoreOS, SUSE, Ubuntu, Red Hat Enterprise Linux, FreeBSD, or Windows Server 2008 R2, 2012 R2, and 2016. You can also use a shared image from the Google Cloud community or bring your own.

14. Per-second billing
Google bills in second-level increments. You pay only for the compute time that you use.

15. Commitment savings
With committed-use discounts, you can save up to 57% with no up-front costs or instance-type lock-in.

16. Container support
Run, manage, and orchestrate Docker containers on Compute Engine VMs with Google Kubernetes Engine.

17. Reservations
Create reservations for VM instances in a specific zone. Use reservations to ensure that your project has resources for future increases in demand. When you no longer need a reservation, delete the reservation to stop incurring charges for it.

18. Right-sizing recommendations
Compute Engine provides machine type recommendations to help you optimize the resource utilization of your virtual machine (VM) instances. Use these recommendations to resize your instance’s machine type to more efficiently use the instance’s resources.

19. OS patch management
With OS patch management, you can apply OS patches across a set of VMs, receive patch compliance data across your environments, and automate installation of OS patches across VMs—all from a centralized location.

20. Placement Policy
Use Placement Policy to specify the location of your underlying hardware instances. Spread Placement Policy provides higher reliability by placing instances on distinct hardware, reducing the impact of underlying hardware failures. Compact Placement Policy provides lower latency between nodes by placing instances close together within the same network infrastructure.

Compute Engine pricing

Pricing for Compute Engine is based on per-second usage of the machine types, persistent disks, and other resources that you select for your virtual machines. If you have a specific project in mind, use the pricing calculator to estimate cost.

Okay so, this must be more than enough to kick start our exploration in the cloud console here Console 🎉🎉.

Thank you so much for sticking around and holding on to the end.

Until next time!

How Internet Message Access Protocol(IMAP) works in Node JS

Venkat — Sat, 02 Oct 2021 09:18:11 +0000

Hello my dear peers 😃! Hope you're doing well. Welcome to my tech blog and this time we are discussing about IMAP package and it's uses in Node JS with real time code snippet examples. In this, first will only focus on reading emails.

node-imap is an IMAP client module for node.js.

Let's open our terminal and hit npm install node-imap. to install IMAP package.

In this blog, we are mainly focusing on how to read email attachments based on the DATE RANGE, FROM particular email address and it's SUBJECT.

Let's see from the below example code which fetches first 3 email messages from the mail box.

var Imap = require('node-imap'),
    inspect = require('util').inspect;

var imap = new Imap({
  user: 'mygmailname@gmail.com',
  password: 'mygmailpassword',
  host: 'imap.gmail.com',
  port: 993,
  tls: true
});

function openInbox(cb) {
  imap.openBox('INBOX', true, cb);
}

imap.once('ready', function() {
  openInbox(function(err, box) {
    if (err) throw err;
    var f = imap.seq.fetch('1:3', {
      bodies: 'HEADER.FIELDS (FROM TO SUBJECT DATE)',
      struct: true
    });
    f.on('message', function(msg, seqno) {
      console.log('Message #%d', seqno);
      var prefix = '(#' + seqno + ') ';
      msg.on('body', function(stream, info) {
        var buffer = '';
        stream.on('data', function(chunk) {
          buffer += chunk.toString('utf8');
        });
        stream.once('end', function() {
          console.log(prefix + 'Parsed header: %s', inspect(Imap.parseHeader(buffer)));
        });
      });
      msg.once('attributes', function(attrs) {
        console.log(prefix + 'Attributes: %s', inspect(attrs, false, 8));
      });
      msg.once('end', function() {
        console.log(prefix + 'Finished');
      });
    });
    f.once('error', function(err) {
      console.log('Fetch error: ' + err);
    });
    f.once('end', function() {
      console.log('Done fetching all messages!');
      imap.end();
    });
  });
});

imap.once('error', function(err) {
  console.log(err);
});

imap.once('end', function() {
  console.log('Connection ended');
});

imap.connect();

There are scenarios where you need to fetch only the attachments from the email and process it for a different purpose. In such cases, please refer below code example.

var imap = new Imap({
  user: 'mygmailname@gmail.com',
  password: 'mygmailpassword',
  host: 'imap.gmail.com',
    port: 993,
    tls: true,
  });
  imap.once("ready", function () {
    var fs = require("fs"),
      fileStream;
    imap.openBox("INBOX", true, function (err, box) {
      if (err) throw err;
      try {
        imap.search(
          [
            ["FROM", FROM_MAIL],
            ["HEADER", "SUBJECT", SUBJECT],
            ["UNSEEN", ["SINCE", "Day, Year"]],
          ],
          function (err, results) {
            if (err) throw err;
            try {
              var f = imap.fetch(results, {
                bodies: ["HEADER.FIELDS (FROM TO SUBJECT DATE)"],
                struct: true,
              });
              f.on("message", function (msg, seqno) {
                console.log("Message #%d", seqno);

                var prefix = "(#" + seqno + ") ";
                msg.on("body", function (stream, info) {
                  var buffer = "";
                  stream.on("data", function (chunk) {
                    buffer += chunk.toString("utf8");
                  });
                  stream.once("end", function () {
                    console.log(
                      prefix + "Parsed header: %s",
                      Imap.parseHeader(buffer)
                    );
                  });
                });
                msg.once("attributes", function (attrs) {
                  // console.log("test", attrs);
                  var attachments = findAttachmentParts(attrs.struct);
                  console.log(
                    prefix + "Has attachments: %d",
                    attachments.length
                  );
                  for (var i = 0, len = attachments.length; i < len; ++i) {
                    var attachment = attachments[i];

                    var f = imap.fetch(attrs.uid, {
                      //do not use imap.seq.fetch here
                      bodies: [attachment.partID],
                      struct: true,
                    });
                    //build function to process attachment message
                    f.on("message", processAttachment(attachment));
                  }
                });
                msg.once("end", function () {
                  console.log(prefix + "Finished email");
                });
              });
              f.once("error", function (err) {
                console.log("Fetch error: " + err);
              });
              f.once("end", function () {
                console.log("Done fetching all messages!");
                imap.end();
              });
            } catch (e) {
              console.log("err", e);
            }
          }
        );
      } catch (e) {
        console.log("log", e);
      }
    });
  });

  imap.once("error", function (err) {
    console.log(err);
  });

  imap.once("end", function () {
    console.log("Connection ended");
  });
  imap.connect();

The downloaded email attachment must be decoded using Base64Decode() method.

function processAttachment(attachment) {
  var filename = attachment.params.name;
  var encoding = attachment.encoding;
  var name = filename.split(".")[1];
  console.log("log", name);

  return function (msg, seqno) {
    if (name === "pdf") {
      var prefix = "(#" + seqno + ") ";
      msg.on("body", function (stream, info) {
        //Create a write stream so that we can stream the attachment to file;
        console.log(
          prefix + "Streaming this attachment to file",
          filename,
          info
        );
        var path = require("path");
       // var dirPath = path.join(__dirname, "/attachments");
        var writeStream = fs.createWriteStream(filename);
        writeStream.on("finish", function () {
          console.log(prefix + "Done writing to file %s", filename);
        });

        if (toUpper(encoding) === "BASE64") {
          stream.pipe(new base64.Base64Decode()).pipe(writeStream);
        } else {
          stream.pipe(writeStream);
        }
      });
      msg.once("end", function () {
        console.log(prefix + "Finished attachment %s", filename);
      });
    }
  };
}

Note: The above process attachment method has a condition check of having only PDF docs.

So, after processing the email attachments would you recommend those emails still be present in same inbox? No not at all, because we need to move that to some other folder so that we can differentiate the newly arrived emails.

So, you can move the processed email to specific folder from the inbox using below code example.

 imap.seq.move(seqno, "Processed", function (err) {
                  if (!err) {
                    console.log(seqno + ": move success");
                  }
                });

Hope you got atleast an idea how to work with imap package and with emails in Node JS 🎉🎉.

References:
https://www.npmjs.com/package/node-imap
https://github.com/mikebevz/node-imap

Thank you for sticking around and holding on to the end.

Until next time!

How to dockerize a React app with Nest JS server code...!

Venkat — Fri, 23 Jul 2021 05:40:01 +0000

Namaste coders :) Welcome to my tech blog on dockerizing React app with one of Node's typescript framework. This is my first ever post in DEV, excited to contribute it 😃.

Basically, there are two ways you can dockerize them,

1. Dockerize both React app and Nest JS separately and compose them.
2. Dockerize both of the apps in a single docker file.

1.Dockerize both React app and Nest JS separately and compose them.

a). Dockerize React app :

Create a docker file as below in React app-



FROM node:14.16.1

WORKDIR /app

COPY package.json ./ 

RUN npm install

COPY . .

EXPOSE 3000

CMD ["npm", "start"]{% raw %}`
```


Also create a .dockerignore file

```
node_modules
.git
.gitignore
```
Next step is that we have to build the docker image of the React app.
```
 docker build . -t react
```

Now run the tagged image as below.
```
 docker run --name react -d -p 80:3000 react
```
Open http://localhost:3000 and you should see React served from Docker.

Also you can check the docker container running as below with `docker ps` command.
```
CONTAINER ID   IMAGE     COMMAND                  CREATED       STATUS         PORTS                                   NAMES
6aea1cf12647   react     "docker-entrypoint.s…"   11 days ago   Up 3 seconds   0.0.0.0:80->3000/tcp, :::80->3000/tcp   react
```
####b). Dockerize Nest JS code :
Create a docker file as below in your server directory- 
```
FROM node:14.16.1

WORKDIR /app

COPY package.json ./

RUN npm install

COPY . .

EXPOSE 5000

CMD [ "npm", "run", "start:dev" ]
```
As similar to above create a .dockerignore file

```
node_modules
.git
.gitignore
```
Next step is that we have to build the docker image of the server app.
```
 docker build . -t server
```

Now run the tagged image as below.
```
 docker run --name server -d -p 80:5000 server
```
Let's check this by hitting http://localhost:5000 and you should see your Nest JS being served from Docker.

So, now we have stepped into the final process of running both simultaneously by creating docker compose yaml file in the project root directory as below. 

```
version: "3"
services:
    frontend:
        container_name: client
        build:
            context: ./client
            dockerfile: Dockerfile
        image: react
        ports:
            - "3000:3000"
        volumes:
            - ./client:/app
    backend:
        container_name: backend
        build:
            context: ./server
            dockerfile: Dockerfile
        image: server
        ports:
            - "5000:5000"
        volumes:
            - ./server:/app

```
Run the command `docker-compose up` and you should see both the apps running.



###2.Dockerize both of the apps in a single docker file.

I would recommend this approach than the above, it's simple and preferred to follow for deploying these kind of applications for dev, qa and prod environments.

As we have both apps React and Nest JS server code. We initially need to build our React UI code and should copy the build folder contents as below - 

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xjfzr93av9h0ria6u4sa.jpg)

After, we need to paste them in public folder of the Nest JS server code root directory. 

**Note:** In Nest JS, you need to place server static module in *app.module.ts* file as below 
```
  ServeStaticModule.forRoot({
    rootPath: join(__dirname, '..', 'public'),
  }),
```

Finally, you are all set to run the docker file with commands below 
```
FROM node:14.16.1:lts-alpine
RUN mkdir -p /usr/src/app

WORKDIR /usr/src/app

COPY . .

RUN cd ./client && npm ci  && npm run build && cd ..

RUN cd ./server && npm ci  && cd ..

RUN mkdir -p /usr/src/app/server/public

RUN cp -r ./client/build/* ./server/public/

WORKDIR  /usr/src/app/server

RUN npm run prebuild

RUN npm run build

EXPOSE 5000

CMD [ "npm", "run", "start:dev" ]
```
Build the docker file 
```
 docker build . -t ReactServer
```
And finally run the image 
```
docker run --name ReactServer -d -p 80:5000 ReactServer
```
Open http://localhost:5000 and you should see the application served from Docker.

Congratulations you successfully dockerized React UI and Nestjs server...🎉🎉🎉

🚀 If you read something interesting from this article, please like and follow me for more posts.