DEV Community: Rajat Kumar Nayak

From Code to Impact

Rajat Kumar Nayak — Wed, 04 Dec 2024 15:43:14 +0000

This is not the typical technology blog I used to publish.

Before getting into the main topic of discussion I thought let’s have a background.

I have an approach and everyone suggested also that I learn better by building projects and implementing the concepts. From college seniors to tech influencers emphasized the fact that you must implement whatever you are learning in projects. Well they are correct and everyone should learn software development by building projects, in short must implement the concepts.

But I came to realize whenever we are building any project we should solve a problem statement.

Why I am saying this?
On 30th November 2024, I came across a folder on my phone which has basically all the applications that I built during my college days to understand the concepts of Android development.

I assumed that none of the applications would be working. Again why I thought that.

It is because I used to integrate Firebase as a backend for all my applications during my initial days of Android development. Now we all know that if we don’t do any transaction in the storage or any of the services of the Firebase for a prolonged period, firebase will automatically stop the service after giving proper email notifications and all.

But a strange thing happened.
I clicked my blood donation app, tried to register as a new user and guess what it worked. The user got successfully registered and it was after 2 years. and when I entered the home screen I could see around 100s of users having their details submitted as the donor.

Seeing over 100 registered donors filled me with pride and reminded me of the potential of even small, college-level projects to contribute meaningfully to society.

What exactly I am trying to say?

It’s not always a fancy tech stack that will help to build a great project. P.S. I am not saying my project is great. (Ha ha..)But what I’m trying to say is that if we think about the final outcome that our product will deliver, that will bring about change and impact in society.

So, the next time you start a project, think beyond the code. Think about the problem you’re solving and the impact you can create. Who knows? Years from now, your work might still be helping someone.

You can find that project here. https://github.com/rajat-gith/BloodDonationApp

Unleashing Real-Time Data: Setting Up Kafka on EC2 and Connecting from Your Local Machine

Rajat Kumar Nayak — Tue, 25 Jun 2024 13:46:36 +0000

This blog post will show you how to install Apache Kafka on an Amazon EC2 instance and connect to it locally to take use of the potential of real-time data. We will guide you through every step of the process, including setting up Kafka on the cloud and managing data streams using a producer and consumer. By the end, you will be able to use a specialized GitHub repository to conduct an analysis of the stock market. For those who want to learn more about real-time analytics and data streaming, this extensive guide is ideal.

So let's begin

Steps to Setup Kafka

Create an EC2 Instance

Create EC2 Instance

Edit Security Group

Edit Security Group

Connect to EC2 Instance
1.Go to the Downloads folder where the key is located
2.Fetch the SSH Connect Command

Sign in to the AWS Management Console.
Navigate to the EC2 service.
In the EC2 Dashboard, click on "Instances" in the left sidebar.
Select the instance you want to connect to by clicking on its checkbox.
Click the "Connect" button at the top of the page.
In the "Connect to instance" page that opens, select the "SSH client" tab.
Scroll down to the section titled "Example". Here, you'll see the exact SSH command to use for connecting to your instance.
You can copy this command directly from the dashboard. It will look something like this:

ssh -i "your-key-name.pem" ec2-user@ec2-12-34-56-78.compute-1.amazonaws.com

Now you are connected to remote EC2 Instance

Run this command to Download Kafka

wget https://downloads.apache.org/kafka/3.7.0/kafka_2.13-3.7.0.tgz

Install Amazon Corretto (OpenJDK):

Amazon Corretto is a no-cost, multiplatform, production-ready distribution of the Open Java Development Kit (OpenJDK).

sudo yum install -y java-1.8.0-amazon-corretto-devel

Navigate to the kafka folder

 cd kafka_2.13-3.7.0/

Now Similarly open two terminal connecting to the EC2 Instance

In the first terminal run the following command to start zookeeper:

bin/zookeeper-server-start.sh config/zookeeper.properties

In the second terminal run the following command to start server:

export KAFKA_HEAP_OPTS="-Xmx256M -Xms128M"

cd kafka_2.13-3.7.0/

bin/kafka-server-start.sh config/server.properties

Hurray you can see your server starting !!

Now stop both the services and now focus on an important step.

There is an important step here:
To make our local system connect to the Kafka that is on the remote EC2 instance we need to make a little bit of configurational changes.

Run the following command in your terminal

sudo nano config/server.properties

Find this line:
ADVERTISED_LISTENERS

In your case, it will be localhost.
Change it to the public IPV4 of your instance.

Press CTRL+X and SHIFT+Y and then Enter

And we are done !!

We can check the running of kafka

cd kafka_2.13-3.7.0

bin/kafka-topics.sh --create --topic sample_topic --bootstrap-server {Public IP of your EC2 Instance:9092} --replication-factor 1 --partitions 1

Start Producer:

bin/kafka-console-producer.sh --topic sample_topic --bootstrap-server {Public IP of your EC2 Instance:9092}

Start Consumer:
Duplicate the session & enter in a new console --

cd kafka_2.13-3.7.0/

bin/kafka-console-consumer.sh --topic sample_topic --bootstrap-server {Public IP of your EC2 Instance:9092}

Write some messages in producer side and you can receive the same in consumer side.

Now we can build our project locally and by properly mentioning the details we can connect to the Kakfa.

I have worked on a project that leads to stock market analysis and plots the graph in real time. You can find the code below.

https://github.com/rajat-gith/stock-market-analysis.git

Effortless Node.js Deployment on Amazon EC2 with tmux: A Step-by-Step Guide

Rajat Kumar Nayak — Sun, 04 Feb 2024 13:37:54 +0000

In this blog, we will have a clear and step-by-step understanding of how to host our node application on an Amazon EC2 instance and how to keep the application running.

Introduction:
Imagine you're a chef in a busy kitchen, about to serve a delicious dish. But just as you're ready, someone unplugs the oven!

Problem:
That's what it's like hosting an application on EC2. When you close the terminal, your app goes offline, leaving users in the lurch.

Solution:
Enter tmux, your kitchen assistant! It keeps your app running in the background, like a backup generator for your oven. With tmux, your app stays online, serving users seamlessly.

So, when hosting on EC2, don't forget tmux. It's the secret ingredient for uninterrupted service!

Now Let's look at the steps to implement it..

The entire steps will be broadly classified into 3 main categories.

Step-1 Develop a Node JS Application.

Initiate a Node Application
npm init -y
Install express dotenv, express npm package.
npm install express dotenv
Create a .env file.

PORT=3000

Create a sample node application.

require("dotenv").config();
const express = require("express");
const app = express();
const PORT = process.env.PORT || 3001;

app.get("/", (req, res) => {
  res.send("Welcome to AWS World");
});


app.listen(PORT, () => {
  console.log(`Server running on ${PORT}`);
});

Now your node application is ready to push into the remote repository.

Initialize a git repo using:
git init
Make sure to add .gitignore file and add node_modules folder inside .gitignore.

Create the main branch
git checkout -b main
Add the remote origin:
git remote add origin <repo-link>
Then Commit the changes with the following commands:
git add .
git commit -m <commit message>
Push the commit into the remote repo using:
git push origin <branch-name>

Now we pushed the code successfully into the remote repository.

Step-2 It's time to start with the AWS EC2 hoisting part.

I have attached a video below which describes how to create an EC2 instance.

After you have the instance now it time to setup the ports to open for our application to display.

Head over to the security groups option in the instance dashboard.
Click on Edit Inbound Rules Option.

Add a new rule.

Now we are ready with setting up the EC2 instance now let's move to connecting our instance and hoisting the application.

Setting up SSH and Hoisting

Set up SSH in your local system.

You can check whether SSH is present in your system by running ssh in your command prompt.

If the output is not the same as above then you can setup SSH by following the steps mentioned in the provided blog link.

Setup SSH

After you are done with SSH follow the below-mentioned steps.

Run the bellow command to connect to the instance. ssh -i <key-path> ubuntu@<public-ip>

Make a directory of your choice name.
mkdir node-app
Move inside the director.
cd node-app

You can check for git using the git --version command.
If not found you can install using sudo apt install git

clone the remote app that we created previously.

Move inside the cloned folder using cd node-app

Then you can install packages using
npm install
It will install all necessary packages.
Install tmux using
sudo apt install tmux

Start a new tmux session: tmux new-session -s session_name

Run the node application in the tmux session using
node index.js
To detach from the tmux session without stopping the Node.js app, press Ctrl+b then d.

Hurray you have hoisted the application with continuous running.

You can again attach to a pre-existing session using the following command. tmux attach -t session_name

So we have pretty much covered how to host a node application in an EC2 instance and run it continuously with Node JS. If you have any doubt I will be more than happy to help. You can mail your queries to 'rajatnayak1582002@gmail.com'.

Caching in NodeJS with Redis

Rajat Kumar Nayak — Fri, 26 Jan 2024 17:47:38 +0000

This blog post will provide a brief explanation on how to use Redis caching to improve server response time, boost performance, and reduce fetch times.

Step 1: To run Redis Stack on Windows we need to install Docker.Download Docker

Step 2: After installing Docker open the terminal and run the following command:

docker run -d --name redis-stack -p 6379:6379 -p 8001:8001 redis/redis-stack:latest

The command pulls the redis-stack image from the docker hub and runs it.

You can open the Docker Desktop and see the status of the image running.

After this, we can visit http://localhost:8001/ and access the redis stack dashboard:

Now that the Redis setup is complete, let's move on to the Node.js portion.

We will first look without Redis, then with Redis, to see the difference.

Initiate a node package with the following command:
npm init
Install express and axios package:
npm install express axios
Find the code below for index.js:

Import the necessary modules:

const express = require("express");
const axios = require("axios");
const app = express();
const port = 3000;

app.get("/todos", async (req, res) => {
  try {
    // Record the start time
    const startTime = Date.now();

    // Make a GET request to the typicode todos endpoint
    const response = await axios.get(
      "https://jsonplaceholder.typicode.com/todos"
    );

    // Record the end time
    const endTime = Date.now();

    // Calculate the total time taken
    const totalTime = endTime - startTime;

    // Extract todos from the response
    const todos = response.data;

    // Send the todos along with the total time as a JSON response
    res.json({ todos: todos, totaltime: totalTime });
  } catch (error) {
    console.error("Error fetching todos:", error.message);
    res.status(500).json({ error: "Internal Server Error" });
  }
});

After running the api in Postman you can see the response.

With Redis
Add the following Lines:

const Redis = require('ioredis');
const redisClient = new Redis();

Initially no data in Redis:

The complete index.js file:

const express = require("express");
const axios = require("axios");
const Redis = require("ioredis");

const app = express();
const port = 3000;

// Create a Redis client
const redisClient = new Redis();

app.get("/todos", async (req, res) => {
  try {
    // Record the start time
    const startTime = Date.now();

    // Check if the response is cached in Redis
    const cachedTodos = await redisClient.get("todos");

    // Record the end time
    const endTime = Date.now();

    if (cachedTodos) {
      // If cached data is found, send it directly
      const parsedTodos = JSON.parse(cachedTodos);

      // Calculate the total time taken for fetching from Redis
      const totalTime = endTime - startTime;

      res.json({ todos: parsedTodos, totalTime, source: "Redis Cache" });
    } else {
      // Make a GET request to the typicode todos endpoint
      const apiStartTime = Date.now();
      const response = await axios.get(
        "https://jsonplaceholder.typicode.com/todos"
      );
      const apiEndTime = Date.now();

      // Calculate the total time taken for fetching from the API
      const apiTotalTime = apiEndTime - apiStartTime;

      // Extract todos from the response
      const todos = response.data;

      // Save the todos to Redis for future use with a TTL of 60 seconds (adjust as needed)
      await redisClient.set("todos", JSON.stringify(todos), "EX", 60);

      // Calculate the total time taken for the entire operation
      const totalTime = apiEndTime - startTime;

      // Send the todos along with the total time as a JSON response
      res.json({ todos, totalTime, apiTotalTime, source: "API" });
    }
  } catch (error) {
    console.error("Error fetching todos:", error.message);
    res.status(500).json({ error: "Internal Server Error" });
  }
});

app.listen(port, () => {
  console.log(`Server is running on http://localhost:${port}`);
});

After running the API in postman:

You can find that a string in the redis stack browser window.

And wohoo!! After using Redis:

The response time has reduced significantly.

So we have pretty much covered how to use Redis as a cache to improve perfomance and efficiency. If you have any doubt I will be more than happy to help. You can mail your queries to 'rajatnayak1582002@gmail.com'.

Linux Permissions

Rajat Kumar Nayak — Sun, 21 Jan 2024 12:46:28 +0000

What are Linux Permissions?

Linux permissions, as the name implies, are a technique of granting different permissions to a person or group of users.

We'll go over how permissions are defined, how people are assigned permissions, and how they're used.

How to check permissions?

Run Command:
ls -a -l
The output of the command will be

Here you can see we have a directory and a file.
we will now focus on the first column which has complete information on the permissions.

The permission column has values of each ten characters.

The first place can be either a 'd' or an empty value.

'd' represents that it is a directory.

Different types of users:

User: We can say that 'User' refers to the owner of the file or directory who has created the file or directory.
Groups: It refers to a group of users who are assigned permissions to execute.

In production circumstances, groups are primarily employed. When building a project, several groups are formed based on the use case.

For a better understanding let us consider working on a project on management systems. Then we categorize users based on their role and assign permissions.

Some groups such as developers have the privilege of editing the source code and making appropriate changes.

Some users entitled as contributors can have limited access to some of the directories which are made public by the owner for review and corresponding changes.

In such scenarios, the owner adds users to groups, and permissions are assigned to the groups so that all the users entitled to the group have the privilege of exercising based on the permission.

Others: This category consists of all other users available and it has maximum possible risk and the directories and files are vulnerable to security threats and attacks.

Types of Permissions

Read: As its name suggests it allows the user or group to read the content of the directory or file.
Write: This permission allows the users to change the contents of the file and make changes.
Execute: This permission allows the users or group to execute the contents of the file. Generally bash scripts that are designed to solve specific purpose are executed.

By default, the owner of the file has read and write permissions enabled.

Why the owner doesn't have the execute permission by default?

When we have a closer look at the 1st set of permissions then we can notice that the owner lacks the execute permission.

This is because not every file is meant for execution at first. So to prevent any accidental execution that can lead to unwanted circumstances the file doesn't have execute permission.

Based on our use case we can give execute permission to the user using the command chmod. We will look into detail how we can assign permissions.

Ways to represent permissions
There are two ways to represent permissions. The first one is to represent with acronyms.
r -> read
w -> write
x -> execute

The other way to represent permission is with octal number. Each permission has a numeric value assigned to it:

r (read): 4
w (write): 2
x (execute): 1

Denoting a permission
Some examples of permissions sets:

Suppose we want to give read and write and permission:

Either we can use 'r', 'w', 'x' : rw
read -> 4 write -> 2 That eventually leads to 6

Use Case 1:

For example, a file might have read, write, and execute permissions for its owner, and only read permission for all other users. That looks like this:

Owner: rwx = 4+2+1 = 7
Group: r-- = 4+0+0 = 4
Others: r-- = 4+0+0 = 4

The chmod Command
The chmod command can be used to either assign permission or remove permission from the user or the group.

+ in the command denotes that the mentioned permission is granted to the group or user.
- represents that the mentioned permission being removed from the user or group.

Some examples of using chmod command:

Assign read and write permissions to the owner:

chmod u+rw filename

Assign execute permissions to the group:

chmod g+x filename

Assign read,write,execute to owner and read,write permission to group.

Way 1:
chmod u=rwx,g=rw filename

Way 2:
read -> 4
write -> 2
execute -> 1

chmod 760 filename

So we have pretty much covered the topic of permissions in Linux. If you have any doubt I will be more than happy to help. You can mail your queries to 'rajatnayak1582002@gmail.com'.

WebRTC Explained: Behind the Scenes of Seamless Video Calls

Rajat Kumar Nayak — Fri, 06 Oct 2023 15:18:17 +0000

Introduction

Hello Everyone!! 👋👋 We all must have come across video calling applications like Google Meet, Zoom, or Microsoft Teams app, and the list goes on. Whether you are a student or a working professional we all must have used this application for educational purposes and to conduct official meetings or get connected with distant family members.

In order to implement a video conferencing platform we need to understand the implementation of WebRTC. So let us have a deep understanding of how WebRTC works.

What is WebRTC?

WebRTC stands for Web Real-Time Communication, and it allows for instant, low-latency communication over the internet.
One most fascinating features of WebRTC is that gets easily integrated into web browsers such as Chrome, Firefox, and Safari. making it readily accessible without the need for additional plugins or installations.

How it works

Let's first understand the fundamental terms we will be using in the next steps before diving into the technical details.

NAT (Network Address Translation): The fundamental use case of NAT is to map a private IP address used in a local network into a public IP address.

You might be thinking why we are considering this?

When two hosts attempt to connect to each other, NAT devices such as routers and firewalls either block or interfere with the connection. To avoid this issue, WebRTC implementation frequently uses different strategies. In the meanwhile, we shall investigate those strategies.

So what the heck is NAT Traversal?

NAT traversal is the process of finding a way for two computers or devices to connect with each other while being separated by various NAT (Network Address Translation) routers.

So in order to facilitate NAT Traversal we make use of STUN and TURN Servers and exchange of ICE Candidates.

Okay, there's too much jargon in this line 😮‍💨😮‍💨. Let's break it down.

What are ICE Candidates?

WebRTC generates ICE candidates, which are IP addresses and port combinations that could be used to make connections. These possibilities come from a variety of sources, including local network interfaces and STUN servers. These candidates assist peers in negotiating the optimum approach to establish a connection while taking NAT setups into account.

STUN:

It is a protocol that allows a device to discover its public IP address and port when it is behind a NAT device. STUN servers are typically located on the public internet.

TURN:

If STUN is unable to make a direct connection, usually due to severe NAT configurations or firewall constraints, ICE can use a TURN server.
The TURN server serves as a relay, accepting data from one device and relaying it to another. When direct peer-to-peer connections are not possible, this relayed communication ensures that data can flow between the devices.
There is an important concept we need to know to have a clear understanding of how things work and that is Signaling.
Signaling is responsible for coordinating communication across WebRTC peers. It entails exchanging session control messages such as SDP (Session Description Protocol) offers, responses, and ICE candidates.
Signaling can be accomplished through the use of numerous protocols, such as WebSocket, HTTP, or bespoke protocols. The application developer selects the signaling protocol and server.
When a WebRTC peer sends an ICE candidate that includes STUN-derived information to the remote peer through signaling, it helps the remote peer understand how to reach it through NAT devices.

So until now, we have a brief understanding with the help of ICE Candidates two peers will be able to connect with each other.

Now taking into consideration video-calling applications we need to transmit applications from one peer to another peer.

So how this is possible? 🤔🤔

The answer is it is possible SDP.

Now what is SDP?

SDP (Session Description Protocol) is a protocol that is used to describe the media capabilities and preferences of a device.

After getting access to the device's camera and microphone, the information is transferred to the other peer through SDP and is referred to as an SDP Offer.
The SDP offer provides information about its media capabilities, network settings, and audio and video preferences.
The receiving peer responds with an SDP response, indicating its own media capabilities, preferences, and any necessary alterations to match the offer.
The SDP offer/answer exchange ensures that the media settings, codecs, and other parameters for the call are agreed upon by both peers.
It supports dynamic configuration and modifications during the call if necessary.

So upon successful SDP exchange and the peer connection is established, the browsers start streaming audio and video directly to each other. This data is encrypted using Secure Real-time Transport Protocol (SRTP) to ensure privacy and security.

Now in all these steps, you have noticed the major steps. So let us summarize the above contents into a brief.

The peers that need to get connected first undergo NAT Traversal.
It is implemented with the help of ICE Candidates and STUN/TURN Servers.
After ICE Candidates, the peers get connected and now the turn comes to exchange media information.
WebRTC asks for permission for media such as audio and video and an SDP offer is created which is then sent to the remote peer and in response it receives the SDP answer from the remote peer.
The media is transmitted in an encrypted format in order to prevent malicious activities.

In this way after an exchange of media information, the browsers start streaming audio and video directly to each other.

So I hope you enjoyed reading the article and that I was able to help you understand what goes on behind the scenes of WebRTC to some extent.