DEV Community: Sanjay R

[Boost]

Sanjay R — Sat, 11 Jan 2025 04:01:11 +0000

Playing with raw image bytes

Sanjay R — Thu, 10 Oct 2024 14:14:11 +0000

Hello, guys today let's play with image raw bytes. What I am gonna do is, manipulate the image raw pixel bytes, and make something silly, this is fun stuff to do. This article contains some theory as well as practical implementation, So let's go…

As we know an image is formed by a bunch of pixels together, pixel is nothing but a combination of RGB (Red, Green, Blue) or RGBA (Red, Green, Blue, Alpha) each with take 1 byte.

The images we view with extensions like PNG or JPG are the compressed formats of the image, PNG is lossless that PNG uses algorithms like DEFLATE to compress without losing the pixel and JPG is lossy compression that it will lose some pixel so there will be some loss in the image quality, if we want to view the image without the compression we need to convert the image to the BMP (Bitmap Image File) or there are also some other formats, if we convert to this we get the uncompressed image. But we don't need this we will extract those raw bytes and play with them and we will again convert them back to PNG or JPG.

First, let's set the client to upload images, I will set a simple react application for this

import axios from "axios";
import { useState } from "react";
import "./App.css";

function App() {
  const [img, setImg] = useState(null);
  const [pending, setPending] = useState(false);

  const handleImg = (e) => {
    setImg(e.target.files[0]);
  };

  const handleSubmit = async (e) => {
    e.preventDefault();
    if (!img) return;

    const formData = new FormData();
    formData.append("img", img);

    try {
      setPending(true);
      const response = await axios.post("http://localhost:4000/img", formData);
      console.log(response.data);

    } catch (error) {
      console.log("Error uploading image:", error);
    } finally {
      setPending(false);
    }
  };

  return (
    <div className="app-container">
      <div className="form-container">
        <form onSubmit={handleSubmit}>
          <input type="file" name="img" accept="image/*" onChange={handleImg} />
          <br />
          <button type="submit" disabled={pending}>
            {pending ? "Uploading..." : "Upload Image"}
          </button>
        </form>
      </div>
    </div>
  );
}

export default App;

So this is simple code, to upload the images, the main part is on the server side.

Now let's manually calculate the image bytes and check with the server-side code.

I have chosen the below image.

The image is from my previous article thumbnail. So this is a PNG file, if we go to the properties section, we can see the width and the height of the image. For this the width and the height are 722 x 407 which is equal to 293854 pixels, also this is not a total number of bytes, it is just a total number of pixels. As we know each pixel is either 3 or 4 bytes, RGB or RGBA. So if the above image is RGB the total bytes would be 722 x 407 x 3 = 881562 or if the image has the alpha channel, then the total bytes would be 722 x 407 x 4 = 1175416.

Let's some to the server side, I am using the node js.

There is a library called multer to parse multiform data.

app.post("/img", upload.single("img"), async (req, res) => {
  const arr = req.file.buffer
  console.log(arr.length)    //output: 30929
  res.send("success")
});

We store the image bytes in the buffer array, if we take the length of the buffer array the answer is 30929, there are these many bytes in the array, but wait the total number of bytes should be 1175416 right? What happens here is multer doesn't do some compression or anything, it just gets the image from the user and stores it in the buffer as it is, so we uploaded the PNG file, the buffer you are seeing is the same size as the PNG image size.

Now let's change the bytes in the compressed image byte.

app.post("/img", upload.single("img"), async (req, res) => {
  const arr = req.file.buffer;
  console.log("multer " + arr.length);
  fs.writeFile("output.png", arr, (err) => {
    console.log(err);
  });
  res.send("successfull");
});

I used the fs to create a new image with the existing one. So now if we change the first-byte arr[0] = 231, the image will not open.

Because the first certain bytes are reserved for the metadata, so if we change those metadata, and then the image can corrupt.

So let's jump to the 500th byte. arr[500] = 123, then write the image. But now, the image is broke, we should not directly manipulate the compressed image bytes because it can change the compression algorithm encoded data.

We need the raw bytes from the image, and then we can independently manipulate the bytes, and for that, we can use a sharp library.

npm install sharp

install the sharp, Now I will create a separate file to handle those logics,

sharp.js

export async function convert(buffer) {
  try {
    const data = await sharp(buffer).metadata();
    console.log(data)
  }catch(err){
    console.log(err)
  }
}

This is an async function, Now let's get the metadata from the png we have uploaded.

{
  format: 'png',
  size: 30929,
  width: 722,
  height: 407,
  space: 'srgb',
  channels: 4,
  depth: 'uchar',
  density: 72,
  isProgressive: false,
  hasProfile: false,
  hasAlpha: true
}

This is the metadata from the image, as we can see the last data hasAlpha: true so it has the alpha channel, so each pixel is 4 bytes.

Now let's get the raw bytes from the image.

const rawBytes = await sharp(buffer)
      .raw()
      .toBuffer({ resolveWithObject: true });

console.log(rawBytes.data.length)  //1175416

Now we can see the array length is equal to our calculation. So this image contains 1175416 bytes. Now we are free.. to change any bytes, Now the metadata is not stored in the buffer, the buffer only contains the raw bytes of the image.

Let's change only one pixel to red.

  rawBytes.data[0] = 225;    //red
  rawBytes.data[1] = 10;     //green
  rawBytes.data[2] = 10;     //blue
  rawBytes.data[3] = Math.floor(0.8 * 255);   //alpha

As we can one pixel is changed to red, we need to zoom in on the image to see the pixel change.

Now let's divide the image and change the color, the top half is yellow and the bottom half is green

const div = rawBytes.data.length / 2;
    for (let i = 0; i < rawBytes.data.length; i += 4) {
      if (i <= div) {
        rawBytes.data[i] = 240;
        rawBytes.data[i + 1] = 255;
        rawBytes.data[i + 2] = 21;
        rawBytes.data[i + 3] = Math.floor(0.8 * 255);
      } else {
        rawBytes.data[i] = 84;
        rawBytes.data[i + 1] = 135;
        rawBytes.data[i + 2] = 21;
        rawBytes.data[i + 3] = Math.floor(0.9 * 255);
      }
    }

We are incrementing the loop by 4 times because we are changing one pixel at each iteration. Now the output will be like this.

We can see the transparency in this image because the Alpha channel is set to 0.8

I forgot to tell for writing the image, we don't need fs to write a new image, we can use the sharp itself.

await sharp(rawBytes.data, {
      raw: {
        width: data.width,
        height: data.height,
        channels: data.channels,
      },
    })
      .png()
      .toFile("demo.png");

we are generating the new image with the same metadata.

Here's the full server side code,

//index.js
import express from "express";
import dotenv from "dotenv";
import multer from "multer";
import cors from "cors";
import { convert } from "./sharp.js";

const app = express();
dotenv.config();
app.use(cors({ origin: "http://localhost:5173" }));
const storage = multer.memoryStorage();
const upload = multer();

app.post("/img", upload.single("img"), async (req, res) => {
  const arr = req.file.buffer;
  await convert(arr);
  res.send("successful");
});

app.listen(process.env.PORT, () => {
  console.log("server started");
});

//sharp.js
import sharp from "sharp";

export async function convert(buffer) {
  try {
    const data = await sharp(buffer).metadata();
    console.log(data);
    //raw data
    const rawBytes = await sharp(buffer)
      .raw()
      .toBuffer({ resolveWithObject: true });
    console.log(rawBytes.data.length);
    const div = rawBytes.data.length / 2;
    for (let i = 0; i < rawBytes.data.length; i += 4) {
      if (i <= div) {
        rawBytes.data[i] = 240;
        rawBytes.data[i + 1] = 255;
        rawBytes.data[i + 2] = 21;
        rawBytes.data[i + 3] = Math.floor(0.8 * 255);
      } else {
        rawBytes.data[i] = 84;
        rawBytes.data[i + 1] = 135;
        rawBytes.data[i + 2] = 21;
        rawBytes.data[i + 3] = Math.floor(0.9 * 255);
      }
    }
    await sharp(rawBytes.data, {
      raw: {
        width: data.width,
        height: data.height,
        channels: data.channels,
      },
    })
      .png()
      .toFile("demo.png");
  } catch (error) {
    console.log(error.message);
  }
}

So this is it, We just played with those pixels. and finally this article thumbnail is made with this one line in the loop.

rawBytes.data[i] = Math.floor(Math.random()*256)

I just randomly changed each byte 😂

for the full code check out my repo: pixel-byte-manipulation

if there are any mistakes kindly do comment

Thank you!!!

Building a CRUD API with NestJS

Sanjay R — Sun, 06 Oct 2024 09:26:46 +0000

Hello guys! In this article, we can make crud APIs in NestJs. if you are unfamiliar with NestJS, let me tell you shortly.

NestJS is a framework for NodeJS to build server-side applications, under the hood by default NestJS uses the express, if required we can also change the platform to Fastify. NestJS provides an abstraction to these frameworks like Express and Fastify.

We can set up the Nest with cli, it will generate the files and basic code. but we don't need that we can set it up from scratch.

npm i --save @nestjs/core @nestjs/common rxjs reflect-metadata

run the above command to initialize the project.

this initializes the project, also we need to install additional libraries

npm install @nestjs/platform-express @nestjs/mongoose ts-node typescript nodemon

in the package.json add the scripts to run the server.

"scripts": {
    "start": "ts-node main.ts",
    "dev": "nodemon --exec ts-node main.ts"
 },

we use the ts-node to run the typescript files directly on the node.

Let's start the project,

Project structure: this is my folder structure

CRUD/
│
├── node_modules/
│
├── schemas/
│   └── todo.schema.ts
│
├── todo/
│   ├── todo.controller.ts
│   ├── todo.module.ts
│   ├── todo.service.ts
│   └── todo.types.ts
│
├── .gitignore
├── app.module.ts
├── main.ts
├── package-lock.json
├── package.json
└── tsconfig.json

before that, we need to config in the tsconfig.json

create a file tsconfig.json and add the following,

the experimentalDecorators and emitDecoratorMetadata allow us to use the Decorators in the ts, if you are unfamiliar with the decorators, let me tell you the high-level definition, "decorators are used to modifying the behavior of a class without modifying the underlying code".

create a main.ts and app.module.ts

In the app.module.ts

import { Module } from "@nestjs/common";
import { MongooseModule } from "@nestjs/mongoose";
import { TodoModule } from "./todo/todo.module";

@Module({
  imports: [
    MongooseModule.forRoot("mongodb://127.0.0.1:27017/nest"),
    TodoModule,
  ],
  controllers: [],
  providers: [],
})
export class AppModule {}

In the main.ts

import { NestFactory } from "@nestjs/core";
import { AppModule } from "./app.module";

async function Server() {
  const app = await NestFactory.create(AppModule);    
  await app.listen(4000);    //server listen on port 4000
}

Server();

The good thing about NestJS is the dependency injection, we can configure the dependency in the module, and inject it across the app and the instance will be created automatically by the nest.

let's define a schema for the database in the todo.schema.ts

import { Prop, Schema, SchemaFactory } from "@nestjs/mongoose";
import { HydratedDocument } from "mongoose";

export type TodoDoc = HydratedDocument<Todo>;

@Schema()
export class Todo {
  @Prop({ required: true })
  name!: string;

  @Prop({ required: true })
  content!: string;
}

export const TodoSchema = SchemaFactory.createForClass(Todo);

we can also define the schema without decorators, as we do in the express application but let's try with the decorators.

Let's create a separate directory for called todo, here we will handle the service, controller, and module regarding todos.

In the todo/todo.module.ts

import { Module } from "@nestjs/common";
import { MongooseModule } from "@nestjs/mongoose";
import { Todo, TodoSchema } from "../schemas/todo.schema";
import { TodoController } from "./todo.controller";
import { TodoService } from "./todo.service";

@Module({
  imports: [
    MongooseModule.forFeature([{ name: Todo.name, schema: TodoSchema }]),
  ],
  controllers: [],
  providers: [],
})
export class TodoModule {}

The MongooseModule.forFeature() registers the schema in NestJS's dependency injection system, making the TodoModel available throughout the app. For now, let's keep the controller's array and the provider's array empty, once we define the service and controllers we can add it here.

Let's make a service, create a file in todo/todo.service.ts

import { Model } from "mongoose";
import { Injectable } from "@nestjs/common";
import { InjectModel } from "@nestjs/mongoose";
import { Todo } from "../schemas/todo.schema";
import { todo } from "./todo.types";

@Injectable()
export class TodoService {
  constructor(@InjectModel(Todo.name) private todoModel: Model<Todo>) {}
  async create(data: todo): Promise<Todo> {
    try {
      const result = new this.todoModel(data);
      return await result.save();
    } catch (error) {
      throw error;
    }
  }

  async getAll(): Promise<Todo[]> {
    try {
      const data = await this.todoModel.find();
      if (!data) {
        throw new Error("No data");
      }
      return data;
    } catch (error) {
      throw error;
    }
  }

  async deleteTodo(id: string): Promise<Todo> {
    try {
      const data = await this.todoModel.findByIdAndDelete(id);
      if (!data) {
        throw new Error("error in deletion");
      }
      return data;
    } catch (error) {
      throw error;
    }
  }

  async updateTodo(id: string, upDate: todo): Promise<Todo> {
    try {
      const data = await this.todoModel.findByIdAndUpdate(id, upDate, {
        new: true,
      });
      if (!data) {
        throw new Error("can't update");
      }
      return data;
    } catch (error) {
      throw error;
    }
  }
}

These are the crud services for our API.

@Injectable(): This is a decorator, that makes our service to be used in the controller, with injection. NestJS provides a powerful Dependency Injection system that automatically resolves and manages dependencies across the application. The @Injectable() decorator marks a class as a provider, and NestJS injects these services where needed without requiring the manual instantiation.

constructor(@InjectModel(Todo.name) private todoModel: Model<Todo>) {}

//the above like Inject the model into our service, so we can use the Mongo methods in it.

remaining are just regular methods for crud operations. As we see that every method returns a Promise<Todo>, it will resolve the Todo structure promise.

Now, update the todo.module.ts Providers array,

@Module({
  imports: [
    MongooseModule.forFeature([{ name: Todo.name, schema: TodoSchema }]),
  ],
  controllers: [],
  providers: [TodoService],
})

Now the TodoService is available for injection, we will inject it into the controllers and use those methods.

Let's Now create a controller todo/todo.controller.ts

import {
  Body,
  Controller,
  Get,
  Post,
  Req,
  Res,
  Param,
  Delete,
  Patch,
} from "@nestjs/common";
import { Request, Response } from "express";
import { TodoService } from "./todo.service";
import { todo } from "./todo.types";

@Controller("todo")
export class TodoController {
  constructor(private todoService: TodoService) {}

//create a new data
  @Post("/create")
  async create(@Body() body: todo, @Res() res: Response) {
    const data = body;
    try {
      const result = await this.todoService.create(data);
      res.status(200).json(result);
    } catch (error) {
      return res.status(500).json({ error: (error as Error).message });
    }
  }

//get all the data
  @Get()
  async getAll(@Res() res: Response) {
    try {
      const result = await this.todoService.getAll();
      return res.status(200).json(result);
    } catch (error) {
      res.status(500).json({ error: (error as Error).message });
    }
  }

//delete a record
  @Delete("/delete/:id")
  async deleteTodo(@Param("id") id: string, @Res() res: Response) {
    try {
      const result = await this.todoService.deleteTodo(id);
      res.status(200).json(result);
    } catch (error) {
      res.status(500).json({ error: (error as Error).message });
    }
  }

//update the record
  @Patch("/update/:id")
  async updateTodo(
    @Param("id") id: string,
    @Body() body: todo,
    @Res() res: Response
  ) {
    try {
      const result = await this.todoService.updateTodo(id, body);
      res.status(200).json(result);
    } catch (error) {
      res.status(500).json({ error: (error as Error).message });
    }
  }
}

Controllers will handle the request and response.

with the @Controller decorator, it indicates that the below class will handle the incoming requests. when we give the @Controller("todo") the API will prefixed with todo, Now we can write a code for curd routes.

As we can see the Request and Response are from the express, we are using the @Req and @Res decorators with the Request and Response types.

Now we can handle the request and the response like we handle in express.

Now update the todo.module.ts

@Module({
  imports: [
    MongooseModule.forFeature([{ name: Todo.name, schema: TodoSchema }]),
  ],
  controllers: [TodoController],
  providers: [TodoService],
})

these are the API endpoints

--   Get: localhost:4000/todo/
--  POST: localhost:4000/todo/create
-- Patch: localhost:4000/todo/update/:id
--Delete: localhost:4000/todo/delete/:id

You can test it in the insomnia or postman

For the full code check out my repo: nestjs-crud

if there are any mistakes, comment below

Thank You!!!

I made a chatbot with gemini API that pretends to think!

Sanjay R — Sun, 22 Sep 2024 10:20:58 +0000

Hello guys, we know that OpenAI's o1 model appears to think before giving answers to users. I don't know how it works under the hood, but I've created something fun that mimics this "thinking…" behavior using the Gemini API. Let me clarify - this isn't a serious project! It's purely for entertainment, and the logic behind it is a bit silly.

The chatbot I created takes time to respond based on the difficulty of the question.

For example,

if the question is "Write a program to add two numbers" will take about 1 second, check the below image

However, if the question is like "Write a program to create an HTTP server in x86 assembly" it will take longer to respond compared to the simpler question.

To be honest, this chatbot doesn't actually "think" before responding. It just generates multiple answers under the hood and analyzes them, but we can pretend it's thinking by showing it in the UI! 😂

Also, just to be clear, this is purely for fun and not how OpenAI's o1 model works.

How it works:

Let me explain the logic under the hood

First I send a utility text of "I need you to rate the question from the programmer's point of view from 1 to 5 hardness from low to high", this will be added to Gemini along with the user prompt. Gemini responds with a difficulty rating. Based on this rating, for example, if the rating is 4, I generate four different answers to the same question and store them in an array.

Finally, I added another utility text of "This is the array of answers and I need you to analyze them and generate a final correct output".
This process generates a final answer, which is returned to the user. I also calculate how much time it takes for the system to generate and analyze the answers.

You can check out the code repo here: so-called-think

Clone the repo, create a .env file, add your Gemini API key, and run it.

If you notice any mistakes, feel free to leave a comment.

Thanks, everyone!

Text Detection in Images Using AWS Rekognition and Node.js

Sanjay R — Mon, 26 Aug 2024 08:51:04 +0000

Hey everyone! In this article, we'll be creating a simple application to perform image text detection using AWS Rekognition with Node.js.

What is AWS Rekognition?

Amazon Rekognition is a service that makes it easy to add image and video analysis to your applications. It offers features like text detection, facial recognition, and even celebrity detection.
While Rekognition can analyze images or videos stored in S3, for this tutorial, we'll be working without S3 to keep things simple.
We'll be using Express for the backend and React for the frontend.

First Steps

Before we start, you'll need to create an AWS account and set up an IAM user. If you already have these, you can skip this section.

Creating IAM user

Log in to AWS: Start by logging into your AWS root account.
Search for IAM: In the AWS console, search for IAM and select it.
Go to the Users section and click Create User.
Set the user name, and under Set Permissions, choose Attach policies directly.
Search for and select the Rekognition policy, then click Next and create the user.
Create Access Keys: After creating the user, select the user, and under the Security credentials tab, create an access key. Be sure to download the .csv file containing your access key and secret access key.
For more detailed instructions, refer to the official AWS documentation: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html

Configuring aws-sdk

Install AWS CLI: Install the AWS CLI on your system.
Verify Installation: Open a terminal or command prompt and type aws --version to ensure the CLI is installed correctly.
Configure the AWS CLI: Run aws configure and provide the access key, secret access key, and region from the .csv file you downloaded.

Project Directory

my-directory/
│
├── client/
│   └── src/
│       └── App.jsx
│   └── public/
│   └── package.json
│   └── ... (other React project files)
│
└── server/
    ├── index.js
    └── rekognition/
        └── aws.rek.js

Setting up frontend

npm create vite @latest . -- --template react
it will create the react project in the client folder.

In the App.jsx

import { useState } from "react";

function App() {
  const [img, setImg] = useState(null);

  const handleImg = (e) => {
    setImg(e.target.files[0]);  // Store the selected image in state
  };

  const handleSubmit = (e) => {
    e.preventDefault();
    if (!img) return;

    const formData = new FormData();
    formData.append("img", img);
    console.log(formData);      // Log the form data to the console
  };

  return (
    <div>
      <form onSubmit={handleSubmit}>
        <input type="file" name="img" accept="image/*" onChange={handleImg} />
        <br />
        <button type="submit">Submit</button>
      </form>
    </div>
  );
}

export default App;

Let's test this out by ensuring the image is logged to the console after submitting.

Now, Let's move to backend and start making the soul, for this project.

Initializing the backend

in the server folder

npm init -y
npm install express cors nodemon multer @aws-sdk/client-rekognition
I have created a separate folder for rekognition, to handle analyzing logic and create a file inside that folder.

//aws.rek.js

import {
  RekognitionClient,
  DetectTextCommand,
} from "@aws-sdk/client-rekognition";

const client = new RekognitionClient({});

export const Reko = async (params) => {
  try {
      const command = new DetectTextCommand(
          {
              Image: {
                  Bytes:params  //we are using Bytes directly instead of S3
              }
        }
    );
    const response = await client.send(command);
    return response
  } catch (error) {
    console.log(error.message);
  }
};

Explanation

We initialize a RekognitionClient object. Since we've already configured the SDK, we can leave the braces empty.
We create an async function Reko to process the image. In this function Initalize a DetectTextCommand object, which takes an image in Bytes.
This DectedTextCommand is specifically used for text detection.
The function waits for a response and returns it.

Creating the API

In the server folder, create a file index.js or what ever name you want.

//index.js

import express from "express"
import multer from "multer"
import cors from "cors"
import { Reko } from "./rekognition/aws.rek.js";

const app = express()
app.use(cors())
const storage = multer.memoryStorage()
const upload = multer()
const texts = []
let data = []

app.post("/img", upload.single("img"), async(req,res) => {
    const file = req.file
    data = await Reko(file.buffer)
    data.TextDetections.map((item) => {
        texts.push(item.DetectedText)
    })
    res.status(200).send(texts)
})

app.listen(3000, () => {
    console.log("server started");
})

Explanation

Initializing the express and starting the server.
We are using the multer to handle the multipart form data, and storing it temporarily in the Buffer.
Creating the post request to get the image from the user. this is an async function.
After the user uploads the image, the image will be available in the req.file
This req.file contains some properties, in that there will be a Buffer property that holds our image data as an 8-bit buffer.
We need that so we are passing that req.file.buffer to the rekognition function. after analyzing it, the function returns the array of objects.
We are sending the texts from those objects to the user.

Coming back to frontend

import axios from "axios";
import { useState } from "react";
import "./App.css"; 

function App() {
  const [img, setImg] = useState(null);
  const [pending, setPending] = useState(false);
  const [texts, setTexts] = useState([]);

  const handleImg = (e) => {
    setImg(e.target.files[0]);
  };

  const handleSubmit = async (e) => {
    e.preventDefault();
    if (!img) return; 

    const formData = new FormData();
    formData.append("img", img);

    try {
      setPending(true);
      const response = await axios.post("http://localhost:3000/img", formData);
      setTexts(response.data);
    } catch (error) {
      console.log("Error uploading image:", error);
    } finally {
      setPending(false);
    }
  };

  return (
    <div className="app-container">
      <div className="form-container">
        <form onSubmit={handleSubmit}>
          <input type="file" name="img" accept="image/*" onChange={handleImg} />
          <br />
          <button type="submit" disabled={pending}>
            {pending ? "Uploading..." : "Upload Image"}
          </button>
        </form>
      </div>

      <div className="result-container">
        {pending && <h1>Loading...</h1>}
        {texts.length > 0 && (
          <ul>
            {texts.map((text, index) => (
              <li key={index}>{text}</li>
            ))}
          </ul>
        )}
      </div>
    </div>
  );
}

export default App;

Using Axios to post the image. and storing the response in the text's state.
Displaying the texts, for now, I am using the index as the Key, but it is not encouraged to use the Index as the key.
I have also added some additional things like loading state and some styles.

Final Output

After clicking the "Upload Image" button, the backend processes the image and returns the detected text, which is then displayed to the user.

For the complete code, check out my: GitHub Repo

Thank You!!!

Follow me on: Medium, GitHub, LinkedIn, X, Instagram

Building a Simple Real-Time Chat Application with Socket.IO

Sanjay R — Sat, 13 Jul 2024 17:36:46 +0000

Let's build a simple chat app using Socket.IO with chat rooms. Here's the live site link Simple Chat. Don't judge the UI 😅. I know it is not good and it is not even responsive. For now, let's focus on the concepts and functionality.

Before diving into the code, let's cover some concepts about WebSockets and Socket.IO.

WebSockets
WebSockets is a protocol that provides full-duplex communication, meaning it allows a connection between the client and the server, enabling real-time, two-way communication.

Socket IO
Socket.IO is a JavaScript library that simplifies the use of WebSockets. It provides an abstraction over WebSockets, making real-time communication easier to implement.

Let's dive into the coding part. Make sure Node.js is installed on your computer.

Folder Structure

server/
├── node_modules/
├── public/
│   ├── chat.html
│   ├── createRoom.html
│   ├── index.html
│   ├── JoinRoom.html
│   ├── script.js
│   └── styles.css
├── .env
├── .gitignore
├── package-lock.json
├── package.json
└── server.js

Initialize the Project

npm init -y

install the required packages, first, let's install the express and nodemon

npm install express nodemon

let's configure the nodemon in package.json

 "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "start": "node server.js",
    "dev":"nodemon server.js"
  },

let's set up our application, and create a file name server.js or index.js as you wish.

const express = require("express");
const env = require("dotenv");
const app = express();
const port = process.env.PORT || 3000;
const { createServer } = require("node:http");  
const server = createServer(app);

//when the user enters the localhost:3000, the request is handled by this route handler
app.get('/', (req, res) => {           //get request
  res.send('<h1>Hello world</h1>');    
});

server.listen(port, () => {               //listen on the port
  console.log(`server started ${port}`);
});

socket.io requires direct access to the underlying HTTP server to establish WebSocket connections. so we require the createServer from the HTTP module. and listen on the port.

npm run dev      //start the server

we are not going to use any frontend frameworks, so let's serve the HTML static file from the server.
create a folder public and create an index.html inside the public folder

const path = require("path");    //require the path module

app.use(express.static("public"));   //make the folder as static 

//in the app.get let's send the html
app.get("/", (req, res) => {
  res.sendFile(path.join(__dirname, "public", "index.html"));
});

server.listen(port, () => {               //listen on the port
  console.log(`server started ${port}`);
});

Now if you open the localhost, you can see the HTML page.

let's integrate socket io

npm install socket.io

const { Server } = require('socket.io');  //require 
const io = new Server(server);  //initialize  the new instance of the socket.io by passing the server object

//listen on the connection event for incoming sockets
io.on('connection', (socket) => {    
  console.log('user connected');
});

Now in the index.html

<body>
    <ul id="messages"></ul>
    <form id="form" action="">
      <input id="input" autocomplete="off" /><button>Send</button>
    </form>

<script src="/socket.io/socket.io.js"></script>
<script>
  const socket = io();   //establish a connection
</script>
</body>

the above script is used to enable real-time communication between the client and the server.
when you open localhost you can see the user connected as the output.

Emit some events

The main idea of socket.io is that you can send and receive any data you want. JSON and Binary data are also supported.

Let's make it so that when the user sends the message, the server gets it as a chat message event.

in the index.html

<script src="/socket.io/socket.io.js"></script>
<script>
  const socket = io();

  const form = document.getElementById('form');
  const input = document.getElementById('input');

  form.addEventListener('submit', (e) => {
    e.preventDefault();
    if (input.value) {
      socket.emit('message', input.value);
      input.value = '';
    }
  });
</script>

and print out the message received from the client.

io.on('connection', (socket) => {
  socket.on('message', (msg) => {
    console.log('message: ' + msg);
  });
});

Broadcasting
Now we have received the message from the client, then we need to broadcast it to all the users.

socket.io gives the io.emit() method, to emit the events or messages.

io.on('connection', (socket) => {
  socket.on('message', (msg) => {
    io.emit('message', msg);
  });
});

So, now the messages from the client are broadcasted to all the users, we need to capture the message and include it in the page.

<script src="/socket.io/socket.io.js"></script>
<script>
  const socket = io();

  const form = document.getElementById('form');
  const input = document.getElementById('input');
  const messages = document.getElementById('messages');

  form.addEventListener('submit', (e) => {
    e.preventDefault();
    if (input.value) {
      socket.emit('message', input.value);
      input.value = '';
    }
  });

  socket.on('message', (msg) => {
    const item = document.createElement('li');
    item.textContent = msg;
    messages.appendChild(item);
  });
</script>

the output image is shown below.

We have done the basic chat application, let's make chat rooms.

How do chat rooms work?

socket io provides functionality for the rooms, so the event is emitted, and only the room members can see the message.

Now I am going to create a 3 html file in the public folder. chat.html , createRoom.html , joinRoom.html

I am going to shift the code from the index.html to the chat.html.

In the index.html

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Index</title>
  </head>
  <body>
    <div class="main">
      <div class="text">
        <h1>Chat</h1>
      </div>
      <button id="create-room">Create Room</button>
      <button id="join-room">Join Room</button>
    </div>
    <script>
      const createRoom = document.getElementById("create-room");
      const joinRoom = document.getElementById("join-room");

      createRoom.addEventListener("click", () => {
        location.href = "./createRoom.html";
      });

      joinRoom.addEventListener("click", () => {
        location.href = "./JoinRoom.html";
      });
    </script>
  </body>
</html>

In the createRoom.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Create Room</title>
</head>
<body>
    <div class="form-container">
        <h1>Create Room</h1>
        <form action="/create" method="post">
            <input type="text" placeholder="Enter the room name" name="roomName" required>
            <input type="submit" value="Create Room">
        </form>
    </div>
</body>
</html>

In the JoinRoom.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Join Room</title>
</head>
<body>
    <div class="form-container">
        <h1>Join Room</h1>
        <form action="/joinRoom" method="post">
            <input type="text" placeholder="Enter the room name" name="Join" required>
            <input type="submit" value="Join Room">
        </form>
    </div>
</body>
</html>

So let me explain what I have done in this file, in the index.html I have made a UI with two buttons to create a room or join a room, it will be redirected to the respective files when it is clicked.

Creating the room
In createRoom.html, there's a form with the POST method, and the action is set to /create. When submitted, the request is sent to the server.

<form action="/create" method="post">
  <input type="text" placeholder="Enter the room name" name="roomName" required>
  <input type="submit" value="Create Room">
</form>

in server.js

app.use(express.urlencoded({ extended: true }));  //parse the data

const rooms = [];    //to store the rooms

app.post("/create", (req, res) => {
  const roomName = req.body.roomName;   //get the room name from the body
  if (rooms.includes(roomName)) {       //see if the room already exists
    res.send("room already exits");
  } else if (roomName) {
    rooms.push(roomName);               //push the  room name to the array
    res.redirect(`/chat.html?room=${roomName}`);   //redirect to the chat.html with the room name
  } else {
    res.redirect("/createRoom.html");
  }
});

we are not using any databases to store the room names, for this tutorial let's use an array to store the room names.

let's do the similar functionalities for the join room

in server.js

app.post("/joinRoom", (req, res) => {
  const roomName = req.body.Join;
  if (rooms.includes(roomName)) {
    res.redirect(`/chat.html?room=${roomName}`);
  } else {
    res.send("enter the valid room");
  }
});

Let's make a separate JavaScript file script.js in the public. I have copied all the JS from the chat.html to a separate file.

script.js

const socket = io();

function getUrlname(name) {
  const urlPara = new URLSearchParams(window.location.search);
  return urlPara.get(name);
}

const roomName = getUrlname("room");
if (roomName) {
  socket.emit("join room", { roomName });
} else {
  console.log("room is not valid");
}

const form = document.getElementById("form");
const input = document.getElementById("input");
const messages = document.getElementById("messages");

form.addEventListener("submit", (e) => {
  e.preventDefault();
  if (input.value) {
    socket.emit("message", { roomName, message: input.value });
    input.value = "";
  }
});

socket.on("message", ({ message }) => {
  if (!message) {
    alert("room is not valid");
  } else {
    const item = document.createElement("li");
    console.log(message);
    item.textContent = message;
    messages.appendChild(item);
  }
});

// Connection state recovery
const disconnectBtn = document.getElementById("disconnect-btn");

disconnectBtn.addEventListener("click", (e) => {
  e.preventDefault();
  if (socket.connected) {
    disconnectBtn.innerText = "Connect";
    socket.disconnect();
  } else {
    disconnectBtn.innerText = "Disconnect";
    socket.connect();
  }
});

In the server, after the create room or join room is handled it is redirected to the chat.html page, we have also sent the room name in the search params.

In the above code, we are extracting the room name from the search params using URLSearchParams and checking if the room is valid.

Each message from the client is emitted with the message and with the room name. With the help of this we can now check from which room the message is from.

in server.js

io.on("connection", (socket) => {
  socket.on("join room", ({ roomName }) => {
    if (rooms.includes(roomName)) {
      socket.join(roomName);
    }
  });

  socket.on("message", ({ roomName, message }) => {
    if (rooms.includes(roomName)) {
      io.to(roomName).emit("message", { message });
    } else {
      socket.emit("message", { message: false });
    }
  });
});

socket provides a join method to join the room, and we check where the message is from, and the message is broadcasted to only that room member. this is done by the io.to(roomName).emit("message",{message})

Now open multiple tabs on the browser create different rooms join in that room and check if this works.

here's the full code for server.js

const express = require("express");
const app = express();
const env = require("dotenv");
const path = require("path");
const { Server } = require("socket.io");
const { createServer } = require("node:http");
const server = createServer(app);
env.config();
const port = process.env.PORT;
app.use(express.static("public"));
app.use(express.json());
app.use(express.urlencoded({ extended: true }));

const io = new Server(server, {
  connectionStateRecovery: {},
});

const rooms = [];

app.get("/", (req, res) => {
  res.sendFile(path.join(__dirname, "public", "index.html"));
});

app.post("/create", (req, res) => {
  const roomName = req.body.roomName;
  if (rooms.includes(roomName)) {
    res.send("room already exits");
  } else if (roomName) {
    rooms.push(roomName);
    res.redirect(`/chat.html?room=${roomName}`);
  } else {
    res.redirect("/createRoom.html");
  }
});

app.post("/joinRoom", (req, res) => {
  const roomName = req.body.Join;
  if (rooms.includes(roomName)) {
    res.redirect(`/chat.html?room=${roomName}`);
  } else {
    res.send("enter the valid room");
  }
});

io.on("connection", (socket) => {
  socket.on("join room", ({ roomName }) => {
    if (rooms.includes(roomName)) {
      socket.join(roomName);
    }
  });

  socket.on("message", ({ roomName, message }) => {
    if (rooms.includes(roomName)) {
      io.to(roomName).emit("message", { message })
    } else {
      socket.emit("message", { message: false });
    }
  });
});

server.listen(port, () => {
  console.log(`server started ${port}`);
});

we have done it!! and finally, let me tell one concept that is called connection state recovery.

In the server code, there is one line

const io = new Server(server, {
  connectionStateRecovery: {},
});

and in the client script.js

// Connection state recovery
const disconnectBtn = document.getElementById("disconnect-btn");

disconnectBtn.addEventListener("click", (e) => {
  e.preventDefault();
  if (socket.connected) {
    disconnectBtn.innerText = "Connect";
    socket.disconnect();
  } else {
    disconnectBtn.innerText = "Disconnect";
    socket.connect();
  }
});

it is used to handle the disconnections by pretending that there was no disconnection.
this feature will temporarily store all the events that are sent by the server and will try to restore the state when the client reconnects.

This is it!!!

here's the link for the source code - GitHub
here's the live site link - Simple Chat

Thank You!!!

Follow me on: Linkedin, Medium, Instagram, X

Commonly Used npm Flags

Sanjay R — Thu, 23 May 2024 18:21:39 +0000

Commonly Used npm Flags

-g (global)

Cmd: npm install -g <package-name>
Usage: Installs the package globally on your system, making it available from any directory. This is useful for tools and utilities you want to use across different projects.
Example: npm install -g create-react-app allows you to use the create-react-app command from anywhere on your machine.

-p (prefix)

Cmd: npm install -p <directory> <package-name>
Usage: Installs the package to a specific directory rather than the default node_modules in the current project. This can be used to manage packages for different environments or users.
Example: npm install -p /usr/local <package-name> installs the package to /usr/local.

-d (save-dev)

Cmd: npm install -D <package-name> or npm install --save-dev <package-name>
Usage: Adds the package as a development dependency. Development dependencies are only needed during development and testing, not in production.
Example: npm install -D jest adds jest to your devDependencies in package.json

{
  "dependencies": {
    "express": "^4.17.1"
  },
  "devDependencies": {
    "jest": "^27.0.6"
  }
}

Other Common npm Flags

--save(-S)

Cmd: npm install -S <package-name> or npm install <package-name>
Usage: Adds the package as a dependency in your package.json. This is the default behavior when you run npm install <package-name>
Example: npm install express will add express to the dependencies in package.json

--save-optional (-O)

Cmd: npm install -O <package-name>
Usage: Adds the package as an optional dependency. These dependencies are not critical for the project to function but can add optional features.
Example: npm install -O fsevents adds fsevents to optionalDependencies.

--save-exact (-E)

Cmd: npm install -E <package-name>
Usage: Installs the exact version specified without updating to newer versions.
Example: npm install -E lodash installs the exact version listed, e.g., lodash@4.17.21

--production

Cmd: npm install --production
Usage: Installs only the dependencies listed under dependencies and skips devDependencies. Useful for deployment.
Example: npm install --production ensures only necessary packages for running the app are installed.

Summary

-g: Global installation
-p: Specify installation directory
-d: Development dependencies
--save: Default dependency installation (often implicit)
-O: Optional dependencies
-E: Exact version installation
--production: Production-only installation

These flags help configure your environment and dependencies according to your project needs.

Thank You!!!

Follow on: Linkedin, Instagram, Medium

Understanding Strings, toString(), and equals() in Java

Sanjay R — Fri, 17 May 2024 13:00:51 +0000

Let's dive into the topic

Strings

A string is a class
It is from a package called Java.lang
String is immutable
The object is implicitly Created

Let's see some examples:

//creating objects for string
String s = new String();
System.out.println(s);

output:
//empty

In the above example, the output will be empty. But if we print an object of another class, some hashcode will be printed. For a String, Why is it not printing anything? Let's see about that.

Let's see some internal work.

Whenever an object is printed, Java internally calls the toString method which returns a string format.

//toString code

public class Object{

    public String toString(){
      return getClass().getName() + "@" + Integer.toHexString(hashCode());
    }

}

This is the internal code for the toString method. We can also override this method. The Object class is the parent class of every class. We can override the Object class methods unless they have the final or static keyword.

Let's get to the point.

In the String class, the toString method is written as:

// This object (already a string!) is returned. 

public String toString() {
    return this;
}

It just returns the string itself. Now, since the string is empty, it prints the empty string.

Overriding the toString method

public class Test {
    @Override
    public String toString(){
        return "Espresso";
    }
    public static void main(String[] args) {
        Test a = new Test();
        System.out.println(a);
    }
}

output:
Espresso

We have overridden the toString method to print espresso every time we call toString.

equals()

It checks whether the two objects are the same. But if we think logically, two objects can't be the same.

Here's an analogy: Imagine there are two phones from the same brand and model. Their properties may be the same, but that doesn't mean they are the same. They are two different physical objects. Likewise, we can't compare the two objects because the two objects are different and it is stored in separate memory locations. But we can compare their properties, and also we override the equals() method.

However, for the String class, the equals() method is overridden to compare the character sequences of the strings, rather than the object reference. This means that two different String objects with the same character sequence will be considered equal when compared using the equals() method.

So, this is it.

I hope it is useful.

Thank You!!!

Authentication and Authorization in Web Applications

Sanjay R — Sat, 27 Apr 2024 10:16:07 +0000

Authentication

It Provides you with who you are, like login into the website, and the website verifies the email and password.

Authentication is like showing your ID to enter a secure building. When you log in to a website or app, it checks your username and password to confirm that you are who you say you are. Once Verified, you’re granted access to your account, just like unlocking a door with a key.

Authentication Under the Hood

It happens on the server, the client sends the login request to the server and the server verifies it and sends a response, in that response there will be a session ID, the session ID is nothing but a unique ID stored as a cookie on your local machine, it is used for server to remember who you are, without session-ID server can’t remember, so every time you make the request the session-ID is sent along with the request to the server.

Authorization

Permission to do this, like users and admin, it’s role-based

Once you’re authenticated the server authorizes you

Authorization is like having different levels of access in a building. Once you’re inside, you might have permission to enter certain rooms, but some rooms are restricted for you. it’s a role-based thing, if you are the manager of the hotel, you have access to certain rooms that are restricted to the client. Similarly, in a web application, authorization determines what actions you’re allowed to take.

Authorization Under the Hood

It is going to send a request to the server, for example, on a website let’s say you want to delete something that you are not permitted to, when you click the button the request is sent to the server along with the session ID. The server verifies that if the user has permission to delete this if the user doesn’t have permission, it will send the response “Not permitted” or something.

JS Bites: Chapter 4 - Fetch API

Sanjay R — Sun, 14 Apr 2024 06:11:25 +0000

Fetch API

fetch()is a function used to send the requests to APIs and retrieve data.

const response = fetch('someapi.com');

How Does It Work?

When you pass the URL to fetch(), JavaScript creates a request object internally.

// Internal working
const request = new Request('someapi.com', {
   method: 'GET'   
});

const response = fetch(request);

The above demonstrates the initial workings of fetch(), abstracting away the complexities for you.

Response Object

fetch() is asynchronous, returning a promise that resolves later.

const url = "https://jsonplaceholder.typicode.com/todos/1"; 

async function fetchData() {
    const response = await fetch(url);
    console.log(response);
}

fetchData();

This will output the response object. To obtain data, you need to convert the response object into JSON.

const url = "https://jsonplaceholder.typicode.com/todos/1";

async function fetchData() {
    const response = await fetch(url); 
    const data = await response.json();
    console.log(data.title);
}

fetchData();

GET Requests Parameters

For specific data, parameters can be added to the URL, but this method isn’t secure, For safety, include parameters in the request object.

const url = 'someUrl';

async function getData() {
    const request = new Request(url, {
        headers: {
            'Authorization': 'token'    
        }
    });

    const response = await fetch(request);
    const data = await response.json();
    console.log(data);
}

getData();

Error Handling

Handle errors properly. If the request is successful but the data retrieval fails. it’s still considered a success.

let’s handle the error if the request is made successfully but the data retrieval fails. Check the status code for error handling.

try {
    const response = await fetch(request);
    if (response.status === 200) {
        const data = await response.json(); 
        console.log('Success', data);
    } else {
        console.log('Server error', data.error.message);
    }
} catch(error) {
    console.log(error);
}

POST Request

When you send data to a server, like when submitting a form, you can use the POST method. To do this with fetch(), you need to include the data you want to send in the request body.

const request = new Request(url, {
    headers: {...},
    body: {...}
});

For more detailed information on the Fetch API, refer to the MDN documentation.

Comment your thoughts…

See you in the next chapter…

JS Bites: Chapter 3 - Async/Await vs Promise

Sanjay R — Fri, 12 Apr 2024 09:22:22 +0000

In promise, if you remember there will be a Promise Creator the one who creates the promise and the one who receives those promises and does operations on them.

what if we can synchronously write the asynchronous code? instead of nesting code.

Here comes the async and await…

function getData(){
    return new Promise(...)
}

const result = getData(); //this will return the promise not the actual data

JS thinks this is the Synchronous operation and doesn't wait for the function async operation unless we put await

const result  = await getData()
console.log(result)

//but this will be an error because 
//await is only valid in async functions and the top-level bodies of modules

correct code is,

function getData(){
  return new Promise(res=>{
    setTimeout(function() {res("hi")}, 1000);
  })
}

async function start(){
  const result  = await getData()
  console.log(result)
}

start()

wrap the await in the async function

these two functions do the same work, but which one do you prefer, obviously the first one because that is simple.

Important points in async/await

Async and Await must be used together.
Await can be used without async with JavaScript modules & Chrome DevTools Console.
Async and Await only affect the Promise receiver
Any function can be converted to an async
All async function returns a promise by default.
Error Handling try/catch

Error Handling try/catch

function getData(a,b){
  return new Promise((resolve,reject)=>{
    setTimeout(function() {
      if(a*b<=10){
        resolve("less than 10")
      }else{
        reject("this is less than 10")
      }
     }, 2000);
  })
}

async function start(a,b){
  try{
    const result  = await getData(a,b)
    console.log(result)
  }catch(error){
    console.log(`error ${error}`)
  }
}

start(2,10)

Conclusion
Why use async/await?

It makes code more readable and enables the writing of code that uses asynchronous functions but looks like synchronous code.

Comment your thoughts…

See you in the next Chapter…

JS Bites: Chapter 2 - Async Made Easy with Promises

Sanjay R — Wed, 10 Apr 2024 12:59:00 +0000

Promises
To overcome the issue of Callback Hell, we use Promises. In Promises, there are two main actors: the "Promise Maker" and the "Promise Receiver."

Promise Maker
The Promise Maker is the one who creates and returns the promise. It initiates an action that will eventually complete, either successfully or with an error.

Example:

function makePromise() {
    return new Promise();
}

Promise Receiver
The Promise Receiver is the one who receives the promise from the Maker and performs actions based on its outcome.

Example:

let promise = makePromise();

When a Promise is initially created, it enters the "pending" state. Then, it can either be fulfilled (completed) or rejected (encounters an error).

Creating and Handling Promises

function isOdd(number) {
    return new Promise((resolve, reject) => {
        if (number % 2 !== 0) {
            resolve(true);  // Resolve if the number is odd
        } else {
            reject(false);  // Reject if the number is even
        }
    });
}

const promise = isOdd(4)
    .then((result) => console.log(result))  // Handling fulfillment
    .catch((error) => console.log(error));  // Handling rejection

Chaining Promises
Promises can be chained together, allowing for sequential execution of asynchronous operations. This is achieved by returning a Promise from within a .then() or .catch() block.

Example:

function asyncTask() {
    return new Promise((resolve, reject) => {
        // Asynchronous task
        resolve('Data from async task');
    });
}

asyncTask()
    .then((data) => {
        console.log(data);
        return anotherAsyncTask();  // Return another Promise
    })
    .then((result) => {
        console.log(result);
    })
    .catch((error) => {
        console.error(error);
    });

Reject and .catch()
These are used to handle errors that occur during the execution of a Promise. If a Promise encounters an error, it gets rejected, and .catch() is used to handle that error.

Example:

const promise = isOdd(4)
    .catch((error) => console.log(error));  // Handling rejection

Finally()
The finally() method is used to execute code after the Promise is settled, regardless of whether it was fulfilled or rejected. It's commonly used for cleanup tasks.

const promise = isOdd(4)
    .then((result) => console.log(result))
    .catch((error) => console.log(error))
    .finally(() => console.log("Promise execution completed"));  // Executes regardless of fulfillment or rejection

See you in the next chapter...