DEV Community: rutvikjani

How To install feed from your live stream in node-media-server.

rutvikjani — Fri, 08 Dec 2023 06:20:06 +0000

Introduction:

In the dynamic world of digital content, live streaming has emerged as a powerful and engaging way to connect with audiences in real-time. Whether you're a content creator, educator, or simply someone looking to share experiences as they happen, setting up a reliable live streaming infrastructure is crucial. In this comprehensive guide, we'll walk you through the process of creating a Node Media Server, seamlessly pushing your live stream with FFmpeg, and even show you how to download the feed in the widely used HLS (HTTP Live Streaming) format.

Prerequisites

Node.js : Make Sure to install the node.js in your system. If you do not already installed node.js then you can download from the link -

nodejs.org
FFmpeg : Download the FFmpeg module in your pc. You can download FFmpeg from -

ffmpeg.org

Modules Used

Node-Media-Server

Node Media Server (NMS) is a powerful and extensible media server built on top of Node.js, a server-side JavaScript runtime. Developed to meet the demands of modern live streaming applications, NMS provides support for various streaming protocols, including Real-Time Messaging Protocol (RTMP), HTTP Live Streaming (HLS), and more.

You can download through vscode terminal by following command

npm install node-media-server

fluent-ffmpeg

fluent-ffmpeg is a Node.js module that provides a fluent interface for working with FFmpeg. It allows developers to use FFmpeg capabilities in a more user-friendly and JavaScript-oriented manner, making it easier to construct complex multimedia processing pipelines.

You can download through vscode terminal by following command

 npm install fluent-ffmpeg

Moment

Moment.js is a lightweight and versatile JavaScript library for parsing, validating, manipulating, and formatting dates and times. It simplifies working with dates in JavaScript, providing a consistent and user-friendly API for various date-related operations.

You can download through vscode terminal by following command

 npm install moment

Now We are all set to start our node-media-server project.

Step 1 : Configure our node-media-server.

First you need to import the node-media-server. By using

const nms = require('node-media-server)

Then we need to write the configurations for node-media server

 const config = {
  rtmp: {
    port: 1935,
    chunk_size: 60000,
    gop_cache: true,
    ping: 60,
    ping_timeout: 30,
  },
  http: {
    port: 8000,
    mediaroot: "./media",
    allow_origin: "*",
  },
};

These are the basic configurations for rtmp(real time messaging protocol) feed and hls(http live streaming) feed. The rtmp configs contains of the -

port

port number 1935 which is by default port number. You can change it to the available port number.

chunk_size

This parameter represents the size of the data chunks that will be used during streaming. Here, it is set to 60000 bytes.
Here, chunk_size: 60000 indicates that the server will use a chunk size of 60,000 bytes for RTMP streaming. Adjusting this value allows you to fine-tune the trade-off between latency, efficiency, and other factors based on the specific requirements of your streaming application. Different streaming platforms or protocols may have their own guidelines or default values for chunk sizes.

gop_cache

The Group of Pictures (GOP) is a key concept in video compression. It represents a group of consecutive frames in a video stream where the inter-frame compression is applied. The GOP structure typically consists of three types of frames: I-frames (keyframes), P-frames (predicted frames), and B-frames (bidirectional frames).

Setting gop_cache to true in the context of an RTMP server, such as with the configuration for node-media-server, means that the server will cache GOP frames. Caching GOP frames can have several benefits: Improved Seeking Performance, Reduce Bandwidth Usage, Stability in Video Playback.

ping

It sets the interval for sending ping messages to clients, in seconds. Here, it's set to 60 seconds.

ping_timeout

The timeout duration for ping responses from clients, in seconds. If a client doesn't respond within this timeout, it may be considered disconnected. Here, it's set to 30 seconds.

http

This configurations is for http live streaming it contains of -

port

Here also the port number is by default set to 8000 and we can change it as per our requirements

mediaroot

It is set to "./media" that means it indicates that the media files are expected to be located in the "./media" directory. The media server will serve content from this directory.

For example

if you have a file named "video.mp4" in the "./media" directory, it would be accessible through the media server at a URL like http://localhost:8000/video.mp4.

So, the mediaroot setting allows you to define the base directory where your media files are stored, and the media server uses this information to serve content to clients or to incorporate it into a streaming workflow. This setting is crucial for the media server to know where to look for and serve media files.

allow_origin

It defines the value of the Access-Control-Allow-Origin header, which controls which domains can make requests to the server. Here, it's set to "*" which means any domain is allowed.

Now we need to make an instance of the node-media-server -

const nms = new NodeMediaServer(config);

Now our node-media-server is set and now the second step is to download the feed on specific time-interval.

Step : 2

Take your static live url in a variable and set the directory in which you want to save your feed recordings -

const inputURL = "Your RTMP url";
const outputDir = "./output/";

Now further going ahead check for the directory if it is already exist or not :-

if (!fs.existsSync(outputDir)) {
  fs.mkdirSync(outputDir);
}

Here the "fs"(file streaming) is an inbuilt module of node.js you only need to import it in your file :-

 const fs = require('fs')

Step : 3

Make a function which will save our recordings in our repository -

  function saveStream() {
  const epochTime = moment().format("HH_mm_ss");
  const outputFilename = `${outputDir}${epochTime}.m3u8`;

  ffmpeg(inputURL)
    .inputFormat("flv")
        .outputOptions([
        `-hls_segment_filename ${tsFileName}.ts`,        
        // "-hls_time 900",
        "-hls_time 30",
        "-hls_list_size 5",
        // "-t 900",
        "-t 30",
        "-f hls",
        ])
    .output(outputFilename)
    .on("end", () => {
      console.log(`Saved stream to ${outputFilename}`);
    })
    .on("error", (err) => {
      console.error("Error:", err);
    })
    .run();
}

saveStream();

This function will save our rtmp feed to our directory this function is simple. Firslty, we take the time when the streaming starts and make our recording file name.

  const epochTime = moment().format("HH_mm_ss");
  const outputFilename = `${outputDir}${epochTime}.m3u8`;

Then, we take the rtmp url and save it in the hls format using fluent ffmpeg module :

  ffmpeg(inputURL)
    .inputFormat("flv")
    .outputOptions([
        "-hls_time 25",   
        "-hls_list_size 5",
        "-t 30",       
        "-f hls",
    ])
    .output(outputFilename)
    .on("end", () => {
      console.log(`Saved stream to ${outputFilename}`);
    })
    .on("error", (err) => {
      console.error("Error:", err);
    })
    .run();

this whole code will take the rtmp url and then it will convert it into hls format. The output format contains :

hls_time - Sets the duration of each segment in the HLS playlist to 25 seconds.

hls_list_size - Sets the maximum number of playlist entries (segments) to 5.

-t 30 - Set the duration of the feed which is getting saved(in seconds)

-f hls - It is specifying the output format as hls

Then we are saving the processed file in our directory with

 .output(outputFilename)

Final part of the function is Event Handling when our processed output is getting saved in our file then it will give us a log otherwise it will throw an error.

.on("end", () => {
      console.log(`Saved stream to ${outputFilename}`);
    })
    .on("error", (err) => {
      console.error("Error:", err);
    })

Lastly Executing our ffmpeg command with -

.run()

After setting everything call the function and set the interval as per your requirements I set it on the 30sec of interval. (to save the memory give the interval as same as your output file duration)

saveStream();

setInterval(saveStream, 30000);

Lastly, Handle event of post publish. Post publish will trigger when the stream is starting on node-media-server. Here I gave a condition because I am giving static URL -

nms.on('postPublish', (_, streamPath, _params) => {
  if (streamPath.endsWith('1efa24f9-0cd0-47c5-b604-c7e3ee118302')) {
    saveStream();
  }
});

Finally, Everything is all set now we can run our instance of node-meda-server so that it can run our script. From this our node-media-server will run.

nms.run();

Our final code will something like this -

const NodeMediaServer = require("node-media-server");
const fs = require("fs");
const moment = require('moment')
const ffmpeg = require("fluent-ffmpeg");

const config = {
  rtmp: {
    port: 1935,
    chunk_size: 60000,
    gop_cache: true,
    ping: 60,
    ping_timeout: 30,
  },
  http: {
    port: 8000,
    mediaroot: "./media",
    allow_origin: "*",
  },
};

const nms = new NodeMediaServer(config);

const inputURL =
  "rtmp://media5.ambicam.com:1938/live/1efa24f9-0cd0-47c5-b604-c7e3ee118302";
const outputDir = "./output/";

if (!fs.existsSync(outputDir)) {
  fs.mkdirSync(outputDir);
}

function saveStream() {
  const epochTime =  moment().format("HH_mm_ss"); 
  const outputFilename = `${outputDir}${epochTime}.m3u8`;

  ffmpeg(inputURL)
    .inputFormat("flv")
    .outputOptions([
        "-hls_time 25",   
        "-hls_list_size 5",
        "-t 30",       
        "-f hls",
    ])
    .output(outputFilename)
    .on("end", () => {
      console.log(`Saved stream to ${outputFilename}`);
    })
    .on("error", (err) => {
      console.error("Error:", err);
    })
    .run();
}

saveStream();

setInterval(saveStream, 30000);

nms.on('postPublish', (_, streamPath, _params) => {
  if (streamPath.endsWith('1efa24f9-0cd0-47c5-b604-c7e3ee118302')) {
    saveStream();
  }
});

nms.run();

Conclusion

In this tutorial, we embarked on a journey to set up a basic Node Media Server (NMS) instance, catering to beginners in the realm of video streaming. We covered the essentials from installing the node-media-server package to configuring a simple server instance capable of handling video streams.

One of the highlights of our exploration was the introduction of a straightforward mechanism to save video feeds at specific time intervals. This feature adds an extra layer of functionality to your streaming setup, enabling you to capture and store your content for future reference or archival purposes.

Creating API with Nest.js and MongoDB.

rutvikjani — Sun, 16 Jul 2023 08:02:29 +0000

Introduction

In this blog post, I will share my experience building a comprehensive and well-defined e-commerce API using NestJS, MongoDB, and Postman.

The API I built includes several endpoints that collectively make up a complete e-commerce API. These endpoints are for admins, end users, products, and categories.

Each endpoint is well-defined and provides a clear and concise interface for interacting with the API. The endpoints are also well-documented, making it easy for developers to understand how to use them.

I hope this blog post will be helpful to other developers who are interested in building e-commerce APIs with NestJS.

Here are some of the key takeaways from this project:

NestJS is a powerful framework that can be used to build comprehensive and well-defined APIs.
MongoDB is a reliable and scalable database that is well-suited for e-commerce applications.
Postman is a powerful tool for testing and debugging APIs.

I encourage you to try building your own e-commerce API using NestJS, MongoDB, and Postman. It is a great way to learn about these technologies and build a valuable skill set.

About Project

Ecommerce systems typically have two types of users: users and admins. Users can register and login to the system, but they can only read products. Admins can register, login, and perform CRUD operations on products.

The password hashing for both users and admins is done with the bcrypt module in npm. This module provides a secure way to hash passwords.

Product categories are defined with the help of the .populate() method. This method allows you to fetch the product categories from the database when you fetch a product. This makes it easy to display the product categories on the product page.

Detailed Explaination :-

Users and admins: Users are the end users of the ecommerce system. They can register and login to the system, and they can view products. Admins are system administrators. They have more privileges than users, and they can perform CRUD operations on products.
Password hashing: Password hashing is a security measure that helps to protect user passwords. When a user registers for an ecommerce system, their password is hashed using a secure algorithm. This means that the password is not stored in plain text, and it cannot be easily decoded.
jsonwebtoken module: The jsonwebtoken module is a popular module for generating and verifying JSON Web Tokens (JWTs). JWTs are a secure way to authenticate users. When a user logs in to an ecommerce system, they are issued a JWT. This JWT can then be used to authenticate the user on subsequent requests.
.populate() method: The .populate() method is a method that is used to fetch related documents from the database. In the context of ecommerce systems, the .populate() method can be used to fetch product categories when a product is fetched. This makes it easy to display the product categories on the product page.
bcrypt module: The bcrypt module uses the bcrypt algorithm to hash passwords. The bcrypt algorithm is a slow algorithm, which makes it difficult for attackers to brute-force passwords. The bcrypt module also uses a salt, which is a random string that is added to the password before it is hashed. This makes the hash more unique and difficult to crack.

How to create Nest Project

First of all we need to install Nestjs in our root directory. To install Nestjs Cli in our root directory we need to follow the command:-

$npm i -g @nestjs/cli

Moreover, for creating boilerplate for our nest project file We need to write following code:-

$nest new project-name

Your created file will look like this :-

Your project file will look like this at first but you can create or delete folders as per your requirement.

Creating first API

Nest APIs are made up of three main components: providers, controllers, and services.

Providers are plain JavaScript classes that provide functionality to the application. They are declared in module files.

Controllers handle HTTP requests. They are responsible for routing requests to the appropriate providers and services.

Services contain the business logic of the application. They are used by controllers to perform tasks such as data access and manipulation.

In other words, providers are the foundation of an API, controllers are the glue that holds it together, and services are the muscle that makes it work.

The normal flow of data is like....
Controller -> Service;
Service -> Repository/Schema;
Repository/Schema -> Database;

So here's how your all your files inside src folder will look like...

Schema

import { Prop, Schema, SchemaFactory } from "@nestjs/mongoose";
import { Document } from "mongoose";

@Schema()
export class Admin extends Document{

    @Prop({required: true})
    firstName: string

    @Prop({required: true})
    lastName: string

    @Prop({unique: true, required: true })
    userName: string

    @Prop({required: true})
    password: string

    @Prop({ unique: true, required: true })
    mobileNo : number

    @Prop({ unique: true, required: true })
    email: string

    @Prop({required: true})
    gender: string

    @Prop()
    isAdmin: boolean
}
 export const AdminSchema = SchemaFactory.createForClass(Admin)

The code above defines a mongoose schema for the admin API. A schema is a blueprint for a document in a MongoDB database. It defines the structure of the document, including the fields that are allowed and the types of data that can be stored in each field.

The mongoose schema in the code above uses decorators to validate the data that is stored in the document. For example, the IsString() decorator ensures that the value of the name field is a string, and the IsNotEmpty() decorator ensures that the value of the email field is not empty.

This schema ensures that the data that is stored in the admin API is valid and consistent. This makes it easier to manage the data and to ensure that the API is working correctly.

Controllers

import { Body, Controller, Post } from '@nestjs/common';
import { adminService } from './admin.service';
import { Admin } from './admin.schema';
import { adminDto } from './dto/admin.dto';
import { adminLoginDto } from './dto/adminLogin.dto';

@Controller('admin')
export class adminController {

  constructor(
    private adminService: adminService, 
    ) {}

  @Post('/register')
  async newAdmin(@Body() adminDto: adminDto): Promise<Admin | object> {
    return this.adminService.newAdmin(adminDto);
  }

  @Post('/login')
  async loginAdmin(@Body() adminLoginDto: adminLoginDto): Promise<object>{
   return this.adminService.loginAdmin(adminLoginDto);
  }
}

The code above shows the Admin API controller file, which is a class that handles HTTP requests and responses for managing admin users. The controller uses only the HTTP POST method for signup and login requests. It also uses the services, as mentioned above, which are classes that provide business logic.

To create a controller file, it is mandatory to use the @Controller() decorator above the controller class. This is a Spring annotation that marks a class as a controller.

The term DTO stands for Data Transfer Object. DTOs are simple objects that are used to transfer data between different components of an application. In this case, they are used to validate data that is flowing in incoming requests.

Here is the Example of a Simple admin login Dto....

import { IsString } from 'class-validator';

export class adminLoginDto {
  @IsString()
  userName: string;

  @IsString()
  password: string;

}

Services

/* eslint-disable prettier/prettier */
import { Injectable, UnauthorizedException } from '@nestjs/common';
import { Admin } from './admin.schema';
import { InjectModel } from '@nestjs/mongoose';
import { Model } from 'mongoose';
import * as bcrypt from 'bcrypt';
import { adminDto } from './dto/admin.dto';
import { JwtService } from '@nestjs/jwt';
import { adminLoginDto } from './dto/adminLogin.dto';

@Injectable()
export class adminService {
  constructor(
    @InjectModel(Admin.name)
    private adminModel: Model<Admin>,
    private jwtService: JwtService,
  ) {}

  async newAdmin(adminDto: adminDto): Promise<Admin | object> {
    const {
      firstName,
      lastName,
      userName,
      password,
      email,
      mobileNo,
      gender,
      isAdmin,
    } = adminDto;

    const salt = await bcrypt.genSalt();
    const hashedPassword = await bcrypt.hash(password, salt);

    const existingUser = await this.adminModel.findOne({
      userName,
      email,
    });
    if (!existingUser) {
      const adminUser = await this.adminModel.create({
        firstName,
        lastName,
        userName,
        password: hashedPassword,
        email,
        mobileNo,
        gender,
        isAdmin: true,
      });
      return adminUser.save();
    } else {
      return {
        statusCode: 400,
        message: `User already exist.`,
      };
    }
  }

  async loginAdmin(adminLoginDto: adminLoginDto): Promise<object> {
    const { userName, password } = adminLoginDto;
    const admin = await this.adminModel.findOne({ userName });
    const isAdmin = admin.isAdmin;
    if (admin && (await bcrypt.compare(password, admin.password))) {
      const payload = { userName, isAdmin };
      const accessToken = await this.jwtService.sign(payload);
      return {
        statusCode: 200,
        message: 'Login Successfull',
        Admin: admin,
        token: accessToken,
      };
    } else {
      throw new UnauthorizedException('User is not authorized');
    }
  }
}

The Services module in a Nest project is responsible for the business logic of the application. It is used by the controller modules to perform tasks such as saving data to the database, verifying passwords, and generating tokens.

In this module, we are saving the admin's data on sign-in with hashed passwords using the bcrypt package. This ensures that the passwords are stored securely and cannot be easily decoded.

The .findOne() method is used to find data in the database based on the username and email fields. These two fields should be unique, so using the .findOne() method ensures that we only return one record.

Array destructuring is used to extract the required fields from the record that we retrieved from the database. These fields will be saved into our database. We also keep a flag to differentiate between end-users and admin users.

The Login function uses the same .findOne() method to find the data from the database. The password is then verified, and if the credentials are correct, the user will receive a token.

The AuthGuard that we used will only give access to the admin to create, delete, and update products. End-users do not have the same authority as the admin.

Conclusion

In this blog post, we discussed the process of creating an admin API for an e-commerce platform. We covered the different endpoints that are typically included in an admin API, as well as the different constraints that you will need to consider when creating an API.

We also discussed the need for additional APIs, such as end-user, product, and category APIs, in order to provide a complete e-commerce experience to users. We briefly touched on the different constraints that you will need to consider when creating these additional APIs.

We hope that this blog post has been helpful. If you have any questions, please feel free to leave a comment below.

Thank you for reading!

Here You can checkout my full project, I tried to keep it as minimal as possible for better understanding..

DEV Community: rutvikjani

How To install feed from your live stream in node-media-server.

Introduction:

Prerequisites

Modules Used

Creating API with Nest.js and MongoDB.

Introduction

About Project

How to create Nest Project

Creating first API

Conclusion

rutvikjani / Ecommerce-Api-Using-Nestjs-MongoDB