DEV Community: Arjun Kava

Build Real-time transcription app using React Hooks

Arjun Kava — Sat, 25 May 2024 17:55:26 +0000

Step-by-Step Tutorial for the Code

This tutorial will guide you through the setup and functioning of the React app using the VideoSDK.live SDK. The app allows users to join or create meetings and provides functionalities for recording and transcription.

Prerequisites

Node.js and npm: Ensure you have Node.js and npm installed.
VideoSDK Account: Sign up at VideoSDK.live and get your API key.

Step 1: Setup the Project

Create a React App: If you don't already have a React app, create one using the following command:

   npx create-react-app my-video-app
   cd my-video-app

Install Dependencies: Install the necessary packages.

   npm install @videosdk.live/react-sdk react-player

Step 2: Setup API Functions

Create an API.js file in the src directory to handle API interactions.

// API.js
export const authToken = "YOUR_VIDEO_SDK_AUTH_TOKEN";

export const createMeeting = async ({ token }) => {
  const response = await fetch("https://api.videosdk.live/v1/meetings", {
    method: "POST",
    headers: {
      Authorization: token,
      "Content-Type": "application/json",
    },
  });
  const data = await response.json();
  return data.meetingId;
};

Step 3: Build the Components

In your App.js, import necessary modules and create components as described below.

JoinScreen Component

Allows users to enter a meeting ID or create a new meeting.

import React, { useState } from "react";

function JoinScreen({ getMeetingAndToken }) {
  const [meetingId, setMeetingId] = useState(null);
  const onClick = async () => {
    await getMeetingAndToken(meetingId);
  };
  return (
    <div>
      <input
        type="text"
        placeholder="Enter Meeting Id"
        onChange={(e) => {
          setMeetingId(e.target.value);
        }}
      />
      <button onClick={onClick}>Join</button>
      {" or "}
      <button onClick={onClick}>Create Meeting</button>
    </div>
  );
}

export default JoinScreen;

ParticipantView Component

Displays the video and audio of a participant.

import React, { useEffect, useMemo, useRef } from "react";
import { useParticipant } from "@videosdk.live/react-sdk";
import ReactPlayer from "react-player";

function ParticipantView(props) {
  const micRef = useRef(null);
  const { webcamStream, micStream, webcamOn, micOn, isLocal, displayName } =
    useParticipant(props.participantId);

  const videoStream = useMemo(() => {
    if (webcamOn && webcamStream) {
      const mediaStream = new MediaStream();
      mediaStream.addTrack(webcamStream.track);
      return mediaStream;
    }
  }, [webcamStream, webcamOn]);

  useEffect(() => {
    if (micRef.current) {
      if (micOn && micStream) {
        const mediaStream = new MediaStream();
        mediaStream.addTrack(micStream.track);

        micRef.current.srcObject = mediaStream;
        micRef.current
          .play()
          .catch((error) =>
            console.error("videoElem.current.play() failed", error)
          );
      } else {
        micRef.current.srcObject = null;
      }
    }
  }, [micStream, micOn]);

  return (
    <div>
      <p>
        Participant: {displayName} | Webcam: {webcamOn ? "ON" : "OFF"} | Mic:{" "}
        {micOn ? "ON" : "OFF"}
      </p>
      <audio ref={micRef} autoPlay playsInline muted={isLocal} />
      {webcamOn && (
        <ReactPlayer
          playsinline
          pip={false}
          light={false}
          controls={false}
          muted={true}
          playing={true}
          url={videoStream}
          height={"300px"}
          width={"300px"}
          onError={(err) => {
            console.log(err, "participant video error");
          }}
        />
      )}
    </div>
  );
}

export default ParticipantView;

Controls Component

Provides buttons to leave the meeting and toggle mic/webcam.

import React from "react";
import { useMeeting } from "@videosdk.live/react-sdk";

function Controls() {
  const { leave, toggleMic, toggleWebcam } = useMeeting();
  return (
    <div>
      <button onClick={() => leave()}>Leave</button>
      <button onClick={() => toggleMic()}>toggleMic</button>
      <button onClick={() => toggleWebcam()}>toggleWebcam</button>
    </div>
  );
}

export default Controls;

MeetingView Component

Main component to handle meeting functionalities like transcription, recording, and displaying participants.

import React, { useState } from "react";
import { useMeeting, useTranscription, Constants } from "@videosdk.live/react-sdk";
import ParticipantView from "./ParticipantView";
import Controls from "./Controls";

function MeetingView(props) {
  const [transcript, setTranscript] = useState("Transcription");
  const [transcriptState, setTranscriptState] = useState("Not Started");
  const tConfig = { webhookUrl: "https://www.example.com" };
  const { startTranscription, stopTranscription } = useTranscription({
    onTranscriptionStateChanged: (data) => {
      const { status } = data;
      if (status === Constants.transcriptionEvents.TRANSCRIPTION_STARTING) {
        setTranscriptState("Transcription Starting");
      } else if (status === Constants.transcriptionEvents.TRANSCRIPTION_STARTED) {
        setTranscriptState("Transcription Started");
      } else if (status === Constants.transcriptionEvents.TRANSCRIPTION_STOPPING) {
        setTranscriptState("Transcription Stopping");
      } else if (status === Constants.transcriptionEvents.TRANSCRIPTION_STOPPED) {
        setTranscriptState("Transcription Stopped");
      }
    },
    onTranscriptionText: (data) => {
      let { participantName, text, timestamp } = data;
      console.log(`${participantName}: ${text} ${timestamp}`);
      setTranscript(transcript + `${participantName}: ${text} ${timestamp}`);
    },
  });

  const { startRecording, stopRecording } = useMeeting();

  const handleStartRecording = () => {
    startRecording("YOUR_WEB_HOOK_URL", "AWS_Directory_Path", {
      layout: { type: "GRID", priority: "SPEAKER", gridSize: 4 },
      theme: "DARK",
      mode: "video-and-audio",
      quality: "high",
      orientation: "landscape",
    });
  };

  const handleStopRecording = () => stopRecording();
  const handleStartTranscription = () => startTranscription(tConfig);
  const handleStopTranscription = () => stopTranscription();

  const [joined, setJoined] = useState(null);
  const { join, participants } = useMeeting({
    onMeetingJoined: () => setJoined("JOINED"),
    onMeetingLeft: () => props.onMeetingLeave(),
  });

  const joinMeeting = () => {
    setJoined("JOINING");
    join();
  };

  return (
    <div className="container">
      <h3>Meeting Id: {props.meetingId}</h3>
      {joined && joined === "JOINED" ? (
        <div>
          <Controls />
          <button onClick={handleStartRecording}>Start Recording</button>
          <button onClick={handleStopRecording}>Stop Recording</button>
          <button onClick={handleStartTranscription}>Start Transcription</button>
          <button onClick={handleStopTranscription}>Stop Transcription</button>
          {[...participants.keys()].map((participantId) => (
            <ParticipantView participantId={participantId} key={participantId} />
          ))}
          <p>State: {transcriptState}</p>
          <p>{transcript}</p>
        </div>
      ) : joined && joined === "JOINING" ? (
        <p>Joining the meeting...</p>
      ) : (
        <button onClick={joinMeeting}>Join</button>
      )}
    </div>
  );
}

export default MeetingView;

App Component

Main component that manages the meeting state and provides the necessary context.

import React, { useState } from "react";
import { MeetingProvider } from "@videosdk.live/react-sdk";
import JoinScreen from "./JoinScreen";
import MeetingView from "./MeetingView";
import { authToken, createMeeting } from "./API";

function App() {
  const [meetingId, setMeetingId] = useState(null);

  const getMeetingAndToken = async (id) => {
    const meetingId =
      id == null ? await createMeeting({ token: authToken }) : id;
    setMeetingId(meetingId);
  };

  const onMeetingLeave = () => setMeetingId(null);

  return authToken && meetingId ? (
    <MeetingProvider
      config={{
        meetingId,
        micEnabled: true,
        webcamEnabled: true,
        name: "C.V. Raman",
      }}
      token={authToken}
    >
      <MeetingView meetingId={meetingId} onMeetingLeave={onMeetingLeave} />
    </MeetingProvider

>
  ) : (
    <JoinScreen getMeetingAndToken={getMeetingAndToken} />
  );
}

export default App;

Step 4: Run the Application

Start the React App:

   npm start

Navigate to Your Browser: Open http://localhost:3000 to view the app.

Optional Step: Add CSS for Styling

To enhance the visual appeal of your app, you can add the following CSS.

Create an App.css file in your src directory with the following content:

/* App.css */

body {
  background-color: #121212;
  color: #e0e0e0;
  font-family: 'Roboto', sans-serif;
  margin: 0;
  padding: 0;
}

input, button {
  background-color: #1e1e1e;
  border: 1px solid #333;
  color: #e0e0e0;
  padding: 10px;
  margin: 5px;
  border-radius: 5px;
}

button:hover {
  background-color: #333;
  cursor: pointer;
}

.container {
  max-width: 800px;
  margin: auto;
  padding: 20px;
  text-align: center;
}

h3 {
  color: #f5f5f5;
}

p {
  margin: 10px 0;
}

audio, .react-player__preview {
  background-color: #333;
  border: 1px solid #555;
  border-radius: 5px;
  margin: 10px 0;
}

.react-player__preview img {
  border-radius: 5px;
}

.react-player__shadow {
  border-radius: 5px;
}

Summary

You have created a functional video meeting application using the VideoSDK.live SDK. This app allows users to join or create meetings, manage participants, and control functionalities like recording and transcription. The optional CSS step ensures a consistent and visually appealing user interface.

Comparing WebRTC and WebSocket: Choosing the Right Technology for Real-Time Experience

Arjun Kava — Sat, 25 May 2024 16:48:06 +0000

WebRTC vs WebSocket: A Comprehensive Comparison

In the realm of real-time communication on the web, two technologies stand out: WebRTC (Web Real-Time Communication) and WebSocket. Both offer unique advantages and are tailored to different use cases. This article provides an in-depth comparison of WebRTC and WebSocket, exploring their features, use cases, advantages, and disadvantages to help you decide which technology is best suited for your application.

Introduction

Real-time communication is integral to modern web applications, enabling functionalities like video conferencing, live streaming, online gaming, and instant messaging. WebRTC and WebSocket are pivotal technologies in this space, each with distinct capabilities.

What is WebRTC?

WebRTC is a peer-to-peer communication protocol that facilitates real-time audio, video, and data exchange directly between browsers or mobile applications without the need for an intermediary server. It is designed to support high-quality media streams with low latency.

Advantages of WebRTC

High-Quality Media Streams: WebRTC is optimized for real-time video and audio communication, offering minimal latency and high quality.
Encryption: Built-in security features such as DTLS and SRTP ensure that all communications are encrypted and secure from eavesdropping.
Browser Support: Supported by most modern browsers, including Chrome, Firefox, and Safari, facilitating easy implementation and broad accessibility.

Disadvantages of WebRTC

Complexity: Implementing WebRTC can be technically challenging due to its complex architecture involving numerous protocols and APIs like ICE, STUN, and TURN.
Resource Intensive: WebRTC applications can be resource-intensive, especially when handling high-definition video streams, requiring significant bandwidth and processing power.
Partial Browser Support: While broadly adopted, certain codecs or features may have inconsistent support across different browsers, potentially leading to compatibility issues.

What is WebSocket?

WebSocket is a full-duplex communication protocol that enables persistent, bidirectional communication between a client and a server over a single TCP connection. This makes it suitable for real-time applications that require continuous data exchange.

Advantages of WebSocket

Low Overhead: WebSocket reduces the overhead associated with traditional HTTP requests, making data transfer more efficient.
Full-Duplex Communication: Allows for simultaneous two-way communication, which is crucial for interactive applications like online gaming and live chat.
Wide Compatibility: Supported by all major browsers and easily implemented across various platforms and devices.

Disadvantages of WebSocket

Stateful Connections: Maintaining a stateful connection can complicate scaling in large applications due to the need for persistent connections to each client.
Connection Recovery: WebSockets don’t automatically recover when connections are terminated; this functionality needs to be implemented by the developer.
Security: While secure over TLS (WSS), WebSocket lacks the built-in encryption mechanisms of WebRTC, requiring additional security implementations.

Key Differences Between WebRTC and WebSocket

Protocol and Connection Type

WebRTC: Uses UDP primarily (can work over TCP) and supports peer-to-peer connections【8†source】【10†source】.
WebSocket: Uses TCP and relies on a client-server model.

Data Transmission and Use Cases

WebRTC: Suitable for high-performance media streaming (video/audio) and real-time communications like video conferencing and live broadcasting.
WebSocket: Ideal for reliable data transmission, making it better suited for real-time messaging, multiplayer gaming, and live notifications.

Latency and Bandwidth

WebRTC: Lower latency and bandwidth usage due to its peer-to-peer nature.
WebSocket: Higher latency and bandwidth usage but ensures data integrity through TCP.

Use Cases

When to Use WebRTC

Video and Audio Conferencing: Real-time video and audio calls directly within web browsers without the need for additional plugins or software.
Live Streaming: For events like webinars or concerts, where low latency is critical.
Real-time Gaming: Low latency communication ideal for online gaming interactions.
File Sharing: Direct transfer of files between users in a peer-to-peer manner.
Telehealth: Secure video consultations between patients and healthcare providers.
Education and E-Learning: Facilitates interactive online learning experiences.

When to Use WebSocket

Chat Applications: Instant messaging where messages need to be exchanged in real-time.
Live Notifications: Updating users with live notifications, such as new posts or social media interactions.
Online Multiplayer Games: Games requiring constant data exchange between players and the server.
Financial Applications: Real-time updates for stock trading and financial transactions.
Interactive Dashboards: Real-time updating of user interfaces, such as live stats or monitoring systems.

When to Use Both WebRTC and WebSocket

In some scenarios, using both WebRTC and WebSocket together can be beneficial:

Signaling in WebRTC: WebSocket can handle the signaling process required to establish WebRTC connections.
Complex Applications: Applications like multiplayer games or live broadcasting with chat might use WebRTC for low-latency media transmission and WebSocket for server-mediated tasks like state synchronization and user matching.

Conclusion

Both WebRTC and WebSocket are crucial for real-time communication, but they cater to different needs. WebRTC is ideal for high-quality, low-latency media communication, while WebSocket is perfect for reliable, persistent data exchange. Understanding their strengths and limitations will help you decide which technology best meets your application's requirements.

FAQs

Can WebRTC and WebSocket be used together?

Yes, WebRTC and WebSocket can be used together to leverage their strengths for different aspects of an application, such as using WebSocket for signaling and WebRTC for media transmission.

What are the primary use cases for WebRTC?

WebRTC is primarily used for video and audio conferencing, live streaming, real-time gaming, file sharing, telehealth, and education.

What are the primary use cases for WebSocket?

WebSocket is best suited for chat applications, live notifications, online multiplayer games, financial applications, and interactive dashboards.

By understanding the capabilities and ideal use cases for WebRTC and WebSocket, developers can make informed decisions to create robust, real-time communication applications tailored to their specific needs.

Cool animations with React-Spring

Arjun Kava — Thu, 23 May 2024 17:31:36 +0000

Introduction

Hey there, fellow devs! 🖐️ Are you tired of boring, static UIs? Do you want to add some pizzazz to your React applications? Well, you're in for a treat! Today, we're diving into the world of react-spring, a powerful library for creating delightful animations in your React apps. Whether you're a beginner or a seasoned pro, there's always something cool to learn with react-spring. So, buckle up and let's make our UIs dance! 🎉

What is React-Spring?

React-spring is a spring-physics-based animation library that powers most UI-related animation in React. It provides a set of hooks to easily create animations and transitions in your components. The best part? It's super flexible and can handle complex animations with ease.

Why Use React-Spring?

Declarative Animations: Write animations the same way you write your React components.
Performance: Built on top of the react-motion library, it's optimized for performance.
Flexibility: From simple animations to complex sequences, react-spring has got you covered.

Getting Started

First things first, let's get react-spring installed. You can do this using npm or yarn:

npm install react-spring
# or
yarn add react-spring

Now that we have it installed, let's create a simple animation.

Simple Fade-In Animation

Let's start with something basic: a fade-in animation. We'll animate a component to fade in when it mounts.

import React from 'react';
import { useSpring, animated } from 'react-spring';

const FadeInComponent = () => {
  const props = useSpring({ opacity: 1, from: { opacity: 0 } });

  return <animated.div style={props}>Hello, I am fading in!</animated.div>;
};

export default FadeInComponent;

Bouncing Ball Animation

Now, let's create a more interesting animation: a bouncing ball. We'll animate a ball that bounces up and down.

import React from 'react';
import { useSpring, animated } from 'react-spring';

const BouncingBall = () => {
  const props = useSpring({
    to: { transform: 'translateY(0px)' },
    from: { transform: 'translateY(-100px)' },
    config: { tension: 170, friction: 12 },
    loop: { reverse: true },
  });

  return (
    <animated.div style={{ ...props, width: 50, height: 50, borderRadius: '50%', background: 'coral', margin: 'auto' }} />
  );
};

export default BouncingBall;

Carousel with React-Spring

Let's create a simple carousel using react-spring. We'll animate the transition between images.

import React, { useState } from 'react';
import { useSpring, animated } from 'react-spring';

const Carousel = ({ images }) => {
  const [index, setIndex] = useState(0);
  const props = useSpring({ opacity: 1, from: { opacity: 0 }, reset: true, reverse: index % 2 === 0 });

  const nextImage = () => setIndex((index + 1) % images.length);

  return (
    <div>
      <animated.img src={images[index]} style={props} alt="carousel" />
      <button onClick={nextImage}>Next</button>
    </div>
  );
};

export default Carousel;

Interactive Spring Animation

Let's create an interactive animation where the user can drag a box around the screen.

import React from 'react';
import { useSpring, animated } from 'react-spring';
import { useDrag } from 'react-use-gesture';

const DraggableBox = () => {
  const [props, set] = useSpring(() => ({ x: 0, y: 0 }));

  const bind = useDrag((params) => set({ x: params.offset[0], y: params.offset[1] }));

  return <animated.div {...bind()} style={{ ...props, width: 100, height: 100, background: 'lightblue', cursor: 'grab' }} />;
};

export default DraggableBox;

Conclusion

React-spring opens up a world of possibilities for adding animations to your React applications. From simple transitions to complex interactive animations, it's a versatile tool that can bring your UI to life. So go ahead, experiment with it, and make your apps more engaging and fun! 🌟

Mastering React Motion: Adding Smooth Animations to Your React Apps

Arjun Kava — Thu, 23 May 2024 17:20:02 +0000

Introduction

Hey there, fellow developers! Are you ready to take your React projects to the next level with some slick animations? Today, we're diving into React Motion, a powerful library that makes it super easy to add smooth, natural animations to your React components. Whether you're building a simple to-do list or a complex dashboard, React Motion can help you create a more dynamic and engaging user experience.

What is React Motion?

React Motion is a popular animation library for React that leverages the power of physics to create natural and fluid animations. Unlike traditional CSS-based animations, React Motion uses a physics-based approach, allowing you to define how elements should move based on principles like stiffness, damping, and precision.

Why Use React Motion?

Smooth and Natural Animations: React Motion's physics-based animations feel more natural compared to traditional CSS animations.
Declarative Syntax: You can describe your animations declaratively within your React components, making your code more readable and maintainable.
Flexibility: Easily animate a variety of properties, including position, size, and opacity.

Getting Started

Let's get our hands dirty with some code. We'll start by setting up a basic React project and adding React Motion to it.

Step 1: Set Up Your React Project

If you don't already have a React project, you can create one using Create React App:

npx create-react-app react-motion-demo
cd react-motion-demo

Step 2: Install React Motion

Next, install React Motion via npm:

npm install react-motion

Step 3: Create Your First Animation

Now, let's create a simple component that uses React Motion to animate a box.

import React from 'react';
import { Motion, spring } from 'react-motion';

const AnimatedBox = () => {
  return (
    <Motion defaultStyle={{ x: 0 }} style={{ x: spring(100) }}>
      {style => (
        <div style={{
          transform: `translateX(${style.x}px)`,
          width: '100px',
          height: '100px',
          backgroundColor: 'tomato',
        }} />
      )}
    </Motion>
  );
};

export default AnimatedBox;

In this example, we use the Motion component to animate a box from x: 0 to x: 100. The spring function is used to create a smooth animation based on spring physics.

Exploring More Complex Animations

Let's add some more complexity by animating multiple properties and incorporating user interactions.

import React, { useState } from 'react';
import { Motion, spring } from 'react-motion';

const InteractiveBox = () => {
  const [isToggled, setIsToggled] = useState(false);

  const handleToggle = () => {
    setIsToggled(!isToggled);
  };

  return (
    <div onClick={handleToggle}>
      <Motion
        defaultStyle={{ x: 0, opacity: 0.5 }}
        style={{
          x: spring(isToggled ? 200 : 0),
          opacity: spring(isToggled ? 1 : 0.5)
        }}
      >
        {style => (
          <div style={{
            transform: `translateX(${style.x}px)`,
            opacity: style.opacity,
            width: '100px',
            height: '100px',
            backgroundColor: 'tomato',
            cursor: 'pointer'
          }} />
        )}
      </Motion>
    </div>
  );
};

export default InteractiveBox;

In this example, clicking the box toggles its position and opacity, creating a more dynamic interaction.

Fine-Tuning Your Animations

React Motion provides several parameters to fine-tune your animations:

Stiffness: Controls the rigidity of the spring. Higher values result in faster animations.
Damping: Controls the resistance. Higher values create slower animations.
Precision: Determines the accuracy of the animation's end state.

Here's how you can customize these parameters:

<Motion
  defaultStyle={{ x: 0 }}
  style={{ x: spring(100, { stiffness: 170, damping: 26, precision: 0.01 }) }}
>
  {style => (
    <div style={{
      transform: `translateX(${style.x}px)`,
      width: '100px',
      height: '100px',
      backgroundColor: 'tomato',
    }} />
  )}
</Motion>

Conclusion

React Motion is a fantastic library for adding smooth, physics-based animations to your React applications. By leveraging its declarative syntax and customizable parameters, you can create engaging and dynamic user experiences with ease. Give it a try in your next project, and watch your UI come to life!

Happy coding!

Feel free to ask any questions in the comments or share your awesome animations with us. We'd love to see what you create with React Motion!

Zooming out Twilio Migration

Arjun Kava — Fri, 08 Dec 2023 15:13:31 +0000

Twilio has decided to shut down its Programmable Video SDK. This decision, while understandable, may leave many developers wondering about the future of their video-based applications.😟

In a recent statement, Twilio CEO Jeff Lawson explained the reason behind this decision: While I was disappointed with this decision, the Twilio Video SDK was one of the best products in town, especially for builders.🏗️

Lastly, we’ve decided to end-of-life (EOL) Twilio Programmable Video as a standalone product. Given it’s such a niche area and a relatively small part of our portfolio, we believe partnering with video industry leaders is the best way to ensure long-term product innovation for our customers.📧

Removing Programmable Video from our portfolio will also allow Communications to more effectively focus on our pillar products - Messaging, Voice, and Email.

I previously wrote about "Domino called exit(); for Twilio's Programmable Video", click the link below if you haven't read it yet.

🌐 Zooming into a situation

Twilio has partnered with Zoom to migrate from the Programmable Video to Zoom Video SDK:

We recommend migrating your application to the API provided by our preferred video partner, Zoom. We've prepared this migration guide to assist you in minimizing any service disruption.

The only reason for Zoom is to not pick up the competition. Zoom is nowhere near a direct competitor to Twilio, although both have similar types of customers such as contact centers, sales/marketing departments in corporates, etc.

Either way, Twilio customers have a year to find a solution unless drastic changes are made to the WebRTC API.

📱 Future of building Real-time Video Apps

I think creating a great video app is an art. Every company has different uses and daily needs. Development flexibility is even more important because each use case requires high-end customization.

🖌️ Compatibility with browsers and mobile devices

First and foremost, compatibility with major browsers and mobile devices is essential. Pre-call checks play a big role in improving the call experience.

The main cause of poor call experience is either audio/video capture failure, or pre-checking on capture will save a lot of time and ensure browser/device support is available.

🔄 End to End Customisable UI/UX Experience

Real-time experiences are about harmony between web apps and mobile apps. Each business requires different features to build.

For example, an education use case is driven by a single speaker while tele-consultancy is a continuous communication between two people and real-time audio broadcast requires a constant change of speakers.

Each application requires different types of user experience to implement and manage according to the user base and their behaviour.

📷 High Quality Audio / Video Experience

In an era where user experience is a key differentiator, a video application with high-quality audio and video capabilities stands out. Users expect nothing less than excellence, and an app that delivers on this front not only meets but exceeds those expectations. This, in turn, contributes to positive reviews, increased user retention, and organic growth of the app's user base.

To achieve that, high bitrates are required to send and receive audio and video with optimal compression mechanisms.

🤖 Native Integration of AI on top of Audio and Video

As the world is going through a generative AI boom. Real-time audio and video will play an important role in the adoption of AI.

Background change, filters, and face tracking are much-needed features depending on different market segments. Apart from that speech, text and transcription are essential features when it comes to video analysis.

This is just the beginning as a growing number of companies are integrating native generative AI capabilities over real-time audio and video to better assist their users.

🤝 Collaboration and Moderation on scale

On a 1:1 basis, small group call or large group call collaborative features like chat, polling, Q&A, raising hands, layout changes, etc. are essential to create a connection between participants.

When it comes to large group calls, moderation controls like mute all, spotlight, waiting room, etc. are very necessary.

🔐 Built-in Data Privacy and Protection

Data privacy and security is the most important aspect that a company should consider before making any decision. It is essential to protect customer information from threats.

A focus on privacy prevents unauthorized access, protects sensitive data, and maintains the app’s reputation in an environment where user trust is paramount.

🎥 Instant recordings with customizable templates

Most use cases around cloud recording are either post-streaming of content or post-production of content. For example, the use-case is streaming in education recordings while in virtual events, it is post-production content for websites and social media.

Instant recording infrastructure plays a big role in such use cases where users do not have to wait for the recording to be processed.

😱 Nightmare of migrating to Zoom SDK

Now let's talk about the elephant in the room, migration to Zoom. Zoom by nature is an MCU architecture, meaning they decrypt audio/video streams on the server and mix them as one stream.

Compared to SFU architecture it is difficult to have a good developer experience and thus it is becoming a nightmare for developers for several reasons but below are the most important ones to consider:

🌍 Not having global connected regions to solve global latency

Zoom forces you to select a region before init the client, and that's why it's very difficult to solve for global latency because anyone from Europe joining a call to a US server will experience latency and massive packet loss.

client.init('en-US', 'Global', { patchJsMedia: true }).then(() => {
  client.join('sessionName', 'VIDEO_SDK_JWT', 'userName', 'sessionPasscode').then(() => {
    stream = client.getMediaStream()
  })
})

🎨 Video Layout with Canvas Painting

Zoom doesn't have raw access to audio and video streams in the SDK, and because of that, you have to calculate a lot of unnecessary mathematical code to manage multiple layouts. Although canvas rendering is good but developing and maintaining layout logic takes almost more time than creating a product.

let participants = client.getAllUser()

stream.renderVideo(document.querySelector('#participant-videos-canvas'), participants[0].userId, 960, 540, 0, 540, 2)
stream.renderVideo(document.querySelector('#participant-videos-canvas'), participants[1].userId, 960, 540, 960, 540, 2)
stream.renderVideo(document.querySelector('#participant-videos-canvas'), participants[2].userId, 960, 540, 0, 0, 2)
stream.renderVideo(document.querySelector('#participant-videos-canvas'), participants[3].userId, 960, 540, 960, 0, 2)

🔍 Raw media Access for Generative AI use-cases

As I said earlier, Zoom doesn't allow raw media stream access which means you can't integrate any third-party SDKs or open source models on the client or server side.

Generative AI is becoming increasingly important for every application integrating text-to-speech, face tracking, face recognition, and server-side audio/video analysis.

All of the above is not possible with the Zoom SDK and cannot be done due to the nature of the technology.

💽 Large Size of SDK Binary

The Zoom SDK averages 97mb+ in mobile while it can go up to 157mb+ (sometimes I've read 500mb+ in community threads) which makes it heavy for a large number of use cases.

🔍 720p+ resolution is not supported

Zoom is not suitable if you are building apps for high-quality experiences due to resolution and bitrate restrictions at 720p.

This kills most use cases like broadcasting, high-resolution screen sharing, high-quality content sharing, etc.

🖼️ Virtual Background, Gallery View with SharedBuffer Array

The ShareadBuffer array is a compatibility and cross-device support killer in the Zoom SDK. It is only supported by two browsers as mentioned below by the Zoom team.

The bad news is that your client has to enable this because Chrome doesn't allow SharedArrayBuffer directly.

As of Chrome and Edge 92, and Firefox 79 SharedArrayBuffer is only available if your web page is Cross-Origin Isolated, or if your web page uses Credentialless headers, or if you have registered for the SharedArrayBuffer Chrome Origin Trial (works only in Chrome and Edge).

❌ Not Compatible majority of browsers

Since the Zoom SDK does not use WebRTC technology and relies on its own MCU infrastructure, it is not able to provide good support for web calls.

🔐 End to End Encryption of Audio / Video Streams

Zoom does not allow you to encrypt streams in the browser. This means that the data transmitted from your browser, including audio and video, is not encrypted throughout its journey to the recipient. While Zoom encrypts the data in transit between its servers, there is a potential vulnerability in the browser-to-server leg of the communication.

When selecting a video conferencing platform, it is crucial to consider security needs. If End to End Encryption is essential for your specific requirements, ensuring the platform you choose offers this functionality is critical.

🔄 Simulcast for Adaptive Bitrate Streaming

Zoom does not allow multi-layer sending and receiving of video with adaptive bitrate streaming due to the same MCU architecture.

This does not allow the Zoom SDK to reflect audio/video bitrate and resolution depending on the volatility and change of internet bandwidth.

👥 Sender / Receiver media track subscription

Zoom does not allow subscribe/unsubscribe from receiving audio/video streams which makes it difficult to implement for use cases such as breakout rooms, backstage, and watch parties.

🔍 Pre-call Testing for best call experience

Zoom does not provide a pre-call test before starting a video call. While checking quality only preview is available, connectivity and all other features are not available.

📈 QOS APIs and Dashboard

Quality of service (QoS) is paramount for any video conferencing platform. Zoom remains committed to providing its users with the tools and insights necessary to ensure optimal communication experiences. To this end, Zoom offers two key avenues for monitoring and managing QoS: APIs and the Dashboard.

Zoom offers a single API that grants developers access to a wealth of QoS data. This API delivers detailed information on key performance metrics. By leveraging the API, developers can integrate custom monitoring tools, automated remediation workflows, and real-time quality feedback into their applications.

🎥 Server side raw video streams for AI use-case

It is not possible to extract raw audio/video streams from the client or server-side using the Zoom Video SDK. This makes it difficult to create custom AI use cases such as transcription, speech-to-text, or any other type of intelligence.

❌ Missing data channel for collaboration and moderation controls

Zoom does not have a proper Data Channel feature within the SDK. That means you can't create collaborative features like polls, Q&A, layout changes, and moderation features like mute all, invite as a host, etc.

What you should know before moving to next API / SDK?

List your requirements and prioritize SDKs based on match score
Create a small POC and test the platform
Check out their support in the community and their knowledge of the space.
Check out the latest releases and continuous updates for your industry.
Get a demo with their team and see how committed they are to your future roadmap and what they're building for the next couple of quarters.
Check if they have been laid off in the last few quarters and see if they will be around for long.
A red flag is people trying to sell demos instead of explaining what's available and not available compared to Twilio.
Another red flag is getting onboard by offering migration credits and offers, trust me it's not worth it.
Make sure you invest in one vendor rather than buying from multiple because in the long run one will be able to justify the usage, business needs and relationships.

👩‍💻 Developer first approach at Video SDK

Video SDK is solving one problem for the best developer experience, reliability, and security of real-time video infrastructure. Compared to Zoom, we have a rich, highly flexible, and developer-friendly SDK. Here is the comparison:

Features	Zoom SDK	Video SDK
Globally Connected Regions	No	Yes
Video Tiles Rendering	In-flexible	Flexible
Raw Media Access	No	Yes
SDK Binary Size	100mb+	<20mb
Max Resolution	720p	2k+
Browser Compatibility	10% browsers	98% browsers
Sender / Receiver media track subscription	No	Yes
QOS API / Dahsboard	No	Yes
Pre-call Check	No	Yes
Data Channel	No	Yes

Follow the migration guide from Twilio to Video SDK, and at least start building a POC to see what works for you and what doesn't. Here are some references to do the same:

Check out Migration guide that we have written:

That's all for today, feel free to reach out if you need any help to navigate through solution, Talk to Our Migration Expert

As I said earlier here is the link about "Domino called exit(); for Twilio's Programmable Video"

Until next time, see you.

Build a Video Chat App with React Hooks

Arjun Kava — Sat, 30 Oct 2021 09:01:32 +0000

It is easy to use functional components with react hooks API. In this tutorial, we are going to use RTC React Video SDK to build video calling app in react js.

Update 1 - 28th Dec, 2021: Refer new video series React Video Chat App with Video SDK

Video SDK React JS SDK makes it easy to build real time video conferencing application. Video SDK handles compatibility with all the browsers, scale up-to 5,000 participants and low latency.

Video SDK RTC React JS SDK supports many features such as

10,000 minutes free each month
Completely Low code and serverless.
Video API with real-time audio, video and data streams
5,000+ participants support
Chat support with rich media.
Screen sharing with HD and Full HD.
Play realtime video in meeting
Connect it with social media such as Facebook, Youtube etc (RTMP out support).
Intelligent speaker switch
Record your meetings on cloud
Customize UI as per your needs.

Index

Create account on Video SDK
Setup Server
Structure of the Project
Start Writing the Code

1. Create account on Video SDK

Start Project at videosdk.live

Navigate to the the start project button and register yourself either using Google Account or Github.

Prerequisites

Node >= 10.16
Npm >= 5.6
React ≥ 16.8

2. Setup Server

A server will require to perform authentication via JWT token. We are going to use official Node JS server example.

Clone following repository and run the server.



$ git clone https://github.com/videosdk-live/videosdk-rtc-nodejs-sdk-example
$ cd nodejs

Note: You can also find other backend language examples in the same repo.

Follow the Node JS Server Setup Guide to run the server.

3. Structure of the Project

You can use react-scripts to generate project template or any other react boilerplate.

Create new project using create-react-app



npx create-react-app videosdk-react-app-v1

This is how your project directory should look like



.
├── node_modules
├── public
├── .env
├── src
│ └── api.js
│ └── App.jsx
│ └── index.css
│ └── index.jsx
├── package.json
...
.

Configure Environment Variables

Before writing the code, configure .env variables.

.env



REACT_APP_SERVER_URL="http://localhost:9000"

Note: For production environment, you have to host this server and need to change URL.

Install the official React JS package

Before jumping to anything else, install videosdk react js sdk.



yarn add @videosdk.live/react-sdk

4.Start Writing the Code

We will first setup API calls then after jump to writing the code.

Calling API to generate auth `token` and `meetingId`

We will start writing the code with api.js. Before starting any meeting, you have to generate authentication token and meetingId

api.js



const API_BASE_URL = process.env.REACT_APP_SERVER_URL;

// API call to generate authentication token
export const getToken = async () => {
  const res = await fetch(`${API_BASE_URL}/get-token`, {
    method: "GET",
  });

  const { token } = await res.json();
  return token;
};

// API call to create meeting
export const createMeeting = async ({ token }) => {
  const res = await fetch(`${API_BASE_URL}/create-meeting`, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ token }),
  });

  const { meetingId } = await res.json();
  return meetingId;
};

Start with `App.jsx`

First of all, Let's setup token generation and meetingId logic before working on video calling view.

App Component



import logo from './logo.svg';
import './App.css';
import React, { useEffect, useRef, useState } from "react";
import {
  MeetingProvider,
  MeetingConsumer,
  useMeeting,
  useParticipant,
} from "@videosdk.live/react-sdk";
import { getToken, validateMeeting, createMeeting } from "./api";

function MeetingGrid() {
  return <h1>Meeting Grid</h1>
}

function JoinScreen() {
  return <h1>Join Screen</h1>
}

function ParticipantView(props){
  return <h1>Participant View</h1>
}

function App() {
  const [token, setToken] = useState(null);
  const [meetingId, setMeetingId] = useState(null);

  const getMeetingAndToken = async () => {
    const token = await getToken();
    const meetingId = await createMeeting({ token });

    setToken(token);
    setMeetingId(meetingId);
  };

  useEffect(getMeetingAndToken, []);
  return token && meetingId ? (
    <MeetingProvider
      config={{
        meetingId,
        micEnabled: true,
        webcamEnabled: false,
        name: "Participant Name",
      }}
      token={token}
    >
      <MeetingConsumer>
        {() => <MeetingGrid />}
      </MeetingConsumer>
    </MeetingProvider>
  ) : (
    <JoinScreen />
  );
}

export default App;

React JS SDK provides two important hooks API:

useMeeting: Responsible to handle meeting environment.
useParticipant: Responsible to handle

Also, React Provider and Consumer to listen changes in meeting environment.

MeetingProvider: Meeting Provider is Context.Provider that allows consuming components to subscribe to meeting changes.
MeetingConsumer: Meeting Consumer is Context.Consumer that subscribes to meeting changes.

Implement Join Screen

We will start with join screen where user can either create meeting or can join using meetingId.

It contains two simple features:

Create New Meeting
Join the meeting

JoinScreen Component



function JoinScreen() {
  <div>
    <input type="text" placeholder="Enter Meeting Id" onChange={(e) => {setMeetingId(e.target.value)}}  />
    <button  onClick={getMeetingAndToken}>
      Join
    </button>
    <button  onClick={getMeetingAndToken}>
      Create Meeting
    </button>
  </div>
}

Implementing Meeting Grid

Meeting grid will include whole meeting interface. It will be responsible for:

Turn On and Off Mic
Turn On and Off WebCam
Participant View



const {
   join, 
   leave,  
   toggleMic,
   toggleWebcam,
   toggleScreenShare
} = useMeeting();

Let's get started and implement it one by one. useMeeting hook will help you to perform join, leave, toggleMic etc.

Meeting Grid Component



// Helper function for participant loop.
const chunk = (arr) => {
  const newArr = [];
  while (arr.length) newArr.push(arr.splice(0, 3));
  return newArr;
};

function MeetingGrid(props) {
  const [joined, setJoined] = useState(false)
  const {
    join, 
    leave,  
    toggleMic,
    toggleWebcam,
    toggleScreenShare
  } = useMeeting()
  const { participants } = useMeeting();
  const joinMeeting = () => {
    setJoined(true)
    join()
  }
  return (
    <div>
      <header>Meeting Id: {props.meetingId}</header>
      {joined ? 
      (
        <div >
          <button  onClick={leave}>
            Leave
          </button>
          <button  onClick={toggleMic}>
            toggleMic
          </button>
          <button  onClick={toggleWebcam}>
            toggleWebcam
          </button>
          <button  onClick={toggleScreenShare}>
            toggleScreenShare
          </button> 
        </div>
      ) 
      : (<button  onClick={joinMeeting}>
        Join
      </button>)}
      <div
        className="wrapper"
      >
        {chunk([...participants.keys()]).map((k) => (
          <div className="box" key={k} style={{ display: "flex" }}>
            {k.map((l) => (
              <ParticipantView key={l} participantId={l} />
            ))}
          </div>
        ))}
      </div>

    </div>
  )
}

Implementing Participant View

To implement participant grid, we are going to use react-simple-flex-grid. It will be helpful to maintain video grid.

Let's first add this package.



yarn add react-simple-flex-grid

Import react-simple-flex-grid in App Component



import { Row, Col } from 'react-simple-flex-grid';
import "react-simple-flex-grid/lib/main.css";

Participant view will include three major features:

Enable/Disable WebCam
Enable/Disable Mic
Share your screen.

Let's explore each of it. Before starting it, we have to understand useRef of audio, video and screen share element



const webcamRef = useRef(null);
const micRef = useRef(null);
const screenShareRef = useRef(null);

Apart from that, useParticipant hook will help you to handle mic, webcam and screen share.



 const {
    displayName,
    webcamStream,
    micStream,
    screenShareStream,
    webcamOn,
    micOn,
    screenShareOn
  } = useParticipant(props.participantId);

After getting the stream, you can add track to using MediaStream API. For example, check out below code to add reference of webCam



 const mediaStream = new MediaStream();
 mediaStream.addTrack(webcamStream.track);

 webcamRef.current.srcObject = mediaStream;
 webcamRef.current
   .play()
   .catch((error) =>
     console.error("videoElem.current.play() failed", error));

After adding the reference in on loading state of component, you can

ParticipantView Component



function ParticipantView(props) {
  const webcamRef = useRef(null);
  const micRef = useRef(null);
  const screenShareRef = useRef(null);

  const {
    displayName,
    webcamStream,
    micStream,
    screenShareStream,
    webcamOn,
    micOn,
    screenShareOn
  } = useParticipant(props.participantId);

  useEffect(() => {
    if (webcamRef.current) {
      if (webcamOn) {
        const mediaStream = new MediaStream();
        mediaStream.addTrack(webcamStream.track);

        webcamRef.current.srcObject = mediaStream;
        webcamRef.current
          .play()
          .catch((error) =>
            console.error("videoElem.current.play() failed", error)
          );
      } else {
        webcamRef.current.srcObject = null;
      }
    }
  }, [webcamStream, webcamOn]);

  useEffect(() => {
    if (micRef.current) {
      if (micOn) {
        const mediaStream = new MediaStream();
        mediaStream.addTrack(micStream.track);

        micRef.current.srcObject = mediaStream;
        micRef.current
          .play()
          .catch((error) =>
            console.error("videoElem.current.play() failed", error)
          );
      } else {
        micRef.current.srcObject = null;
      }
    }
  }, [micStream, micOn]);

  useEffect(() => {
    if (screenShareRef.current) {
      if (screenShareOn) {
        const mediaStream = new MediaStream();
        mediaStream.addTrack(screenShareStream.track);

        screenShareRef.current.srcObject = mediaStream;
        screenShareRef.current
          .play()
          .catch((error) =>
            console.error("videoElem.current.play() failed", error)
          );
      } else {
        screenShareRef.current.srcObject = null;
      }
    }
  }, [screenShareStream, screenShareOn]);


  return (
    <div key={props.participantId} >
      <audio ref={micRef} autoPlay />
      {webcamRef ||  micOn ? (<div>
      <h2>{displayName}</h2>
      <video
        height={"100%"}
        width={"100%"}
        ref={webcamRef}
        autoPlay
      />
      </div>) : null }
      {screenShareOn ? (
      <div>
        <h2>Screen Shared</h2>
        <video
          height={"100%"}
          width={"100%"}
          ref={screenShareRef}
          autoPlay
        />
      </div>) : null }
      <br/>
      <span>Mic:{micOn ? "Yes": "No"}, Camera: {webcamOn ? "Yes" : "No"}, Screen Share: {screenShareOn ? "Yes" : "No"}</span>
    </div>
  );
}

You can implement further features like cloud recording, chat, whiteboard, social media live etc by refereeing official guide of Video SDK

Find out code whole code on videosdk-react-sdk-tutorial-example

Conclusion

That's how we can integrate video calling API using React JS hooks. You can also check official example videosdk-rtc-react-js-example to extend this application with lots of other amazing features.

You can use this basic video chat app and change a few lines of code for live audio/video streaming.

Embed Video Call on Website & Application

Arjun Kava — Tue, 05 Oct 2021 12:49:59 +0000

Embed and Add video call and chat feature can be easily integrated into any website and application with just a few lines of code in just 5 steps and just 5 minutes. enabling 5000 users to video chat in a web and app.

UPDATE 1 (31st Oct 2021): Updated article with pin screen and layouts configuration

UPDATE 2 (15th Dec 2021): Updated article with whiteboard configuration

Check official Guide: Video SDK Prebuilt SDK for more information.

We will also implement a demo and use Video SDK to make a video call using rtc-js-prebuilt

Introduction

Video calling is becoming norm for everyone. According to Cisco, 87% of the workforce says they feel more connected to each other when using video.

What is video call embed?

Video SDK is provide embed video call in prebuilt SDK adding video calls on your website and application with just a few lines of code and nothing could be simpler than this best video chat experience with just 10 lines of code.

Embed video calling enables the opportunity to integrate real-time communication SDK without writing explicit code. It supports all modern javascript frameworks such as React JS, Vue, and Angular including Vanilla JS.

10,000 minutes free each month
Completely Low code and serverless.
Video API with real-time audio, video and data streams
5,000+ participants support
Chat support with rich media.
Screen sharing with HD and Full HD.
Play realtime video in meeting
Connect it with social media such as Facebook, Youtube etc (RTMP out support).
Intelligent speaker switch
Record your meetings on cloud
Inbuilt support of whiteboard, poll and Q & A.
Customize UI as per your needs.

Let's begin with the prebuilt sdk setup. Read all the steps carefully before installing one in your application.

It supports all the modern frameworks such as plain JavaScript, React JS, Vue JS and Angular JS.

Video Calling Embed in Any Website and App

1: First of all, Register yourself to app.videosdk.live

2: Generate an API key and Secret

Implementation

3: Import the script in your html page



 <script src="https://sdk.videosdk.live/rtc-js-prebuilt/0.3.21/rtc-js-prebuilt.js"></script>

Note: You can also use official Npm package: rtc-js-prebuilt

4: Embed Video Calling javascript prebuilt code to your HTML page



<script>
  var script = document.createElement("script");
  script.type = "text/javascript";

  script.addEventListener("load", function (event) {
    const meeting = new VideoSDKMeeting();

    const config = {
      name: "John Doe",
      apiKey: "<API KEY>", // generated in Setup
      meetingId: "milkyway", // enter your meeting id

      containerId: null,
      redirectOnLeave: "https://www.videosdk.live/",

      micEnabled: true,
      webcamEnabled: true,
      participantCanToggleSelfWebcam: true,
      participantCanToggleSelfMic: true,

      chatEnabled: true,
      screenShareEnabled: true,
      pollEnabled: true,
      whiteBoardEnabled: true,
      raiseHandEnabled: true,

      recordingEnabled: true,
      recordingWebhookUrl: "https://www.videosdk.live/callback",
      participantCanToggleRecording: true,

      brandingEnabled: true,
      brandLogoURL: "https://picsum.photos/200",
      brandName: "Awesome startup",
      poweredBy: true,

      participantCanLeave: true, // if false, leave button won't be visible

      // Live stream meeting to youtube
      livestream: {
        autoStart: true,
        outputs: [
          // {
          //   url: "rtmp://x.rtmp.youtube.com/live2",
          //   streamKey: "<STREAM KEY FROM YOUTUBE>",
          // },
        ],
      },

      whiteboardEnabled: true,

      permissions: {
        askToJoin: false, // Ask joined participants for entry in meeting
        toggleParticipantMic: true, // Can toggle other participant's mic
        toggleParticipantWebcam: true, // Can toggle other participant's webcam
        drawOnWhiteboard: true,
        toggleWhiteboard: true,

      },

      joinScreen: {
        visible: true, // Show the join screen ?
        title: "Daily scrum", // Meeting title
        meetingUrl: window.location.href, // Meeting joining url
      },
    };

    meeting.init(config);
  });

  script.src =
    "https://sdk.videosdk.live/rtc-js-prebuilt/0.1.21/rtc-js-prebuilt.js";
  document.getElementsByTagName("head")[0].appendChild(script);
</script>

5: Insert API key in your code to run it



apiKey: "<API-KEY>"

Note: To restrict use of your API key on other domains, you can add domain URL which will block use of your API key in other domains.

Run the demo

To run the demo, you have to use any local server. Best could be Xampp or IIS as per your environment. We are going to use live-server package to run simple http server.



$ npm install -g live-server
$ live-server --port=8000

and open http://localhost:8000 in your web browser.

Record the meeting

To record the meeting, You have to first enable recording flag in the configuration.

recordingEnabled: set true or false to enable and disable online recordings
recordingWebhookUrl: Callback url, generally helps to store recording details on your database.



 recordingEnabled: true,
 recordingWebhookUrl: "https://www.videosdk.live/callback",

After enabling recording option, you'll be able to see recording button on the screen. Click on it to start and stop recording

Enable WhiteBoard

To enable whiteboard, you have to configure whiteboard flag and permissions.

whiteboardEnabled: if true, it will enable whiteboard
permission: you can also set permission of who can drawOnWhiteboard and toggleWhiteboard by using boolean value



whiteboardEnabled: true,
  permissions: {
    drawOnWhiteboard: true,
    toggleWhiteboard: true,
  },

Go live on social media

To enable live streaming, you have to configure list of stream key and URL. We only support RTMP protocol enabled broadcasting.

autoStart: If true, it will trigger re-streaming automatically when meeting will start.



livestream: {

    autoStart: true,

    outputs: [

        // {

        //   url: "rtmp://x.rtmp.youtube.com/live2",

        //   streamKey: "<STREAM KEY FROM YOUTUBE>",

        // },

    ],

}

Pin Screen

To enable pin screen, you have to configure pin property. It includes two major properties:

allowed: If it is true then that participant can pin or unpin other participants in the meeting, by default it is set to false.
layout: It should be a any of GRID or SPOTLIGHT or SIDEBAR, by default it is set to GRID.

Better understanding of Layouts

GRID

GRID: This layout is default layout if no participants are pinned, it will look same as a normal meeting grid layout, when any participant is pinned that participant will be moved on top of the main screen grid above all non pinned participants

SIDEBAR

SIDEBAR: This layout will have one main screen view and other sidebar grid layout. Only pinned participant will be visible in this layout, all unpinned participants will not be visible in this layout. If more than one participant is pinned then the first participant who was pinned will appear in main screen layout and all other remaining pinned participants will be visible in sidebar.

SPOTLIGHT

SPOTLIGHT: This layout will only contain main screen layout, multiple pinned participants will be visible in main screen view. Same as SIDEBAR layout only pinned participants will be visible in main screen.



pin: {
 allowed: true,
 layout: "SPOTLIGHT",
}

Conclusion

In this article, I have explained Video Chat in JavaScript. You can modify couple of parameters in code according to your need. This was the small introduction to Video SDK Prebuilt SDK.

Here is video version of the same.

I hope this blog helped you understand Video Calling of Video SDK. For more information visit official documentation of Video SDK.

P.S. Find official example of Video SDK No Code Prebuilt App

Official Github

videosdk-live / videosdk-rtc-js-prebuilt-embedded-example

Low Code Prebuilt example of Video SDK using plain JS and HTML.

<p>

Video SDK No Code Prebuilt App.

THIS REPO is DEPRECATED. Please refer to https://github.com/videosdk-live/videosdk-rtc-prebuilt-examples

What is it?

This code sample demonstrates a one-to-one and group video call application built with Video SDK RTC Prebuilt SDK and Video SDK RTC JS SDK

Built for serverless video calling experience.
Scale it upto 5,000 participants with low code.
10,000 minutes free on monthly basis.
Inbuilt video and audio quality optimization.
Inbuilt chat poll, whiteboard, Q and A support.

Features

Completely Low code and serverless.
Video API with real-time audio, video and data streams
5,000+ participants support
Chat support with rich media.
Screen sharing with HD and Full HD.
Play realtime video in meeting
Connect it with social media such as Facebook, Youtube etc (RTMP out support).
Intelligent speaker switch
Record your meetings on cloud
Inbuilt support of whiteboard, poll and Q & A.
Customize UI as per your needs.

Browser Support

Visit our…

View on GitHub