DEV Community: Video SDK

Build AI Voice Agent with Gemini 3.1 Flash Live and VideoSDK

Chaitrali Kakde — Tue, 31 Mar 2026 13:20:13 +0000

Google just launched Gemini 3.1 Flash Live Preview its most capable real-time voice and audio model yet. If you're building AI voice agents, conversational apps, or anything that needs low-latency audio intelligence, this model is a big deal. And with VideoSDK's Python SDK, plugging it into your app takes just a few minutes.

In this blog, we'll walk through what the new model can do, and then build a working voice agent step by step using VideoSDK.

What's New in Gemini 3.1 Flash Live Preview

Google describes this as its "highest-quality audio and voice model yet," and there are a few things that actually back that up.

It's built for real-time, audio-first experiences. Unlike models that convert speech to text and then process it, Gemini 3.1 Flash Live works audio-to-audio meaning it hears you and responds as audio, keeping the conversation feeling natural and fast.

Here's what stands out:

Lower latency than before. Compared to 2.5 Flash Native Audio, this model is noticeably faster. Fewer awkward pauses, snappier responses. That matters a lot when you're building voice agents where delays break the experience.
It actually understands how you say things. The model picks up on acoustic nuances, pitch, pace, tone. So it can tell when you're asking a casual question vs. when you sound urgent or confused.
Better background noise handling. It filters out noise more effectively, which means it works in real environments, not just quiet studios.
Multilingual out of the box. Over 90 languages supported for real-time conversations.
Longer conversation memory. It can follow the thread of a conversation for twice as long as the previous generation. So your agent won't "forget" what was said earlier in a long session.
Tool use during live conversations. This one is huge for agent builders. The model can now trigger external tools (APIs, functions, searches) while a live conversation is happening not just at the end of a turn.
Multimodal awareness. It handles audio and video inputs together, so you can build agents that respond to what they see and hear at the same time.
The model ID is: gemini-3.1-flash-live-preview

Building a Voice Agent with VideoSDK

VideoSDK gives you everything you need to wire Gemini 3.1 Flash Live into a real voice application. Here's how to get set up from scratch.

Step 1 : Create and Activate a Python Virtual Environment

First, create a clean Python environment so your project dependencies stay isolated.

python3 -m venv venv

Activate it:

macOS/Linux

source venv/bin/activate

Windows

venv\Scripts\activate

You should see (venv) in your terminal, which means you're good to go.

Step 2 : Set Up Your Environment Variables

Create a .env file in your project root and add your API keys:

VIDEOSDK_AUTH_TOKEN=your_videosdk_token_here
GOOGLE_API_KEY=your_google_api_key_here

You can get your VideoSDK auth token from the VideoSDK dashboard and your Google API key from Google AI Studio.

Important: when GOOGLE_API_KEY is set in your .env file, do not pass api_key as a parameter in your code the SDK picks it up automatically.

Step 3 : Install the Required Packages

Install VideoSDK's agents SDK along with the Google plugin:

pip install "videosdk-agents[google]"

Step 4 : Create Your Agent (main.py)

Create a file called main.py in your project folder and paste in the following code:

from videosdk.agents import Agent, AgentSession, Pipeline, JobContext, RoomOptions, WorkerJob
from videosdk.plugins.google import GeminiRealtime, GeminiLiveConfig
import logging
logging.basicConfig(level=logging.INFO, format="%(asctime)s - %(name)s - %(levelname)s - %(message)s", handlers=[logging.StreamHandler()])

class MyVoiceAgent(Agent):
    def __init__(self):
        super().__init__(
            instructions="You Are VideoSDK's Voice Agent.You are a helpful voice assistant that can answer questions and help with tasks.",
        )

    async def on_enter(self) -> None:
        await self.session.say("Hello, how can I help you today?")

    async def on_exit(self) -> None:
        await self.session.say("Goodbye!")

async def start_session(context: JobContext):
    agent = MyVoiceAgent()
    model = GeminiRealtime(
        model="gemini-3.1-flash-live-preview",
        # When GOOGLE_API_KEY is set in .env - DON'T pass api_key parameter
        # api_key="AIXXXXXXXXXXXXXXXXXXXX", 
        config=GeminiLiveConfig(
            voice="Leda", # Puck, Charon, Kore, Fenrir, Aoede, Leda, Orus, and Zephyr.
            response_modalities=["AUDIO"]
        )
    )

    pipeline = Pipeline(llm=model)
    session = AgentSession(
        agent=agent,
        pipeline=pipeline
    )

    await session.start(wait_for_participant=True, run_until_shutdown=True)

def make_context() -> JobContext:
    room_options = RoomOptions(
        # room_id="<room_id>", # Replace it with your actual room_id
        name="Gemini Realtime Agent",
        playground=True,
    )

    return JobContext(room_options=room_options)

if __name__ == "__main__":
    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
    job.start()

To run the agent:

python main.py

Once you run this command, a playground URL will appear in your terminal. You can use this URL to interact with your AI agent.

What Can You Build With This?

Gemini 3.1 Flash Live + VideoSDK opens up a pretty wide range of real-world use cases:

Customer support voice bots. Replace or supplement your call center with agents that actually understand tone and can handle multilingual customers in real time.
AI meeting assistants. Agents that join calls, take notes, answer questions from participants, and trigger follow-up actions mid-conversation.
Healthcare intake agents. Voice-based triage agents that collect patient information, ask follow-up questions, and route to the right department all in a natural spoken conversation.
Language tutors. Real-time conversation partners that catch pronunciation issues, adjust their pace based on the learner, and respond naturally.
Voice-controlled IoT and home automation. Agents that listen continuously, understand context, and trigger device actions through tool use all in sub-second response times.
Live interview prep tools. Candidates practice answering questions aloud and get spoken feedback instantly.

Conclusion

Gemini 3.1 Flash Live Preview is a meaningful step forward for real-time voice AI. The improvements in latency, noise handling, multilingual support, and especially live tool use make it a strong foundation for production voice agents.

VideoSDK wraps all of that into a clean Python SDK that gets you from zero to a running agent in a handful of lines. Whether you're prototyping or building something you intend to ship, the setup here gives you a solid starting point.

Next Steps and Resources

Check Gemini3.1 implementation docs
Learn how to deploy your agents
👉 Share your thoughts, roadblocks, or success stories in the comments or join our Discord community ↗. We’re excited to learn from your journey and help you build even better AI-powered communication

Announcing VideoSDK Inference: One Magic API for Every Voice AI Model

Chaitrali Kakde — Tue, 17 Feb 2026 07:08:59 +0000

Building AI voice agents has always been powerful, but slow. You had models STT, LLMs, TTS and the tools to use them. But maintaining accounts across multiple vendors for speech recognition, language models, and speech synthesis, each with its own keys, quotas, billing, and APIs was a major challenge.

Today, that changes.

We’re thrilled to announce Inferencing in VideoSDK AI Voice Agents a unified way to run STT, LLM, TTS, and Realtime models directly inside your voice pipeline without managing multiple accounts through Agent Runtime Dashboard and Python Agents SDK.

Inferencing works in both the CascadingPipeline and the RealtimePipeline, giving you full flexibility to build modular or fully streaming voice agents. Whether you want incremental transcripts, staged execution, or fully native realtime audio, Inferencing makes it easy.

Supported Categories

STT: Sarvam, Google, Deepgram
LLMs: Google Gemini
TTS: Sarvam, Google, Cartesia
Realtime: Gemini Native Audio

What is VideoSDK Inference?

VideoSDK Inference is a managed gateway that gives you access to multiple AI models. All without providing your own API keys for providers like Sarvam AI or Google Gemini.

Authentication, routing, retries, and billing are handled by VideoSDK usage is simply charged against your VideoSDK account balance.

Inferencing via Agent Runtime Dashboard

Inferencing in VideoSDK is now fully accessible through the dashboard, giving developers direct control over model selection and pipeline configuration without needing to manage infrastructure manually.

From the dashboard, developers can:

Select STT, LLM, TTS, or Realtime models and enable them in the pipeline with a single click.
Switch providers instantly, allowing rapid experimentation and iteration .
Attach deployment endpoints for web or telephony, making the agent immediately accessible to users.
With this approach, ideas move from configuration to live, interactive conversations in minutes, making it possible to test new workflows, swap models, or iterate on conversational design almost instantly.

Watch this video to get started 👇

Inferencing via Code (Agents SDK)

With VideoSDK Inferencing, developers can now integrate STT, LLM, TTS, and Realtime models directly into their voice agents all handled inside the VideoSDK. This enables rapid experimentation, modular pipelines, and low-latency real-time conversations.

Installation

The Inference plugin is included in the core VideoSDK Agents SDK. Install it via

pip install videosdk-agents

CascadingPipeline Example

The CascadingPipeline is ideal for modular, stage-by-stage processing. Here’s an example of building a simple agent using STT, LLM, and TTS via the VideoSDK Inference Gateway:

pipeline = CascadingPipeline(
        stt=STT.sarvam(model_id="saarika:v2.5", language="en-IN"),
        llm=LLM.google(model="gemini-2.5-flash"),
        tts=TTS.sarvam(model_id="bulbul:v2", speaker="anushka", language="en-IN"),
        vad=SileroVAD()
)

RealTimePipeline Example

For low-latency, fully streaming voice agents, the RealTimePipeline handles Realtime inference with minimal delay. Here’s an example using Gemini Live Native Audio:

pipeline = RealTimePipeline(
        model=Realtime.gemini(
            model="gemini-2.5-flash-native-audio-preview-12-2025",
            voice="Puck",
            language_code="en-US",
            response_modalities=["AUDIO"],
            temperature=0.7
        )
)

*With this approach, developers retain *

Full programmatic control over pipeline stages, model parameters, and execution behavior.
Modular provider replacement, making it easy to swap STT, LLM, or TTS engines.

The result: a fully configurable, production-ready AI voice agent that can be deployed in minutes.

Conclusion

Voice AI is no longer limited by model capability. It’s limited by how fast you can deploy it. With Inferencing in VideoSDK AI Voice Agents, deployment becomes effortless. Whether through the dashboard or programmatically via the SDK, you can build, select, enable, and go live in minutes.

The era of modular, low-latency, real-time voice agents is here. With Inferencing, your ideas move from concept to conversation faster than ever.

Build. Select. Configure. Go live.

Resources and Next Steps

For More Information Read the Inference documentation.
Learn how to deploy your AI Agents.
Sign up at VideoSDK Dashboard
👉 Share your thoughts, roadblocks, or success stories in the comments or join our Discord community ↗. We’re excited to learn from your journey and help you build even better AI-powered communication tools!

Introducing Phone Numbers: Build AI Telephony Agents in 60 seconds

Chaitrali Kakde — Tue, 17 Feb 2026 06:55:33 +0000

Today, we’re launching VideoSDK Phone Numbers a first-party telephony capability that lets you connect voice agents and calling workflows directly to the phone network, without relying on third-party providers like Twilio.

You can now purchase phone numbers straight from the VideoSDK Dashboard, attach them to your agent or calling logic, and go live in minutes.

No external accounts. No SIP trunk configuration. No extra setup.

Fewer Hops. Better Calls.

Until now, developers building phone-based voice experiences on VideoSDK typically had to:

Sign up with a third-party telephony provider (like Twilio)
Purchase and manage phone numbers externally
Configure SIP trunks and credentials
Integrate and debug multiple systems before making the first call
By offering native phone numbers directly within VideoSDK, we remove those extra layers and make phone-based voice apps significantly easier to build and operate in just 3 steps.

How It Works

Getting started with VideoSDK Phone Numbers is simple:

Buy a Phone Number : Search and purchase available phone numbers directly from the VideoSDK Dashboard.
Attach Your Agent or Logic : Associate the phone number with your voice agent or inbound call routing logic so incoming calls are handled automatically.
Go Live : Call the number and your agent answers instantly.
Manage numbers, update routing, and monitor usage all from one place.

What You Can Build

Phone calls remain critical across many industries. With VideoSDK Phone Numbers, you can spin up phone-based voice agents in minutes for:

Sales qualification and lead routing
Order tracking and delivery updates
Appointment booking and status updates
Restaurant and service business call automation
If your application involves real-time voice and the phone network, this feature is built for you

Resources

Sign up at VideoSDK - dashboard
Have feedback or ideas? Comment below or join our Discord to build and ship AI call agents faster with VideoSDK Discord community ↗. We’re excited to learn from your journey and help you build even better AI-powered communication tools!

Build Real-time transcription app using React Hooks

Arjun Kava — Sat, 25 May 2024 17:55:26 +0000

Step-by-Step Tutorial for the Code

This tutorial will guide you through the setup and functioning of the React app using the VideoSDK.live SDK. The app allows users to join or create meetings and provides functionalities for recording and transcription.

Prerequisites

Node.js and npm: Ensure you have Node.js and npm installed.
VideoSDK Account: Sign up at VideoSDK.live and get your API key.

Step 1: Setup the Project

Create a React App: If you don't already have a React app, create one using the following command:

   npx create-react-app my-video-app
   cd my-video-app

Install Dependencies: Install the necessary packages.

   npm install @videosdk.live/react-sdk react-player

Step 2: Setup API Functions

Create an API.js file in the src directory to handle API interactions.

// API.js
export const authToken = "YOUR_VIDEO_SDK_AUTH_TOKEN";

export const createMeeting = async ({ token }) => {
  const response = await fetch("https://api.videosdk.live/v1/meetings", {
    method: "POST",
    headers: {
      Authorization: token,
      "Content-Type": "application/json",
    },
  });
  const data = await response.json();
  return data.meetingId;
};

Step 3: Build the Components

In your App.js, import necessary modules and create components as described below.

JoinScreen Component

Allows users to enter a meeting ID or create a new meeting.

import React, { useState } from "react";

function JoinScreen({ getMeetingAndToken }) {
  const [meetingId, setMeetingId] = useState(null);
  const onClick = async () => {
    await getMeetingAndToken(meetingId);
  };
  return (
    <div>
      <input
        type="text"
        placeholder="Enter Meeting Id"
        onChange={(e) => {
          setMeetingId(e.target.value);
        }}
      />
      <button onClick={onClick}>Join</button>
      {" or "}
      <button onClick={onClick}>Create Meeting</button>
    </div>
  );
}

export default JoinScreen;

ParticipantView Component

Displays the video and audio of a participant.

import React, { useEffect, useMemo, useRef } from "react";
import { useParticipant } from "@videosdk.live/react-sdk";
import ReactPlayer from "react-player";

function ParticipantView(props) {
  const micRef = useRef(null);
  const { webcamStream, micStream, webcamOn, micOn, isLocal, displayName } =
    useParticipant(props.participantId);

  const videoStream = useMemo(() => {
    if (webcamOn && webcamStream) {
      const mediaStream = new MediaStream();
      mediaStream.addTrack(webcamStream.track);
      return mediaStream;
    }
  }, [webcamStream, webcamOn]);

  useEffect(() => {
    if (micRef.current) {
      if (micOn && micStream) {
        const mediaStream = new MediaStream();
        mediaStream.addTrack(micStream.track);

        micRef.current.srcObject = mediaStream;
        micRef.current
          .play()
          .catch((error) =>
            console.error("videoElem.current.play() failed", error)
          );
      } else {
        micRef.current.srcObject = null;
      }
    }
  }, [micStream, micOn]);

  return (
    <div>
      <p>
        Participant: {displayName} | Webcam: {webcamOn ? "ON" : "OFF"} | Mic:{" "}
        {micOn ? "ON" : "OFF"}
      </p>
      <audio ref={micRef} autoPlay playsInline muted={isLocal} />
      {webcamOn && (
        <ReactPlayer
          playsinline
          pip={false}
          light={false}
          controls={false}
          muted={true}
          playing={true}
          url={videoStream}
          height={"300px"}
          width={"300px"}
          onError={(err) => {
            console.log(err, "participant video error");
          }}
        />
      )}
    </div>
  );
}

export default ParticipantView;

Controls Component

Provides buttons to leave the meeting and toggle mic/webcam.

import React from "react";
import { useMeeting } from "@videosdk.live/react-sdk";

function Controls() {
  const { leave, toggleMic, toggleWebcam } = useMeeting();
  return (
    <div>
      <button onClick={() => leave()}>Leave</button>
      <button onClick={() => toggleMic()}>toggleMic</button>
      <button onClick={() => toggleWebcam()}>toggleWebcam</button>
    </div>
  );
}

export default Controls;

MeetingView Component

Main component to handle meeting functionalities like transcription, recording, and displaying participants.

import React, { useState } from "react";
import { useMeeting, useTranscription, Constants } from "@videosdk.live/react-sdk";
import ParticipantView from "./ParticipantView";
import Controls from "./Controls";

function MeetingView(props) {
  const [transcript, setTranscript] = useState("Transcription");
  const [transcriptState, setTranscriptState] = useState("Not Started");
  const tConfig = { webhookUrl: "https://www.example.com" };
  const { startTranscription, stopTranscription } = useTranscription({
    onTranscriptionStateChanged: (data) => {
      const { status } = data;
      if (status === Constants.transcriptionEvents.TRANSCRIPTION_STARTING) {
        setTranscriptState("Transcription Starting");
      } else if (status === Constants.transcriptionEvents.TRANSCRIPTION_STARTED) {
        setTranscriptState("Transcription Started");
      } else if (status === Constants.transcriptionEvents.TRANSCRIPTION_STOPPING) {
        setTranscriptState("Transcription Stopping");
      } else if (status === Constants.transcriptionEvents.TRANSCRIPTION_STOPPED) {
        setTranscriptState("Transcription Stopped");
      }
    },
    onTranscriptionText: (data) => {
      let { participantName, text, timestamp } = data;
      console.log(`${participantName}: ${text} ${timestamp}`);
      setTranscript(transcript + `${participantName}: ${text} ${timestamp}`);
    },
  });

  const { startRecording, stopRecording } = useMeeting();

  const handleStartRecording = () => {
    startRecording("YOUR_WEB_HOOK_URL", "AWS_Directory_Path", {
      layout: { type: "GRID", priority: "SPEAKER", gridSize: 4 },
      theme: "DARK",
      mode: "video-and-audio",
      quality: "high",
      orientation: "landscape",
    });
  };

  const handleStopRecording = () => stopRecording();
  const handleStartTranscription = () => startTranscription(tConfig);
  const handleStopTranscription = () => stopTranscription();

  const [joined, setJoined] = useState(null);
  const { join, participants } = useMeeting({
    onMeetingJoined: () => setJoined("JOINED"),
    onMeetingLeft: () => props.onMeetingLeave(),
  });

  const joinMeeting = () => {
    setJoined("JOINING");
    join();
  };

  return (
    <div className="container">
      <h3>Meeting Id: {props.meetingId}</h3>
      {joined && joined === "JOINED" ? (
        <div>
          <Controls />
          <button onClick={handleStartRecording}>Start Recording</button>
          <button onClick={handleStopRecording}>Stop Recording</button>
          <button onClick={handleStartTranscription}>Start Transcription</button>
          <button onClick={handleStopTranscription}>Stop Transcription</button>
          {[...participants.keys()].map((participantId) => (
            <ParticipantView participantId={participantId} key={participantId} />
          ))}
          <p>State: {transcriptState}</p>
          <p>{transcript}</p>
        </div>
      ) : joined && joined === "JOINING" ? (
        <p>Joining the meeting...</p>
      ) : (
        <button onClick={joinMeeting}>Join</button>
      )}
    </div>
  );
}

export default MeetingView;

App Component

Main component that manages the meeting state and provides the necessary context.

import React, { useState } from "react";
import { MeetingProvider } from "@videosdk.live/react-sdk";
import JoinScreen from "./JoinScreen";
import MeetingView from "./MeetingView";
import { authToken, createMeeting } from "./API";

function App() {
  const [meetingId, setMeetingId] = useState(null);

  const getMeetingAndToken = async (id) => {
    const meetingId =
      id == null ? await createMeeting({ token: authToken }) : id;
    setMeetingId(meetingId);
  };

  const onMeetingLeave = () => setMeetingId(null);

  return authToken && meetingId ? (
    <MeetingProvider
      config={{
        meetingId,
        micEnabled: true,
        webcamEnabled: true,
        name: "C.V. Raman",
      }}
      token={authToken}
    >
      <MeetingView meetingId={meetingId} onMeetingLeave={onMeetingLeave} />
    </MeetingProvider

>
  ) : (
    <JoinScreen getMeetingAndToken={getMeetingAndToken} />
  );
}

export default App;

Step 4: Run the Application

Start the React App:

   npm start

Navigate to Your Browser: Open http://localhost:3000 to view the app.

Optional Step: Add CSS for Styling

To enhance the visual appeal of your app, you can add the following CSS.

Create an App.css file in your src directory with the following content:

/* App.css */

body {
  background-color: #121212;
  color: #e0e0e0;
  font-family: 'Roboto', sans-serif;
  margin: 0;
  padding: 0;
}

input, button {
  background-color: #1e1e1e;
  border: 1px solid #333;
  color: #e0e0e0;
  padding: 10px;
  margin: 5px;
  border-radius: 5px;
}

button:hover {
  background-color: #333;
  cursor: pointer;
}

.container {
  max-width: 800px;
  margin: auto;
  padding: 20px;
  text-align: center;
}

h3 {
  color: #f5f5f5;
}

p {
  margin: 10px 0;
}

audio, .react-player__preview {
  background-color: #333;
  border: 1px solid #555;
  border-radius: 5px;
  margin: 10px 0;
}

.react-player__preview img {
  border-radius: 5px;
}

.react-player__shadow {
  border-radius: 5px;
}

Summary

You have created a functional video meeting application using the VideoSDK.live SDK. This app allows users to join or create meetings, manage participants, and control functionalities like recording and transcription. The optional CSS step ensures a consistent and visually appealing user interface.

How to Integrate Image Capture in JavaScript Video Chat App?

Kishan Nakrani — Fri, 03 May 2024 05:49:42 +0000

📌 Introduction

Integrating image capture in a JavaScript video chat app enhances user interaction and functionality. With this feature, users can take snapshots during video calls, adding a new dimension to their communication experience. By incorporating image capture, the app enables users to capture memorable moments or important information shared during the call. JavaScript's flexibility allows seamless integration of image capture, ensuring a smooth user experience across different devices and browsers.

Benefits of Integrate Image Capture in JavaScript Video Chat App:

Enhanced Communication : Image capture adds a visual element to video calls, improving communication and understanding.
Memorable Moments : Users can capture snapshots of important moments during the call, preserving memories.
Increased Engagement : Interactive features like image capture keep users engaged and active during video calls.

Use Case of Integrate Image Capture in JavaScript Video Chat App:

Education: Students can capture whiteboard content or diagrams shared during online classes for later review.
Business Meetings: Participants can capture key points discussed in meetings or presentations, ensuring clarity and accountability.
Remote Collaboration: Teams working remotely can capture design mockups, charts, or code snippets for collaborative brainstorming sessions.

This tutorial will guide you through a step-by-step process of how to integrate image capture functionality in a JavaScript chat App with VideoSDK.

🚀 Getting Started with VideoSDK

To take advantage of Image Feature Integration functionality, we must use the capabilities that the VideoSDK offers. Before diving into the implementation steps, let's ensure you complete the necessary prerequisites.

Create a VideoSDK Account

Go to your VideoSDK dashboard. This account gives you access to the required Video SDK token, which acts as an authentication key that allows your application to interact with VideoSDK functionality.

Generate your Auth Token

Visit your VideoSDK dashboard and navigate to the "API Key" section to generate your auth token. This token is crucial in authorizing your application to use VideoSDK features. For a more visual understanding of the account creation and token generation process, consider referring to the provided tutorial.

Prerequisites

Before proceeding, ensure that your development environment meets the following requirements:

VideoSDK Developer Account (if you do not have one, follow VideoSDK Dashboard)
Have Node and NPM installed on your device.

🛠️ Install VideoSDK

Import VideoSDK using the <script> tag or install it using the following npm command. Make sure you are in your app directory before you run this command.

// JavaScript
<html>
  <head>
    <!--.....-->
  </head>
  <body>
    <!--.....-->
    <script src="https://sdk.videosdk.live/js-sdk/0.0.83/videosdk.js"></script>
  </body>
</html>

npm

npm install @videosdk.live/js-sdk

Yarn

yarn add @videosdk.live/js-sdk

Structure of the project

Your project structure should look like this.

  root
   ├── index.html
   ├── config.js
   ├── index.js

You will be working on the following files:

index.html: Responsible for creating a basic UI.
config.js: Responsible for storing the token.
index.js: Responsible for rendering the meeting view and the join meeting functionality.

🎥 Essential Steps to Implement Video Call Functionality

Once you've successfully installed VideoSDK in your project, you'll have access to a range of functionalities for building your video call application. Image Capture is one such feature that leverages VideoSDK's capabilities. It leverages VideoSDK's capabilities to identify the user with the strongest audio signal (the one speaking).

Step 1: Design the user interface (UI)

Create an HTML file containing the screens, join-screen and grid-screen.

<!DOCTYPE html>
<html>
  <head> </head>

  <body>
    <div id="join-screen">
      <!-- Create new Meeting Button -->
      <button id="createMeetingBtn">New Meeting</button>
      OR
      <!-- Join existing Meeting -->
      <input type="text" id="meetingIdTxt" placeholder="Enter Meeting id" />
      <button id="joinBtn">Join Meeting</button>
    </div>

    <!-- for Managing meeting status -->
    <div id="textDiv"></div>

    <div id="grid-screen" style="display: none">
      <!-- To Display MeetingId -->
      <h3 id="meetingIdHeading"></h3>

      <!-- Controllers -->
      <button id="leaveBtn">Leave</button>
      <button id="toggleMicBtn">Toggle Mic</button>
      <button id="toggleWebCamBtn">Toggle WebCam</button>

      <!-- render Video -->
      <div class="row" id="videoContainer"></div>
    </div>

    <!-- Add VideoSDK script -->
    <script src="https://sdk.videosdk.live/js-sdk/0.0.83/videosdk.js"></script>
    <script src="config.js"></script>
    <script src="index.js"></script>
  </body>
</html>

Step 2: Implement Join Screen

Configure the token in the config.js file, which you can obtain from the VideoSDK Dashboard.

// Auth token will be used to generate a meeting and connect to it
TOKEN = "Your_Token_Here";

Next, retrieve all the elements from the DOM and declare the following variables in the index.js file. Then, add an event listener to the join and create meeting buttons.

// Getting Elements from DOM
const joinButton = document.getElementById("joinBtn");
const leaveButton = document.getElementById("leaveBtn");
const toggleMicButton = document.getElementById("toggleMicBtn");
const toggleWebCamButton = document.getElementById("toggleWebCamBtn");
const createButton = document.getElementById("createMeetingBtn");
const videoContainer = document.getElementById("videoContainer");
const textDiv = document.getElementById("textDiv");

// Declare Variables
let meeting = null;
let meetingId = "";
let isMicOn = false;
let isWebCamOn = false;

function initializeMeeting() {}

function createLocalParticipant() {}

function createVideoElement() {}

function createAudioElement() {}

function setTrack() {}

// Join Meeting Button Event Listener
joinButton.addEventListener("click", async () => {
  document.getElementById("join-screen").style.display = "none";
  textDiv.textContent = "Joining the meeting...";

  roomId = document.getElementById("meetingIdTxt").value;
  meetingId = roomId;

  initializeMeeting();
});

// Create Meeting Button Event Listener
createButton.addEventListener("click", async () => {
  document.getElementById("join-screen").style.display = "none";
  textDiv.textContent = "Please wait, we are joining the meeting";

  // API call to create meeting
  const url = `https://api.videosdk.live/v2/rooms`;
  const options = {
    method: "POST",
    headers: { Authorization: TOKEN, "Content-Type": "application/json" },
  };

  const { roomId } = await fetch(url, options)
    .then((response) => response.json())
    .catch((error) => alert("error", error));
  meetingId = roomId;

  initializeMeeting();
});

Step 3: Initialize Meeting

Following that, initialize the meeting using the initMeeting() function and proceed to join the meeting.

// Initialize meeting
function initializeMeeting() {
  window.VideoSDK.config(TOKEN);

  meeting = window.VideoSDK.initMeeting({
    meetingId: meetingId, // required
    name: "Thomas Edison", // required
    micEnabled: true, // optional, default: true
    webcamEnabled: true, // optional, default: true
  });

  meeting.join();

  // Creating local participant
  createLocalParticipant();

  // Setting local participant stream
  meeting.localParticipant.on("stream-enabled", (stream) => {
    setTrack(stream, null, meeting.localParticipant, true);
  });

  // meeting joined event
  meeting.on("meeting-joined", () => {
    textDiv.style.display = "none";
    document.getElementById("grid-screen").style.display = "block";
    document.getElementById(
      "meetingIdHeading"
    ).textContent = `Meeting Id: ${meetingId}`;
  });

  // meeting left event
  meeting.on("meeting-left", () => {
    videoContainer.innerHTML = "";
  });

  // Remote participants Event
  // participant joined
  meeting.on("participant-joined", (participant) => {
    // ...
  });

  // participant left
  meeting.on("participant-left", (participant) => {
    // ...
  });
}

Step 4: Create the Media Elements

In this step, Create a function to generate audio and video elements for displaying both local and remote participants. Set the corresponding media track based on whether it's a video or audio stream.

// creating video element
function createVideoElement(pId, name) {
  let videoFrame = document.createElement("div");
  videoFrame.setAttribute("id", `f-${pId}`);
  videoFrame.style.width = "300px";


  //create video
  let videoElement = document.createElement("video");
  videoElement.classList.add("video-frame");
  videoElement.setAttribute("id", `v-${pId}`);
  videoElement.setAttribute("playsinline", true);
  videoElement.setAttribute("width", "300");
  videoFrame.appendChild(videoElement);

  let displayName = document.createElement("div");
  displayName.innerHTML = `Name : ${name}`;

  videoFrame.appendChild(displayName);
  return videoFrame;
}

// creating audio element
function createAudioElement(pId) {
  let audioElement = document.createElement("audio");
  audioElement.setAttribute("autoPlay", "false");
  audioElement.setAttribute("playsInline", "true");
  audioElement.setAttribute("controls", "false");
  audioElement.setAttribute("id", `a-${pId}`);
  audioElement.style.display = "none";
  return audioElement;
}

// creating local participant
function createLocalParticipant() {
  let localParticipant = createVideoElement(
    meeting.localParticipant.id,
    meeting.localParticipant.displayName
  );
  videoContainer.appendChild(localParticipant);
}

// setting media track
function setTrack(stream, audioElement, participant, isLocal) {
  if (stream.kind == "video") {
    isWebCamOn = true;
    const mediaStream = new MediaStream();
    mediaStream.addTrack(stream.track);
    let videoElm = document.getElementById(`v-${participant.id}`);
    videoElm.srcObject = mediaStream;
    videoElm
      .play()
      .catch((error) =>
        console.error("videoElem.current.play() failed", error)
      );
  }
  if (stream.kind == "audio") {
    if (isLocal) {
      isMicOn = true;
    } else {
      const mediaStream = new MediaStream();
      mediaStream.addTrack(stream.track);
      audioElement.srcObject = mediaStream;
      audioElement
        .play()
        .catch((error) => console.error("audioElem.play() failed", error));
    }
  }
}

Step 5: Handle participant events

Thereafter, implement the events related to the participants and the stream.

The following are the events to be executed in this step:

participant-joined: When a remote participant joins, this event will trigger. In the event callback, create video and audio elements previously defined for rendering their video and audio streams.
participant-left: When a remote participant leaves, this event will trigger. In the event callback, remove the corresponding video and audio elements.
stream-enabled: This event manages the media track of a specific participant by associating it with the appropriate video or audio element.

// Initialize meeting
function initializeMeeting() {
  // ...

  // participant joined
  meeting.on("participant-joined", (participant) => {
    let videoElement = createVideoElement(
      participant.id,
      participant.displayName
    );
    let audioElement = createAudioElement(participant.id);
    // stream-enabled
    participant.on("stream-enabled", (stream) => {
      setTrack(stream, audioElement, participant, false);
    });
    videoContainer.appendChild(videoElement);
    videoContainer.appendChild(audioElement);
  });

  // participants left
  meeting.on("participant-left", (participant) => {
    let vElement = document.getElementById(`f-${participant.id}`);
    vElement.remove(vElement);

    let aElement = document.getElementById(`a-${participant.id}`);
    aElement.remove(aElement);
  });
}

Step 6: Implement Controls

Next, implement the meeting controls such as toggleMic, toggleWebcam, and leave the meeting.

// leave Meeting Button Event Listener
leaveButton.addEventListener("click", async () => {
  meeting?.leave();
  document.getElementById("grid-screen").style.display = "none";
  document.getElementById("join-screen").style.display = "block";
});

// Toggle Mic Button Event Listener
toggleMicButton.addEventListener("click", async () => {
  if (isMicOn) {
    // Disable Mic in Meeting
    meeting?.muteMic();
  } else {
    // Enable Mic in Meeting
    meeting?.unmuteMic();
  }
  isMicOn = !isMicOn;
});

// Toggle Web Cam Button Event Listener
toggleWebCamButton.addEventListener("click", async () => {
  if (isWebCamOn) {
    // Disable Webcam in Meeting
    meeting?.disableWebcam();

    let vElement = document.getElementById(`f-${meeting.localParticipant.id}`);
    vElement.style.display = "none";
  } else {
    // Enable Webcam in Meeting
    meeting?.enableWebcam();

    let vElement = document.getElementById(`f-${meeting.localParticipant.id}`);
    vElement.style.display = "inline";
  }
  isWebCamOn = !isWebCamOn;
});

You can check out the complete code here.

Integrate Image Capture Feature

This guide provides instructions on capturing images of participants from a video stream. This capability proves particularly valuable in Video KYC scenarios, enabling the capture of images where users can hold up their identity for verification.

By using the captureImage() method of the Participant class, you can capture an image of a local participant from their video stream.

You have the option to specify the desired height and width in the captureImage() function; however, these parameters are optional. If not provided, the VideoSDK will automatically use the dimensions of the local participant's webcamStream.
The captureImage() function returns the image in the form of a base64 string.

let meeting;

// Initialize Meeting
meeting = VideoSDK.initMeeting({
  // ...
});

let isWebcamOn; // status of your webcam, true/false

async function imageCapture() {
  if (isWebcamOn) {
    const base64Data = await meeting.localParticipant.captureImage(); // captureImage will return base64 string
    console.log("base64", base64);
  } else {
    console.error("Camera must be on to capture an image");
  }
}

TIP:

Rather than utilizing the participants.get(participantId).captureImage() method to capture an image of a remote participant, it is advisable to refer to the provided documentation for a more effective approach.

The participants.get(participantId).captureImage() method captures an image from the current video stream being consumed from the remote participant. The alternative documentation is likely to provide a better and more appropriate method to achieve the desired result.

How to capture an image of a remote participant?

Before proceeding, it's crucial to understand VideoSDK's temporary file storage system and the underlying pubsub mechanism.
Here's a breakdown of the steps, using the names Participant A and Participant B for clarity:

Step 1: Initiate Image Capture Request

In this step, you have to first send a request to Participant B, whose image you want to capture, using Pubsub.
To do that, you have to create a Pubsub topic called IMAGE_CAPTURE the index.js File.
Here, you will be using the sendOnly property of the publish() method. Therefore, the request will be sent to that participant only.

let meeting;

// Initialize Meeting
meeting = VideoSDK.initMeeting({
  // ...
});

function sendRequest({ participantId }) {
  // Pass the participantId of the participant twhose image you want to capture 
  // Here, it will be Participant B's id, as you want to capture the the image of Participant B
  let message = "Sending request to capture image";
  meeting.pubSub
    .publish("IMAGE_CAPTURE", message, {
      persist: false,
      sendOnly: [participantId],
    })
    .then((res) => console.log(`response of publish : ${res}`))
    .catch((err) => console.log(`error of publish : ${err}`));
}

Place one button to capture an image of a remote participant in index.js file inside createVideoElement function.

// creating video element
function createVideoElement(pId, name) {
// ...

  // Create wrapper div
  let wrapperDiv = document.createElement("div");

  // Create button element
  let buttonElement = document.createElement("button");
  buttonElement.setAttribute("id", `btnCaptureImage-${pId}`);
  buttonElement.textContent = "CaptureImage";
  buttonElement.classList.add("capture-button");

  // Append video and button elements to the wrapper div
  wrapperDiv.appendChild(videoElement);
  wrapperDiv.appendChild(buttonElement);

  // Append the wrapper div before the video element
  division.appendChild(wrapperDiv);

  // Set up event listener for hover effect

  wrapperDiv.addEventListener("mouseover", function () {
    buttonElement.style.display = "block";
  });

  wrapperDiv.addEventListener("mouseout", function () {
    buttonElement.style.display = "none";
  });

  wrapperDiv.style.width = `${videoElement.getAttribute("width")}px`;
  wrapperDiv.style.height = `${videoElement.getAttribute("height")}px`;

//...
}

Now on the capture Image button click the call sendRequest() in index.js

// creating video element
function createVideoElement(pId, name) {
// ...

    buttonElement.addEventListener("click", async function () {
    if (pId == meeting.localParticipant.id) {
      const base64Data = await meeting.localParticipant.captureImage();
      base64 = "data:data:image/jpeg;base64," + base64Data;
      captureImageDiv.style.display = "block";
      captureImage.src = base64;
      captureImage.onload = function () {
        alert(this.width + "x" + this.height);
      };
    } else {
      let participantId = await participants.get(pId);
      sendRequest({ participantId: participantId.id });
    }
  });
}

Step 2: Capture and Upload File

To capture an image from the remote participant [Participant B], you have to subscribe to the pub-sub topic on the meeting-joined event of the Meeting class. When a participant receives an image capture request, this component uses the captureImage method of the Participant class to capture the image.

let meeting;

// Initialize Meeting
meeting = VideoSDK.initMeeting({
  // ...
});

async function captureAndStoreImage() {
  // capture image
  const base64Data = await meeting.localParticipant.captureImage();
  console.log("base64Data", base64Data);
}

const _handleOnImageCaptureMessageReceived = (message) => {
  try {
    if (message.senderId !== meeting.localParticipant.id) {
      // capture and store image when message received
      captureAndStoreImage();
    }
  } catch (err) {
    console.log("error on image capture", err);
  }
};

meeting.on("meeting-joined", () => {
  // ...
  meeting.pubSub.subscribe("IMAGE_CAPTURE", (data) => {
    _handleOnImageCaptureMessageReceived(data);
  });
});

The captured image is then stored in VideoSDK's temporary file storage system using the uploadBase64File() function of the Meeting class. This operation returns a unique fileUrl of the stored image.

let meeting;

// Initialize Meeting
meeting = VideoSDK.initMeeting({
  // ...
});

async function captureAndStoreImage() {
  // capture image
  const base64Data = await meeting.localParticipant.captureImage();
  // upload image to videosdk storage system
  const fileUrl = await meeting.uploadBase64File({
    base64Data,
    token: "VIDEOSDK_TOKEN",
    fileName: "myImage.jpeg", // specify a name for image file with extension
  });

  console.log("fileUrl", fileUrl);
}

Next, the fileUrl is sent back to the participant who initiated the request using the IMAGE_TRANSFER topic.

async function captureAndStoreImage() {
  // ...

  const fileUrl = await meeting.uploadBase64File({
    base64Data,
    token: "VIDEOSDK_TOKEN",
    fileName: "myImage.jpeg", // specify a name for image file with extension
  });

  // publish image Transfer
  meeting.pubSub
    .publish("IMAGE_TRANSFER", fileUrl, {
      persist: false,
      sendOnly: [senderId],
    })
    .then((res) => console.log(`response of publish : ${res}`))
    .catch((err) => console.log(`error of publish : ${err}`));
}

Step 2: Fetch and Display Image

Upon publishing on the IMAGE_TRANSFER topic, subscribe to the same topic within the meeting-joined event of the Meeting class. This will provide access to the fileUrl associated with the captured image. Once obtained, use the fetchBase64File() method of the Meeting class to retrieve the file in base64 format from VideoSDK's temporary storage.

async function captureImageAndDisplay(message) {
  const token = "VIDEOSDK_TOKEN";
  let base64 = await meeting.fetchBase64File({
    url: message.message,
    token,
  });
  console.log("base64", base64); // here is your image in a form of base64
}

meeting.on("meeting-joined", () => {
  // ...
  meeting.pubSub.subscribe("IMAGE_TRANSFER", (data) => {
    if (data.senderId !== meeting.localParticipant.id) {
      captureImageAndDisplay(data);
    }
  });
});

With the base64 data in hand, you can now display the image.

let captureImage = document.getElementById("captureImage");

async function captureImageAndDisplay(message) {
  const token = "VIDEOSDK_TOKEN";
  let base64 = await meeting.fetchBase64File({
    url: message.message,
    token,
  });
  console.log("base64", base64); // here is your image in a form of base64

  base64 = "data:data:image/jpeg;base64," + base64;
  captureImage.src = base64;
}

<!-- -->
<img id="captureImage" />

NOTE:

The file stored in the VideoSDK's temporary file storage system will be automatically deleted once the current room/meeting comes to an end.

🔚 Conclusion

Integrating image capture into a JavaScript video call app significantly enhances its functionality and user experience. This feature not only adds a visual dimension to communication but also provides users with a valuable tool for capturing and preserving important moments, information, or visuals shared during the call.

VideoSDK's capabilities ensure seamless integration of image capture functionality with VideoSDK, providing a smooth user experience across different platforms and devices. By incorporating this feature, the JavaScript video call app becomes more versatile, empowering users with enhanced communication tools and fostering a more interactive and engaging environment.

Ready to get started? Sign up today and take advantage of 10,000 free minutes to experiment with image capture and explore the other powerful features VideoSDK offers.

How to Implement Active Speaker Indication in React JS Video Call App?

Kishan Nakrani — Wed, 01 May 2024 07:04:03 +0000

📌 Introduction

In your React JS video call application, implementing Active Speaker Indication can enhance the user experience by highlighting the participant currently speaking. This feature helps users easily identify who is talking, fostering smoother communication and engagement within the call. By dynamically updating the interface to visually indicate the active speaker, such as through border highlights or avatar animations, participants can focus their attention more effectively during group discussions.

Benefits of Implement Active Speaker Indication in React JS Video App:

Enhanced Communication : Active Speaker Indication improves communication by helping participants easily identify who is speaking, reducing confusion and interruptions.
Improved Engagement : Users stay more engaged in the conversation when they can easily follow who is speaking, leading to more active participation.
Smooth User Experience : It provides a smoother user experience by making it effortless for participants to understand the flow of conversation.

Use case of Implement Active Speaker Indication in React JS Video App:

Group Meetings : During group meetings, Active Speaker Indication helps participants follow the conversation flow and know who is speaking at any given moment.
Remote Work : In remote work scenarios, where team members communicate through video calls, this feature ensures smoother collaboration and better engagement.
Educational Webinars : In online educational webinars or virtual classrooms, it help students identify the speaker and stay focused on the lesson.

In the below guide, you will learn how to highlight active speakers in the React JS Video chat application.

🚀 Getting Started with VideoSDK

To take advantage of the active speaker functionality, we must use the capabilities that the VideoSDK offers. Before diving into the implementation steps, let's ensure you complete the necessary prerequisites.

Create a VideoSDK Account

Go to your VideoSDK dashboard and sign up if you don't have an account. This account gives you access to the required Video SDK token, which acts as an authentication key that allows your application to interact with VideoSDK functionality.

Generate your Auth Token

Visit your VideoSDK dashboard and navigate to the "API Key" section to generate your auth token. This token is crucial in authorizing your application to use VideoSDK features.

For a more visual understanding of the account creation and token generation process, consider referring to the provided tutorial.

Prerequisites and Setup

Before proceeding, ensure that your development environment meets the following requirements:

VideoSDK Developer Account (Not having one? Follow VideoSDK Dashboard)
Basic understanding of React.
React VideoSDK
Make sure Node and NPM are installed on your device.
Basic understanding of Hooks (useState, useRef, useEffect).
React Context API (optional).

Follow the steps to create the environment necessary to add video calls to your app. You can also find the code sample for Quickstart here.

Create a new React App using the below command.

$ npx create-react-app videosdk-rtc-react-app

🛠️ Install VideoSDK

It is necessary to set up VideoSDK within your project before going into the details of integrating the Screen Share feature. Installing VideoSDK using NPM or Yarn will depend on the needs of your project.

For NPM

$ npm install "@videosdk.live/react-sdk"

//For the Participants Video
$ npm install "react-player"

For Yarn

$ yarn add "@videosdk.live/react-sdk"

//For the Participants Video
$ yarn add "react-player"

You are going to use functional components to leverage React's reusable component architecture. There will be components for users, videos and controls (mic, camera, leave) over the video.

App Architecture

The App will contain a MeetingView component which includes a ParticipantView component which will render the participant's name, video, audio, etc. It will also have a Controls component that will allow the user to perform operations like leave and toggle media.

You will be working on the following files:

API.js: Responsible for handling API calls such as generating unique meetingId tokens.
App.js: Responsible for rendering MeetingView and joining the meeting.

🎥 Essential Steps to Implement Video Calling Functionality

To add video capability to your React application, you must first complete a sequence of prerequisites.

Step 1: Get started with API.js

Before moving on, you must create an API request to generate a unique meetingId. You will need an authentication token, which you can create either through the videosdk-rtc-api-server-examples or directly from the VideoSDK Dashboard for developers.

//This is the Auth token, you will use it to generate a meeting and connect to it
export const authToken = "<Generated-from-dashbaord>";
// API call to create a meeting
export const createMeeting = async ({ token }) => {
  const res = await fetch(`https://api.videosdk.live/v2/rooms`, {
    method: "POST",
    headers: {
      authorization: `${authToken}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({}),
  });
  //Destructuring the roomId from the response
  const { roomId } = await res.json();
  return roomId;
};

Step 2: Wireframe App.js with all the components

To build up a wireframe of App.js, you need to use VideoSDK Hooks and Context Providers. VideoSDK provides MeetingProvider, MeetingConsumer, useMeeting, and useParticipant hooks.

First, you need to understand the Context Provider and Consumer. Context is primarily used when some data needs to be accessible by many components at different nesting levels.

MeetingProvider : This is the Context Provider. It accepts value config and token as a prop. The Provider component accepts a value prop to be passed to consuming components that are descendants of this Provider. One Provider can be connected to many consumers. Providers can be nested to override values deeper within the tree.
MeetingConsumer : This is the Context Consumer. All consumers that are descendants of a Provider will re-render whenever the Provider’s value proposition changes.
useMeeting : This is the meeting hook API. It includes all the information related to meetings, such as join/leave, enable/disable the mic or webcam, etc.
useParticipant : This is the participant hook API. It is responsible for handling all the events and props related to one particular participant, such as name , webcamStream , micStream , etc.

The Meeting Context provides a way to listen for any changes that occur when a participant joins the meeting or makes modifications to their microphone, camera, and other settings.

Begin by making a few changes to the code in the App.js file.
App.js

import "./App.css";
import React, { useEffect, useMemo, useRef, useState } from "react";
import {
  MeetingProvider,
  MeetingConsumer,
  useMeeting,
  useParticipant,
} from "@videosdk.live/react-sdk";
import { authToken, createMeeting } from "./API";
import ReactPlayer from "react-player";

function JoinScreen({ getMeetingAndToken }) {
  return null;
}

function ParticipantView(props) {
  return null;
}

function Controls(props) {
  return null;
}

function MeetingView(props) {
  return null;
}

function ChatView() {
  return null;
}

function App() {
  const [meetingId, setMeetingId] = useState(null);

  //Getting the meeting id by calling the api we just wrote
  const getMeetingAndToken = async (id) => {
    const meetingId =
      id == null ? await createMeeting({ token: authToken }) : id;
    setMeetingId(meetingId);
  };

  //This will set Meeting Id to null when meeting is left or ended
  const onMeetingLeave = () => {
    setMeetingId(null);
  };

  return authToken && meetingId ? (
    <MeetingProvider
      config={{
        meetingId,
        micEnabled: true,
        webcamEnabled: true,
        name: "C.V. Raman",
      }}
      token={authToken}
    >
      <MeetingView meetingId={meetingId} onMeetingLeave={onMeetingLeave} />
    </MeetingProvider>
  ) : (
    <JoinScreen getMeetingAndToken={getMeetingAndToken} />
  );
}

export default App;

Step 3: Implement Join Screen

The join screen will serve as a medium to either schedule a new meeting or join an existing one.

function JoinScreen({ getMeetingAndToken }) {
  const [meetingId, setMeetingId] = useState(null);
  const onClick = async () => {
    await getMeetingAndToken(meetingId);
  };
  return (
    <div>
      <input
        type="text"
        placeholder="Enter Meeting Id"
        onChange={(e) => {
          setMeetingId(e.target.value);
        }}
      />
      <button onClick={onClick}>Join</button>
      {" or "}
      <button onClick={onClick}>Create Meeting</button>
    </div>
  );
}

Output

Step 4: Implement MeetingView and Controls

The next step is to create MeetingView and Controls components to manage features such as join, leave, mute, and unmute.

function MeetingView(props) {
  const [joined, setJoined] = useState(null);
  //Get the method which will be used to join the meeting.
  //We will also get the participants list to display all participants
  const { join, participants } = useMeeting({
    //callback for when meeting is joined successfully
    onMeetingJoined: () => {
      setJoined("JOINED");
    },
    //callback for when meeting is left
    onMeetingLeft: () => {
      props.onMeetingLeave();
    },
  });
  const joinMeeting = () => {
    setJoined("JOINING");
    join();
  };

  return (
    <div className="container">
      <h3>Meeting Id: {props.meetingId}</h3>
      {joined && joined == "JOINED" ? (
        <div>
          <Controls />
          //For rendering all the participants in the meeting
          {[...participants.keys()].map((participantId) => (
            <ParticipantView
              participantId={participantId}
              key={participantId}
            />
          ))}
          <ChatView />
        </div>
      ) : joined && joined == "JOINING" ? (
        <p>Joining the meeting...</p>
      ) : (
        <button onClick={joinMeeting}>Join</button>
      )}
    </div>
  );
}

function Controls() {
  const { leave, toggleMic, toggleWebcam } = useMeeting();
  return (
    <div>
      <button onClick={() => leave()}>Leave</button>
      <button onClick={() => toggleMic()}>toggleMic</button>
      <button onClick={() => toggleWebcam()}>toggleWebcam</button>
    </div>
  );
}

Control Component

Output of Controls Component

Step 5: Implement Participant View

Before implementing the participant view, you need to understand a couple of concepts.

5.1 Forwarding Ref for mic and camera

The useRef hook is responsible for referencing the audio and video components. It will be used to play and stop the audio and video of the participant.

const webcamRef = useRef(null);
const micRef = useRef(null);

Forwarding Ref for mic and camera

5.2 useParticipant Hook

The useParticipant hook is responsible for handling all the properties and events of one particular participant who joined the meeting. It will take participantId as an argument.

const { webcamStream, micStream, webcamOn, micOn } = useParticipant(
  props.participantId
);

5.3 MediaStream API

The MediaStream API is beneficial for adding a MediaTrack to the audio/video tag, enabling the playback of audio or video.

const webcamRef = useRef(null);
const mediaStream = new MediaStream();
mediaStream.addTrack(webcamStream.track);

webcamRef.current.srcObject = mediaStream;
webcamRef.current
  .play()
  .catch((error) => console.error("videoElem.current.play() failed", error));

5.4 Implement `ParticipantView`

Now you can use both of the hooks and the API to create ParticipantView

function ParticipantView(props) {
  const micRef = useRef(null);
  const { webcamStream, micStream, webcamOn, micOn, isLocal, displayName } =
    useParticipant(props.participantId);

  const videoStream = useMemo(() => {
    if (webcamOn && webcamStream) {
      const mediaStream = new MediaStream();
      mediaStream.addTrack(webcamStream.track);
      return mediaStream;
    }
  }, [webcamStream, webcamOn]);

  useEffect(() => {
    if (micRef.current) {
      if (micOn && micStream) {
        const mediaStream = new MediaStream();
        mediaStream.addTrack(micStream.track);

        micRef.current.srcObject = mediaStream;
        micRef.current
          .play()
          .catch((error) =>
            console.error("videoElem.current.play() failed", error)
          );
      } else {
        micRef.current.srcObject = null;
      }
    }
  }, [micStream, micOn]);

  return (
    <div>
      <p>
        Participant: {displayName} | Webcam: {webcamOn ? "ON" : "OFF"} | Mic:{" "}
        {micOn ? "ON" : "OFF"}
      </p>
      <audio ref={micRef} autoPlay playsInline muted={isLocal} />
      {webcamOn && (
        <ReactPlayer
          //
          playsinline // extremely crucial prop
          pip={false}
          light={false}
          controls={false}
          muted={true}
          playing={true}
          //
          url={videoStream}
          //
          height={"300px"}
          width={"300px"}
          onError={(err) => {
            console.log(err, "participant video error");
          }}
        />
      )}
    </div>
  );
}

Integrate Active Speaker Feature

The Active Speaker Indication feature in VideoSDK allows you to identify the participant who is currently the active speaker in a meeting. This feature proves especially valuable in larger meetings or webinars, where numerous participants can make it challenging to identify the active speaker.

Whenever any participant speaks in a meeting, the onSpeakerChanged event will trigger, providing the participant ID of the active speaker.

For example, the meeting is running with Alice and Bob. Whenever any of them speaks, onSpeakerChanged the event will trigger and return the speaker's participantId.

import { useMeeting } from "@videosdk.live/react-sdk";

const MeetingView = () => {
  /** useMeeting hooks events */
  const {
    /** Methods */
  } = useMeeting({
    onSpeakerChanged: (activeSpeakerId) => {
      console.log("Active Speaker participantId", activeSpeakerId);
    },
  });
};

🎯 Conclusion

By implementing active speaker indication, you can significantly enhance the usability and clarity of your React JS video call application. Not only does it improve user experience, but it also fosters a more natural and focused communication flow. With VideoSDK, this feature is straightforward to integrate, allowing you to focus on building a captivating user experience.

Enhance your React video call app today with VideoSDK and offer an unparalleled video calling experience to your users. Remember, VideoSDK provides 10,000 free minutes to empower your video call app with advanced features without breaking any initial investment. Sign up with VideoSDK today and take the video app to the next level.

Integrate Screen Share in JavaScript Video Chat App

Kishan Nakrani — Tue, 30 Apr 2024 09:10:33 +0000

📌 Introduction

Integrating screen sharing into your JavaScript video chat app expands its capabilities, allowing users to share their screens during calls. This feature enhances collaboration by enabling participants to show presentations, documents, or other content directly from their screens. With seamless integration, users can easily switch between video chat and screen-sharing modes, making discussions more interactive and productive.

Benefits of Integrating Screen Share in a JavaScript Video Chat App:

Enhanced Collaboration: Screen sharing facilitates real-time collaboration by allowing users to share their screens, enabling them to demonstrate concepts, share documents, or provide visual instructions during video calls.
Improved Communication: Visual aids provided by screen sharing enhance communication clarity, ensuring that all participants are on the same page, reducing misunderstandings, and improving overall comprehension.
Interactive Learning: In educational settings, screen sharing facilitates interactive learning experiences, enabling instructors to demonstrate concepts, showcase multimedia content, or lead virtual workshops effectively.

In the below-provided guide, you'll be able to implement screen-sharing functionality smoothly into your app using JavaScript with VideoSDK.

🚀 Getting Started with VideoSDK

To take advantage of Screen Share functionality, we must use the capabilities that the VideoSDK offers. Before diving into the implementation steps, let's ensure you complete the necessary prerequisites.

Create a VideoSDK Account

Generate your Auth Token

Prerequisites

Before proceeding, ensure that your development environment meets the following requirements:

VideoSDK Developer Account (if you do not have one, follow VideoSDK Dashboard)
Have Node and NPM installed on your device.

🛠️ Install VideoSDK

Import VideoSDK using the <script> tag or install it using the following npm command. Make sure you are in your app directory before you run this command.

<html>
  <head>
    <!--.....-->
  </head>
  <body>
    <!--.....-->
    <script src="https://sdk.videosdk.live/js-sdk/0.0.85/videosdk.js"></script>
  </body>
</html>

npm

npm install @videosdk.live/js-sdk

Yarn

yarn add @videosdk.live/js-sdk

Structure of the project

Your project structure should look like this.

  root
   ├── index.html
   ├── config.js
   ├── index.js

You will be working on the following files:

index.html: Responsible for creating a basic UI.
config.js: Responsible for storing the token.
index.js: Responsible for rendering the meeting view and the join meeting functionality.

🎥 Essential Steps to Implement Video Call Functionality

Once you've successfully installed VideoSDK in your project, you'll have access to a range of functionalities for building your video call application. Screen Share is one such feature that leverages VideoSDK's capabilities. It leverages VideoSDK's capabilities to identify the user with the strongest audio signal (the one speaking)

Step 1: Design the user interface (UI)

Create an HTML file containing the screens, join-screen and grid-screen.

<!DOCTYPE html>
<html>
  <head> </head>

  <body>
    <div id="join-screen">
      <!-- Create new Meeting Button -->
      <button id="createMeetingBtn">New Meeting</button>
      OR
      <!-- Join existing Meeting -->
      <input type="text" id="meetingIdTxt" placeholder="Enter Meeting id" />
      <button id="joinBtn">Join Meeting</button>
    </div>

    <!-- for Managing meeting status -->
    <div id="textDiv"></div>

    <div id="grid-screen" style="display: none">
      <!-- To Display MeetingId -->
      <h3 id="meetingIdHeading"></h3>

      <!-- Controllers -->
      <button id="leaveBtn">Leave</button>
      <button id="toggleMicBtn">Toggle Mic</button>
      <button id="toggleWebCamBtn">Toggle WebCam</button>

      <!-- render Video -->
      <div class="row" id="videoContainer"></div>
    </div

      <!-- render Screen Share Video -->
      <div class="row" id="screenShareVideoContainer"></div>

    <!-- Add VideoSDK script -->
    <script src="https://sdk.videosdk.live/js-sdk/0.0.85/videosdk.js"></script>
    <script src="config.js"></script>
    <script src="index.js"></script>
  </body>
</html>

Step 2: Implement Join Screen

Configure the token in the config.js file, which you can obtain from the VideoSDK Dashboard.

// Auth token will be used to generate a meeting and connect to it
TOKEN = "Your_Token_Here";

Next, retrieve all the elements from the DOM and declare the following variables in the index.js file. Then, add an event listener to the join and create meeting buttons.

// Getting Elements from DOM
const joinButton = document.getElementById("joinBtn");
const enableScreenShareButton = document.getElementById("enableScreenShareBtn");
const disableScreenShareButton = document.getElementById(
  "disableScreenShareBtn"
);
const leaveButton = document.getElementById("leaveBtn");
const toggleMicButton = document.getElementById("toggleMicBtn");
const toggleWebCamButton = document.getElementById("toggleWebCamBtn");
const createButton = document.getElementById("createMeetingBtn");
const videoContainer = document.getElementById("videoContainer");
const screenShareVideoContainer = document.getElementById(
  "screenShareVideoContainer"
);
const textDiv = document.getElementById("textDiv");

// Declare Variables
let meeting = null;
let meetingId = "";
let isMicOn = false;
let isWebCamOn = false;

function initializeMeeting() {}

function createLocalParticipant() {}

function createVideoElement() {}

function createAudioElement() {}

function setTrack() {}

// Join Meeting Button Event Listener
joinButton.addEventListener("click", async () => {
  document.getElementById("join-screen").style.display = "none";
  textDiv.textContent = "Joining the meeting...";

  roomId = document.getElementById("meetingIdTxt").value;
  meetingId = roomId;

  initializeMeeting();
});

// Create Meeting Button Event Listener
createButton.addEventListener("click", async () => {
  document.getElementById("join-screen").style.display = "none";
  textDiv.textContent = "Please wait, we are joining the meeting";

  // API call to create meeting
  const url = `https://api.videosdk.live/v2/rooms`;
  const options = {
    method: "POST",
    headers: { Authorization: TOKEN, "Content-Type": "application/json" },
  };

  const { roomId } = await fetch(url, options)
    .then((response) => response.json())
    .catch((error) => alert("error", error));
  meetingId = roomId;

  initializeMeeting();
});

Step 3: Initialize Meeting

Following that, initialize the meeting using the initMeeting() function and proceed to join the meeting.

// Initialize meeting
function initializeMeeting() {
  window.VideoSDK.config(TOKEN);

  meeting = window.VideoSDK.initMeeting({
    meetingId: meetingId, // required
    name: "Thomas Edison", // required
    micEnabled: true, // optional, default: true
    webcamEnabled: true, // optional, default: true
  });

  meeting.join();

  // Creating local participant
  createLocalParticipant();

  // Setting local participant stream
  meeting.localParticipant.on("stream-enabled", (stream) => {
    setTrack(stream, null, meeting.localParticipant, true);
  });

  // meeting joined event
  meeting.on("meeting-joined", () => {
    textDiv.style.display = "none";
    document.getElementById("grid-screen").style.display = "block";
    document.getElementById(
      "meetingIdHeading"
    ).textContent = `Meeting Id: ${meetingId}`;
  });

  // meeting left event
  meeting.on("meeting-left", () => {
    videoContainer.innerHTML = "";
  });

  // Remote participants Event
  // participant joined
  meeting.on("participant-joined", (participant) => {
    // ...
  });

  // participant left
  meeting.on("participant-left", (participant) => {
    // ...
  });
}

Step 4: Create the Media Elements

// creating video element
function createVideoElement(pId, name) {
  let videoFrame = document.createElement("div");
  videoFrame.setAttribute("id", `f-${pId}`);
  videoFrame.style.width = "300px";


  //create video
  let videoElement = document.createElement("video");
  videoElement.classList.add("video-frame");
  videoElement.setAttribute("id", `v-${pId}`);
  videoElement.setAttribute("playsinline", true);
  videoElement.setAttribute("width", "300");
  videoFrame.appendChild(videoElement);

  let displayName = document.createElement("div");
  displayName.innerHTML = `Name : ${name}`;

  videoFrame.appendChild(displayName);
  return videoFrame;
}

// creating audio element
function createAudioElement(pId) {
  let audioElement = document.createElement("audio");
  audioElement.setAttribute("autoPlay", "false");
  audioElement.setAttribute("playsInline", "true");
  audioElement.setAttribute("controls", "false");
  audioElement.setAttribute("id", `a-${pId}`);
  audioElement.style.display = "none";
  return audioElement;
}

// creating local participant
function createLocalParticipant() {
  let localParticipant = createVideoElement(
    meeting.localParticipant.id,
    meeting.localParticipant.displayName
  );
  videoContainer.appendChild(localParticipant);
}

// setting media track
function setTrack(stream, audioElement, participant, isLocal) {
  if (stream.kind == "video") {
    isWebCamOn = true;
    const mediaStream = new MediaStream();
    mediaStream.addTrack(stream.track);
    let videoElm = document.getElementById(`v-${participant.id}`);
    videoElm.srcObject = mediaStream;
    videoElm
      .play()
      .catch((error) =>
        console.error("videoElem.current.play() failed", error)
      );
  }
  if (stream.kind == "audio") {
    if (isLocal) {
      isMicOn = true;
    } else {
      const mediaStream = new MediaStream();
      mediaStream.addTrack(stream.track);
      audioElement.srcObject = mediaStream;
      audioElement
        .play()
        .catch((error) => console.error("audioElem.play() failed", error));
    }
  }
}

Step 5: Handle participant events

Thereafter, implement the events related to the participants and the stream.

The following are the events to be executed in this step:

participant-joined: When a remote participant joins, this event will trigger. In the event callback, create video and audio elements previously defined for rendering their video and audio streams.
participant-left: When a remote participant leaves, this event will trigger. In the event callback, remove the corresponding video and audio elements.
stream-enabled: This event manages the media track of a specific participant by associating it with the appropriate video or audio element.

// Initialize meeting
function initializeMeeting() {
  // ...

  // participant joined
  meeting.on("participant-joined", (participant) => {
    let videoElement = createVideoElement(
      participant.id,
      participant.displayName
    );
    let audioElement = createAudioElement(participant.id);
    // stream-enabled
    participant.on("stream-enabled", (stream) => {
      setTrack(stream, audioElement, participant, false);
    });
    videoContainer.appendChild(videoElement);
    videoContainer.appendChild(audioElement);
  });

  // participants left
  meeting.on("participant-left", (participant) => {
    let vElement = document.getElementById(`f-${participant.id}`);
    vElement.remove(vElement);

    let aElement = document.getElementById(`a-${participant.id}`);
    aElement.remove(aElement);
  });
}

Step 6: Implement Controls

Next, implement the meeting controls such as toggleMic, toggleWebcam, and leave the meeting.

// leave Meeting Button Event Listener
leaveButton.addEventListener("click", async () => {
  meeting?.leave();
  document.getElementById("grid-screen").style.display = "none";
  document.getElementById("join-screen").style.display = "block";
});

// Toggle Mic Button Event Listener
toggleMicButton.addEventListener("click", async () => {
  if (isMicOn) {
    // Disable Mic in Meeting
    meeting?.muteMic();
  } else {
    // Enable Mic in Meeting
    meeting?.unmuteMic();
  }
  isMicOn = !isMicOn;
});

// Toggle Web Cam Button Event Listener
toggleWebCamButton.addEventListener("click", async () => {
  if (isWebCamOn) {
    // Disable Webcam in Meeting
    meeting?.disableWebcam();

    let vElement = document.getElementById(`f-${meeting.localParticipant.id}`);
    vElement.style.display = "none";
  } else {
    // Enable Webcam in Meeting
    meeting?.enableWebcam();

    let vElement = document.getElementById(`f-${meeting.localParticipant.id}`);
    vElement.style.display = "inline";
  }
  isWebCamOn = !isWebCamOn;
});

You can check out the complete here.

Integrate Screen Share Feature

Screen sharing in a meeting is the process of sharing your computer screen with other participants in the meeting. It allows everyone in the meeting to see exactly what you are seeing on your screen, which can be helpful for presentations, demonstrations, or collaborations.

Enable Screen Share

By using the enableScreenShare() function of the meeting object, the local participant can share their desktop screen to other participants.

You can also pass a customised screenshare track in enableScreenShare() by using Custom Screen Share Track.
The Screen Share stream of a participant can be accessed from the streams property of the Participant object.

Disable Screen Share

By using disableScreenShare() function of the meeting object, the local participant can stop sharing their desktop screen to other participants.

NOTE: Screen Sharing is only supported in the Desktop browsers and not in mobile/tab browser.

let meeting;

// Initialize Meeting
meeting = VideoSDK.initMeeting({
  // ...
});

enableScreenShareButton.addEventListener("click", () => {
  // Enabling ScreenShare
  meeting?.enableScreenShare();
});

disableScreenShareButton.addEventListener("click", () => {
  // Disabling ScreenShare
  meeting?.disableScreenShare();
});

Events associated with Screen Share

enableScreenShare

Every Participant will receive a callback on stream-enabled event of the participant object with Stream object.
Every Participant will receive a callback on presenter-changed event of the meeting object with presenterId.  #### disableScreenShare
Every Participant will receive a callback on stream-disabled event of the participant object with the Stream object.
Every Participant will receive a callback on presenter-changed event of the meeting object with null value indicating there is no current presenter.

let meeting;

// Initialize Meeting
meeting = VideoSDK.initMeeting({
  // ...
});

  // participant joined
  meeting.on("participant-joined", (participant) => {
    // ...
    // ...
    participant.on("stream-enabled", (stream) => {
      setTrack(stream, audioElement, participant, false);

      if (stream.kind == "share") {
        //particiapnt turned on screen share
        //Render screenshare logic here
      }
    });
    // ...
    // ...

   participant.on("stream-disabled", (stream) => {
      if (stream.kind === "share") {
        //particiapnt turned off screenshare
        //remove screenshare logic here
      }
    });
  });

meeting.on("presenter-changed", (presenterId) => {
  if (presenterId) {
    //someone start presenting
  } else {
    //someone stopped presenting
  }
});

Screen Share with Audio

To enable screen sharing with audio, select the Share tab audio option when sharing the chrome tab, as shown below.

After clicking the Share button, you will receive the selected tab's audio stream in the participant's screenShareAudioStream.

NOTE:

Screen Share with Audio is only supported while sharing Chrome Tab in a Chromium-based browser like Google Chrome, Brave etc.

Rendering Screen Share and Screen Share Audio

To display the screenshare video stream, you will receive it in the participant's stream-enabled callback with the stream kind set as "share".

participant.on("stream-enabled", (stream) => {
  if (stream.kind == "share") {
  const videoElem = createShareVideoElement(participant.id, stream);

  //add videoElem to your container
  screenShareVideoContainer.appendChild(videoElem);
  }

  if (stream.kind == "shareAudio") {
  }
});

// creating video element
function createShareVideoElement(pId, stream) {
  if (pId == meeting.localParticipant.id) return;

  let videoElement = document.createElement("video");
  videoElement.setAttribute("autoPlay", false);
  videoElement.setAttribute("controls", "false");
  videoElement.setAttribute("id", `v-share-${pId}`);

  const mediaStream = new MediaStream();
  mediaStream.addTrack(stream.track);
  videoElement.srcObject = mediaStream;
  videoElement
    .play()
    .catch((error) => console.error("audioElem.play() failed", error));
  return videoElement;
}

// creating audio element
function createShareAudioElement(pId, stream) {}

Now to render the screenshare audio stream, you will receive it in the participant's stream-enabled callback with the stream kind set as "shareAudio".

participant.on("stream-enabled", (stream) => {
  if (stream.kind == "share") {
  }
  if (stream.kind == "shareAudio") {
    const audioElem = createShareAudioElement(participant.id, stream);

    //add audioElem to your container
    screenShareVideoContainer.appendChild(audioElem);
  }
});

// creating video element
function createShareVideoElement(pId, stream) {}

// creating audio element
function createShareAudioElement(pId, stream) {
  if (pId == meeting.localParticipant.id) return;

  let audioElement = document.createElement("audio");
  audioElement.setAttribute("autoPlay", false);
  audioElement.setAttribute("playsInline", "false");
  audioElement.setAttribute("controls", "false");
  audioElement.setAttribute("id", `a-share-${pId}`);
  audioElement.style.display = "none";

  const mediaStream = new MediaStream();
  mediaStream.addTrack(stream.track);
  audioElement.srcObject = mediaStream;
  audioElement
    .play()
    .catch((error) => console.error("audioElem.play() failed", error));
  return audioElement;
}

🔚 Conclusion

Integrating screen sharing into your JavaScript video chat app is a powerful way to enhance communication and collaboration. By enabling users to share their screens during calls, you empower them to demonstrate ideas, share presentations, and collaborate on documents more effectively. With a seamless integration process provided by VideoSDK, implementing screen sharing becomes straightforward, ensuring a smooth user experience. Whether for remote work, online education, or virtual meetings, this feature adds significant value to your app, making it more versatile and engaging for users.

Unlock the full potential of VideoSDK and create seamless video experiences today! Sign up now and get 10,000 free minutes to elevate your video app to the next level.

How to Integrate Active Speaker Indication in iOS Video Call App

Kishan Nakrani — Tue, 30 Apr 2024 07:34:21 +0000

📌 Introduction

Integrating Active Speaker Indication in your iOS Video Call App enhances user experience by highlighting the current speaker, making conversations more engaging and natural. With this feature, users can easily identify who is speaking, leading to smoother communication and better interaction.

Benefits of Integrate Active Speaker Indication in iOS Video Call App:

Enhanced Communication: Active Speaker Indication allows participants to easily identify who is speaking during video calls, facilitating smoother communication and reducing interruptions.
Improved Engagement: By highlighting the current speaker, this feature keeps participants more engaged in the conversation, leading to better interaction and collaboration.
Reduced Confusion: Active Speaker Indication reduces confusion by providing visual cues, eliminating the need for participants to guess who is talking and preventing talking over each other.

Use Case of Integrate Active Speaker Indication in iOS Video Call App:

Enhanced Collaboration: During brainstorming sessions, when one team member starts speaking, their video feed becomes highlighted, allowing others to know who is presenting their ideas.
Reduced Miscommunication: In discussions involving multiple participants, Active Speaker Indication helps avoid confusion by clearly indicating the current speaker, preventing overlapping conversations.
Professional Meetings: Whether it's client meetings or internal discussions, Active Speaker Indication adds a level of professionalism to the video calls, making the interactions more effective and impactful.

By following the guide provided by VideoSDK, you'll seamlessly implement Active Speaker Indication into your iOS app, ensuring that users can see who is actively speaking during video calls. This not only improves the overall usability of your app but also adds a professional touch to your video calling experience. With clear visual cues, participants can focus more on the conversation rather than guessing who is talking.

🚀 Getting Started with VideoSDK

VideoSDK enables the opportunity to integrate video & audio calling into Web, Android, and iOS applications with so many different frameworks. It is the best infrastructure solution that provides programmable SDKs and REST APIs to build scalable video conferencing applications. This guide will get you running with the VideoSDK video & audio calling in minutes.

Create a VideoSDK Account

Generate your Auth Token

Prerequisites and Setup

iOS 11.0+
Xcode 12.0+
Swift 5.0+

This App will contain two screens:

Join Screen : This screen allows the user to either create a meeting or join the predefined meeting.

Meeting Screen : This screen basically contains local and remote participant views and some meeting controls such as Enable/Disable the mic & Camera and Leave meeting.

🛠️ Integrate VideoSDK

To install VideoSDK, you must initialize the pod on the project by running the following command:

pod init

It will create the podfile in your project folder, Open that file and add the dependency for the VideoSDK, like below:

pod 'VideoSDKRTC', :git => 'https://github.com/videosdk-live/videosdk-rtc-ios-sdk.git'

then run the below code to install the pod:

pod install

then declare the permissions in Info.plist :

<key>NSCameraUsageDescription</key>
<string>Camera permission description</string>
<key>NSMicrophoneUsageDescription</key>
<string>Microphone permission description</string>

Project Structure

iOSQuickStartDemo
   ├── Models
        ├── RoomStruct.swift
        └── MeetingData.swift
   ├── ViewControllers
        ├── StartMeetingViewController.swift
        └── MeetingViewController.swift
   ├── AppDelegate.swift // Default
   ├── SceneDelegate.swift // Default
   └── APIService
           └── APIService.swift
   ├── Main.storyboard // Default
   ├── LaunchScreen.storyboard // Default
   └── Info.plist // Default
 Pods
     └── Podfile

Create models

Create a swift file for MeetingData and RoomStruct class model for setting data in an object pattern.

import Foundation
struct MeetingData {
    let token: String
    let name: String
    let meetingId: String
    let micEnabled: Bool
    let cameraEnabled: Bool
}

MeetingData.swift

import Foundation
struct RoomsStruct: Codable {
    let createdAt, updatedAt, roomID: String?
    let links: Links?
    let id: String?
    enum CodingKeys: String, CodingKey {
        case createdAt, updatedAt
        case roomID = "roomId"
        case links, id
    }
}

// MARK: - Links
struct Links: Codable {
    let getRoom, getSession: String?
    enum CodingKeys: String, CodingKey {
        case getRoom = "get_room"
        case getSession = "get_session"
    }
}

RoomStruct.swift

🎥 Essential Steps for Building the Video Calling

This guide is designed to walk you through the process of integrating Active Speaker Indication with VideoSDK. We'll cover everything from setting up the SDK to incorporating the visual cues into your app's interface, ensuring a smooth and efficient implementation process.

Step 1: Get started with APIClient

Before jumping to anything else, we have to write an API to generate a unique meetingId. You will require an authentication token; you can generate it either using videosdk-server-api-example or from the VideoSDK Dashboard for developers.

import Foundation

let TOKEN_STRING: String = "<AUTH_TOKEN>"

class APIService {

  class func createMeeting(token: String, completion: @escaping (Result<String, Error>) -> Void) {

    let url = URL(string: "https://api.videosdk.live/v2/rooms")!

    var request = URLRequest(url: url)
    request.httpMethod = "POST"
    request.addValue(TOKEN_STRING, forHTTPHeaderField: "authorization")

    URLSession.shared.dataTask(
      with: request,
      completionHandler: { (data: Data?, response: URLResponse?, error: Error?) in

        DispatchQueue.main.async {

          if let data = data, let utf8Text = String(data: data, encoding: .utf8) {
            do {
              let dataArray = try JSONDecoder().decode(RoomsStruct.self, from: data)

              completion(.success(dataArray.roomID ?? ""))
            } catch {
              print("Error while creating a meeting: \(error)")
              completion(.failure(error))
            }
          }
        }
      }
    ).resume()
  }
}

APIService.swift

Step 2: Implement Join Screen

The Join Screen will work as a medium to either schedule a new meeting or join an existing meeting.

import Foundation
import UIKit

class StartMeetingViewController: UIViewController, UITextFieldDelegate {

  private var serverToken = ""

  /// MARK: outlet for create meeting button
  @IBOutlet weak var btnCreateMeeting: UIButton!

  /// MARK: outlet for join meeting button
  @IBOutlet weak var btnJoinMeeting: UIButton!

  /// MARK: outlet for meetingId textfield
  @IBOutlet weak var txtMeetingId: UITextField!

  /// MARK: Initialize the private variable with TOKEN_STRING &
  /// setting the meeting id in the textfield
  override func viewDidLoad() {
    txtMeetingId.delegate = self
    serverToken = TOKEN_STRING
    txtMeetingId.text = "PROVIDE-STATIC-MEETING-ID"
  }

  /// MARK: method for joining meeting through seague named as "StartMeeting"
  /// after validating the serverToken in not empty
  func joinMeeting() {

    txtMeetingId.resignFirstResponder()

    if !serverToken.isEmpty {
      DispatchQueue.main.async {
        self.dismiss(animated: true) {
          self.performSegue(withIdentifier: "StartMeeting", sender: nil)
        }
      }
    } else {
      print("Please provide auth token to start the meeting.")
    }
  }

  /// MARK: outlet for create meeting button tap event
  @IBAction func btnCreateMeetingTapped(_ sender: Any) {
    print("show loader while meeting gets connected with server")
    joinRoom()
  }

  /// MARK: outlet for join meeting button tap event
  @IBAction func btnJoinMeetingTapped(_ sender: Any) {
    if (txtMeetingId.text ?? "").isEmpty {

      print("Please provide meeting id to start the meeting.")
      txtMeetingId.resignFirstResponder()
    } else {
      joinMeeting()
    }
  }

  // MARK: - method for creating room api call and getting meetingId for joining meeting

  func joinRoom() {

    APIService.createMeeting(token: self.serverToken) { result in
      if case .success(let meetingId) = result {
        DispatchQueue.main.async {
          self.txtMeetingId.text = meetingId
          self.joinMeeting()
        }
      }
    }
  }

  /// MARK: preparing to animate to meetingViewController screen
  override func prepare(for segue: UIStoryboardSegue, sender: Any?) {

    guard let navigation = segue.destination as? UINavigationController,

      let meetingViewController = navigation.topViewController as? MeetingViewController
    else {
      return
    }

    meetingViewController.meetingData = MeetingData(
      token: serverToken,
      name: txtMeetingId.text ?? "Guest",
      meetingId: txtMeetingId.text ?? "",
      micEnabled: true,
      cameraEnabled: true
    )
  }
}

StartMeetingViewController.swift

Step 3: Initialize and Join Meeting

Using the provided token and meetingId, we will configure and initialize the meeting in viewDidLoad().

Then, we'll add @IBOutlet for localParticipantVideoView and remoteParticipantVideoView, which can render local and remote participant media respectively.

class MeetingViewController: UIViewController {

import UIKit
import VideoSDKRTC
import WebRTC
import AVFoundation

class MeetingViewController: UIViewController {

// MARK: - Properties
// outlet for local participant container view
   @IBOutlet weak var localParticipantViewContainer: UIView!

// outlet for label for meeting Id
   @IBOutlet weak var lblMeetingId: UILabel!

// outlet for local participant video view
   @IBOutlet weak var localParticipantVideoView: RTCMTLVideoView!

// outlet for remote participant video view
   @IBOutlet weak var remoteParticipantVideoView: RTCMTLVideoView!

// outlet for remote participant no media label
   @IBOutlet weak var lblRemoteParticipantNoMedia: UILabel!

// outlet for remote participant container view
   @IBOutlet weak var remoteParticipantViewContainer: UIView!

// outlet for local participant no media label
   @IBOutlet weak var lblLocalParticipantNoMedia: UILabel!

// Meeting data - required to start
   var meetingData: MeetingData!

// current meeting reference
   private var meeting: Meeting?

    // MARK: - video participants including self to show in UI
    private var participants: [Participant] = []

        // MARK: - Lifecycle Events

        override func viewDidLoad() {
        super.viewDidLoad()
        // configure the VideoSDK with token
        VideoSDK.config(token: meetingData.token)

        // init meeting
        initializeMeeting()

        // set meeting id in button text
        lblMeetingId.text = "Meeting Id: \(meetingData.meetingId)"
      }

      override func viewWillAppear(_ animated: Bool) {
          super.viewWillAppear(animated)
          navigationController?.navigationBar.isHidden = true
      }

    override func viewWillDisappear(_ animated: Bool) {
        super.viewWillDisappear(animated)
        navigationController?.navigationBar.isHidden = false
        NotificationCenter.default.removeObserver(self)
    }

        // MARK: - Meeting

        private func initializeMeeting() {

            // Initialize the VideoSDK
            meeting = VideoSDK.initMeeting(
                meetingId: meetingData.meetingId,
                participantName: meetingData.name,
                micEnabled: meetingData.micEnabled,
                webcamEnabled: meetingData.cameraEnabled
            )

            // Adding the listener to meeting
            meeting?.addEventListener(self)

            // joining the meeting
            meeting?.join()
        }
}

MeetingViewController.swift

Step 4: Implement Controls

After initializing the meeting in the previous step, we will now add @IBOutlet for btnLeave, btnToggleVideo and btnToggleMic which can control the media in the meeting.

class MeetingViewController: UIViewController {

...

    // outlet for leave button
    @IBOutlet weak var btnLeave: UIButton!

    // outlet for toggle video button
    @IBOutlet weak var btnToggleVideo: UIButton!

    // outlet for toggle audio button
    @IBOutlet weak var btnToggleMic: UIButton!

    // bool for mic
    var micEnabled = true
    // bool for video
    var videoEnabled = true

    // outlet for leave button click event
    @IBAction func btnLeaveTapped(_ sender: Any) {
            DispatchQueue.main.async {
                self.meeting?.leave()
                self.dismiss(animated: true)
            }
        }

    // outlet for toggle mic button click event
    @IBAction func btnToggleMicTapped(_ sender: Any) {
        if micEnabled {
            micEnabled = !micEnabled // false
            self.meeting?.muteMic()
        } else {
            micEnabled = !micEnabled // true
            self.meeting?.unmuteMic()
        }
    }

    // outlet for toggle video button click event
    @IBAction func btnToggleVideoTapped(_ sender: Any) {
        if videoEnabled {
            videoEnabled = !videoEnabled // false
            self.meeting?.disableWebcam()
        } else {
            videoEnabled = !videoEnabled // true
            self.meeting?.enableWebcam()
        }
    }

...

}

MeetingViewController.swift

Step 5: Implementing `MeetingEventListener`

In this step, we'll create an extension for the MeetingViewController that implements the MeetingEventListener, which implements the onMeetingJoined, onMeetingLeft, onParticipantJoined, onParticipantLeft, onParticipantChanged, onSpeakerChanged, etc. methods.


class MeetingViewController: UIViewController {

...

extension MeetingViewController: MeetingEventListener {

        /// Meeting started
        func onMeetingJoined() {

            // handle local participant on start
            guard let localParticipant = self.meeting?.localParticipant else { return }
            // add to list
            participants.append(localParticipant)

            // add event listener
            localParticipant.addEventListener(self)

            localParticipant.setQuality(.high)

            if(localParticipant.isLocal){
                self.localParticipantViewContainer.isHidden = false
            } else {
                self.remoteParticipantViewContainer.isHidden = false
            }
        }

        /// Meeting ended
        func onMeetingLeft() {
            // remove listeners
            meeting?.localParticipant.removeEventListener(self)
            meeting?.removeEventListener(self)
        }

        /// A new participant joined
        func onParticipantJoined(_ participant: Participant) {
            participants.append(participant)

            // add listener
            participant.addEventListener(self)

            participant.setQuality(.high)

            if(participant.isLocal){
                self.localParticipantViewContainer.isHidden = false
            } else {
                self.remoteParticipantViewContainer.isHidden = false
            }
        }

        /// A participant left from the meeting
        /// - Parameter participant: participant object
        func onParticipantLeft(_ participant: Participant) {
            participant.removeEventListener(self)
            guard let index = self.participants.firstIndex(where: { $0.id == participant.id }) else {
                return
            }
            // remove participant from list
            participants.remove(at: index)
            // hide from ui
            UIView.animate(withDuration: 0.5){
                if(!participant.isLocal){
                    self.remoteParticipantViewContainer.isHidden = true
                }
            }
        }

        /// Called when speaker is changed
        /// - Parameter participantId: participant id of the speaker, nil when no one is speaking.
        func onSpeakerChanged(participantId: String?) {

            // show indication for active speaker
            if let participant = participants.first(where: { $0.id == participantId }) {
                self.showActiveSpeakerIndicator(participant.isLocal ? localParticipantViewContainer : remoteParticipantViewContainer, true)
            }

            // hide indication for others participants
            let otherParticipants = participants.filter { $0.id != participantId }
            for participant in otherParticipants {
                if participants.count > 1 && participant.isLocal {
                    showActiveSpeakerIndicator(localParticipantViewContainer, false)
                } else {
                    showActiveSpeakerIndicator(remoteParticipantViewContainer, false)
                }
            }
        }

        func showActiveSpeakerIndicator(_ view: UIView, _ show: Bool) {
            view.layer.borderWidth = 4.0
            view.layer.borderColor = show ? UIColor.blue.cgColor : UIColor.clear.cgColor
        }

}

...

MeetingViewController.swift

Step 6: Implementing `ParticipantEventListener`

In this stage, we'll add an extension for the MeetingViewController that implements the ParticipantEventListener, which implements the onStreamEnabled and onStreamDisabled methods for the audio and video of MediaStreams enabled or disabled.

The function updateUI is frequently used to control or modify the user interface (enable/disable camera & mic) by the MediaStream state.

class MeetingViewController: UIViewController {

...

extension MeetingViewController: ParticipantEventListener {

/// Participant has enabled mic, video or screenshare
/// - Parameters:
/// - stream: enabled stream object
/// - participant: participant object
func onStreamEnabled(_ stream: MediaStream, forParticipant participant: Participant) {
    updateUI(participant: participant, forStream: stream, enabled: true)
 }

/// Participant has disabled mic, video or screenshare
/// - Parameters:
/// - stream: disabled stream object
/// - participant: participant object

func onStreamDisabled(_ stream: MediaStream, 
            forParticipant participant: Participant) {

  updateUI(participant: participant, forStream: stream, enabled: false)
 }

}

private extension MeetingViewController {

 func updateUI(participant: Participant, forStream stream: MediaStream, enabled: Bool) { // true
        switch stream.kind {
        case .state(value: .video):
            if let videotrack = stream.track as? RTCVideoTrack {
                if enabled {
                    DispatchQueue.main.async {
                        UIView.animate(withDuration: 0.5){

                            if(participant.isLocal) {

        self.localParticipantViewContainer.isHidden = false
    self.localParticipantVideoView.isHidden = false       
    self.localParticipantVideoView.videoContentMode = .scaleAspectFill self.localParticipantViewContainer.bringSubviewToFront(self.localParticipantVideoView)                                                           
    videotrack.add(self.localParticipantVideoView)
    self.lblLocalParticipantNoMedia.isHidden = true

} else {
        self.remoteParticipantViewContainer.isHidden = false
            self.remoteParticipantVideoView.isHidden = false
                                self.remoteParticipantVideoView.videoContentMode = .scaleAspectFill
                                self.remoteParticipantViewContainer.bringSubviewToFront(self.remoteParticipantVideoView)
                                                videotrack.add(self.remoteParticipantVideoView)
 self.lblRemoteParticipantNoMedia.isHidden = true
        }
     }
  }
} else {
         UIView.animate(withDuration: 0.5){
                if(participant.isLocal){

                    self.localParticipantViewContainer.isHidden = false
                    self.localParticipantVideoView.isHidden = true
                    self.lblLocalParticipantNoMedia.isHidden = false
                            videotrack.remove(self.localParticipantVideoView)
} else {
                   self.remoteParticipantViewContainer.isHidden = false
                   self.remoteParticipantVideoView.isHidden = true
                   self.lblRemoteParticipantNoMedia.isHidden = false
                            videotrack.remove(self.remoteParticipantVideoView)
      }
    }
  }
}

     case .state(value: .audio):
            if participant.isLocal {

               localParticipantViewContainer.layer.borderWidth = 4.0
               localParticipantViewContainer.layer.borderColor = enabled ? UIColor.clear.cgColor : UIColor.red.cgColor
            } else {
                remoteParticipantViewContainer.layer.borderWidth = 4.0
                remoteParticipantViewContainer.layer.borderColor = enabled ? UIColor.clear.cgColor : UIColor.red.cgColor
            }
        default:
            break
        }
    }
}

...

Known Issue

Please add the following line to the MeetingViewController.swift file's viewDidLoad method If you get your video out of the container view like the below image.

override func viewDidLoad() {

  localParticipantVideoView.frame = CGRect(x: 10, y: 0, 
            width: localParticipantViewContainer.frame.width, 
        height: localParticipantViewContainer.frame.height)

  localParticipantVideoView.bounds = CGRect(x: 10, y: 0, 
        width: localParticipantViewContainer.frame.width, 
            height: localParticipantViewContainer.frame.height)

  localParticipantVideoView.clipsToBounds = true

  remoteParticipantVideoView.frame = CGRect(x: 10, y: 0, 
        width: remoteParticipantViewContainer.frame.width, 
            height: remoteParticipantViewContainer.frame.height)

  remoteParticipantVideoView.bounds = CGRect(x: 10, y: 0, 
        width: remoteParticipantViewContainer.frame.width, 
            height: remoteParticipantViewContainer.frame.height)

    remoteParticipantVideoView.clipsToBounds = true
}

MeetingViewController.swift

TIP:

Stuck anywhere? Check out this example code on GitHub.

Once you've successfully installed the videoSDK into your iOS project, you'll unlock a range of functionality to enhance your video call application. This feature uses VideoSDK's advanced audio processing capabilities to identify the participant with the strongest audio signal in real time. This translates to directing the active speaker during a call, allowing you to provide visual feedback to users.

📸 Integrate Active Speaker Indication

This feature can be especially useful in large meetings or webinars, where there may be many participants and it may be difficult to tell who is speaking. Integrating Active Speaker Indication can significantly improve the user experience in various ways. It facilitates smoother communication by clearly indicating the current speaker, thus reducing confusion, particularly in larger group calls, and creating a more engaging environment for all participants involved.

Whenever any participant speaks in the meeting, the onSpeakerChanged event will be triggered with the participant ID of the active speaker.

For example, Alice and Bob are in a meeting. Whenever one of them speaks, the onSpeakerChanged event will be triggered and the speaker will return the participantId.

extension MeetingViewController: MeetingEventListener {
/// Called when speaker is changed
/// - Parameter participantId: participant id of the speaker, nil when no one is speaking.
func onSpeakerChanged(participantId: String?) {

    // show indicator for active speaker
    if let participant = participants.first(where: { $0.id == participantId }),
        // show indication for active speaker
        // ex. show border color
        // cell.contentView.layer.borderColor = UIColor.blue.cgColor : UIColor.clear.cgColor
    }

    // hide indicator for others participants
    let otherParticipants = participants.filter { $0.id != participantId }
    for participant in otherParticipants {
        // ex. remove border color
        //cell.contentView.layer.borderColor = UIColor.clear.cgColor
    }
}

🔚 Conclusion

Active Speaker Indication is a valuable feature for enhancing the functionality and user experience of your iOS video call app. By providing visual cues to identify the current speaker, this feature improves communication, engagement, and overall efficiency during virtual meetings.

With Active Speaker Indication, users benefit from clearer understanding, reduced confusion, and a more professional video calling experience. Whether it's team collaborations, client meetings, or remote learning sessions, this feature ensures smoother interactions and more productive discussions.

Enhance your iOS video call app today with VideoSDK and offer an unparalleled video calling experience to your users. Remember, VideoSDK provides 10,000 free minutes to empower your video call app with advanced features without breaking any initial investment. Sign up with VideoSDK today and take the video app to the next level.

How to Integrate RTMP Live Stream in React JS Video Call App?

Kishan Nakrani — Tue, 30 Apr 2024 07:09:41 +0000

Introduction

Integrating RTMP into a React JS video call application seamlessly adds cutting-edge technology with a user-friendly interface. Experience real-time communication with high-quality video streaming, as RTMP (Real-Time Messaging Protocol) ensures quick transmission of audiovisual data over the internet. Integrating this protocol into React JS empowers developers to create immersive video call experiences within their applications.

Benefits of Integrate RTMP Live Stream in React JS Video Call App:

Enhanced Multimedia : RTMP integration enriches video calls with high-quality streaming, supporting dynamic content sharing.
Low Latency : Real-time transmission ensures minimal delays, enabling smoother interactions and seamless communication.
Scalability : React JS's flexibility combined with RTMP streaming allows for scalable solutions catering to varying user demands.

Use Cases of Integrate RTMP Live Stream in React JS Video Call App:

Virtual Meetings : Conduct seamless video conferences with RTMP streaming for engaging discussions and presentations.
Live Events : Stream live events, webinars, and workshops to a global audience in real time.
Education : Facilitate interactive online classes and virtual lectures with multimedia-rich content delivery.

In the below guide, you will learn modern ReactJS video-calling applications with live streaming using RTMP and VideoSDK.

🚀 Getting Started with VideoSDK

To utilize the RTMP feature, we need to leverage the capabilities provided by VideoSDK. Before we delve into the implementation steps, let's make sure you have completed the necessary prerequisites.

Create a VideoSDK Account

Generate your Auth Token

Visit your VideoSDK dashboard and navigate to the "API Key" section to generate your auth token. This token is crucial in authorizing your application to use VideoSDK features.

For a more visual understanding of the account creation and token generation process, consider referring to the provided tutorial.

Prerequisites and Setup

Before proceeding, ensure that your development environment meets the following requirements:

VideoSDK Developer Account
Basic understanding of React.
React VideoSDK
Make sure Node and NPM are installed on your device.
Basic understanding of Hooks (useState, useRef, useEffect)
React Context API (optional)

Follow the steps to create the environment necessary to add video calls to your app. You can also find the code sample for Quickstart here.

Create a new React App using the below command.

$ npx create-react-app videosdk-rtc-react-app

🛠️ Install VideoSDK

For NPM

$ npm install "@videosdk.live/react-sdk"

//For the Participants Video
$ npm install "react-player"

For Yarn

$ yarn add "@videosdk.live/react-sdk"

//For the Participants Video
$ yarn add "react-player"

You are going to use functional components to leverage React's reusable component architecture. There will be components for users, videos and controls (mic, camera, leave) over the video.

App Architecture

You will be working on the following files:

API.js: Responsible for handling API calls such as generating unique meetingId and token
App.js: Responsible for rendering MeetingView and joining the meeting.

🎥 Essential Steps to Implement Video Calling Functionality

To add video capability to your React application, you must first complete a sequence of prerequisites.

Step 1: Get started with API.js

Before moving on, you must create an API request to generate a unique meetingId. You will need an authentication token, which you can create either through the videosdk-rtc-api-server-examples or directly from the VideoSDK Dashboard for developers.

//This is the Auth token, you will use it to generate a meeting and connect to it
export const authToken = "<Generated-from-dashbaord>";
// API call to create a meeting
export const createMeeting = async ({ token }) => {
  const res = await fetch(`https://api.videosdk.live/v2/rooms`, {
    method: "POST",
    headers: {
      authorization: `${authToken}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({}),
  });
  //Destructuring the roomId from the response
  const { roomId } = await res.json();
  return roomId;
};

Step 2: Wireframe App.js with all the components

To build up a wireframe of App.js, you need to use VideoSDK Hooks and Context Providers. VideoSDK provides MeetingProvider, MeetingConsumer, useMeeting, and useParticipant hooks.

First, you need to understand the Context Provider and Consumer. Context is primarily used when some data needs to be accessible by many components at different nesting levels.

MeetingProvider : This is the Context Provider. It accepts value config and token as props. The Provider component accepts a value prop to be passed to consuming components that are descendants of this Provider. One Provider can be connected to many consumers. Providers can be nested to override values deeper within the tree.
MeetingConsumer : This is the Context Consumer. All consumers that are descendants of a Provider will re-render whenever the Provider’s value prop changes.
useMeeting : This is the meeting hook API. It includes all the information related to meetings such as join/leave, enable/disable the mic or webcam, etc.
useParticipant : This is the participant hook API. It is responsible for handling all the events and props related to one particular participant such as name , webcamStream , micStream , etc.

The Meeting Context provides a way to listen for any changes that occur when a participant joins the meeting or makes modifications to their microphone, camera, and other settings.

Begin by making a few changes to the code in the App.js file.

import "./App.css";
import React, { useEffect, useMemo, useRef, useState } from "react";
import {
  MeetingProvider,
  MeetingConsumer,
  useMeeting,
  useParticipant,
} from "@videosdk.live/react-sdk";
import { authToken, createMeeting } from "./API";
import ReactPlayer from "react-player";

function JoinScreen({ getMeetingAndToken }) {
  return null;
}

function ParticipantView(props) {
  return null;
}

function Controls(props) {
  return null;
}

function MeetingView(props) {
  return null;
}

function App() {
  const [meetingId, setMeetingId] = useState(null);

  //Getting the meeting id by calling the api we just wrote
  const getMeetingAndToken = async (id) => {
    const meetingId =
      id == null ? await createMeeting({ token: authToken }) : id;
    setMeetingId(meetingId);
  };

  //This will set Meeting Id to null when meeting is left or ended
  const onMeetingLeave = () => {
    setMeetingId(null);
  };

  return authToken && meetingId ? (
    <MeetingProvider
      config={{
        meetingId,
        micEnabled: true,
        webcamEnabled: true,
        name: "C.V. Raman",
      }}
      token={authToken}
    >
      <MeetingView meetingId={meetingId} onMeetingLeave={onMeetingLeave} />
    </MeetingProvider>
  ) : (
    <JoinScreen getMeetingAndToken={getMeetingAndToken} />
  );
}

export default App;

Step 3: Implement Join Screen

The join screen will serve as a medium to either schedule a new meeting or join an existing one.

function JoinScreen({ getMeetingAndToken }) {
  const [meetingId, setMeetingId] = useState(null);
  const onClick = async () => {
    await getMeetingAndToken(meetingId);
  };
  return (
    <div>
      <input
        type="text"
        placeholder="Enter Meeting Id"
        onChange={(e) => {
          setMeetingId(e.target.value);
        }}
      />
      <button onClick={onClick}>Join</button>
      {" or "}
      <button onClick={onClick}>Create Meeting</button>
    </div>
  );
}

Output

Step 4: Implement MeetingView and Controls

The next step is to create MeetingView and Controls components to manage features such as join, leave, mute, and unmute.

function MeetingView(props) {
  const [joined, setJoined] = useState(null);
  //Get the method which will be used to join the meeting.
  //We will also get the participants list to display all participants
  const { join, participants } = useMeeting({
    //callback for when meeting is joined successfully
    onMeetingJoined: () => {
      setJoined("JOINED");
    },
    //callback for when meeting is left
    onMeetingLeft: () => {
      props.onMeetingLeave();
    },
  });
  const joinMeeting = () => {
    setJoined("JOINING");
    join();
  };

  return (
    <div className="container">
      <h3>Meeting Id: {props.meetingId}</h3>
      {joined && joined == "JOINED" ? (
        <div>
          <Controls />
          //For rendering all the participants in the meeting
          {[...participants.keys()].map((participantId) => (
            <ParticipantView
              participantId={participantId}
              key={participantId}
            />
          ))}
        </div>
      ) : joined && joined == "JOINING" ? (
        <p>Joining the meeting...</p>
      ) : (
        <button onClick={joinMeeting}>Join</button>
      )}
    </div>
  );
}

function Controls() {
  const { leave, toggleMic, toggleWebcam } = useMeeting();
  return (
    <div>
      <button onClick={() => leave()}>Leave</button>
      <button onClick={() => toggleMic()}>toggleMic</button>
      <button onClick={() => toggleWebcam()}>toggleWebcam</button>
    </div>
  );
}

Control Component

Output of Controls Component

Step 5: Implement Participant View

Before implementing the participant view, you need to understand a couple of concepts.

5.1 Forwarding Ref for mic and camera

The useRef hook is responsible for referencing the audio and video components. It will be used to play and stop the audio and video of the participant.

const webcamRef = useRef(null);
const micRef = useRef(null);

Forwarding Ref for mic and camera

5.2 useParticipant Hook

The useParticipant hook is responsible for handling all the properties and events of one particular participant who joined the meeting. It will take participantId as an argument.

const { webcamStream, micStream, webcamOn, micOn } = useParticipant(
  props.participantId
);

5.3 MediaStream API

The MediaStream API is beneficial for adding a MediaTrack to the audio/video tag, enabling the playback of audio or video.

const webcamRef = useRef(null);
const mediaStream = new MediaStream();
mediaStream.addTrack(webcamStream.track);

webcamRef.current.srcObject = mediaStream;
webcamRef.current
  .play()
  .catch((error) => console.error("videoElem.current.play() failed", error));

5.4 Implement `ParticipantView`

Now you can use both of the hooks and the API to create ParticipantView

function ParticipantView(props) {
  const micRef = useRef(null);
  const { webcamStream, micStream, webcamOn, micOn, isLocal, displayName } =
    useParticipant(props.participantId);

  const videoStream = useMemo(() => {
    if (webcamOn && webcamStream) {
      const mediaStream = new MediaStream();
      mediaStream.addTrack(webcamStream.track);
      return mediaStream;
    }
  }, [webcamStream, webcamOn]);

  useEffect(() => {
    if (micRef.current) {
      if (micOn && micStream) {
        const mediaStream = new MediaStream();
        mediaStream.addTrack(micStream.track);

        micRef.current.srcObject = mediaStream;
        micRef.current
          .play()
          .catch((error) =>
            console.error("videoElem.current.play() failed", error)
          );
      } else {
        micRef.current.srcObject = null;
      }
    }
  }, [micStream, micOn]);

  return (
    <div>
      <p>
        Participant: {displayName} | Webcam: {webcamOn ? "ON" : "OFF"} | Mic:{" "}
        {micOn ? "ON" : "OFF"}
      </p>
      <audio ref={micRef} autoPlay playsInline muted={isLocal} />
      {webcamOn && (
        <ReactPlayer
          //
          playsinline // extremely crucial prop
          pip={false}
          light={false}
          controls={false}
          muted={true}
          playing={true}
          //
          url={videoStream}
          //
          height={"300px"}
          width={"300px"}
          onError={(err) => {
            console.log(err, "participant video error");
          }}
        />
      )}
    </div>
  );
}

You can check out the complete quick start example here.

Integrate RTMP Stream

RTMP is a widely used protocol for live streaming video content from VideoSDK to platforms like YouTube, Twitch, Facebook, and others.

To initiate live streaming from VideoSDK to platforms supporting RTMP ingestion, you simply need to provide the platform-specific stream key and stream URL. This enables VideoSDK to connect to the platform's RTMP server and transmit the live video stream.

Furthermore, VideoSDK offers flexibility in configuring livestream layouts. You can achieve this by either selecting different prebuilt layouts in the configuration or by providing your custom template for live streaming, catering to your specific layout preferences.

This guide will provide an overview of how to implement starting and stopping RTMP live streaming with VideoSDK.

Start Live Stream

The startLivestream() method, accessible from the useMeeting hook, is used to initiate the RTMP livestream of a meeting. This method accepts the following two parameters:

1. Output: This parameter takes an array of objects containing the RTMP url and streamKey specific to the platform where you want to initiate the livestream.
2. config (optional): This parameter defines the layout configuration for the livestream.

const config = {
  // Layout Configuration
  layout: {
    type: "GRID", // "SPOTLIGHT" | "SIDEBAR", Default : "GRID"
    priority: "SPEAKER", // "PIN", Default : "SPEAKER"
    gridSize: 4, // MAX : 4
  },

  // Theme of livestream layout
  theme: "DARK", // "LIGHT" | "DEFAULT"
};

const outputs = [
  {
    url: "<RTMP_URL>",
    streamKey: "<RTMP_STREAM_KEY>",
  },
];

startLivestream(outputs, config);

Stop Live Stream

The stopLivestream() method, accessible from the useMeeting hook is used to stop the RTMP livestream of a meeting.

Example:

import { useMeeting } from "@videosdk.live/react-sdk";

const MeetingView = () => {
  const { startLivestream, stopLivestream } = useMeeting();

  const handleStartLivestream = () => {
    // Start Livestream
    startLivestream(
      [
        {
          url: "rtmp://a.rtmp.youtube.com/live2",
          streamKey: "key",
        },
      ],
      {
        layout: {
          type: "GRID",
          priority: "SPEAKER",
          gridSize: 4,
        },
        theme: "DARK",
      }
    );
  };

  const handleStopLivestream = () => {
    // Start Livestream
    stopLivestream();
  };

  return (
    <>
      <button onClick={handleStartLivestream}>Start Livestream</button>
      <button onClick={handleStopLivestream}>Stop Livestream</button>
    </>
  );
};

Event associated with Livestream

onLivestreamStateChanged - The onLivestreamStateChanged() event is triggered whenever the state of meeting livestream changes.

import { Constants, useMeeting } from "@videosdk.live/react-sdk";

function onLivestreamStateChanged(data) {
  const { status } = data;

  if (status === Constants.livestreamEvents.LIVESTREAM_STARTING) {
    console.log("Meeting livestream is starting");
  } else if (status === Constants.livestreamEvents.LIVESTREAM_STARTED) {
    console.log("Meeting livestream is started");
  } else if (status === Constants.livestreamEvents.LIVESTREAM_STOPPING) {
    console.log("Meeting livestream is stopping");
  } else if (status === Constants.livestreamEvents.LIVESTREAM_STOPPED) {
    console.log("Meeting livestream is stopped");
  } else {
    //
  }
 }

const {
  meetingId
  ...
} = useMeeting({
  onLivestreamStateChanged,
});

Custom Template

With VideoSDK, you have the option to employ your own custom-designed layout template for livestreaming a meeting. To use a custom template, Follow this guide to create and set up the template. Once the template is configured, you can initiate recording using the REST API, specifying the templateURL parameter.

🔚 Conclusion

Integrating RTMP live streaming into your React JS video call app with the VideoSDK React API helps with real-time communication. You can seamlessly incorporate RTMP functionality, enhancing the app's capabilities and user experience. With RTMP, users can engage in high-quality live video streaming, making the communication process more immersive and dynamic. Elevate your app today and bring your community closer through the power of live streaming with VideoSDK.

Remember, VideoSDK provides 10,000 free minutes to empower your video call app with advanced features without breaking any initial investment. Sign up with VideoSDK today and take the video app to the next level!

Integrate RTMP Live Stream in React Native Video Call App

Kishan Nakrani — Tue, 30 Apr 2024 06:58:22 +0000

📌 Introduction

Integrating RTMP (Real-Time Messaging Protocol) into your React Native video call app enables seamless live streaming functionality. With this feature, users can broadcast their video calls to a wider audience in real time. Implementing RTMP in React Native is achievable through utilizing the appropriate SDKs and APIs. RTMP integration adds versatility and dynamism to your React Native video call app, enriching user engagement and satisfaction.

Benefits of integrating RTMP feature into a React Native video call app:

Real-Time Broadcasting : RTMP enables real-time streaming of video and audio content, ensuring instant communication with minimal delay.
Scalability : RTMP supports scalable streaming, allowing your app to handle a large number of concurrent viewers without compromising performance.
High-Quality Streaming : RTMP offers high-quality video and audio streaming, providing users with a superior viewing experience.

Benefits of integrating RTMP feature into a React Native video call app:

Virtual Events : Host virtual conferences, webinars, and live seminars, allowing participants to join remotely and engage in real time.
Live Tutorials : Conduct live tutorials and workshops, enabling instructors to interact with students in real time, answer questions, and provide feedback.
Remote Collaboration : Facilitate remote team meetings, enabling teams to collaborate effectively regardless of their geographical locations.

By following the provided guide, you can incorporate RTMP live streaming effortlessly, enhancing the capabilities of your app.

How to build a React Native Video Call App with RTMP & VideoSDK

🚀 Getting Started with VideoSDK

To take advantage of the chat functionality, we must use the capabilities that the VideoSDK offers. Before diving into the implementation steps, let's ensure you complete the necessary prerequisites.

Create a VideoSDK Account

Generate your Auth Token

Visit your VideoSDK dashboard and navigate to the "API Key" section to generate your auth token. This token is crucial in authorizing your application to use VideoSDK features.

For a more visual understanding of the account creation and token generation process, consider referring to the provided tutorial.

Prerequisites and Setup

Make sure your development environment meets the following requirements:

Node.js v12+
NPM v6+ (comes installed with newer Node versions)
Android Studio or Xcode installed

🛠️ Install VideoSDK Config.

It is necessary to set up VideoSDK within your project before going into the details of integrating the Image Capture feature. Installing VideoSDK using NPM or Yarn will depend on the needs of your project.

For NPM

npm install "@videosdk.live/react-native-sdk" "@videosdk.live/react-native-incallmanager"

For Yarn

yarn add "@videosdk.live/react-native-sdk" "@videosdk.live/react-native-incallmanager"

Project Configuration

Before integrating the Image Capture functionality, ensure that your project is correctly prepared to handle the integration. This setup consists of a sequence of steps for configuring rights, dependencies, and platform-specific parameters so that VideoSDK can function seamlessly inside your application context.

Android Setup

Add the required permissions to the AndroidManifest.xml file.

<manifest
  xmlns:android="http://schemas.android.com/apk/res/android"
  package="com.cool.app"
>
    <!-- Give all the required permissions to app -->
    <uses-permission android:name="android.permission.INTERNET" />
    <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
    <!-- Needed to communicate with already-paired Bluetooth devices. (Legacy up to Android 11) -->
    <uses-permission
        android:name="android.permission.BLUETOOTH"
        android:maxSdkVersion="30" />
    <uses-permission
        android:name="android.permission.BLUETOOTH_ADMIN"
        android:maxSdkVersion="30" />

    <!-- Needed to communicate with already-paired Bluetooth devices. (Android 12 upwards)-->
    <uses-permission android:name="android.permission.BLUETOOTH_CONNECT" />

    <uses-permission android:name="android.permission.CAMERA" />
    <uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />
    <uses-permission android:name="android.permission.RECORD_AUDIO" />
    <uses-permission android:name="android.permission.SYSTEM_ALERT_WINDOW" />
    <uses-permission android:name="android.permission.FOREGROUND_SERVICE"/>
    <uses-permission android:name="android.permission.WAKE_LOCK" />

    <application>
   <meta-data
      android:name="live.videosdk.rnfgservice.notification_channel_name"
      android:value="Meeting Notification"
     />
    <meta-data
    android:name="live.videosdk.rnfgservice.notification_channel_description"
    android:value="Whenever meeting started notification will appear."
    />
    <meta-data
    android:name="live.videosdk.rnfgservice.notification_color"
    android:resource="@color/red"
    />
    <service android:name="live.videosdk.rnfgservice.ForegroundService" android:foregroundServiceType="mediaProjection"></service>
    <service android:name="live.videosdk.rnfgservice.ForegroundServiceTask"></service>
  </application>
</manifest>

AndroidManifest.xml

Update your colors.xml file for internal dependencies.

<resources>
  <item name="red" type="color">
    #FC0303
  </item>
  <integer-array name="androidcolors">
    <item>@color/red</item>
  </integer-array>
</resources>

android/app/src/main/res/values/colors.xml

Link the necessary VideoSDK dependencies.

  dependencies {
   implementation project(':rnwebrtc')
   implementation project(':rnfgservice')
  }

android/app/build.gradle

include ':rnwebrtc'
project(':rnwebrtc').projectDir = new File(rootProject.projectDir, '../node_modules/@videosdk.live/react-native-webrtc/android')

include ':rnfgservice'
project(':rnfgservice').projectDir = new File(rootProject.projectDir, '../node_modules/@videosdk.live/react-native-foreground-service/android')

android/settings.gradle

import live.videosdk.rnwebrtc.WebRTCModulePackage;
import live.videosdk.rnfgservice.ForegroundServicePackage;

public class MainApplication extends Application implements ReactApplication {
  private static List<ReactPackage> getPackages() {
      @SuppressWarnings("UnnecessaryLocalVariable")
      List<ReactPackage> packages = new PackageList(this).getPackages();
      // Packages that cannot be autolinked yet can be added manually here, for example:

      packages.add(new ForegroundServicePackage());
      packages.add(new WebRTCModulePackage());

      return packages;
  }
}

MainApplication.java

/* This one fixes a weird WebRTC runtime problem on some devices. */
android.enableDexingArtifactTransform.desugaring=false

android/gradle.properties

Include the following line in your proguard-rules.pro file (optional: if you are using Proguard)

-keep class org.webrtc.** { *; }

android/app/proguard-rules.pro

In your build.gradle file, update the minimum OS/SDK version to 23.

buildscript {
  ext {
      minSdkVersion = 23
  }
}

build.gradle

iOS Setup

IMPORTANT: Ensure that you are using CocoaPods version 1.10 or later.

To update CocoaPods, you can reinstall the gem using the following command:

$ sudo gem install cocoapods

2. Manually link react-native-incall-manager (if it is not linked automatically).

Select Your_Xcode_Project/TARGETS/BuildSettings, in Header Search Paths, add "$(SRCROOT)/../node_modules/@videosdk.live/react-native-incall-manager/ios/RNInCallManager"

3. Change the path of react-native-webrtc using the following command:

pod ‘react-native-webrtc’, :path => ‘../node_modules/@videosdk.live/react-native-webrtc’

4. Change the version of your platform.

You need to change the platform field in the Podfile to 12.0 or above because react-native-webrtc doesn't support iOS versions earlier than 12.0. Update the line: platform: ios, ‘12.0’.

5. Install pods.

After updating the version, you need to install the pods by running the following command:

Pod install

6. Add “ libreact-native-webrtc.a ” binary.

Add the " libreact-native-webrtc.a" binary to the "Link Binary With Libraries" section in the target of your main project folder.

7. Declare permissions in Info.plist :

Add the following lines to your info.plist file located at (project folder/ios/projectname/info.plist):

<key>NSCameraUsageDescription</key>
<string>Camera permission description</string>
<key>NSMicrophoneUsageDescription</key>
<string>Microphone permission description</string>

ios/projectname/info.plist

Register Service

import { AppRegistry } from "react-native";
import App from "./App";
import { name as appName } from "./app.json";
import { register } from "@videosdk.live/react-native-sdk";

register();

AppRegistry.registerComponent(appName, () => App);

index.js

🎥 Essential Steps for Implement the Video Calling Functionality

Step 1: Get started with api.js

export const token = "<Generated-from-dashbaord>";
// API call to create meeting
export const createMeeting = async ({ token }) => {
  const res = await fetch(`https://api.videosdk.live/v2/rooms`, {
    method: "POST",
    headers: {
      authorization: `${token}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({}),
  });

  const { roomId } = await res.json();
  return roomId;
};

api.js

Step 2: Wireframe App.js with all the components

To build up a wireframe of App.js, you need to use VideoSDK Hooks and Context Providers. VideoSDK provides MeetingProvider, MeetingConsumer, useMeeting, and useParticipant hooks.

First, you need to understand the Context Provider and Consumer. Context is primarily used when some data needs to be accessible by many components at different nesting levels.

MeetingProvider : This is the Context Provider. It accepts value config and token as a prop. The Provider component accepts a value prop to be passed to consuming components that are descendants of this Provider. One Provider can be connected to many consumers. Providers can be nested to override values deeper within the tree.
MeetingConsumer : This is the Context Consumer. All consumers that are descendants of a Provider will re-render whenever the Provider’s value prop changes.
useMeeting : This is the meeting hook API. It includes all the information related to meetings such as join, leave, enable/disable the mic or webcam, etc.
useParticipant : This is the participant hook API. It is responsible for handling all the events and props related to one particular participant such as name , webcamStream , micStream , etc.

The Meeting Context provides a way to listen for any changes that occur when a participant joins the meeting or makes modifications to their microphone, camera, and other settings.

Begin by making a few changes to the code in the App.js file.

import React, { useState } from "react";
import {
  SafeAreaView,
  TouchableOpacity,
  Text,
  TextInput,
  View,
  FlatList,
} from "react-native";
import {
  MeetingProvider,
  useMeeting,
  useParticipant,
  MediaStream,
  RTCView,
} from "@videosdk.live/react-native-sdk";
import { createMeeting, token } from "./api";

function JoinScreen(props) {
  return null;
}

function ControlsContainer() {
  return null;
}

function MeetingView() {
  return null;
}

export default function App() {
  const [meetingId, setMeetingId] = useState(null);

  const getMeetingId = async (id) => {
    const meetingId = id == null ? await createMeeting({ token }) : id;
    setMeetingId(meetingId);
  };

  return meetingId ? (
    <SafeAreaView style={{ flex: 1, backgroundColor: "#F6F6FF" }}>
      <MeetingProvider
        config={{
          meetingId,
          micEnabled: false,
          webcamEnabled: true,
          name: "Test User",
        }}
        token={token}
      >
        <MeetingView />
      </MeetingProvider>
    </SafeAreaView>
  ) : (
    <JoinScreen getMeetingId={getMeetingId} />
  );
}

App.js

Step 3: Implement Join Screen

The join screen will serve as a medium to either schedule a new meeting or join an existing one.

function JoinScreen(props) {
  const [meetingVal, setMeetingVal] = useState("");
  return (
    <SafeAreaView
      style={{
        flex: 1,
        backgroundColor: "#F6F6FF",
        justifyContent: "center",
        paddingHorizontal: 6 * 10,
      }}
    >
      <TouchableOpacity
        onPress={() => {
          props.getMeetingId();
        }}
        style={{ backgroundColor: "#1178F8", padding: 12, borderRadius: 6 }}
      >
        <Text style={{ color: "white", alignSelf: "center", fontSize: 18 }}>
          Create Meeting
        </Text>
      </TouchableOpacity>

      <Text
        style={{
          alignSelf: "center",
          fontSize: 22,
          marginVertical: 16,
          fontStyle: "italic",
          color: "grey",
        }}
      >
        ---------- OR ----------
      </Text>
      <TextInput
        value={meetingVal}
        onChangeText={setMeetingVal}
        placeholder={"XXXX-XXXX-XXXX"}
        style={{
          padding: 12,
          borderWidth: 1,
          borderRadius: 6,
          fontStyle: "italic",
        }}
      />
      <TouchableOpacity
        style={{
          backgroundColor: "#1178F8",
          padding: 12,
          marginTop: 14,
          borderRadius: 6,
        }}
        onPress={() => {
          props.getMeetingId(meetingVal);
        }}
      >
        <Text style={{ color: "white", alignSelf: "center", fontSize: 18 }}>
          Join Meeting
        </Text>
      </TouchableOpacity>
    </SafeAreaView>
  );
}

JoinScreen Component

Step 4: Implement Controls

The next step is to create a ControlsContainer component to manage features such as Join or leave a Meeting and Enable or Disable the Webcam/Mic.

In this step, the useMeeting hook is utilized to acquire all the required methods such as join(), leave(), toggleWebcam and toggleMic.

const Button = ({ onPress, buttonText, backgroundColor }) => {
  return (
    <TouchableOpacity
      onPress={onPress}
      style={{
        backgroundColor: backgroundColor,
        justifyContent: "center",
        alignItems: "center",
        padding: 12,
        borderRadius: 4,
      }}
    >
      <Text style={{ color: "white", fontSize: 12 }}>{buttonText}</Text>
    </TouchableOpacity>
  );
};

function ControlsContainer({ join, leave, toggleWebcam, toggleMic }) {
  return (
    <View
      style={{
        padding: 24,
        flexDirection: "row",
        justifyContent: "space-between",
      }}
    >
      <Button
        onPress={() => {
          join();
        }}
        buttonText={"Join"}
        backgroundColor={"#1178F8"}
      />
      <Button
        onPress={() => {
          toggleWebcam();
        }}
        buttonText={"Toggle Webcam"}
        backgroundColor={"#1178F8"}
      />
      <Button
        onPress={() => {
          toggleMic();
        }}
        buttonText={"Toggle Mic"}
        backgroundColor={"#1178F8"}
      />
      <Button
        onPress={() => {
          leave();
        }}
        buttonText={"Leave"}
        backgroundColor={"#FF0000"}
      />
    </View>
  );
}

ControlsContainer Component

function ParticipantList() {
  return null;
}
function MeetingView() {
  const { join, leave, toggleWebcam, toggleMic, meetingId } = useMeeting({});

  return (
    <View style={{ flex: 1 }}>
      {meetingId ? (
        <Text style={{ fontSize: 18, padding: 12 }}>
          Meeting Id :{meetingId}
        </Text>
      ) : null}
      <ParticipantList />
      <ControlsContainer
        join={join}
        leave={leave}
        toggleWebcam={toggleWebcam}
        toggleMic={toggleMic}
      />
    </View>
  );
}

MeetingView Component

Step 5: Render Participant List

After implementing the controls, the next step is to render the joined participants.

You can get all the joined participants from the useMeeting Hook.

function ParticipantView() {
  return null;
}

function ParticipantList({ participants }) {
  return participants.length > 0 ? (
    <FlatList
      data={participants}
      renderItem={({ item }) => {
        return <ParticipantView participantId={item} />;
      }}
    />
  ) : (
    <View
      style={{
        flex: 1,
        backgroundColor: "#F6F6FF",
        justifyContent: "center",
        alignItems: "center",
      }}
    >
      <Text style={{ fontSize: 20 }}>Press Join button to enter meeting.</Text>
    </View>
  );
}

ParticipantList Component

function MeetingView() {
  // Get `participants` from useMeeting Hook
  const { join, leave, toggleWebcam, toggleMic, participants } = useMeeting({});
  const participantsArrId = [...participants.keys()];

  return (
    <View style={{ flex: 1 }}>
      <ParticipantList participants={participantsArrId} />
      <ControlsContainer
        join={join}
        leave={leave}
        toggleWebcam={toggleWebcam}
        toggleMic={toggleMic}
      />
    </View>
  );
}

MeetingView Component

Step 6: Handling Participant's Media

Before Handling the Participant's Media, you need to understand a couple of concepts.

1. useParticipant Hook

The useParticipant hook is responsible for handling all the properties and events of one particular participant who joined the meeting. It will take participantId as argument.

const { webcamStream, webcamOn, displayName } = useParticipant(participantId);

useParticipant Hook Example

2. MediaStream API

The MediaStream API is beneficial for adding a MediaTrack to the RTCView component, enabling the playback of audio or video.

<RTCView
  streamURL={new MediaStream([webcamStream.track]).toURL()}
  objectFit={"cover"}
  style={{
    height: 300,
    marginVertical: 8,
    marginHorizontal: 8,
  }}
/>

useParticipant Hook Example

Rendering Participant Media

function ParticipantView({ participantId }) {
  const { webcamStream, webcamOn } = useParticipant(participantId);

  return webcamOn && webcamStream ? (
    <RTCView
      streamURL={new MediaStream([webcamStream.track]).toURL()}
      objectFit={"cover"}
      style={{
        height: 300,
        marginVertical: 8,
        marginHorizontal: 8,
      }}
    />
  ) : (
    <View
      style={{
        backgroundColor: "grey",
        height: 300,
        justifyContent: "center",
        alignItems: "center",
      }}
    >
      <Text style={{ fontSize: 16 }}>NO MEDIA</Text>
    </View>
  );
}

ParticipantView Component

Congratulations! By following these steps, you're on your way to unlocking the video within your application. Now, we are moving forward to integrate the RTMP feature that builds immersive video experiences for your users!

Integrate RTMP Live Stream Feature

RTMP is a widely used protocol for live streaming video content from VideoSDK to platforms like YouTube, Twitch, Facebook, and others.

Furthermore, VideoSDK offers flexibility in configuring livestream layouts. You can achieve this by either selecting different prebuilt layouts in the configuration or by providing your own custom template for livestreaming, catering to your specific layout preferences.

This guide will provide an overview of how to implement starting and stopping RTMP live streaming with VideoSDK.

Start Livestream

The startLivestream() method, accessible from the useMeeting hook, is used to initiate the RTMP live stream of a meeting. This method accepts the following two parameters:

1. outputs: This parameter takes an array of objects containing the RTMP url and streamKey specific to the platform where you want to initiate the live stream.
2. config (optional): This parameter defines the layout configuration for the live stream.

const config = {
  // Layout Configuration
  layout: {
    type: "GRID", // "SPOTLIGHT" | "SIDEBAR", Default : "GRID"
    priority: "SPEAKER", // "PIN", Default : "SPEAKER"
    gridSize: 4, // MAX : 4
  },

  // Theme of RTMP
  theme: "DARK", // "LIGHT" | "DEFAULT"
};

const outputs = [
  {
    url: "<RTMP_URL>",
    streamKey: "<RTMP_STREAM_KEY>",
  },
];

startLivestream(outputs, config);

Stop Livestream

The stopLivestream() method, accessible from the useMeeting hook is used to stop the RTMP live stream of a meeting.

import { useMeeting } from "@videosdk.live/react-native-sdk";
import { TouchableOpacity, Text } from "react-native";

const MeetingView = () => {
  const { startLivestream, stopLivestream } = useMeeting();

  const handleStartLivestream = () => {
    // Start Livestream
    startLivestream(
      [
        {
          url: "rtmp://a.rtmp.youtube.com/live2",
          streamKey: "key",
        },
      ],
      {
        layout: {
          type: "GRID",
          priority: "SPEAKER",
          gridSize: 4,
        },
        theme: "DARK",
      }
    );
  };

  const handleStopLivestream = () => {
    // Stop Livestream
    stopLivestream();
  };

  return (
    <>
      <TouchableOpacity
        onPress={() => {
          handleStartLivestream();
        }}
      >
        <Text>Start Livestream</Text>
      </TouchableOpacity>

      <TouchableOpacity
        onPress={() => {
          handleStopLivestream();
        }}
      >
        <Text>Stop Livestream</Text>
      </TouchableOpacity>
    </>
  );
};

Event associated with Livestream

onLivestreamStateChanged - The onLivestreamStateChanged() event is triggered whenever the state of the meeting livestream changes.

import { Constants, useMeeting } from "@videosdk.live/react-native-sdk";

function onLivestreamStateChanged(data) {
  const { status } = data;

  if (status === Constants.livestreamEvents.LIVESTREAM_STARTING) {
    console.log("Meeting livestream is starting");
  } else if (status === Constants.livestreamEvents.LIVESTREAM_STARTED) {
    console.log("Meeting livestream is started");
  } else if (status === Constants.livestreamEvents.LIVESTREAM_STOPPING) {
    console.log("Meeting livestream is stopping");
  } else if (status === Constants.livestreamEvents.LIVESTREAM_STOPPED) {
    console.log("Meeting livestream is stopped");
  } else {
    //
  }
 }

const {
  meetingId
  ...
} = useMeeting({
  onLivestreamStateChanged,
});

Custom Template

With VideoSDK, you can also use your custom-designed layout template to record the meetings. To use the custom template, you need to create a template, for which you can follow this guide. Once you have set up the template, you can use the REST API to start the recording with the templateURL parameter.

🔚 Conclusion

Integrating RTMP functionality into your React Native app with videoSDK empowers you to create a more versatile and engaging user experience. This guide provided a roadmap for setting up the development environment, configuring the SDK, and establishing a connection to your RTMP server. With RTMP support, you can enable features like live streaming to external platforms or broadcasting calls to a wider audience.

Remember, VideoSDK offers a generous free tier that includes 10,000 minutes allowing you to experiment and build your video call application without initial investment. Start your free trial today and unlock the potential of RTMP streaming in your React Native app!

How to Integrate RTMP Livestream in JavaScript Video Chat App?

Kishan Nakrani — Tue, 30 Apr 2024 06:29:26 +0000

📌 Introduction

Integrating RTMP (Real-Time Messaging Protocol) live streaming into your JavaScript video chat application can enhance your platform's capabilities, allowing users to broadcast their video chats in real-time. With RTMP, you can seamlessly stream video and audio content to various platforms such as YouTube, Twitch, or your own streaming server.

Benefits of Integrating RTMP Livestream in a JavaScript video chat app:

Enhanced Interactivity : RTMP allows for real-time streaming, enabling users to engage in live interactions, and making video chats more dynamic and engaging.
Scalability : With RTMP, your app can handle a large number of concurrent streams, ensuring scalability as your user base grows.
Low Latency : RTMP provides low-latency streaming, reducing the delay between sending and receiving video data, resulting in smoother conversations.

Use Cases of Integrating RTMP Livestream in a JavaScript video chat app:

Live Events : Broadcasting live events such as conferences, webinars, or concerts where real-time interaction with viewers is crucial.
Online Education : Facilitating live classes or tutoring sessions with the ability for students to ask questions and interact with the instructor in real time.
Virtual Meetings : Enhancing video conferencing apps with real-time streaming capabilities for more engaging and productive meetings.

This tutorial will guide you through a step-by-step process, enabling you to build a JavaScript video chat app with RTMP integration & VideoSDK.

🚀 Getting Started with VideoSDK

Before we get into implementing RTMP Live Stream functionality, let's make sure you've completed the necessary prerequisites to take advantage of it with the VideoSDK capabilities.

Create a VideoSDK Account

Generate your Auth Token

Prerequisites

Before proceeding, ensure that your development environment meets the following requirements:

VideoSDK Developer Account (if you do not have one, follow VideoSDK Dashboard)
Have Node and NPM installed on your device.

🛠️ Install VideoSDK

Import VideoSDK using the <script> tag or install it using the following npm command. Make sure you are in your app directory before you run this command.

<html>
  <head>
    <!--.....-->
  </head>
  <body>
    <!--.....-->
    <script src="https://sdk.videosdk.live/js-sdk/0.0.83/videosdk.js"></script>
  </body>
</html>

npm

npm install @videosdk.live/js-sdk

Yarn

yarn add @videosdk.live/js-sdk

Structure of the project

Your project structure should look like this.

  root
   ├── index.html
   ├── config.js
   ├── index.js

You will be working on the following files:

index.html : Responsible for creating a basic UI.
config.js : Responsible for storing the token.
index.js : Responsible for rendering the meeting view and the join meeting functionality.

🎥 Essential Steps to Implement Video Call Functionality

Once you've successfully installed VideoSDK in your project, you'll have access to a range of functionalities for building your video call application. Active Speaker Indication is one such feature that leverages VideoSDK's capabilities. It leverages VideoSDK's capabilities to identify the user with the strongest audio signal (the one speaking)

Step 1: Design the user interface (UI)

Create an HTML file containing the screens, join-screen and grid-screen.

<!DOCTYPE html>
<html>
  <head> </head>

  <body>
    <div id="join-screen">
      <!-- Create new Meeting Button -->
      <button id="createMeetingBtn">New Meeting</button>
      OR
      <!-- Join existing Meeting -->
      <input type="text" id="meetingIdTxt" placeholder="Enter Meeting id" />
      <button id="joinBtn">Join Meeting</button>
    </div>

    <!-- for Managing meeting status -->
    <div id="textDiv"></div>

    <div id="grid-screen" style="display: none">
      <!-- To Display MeetingId -->
      <h3 id="meetingIdHeading"></h3>

      <!-- Controllers -->
      <button id="leaveBtn">Leave</button>
      <button id="toggleMicBtn">Toggle Mic</button>
      <button id="toggleWebCamBtn">Toggle WebCam</button>

      <!-- render Video -->
      <div class="row" id="videoContainer"></div>
    </div>

    <!-- Add VideoSDK script -->
    <script src="https://sdk.videosdk.live/js-sdk/0.0.83/videosdk.js"></script>
    <script src="config.js"></script>
    <script src="index.js"></script>
  </body>
</html>

Step 2: Implement Join Screen

Configure the token in the config.js file, which you can obtain from the VideoSDK Dashbord.

// Auth token will be used to generate a meeting and connect to it
TOKEN = "Your_Token_Here";

Next, retrieve all the elements from the DOM and declare the following variables in the index.js file. Then, add an event listener to the join and create meeting buttons.

// Getting Elements from DOM
const joinButton = document.getElementById("joinBtn");
const leaveButton = document.getElementById("leaveBtn");
const toggleMicButton = document.getElementById("toggleMicBtn");
const toggleWebCamButton = document.getElementById("toggleWebCamBtn");
const createButton = document.getElementById("createMeetingBtn");
const videoContainer = document.getElementById("videoContainer");
const textDiv = document.getElementById("textDiv");

// Declare Variables
let meeting = null;
let meetingId = "";
let isMicOn = false;
let isWebCamOn = false;

function initializeMeeting() {}

function createLocalParticipant() {}

function createVideoElement() {}

function createAudioElement() {}

function setTrack() {}

// Join Meeting Button Event Listener
joinButton.addEventListener("click", async () => {
  document.getElementById("join-screen").style.display = "none";
  textDiv.textContent = "Joining the meeting...";

  roomId = document.getElementById("meetingIdTxt").value;
  meetingId = roomId;

  initializeMeeting();
});

// Create Meeting Button Event Listener
createButton.addEventListener("click", async () => {
  document.getElementById("join-screen").style.display = "none";
  textDiv.textContent = "Please wait, we are joining the meeting";

  // API call to create meeting
  const url = `https://api.videosdk.live/v2/rooms`;
  const options = {
    method: "POST",
    headers: { Authorization: TOKEN, "Content-Type": "application/json" },
  };

  const { roomId } = await fetch(url, options)
    .then((response) => response.json())
    .catch((error) => alert("error", error));
  meetingId = roomId;

  initializeMeeting();
});

Step 3: Initialize Meeting

Following that, initialize the meeting using the initMeeting() function and proceed to join the meeting.

// Initialize meeting
function initializeMeeting() {
  window.VideoSDK.config(TOKEN);

  meeting = window.VideoSDK.initMeeting({
    meetingId: meetingId, // required
    name: "Thomas Edison", // required
    micEnabled: true, // optional, default: true
    webcamEnabled: true, // optional, default: true
  });

  meeting.join();

  // Creating local participant
  createLocalParticipant();

  // Setting local participant stream
  meeting.localParticipant.on("stream-enabled", (stream) => {
    setTrack(stream, null, meeting.localParticipant, true);
  });

  // meeting joined event
  meeting.on("meeting-joined", () => {
    textDiv.style.display = "none";
    document.getElementById("grid-screen").style.display = "block";
    document.getElementById(
      "meetingIdHeading"
    ).textContent = `Meeting Id: ${meetingId}`;
  });

  // meeting left event
  meeting.on("meeting-left", () => {
    videoContainer.innerHTML = "";
  });

  // Remote participants Event
  // participant joined
  meeting.on("participant-joined", (participant) => {
    // ...
  });

  // participant left
  meeting.on("participant-left", (participant) => {
    // ...
  });
}

Step 4: Create the Media Elements

// creating video element
function createVideoElement(pId, name) {
  let videoFrame = document.createElement("div");
  videoFrame.setAttribute("id", `f-${pId}`);
  videoFrame.style.width = "300px";


  //create video
  let videoElement = document.createElement("video");
  videoElement.classList.add("video-frame");
  videoElement.setAttribute("id", `v-${pId}`);
  videoElement.setAttribute("playsinline", true);
  videoElement.setAttribute("width", "300");
  videoFrame.appendChild(videoElement);

  let displayName = document.createElement("div");
  displayName.innerHTML = `Name : ${name}`;

  videoFrame.appendChild(displayName);
  return videoFrame;
}

// creating audio element
function createAudioElement(pId) {
  let audioElement = document.createElement("audio");
  audioElement.setAttribute("autoPlay", "false");
  audioElement.setAttribute("playsInline", "true");
  audioElement.setAttribute("controls", "false");
  audioElement.setAttribute("id", `a-${pId}`);
  audioElement.style.display = "none";
  return audioElement;
}

// creating local participant
function createLocalParticipant() {
  let localParticipant = createVideoElement(
    meeting.localParticipant.id,
    meeting.localParticipant.displayName
  );
  videoContainer.appendChild(localParticipant);
}

// setting media track
function setTrack(stream, audioElement, participant, isLocal) {
  if (stream.kind == "video") {
    isWebCamOn = true;
    const mediaStream = new MediaStream();
    mediaStream.addTrack(stream.track);
    let videoElm = document.getElementById(`v-${participant.id}`);
    videoElm.srcObject = mediaStream;
    videoElm
      .play()
      .catch((error) =>
        console.error("videoElem.current.play() failed", error)
      );
  }
  if (stream.kind == "audio") {
    if (isLocal) {
      isMicOn = true;
    } else {
      const mediaStream = new MediaStream();
      mediaStream.addTrack(stream.track);
      audioElement.srcObject = mediaStream;
      audioElement
        .play()
        .catch((error) => console.error("audioElem.play() failed", error));
    }
  }
}

Step 5: Handle participant events

Thereafter, implement the events related to the participants and the stream.

The following are the events to be executed in this step:

participant-joined: When a remote participant joins, this event will trigger. In the event callback, create video and audio elements previously defined for rendering their video and audio streams.
participant-left: When a remote participant leaves, this event will trigger. In the event callback, remove the corresponding video and audio elements.
stream-enabled: This event manages the media track of a specific participant by associating it with the appropriate video or audio element.

// Initialize meeting
function initializeMeeting() {
  // ...

  // participant joined
  meeting.on("participant-joined", (participant) => {
    let videoElement = createVideoElement(
      participant.id,
      participant.displayName
    );
    let audioElement = createAudioElement(participant.id);
    // stream-enabled
    participant.on("stream-enabled", (stream) => {
      setTrack(stream, audioElement, participant, false);
    });
    videoContainer.appendChild(videoElement);
    videoContainer.appendChild(audioElement);
  });

  // participants left
  meeting.on("participant-left", (participant) => {
    let vElement = document.getElementById(`f-${participant.id}`);
    vElement.remove(vElement);

    let aElement = document.getElementById(`a-${participant.id}`);
    aElement.remove(aElement);
  });
}

Step 6: Implement Controls

Next, implement the meeting controls, such as toggleMic, toggleWebcam, and leave the meeting.

// leave Meeting Button Event Listener
leaveButton.addEventListener("click", async () => {
  meeting?.leave();
  document.getElementById("grid-screen").style.display = "none";
  document.getElementById("join-screen").style.display = "block";
});

// Toggle Mic Button Event Listener
toggleMicButton.addEventListener("click", async () => {
  if (isMicOn) {
    // Disable Mic in Meeting
    meeting?.muteMic();
  } else {
    // Enable Mic in Meeting
    meeting?.unmuteMic();
  }
  isMicOn = !isMicOn;
});

// Toggle Web Cam Button Event Listener
toggleWebCamButton.addEventListener("click", async () => {
  if (isWebCamOn) {
    // Disable Webcam in Meeting
    meeting?.disableWebcam();

    let vElement = document.getElementById(`f-${meeting.localParticipant.id}`);
    vElement.style.display = "none";
  } else {
    // Enable Webcam in Meeting
    meeting?.enableWebcam();

    let vElement = document.getElementById(`f-${meeting.localParticipant.id}`);
    vElement.style.display = "inline";
  }
  isWebCamOn = !isWebCamOn;
});

You can check out the complete guide here:

After you've installed the video SDK in your video call application, you're ready to explore the exciting world of RTMP live streaming. RTMP is a popular protocol designed specifically for real-time video and audio transmission. This enables broadcasters to share live experiences, host presentations, or run live tutorials within your app, expanding its reach and functionality.

Integrate RTMP Livestream

RTMP is a widely used protocol for live streaming video content from VideoSDK to platforms like YouTube, Twitch, Facebook, and others.

Furthermore, VideoSDK offers flexibility in configuring livestream layouts. You can achieve this by either selecting different prebuilt layouts in the configuration or by providing your own custom template for live streaming, catering to your specific layout preferences.

This guide will provide an overview of how to implement starting and stopping RTMP live streaming with VideoSDK.

Start Live Stream

The startLivestream() method, accessible from the meeting object, is used to initiate the RTMP live stream of a meeting. This method accepts the following two parameters:

1. outputs: This parameter takes an array of objects containing the RTMP url and streamKey specific to the platform where you want to initiate the live stream.
2. config (optional): This parameter defines the layout configuration for the live stream.

let meeting;

// Initialize Meeting
meeting = VideoSDK.initMeeting({
  // ...
});

const startLivestreamBtn = document.getElementById("startLivestreamBtn");
startLivestreamBtn.addEventListener("click", () => {
  // Start Livestream
  meeting?.startLivestream(
    [
      {
        url: "rtmp://a.rtmp.youtube.com/live2",
        streamKey: "key",
      },
    ],
    {
      layout: {
        type: "GRID",
        priority: "SPEAKER",
        gridSize: 4,
      },
      theme: "DARK",
    }
  );
});

Stop Live Stream

The stopLivestream() method, accessible from the meeting object is used to stop the RTMP live stream of a meeting.

let meeting;

// Initialize Meeting
meeting = VideoSDK.initMeeting({
  // ...
});

const stopLivestreamBtn = document.getElementById("stopLivestreamBtn");
stopLivestreamBtn.addEventListener("click", () => {
  // Stop Livestream
  meeting?.stopLivestream();
});

Event associated with Livestream

livestream-state-changed : Whenever a livestream state changes, then livestream-state-changed the event will trigger.

let meeting;

// Initialize Meeting
meeting = VideoSDK.initMeeting({
  // ...
});

const Constants = VideoSDK.Constants;

meeting.on("livestream-state-changed", (data) => {
  const { status } = data;

  if (status === Constants.livestreamEvents.LIVESTREAM_STARTING) {
    console.log("Meeting livestream is starting");
  } else if (status === Constants.livestreamEvents.LIVESTREAM_STARTED) {
    console.log("Meeting livestream is started");
  } else if (status === Constants.livestreamEvents.LIVESTREAM_STOPPING) {
    console.log("Meeting livestream is stopping");
  } else if (status === Constants.livestreamEvents.LIVESTREAM_STOPPED) {
    console.log("Meeting livestream is stopped");
  } else {
    //
  }
});

Custom Template

With VideoSDK, you have the option to employ your own custom-designed layout template for livestreaming a meeting. To use a custom template, follow this guide to create and set up the template. Once the template is configured, you can initiate recording using the REST API, specifying the templateURL parameter.

🔚 Conclusion

Integrating RTMP Livestream into your JavaScript Video Chat App is a powerful way to enhance its capabilities. By using the VideoSDK, you can easily implement this feature and provide users with seamless live streaming experiences. Whether it's for hosting live events, broadcasting presentations, or enabling interactive streaming sessions, RTMP Livestream opens up new possibilities for your app. Take your video chat app to the next level with VideoSDK and empower your users with high-quality live streaming. Get started today and revolutionize your app's video capabilities

VideoSDK offers 10,000 free minutes to get you started. So, why wait? Follow this guide, integrate RTMP functionality, and unleash the power of live streaming in your video call app today!

How to Integrate Screen Share in iOS Video Call App?

Kishan Nakrani — Mon, 29 Apr 2024 12:50:51 +0000

📌 Introduction

Integrating screen share in an iOS video call app enhances user experience and collaboration. With this feature, users can seamlessly share their screen during calls, facilitating presentations, demonstrations, and remote assistance. Implementing screen sharing requires integrating APIs for capturing device screens, ensuring smooth transmission of visuals, and maintaining privacy controls.

Benefits of Screen Share in iOS Video Call App:

Enhanced Collaboration: Screen share enables users to collaborate more effectively by sharing documents, presentations, or designs during video calls, fostering better understanding and teamwork.
Improved Communication: Visual aids facilitate clearer communication, especially for technical support, education, or remote work scenarios, leading to faster issue resolution and knowledge transfer.
Increased Productivity: With real-time sharing, teams can discuss projects or review documents without the need for additional tools or meetings, saving time and boosting productivity.

Use Cases of Screen Share in iOS Video Call App:

Business Meetings: Sales teams can share presentations or product demos with clients, enhancing engagement and closing deals more effectively.
Remote Work: Colleagues can collaborate on projects by sharing screens to discuss documents, designs, or code, replicating in-person collaboration remotely.
Technical Support: Customer support agents can visually guide users through troubleshooting steps by sharing screens, and resolving issues efficiently.

This tutorial guides you through integrating this valuable feature into your JavaScript video call application using VideoSDK. We'll cover the steps required to leverage VideoSDK's capabilities and implement visual cues that highlight the active speaker within your app's interface.

🚀 Getting Started with VideoSDK

Create a VideoSDK Account

Generate your Auth Token

Prerequisites and Setup

iOS 11.0+
Xcode 12.0+
Swift 5.0+

This App will contain two screens:

Join Screen : This screen allows the user to either create a meeting or join the predefined meeting.

Meeting Screen : This screen basically contains local and remote participant views and some meeting controls such as Enable/Disable the mic & Camera and Leave meeting.

🛠️ Integrate VideoSDK

To install VideoSDK, you must initialise the pod on the project by running the following command:

pod init

It will create the podfile in your project folder, Open that file and add the dependency for the VideoSDK, like below:

pod 'VideoSDKRTC', :git => 'https://github.com/videosdk-live/videosdk-rtc-ios-sdk.git'

then run the below code to install the pod:

pod install

then declare the permissions in Info.plist :

<key>NSCameraUsageDescription</key>
<string>Camera permission description</string>
<key>NSMicrophoneUsageDescription</key>
<string>Microphone permission description</string>

Project Structure

iOSQuickStartDemo
   ├── Models
        ├── RoomStruct.swift
        └── MeetingData.swift
   ├── ViewControllers
        ├── StartMeetingViewController.swift
        └── MeetingViewController.swift
   ├── AppDelegate.swift // Default
   ├── SceneDelegate.swift // Default
   └── APIService
           └── APIService.swift
   ├── Main.storyboard // Default
   ├── LaunchScreen.storyboard // Default
   └── Info.plist // Default
 BroadcastExtension
   ├── SampleHandler.swift // Default
   ├── Atomic.swift
   └── SocketConnection.swift
   ├── DarwinNotification.swift
   ├── SampleUploader.swift
   └── Info.plist // Default
 Pods
     └── Podfile

Create models

Create a swift file for MeetingData and RoomStruct class model for setting data in object pattern.

import Foundation
struct MeetingData {
    let token: String
    let name: String
    let meetingId: String
    let micEnabled: Bool
    let cameraEnabled: Bool
}

MeetingData.swift

import Foundation
struct RoomsStruct: Codable {
    let createdAt, updatedAt, roomID: String?
    let links: Links?
    let id: String?
    enum CodingKeys: String, CodingKey {
        case createdAt, updatedAt
        case roomID = "roomId"
        case links, id
    }
}

// MARK: - Links
struct Links: Codable {
    let getRoom, getSession: String?
    enum CodingKeys: String, CodingKey {
        case getRoom = "get_room"
        case getSession = "get_session"
    }
}

RoomStruct.swift

🎥 Essential Steps for Building the Video Calling

This guide is designed to walk you through the process of integrating Screen Share with VideoSDK. We'll cover everything from setting up the SDK to incorporating the visual cues into your app's interface, ensuring a smooth and efficient implementation process.

Step 1: Get started with APIClient

Before jumping to anything else, we have to write an API to generate unique meetingId. You will require an authentication token; you can generate it either using videosdk-server-api-example or from the VideoSDK Dashboard for developers.

import Foundation

let TOKEN_STRING: String = "<AUTH_TOKEN>"

class APIService {

  class func createMeeting(token: String, completion: @escaping (Result<String, Error>) -> Void) {

    let url = URL(string: "https://api.videosdk.live/v2/rooms")!

    var request = URLRequest(url: url)
    request.httpMethod = "POST"
    request.addValue(TOKEN_STRING, forHTTPHeaderField: "authorization")

    URLSession.shared.dataTask(
      with: request,
      completionHandler: { (data: Data?, response: URLResponse?, error: Error?) in

        DispatchQueue.main.async {

          if let data = data, let utf8Text = String(data: data, encoding: .utf8) {
            do {
              let dataArray = try JSONDecoder().decode(RoomsStruct.self, from: data)

              completion(.success(dataArray.roomID ?? ""))
            } catch {
              print("Error while creating a meeting: \(error)")
              completion(.failure(error))
            }
          }
        }
      }
    ).resume()
  }
}

APIService.swift

Step 2 : Implement Join Screen

The Join Screen will work as a medium to either schedule a new meeting or join an existing meeting.

import Foundation
import UIKit

class StartMeetingViewController: UIViewController, UITextFieldDelegate {

  private var serverToken = ""

  /// MARK: outlet for create meeting button
  @IBOutlet weak var btnCreateMeeting: UIButton!

  /// MARK: outlet for join meeting button
  @IBOutlet weak var btnJoinMeeting: UIButton!

  /// MARK: outlet for meetingId textfield
  @IBOutlet weak var txtMeetingId: UITextField!

  /// MARK: Initialize the private variable with TOKEN_STRING &
  /// setting the meeting id in the textfield
  override func viewDidLoad() {
    txtMeetingId.delegate = self
    serverToken = TOKEN_STRING
    txtMeetingId.text = "PROVIDE-STATIC-MEETING-ID"
  }

  /// MARK: method for joining meeting through seague named as "StartMeeting"
  /// after validating the serverToken in not empty
  func joinMeeting() {

    txtMeetingId.resignFirstResponder()

    if !serverToken.isEmpty {
      DispatchQueue.main.async {
        self.dismiss(animated: true) {
          self.performSegue(withIdentifier: "StartMeeting", sender: nil)
        }
      }
    } else {
      print("Please provide auth token to start the meeting.")
    }
  }

  /// MARK: outlet for create meeting button tap event
  @IBAction func btnCreateMeetingTapped(_ sender: Any) {
    print("show loader while meeting gets connected with server")
    joinRoom()
  }

  /// MARK: outlet for join meeting button tap event
  @IBAction func btnJoinMeetingTapped(_ sender: Any) {
    if (txtMeetingId.text ?? "").isEmpty {

      print("Please provide meeting id to start the meeting.")
      txtMeetingId.resignFirstResponder()
    } else {
      joinMeeting()
    }
  }

  // MARK: - method for creating room api call and getting meetingId for joining meeting

  func joinRoom() {

    APIService.createMeeting(token: self.serverToken) { result in
      if case .success(let meetingId) = result {
        DispatchQueue.main.async {
          self.txtMeetingId.text = meetingId
          self.joinMeeting()
        }
      }
    }
  }

  /// MARK: preparing to animate to meetingViewController screen
  override func prepare(for segue: UIStoryboardSegue, sender: Any?) {

    guard let navigation = segue.destination as? UINavigationController,

      let meetingViewController = navigation.topViewController as? MeetingViewController
    else {
      return
    }

    meetingViewController.meetingData = MeetingData(
      token: serverToken,
      name: txtMeetingId.text ?? "Guest",
      meetingId: txtMeetingId.text ?? "",
      micEnabled: true,
      cameraEnabled: true
    )
  }
}

StartMeetingViewController.swift

Step 3 : Initialize and Join Meeting

Using the provided token and meetingId, we will configure and initialize the meeting in viewDidLoad().

Then, we'll add @IBOutlet for localParticipantVideoView and remoteParticipantVideoView, which can render local and remote participant media, respectively.

class MeetingViewController: UIViewController {

import UIKit
import VideoSDKRTC
import WebRTC
import AVFoundation

class MeetingViewController: UIViewController {

// MARK: - Properties
// outlet for local participant container view
   @IBOutlet weak var localParticipantViewContainer: UIView!

// outlet for label for meeting Id
   @IBOutlet weak var lblMeetingId: UILabel!

// outlet for local participant video view
   @IBOutlet weak var localParticipantVideoView: RTCMTLVideoView!

// outlet for remote participant video view
   @IBOutlet weak var remoteParticipantVideoView: RTCMTLVideoView!

// outlet for remote participant no media label
   @IBOutlet weak var lblRemoteParticipantNoMedia: UILabel!

// outlet for remote participant container view
   @IBOutlet weak var remoteParticipantViewContainer: UIView!

// outlet for local participant no media label
   @IBOutlet weak var lblLocalParticipantNoMedia: UILabel!

// Meeting data - required to start
   var meetingData: MeetingData!

// current meeting reference
   private var meeting: Meeting?

    // MARK: - video participants including self to show in UI
    private var participants: [Participant] = []

        // MARK: - Lifecycle Events

        override func viewDidLoad() {
        super.viewDidLoad()
        // configure the VideoSDK with token
        VideoSDK.config(token: meetingData.token)

        // init meeting
        initializeMeeting()

        // set meeting id in button text
        lblMeetingId.text = "Meeting Id: \(meetingData.meetingId)"
      }

      override func viewWillAppear(_ animated: Bool) {
          super.viewWillAppear(animated)
          navigationController?.navigationBar.isHidden = true
      }

    override func viewWillDisappear(_ animated: Bool) {
        super.viewWillDisappear(animated)
        navigationController?.navigationBar.isHidden = false
        NotificationCenter.default.removeObserver(self)
    }

        // MARK: - Meeting

        private func initializeMeeting() {

            // Initialize the VideoSDK
            meeting = VideoSDK.initMeeting(
                meetingId: meetingData.meetingId,
                participantName: meetingData.name,
                micEnabled: meetingData.micEnabled,
                webcamEnabled: meetingData.cameraEnabled
            )

            // Adding the listener to meeting
            meeting?.addEventListener(self)

            // joining the meeting
            meeting?.join()
        }
}

MeetingViewController.swift

Step 4 : Implement Controls

After initializing the meeting in the previous step, we will now add @IBOutlet for btnLeave, btnToggleVideo and btnToggleMic which can control the media in the meeting.

class MeetingViewController: UIViewController {

...

    // outlet for leave button
    @IBOutlet weak var btnLeave: UIButton!

    // outlet for toggle video button
    @IBOutlet weak var btnToggleVideo: UIButton!

    // outlet for toggle audio button
    @IBOutlet weak var btnToggleMic: UIButton!

    // bool for mic
    var micEnabled = true
    // bool for video
    var videoEnabled = true

    // outlet for leave button click event
    @IBAction func btnLeaveTapped(_ sender: Any) {
            DispatchQueue.main.async {
                self.meeting?.leave()
                self.dismiss(animated: true)
            }
        }

    // outlet for toggle mic button click event
    @IBAction func btnToggleMicTapped(_ sender: Any) {
        if micEnabled {
            micEnabled = !micEnabled // false
            self.meeting?.muteMic()
        } else {
            micEnabled = !micEnabled // true
            self.meeting?.unmuteMic()
        }
    }

    // outlet for toggle video button click event
    @IBAction func btnToggleVideoTapped(_ sender: Any) {
        if videoEnabled {
            videoEnabled = !videoEnabled // false
            self.meeting?.disableWebcam()
        } else {
            videoEnabled = !videoEnabled // true
            self.meeting?.enableWebcam()
        }
    }

...

}

MeetingViewController.swift

Step 5 : Implementing `MeetingEventListener`


class MeetingViewController: UIViewController {

...

extension MeetingViewController: MeetingEventListener {

        /// Meeting started
        func onMeetingJoined() {

            // handle local participant on start
            guard let localParticipant = self.meeting?.localParticipant else { return }
            // add to list
            participants.append(localParticipant)

            // add event listener
            localParticipant.addEventListener(self)

            localParticipant.setQuality(.high)

            if(localParticipant.isLocal){
                self.localParticipantViewContainer.isHidden = false
            } else {
                self.remoteParticipantViewContainer.isHidden = false
            }
        }

        /// Meeting ended
        func onMeetingLeft() {
            // remove listeners
            meeting?.localParticipant.removeEventListener(self)
            meeting?.removeEventListener(self)
        }

        /// A new participant joined
        func onParticipantJoined(_ participant: Participant) {
            participants.append(participant)

            // add listener
            participant.addEventListener(self)

            participant.setQuality(.high)

            if(participant.isLocal){
                self.localParticipantViewContainer.isHidden = false
            } else {
                self.remoteParticipantViewContainer.isHidden = false
            }
        }

        /// A participant left the meeting
        /// - Parameter participant: participant object
        func onParticipantLeft(_ participant: Participant) {
            participant.removeEventListener(self)
            guard let index = self.participants.firstIndex(where: { $0.id == participant.id }) else {
                return
            }
            // remove participant from list
            participants.remove(at: index)
            // hide from ui
            UIView.animate(withDuration: 0.5){
                if(!participant.isLocal){
                    self.remoteParticipantViewContainer.isHidden = true
                }
            }
        }

        /// Called when speaker is changed
        /// - Parameter participantId: participant id of the speaker, nil when no one is speaking.
        func onSpeakerChanged(participantId: String?) {

            // show indication for active speaker
            if let participant = participants.first(where: { $0.id == participantId }) {
                self.showActiveSpeakerIndicator(participant.isLocal ? localParticipantViewContainer : remoteParticipantViewContainer, true)
            }

            // hide indication for others participants
            let otherParticipants = participants.filter { $0.id != participantId }
            for participant in otherParticipants {
                if participants.count > 1 && participant.isLocal {
                    showActiveSpeakerIndicator(localParticipantViewContainer, false)
                } else {
                    showActiveSpeakerIndicator(remoteParticipantViewContainer, false)
                }
            }
        }

        func showActiveSpeakerIndicator(_ view: UIView, _ show: Bool) {
            view.layer.borderWidth = 4.0
            view.layer.borderColor = show ? UIColor.blue.cgColor : UIColor.clear.cgColor
        }

}

...

MeetingViewController.swift

Step 6: Implementing `ParticipantEventListener`

The function updateUI is frequently used to control or modify the user interface (enable/disable camera & mic) following MediaStream state.

class MeetingViewController: UIViewController {

...

extension MeetingViewController: ParticipantEventListener {

/// Participant has enabled mic, video or screenshare
/// - Parameters:
/// - stream: enabled stream object
/// - participant: participant object
func onStreamEnabled(_ stream: MediaStream, forParticipant participant: Participant) {
    updateUI(participant: participant, forStream: stream, enabled: true)
 }

/// Participant has disabled mic, video or screenshare
/// - Parameters:
/// - stream: disabled stream object
/// - participant: participant object

func onStreamDisabled(_ stream: MediaStream, 
            forParticipant participant: Participant) {

  updateUI(participant: participant, forStream: stream, enabled: false)
 }

}

private extension MeetingViewController {

 func updateUI(participant: Participant, forStream stream: MediaStream, enabled: Bool) { // true
        switch stream.kind {
        case .state(value: .video):
            if let videotrack = stream.track as? RTCVideoTrack {
                if enabled {
                    DispatchQueue.main.async {
                        UIView.animate(withDuration: 0.5){

                            if(participant.isLocal) {

        self.localParticipantViewContainer.isHidden = false
    self.localParticipantVideoView.isHidden = false       
    self.localParticipantVideoView.videoContentMode = .scaleAspectFill self.localParticipantViewContainer.bringSubviewToFront(self.localParticipantVideoView)                                                           
    videotrack.add(self.localParticipantVideoView)
    self.lblLocalParticipantNoMedia.isHidden = true

} else {
        self.remoteParticipantViewContainer.isHidden = false
            self.remoteParticipantVideoView.isHidden = false
                                self.remoteParticipantVideoView.videoContentMode = .scaleAspectFill
                                self.remoteParticipantViewContainer.bringSubviewToFront(self.remoteParticipantVideoView)
                                                videotrack.add(self.remoteParticipantVideoView)
 self.lblRemoteParticipantNoMedia.isHidden = true
        }
     }
  }
} else {
         UIView.animate(withDuration: 0.5){
                if(participant.isLocal){

                    self.localParticipantViewContainer.isHidden = false
                    self.localParticipantVideoView.isHidden = true
                    self.lblLocalParticipantNoMedia.isHidden = false
                            videotrack.remove(self.localParticipantVideoView)
} else {
                   self.remoteParticipantViewContainer.isHidden = false
                   self.remoteParticipantVideoView.isHidden = true
                   self.lblRemoteParticipantNoMedia.isHidden = false
                            videotrack.remove(self.remoteParticipantVideoView)
      }
    }
  }
}

     case .state(value: .audio):
            if participant.isLocal {

               localParticipantViewContainer.layer.borderWidth = 4.0
               localParticipantViewContainer.layer.borderColor = enabled ? UIColor.clear.cgColor : UIColor.red.cgColor
            } else {
                remoteParticipantViewContainer.layer.borderWidth = 4.0
                remoteParticipantViewContainer.layer.borderColor = enabled ? UIColor.clear.cgColor : UIColor.red.cgColor
            }
        default:
            break
        }
    }
}

...

Known Issue

Please add the following line in the MeetingViewController.swift file's viewDidLoad method If you get your video out of the container.

override func viewDidLoad() {

  localParticipantVideoView.frame = CGRect(x: 10, y: 0, 
            width: localParticipantViewContainer.frame.width, 
        height: localParticipantViewContainer.frame.height)

  localParticipantVideoView.bounds = CGRect(x: 10, y: 0, 
        width: localParticipantViewContainer.frame.width, 
            height: localParticipantViewContainer.frame.height)

  localParticipantVideoView.clipsToBounds = true

  remoteParticipantVideoView.frame = CGRect(x: 10, y: 0, 
        width: remoteParticipantViewContainer.frame.width, 
            height: remoteParticipantViewContainer.frame.height)

  remoteParticipantVideoView.bounds = CGRect(x: 10, y: 0, 
        width: remoteParticipantViewContainer.frame.width, 
            height: remoteParticipantViewContainer.frame.height)

    remoteParticipantVideoView.clipsToBounds = true
}

MeetingViewController.swift

TIP:

Stuck anywhere? Check out this example code on GitHub.

📸 Integrate Screen Share in Video App

Step 1 : Open Target

Open your project with XCode, then select File > New > Target in the menu bar.

Step 2: Select Target

Select Broadcast Upload Extension and click

Step 3: Configure Broadcast Upload Extension

Enter the extension's name in the Product Name field, choose the team from the dropdown, uncheck the "Include UI extension" field, and click "Finish."

Step 4: Activate Extension scheme

You will be prompted with a popup : Activate "Your-Extension-name" scheme? , click on activate.

Now, the "Broadcast" folder will appear in the Xcode left side bar.

Step 5: Add External file in the created extension.

Open the videosdk-rtc-ios-sdk-example repository, and copy the following files: SampleUploader.swift, SocketConnection.swift, DarwinNotificationCenter.swift, and Atomic.swift to your extension's folder. Ensure that these files are added to the target.

Step 6: Update `SampleHandler.swift` file

Open SampleHandler.swift, and copy the content of the file. Paste this content into your extension's SampleHandler.swift file.

Step 7: Add Capability to App

In Xcode, navigate to YourappName > Signing & Capabilities , and click on +Capability to configure the app group.

Choose App Groups from the list.

After that, select or add the generated App Group ID that you have created before.

Step 8: Add Capability in Extension

Go to Your-Extension-Name > Signing & Capabilities and configure App Group functionality which we had perform in previous steps. (Group id should be same for both targets).

Step 9: Add App Group Id in Extension File

Go to the extension's SampleHandler.swift file and paste your group ID in the appGroupIdentifier constant.

Step 10: Update App level info.plist file

Add a new key, RTCScreenSharingExtension in Info.plist with the extension's Bundle Identifier as the value.
Add a new key RTCAppGroupIdentifier in Info.plist with the extension's App groups Id as the value.

Note : For the extension's Bundle Identifier, go to TARGETS > Your-Extension-Name > Signing & Capabilities.

NOTE:

You can also check out the extension's example code on Github.

Integrate ScreenShare in your App

After successfully creating Broadcast Upload Extension using the above-listed steps, we can start using the enableScreenShare and disableScreenShare functions of the Meeting class.

How to use the ScreenShare functions

Use these functions in your app's Meeting Screen.

@IBAction func ScreenShareButtonTapped(_ sender: Any) {
    Task {
      self.meeting?.enableScreenShare()
    }
}

@IBAction func StopScreenShareButtonTapped(_ sender: Any) {
    Task {
      self.meeting?.disableScreenShare()
    }
}

CAUTION :

The function enableScreenShare and disableScreenShare are async functions; therefore use the above syntax to call the ScreenShare functions.

Calling the enableScreenShare() will prompt a RPBroadcastPickerView with the extension that was created using the above steps.

After clicking the Start Broadcast button, you will be able to get the screen share stream in the session.

When the broadcast is started, it creates a Stream that has MediaStream.kind = .share. Using the stream kind, you can prompt a ScreenShare view for remote peers when ScreenShare is started by the local peer.
Similarly, you can use the same kind to dismiss the ScreenShare view on the remote peer when the ScreenShare is stopped.

extension MeetingViewController: ParticipantEventListener {
    /// Participant has enabled mic, video or screenshare
    /// - Parameters:
    /// - stream: enabled stream object
    /// - participant: participant object
    func onStreamEnabled(_ stream: MediaStream, forParticipant participant: Participant) {

        if stream.kind == .share {
        // show screen share
            showScreenSharingView(true)
            screenSharingView.showMediastream(stream)
            return
        }

        // show stream in cell
        if let cell = self.cellForParticipant(participant) {
            cell.updateView(forStream: stream, enabled: true)
        }

        if participant.isLocal {
        // turn on controls for local participant
            self.buttonControlsView.updateButtons(forStream: stream, enabled: true)
        }
    }

    /// Participant has disabled mic, video or screenshare
    /// - Parameters:
    /// - stream: disabled stream object
    /// - participant: participant object
    func onStreamDisabled(_ stream: MediaStream, forParticipant participant: Participant) {

        if stream.kind == .share {
        // remove screen share
            showScreenSharingView(false)
            screenSharingView.hideMediastream(stream)
            return
        }

        // hide stream in cell
        if let cell = self.cellForParticipant(participant) {
            cell.updateView(forStream: stream, enabled: false)
        }

        if participant.isLocal {
        // turn off controls for local participant
            self.buttonControlsView.updateButtons(forStream: stream, enabled: false)
        }
    }
}

🔚 Conclusion

Integrating screen sharing into your iOS video call app with VideoSDK is a straightforward process that unlocks a powerful new feature for your users. This functionality can streamline communication, boost productivity, and open doors for innovative use cases within your app.

Unlock the power of seamless video communication with VideoSDK! Sign up now and dive into an extraordinary world of interactive video calling experiences.

With VideoSDK, you get a generous 10,000 free minutes to kickstart your journey towards creating engaging and immersive connections. Whether you're building a social app, a collaboration tool, or an e-learning platform, our platform provides the tools you need to elevate your user experience to the next level.

DEV Community: Video SDK

Build AI Voice Agent with Gemini 3.1 Flash Live and VideoSDK

What's New in Gemini 3.1 Flash Live Preview

Here's what stands out:

Building a Voice Agent with VideoSDK

Step 1 : Create and Activate a Python Virtual Environment

Step 2 : Set Up Your Environment Variables

Step 3 : Install the Required Packages

Step 4 : Create Your Agent (main.py)

To run the agent:

What Can You Build With This?

Conclusion

Next Steps and Resources

Announcing VideoSDK Inference: One Magic API for Every Voice AI Model

Supported Categories

What is VideoSDK Inference?

Inferencing via Agent Runtime Dashboard

Inferencing via Code (Agents SDK)

CascadingPipeline Example

Conclusion

Resources and Next Steps

Introducing Phone Numbers: Build AI Telephony Agents in 60 seconds

Fewer Hops. Better Calls.

How It Works

What You Can Build

Resources

Build Real-time transcription app using React Hooks

Step-by-Step Tutorial for the Code

Prerequisites

Step 1: Setup the Project

Step 2: Setup API Functions

Step 3: Build the Components

JoinScreen Component

ParticipantView Component

Controls Component

MeetingView Component

App Component

Step 4: Run the Application

Optional Step: Add CSS for Styling

Summary

How to Integrate Image Capture in JavaScript Video Chat App?

📌 Introduction

🚀 Getting Started with VideoSDK

Create a VideoSDK Account

Generate your Auth Token

Prerequisites

🛠️ Install VideoSDK

Structure of the project

🎥 Essential Steps to Implement Video Call Functionality

Step 1: Design the user interface (UI)​

Step 2: Implement Join Screen​

Step 3: Initialize Meeting​

Step 4: Create the Media Elements​

Step 5: Handle participant events​

Step 6: Implement Controls​

Integrate Image Capture Feature

How to capture an image of a remote participant?​

Step 1: Initiate Image Capture Request​

Step 2: Capture and Upload File​

Step 2: Fetch and Display Image​

🔚 Conclusion

How to Implement Active Speaker Indication in React JS Video Call App?

📌 Introduction

🚀 Getting Started with VideoSDK

Create a VideoSDK Account

Generate your Auth Token

Prerequisites and Setup

🛠️ Install VideoSDK​

App Architecture

🎥 Essential Steps to Implement Video Calling Functionality

Step 1: Get started with API.js​

Step 2: Wireframe App.js with all the components​

Step 3: Implement Join Screen

Output

Step 4: Implement MeetingView and Controls​

Output of Controls Component

Step 5: Implement Participant View​

5.1 Forwarding Ref for mic and camera

5.2 useParticipant Hook

5.3 MediaStream API

Step 1: Design the user interface (UI)

Step 2: Implement Join Screen

Step 3: Initialize Meeting

Step 4: Create the Media Elements

Step 5: Handle participant events

Step 6: Implement Controls

How to capture an image of a remote participant?

Step 1: Initiate Image Capture Request

Step 2: Capture and Upload File

Step 2: Fetch and Display Image

🛠️ Install VideoSDK

Step 1: Get started with API.js

Step 2: Wireframe App.js with all the components

Step 4: Implement MeetingView and Controls

Step 5: Implement Participant View

5.4 Implement `ParticipantView`

Step 1: Design the user interface (UI)

Step 2: Implement Join Screen

Step 3: Initialize Meeting

Step 4: Create the Media Elements

Step 5: Handle participant events

Step 6: Implement Controls

Rendering Screen Share and Screen Share Audio

🛠️ Integrate VideoSDK

Create models

Step 1: Get started with APIClient

Step 2: Implement Join Screen

Step 3: Initialize and Join Meeting

Step 4: Implement Controls

Step 5: Implementing `MeetingEventListener`

Step 6: Implementing `ParticipantEventListener`

Known Issue

🛠️ Install VideoSDK

Step 1: Get started with API.js

Step 2: Wireframe App.js with all the components

Step 3: Implement Join Screen

Step 4: Implement MeetingView and Controls

Step 5: Implement Participant View

5.4 Implement `ParticipantView`

Event associated with Livestream

Custom Template