Screen Sharing with WebRTC: Harnessing JavaScript for Seamless Streaming

Filipe Melo — Fri, 15 Mar 2024 20:22:42 +0000

Introduction

WebRTC is a technology that allows web applications to exchange data directly with other browsers, "without" an intermediary. It uses a variety of protocols that work together to achieve this.
Unfortunately, webRTC cannot initiate a connection on its own, so we need a server that will transmit the connection Offer between the clients: we call it Signal Channel or Signaling Server; once the Offer is accepted, the browsers can then share information between themselves.
In this article, we are going to create a screen-sharing application, using webRTC and a Websocket server as our Signaling Server (which we are also going to build ourselves).

Creating the initial project

To keep this project simple, we are going to start a React + Typescript project using Vite:

npm create vite scrshr-app

For now, we are going to modify the App.tsx by adding a form with an input and two buttons. We are going to use this form to allow users to create/join a stream "room".

You can also delete some of the files it generated, such as App.css and index.css.

// App.tsx

import { useAppState } from "./app.state"

function App() {

  const {
    inputRef,
    onCreateRoom,
    onJoinRoom
  } = useAppState();
  return (
    <>
      <form>
        <input type='text' ref={inputRef}/>
        <button type="button" onClick={onJoinRoom}>Join</button>
        <button type="button" onClick={onCreateRoom}>Create</button>
      </form>
      <video width="320" height="240"/>
    </>
  )
}

export default App

// app.state.ts

import { useRef } from "react"

export function useAppState() {

  const inputRef = useRef<HTMLInputElement>(null);

  const onCreateRoom = () => {
    console.log("Create room")
  }

  const onJoinRoom = () => {
    console.log("Join room")
  }

  return {
    onCreateRoom,
    onJoinRoom,
    inputRef
  }
}

Now that we have the UI components on the screen, we can start building the logic for transmitting the webRTC offers with WebSocket. So before we continue with the code, we need to install the Socket.io:

npm i socket.io-client

Handling the Websocket Events (Client)
Let's go ahead now and create a new file called websocket.service.ts. In this file, we are going to implement our business rule related to the websocket connection. There, we will add listeners for the events, and also, trigger our events to the server. Below, is a list of all the events we will be triggering/listening to, and their purpose:

createRoom: this event will be responsible for creating a "room" in the backend, so we can listen to it when a new user joins the room;
joinRoom: this event will be triggered when the user clicks on the "join room" button;
onNewUserJoined: this event will tell the room's owner that a user has joined, therefore, it will be able to send an offer;
sendOffer: this event will be triggered once a new user joins; we will send the webRTC Offer object to the new user;
receiveOffer: this event will bring the offer to the user who's just joined, that way they can trigger the sendAnswer method explained below;
sendAnswer: this event will be triggered by the user who's received an Offer, allowing them to respond with their webRFC Answer object;
receiveAnswer: this event will be triggered once the user who joined has responded, allowing them to finally communicate.

It is important to note that there are some more events we will need to implement, but we will get into more details further.

import { Socket, io } from "socket.io-client";

export class WebsocketService {
  websocket: Socket;
  constructor() {
    this.websocket = io("http://localhost:3000");
  }

  createRoom(roomName: string) {
    this.websocket.emit("room/create", {
      roomName,
    });
  }

  joinRoom(roomName: string) {
    this.websocket.emit("room/join", {
      roomName,
    });
  }

  onNewUserJoined(callback: () => void) {
    this.websocket.on("user/joined", callback);
  }

  sendOffer(roomName: string, offer: RTCSessionDescriptionInit) {
    this.websocket.emit("offer/send", {
      offer,
      roomName,
    });
  }

  receiveOffer(callback: (offer: RTCSessionDescriptionInit) => void) {
    this.websocket.on("offer/receive", ({ offer }) => callback(offer));
  }

  sendAnswer(roomName: string, answer: RTCSessionDescriptionInit) {
    this.websocket.emit("answer/send", {
      answer,
      roomName,
    });
  }

  receiveAnswer(callback: (answer: RTCSessionDescriptionInit) => void) {
    this.websocket.on("answer/receive", ({ answer }) => callback(answer));
  }
}

Creating the Signaling Server

We can get started creating the Signaling Server. Let's create a new directory called server and run the following commands:

npm init -y

This will generate for us the package.json.
Since we are using typescript on the Front-End, I will add some extra steps here if you want to follow along, but this is entirely optional.

npm install typescript ts-node-dev -D

Now we can initiate the typescript by running the following command:

npx tsc --init

I made a few adjustments to the tsconfig.json and package.json files, here is what they look like:

{
  "compilerOptions": {
    "target": "es2020",
    "module": "commonjs",
    "rootDir": "./src",
    "outDir": "./build",
    "esModuleInterop": true,
    "forceConsistentCasingInFileNames": true,
    "strict": true,
    "skipLibCheck": true
  }
}

{
  "name": "server",
  "version": "1.0.0",
  "description": "",
  "main": "./src/server.ts",
  "scripts": {
    "start": "node ./build/server.js",
    "dev": "ts-node-dev .",
    "build": "tsc "
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "ts-node-dev": "^2.0.0",
    "typescript": "^5.4.2"
  }
}

Now we can install both Express and Socket.io libraries:

npm install express socket.io
npm install @types/express -D

To make this quick, we can follow the steps to setup socket.io along with express described here:

//server.ts
import express from "express";
import { createServer } from "node:http";
import { Server } from "socket.io";

const app = express();
const httpServer = createServer(app);

const io = new Server(httpServer, {
  cors: {
    origin: "http://localhost:5173",
  },
});

io.on("connection", () => {
  console.log("connected");
});

httpServer.listen(3000);

Finally, we can create a new file and place where the event listeners to handle the WebSocket requests coming from the clients, as we saw earlier.

//room.service.ts

import { Server } from "socket.io";

interface Room {
  owner: string;
  guest: string | null;
};

export class RoomService {
  rooms = new Map<string, Room>();

  constructor(private server: Server) {
  }

  initialize() {
    this.server.on("connection", (socket) => {
      console.log("user connected", socket.id)
    })
  }

}

And make sure to instantiate this service on the server.ts file as shown below:

//server.ts
import express from "express";
import { createServer } from "node:http";
import { Server } from "socket.io";
import { RoomService } from "./room.service";

const app = express();
const httpServer = createServer(app);

const io = new Server(httpServer, {
  cors: {
    origin: "http://localhost:5173",
  },
});

const roomService = new RoomService(io);
roomService.initialize();

httpServer.listen(3000);

To give this all a try, let's make a few adjustments on the Front-End client.

// app.state.ts
...
const socketService = new WebsocketService();

export function useAppState() {
  ...
  const onCreateRoom = () => {
    if (!inputRef.current?.value) {
      return;
    }
    socketService.createRoom(inputRef.current.value)
  }
  ...
}

As you can see above, we have the server printing that our client has connected to the websocket successfully. We can now implement each handler to handle each event, as well as emit events when necessary.

Handling the Websocket Events (Server)

Let's go ahead now and implement all the logic we need on the server so it can exchange information.

// room.service.ts

import { Server, Socket } from "socket.io";

interface Room {
  owner: string;
  guest: string | null;
}

export class RoomService {
  rooms = new Map<string, Room>();

  constructor(private server: Server) {}

  initialize() {
    this.server.on("connection", (socket) => {
      this.handleOnCreateRoom(socket);
      this.handleOnJoinRoom(socket);
      this.handleOnSendOffer(socket);
      this.handleOnSendAnswer(socket);
    });
  }

  handleOnCreateRoom(socket: Socket) {
    socket.on("room/create", (payload) => {
      const { roomName } = payload;
      this.rooms.set(roomName, {
        owner: socket.id,
        guest: null,
      });
    });
  }

  handleOnJoinRoom(socket: Socket) {
    socket.on("room/join", (payload) => {
      const {roomName} = payload;

      const room = this.rooms.get(roomName);

      if (!room) {
        return;
      }
      room.guest = socket.id;
      socket.to(room.owner).emit("user/joined");
    })
  }

  handleOnSendOffer(socket: Socket) {
    socket.on("offer/send", (payload) => {
      const { roomName, offer } = payload;

      const room = this.rooms.get(roomName);

      if (!room || !room.guest) {
        return;
      }
      socket.to(room.guest).emit("offer/receive", {
        offer
      })
    })
  }

  handleOnSendAnswer(socket: Socket) {
    socket.on("answer/send", (payload) => {
      const { answer, roomName } = payload;

      const room = this.rooms.get(roomName);

      if (!room) {
        return;
      }

      socket.to(room.owner).emit("answer/receive", {
        answer
      })
    })
  }
}

Implementing the WebRTC service
Now, let's get started setting up our webRTC service. Firstly, we start by setting up the RTC Peer Connection Object:

// webrtc.service.ts
export class WebRTCService {
  private peerConnection: RTCPeerConnection;
  constructor() {
    const configuration: RTCConfiguration = {
      iceServers: [{ urls: "stun:stun.l.google.com:19302" }],
    };
    this.peerConnection = new RTCPeerConnection(configuration);
  }
  ...
}

Then, we are going to implement the following methods for now:

makeOffer: this method will be responsible for creating the "offer", which will contain all the information about the current connection, including its media streams;
makeAnswer: this method, similar to make offer, creates an answer for an offer;
setRemoteOffer: once we receive either an answer or offer from a remote client, we pass it down to the setRemoteDescription;
setLocalOffer: once we make an offer or an answer, we need to specify the properties of the local end of the connection;
getMediaStream: using the MediaDevices API, we can capture the contents of a display (or specific app), and stream it over the webRTC connection;
onStream: using the ontrack event, we can receive the stream coming over the webRTC stream.

// webrtc.service.ts
export class WebRTCService {
  private peerConnection: RTCPeerConnection;

  constructor() {
    const configuration: RTCConfiguration = {
      iceServers: [{ urls: "stun:stun.l.google.com:19302" }],
    };
    this.peerConnection = new RTCPeerConnection(configuration);
  }

  async makeOffer() {
    const offer = await this.peerConnection.createOffer();
    return offer;
  }

  async makeAnswer() {
    const answer = await this.peerConnection.createAnswer();
    return answer;
  }

  async setRemoteOffer(offer: RTCSessionDescriptionInit) {
    await this.peerConnection.setRemoteDescription(new RTCSessionDescription(offer));
  }

  async setLocalOffer(offer: RTCSessionDescriptionInit) {
    await this.peerConnection.setLocalDescription(offer);
  }

  async getMediaStream() {
    const mediaStream = await navigator.mediaDevices.getDisplayMedia({
      video: true,
    });
    mediaStream.getTracks().forEach((track) => {
      this.peerConnection.addTrack(track, mediaStream);
    });
    return mediaStream;
  }

  onStream(cb: (media: MediaStream) => void ) {
    this.peerConnection.ontrack = function ({ streams: [stream] }) {

      cb(stream);
    };
  }
}

Let's move back to the app.state.ts file and implement the logic there. We are going to start from the perspective of a user creating a room.

// app.state.ts
...
const webRTCService = new WebRTCService();

export function useAppState() {
  ...
  const onCreateRoom = () => {
    ...
    const roomName = inputRef.current.value
    socketService.createRoom(roomName)
    socketService.onNewUserJoined(async () => {
      const offer = await webRTCService.makeOffer();
      webRTCService.setLocalOffer(offer);
      socketService.sendOffer(roomName, offer);
    })

    socketService.receiveAnswer(async (answer) => {
      await webRTCService.setRemoteOffer(answer);
    })
  }
 ...
}

As you can see above, once the user creates a room, we start listening to the websocket events. As soon as another user joins the session, we will be making the offer to send it over the websocket.
Finally, let's implement the business rule for when the user joins a session:

// app.state.ts
...
export function useAppState() {
...  
  const onJoinRoom = () => {
    if (!inputRef.current?.value) {
      return;
    }

    const roomName = inputRef.current.value

    socketService.joinRoom(roomName);
    socketService.receiveOffer(async (offer) => {
      await webRTCService.setRemoteOffer(offer);
      const answer = await webRTCService.makeAnswer();
      await webRTCService.setLocalOffer(answer);
      socketService.sendAnswer(roomName, answer);
    })
  }

...
}

Since now the connection is being established, it is time we finally add the stream track to the connection. We are going to do this when the user creates the room, by calling the getMediaStream method. We are also, going to use its return so we can show our stream locally as well, using the video tag we added earlier.

// app.state.ts

...
export function useAppState() {
  ...
  const videoRef = useRef<HTMLVideoElement>(null);

  const onCreateRoom = async () => {
    if (!inputRef.current?.value) {
      return;
    }

    const roomName = inputRef.current.value;
    await startLocalStream();
    socketService.createRoom(roomName)
    ...
  }

  ...

  async function startLocalStream () {
    const mediaStream = await webRTCService.getMediaStream();
    startStream(mediaStream, true);
  }

  function startStream(mediaStream: MediaStream, isLocal = false ) {
    if (!videoRef.current) {
      return;
    }
    videoRef.current.srcObject = mediaStream;
    videoRef.current.muted = isLocal;
    videoRef.current.play();
  }

  return {
    ...
    videoRef
  }
}

// App.tsx
...

function App() {

  const {
    ...
    videoRef
  } = useAppState();
  return (
    <>
      ...
      <video width="500" ref={videoRef}/>
    </>
  )
}

export default App

Lastly, we need to listen to a stream coming from the webRTC callback.

// app.state.ts

  const onJoinRoom = () => {
    ...
    webRTCService.onStream(startStream)
  }

Fixing the latency problem

When starting a WebRTC peer connection, typically a number of candidates are proposed by each end of the connection, until they mutually agree upon one which describes the connection they decide will be best. WebRTC then uses that candidate's details to initiate the connection.

As you may have noticed, we don't get any errors when connecting, but we still cannot see the stream on the other client. This is happening because we haven't implemented the logic that will share the RTCIceCandidate. We are going to create a few more socket events so both clients can finally exchange information and connect much more quickly.
Firstly, let's implement the new methods in our webRTC events so we can listen to them when they change:

// webrtc.service.ts

export class WebRTCService {
 ...
  onICECandidateChange(cb: (agr: RTCIceCandidate ) => void ) {
    this.peerConnection.onicecandidate = (event: RTCPeerConnectionIceEvent) => {
      if (event.candidate) {
        cb(event.candidate);
      }
    }
  }

  async setICECandidate(candidate: RTCIceCandidate) {
    await this.peerConnection.addIceCandidate(candidate);
  }
}

Now we are going to implement two new socket events: ice/send and ice/receive:

// websocket.service.ts

export class WebsocketService {
  ...
  sendIceCandidate(roomName: string, iceCandidate: RTCIceCandidate) {
    this.websocket.emit("ice/send", {
      roomName,
      iceCandidate
    })
  }

  receiveIceCandidate(cb: (arg: RTCIceCandidate) => void ) {
    this.websocket.on("ice/receive", ({iceCandidate}) => {
      cb(iceCandidate);
    })
  }
}

On the backend, since either the owner or guest could send their ICECandidates; we can add a very simple verification to send the candidate to the appropriate client.

// room.service.ts
...
export class RoomService {
  ...
  initialize() {
    this.server.on("connection", (socket) => {
       ...
      this.handleOnSendIceCandidate(socket);
    });
  }

  ...

  handleOnSendIceCandidate(socket: Socket) {
    socket.on("ice/send", (payload) => {
      const { roomName, iceCandidate } = payload;

      const room = this.rooms.get(roomName);

      if (!room || !room.guest) {
        return;
      }

      const isOwner = room.owner === socket.id;
      const to = isOwner ? room.guest : room.owner;

      socket.to(to).emit("ice/receive", {
        iceCandidate,
      });
    });
  }
}

Finally, we can listen to the events when the user creates or joins:

// app.state.ts

...

  const onCreateRoom = async () => {
    ...
    setupIceCandidate(roomName)
  }

  const onJoinRoom = () => {
    ...
    setupIceCandidate(roomName)
  }

  function setupIceCandidate (roomName: string) {
    webRTCService.onICECandidateChange((candidate) => {
      socketService.sendIceCandidate(roomName, candidate)
    })
    socketService.receiveIceCandidate(webRTCService.setICECandidate)
  }
...

DEV Community: Filipe Melo

Screen Sharing with WebRTC: Harnessing JavaScript for Seamless Streaming