DEV Community: Kofi Mupati

How to Implement Partial Screenshare

Kofi Mupati — Thu, 09 Nov 2023 17:33:47 +0000

The rise of video call applications in many domains also comes with some privacy considerations. A typical scenario worth looking at is screen sharing.

How do we prevent mistakenly sharing sensitive material during screen sharing in Telehealth, E-learning, and many other use cases?

Zoom provides an advanced screen-sharing functionality that doesn’t come with the regular browser screen-share dialog.

I will walk you through how to implement similar functionality which can be later integrated into your main Video Call application or solution.

Click here to see the full source code.

Selecting the Screen to Share

We use the browser media devices API to bring up the screen selection dialog. After this, we handle the selection of a portion of interest using CropperJs.

    async function startCapture() {
        try {
          captureStream = await navigator.mediaDevices.getDisplayMedia();

          const imageUrl = await getImageFromVideoStream(captureStream);
          const imgElem = document.getElementById("previewImage");
          imgElem.style.display = "block";
          imgElem.src = imageUrl;

          const videoDisplay = document.getElementById("screenshareDisplay");
          videoDisplay.style.display = "block";
          new Cropper(imgElem, {
            zoomable: false,
            restore: false,
            crop(event) {
              coordinates.x = event.detail.x;
              coordinates.y = event.detail.y;
              coordinates.height = event.detail.height;
              coordinates.width = event.detail.width;

              const generatedStream = processVideoTrack(captureStream);
              videoDisplay.srcObject = generatedStream;
            },
          });
        } catch (err) {
          console.error(`Error: ${err}`);
        }
      }

Getting the Coordinates of the Area of Interest

To get the portion we want to share, we grab an image from the screen we selected and pass it to the Cropper Library. This way we can get the coordinates that reflect the selected screen.

      async function getImageFromVideoStream(stream) {
        const canvas = document.createElement("canvas");
        if ("ImageCapture" in window) {
          const videoTrack = stream.getVideoTracks()[0];
          const imageCapture = new window.ImageCapture(videoTrack);
          const bitmap = await imageCapture.grabFrame();
          canvas.height = bitmap.height;
          canvas.width = bitmap.width;
          canvas.getContext("2d").drawImage(bitmap, 0, 0);
          return canvas.toDataURL();
        }
        const video = document.createElement("video");
        video.srcObject = stream;
        return new Promise((resolve, reject) => {
          video.addEventListener("loadeddata", async () => {
            const { videoWidth, videoHeight } = video;
            canvas.width = videoWidth;
            canvas.height = videoHeight;

            try {
              await video.play();
              canvas
                .getContext("2d")
                .drawImage(video, 0, 0, videoWidth, videoHeight);
              return resolve(canvas.toDataURL());
            } catch (error) {
              return reject(error);
            }
          });
        });
      }

The image is then passed to the Cropper instance. At this point, we display the box for cropping the area of interest on the image. Dragging the box updates the coordinates of the selected area which we use to crop the video stream from the screen share, frame by frame.

new Cropper(imgElem, {

    zoomable: false,

    restore: false,

    crop(event) {

        coordinates.x = event.detail.x;

        coordinates.y = event.detail.y;

        coordinates.height = event.detail.height;

        coordinates.width = event.detail.width;

     const generatedStream = processVideoTrack(captureStream);
     videoDisplay.srcObject = generatedStream;
      },
  });

Cropping the Video track from the Screenshare

This is where we see the insertable stream at play. We modify each frame by cropping to the selected screen portion coordinates we get from the cropper.

// Cropping each frame

function cropVideoFramesWithCoordinates(frame, controller) {

        const newFrame = new window.VideoFrame(frame, {

          visibleRect: {

            x: getSampleAlignedCoordinates(Math.round(coordinates.x)),

            y: getSampleAlignedCoordinates(Math.round(coordinates.y)),

            width: getSampleAlignedCoordinates(Math.round(coordinates.width)),

            height: getSampleAlignedCoordinates(Math.round(coordinates.height)),

          },

        });

        controller.enqueue(newFrame);

        frame.close();

      }

// Insertable stream logic

  function processVideoTrack(track) {
    const mainTrack = track.getVideoTracks()[0] ?? track;
    const generator = new window.MediaStreamTrackGenerator({
      kind: "video",
    });
    const generatedStream = new window.MediaStream([generator]);
    const processor = new window.MediaStreamTrackProcessor({
      track: mainTrack,
    });

    processor.readable
      .pipeThrough(
        new window.TransformStream({
          transform: cropVideoFramesWithCoordinates,
        })
      )
      .pipeTo(generator.writable)
      .catch((err) =&gt; {
        // TODO: Figure out how to prevent this error
        console.log("pipe error: ", { err });
      });
    return generatedStream;
  }

Demo

Conclusion

This shows you the general approach to achieving partial screen share as we have in Zoom. You can always improve the UX to suit your use case.

In subsequent articles, I’ll demonstrate how this can be integrated into existing video call platforms.

How to Set up Cascaded Jitsi Videobridges

Kofi Mupati — Wed, 01 Nov 2023 21:56:12 +0000

INTRODUCTION

After self-hosting a Jitsi server, one will soon be required to add multiple videobridges to support the load.
I take you through a manual step in adding multiple videobridges. The understanding of this can help you write your ansible playbook for automating this process.

This article assumes the following:

Ability to create and configure droplets on DigitalOcean.
Set up a Jitsi server by following the self-hosting guide.
Configured a subdomain for hosting your Jitsi server. I’ll use meet.kofimupati.com (At the time of reading the article I would have shut down the servers)

We’ll create and configure 2 additional video bridge servers:
jvb1 and jvb2 which will then be connected to our existing Jitsi server.

UPDATING JITSI SERVER CONFIGURATION

Modify the Videobridge Connected to the Jitsi Server
“Jitsi Videobridge is a WebRTC compatible video router or SFU that lets build highly scalable video conferencing infrastructure (i.e., up to hundreds of conferences per server).”

Allow the following ports as well:

9090/tcp
9090/udp
5222/tcp
5222/udp

Update the config: /etc/jitsi/videobridge/config

-- JVB_HOST=
++ JVB_HOST=meet.kofimupati.com
++ JVB_OPTS="--apis=rest,xmpp --subdomain=jitsi-videobridge"
++ AUTHBIND=yes

Update jvb.conf: /etc/jitsi/videobridge/jvb.conf

The xmpp-client.configs.xmpp-server-1 values can be found in the default /etc/jitsi/videobridge/sip-communicator.properties

The region and relay-id under relay are defined to distinguish each video bridge in the cascaded videobridge setup.

videobridge {
  stats {
    enabled = true
    transports = [
      { type = "muc" }
    ]
  }

  apis {
    rest {
      enabled = true
    }
    xmpp-client {
      configs {
        xmpp-server-1 {
        hostname="meet.kofimupati.com"
        domain = "auth.meet.kofimupati.com"
        username = "jvb"
        password = "18YlmYSH"
        muc_jids = "JvbBrewery@internal.auth.kofimupati.com"
        muc_nickname = "08a74380-a3d3-4763-adf4-545c55743b3e"
      }
    }
   }
  }

  cc {
    max-time-between-calculations = 5 seconds
    bwe-change-threshold = 0.1
    padding-period = 10ms
    jvb-last-n = -1
    trust-bwe = false
  }

  http-servers {
    public {
    port = 9090
   }
 }

  websockets {
    enabled = true
    domain = "meet.kofimupati.com:443"
    tls = true
  }

  relay {
    enabled=true
    region="region1"
    relay-id="jitsi-videobridge"
  }

}

Note: The use of sip-communicator.properties is currently deprecated.
Update the sip-communicator.properties: /etc/jitsi/videobridge/sip-communicator.properties

org.ice4j.ice.harvest.DISABLE_AWS_HARVESTER=true
org.ice4j.ice.harvest.STUN_MAPPING_HARVESTER_ADDRESSES=meet-jit-si-turnrelay.jitsi.net:443

++org.ice4j.ice.harvest.NAT_HARVESTER_LOCAL_ADDRESS=10.106.0.3 #Private IP of the Server
++org.ice4j.ice.harvest.NAT_HARVESTER_PUBLIC_ADDRESS=178.62.18.4. #Public IP of the Server

org.jitsi.videobridge.ENABLE_STATISTICS=true

--org.jitsi.videobridge.STATISTICS_TRANSPORT=muc
++org.jitsi.videobridge.STATISTICS_TRANSPORT=muc,colibri

++org.jitsi.videobridge.STATISTICS_INTERVAL=2000

--org.jitsi.videobridge.xmpp.user.shard.HOSTNAME=localhost
++org.jitsi.videobridge.xmpp.user.shard.HOSTNAME=meet.kofimupati.com

org.jitsi.videobridge.xmpp.user.shard.DOMAIN=auth.meet.kofimupati.com
org.jitsi.videobridge.xmpp.user.shard.USERNAME=jvb
org.jitsi.videobridge.xmpp.user.shard.PASSWORD=18YlmYSH
org.jitsi.videobridge.xmpp.user.shard.MUC_JIDS=JvbBrewery@internal.auth.meet.virtualcp.app
org.jitsi.videobridge.xmpp.user.shard.MUC_NICKNAME=be350741-2b3d-4829-95ee-a60c4fbae28b

++org.jitsi.videobridge.SINGLE_PORT_HARVESTER_PORT=10000
++org.jitsi.videobridge.TCP_HARVESTER_PORT=443
++org.jitsi.videobridge.octo.BIND_ADDRESS=10.106.0.3 #Private IP of the Server
++org.jitsi.videobridge.octo.PUBLIC_ADDRESS=178.62.18.4 #Public IP of the Server
++org.jitsi.videobridge.octo.BIND_PORT=4096
+++org.jitsi.videobridge.REGION=region1

**Modify the Jicofo Configuration on the Jitsi Server
**It is responsible for managing media sessions between each of the participants and the video bridge. Link to repository

Update the config: /etc/jitsi/jicofo/config

--JICOFO_HOST=localhost
++JICOFO_HOST=meet.kofimupati.com

Update the jicofo.conf: /etc/jitsi/jicofo/jicofo.conf

selection-strategy: This is used to define how “participants” on a call are assigned to videobridges.
octo: Enabling octo is what causes Jicofo to assign “participants” in a single conference across multiple videobridges.

jicofo {
  xmpp: {
    client: {
    client-proxy: focus.meet.kofimupati.com
   }  
    trusted-domains: [ "recorder.meet.kofimupati.com" ]
  }

  bridge {
    brewery-jid: "JvbBrewery@internal.auth.meet.kofimupati.com"
    selection-strategy = SplitBridgeSelectionStrategy
  }

  octo {
    enabled = true
    id = "1"
  }
}

Modify the Jicofo Configuration on the Jitsi Server

Configure the app-config.js i.e **/etc/jitsi/meet/meet.kofimupati.com-config.js **to enable WebSocket and any configuration needed for your use case.
This might not be necessary for you.

--// websocket: 'wss://meet.kofimupati.com/' + subdir + 'xmpp-websocket',
++websocket: 'wss://meet.kofimupati.com/' + subdir + 'xmpp-websocket',

// Add the following line just before the section which begins with the comment // UI
openBridgeChannel: 'websocket',

Modify the Prosody Configuration on the Jitsi Server

Prosody is the XMPP component of Jitsi that handles the messaging.
We also enable communication via WebSocket instead of BOSH which happens to be the default on a fresh installation.

Enable ports and allow all IPs to reach the prosody server by adding the following to /etc/prosody/prosody.cfg.lua

admin = {}
component_ports = { 5347 }
component_interface = "0.0.0.0"

--use_libevent = true
use_libevent = true

-- Uncomment websocket under HTTP modules
--"websocket"
"websocket"

-- Comment out VirtualHost "localhost" under the Virtual hosts section

VirtualHost "localhost"
--VirtualHost "localhost"

Configure the domain-specific prosody settings by modifying /etc/prosody/conf.d/meet.virtualcp.app.cfg.lua

-- Add the following to allow the use of websocket with the low-level library.
cross_domain_bosh = false;
consider_bosh_secure = true;
cross_domain_websocket = true;
consider_websocket_secure = true;


-- Add smacks and websocket to modules enabled and configurations for smacks
modules_enables = {
  "smacks";
  "websocket";
}

smacks_max_unacked_stanzas = 5;
smacks_hibernation_time = 60;
smacks_max_hibernated_sessions = 1;
smacks_max_old_sessions = 1;

**Modify the Nginx Configuration on the Jitsi Server
**Add the following to enable WebSocket connection between external video bridges which will be configured later. The following nginx location blocks are needed for the cascaded videobridges to work.

Add them after the location ~ ^/colibri-ws/default-id/(.*) block.

location ~ ^/colibri-ws/([0-9.]*)/(.*) {
  proxy_pass http://$1:9090/colibri-ws/$1/$2$is_args$args;
  proxy_http_version 1.1;
  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection "upgrade";
  tcp_nodelay on;
}

# colibri secure-octo relay websockets for jvb1

location ~ ^/colibri-relay-ws/default-id/(.*) {
  proxy_pass http://jvb1/colibri-relay-ws/default-id/$1$is_args$args;
  proxy_http_version 1.1;
  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection "upgrade";
  tcp_nodelay on;
}


location ~ ^/colibri-relay-ws/([0-9.]*)/(.*) {
  proxy_pass http://$1:9090/colibri-relay-ws/$1/$2$is_args$args;
  proxy_http_version 1.1;
  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection "upgrade";
  proxy_set_header Host meet.virtualcp.app;
  tcp_nodelay on;
}

CONFIGURING EXTERNAL VIDEO BRIDGES

Install just the jitsi-videobridge2 component on each of the external JVBs.

add jitsi package repository

curl -sL https://download.jitsi.org/jitsi-key.gpg.key | sudo sh -c 'gpg --dearmor > /usr/share/keyrings/jitsi-keyring.gpg'

echo "deb [signed-by=/usr/share/keyrings/jitsi-keyring.gpg] https://download.jitsi.org stable/" | sudo tee /etc/apt/sources.list.d/jitsi-stable.list

sudo apt install jitsi-videobridge2

During installation enter the domain of the Jitsi Server in the prompt that shows up i.e. meet.kofimupati.com in my case.
Allow access to the following ports on the firewall.

    ufw allow 9090/tcp
    ufw allow 9090/udp
    ufw allow 5347/tcp
    ufw allow 5347/udp
    ufw allow 5222/tcp
    ufw allow 5222/udp
    ufw allow 10000/tcp
    ufw allow 10000/udp
    ufw allow 4096/tcp
    ufw allow 4096/udp
    ufw enable

Configure the VideoBridge Replace jvb.conf, sip-communicator.properties, and **config **on the external video bridge with what was configured on the Jitsi Server.

Using jvb1 as an example:

Changes to **/etc/jitsi/videobridge/config
*JVB_SECRET* should have the same value for the main and external video bridges.

JVB_HOST=
JVB_SECRET=18YlmYSH
JVB_OPTS="--apis=rest,xmpp --subdomain=jvb1"

Changes to ***/etc/jitsi/videobridge/jvb.conf

*Add **disable_certificate_verification = true to **xmpp-client.configs.xmpp-server-1
Set the **muc_nickname with the value found in sip-communicator.properties of the current external videobridge (jvb1) being configured.
Set a different region and relay-id for each external videobridge.
Add a server-id option to the WebSocket block of only the external videobridges. This value should be the Public IP of the videobridge server.

    videobridge {
      stats {
        enabled = true
        transports = [
          { type = "muc" }
        ]
      }

      apis {
        rest {
          enabled = true
        }
        xmpp-client {
          configs {
            xmpp-server-1 {
            hostname="meet.kofimupati.com"
            domain = "auth.meet.kofimupati.com"
            username = "jvb"
            password = "18YlmYSH"
            muc_jids = "JvbBrewery@internal.auth.kofimupati.com"
            muc_nickname = "44643c18-e111-4bfc-9fcc-29f57bb415d8"
            disable_certificate_verification = true
          }
        }
       }
      }

      cc {
        max-time-between-calculations = 5 seconds
        bwe-change-threshold = 0.1
        padding-period = 10ms
        jvb-last-n = -1
        trust-bwe = false
      }

      http-servers {
        public {
        port = 9090
       }
     }

      websockets {
        server-id = "203.500.130.37"
        enabled = true
        domain = "meet.kofimupati.com:443"
        tls = true
      }

      relay {
        enabled=true
        region="region2"
        relay-id="jvb1"
      }

    }

Changes to /etc/jitsi/videobridge/sip-communicator.properties

--org.jitsi.videobridge.xmpp.user.shard
++org.jitsi.videobridge.xmpp.user.jvb1

org.ice4j.ice.harvest.DISABLE_AWS_HARVESTER=true
org.ice4j.ice.harvest.STUN_MAPPING_HARVESTER_ADDRESSES=meet-jit-si-turnrelay.jitsi.net:443

++org.ice4j.ice.harvest.NAT_HARVESTER_LOCAL_ADDRESS=10.106.0.3 #Private IP of the jvb1
++org.ice4j.ice.harvest.NAT_HARVESTER_PUBLIC_ADDRESS=178.62.18.4. #Public IP of the jvb1

org.jitsi.videobridge.ENABLE_STATISTICS=true

--org.jitsi.videobridge.STATISTICS_TRANSPORT=muc
++org.jitsi.videobridge.STATISTICS_TRANSPORT=muc,colibri

++org.jitsi.videobridge.STATISTICS_INTERVAL=2000

--org.jitsi.videobridge.xmpp.user.shard.HOSTNAME=localhost
++org.jitsi.videobridge.xmpp.user.shard.HOSTNAME=meet.kofimupati.com

org.jitsi.videobridge.xmpp.user.jvb1.DOMAIN=auth.meet.kofimupati.com
org.jitsi.videobridge.xmpp.user.jvb1.USERNAME=jvb
org.jitsi.videobridge.xmpp.user.jvb1.PASSWORD=18YlmYSH
org.jitsi.videobridge.xmpp.user.jvb1.MUC_JIDS=JvbBrewery@internal.auth.meet.virtualcp.app
org.jitsi.videobridge.xmpp.user.jvb1.MUC_NICKNAME=44643c18-e111-4bfc-9fcc-29f57bb415d8
++org.jitsi.videobridge.xmpp.user.jvb1.DISABLE_CERTIFICATE_VERIFICATION=true

++org.jitsi.videobridge.SINGLE_PORT_HARVESTER_PORT=10000
++org.jitsi.videobridge.TCP_HARVESTER_PORT=443
++org.jitsi.videobridge.octo.BIND_ADDRESS=10.106.0.3 #Private IP of the jvb1
++org.jitsi.videobridge.octo.PUBLIC_ADDRESS=178.62.18.4 #Public IP of the jvb1
++org.jitsi.videobridge.octo.BIND_PORT=4096
+++org.jitsi.videobridge.REGION=region2

Repeat the above external video bridge configuration steps for all the videobridges you want to use.

Conclusion

Jitsi is an open-source video conferencing solution that can serve most of your needs. You can self-host and use the **lib-meet-jitsi **to build whichever experience you want.

If you need help building your custom application on top of Jitsi, you know what to do 😉

Build a Scalable Video Chat App with Agora in Django

Kofi Mupati — Tue, 13 Apr 2021 18:01:27 +0000

Introduction

Django is a high-level Python Web framework that takes care of much of the hassle of Web development so that you can focus on writing your app without needing to reinvent the wheel. While Agora takes away the hassle of building a video chat application from scratch.

WebRTC is only one of the ways that people can implement video chat features. Companies like Agora also provide a fully packaged video chat SDK to provide a high-quality Real-Time Engagement video chat experience. As someone who has WebRTC development experience, I can tell you there are some limitations with WebRTC, such as:

Quality of experience: Since WebRTC is transmitted over the Internet, which is a public domain, the quality of experience is hard to guarantee.
Scalability: Scalability is fairly limited on group video calls due to the peer-to-peer nature of WebRTC.

After I was introduced to the Agora platform, I was impressed that setting up the same video call feature is easier with the Agora SDK than with WebRTC. I went ahead to build a video chat application with Agora and Laravel. In this article, however, I don't want Django developers to be left out so we are going to implement a video chat application with Django and Agora.

Why Agora Is the Preferred Solution

After building a video chat app with Agora, I want to highlight some of the advantages:

There's one SDK for everything - voice, video, live streaming, screen sharing, and so on.
I didn't have to set up a turn server with coturn on Amazon EC2 as I did in the other implementation to relay traffic between peers on different networks.
You get 10,000 minutes every month free, and this gives you the flexibility to develop your solution prototype for free.
You don't have the challenge of managing the underlying infrastructure supporting the video call functionality.
Intuitive API documentation is available.

Prerequisites

Python 3.8.5
An average knowledge about how to create a Django project and app. The following tutorial can help you: Writing your first Django app.
A free pusher account on pusher.com
An understanding of pusher presence channels and the python server library.
Agora Developer Account: (See How to get started with Agora)

Project Setup

Create and activate a python3 virtual environment for this project.
Open your terminal or command prompt and navigate to your Django project directory. We will use mysite as the project name for this tutorial.
Install the necessary packages from your terminal or command prompt.
```
    pip install pusher python-dotenv
```
Create a new app called agora. Run the following from your terminal.
```
   python manage.py startapp agora
```

In the project directory i.e mysite, run your migrations and create new super users by running the following command from your terminal.


    python manage.py migrate
// run the next command multiple times to create more users
    python manage.py createsuperuser

Download the AgoraDynamicKey Python3 code from the Agora repository: AgoraDynamicKey

Keep the downloaded folder in a location outside the project folder. Some of these files from the folder will be copied into our project when we're configuring the back end.

Configuring the Backend

We will create the views and classes with the methods needed to generate the Agora token to establish a call. We will set up Pusher at the server-side as well.

1. Add agora to the installed apps in mysite/settings.py

2. Add application routes

Create a file named urls.py in the agora directory and add the following code.
From your terminal or command prompt:

```bash

   touch agora/urls.py

```

Add the following to agora/urls.py

3. Add the downloaded AgoraDynamicKey generator files

Open your command prompt, and in the agora directory, create a sub-directory named agora_key.
```
   cd agora
   mkdir agora_key
```
Copy AccessToken.py and RtcTokenBuilder.py from the src directory in the downloaded files and add them to the agora_key directory.

4. Create the views for the Agora app in agora/views.py

Add the following block of code to the agora/views.py file.

Breakdown of Methods in agora/views.py

index: To view the video call page. Only authenticated users can view the page but non-authenticated users are redirected to the login page. We return a list of all the users apart from the currently authenticated user to be rendered on the front end.
pusher_auth: It serves as the endpoint for authenticating the logged-in user as they join the pusher's presence channel. The ID and name of the user are returned after successful authentication with the pusher.
generate_agora_token: To generate the Agora dynamic token. The token is used to authenticate app users when they join the agora channel to establish a call.
call_user: This triggers a make-agora-call event on the presence-online-channel to which all logged-in users are subscribed.

The data broadcast with the make-agora-call event across the presence-online-channel contains the following:

userToCall: This is the ID of the user who is supposed to receive a call from a caller.
channelName: This is the call channel that the caller has already joined on the front end. This is a channel created with the Agora SDK on the client-side. It is the room the caller has already joined, waiting for the callee to also join to establish a call connection.
from: The ID of the caller.

From the make-agora-call event, a user can determine whether they are being called if the userToCall value matches their ID. We show an incoming call notification with a button to accept the call. They know who the caller is by the value of from.

Configuring the Front End

We are going to create the user interface for making and receiving the video call with the ability to toggle the on and off states of the camera and the microphone.

1. Create the HTML file for the index view.

The HTML file will contain the links to the CDN for Agora SDK, Vue.js, Pusher, Bootstrap for styling, and our custom CSS and Javascript.

In your terminal navigate to the agora directory, create a templates directory and an agora subdirectory within it.
Create your index.html file in the agora subdirectory.

    cd agora
    mkdir -p templates/agora
    touch templates/agora/index.html

Add the following to the index.html file:

NOTE:

In the Pusher instance, replace the key with your own Pusher Key. Replace "420e941c25574fda6378" with yours.

      window.pusher = new Pusher("420e941c25574fda6378", {
        cluster: "APP_CLUSTER",
        authEndpoint: "{% url 'agora-pusher-auth' %}",
        auth: {
          headers: {
            "X-CSRFToken": "{{ csrf_token }}",
          },
        },
      });

2. Create the static files

We have index.css for custom styling and index.js; our script for handling the call logic.

Add the following to index.js

Add the following to index.css

Breakdown of the Agora Call Page

On the video call page, i.e app/templates/agora/ndex.html, we display buttons that bear the name of each registered user and whether they are online or offline at the moment.

To place a call, we click the button of a user with online status. An online user indicates one who is available to receive a call. For our demo, we see a list of users. The user named Bar is indicated as being online. The caller named Foo can call Bar by clicking the button.

Bar gets an incoming call notification with Accept and Decline buttons and the name of the caller.

From the call notification image above, we see that the caller's name is Foo. Bar can then accept the call for a connection to be established.

The following diagram explains the call logic in terms of the code:

3. Update env variables with Pusher and Agora keys

The .env file is located at the root of your project folder. Add the credentials you got from Agora and Pusher.

PUSHER_APP_ID=
PUSHER_KEY=
PUSHER_SECRET=
PUSHER_CLUSTER=

AGORA_APP_ID=
AGORA_APP_CERTIFICATE=

Testing

Start the Django development server from your terminal.
```
    python manage.py runserver
```
Open two different browsers or two instances of the same browser, with one instance in incognito mode, and go to http://127.0.0.1:8000
You are presented with the Django admin login page if you are not already logged in.
After successful login, you will be taken to the Django admin dashboard. Click on the VIEW SITE link at the top right to navigate to the video call page.
In each of the browsers you opened, the other users registered on the application are displayed.
In one browser, you can call the user who is logged in and on the other browser by clicking the button that bears their name.
The other user is prompted to click the Accept button to fully establish the call.

Video Demonstration of the Video Call

To confirm that your demo is functioning properly, see my demo video as an example of how the finished project should look and function:

Conclusion

You have now implemented the video call feature in your Django application! It's not that hard, right?

To include video calling functionality in your web app, you don't have to build it from scratch.

Agora provides a lot of great features out of the box. It also helps businesses save development hours when implementing video chat into existing projects. The only thing a developer has to do is build a compelling front end - Agora handles the video chat back end.

Link to project repository: https://github.com/Mupati/agora-django-video-call

Online Demo Link: https://fleet-server.herokuapp.com/agora/login/?next=/agora/dashboard/

Make sure the demo link or production version is served on HTTPS.

Test accounts:

foo@example.com: DY6m7feJtbnx3ud

bar@example.com: Me3tm5reQpWcn3Q

Other Resources

Available events on the Agora Client
For more information about Agora.io applications, take a look at the: Agora Quickstart Guides
Take a look at the complete documentation for the functions discussed above and many more: Agora Web SDK API

I also invite you to join the Agora.io Developer Slack community.

Build a Scalable Video Chat App with Agora in Flask

Kofi Mupati — Tue, 13 Apr 2021 14:56:46 +0000

Introduction

Flask is a micro web framework written in Python. It is mostly used to build API endpoints but can be extended to build fully-fledged applications. We are going to build a one-on-one video call application with Flask on the hassle-free Real-Time Engagement platform; Agora.

Previously, I built a video chat app with WebRTC and Laravel and wrote about it here:

Adding Video Chat To Your Laravel App

Kofi Mupati ・ Nov 12 '20

#laravel #vue #webrtc #videochat

Quality of experience: Since WebRTC is transmitted over the Internet, which is a public domain, the quality of experience is hard to guarantee.
Scalability: Scalability is fairly limited on group video calls due to the peer-to-peer nature of WebRTC.

After my experience with the Agora platform, I concluded it was not worthwhile for a company to invest a lot of development hours into building a video call feature as part of their main application with WebRTC. Agora is here to save you in that light. Through this article, I'm going to show you how to achieve that in your application you built with Flask.

Why Agora Is the Preferred Solution

After building a video chat app with Agora, I want to highlight some of the advantages:

There's one SDK for everything - voice, video, live streaming, screen sharing, and so on.
I didn't have to set up a turn server with coturn on Amazon EC2 as I did in the other implementation to relay traffic between peers on different networks.
You get 10,000 minutes every month free, and this gives you the flexibility to develop your solution prototype for free.
You don't have the challenge of managing the underlying infrastructure supporting the video call functionality.
Intuitive API documentation is available.

Prerequisites

Python 3.8.5
A Flask application with authentication. Download the following starter project to get started: Flask Auth Starter Project. We are going to build on top of this starter project.
A fair understanding of the factory and blueprint pattern in Flask. The starter project has been structured that way.
A free pusher account on pusher.com
An understanding of pusher presence channels and the python server library.
Agora Developer Account: (See How to get started with Agora)

Project Setup

Create and activate a python3 virtual environment for this project.
Open your terminal or command prompt and navigate to the starter project you downloaded as part of the prerequisites. The folder is named agora-flask-starter.
Follow the instructions in the README.md file in the agora-flask starter to set up the application.
Install additional dependencies from your terminal or command prompt.
```
    pip install pusher
```
Download the AgoraDynamicKey Python3 code from the Agora repository: AgoraDynamicKey

Keep the downloaded folder in a location outside the project folder. Some of these files from the folder will be copied into our project when we're configuring the back end.

Configuring the Backend

We are going to create the views for the agora blueprint and import the classes needed to generate the Agora token to establish a call. We will set up Pusher at the server-side as well.

1. Add the downloaded AgoraDynamicKey generator files

Open your terminal or command prompt and navigate into the agora blueprint directory.
```
    cd app/agora
```
Create a subdirectory named agora_key.
```
    mkdir agora_key
```
Copy AccessToken.py and RtcTokenBuilder.py from the src directory in the downloaded files and add them to the agora_key directory.

2. Create the agora view

Add the following code to the views.py file in the app/agora directory.

Breakdown of Methods in agora/views.py

index: To view the video call page. Only authenticated users can view the page but non-authenticated users are redirected to the login page. We return a list of all the registered users.
pusher_auth: It serves as the endpoint for authenticating the logged-in user as they join the pusher's presence channel. The ID and name of the user are returned after successful authentication with the pusher.
generate_agora_token: To generate the Agora dynamic token. The token is used to authenticate app users when they join the agora channel to establish a call.
call_user: This triggers a make-agora-call event on the presence-online-channel to which all logged-in users are subscribed.

The data broadcast with the make-agora-call event across the presence-online-channel contains the following:

userToCall: This is the ID of the user who is supposed to receive a call from a caller.
channelName: This is the call channel that the caller has already joined on the front end. This is a channel created with the Agora SDK on the client-side. It is the room the caller has already joined, waiting for the callee to also join to establish a call connection.
from: The ID of the caller.

Configuring the Front End

We are going to create the user interface for making and receiving the video call with the ability to toggle the on and off states of the camera and the microphone.

1. Update the HTML file for the index view.

The HTML file will contain the links to the CDN for Agora SDK, Vue.js, Pusher, Bootstrap for styling, and our custom CSS and Javascript.

The index.html file will also inherit a base template which is used to render the view.

Open the index.html file in the agora subdirectory in the templates directory. The location is app/templates/agora Currently, it only displays a welcome message for an authenticated user
Update the index.html file with the following code.

We use Flask's templating language to help with code reuse. As indicated early on, we inherit a base template named base.html. It has the following blocks:

head_scripts: This is the block where we place the link to the Agora SDK and our index.css for styling the video call page.
content: The content block contains the user interface for rendering the video stream with its control buttons.
bottom_scripts: This block contains the links to the Pusher SDK, an instance of the Pusher Class with the presence channel authentication, Axios for sending AJAX requests, and Vuejs for writing the client-side logic for our video chat application.

2. Create the static files

We have index.css for custom styling and index.js; our script for handling the call logic.

Run the following command to create the files from your terminal or command prompt.

    cd static/agora/
    touch index.css index.js

Add the following to index.js

Add the following to index.css

Breakdown of the Agora Call Page

On the video call page, i.e app/templates/agora/index.html, we display buttons that bear the name of each registered user and whether they are online or offline at the moment.

Bar gets an incoming call notification with Accept and Decline buttons and the name of the caller.

From the call notification image above, we see that the caller's name is Foo. Bar can then accept the call for a connection to be established.

The following diagram explains the call logic in terms of the code:

3. Update flaskenv variables with Pusher and Agora keys

The .flaskenv file is located at the root of your project folder. Add the credentials you got from Agora and Pusher. Add the secret key too if you didn't add it during the project setup.

Testing

Start the Flask development server from your terminal.
```
    flask run
```
Open two different browsers or two instances of the same browser, with one instance in incognito mode, and navigate to the registration page.

http://127.0.0.1:5000/register
In one of the browsers create about 4 users by registering 4 times.
Login with the account details you just created on each of the browsers from the login page: http://127.0.0.1:5000/login
In each of the browsers you opened, the other users registered on the application are displayed.
In one browser, you can call the user who is logged in and on the other browser by clicking the button that bears their name.
The other user is prompted to click the Accept button to fully establish the call.

Video Demonstration of the Video Call

To confirm that your demo is functioning properly, see my demo video as an example of how the finished project should look and function:

Conclusion

You have now implemented the video call feature in your Flask application! It's not that hard, right?

To include video calling functionality in your web app, you don't have to build it from scratch.

Online Demo link: https://watermark-remover.herokuapp.com/auth/login?next=%2Fagora

Completed Project Repository (it is located on the branch named completed in the starter kit):
https://github.com/Mupati/agora-flask-starter/tree/completed

Make sure the demo link or production version is served on HTTPS.

Test accounts:

foo@example.com: DY6m7feJtbnx3ud

bar@example.com: Me3tm5reQpWcn3Q

Other Resources

Available events on the Agora Client
For more information about Agora.io applications, take a look at the: Agora Quickstart Guides
Take a look at the complete documentation for the functions discussed above and many more: Agora Web SDK API

I also invite you to join the Agora.io Developer Slack community.

Live stream with WebRTC in your Laravel application

Kofi Mupati — Tue, 23 Mar 2021 08:32:33 +0000

Introduction

My first attempt at WebRTC was to implement a video call feature within a Laravel Application. The implementation involved placing a call, showing an incoming call notification, and the ability of the receiver to accept the call. I wrote about it over here:

Adding Video Chat To Your Laravel App

Kofi Mupati ・ Nov 12 '20

#laravel #vue #webrtc #videochat

One of my readers asked whether it was possible to build a live streaming application with WebRTC in a Laravel Application. I took up this challenge and even though WebRTC has limitations, I came up with a simple live streaming implementation.

We'll go through my implementation in this article.

Final Project Repository: https://github.com/Mupati/laravel-video-chat Note that this repository contains code for some other technical articles.

Requirements

This tutorial assumes you know how to set up a new Laravel project with VueJs authentication. Create some users after setting up your project. You should be familiar with Laravel's broadcasting mechanism and have a fair idea of how WebSockets work. You may use this starter project I created: Laravel 8 Vue Auth Starter
Set up a free pusher account on pusher.com
Set up your ICE SERVER (TURN SERVER) details. This tutorial is a good guide. HOW TO INSTALL COTURN.

Project Setup

# Install needed packages
composer require pusher/pusher-PHP-server "~4.0"
npm install --save laravel-echo pusher-js simple-peer

Configuring Backend

Add routes for streaming pages in routes/web.php. The routes will be used to visit the live stream page, start a live stream from the device camera, and generate a broadcast link for other authenticated users to view your live stream.

    Route::get('/streaming', 'App\Http\Controllers\WebrtcStreamingController@index');
    Route::get('/streaming/{streamId}', 'App\Http\Controllers\WebrtcStreamingController@consumer');
    Route::post('/stream-offer', 'App\Http\Controllers\WebrtcStreamingController@makeStreamOffer');
    Route::post('/stream-answer', 'App\Http\Controllers\WebrtcStreamingController@makeStreamAnswer');

Uncomment BroadcastServiceProvider in config/app.php. This allows us to use Laravel's broadcasting system.

+ App\Providers\BroadcastServiceProvider::class
- //App\Providers\BroadcastServiceProvider::class

Create Dynamic Presence and Private Channel in routes/channels.php.

Authenticated users subscribe to both channels.
The presence channel is dynamically created with a streamId generated by the broadcaster. This way, we can detect all users who have joined the live broadcast.

Signaling information is exchanged between the broadcaster and the viewer through the private channel.

// Dynamic Presence Channel for Streaming
Broadcast::channel('streaming-channel.{streamId}', function ($user) {
    return ['id' => $user->id, 'name' => $user->name];
});

// Signaling Offer and Answer Channels
Broadcast::channel('stream-signal-channel.{userId}', function ($user, $userId) {
    return (int) $user->id === (int) $userId;
});

Create StreamOffer and StreamAnswer events. Signaling information is broadcast on the stream-signal-channel-{userId} private channel we created early on.

The broadcaster sends an offer to a new user who joins the live stream when we emit the StreamOffer event and the viewer replies with an answer using the StreamAnswer event.

php artisan make:event StreamOffer
php artisan make:event StreamAnswer

Add the following code to app/Events/StreamOffer.php.

<?php

namespace App\Events;

use Illuminate\Broadcasting\InteractsWithSockets;
use Illuminate\Broadcasting\PrivateChannel;
use Illuminate\Contracts\Broadcasting\ShouldBroadcast;
use Illuminate\Foundation\Events\Dispatchable;
use Illuminate\Queue\SerializesModels;

class StreamOffer implements ShouldBroadcast
{
    use Dispatchable, InteractsWithSockets, SerializesModels;

    public $data;
    /**
     * Create a new event instance.
     *
     * @return void
     */
    public function __construct($data)
    {
        $this->data = $data;
    }

    /**
     * Get the channels the event should broadcast on.
     *
     * @return \Illuminate\Broadcasting\Channel|array
     */
    public function broadcastOn()
    {
        // stream offer can broadcast on a private channel
        return  new PrivateChannel('stream-signal-channel.' . $this->data['receiver']['id']);
    }
}

Add the following code to app/Events/StreamAnswer.php.

<?php

namespace App\Events;

use Illuminate\Broadcasting\InteractsWithSockets;
use Illuminate\Broadcasting\PrivateChannel;
use Illuminate\Contracts\Broadcasting\ShouldBroadcast;
use Illuminate\Foundation\Events\Dispatchable;
use Illuminate\Queue\SerializesModels;

class StreamAnswer implements ShouldBroadcast
{
    use Dispatchable, InteractsWithSockets, SerializesModels;

    public $data;
    /**
     * Create a new event instance.
     *
     * @return void
     */
    public function __construct($data)
    {
        $this->data = $data;
    }

    /**
     * Get the channels the event should broadcast on.
     *
     * @return \Illuminate\Broadcasting\Channel|array
     */
    public function broadcastOn()
    {
        return  new PrivateChannel('stream-signal-channel.' . $this->data['broadcaster']);
    }
}

Create WebrtcStreamingController to handle the broadcasting, viewing, and signaling for the live stream.

php artisan make:controller WebrtcStreamingController

Add the following to the WebrtcStreamingController

<?php

namespace App\Http\Controllers;

use App\Events\StreamAnswer;
use App\Events\StreamOffer;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\Auth;

class WebrtcStreamingController extends Controller
{

    public function index()
    {
        return view('video-broadcast', ['type' => 'broadcaster', 'id' => Auth::id()]);
    }

    public function consumer(Request $request, $streamId)
    {
        return view('video-broadcast', ['type' => 'consumer', 'streamId' => $streamId, 'id' => Auth::id()]);
    }

    public function makeStreamOffer(Request $request)
    {
        $data['broadcaster'] = $request->broadcaster;
        $data['receiver'] = $request->receiver;
        $data['offer'] = $request->offer;

        event(new StreamOffer($data));
    }

    public function makeStreamAnswer(Request $request)
    {
        $data['broadcaster'] = $request->broadcaster;
        $data['answer'] = $request->answer;
        event(new StreamAnswer($data));
    }
}

Methods in the WebrtcStreamingController

Let's explore what the methods in the controller are doing.

index: This returns the view for the broadcaster. We pass a 'type': broadcaster and the user's ID into the view to help identify who the user is.
consumer: It returns the view for a new user who wants to join the live stream. We pass a 'type': consumer, the 'streamId' we extract from the broadcasting link, and the user's ID.
makeStreamOffer: It broadcasts an offer signal sent by the broadcaster to a specific user who just joined. The following data is sent:
- broadcaster: The user ID of the one who initiated the live stream i.e the broadcaster
- receiver: The ID of the user to whom the signaling offer is being sent.
- offer: This is the WebRTC offer from the broadcaster.
makeStreamAnswer: It sends an answer signal to the broadcaster to fully establish the peer connection.
- broadcaster: The user ID of the one who initiated the live stream i.e the broadcaster.
- answer: This is the WebRTC answer from the viewer after, sent after receiving an offer from the broadcaster.

Configuring Frontend

Instantiate Laravel Echo and Pusher in resources/js/bootstrap.js by uncommenting the following block of code.

+ import Echo from 'laravel-echo';
+ window.Pusher = require('pusher-js');
+ window.Echo = new Echo({
+     broadcaster: 'pusher',
+     key: process.env.MIX_PUSHER_APP_KEY,
+     cluster: process.env.MIX_PUSHER_APP_CLUSTER,
+     forceTLS: true
+ });
- import Echo from 'laravel-echo';
- window.Pusher = require('pusher-js');
- window.Echo = new Echo({
-     broadcaster: 'pusher',
-     key: process.env.MIX_PUSHER_APP_KEY,
-     cluster: process.env.MIX_PUSHER_APP_CLUSTER,
-     forceTLS: true
-});

Create resources/js/helpers.js. Add a getPermissions function to help with permission access for the microphone and camera. This method handles the video and audio permission that is required by browsers to make the video calls. It waits for the user to accept the permissions before we can proceed with the video call. We allow both audio and video. Read more on MDN Website.

export const getPermissions = () => {
    // Older browsers might not implement mediaDevices at all, so we set an empty object first
    if (navigator.mediaDevices === undefined) {
        navigator.mediaDevices = {};
    }

    // Some browsers partially implement media devices. We can't just assign an object
    // with getUserMedia as it would overwrite existing properties.
    // Here, we will just add the getUserMedia property if it's missing.
    if (navigator.mediaDevices.getUserMedia === undefined) {
        navigator.mediaDevices.getUserMedia = function(constraints) {
            // First get ahold of the legacy getUserMedia, if present
            const getUserMedia =
                navigator.webkitGetUserMedia || navigator.mozGetUserMedia;

            // Some browsers just don't implement it - return a rejected promise with an error
            // to keep a consistent interface
            if (!getUserMedia) {
                return Promise.reject(
                    new Error("getUserMedia is not implemented in this browser")
                );
            }

            // Otherwise, wrap the call to the old navigator.getUserMedia with a Promise
            return new Promise((resolve, reject) => {
                getUserMedia.call(navigator, constraints, resolve, reject);
            });
        };
    }
    navigator.mediaDevices.getUserMedia =
        navigator.mediaDevices.getUserMedia ||
        navigator.webkitGetUserMedia ||
        navigator.mozGetUserMedia;

    return new Promise((resolve, reject) => {
        navigator.mediaDevices
            .getUserMedia({ video: true, audio: true })
            .then(stream => {
                resolve(stream);
            })
            .catch(err => {
                reject(err);
                //   throw new Error(`Unable to fetch stream ${err}`);
            });
    });
};

Create a component for the Broadcaster named Broadcaster.vue in resources/js/components/Broadcaster.vue.
Create a component for the Viewer named Viewer.vue in resources/js/components/Viewer.vue.

Explanation of the Broadcaster and Viewer components.

The following video explains the call logic on the client-side from the perspective of both Broadcaster and Viewers.

//  Streaming Components
Vue.component("broadcaster", require("./components/Broadcaster.vue").default);
Vue.component("viewer", require("./components/Viewer.vue").default);

Create the video broadcast view in resources/views/video-broadcast.blade.php
Update env variables. Insert your Pusher API keys

APP_ENV=

BROADCAST_DRIVER=pusher

PUSHER_APP_ID=
PUSHER_APP_KEY=
PUSHER_APP_SECRET=
PUSHER_APP_CLUSTER=

TURN_SERVER_URL=
TURN_SERVER_USERNAME=
TURN_SERVER_CREDENTIAL=

Live Stream Demo

Final Thoughts

The logic for this live streaming application can be likened to a group video call where only one person's stream is seen.

The stream of the broadcaster is rendered on the viewers' browser but the broadcaster doesn't receive anything from the viewers after exchanging the signaling information which is required in WebRTC.

This looks like a star topology and there is a limitation on how many peers can be connected to a single user.

I want to explore the option of turning some of the viewers into broadcasters after the initial broadcaster's peer has connected to about 4 users.

The goal is to rebroadcast the stream they received from the original broadcaster.

Is it possible? I can't tell. This will be an interesting challenge to explore.

Stay tuned!!!.

Build a Scalable Video Chat App with Agora in Laravel

Kofi Mupati — Sun, 17 Jan 2021 00:45:35 +0000

Introduction

Laravel is a powerful PHP framework that aims to make the web development process easier without sacrificing application functionality. This is especially true when you try to build a video chat app with Laravel.

Previously, I built a video chat app with WebRTC and Laravel and wrote about it here: Adding Video Chat To Your Laravel App. WebRTC is only one of the ways that people can implement video chat features. Companies like Agora also provide a fully packaged video chat SDK to provide a high-quality Real-Time Engagement video chat experience. As someone who has WebRTC development experience, I can tell you there are some limitations with WebRTC, such as:

Quality of experience: Since WebRTC is transmitted over the Internet, which is a public domain, the quality of experience is hard to guarantee.
Scalability: Scalability is fairly limited on group video calls due to the peer-to-peer nature of WebRTC.

After I was introduced to Agora, I was impressed that setting up the same video call feature is easier with Agora than with WebRTC. Let me walk you through building a video chat app with Agora in Laravel.

Why Agora Is the Preferred Solution

After building a video chat app with Agora, I want to highlight some of the advantages:

There's one SDK for everything - voice, video, live streaming, screen sharing, and so on.
I didn't have to set up a turn server with coturn on Amazon EC2 as I did in the other implementation to relay traffic between peers on different networks.
You get 10,000 minutes every month free, and this gives you the flexibility to develop your solution prototype for free.
You don't have the challenge of managing the underlying infrastructure supporting the video call functionality.
Intuitive API documentation is available.

Prerequisites

A Laravel project with VueJs authentication with some created users. You can download the following base project to get started: Laravel 8 Vue Auth Starter
An understanding of Laravel's broadcasting and event system and WebSockets in general.
A free pusher account on pusher.com
An Agora Account: How to get started with Agora

Project Setup

Open your terminal or console and navigate to your Laravel project directory.
Install the necessary packages. ```bash composer require pusher/pusher-php-server "~4.0" npm install --save laravel-echo pusher-js ```
Download the AgoraDynamicKey PHP code from the Agora repository: AgoraDynamicKey Keep the downloaded folder in a location outside the project folder. Some files from the folder will be copied into our project when we're configuring the back end.

Configuring the Backend

We will set up the various controllers and classes with methods needed to generate the Agora token to establish a call. Laravel's broadcasting system will also be activated.
We begin with the endpoints that will be accessed from the front end.

1. Add Application Routes.

Add the following code to routes/web.php.

2. Activate Laravel's Broadcasting System.

Uncomment BroadcastServiceProvider in config/app.php in your project folder.

+ App\Providers\BroadcastServiceProvider::class
- //App\Providers\BroadcastServiceProvider::class

3. Create a presence channel in routes/channels.php

We get to know online users by looking at who has subscribed to this channel on the front end. We return their ID and name.

4. Create an event for making a call named MakeAgoraCall

This event will be used to make a call by broadcasting some data over the agora-online-channel. Run the following command in your terminal/console:

php artisan make:event MakeAgoraCall

Add the following code to MakeAgoraCall.php

5. Add the downloaded AgoraDynamicKey generator files

Open your command line, and in your project's app directory create a directory named Class along with a subdirectory named AgoraDynamicKey. ```bash cd app mkdir -p Class/AgoraDynamicKey ```
Open the AgoraDynamicKey folder that was downloaded as part of the project setup. Next, copy the AccessToken.php and RtcTokenBuilder.php from the src directory into the AgoraDynamicKey directory that was created in the previous step.
Open AccessToken.php and RtcTokenBuilder.php, and add the project's namespace to make them accessible in a controller.

RtcTokenBuilder.php becomes:

AccessToken.php becomes:

6. Create the AgoraVideoController.php

Next, create the AgoraVideoController using the command line.

php artisan make:controller AgoraVideoController

Add the following code to AgoraVideoController.php:

Breakdown of Methods in the AgoraVideoController

index: To view the video call page.
token: To generate the Agora dynamic token. The token generation code is taken from sample/RtcTokenBuilderSample.php, which can be found in the files downloaded during the project setup.
callUser: To call a particular user by broadcasting a MakeAgoraCall event across a Laravel presence channel that I've named agora-online-channel. All logged-in users subscribe and listen to events on this channel.

The data received from the MakeAgoraEvent by the users subscribed to the agora-online-channel contains the following:

userToCall: This is the ID of the user who is supposed to receive a call from a caller.
channelName: This is the call channel that the caller has already joined on the front end. This is a channel created with the Agora SDK on the front end. It is the room the caller has already joined, waiting for the callee to also join to establish a call connection.
from: The ID of the caller.

From the MakeAgoraEvent, a user can determine whether they are being called if the userToCall value matches their ID. We show an incoming call notification with a button to accept the call.
They know who the caller is by the value of from.

Configuring the Front End

We are going to create the user interface for making and receiving the video call with the ability to toggle the on and off states of the camera and the microphone.

1. Add a link to the Agora SDK

Add the following to the head tag of resources/views/layouts/app.blade.php.

It then becomes:

2. Instantiate Laravel Echo and Pusher Instantiate Laravel Echo and Pusher in resources/js/bootstrap.js by uncommenting the following block of code:

3. Create an Agora chat component

On your terminal or command line, create a component called AgoraChat.vue with the following command:

touch resources/js/components/AgoraChat.vue

Add the following code:

Breakdown of the AgoraChat Component

On the AgoraChat page, we display buttons that bear the name of each registered user and whether they are online or offline at the moment.

To place a call, we click the button of a user with online status. An online user indicates one who is readily available to receive a call. For our demo, we see a list of users. The user named Bar is indicated as being online. The caller named Foo can call Bar by clicking the button.

Bar gets an incoming call notification with Accept and Decline buttons and the name of the caller.

From the call notification image above, we see that the caller's name is Foo. Bar can then accept the call for a connection to be established.

The following diagram explains the call logic in terms of the code:

4. Register the AgoraChat.vue component

Add the following code to resources/js/app.js:

5. Create an Agora chat view

On your terminal or command line, create a view called agora-chat.blade.php with the following command:

touch resources/views/agora-chat.blade.php

Add the following code to the agora-chat.blade.php:

6. Update env variables with Pusher and Agora keys

The .env file is located at the root of your project folder. Add the credentials you got from Agora and Pusher.

Testing

Start the Laravel development server from your terminal.br/> ```bash php artisan serve ```
Open another terminal and run the front end. ```bash npm run dev ```
Open two different browsers or two instances of the same browser, with one instance in incognito mode, and go to http://127.0.0.1:8000/agora-chat.
You are presented with a login page if you are not already logged in.
After successful login, you are automatically redirected to the video chat page, where you see the buttons with names of the users with an indication of their online status.
In each of the browsers you opened, the other users registered on the application are displayed as described in 5.
In one browser, you can call the user who is logged in on the other browser by clicking the button that bears their name.
The other user is prompted to click the Accept button to fully establish the call.

Video Demonstration of the Video Call

To confirm that your demo is functioning properly, see my demo video as an example of how the finished project should look and function:

Conclusion

You have now implemented the video chat feature in Laravel! It's not that hard, right?

To include video calling functionality in your web app, you don't have to build it from scratch.

Link to project repository: https://github.com/Mupati/laravel-video-chat

Demo link: https://laravel-video-call.herokuapp.com/agora-chat

Make sure the demo link or production version is served on HTTPS.

Test accounts:

foo@example.com: DY6m7feJtbnx3ud

bar@example.com: Me3tm5reQpWcn3Q

Other Resources

I also invite you to join the Agora.io Developer Slack community.

Adding Video Chat To Your Laravel App

Kofi Mupati — Thu, 12 Nov 2020 16:31:06 +0000

Introduction

I was required to build a custom video chat application for a Vuejs and Laravel project. I went through a lot of hoops to get it working. I will share all that I learnt throughout the process over here.

Final Project Repository: https://github.com/Mupati/laravel-video-chat

Requirements

This tutorial assumes you know how to set up a new Laravel project with VueJs authentication. Create some users after setting up your project. You should be familiar with Laravel's broadcasting mechanism and have a fair idea of how WebSockets work.
Set up a free pusher account on pusher.com
Set up your ICE SERVER (TURN SERVER) details. This tutorial is a good guide. HOW TO INSTALL COTURN.

Project Setup

# Install needed packages
composer require pusher/pusher-php-server "~4.0"
npm install --save laravel-echo pusher-js simple-peer

Configuring Backend

Add routes for video page in routes/web.php. The routes will be used to visit the video call page, make calls and receive calls.

Route::get('/video-chat', function () {
    // fetch all users apart from the authenticated user
    $users = User::where('id', '<>', Auth::id())->get();
    return view('video-chat', ['users' => $users]);
});

// Endpoints to call or receive calls.
Route::post('/video/call-user', 'App\Http\Controllers\VideoChatController@callUser');
Route::post('/video/accept-call', 'App\Http\Controllers\VideoChatController@acceptCall');

Uncomment BroadcastServiceProvider in config/app.php. This allows us to use Laravel's broadcasting system.

+ App\Providers\BroadcastServiceProvider::class
- //App\Providers\BroadcastServiceProvider::class

Create a Presence Channel for the Video Chat Application in routes/channels.php. When an authenticated user subscribes to the channel (presence-video-channel), we return the users's id and name. This is how we are able to get the user who is logged in and can be called.

Broadcast::channel('presence-video-channel', function($user) {
    return ['id' => $user->id, 'name' => $user->name];
});

Create StartVideoChat event.This event will be called when placing a call or accepting a call and it will broadcast on the presence-video-call channel. Users subscribed to the channel will be listening to this event on the frontend so that incoming call notifications can be triggered.

php artisan make:event StartVideoChat

Add the following code to app/Events/StartVideoChat.php. The StartVideoChat event broadcasts to presence-video-channel so that the data needed to initiate the video call is shared on the channel.

<?php

namespace App\Events;

use Illuminate\Broadcasting\InteractsWithSockets;
use Illuminate\Broadcasting\PresenceChannel;
use Illuminate\Contracts\Broadcasting\ShouldBroadcast;
use Illuminate\Foundation\Events\Dispatchable;
use Illuminate\Queue\SerializesModels;

class StartVideoChat implements ShouldBroadcast
{
    use Dispatchable, InteractsWithSockets, SerializesModels;

    public $data;
    /**
     * Create a new event instance.
     *
     * @return void
     */
    public function __construct($data)
    {
        $this->data = $data;
    }

    /**
     * Get the channels the event should broadcast on.
     *
     * @return \Illuminate\Broadcasting\Channel|array
     */
    public function broadcastOn()
    {
        return new PresenceChannel('presence-video-channel');
    }
}

Create VideoChatController to make and accept calls.

php artisan make:controller VideoChatController

Add the following to the VideoChatController

<?php

namespace App\Http\Controllers;

use Illuminate\Support\Facades\Auth;
use Illuminate\Http\Request;
use App\Events\StartVideoChat;

class VideoChatController extends Controller
{

    public function callUser(Request $request)
    {
        $data['userToCall'] = $request->user_to_call;
        $data['signalData'] = $request->signal_data;
        $data['from'] = Auth::id();
        $data['type'] = 'incomingCall';

        broadcast(new StartVideoChat($data))->toOthers();
    }
    public function acceptCall(Request $request)
    {
        $data['signal'] = $request->signal;
        $data['to'] = $request->to;
        $data['type'] = 'callAccepted';
        broadcast(new StartVideoChat($data))->toOthers();
    }
}

Methods in the VideoChatController

One thing to understand is that the VideoChat Application is a realtime application that works with web sockets. The endpoints are just needed to establish the link between the 2 calling parties after which the communication data is exchanged via websockets.

Let's try to understand what the 2 methods in the Controller are doing:

`callUser` Method

user_to_call: id of the user the initiator of the call wants to reach.
signal_data: The initial signal data (offer) sent by the caller from the webrtc client (simple-peerjs is the webrtc wrapper we are using). These are the parameters received. We create a data object with 2 additional properties,from and type then broadcast the data with the StartVideoChat event which will be listened to on the frontend.
from: is the id of the user placing the call. We use the authenticated user's id.
type: is a property of the data which will indicate that there is an incoming call on the channel. The notification will be shown to the user whose id corresponds to the value of user_to_call.

`acceptCall` Method

signal: This is the callee's answer data.
to: The caller's of the call's id. The signal data for the answered call is sent to the user whose id matches to and this is supposed to be the caller's id.
type: A property added to the data to be sent over the channel indicating that the call recipient has accepted the call.

Configuring Frontend

Instantiate Laravel Echo and Pusher in resources/js/bootstrap.js by uncommenting the following block of code.

+ import Echo from 'laravel-echo';
+ window.Pusher = require('pusher-js');
+ window.Echo = new Echo({
+     broadcaster: 'pusher',
+     key: process.env.MIX_PUSHER_APP_KEY,
+     cluster: process.env.MIX_PUSHER_APP_CLUSTER,
+     forceTLS: true
+ });
- import Echo from 'laravel-echo';
- window.Pusher = require('pusher-js');
- window.Echo = new Echo({
-     broadcaster: 'pusher',
-     key: process.env.MIX_PUSHER_APP_KEY,
-     cluster: process.env.MIX_PUSHER_APP_CLUSTER,
-     forceTLS: true
-});

Create resources/js/helpers.js. Add a getPermissions function to help with permission access for microphone and videos. This method handles the video and audio permission that is required by browsers to make the video calls. It waits for the user to accept the permissions before we can proceed with the video call. We allow both audio and video. Read more on MDN Website.

export const getPermissions = () => {
    // Older browsers might not implement mediaDevices at all, so we set an empty object first
    if (navigator.mediaDevices === undefined) {
        navigator.mediaDevices = {};
    }

    // Some browsers partially implement mediaDevices. We can't just assign an object
    // with getUserMedia as it would overwrite existing properties.
    // Here, we will just add the getUserMedia property if it's missing.
    if (navigator.mediaDevices.getUserMedia === undefined) {
        navigator.mediaDevices.getUserMedia = function(constraints) {
            // First get ahold of the legacy getUserMedia, if present
            const getUserMedia =
                navigator.webkitGetUserMedia || navigator.mozGetUserMedia;

            // Some browsers just don't implement it - return a rejected promise with an error
            // to keep a consistent interface
            if (!getUserMedia) {
                return Promise.reject(
                    new Error("getUserMedia is not implemented in this browser")
                );
            }

            // Otherwise, wrap the call to the old navigator.getUserMedia with a Promise
            return new Promise((resolve, reject) => {
                getUserMedia.call(navigator, constraints, resolve, reject);
            });
        };
    }
    navigator.mediaDevices.getUserMedia =
        navigator.mediaDevices.getUserMedia ||
        navigator.webkitGetUserMedia ||
        navigator.mozGetUserMedia;

    return new Promise((resolve, reject) => {
        navigator.mediaDevices
            .getUserMedia({ video: true, audio: true })
            .then(stream => {
                resolve(stream);
            })
            .catch(err => {
                reject(err);
                //   throw new Error(`Unable to fetch stream ${err}`);
            });
    });
};

Create a Video Chat Component, resources/js/components/VideoChat.vue.

<template>
  <div>
    <div class="container">
      <div class="row">
        <div class="col">
          <div class="btn-group" role="group">
            <button
              type="button"
              class="btn btn-primary mr-2"
              v-for="user in allusers"
              :key="user.id"
              @click="placeVideoCall(user.id, user.name)"
            >
              Call {{ user.name }}
              <span class="badge badge-light">{{
                getUserOnlineStatus(user.id)
              }}</span>
            </button>
          </div>
        </div>
      </div>
      <!--Placing Video Call-->
      <div class="row mt-5" id="video-row">
        <div class="col-12 video-container" v-if="callPlaced">
          <video
            ref="userVideo"
            muted
            playsinline
            autoplay
            class="cursor-pointer"
            :class="isFocusMyself === true ? 'user-video' : 'partner-video'"
            @click="toggleCameraArea"
          />
          <video
            ref="partnerVideo"
            playsinline
            autoplay
            class="cursor-pointer"
            :class="isFocusMyself === true ? 'partner-video' : 'user-video'"
            @click="toggleCameraArea"
            v-if="videoCallParams.callAccepted"
          />
          <div class="partner-video" v-else>
            <div v-if="callPartner" class="column items-center q-pt-xl">
              <div class="col q-gutter-y-md text-center">
                <p class="q-pt-md">
                  <strong>{{ callPartner }}</strong>
                </p>
                <p>calling...</p>
              </div>
            </div>
          </div>
          <div class="action-btns">
            <button type="button" class="btn btn-info" @click="toggleMuteAudio">
              {{ mutedAudio ? "Unmute" : "Mute" }}
            </button>
            <button
              type="button"
              class="btn btn-primary mx-4"
              @click="toggleMuteVideo"
            >
              {{ mutedVideo ? "ShowVideo" : "HideVideo" }}
            </button>
            <button type="button" class="btn btn-danger" @click="endCall">
              EndCall
            </button>
          </div>
        </div>
      </div>
      <!-- End of Placing Video Call  -->

      <!-- Incoming Call  -->
      <div class="row" v-if="incomingCallDialog">
        <div class="col">
          <p>
            Incoming Call From <strong>{{ callerDetails.name }}</strong>
          </p>
          <div class="btn-group" role="group">
            <button
              type="button"
              class="btn btn-danger"
              data-dismiss="modal"
              @click="declineCall"
            >
              Decline
            </button>
            <button
              type="button"
              class="btn btn-success ml-5"
              @click="acceptCall"
            >
              Accept
            </button>
          </div>
        </div>
      </div>
      <!-- End of Incoming Call  -->
    </div>
  </div>
</template>

<script>
import Peer from "simple-peer";
import { getPermissions } from "../helpers";
export default {
  props: [
    "allusers",
    "authuserid",
    "turn_url",
    "turn_username",
    "turn_credential",
  ],
  data() {
    return {
      isFocusMyself: true,
      callPlaced: false,
      callPartner: null,
      mutedAudio: false,
      mutedVideo: false,
      videoCallParams: {
        users: [],
        stream: null,
        receivingCall: false,
        caller: null,
        callerSignal: null,
        callAccepted: false,
        channel: null,
        peer1: null,
        peer2: null,
      },
    };
  },

  mounted() {
    this.initializeChannel(); // this initializes laravel echo
    this.initializeCallListeners(); // subscribes to video presence channel and listens to video events
  },
  computed: {
    incomingCallDialog() {
      if (
        this.videoCallParams.receivingCall &&
        this.videoCallParams.caller !== this.authuserid
      ) {
        return true;
      }
      return false;
    },

    callerDetails() {
      if (
        this.videoCallParams.caller &&
        this.videoCallParams.caller !== this.authuserid
      ) {
        const incomingCaller = this.allusers.filter(
          (user) => user.id === this.videoCallParams.caller
        );

        return {
          id: this.videoCallParams.caller,
          name: `${incomingCaller[0].name}`,
        };
      }
      return null;
    },
  },
  methods: {
    initializeChannel() {
      this.videoCallParams.channel = window.Echo.join("presence-video-channel");
    },

    getMediaPermission() {
      return getPermissions()
        .then((stream) => {
          this.videoCallParams.stream = stream;
          if (this.$refs.userVideo) {
            this.$refs.userVideo.srcObject = stream;
          }
        })
        .catch((error) => {
          console.log(error);
        });
    },

    initializeCallListeners() {
      this.videoCallParams.channel.here((users) => {
        this.videoCallParams.users = users;
      });

      this.videoCallParams.channel.joining((user) => {
        // check user availability
        const joiningUserIndex = this.videoCallParams.users.findIndex(
          (data) => data.id === user.id
        );
        if (joiningUserIndex < 0) {
          this.videoCallParams.users.push(user);
        }
      });

      this.videoCallParams.channel.leaving((user) => {
        const leavingUserIndex = this.videoCallParams.users.findIndex(
          (data) => data.id === user.id
        );
        this.videoCallParams.users.splice(leavingUserIndex, 1);
      });
      // listen to incomming call
      this.videoCallParams.channel.listen("StartVideoChat", ({ data }) => {
        if (data.type === "incomingCall") {
          // add a new line to the sdp to take care of error
          const updatedSignal = {
            ...data.signalData,
            sdp: `${data.signalData.sdp}\n`,
          };

          this.videoCallParams.receivingCall = true;
          this.videoCallParams.caller = data.from;
          this.videoCallParams.callerSignal = updatedSignal;
        }
      });
    },
    async placeVideoCall(id, name) {
      this.callPlaced = true;
      this.callPartner = name;
      await this.getMediaPermission();
      this.videoCallParams.peer1 = new Peer({
        initiator: true,
        trickle: false,
        stream: this.videoCallParams.stream,
        config: {
          iceServers: [
            {
              urls: this.turn_url,
              username: this.turn_username,
              credential: this.turn_credential,
            },
          ],
        },
      });

      this.videoCallParams.peer1.on("signal", (data) => {
        // send user call signal
        axios
          .post("/video/call-user", {
            user_to_call: id,
            signal_data: data,
            from: this.authuserid,
          })
          .then(() => {})
          .catch((error) => {
            console.log(error);
          });
      });

      this.videoCallParams.peer1.on("stream", (stream) => {
        console.log("call streaming");
        if (this.$refs.partnerVideo) {
          this.$refs.partnerVideo.srcObject = stream;
        }
      });

      this.videoCallParams.peer1.on("connect", () => {
        console.log("peer connected");
      });

      this.videoCallParams.peer1.on("error", (err) => {
        console.log(err);
      });

      this.videoCallParams.peer1.on("close", () => {
        console.log("call closed caller");
      });

      this.videoCallParams.channel.listen("StartVideoChat", ({ data }) => {
        if (data.type === "callAccepted") {
          if (data.signal.renegotiate) {
            console.log("renegotating");
          }
          if (data.signal.sdp) {
            this.videoCallParams.callAccepted = true;
            const updatedSignal = {
              ...data.signal,
              sdp: `${data.signal.sdp}\n`,
            };
            this.videoCallParams.peer1.signal(updatedSignal);
          }
        }
      });
    },

    async acceptCall() {
      this.callPlaced = true;
      this.videoCallParams.callAccepted = true;
      await this.getMediaPermission();
      this.videoCallParams.peer2 = new Peer({
        initiator: false,
        trickle: false,
        stream: this.videoCallParams.stream,
        config: {
          iceServers: [
            {
              urls: this.turn_url,
              username: this.turn_username,
              credential: this.turn_credential,
            },
          ],
        },
      });
      this.videoCallParams.receivingCall = false;
      this.videoCallParams.peer2.on("signal", (data) => {
        axios
          .post("/video/accept-call", {
            signal: data,
            to: this.videoCallParams.caller,
          })
          .then(() => {})
          .catch((error) => {
            console.log(error);
          });
      });

      this.videoCallParams.peer2.on("stream", (stream) => {
        this.videoCallParams.callAccepted = true;
        this.$refs.partnerVideo.srcObject = stream;
      });

      this.videoCallParams.peer2.on("connect", () => {
        console.log("peer connected");
        this.videoCallParams.callAccepted = true;
      });

      this.videoCallParams.peer2.on("error", (err) => {
        console.log(err);
      });

      this.videoCallParams.peer2.on("close", () => {
        console.log("call closed accepter");
      });

      this.videoCallParams.peer2.signal(this.videoCallParams.callerSignal);
    },
    toggleCameraArea() {
      if (this.videoCallParams.callAccepted) {
        this.isFocusMyself = !this.isFocusMyself;
      }
    },
    getUserOnlineStatus(id) {
      const onlineUserIndex = this.videoCallParams.users.findIndex(
        (data) => data.id === id
      );
      if (onlineUserIndex < 0) {
        return "Offline";
      }
      return "Online";
    },
    declineCall() {
      this.videoCallParams.receivingCall = false;
    },

    toggleMuteAudio() {
      if (this.mutedAudio) {
        this.$refs.userVideo.srcObject.getAudioTracks()[0].enabled = true;
        this.mutedAudio = false;
      } else {
        this.$refs.userVideo.srcObject.getAudioTracks()[0].enabled = false;
        this.mutedAudio = true;
      }
    },

    toggleMuteVideo() {
      if (this.mutedVideo) {
        this.$refs.userVideo.srcObject.getVideoTracks()[0].enabled = true;
        this.mutedVideo = false;
      } else {
        this.$refs.userVideo.srcObject.getVideoTracks()[0].enabled = false;
        this.mutedVideo = true;
      }
    },

    stopStreamedVideo(videoElem) {
      const stream = videoElem.srcObject;
      const tracks = stream.getTracks();
      tracks.forEach((track) => {
        track.stop();
      });
      videoElem.srcObject = null;
    },
    endCall() {
      // if video or audio is muted, enable it so that the stopStreamedVideo method will work
      if (!this.mutedVideo) this.toggleMuteVideo();
      if (!this.mutedAudio) this.toggleMuteAudio();
      this.stopStreamedVideo(this.$refs.userVideo);
      if (this.authuserid === this.videoCallParams.caller) {
        this.videoCallParams.peer1.destroy();
      } else {
        this.videoCallParams.peer2.destroy();
      }
      this.videoCallParams.channel.pusher.channels.channels[
        "presence-presence-video-channel"
      ].disconnect();

      setTimeout(() => {
        this.callPlaced = false;
      }, 3000);
    },
  },
};
</script>

<style scoped>
#video-row {
  width: 700px;
  max-width: 90vw;
}

#incoming-call-card {
  border: 1px solid #0acf83;
}

.video-container {
  width: 700px;
  height: 500px;
  max-width: 90vw;
  max-height: 50vh;
  margin: 0 auto;
  border: 1px solid #0acf83;
  position: relative;
  box-shadow: 1px 1px 11px #9e9e9e;
  background-color: #fff;
}

.video-container .user-video {
  width: 30%;
  position: absolute;
  left: 10px;
  bottom: 10px;
  border: 1px solid #fff;
  border-radius: 6px;
  z-index: 2;
}

.video-container .partner-video {
  width: 100%;
  height: 100%;
  position: absolute;
  left: 0;
  right: 0;
  bottom: 0;
  top: 0;
  z-index: 1;
  margin: 0;
  padding: 0;
}

.video-container .action-btns {
  position: absolute;
  bottom: 20px;
  left: 50%;
  margin-left: -50px;
  z-index: 3;
  display: flex;
  flex-direction: row;
}

/* Mobiel Styles */
@media only screen and (max-width: 768px) {
  .video-container {
    height: 50vh;
  }
}
</style>

Breakdown of the video-chat component.

We import Peer from simple-peer which is the package that makes interacting with webrtc easier for us.
The component accepts the following props:

allusers: All registered users excluding the currently authenticated user. These users will be displayed. We don't want to permit the authenticated user to call oneself.

authuserid: The id of the authenticated user.

turn_url: The URL of your turn server to be used in an instance of simple-peer i.e Peer.

turn_username: Username from TURN Server.

turn_credential: The password for the turn_username.
When the component is mounted we subscribe to the presence-video-channel with the initializeChannel method. We use Laravel-echo for that.
We initializeCallListeners on the channel we subscribed to. There are methods provided by Laravel-echo to know how many users have subscribed to the channel, users who are joining and users leaving the channel. We also listen to the StartVideoChat event on the presence-video-channel for incoming calls.
We list all the users in the database from the allUsers prop and indicate whether they are online or not. Online means they have also subscribed to the presence-video-channel. This will take effect on whichever page you place the video-chat component. In this tutorial, we have a video-chat page where we place the component.
placeVideoCall is used to make a call. We pass the id and name of the user being called as parameters.
We ask the user to grant the browser access to the microphone and camera with getMediaPermission method. The streaming data is displayed in the browser. The caller now sees their face in the browser.
We create a Peer for the caller. When there is a signalling event, peer.on('signal',..) we send the signalling data to the /video/call-user endpoint on our backend.
The recipient receives the incoming call notification and when they accept the call, we signal the caller with the caller with an answer signal.

The peer.on('stream',...)listener receives the streaming data which is displayed on the recipients part in the browser.
acceptCall method is used to accept an incoming call. When a user sees an incoming call notification, they click on the accept button. We signal the receiver with the signal data received from the caller.

We get permission to access the camera and microphone and display the streaming data on our UI.

This creates a second instance of the Peer with the initiator property set to false to indicate that the new Peer is a receiver.

We hit the accept-call endpoint and send our signalling data (an answer) to the caller.

When the streaming starts, we display the caller's stream in the browser as well and now communication continues without hitting our backend but through the websocket powered by pusher.

The remaining functions are used for muting audio, disable video stream and to end the call.

Vue.component("video-chat", require("./components/VideoChat.vue").default);

Create the video chat view in resources/views/video-chat.blade.php

@extends('layouts.app')

@section('content')
    <video-chat :allusers="{{ $users }}" :authUserId="{{ auth()->id() }}" turn_url="{{ env('TURN_SERVER_URL') }}"
        turn_username="{{ env('TURN_SERVER_USERNAME') }}" turn_credential="{{ env('TURN_SERVER_CREDENTIAL') }}" />
@endsection

Update env variables. Insert your Pusher API keys

BROADCAST_DRIVER=pusher

PUSHER_APP_ID=
PUSHER_APP_KEY=
PUSHER_APP_SECRET=
PUSHER_APP_CLUSTER=

TURN_SERVER_URL=
TURN_SERVER_USERNAME=
TURN_SERVER_CREDENTIAL=

Credits

I found a lot of resources beneficial which I cannot share all over here, but the following YouTube videos helped in my understanding and arriving at this implementation.

I'd like to hear your thoughts about how easy it is to follow through with this article.

Update

I just published a Live streaming implementation with WebRTC. Check it out over here:

Live stream with WebRTC in your Laravel application

Kofi Mupati ・ Mar 23 '21

#webrtc #livestreaming #laravel #vue

SETTING UP DISCOURSE ON A SUBDOMAIN

Kofi Mupati — Wed, 01 Apr 2020 22:43:36 +0000

Introduction

Discourse is a great platform for creating discussion forums for different purposes. It is an Open Source project and can be self-hosted. You can check out the various pricing packages offered by Discourse on their Cloud Hosting.
This tutorial was greatly inspired by the following articles:

I could not get it to run by just following the tutorials. I encountered some issues because I already had a lot of websites on various subdomains hosted on my server. This is to explain any similarity that is seen in my article to address any plagiarism concern.

Prerequisites

You should own a server. I used my droplet on Digital Ocean for this.
Have Nginx installed and configured on the server. Learn how to do it over here.
Have a personal email/ mail server attached to your domain. In this tutorial, we will use Zoho mail. It is affordable and provides 5 free e-mails albeit, with some limitations.

Create Subdomain on Digital Ocean

Create the subdomain you want to use for the discussion forum you are creating with Discourse on your existing droplet.
The one I will use in this tutorial is sme.ek-brandconsult.com

Stop The Web server - Nginx

sudo systemctl stop nginx

I had to stop the webserver because the configurations in the Discourse image was conflicting with Nginx.

Download and Install Discourse as a `sudo` user

sudo git clone https://github.com/discourse/discourse_docker.git /var/discourse
cd /var/discourse 
sudo ./discourse-setup

Go through the installation process and answer the questions that are posed.

Hostname for your Discourse? [sme.ek-brandconsult.com]: 
Email address for admin account(s)? [forum@ek-brandconsult.com]: 
SMTP server address? [smtp.zoho.com]: 
SMTP port? [587]: 
SMTP user name? [forum@ek-brandconsult.com]: 
SMTP password? [password_of_the_email]: 
Let's Encrypt account email? (ENTER to skip) [me@domain.com]:

After the installation, open the subdomain in your browser. In my case, sme.ek-brandconsult.com. You should see a welcome screen that walks you through the registration process.

The Challenge

I faced an issue where the activation email that is supposed to be sent through never hits my inbox.
To fix this, we stop the running container and edit the /var/discourse/containers/app.yml file. Locate the lines

Stop the docker container the Discourse installation is running on.

cd /var/discourse
sudo ./launcher stop app

Change the port number that is exposed in the docker config. Note the port number. It will also be used in setting up Reverse Proxy on Nginx virtual host for the subdomain.

  - "80:80"   # http
changes to
  - "3035:80"   # http

Set the notification email in the config to be used for the initial setup.

#- exec: rails r "SiteSetting.notification_email='mail@domain.com'"
changes to
- exec: rails r "SiteSetting.notification_email='forum@ek-brandconsult.com'"

Set Up Virtual Host for the Subdomain.

Start Nginx once again.

sudo systemctl start nginx

Create a file /etc/nginx/sites-available/sme.ek-brandconsult.com. Replace sme.ek-brandconsult.com with your subdomain.
Place these configs in these files and save them.

server {
 listen 80; listen [::]:80;
 server_name sme.ek-brandconsult.com;
 return 301 https://$host$request_uri;

 include /etc/nginx/ssl/ssl-params.conf;
}

server {
  listen 443 ssl http2;  
  server_name sme.ek-brandconsult.com;

  location / {
    proxy_pass http://sme.ek-brandconsult.com:3035/;
    proxy_set_header Host $http_host;
    proxy_http_version 1.1;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_redirect http://sme.ek-brandconsult.com:3035/ https://sme.ek-brandconsult.com;
  }
}

Contents of /etc/nginx/ssl/ssl-params.conf

    ssl_session_cache shared:SSL:10m;
    ssl_session_tickets off;

    ssl_ecdh_curve secp384r1;

    resolver 8.8.8.8 8.8.4.4 valid=300s;
    resolver_timeout 5s;

    add_header Strict-Transport-Security "max-age=604800";

    add_header X-Content-Type-Options nosniff;

    ssl_session_timeout  10m;
    ssl_stapling on;
    ssl_stapling_verify on;

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_prefer_server_ciphers on;
    ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA';

Activate the virtual host you created and check whether Nginx settings are correct.

sudo ln -s /etc/nginx/sites-available/sme.ek-brandconsult.com /etc/nginx/sites-enabled/
sudo nginx -t

Rebuild Discourse App with the new configurations

After editing the /var/discourse/container/app.yml file, we have to rebuild our Discourse App.

cd /var/discourse
sudo ./launcher rebuild app

Complete the Installation

Open the subdomain (https://sme.ek-brandconsult.com in my case) in the browser and complete the installation.
This time the activation email should hit your inbox and you should be able to complete the installation.

It is advised that the notification email that you set in the /var/discourse/container/app.yml be commented out after successful installation and rebuild the app.

- exec: rails r "SiteSetting.notification_email='forum@ek-brandconsult.com'"
changes back to
#- exec: rails r "SiteSetting.notification_email='mail@domain.com'"

This is the only way I was able to successfully set up the biggest discussion forum for SMEs in Ghana; SME GHANA after struggling to use the various tutorials that are available.

Further additions - Configuring SSL

YOu can further configure SSL for your subdomain using Let's Encrypt

Import CSV Data Into SQL Databases From The Terminal

Kofi Mupati — Sat, 07 Mar 2020 18:12:11 +0000

The Data Scientist and several professionals in the same space work with data from several sources. It does not take long for one to face the challenge of importing data from a CSV file into an SQL database. I am a great fan of the terminal so I want to show you how to do it from the terminal.

Prerequisites

I am going to use a Python Package for this. I suggest that you work in a virtual environment. The Real Python's article on virtual environments is a good resource if need be. I assume you are using Python3.

Install the needed packages

pip install PyMySQL psycopg2 csvkit

PyMySQL: This is a pure Python MySQL driver which helps us to connect to a MySQL database using Python.
psycopg2: This the most popular Postgresql database adapter for Python.
csvkit: It is a suite of command-line tools for converting to and working with CSV. It provides the csvsql module we are going to use to import the data from our CSV file.

Download the sample CSV file,customer_info.csv.

For the sake of this tutorial, I have uploaded a CSV file which contains some customer information you can use to test this tool I am introducing. Click this link to download.

Create a new MySQL or Postgresql database. I named mine test. The data will be imported into a table I have named customer_info.

Importing Your Data

The Quick Approach.

This approach automatically generates the Schema from the CSV data, creates the table and inserts the data into the table.
db_user: This is the database user with access to the test database you created.
password: This is the password of the user.

For MySQL

csvsql --db 'mysql+pymysql://db_user:password@localhost/test'  --tables customer_info --insert customer_info.csv

For Postgresql

csvsql --db 'postgresql:///test' --tables customer_info  --insert customer_info.csv

The Lengthier (and Safer) Approach.

Generate the query for creating the customer_info table from your terminal.

For MySQL

csvsql -i mysql customer_info.csv

Expected Result:

CREATE TABLE customer_info (
    `contractId` DECIMAL(38, 0) NOT NULL, 
    `AccountNumber` DECIMAL(38, 0) NOT NULL, 
    `CreatedAt` TIMESTAMP NULL, 
    `Amount` DECIMAL(38, 0) NOT NULL, 
    `Count` DECIMAL(38, 0) NOT NULL, 
    `Duration` DECIMAL(38, 0) NOT NULL
);

The result above is the query we are going to use to create the customer_info table in the test database. You can edit the generated schema to suit your need. I will leave it as it is in this tutorial.

For Postgresql

csvsql -i postgresql

Expected Result:

CREATE TABLE customer_info (
    "contractId" DECIMAL NOT NULL, 
    "AccountNumber" DECIMAL NOT NULL, 
    "CreatedAt" TIMESTAMP WITHOUT TIME ZONE, 
    "Amount" DECIMAL NOT NULL, 
    "Count" DECIMAL NOT NULL, 
    "Duration" DECIMAL NOT NULL
);

NOTE THE DIFFERENCE IN HOW EACH DATABASE EXPECTS THE QUERY TO BE. MySQL is wrapped in single quotes but Postgresql is wrapped in double-quotes.

Import your data into the table.

In MySQL

csvsql --db 'mysql+pymysql://db_user:password@host/test' --no-create  --insert customer_info.csv

In Postgresql

csvsql --db 'postgresql:///test' --no-create  --insert customer_info.csv

SUMMARY

We just learnt how to import data from a CSV file into tables in our MySQL and Postgresql databases with the csvkit package.You can take the direct approach which automatically creates your table and inserts the data.The longer and in my opinion safer approach is to generate the query that creates the table, inspect and edit the generated schema and import your data into the table.

CSVKIT is a powerful tool every Data Scientist should have in his or her toolbox.

Optimize Background Images Sourced From Storyblok in Gatsby

Kofi Mupati — Sat, 07 Mar 2020 11:47:56 +0000

In this tutorial, I am going to show you how to serve optimized background images sourced from Storyblok (The Only Headless CMS with a Visual Editor). I assume you are familiar with Gatsby and Storyblok. If you are not familiar with these, visit this link; The Complete Guide to Build a Full-Blown Multilanguage Website with Gatsby.js to get up to speed.

The Challenge

The gatsby-image plugin is what is generally used to serve optimized images in Gatsby. The challenge, however, is the fact that gatsby-image plugin (as at the time of writing) works for images from the file system. This poses a challenge to the various Headless CMS available.

When your content comes from the Headless CMS you lose the optimization of your images if the CMS does not provide a way for you to utilize the gatsby-image plugin. The gatsby-storyblok-image package helps us when using Storyblok.

One faces an extra hurdle when you want to use an optimized background image. Luckily for us, we already have the gatsby-background-image plugin helps with background images.

The Proposed Solution

We are going to combine the gatsby-background-image and gatsby-storyblok-image plugins to achieve this. This implementation is what I used in my blog. The blog is open-sourced and can be downloaded over here. You can get the full version from there.

Install the necessary dependencies

yarn add gatsby-background-image gatsby-storyblok-image styled-components

Create helper functions in a file src/utils/gatsbyImageTransform.js to use the fluid and fixed queries provided by gatsby-storyblok-image

import { getFixedGatsbyImage, getFluidGatsbyImage } from "gatsby-storyblok-image"
export const fluidImage = (image, params) => {
  const fluidProps = getFluidGatsbyImage(image, { ...params })
  return fluidProps
}
export const fixedImage = (image, params) => {
  const fixedProps = getFixedGatsbyImage(image, { ...params })
  return fixedProps
}

Create a hero component, hero.js that uses the helper function to serve optimized background images

import React from "react"
import SbEditable from "storyblok-react"
import styled from "styled-components"
import BackgroundImage from "gatsby-background-image"

import { fluidImage } from "../../utils/gatsbyImageTransform"

const Hero = props => {
  let optimizedFluidImage = fluidImage(props.blok.banner)
  return (
    <SbEditable content={props.blok}>
      <Wrapper fluid={optimizedFluidImage}>
        <Overlay />
        <Heading>{props.blok.heading}</Heading>
        {props.path === "index" && props.blok.description && (
          <Description>{props.blok.description}</Description>
        )}
      </Wrapper>
    </SbEditable>
  )
}
export default Hero
const Wrapper = styled(props => <BackgroundImage {...props} />)`
  position: relative;
  display: flex;
  flex-direction: column;
  justify-content: center;
  align-items: center;
  text-align: center;
  height: 50vh;
  margin-bottom: 2em;
  }
`
const Heading = styled.h2`
  z-index: 2;
  color: #06c4d1;
  font-size: 5em;
  @media (max-width: 768px) {
    font-size: 2.3em;
  }
`
const Description = styled.p`
  color: #fff;
  z-index: 2;
  font: 400 25px/33px "Montserrat";
  @media (max-width: 768px) {
    font-size: 15px;
  }
`
const Overlay = styled.div`
  position: absolute;
  top: 0px;
  left: 0px;
  width: 100%;
  height: 100%;
  background-color: rgb(19, 17, 16);
  opacity: 0.7;
`

The helper function is what provides the image data: fluidImage and fixedImage that is supplied to the BackgroundImage component which is later used as the background image for the hero component.

A Quick Recap

We looked at how to serve optimized background images to the visitors of our website. It involved 2 open-sourced packages; the gatsby-background-image and gatsby-storyblok-image packages. We use the former to get the image data that is used originally in the gatsby-image plugin and supply this data to the gatsby-background-image plugin to be able to achieve the optimization we need.

I'd like to hear your views about this approach and the alternatives you might have used in your project. The gatsby-storyblok-image package is also quite young so the open-source community will be very helpful in bringing all the features gatsby-image provides.