DEV Community: Arnelle Balane

Let's build a video calling app! - Part One

Arnelle Balane — Thu, 06 Oct 2022 16:27:11 +0000

Introduction

In the past two years or so we've been kind of forced into using video calling apps as we transitioned into work-from-home setups. Services like Google Meet and Zoom, among others, have been instrumental in easing this transition, allowing us to continue performing our work, attending our classes, and generally connecting with other people.

In this first article of a two-part post, we're going to take a look at how we can build a basic video calling app using Web technologies including WebRTC. In the second part, we will integrate Firebase into the app to make it support multiple calls/meetings with multiple participants.

This article will not be focusing on "building the next Zoom app" (although I won't stop you from doing so!) but instead on exploring and experimenting with the fundamental concepts of how video calling apps work in general, so we can build whatever we want on top of it.

Concepts

Video calling apps use WebRTC under the hood for peer-to-peer communication of streaming data between browsers (also known as peers). These apps need to do several things:

Get streaming audio and video data
Determine network information, such as IP addresses and ports to use
Determine information about media capabilities, such as resolutions and available media codecs
Coordinate communications to initiate or close connections, and to exchange network and media capabilities information
Stream audio and video data

MediaStream API

For #1 on our list, we can use the MediaStream API in order to get the audio and video data that we can stream to our peers. We can obtain a media stream in different ways:

navigator.mediaDevices.getUserMedia() to get a media stream from media input devices such as the user's camera and microphone
navigator.mediaDevices.getDisplayMedia() to get a media stream from a display device, useful for a screen-sharing functionality
canvasElement.captureStream() to get a media stream of an HTML canvas element

A MediaStream object represents a stream of media content, and can contain multiple "tracks". For example, we can have a media stream that contains a video track from the camera and a separate video track from the microphone.

Additionally, we can obtain media streams with certain constraints, such as if it should only contain audio or video or both, what video resolution to get, which camera device to use (front or rear camera, or a specific device), etc. We won't get into all of these options in this post, but all of them are described in the linked documentation.

As an example for this article, let's get a media stream containing video and audio tracks:

const mediaStream = await navigator.mediaDevices.getUserMedia({
  video: true,
  audio: true,
});

RTCPeerConnection API

The primary way for using WebRTC is through the RTCPeerConnection API, which represents a connection between two peers. Let's say you want to call your friend. On your end you will need to create a new RTCPeerConnection object, and you are considered the local peer while your friend is the remote peer (conversely, from your friend's end they will also create a new RTCPeerConnection object where they are the local peer and you are the remote peer).

const peerConnection = new RTCPeerConnection();

In order to stream our media stream to the other peer once our peer connection is established (#5 on our list), we need to add all of its tracks into the peer connection:

mediaStream.getTracks().forEach((track) => {
  peerConnection.addTrack(track, mediaStream);
});

Interactive Connectivity Establishment (ICE)

Next, for #2 on our list, each peer needs to determine how the other peer can connect to it. A technique called Interactive Connectivity Establishment or ICE is used to negotiate the best way to connect the peers.

Initially, ICE will try to connect peers directly through UDP to achieve the lowest possible latency. If UDP fails, ICE will try TCP. In both cases, a peer's public IP address and port needs to be determined, which is the task of a STUN (Session Traversal Utilities for NAT) server. A peer will basically ask a STUN server what its public IP address and port are, and the STUN server will respond with the the IP address and port that it sees when the peer connected to it.

When direct connection is not possible, for example because of NAT traversals and firewalls, ICE will try to use an intermediary TURN (Traversal Using Relay NAT) server to relay streaming data. This process of determining IP addresses and ports is usually referred to as "finding candidates", and each viable option that can be used to establish a peer-to-peer connection is called an "ICE candidate".

Image from https://web.dev/webrtc-basics: Get started with WebRTC

Now that's a lot of terms and acronyms and servers to remember right now, but thankfully in our code it's as simple as passing additional options to our RTCPeerConnection object, and the WebRTC API will take care of all the complexities of finding ICE candidates. Let's create an RTCPeerConnection that uses one of Google's free STUN servers:

const peerConnection = new RTCPeerConnection({
  iceServers: [
    {
      urls: ['stun:stun.l.google.com:19302'],
    },
  ],
});

Session Description Protocol (SDP)

Earlier we added our media stream to the peer connection, now we can do #3 on our list and determine the media capabilities that our browser supports. This information (together with the network information) are described in a format called the Session Description Protocol (SDP).

Here's an example of what an SDP for our peer might look like. In here, we can see details about what media types we are going to stream, available ICE candidates, and supported media codecs.

v=0
o=- 6273550205898152643 4 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 2
a=extmap-allow-mixed
a=msid-semantic: WMS i5NpLCJMg7hiYh1kG5YyR6EIvxcfO8QgZ3OA
m=video 60159 UDP/TLS/RTP/SAVPF 96 97 102 122 127 121 125 107 108 109 124 120 39 40 45 46 98 99 100 101 123 119 114 115 116
c=IN IP4 49.145.37.142
a=rtcp:9 IN IP4 0.0.0.0
a=candidate:4077567720 1 udp 2122260223 192.168.1.10 65452 typ host generation 0 network-id 1 network-cost 10
a=candidate:85641020 1 udp 1686052607 49.145.37.142 60159 typ srflx raddr 192.168.1.10 rport 65452 generation 0 network-id 1 network-cost 10
a=candidate:3179889176 1 tcp 1518280447 192.168.1.10 9 typ host tcptype active generation 0 network-id 1 network-cost 10
a=ice-ufrag:PUyC
a=ice-pwd:eJeVV3MKP5+MDfPYAA6WUcKV
a=rtpmap:96 VP8/90000
a=rtpmap:108 H264/90000

Note that I had to remove a lot of lines from the actual SDP to keep it concise for illustration purposes, but a real SDP contains so much more information than this. Thankfully we don't need to deal directly with SDP, as the WebRTC API already handles this for us.

So now we're probably wondering, how do we obtain an SDP, and how do we share it to the other peer?

Signaling

What brings all of these together is the process called signaling. This is how one peer communicates its intent to call the other peer, and how both peers exchange their SDP containing their network and media information. This is also the part of our video calling app where we write the most code for, since signaling methods and protocols are not specified by WebRTC.

To implement a signaling mechanism (#4 on our list), we can choose any two-way communication channel. This could be implemented using WebSockets, or a combination of Fetch API and Server-Sent Events. This can also be done using Firebase Cloud Firestore as we will see in the second part of this post. For this article, we'll keep it simple and use the [BroadcastChannel API](https://developer.mozilla.org/en-US/docs/Web/API/Broadcast_Channel_API) so we can let two local browser tabs call each other. First let's create our channel for signaling:

const signaling = new BroadcastChannel('signaling');

Now let's say you want to initiate a call to your friend. The flow is going to be as follows:

You create what we call an offer (which is simply the SDP describing your browser's capabilities and ICE candidates) and set it as the local peer in your RTCPeerConnection object.

   // YOU
   const offer = await peerConnection.createOffer();
   await peerConnection.setLocalDescription(offer);
   // to be continued in the next step...

You then send your offer to your friend through the signaling mechanism.

   // YOU
   signaling.postMessage({
     type: 'offer',
     payload: offer.toJSON(),
   });

Your friend then receives your offer through the signaling mechanism.

   // YOUR FRIEND
   signaling.addEventListener('message', async (event) => {
     const { type, payload } = event.data;
     if (type === 'offer') {
       // to be continued in the next step...
     }
   });

Your friend then creates their own RTCPeerConnection object and sets your offer as their remote peer.

   // YOUR FRIEND
   const peerConnection = new RTCPeerConnection();
   await peerConnection.setRemoteDescription(payload); // payload contains your offer
   // to be continued in the next step...

Your friend then creates what we call an answer (which again is simply the SDP describing their browser's capabilities and ICE candidates) and sets it as the local peer in their RTCPeerConnection object.

   // YOUR FRIEND
   const answer = await peerConnection.createAnwer();
   await peerConnection.setLocalDescription(answer);
   // to be continued in the next step...

Your friend then sends their answer to you through the signaling mechanism.

   // YOUR FRIEND
   signaling.postMessage({
     type: 'answer',
     payload: answer.toJSON(),
   });

Finally, you then receive your friend's answer through the signaling mechanism and set it as the remote peer in your RTCPeerConnection object.

   // YOU
   signaling.addEventListener('message', async (event) => {
     const { type, payload } = event.data;
     if (type === 'answer') {
       await peerConnection.setRemoteDescription(payload); // payload is your friend's answer
     }
   });

And just like that, the peer-to-peer connection between you and your friend is now established!

Events

Just a couple more things to take care of to complete our video calling app, and both have to do with events that get sent to the RTCPeerConnection object.

First is the icecandidate event, which gets sent when new ICE candidates are identified and added to the local peer, which usually happens when we call the peerConnection.setLocalDescription method. When we receive such events, we need to send it to the remote peer through the signaling channel, so that the remote peer can add it to its set of remote candidates.

// YOU
peerConnection.addEventListener('icecandiate', (event) => {
  signaling.postMessage({
    type: 'candidate',
    payload: event.candidate.toJSON(),
  });
});

// YOUR FRIEND
signaling.addEventListener('message', async (event) => {
  const { type, payload } = event.data;
  if (type === 'candidate') {
    await peerConnection.addIceCandidate(payload); // payload is your ice candidate
  }
});

Second is the track event, which lets us know when a media stream from the remote peer arrives to us through the established peer connection. When we receive such events, we need to display the media stream in the page so we can see the peer we're on a call with.

peerConnection.addEventListener('track', (event) => {
  displayMediaStream(event.streams[0]);
});

Implementation

At this point we've actually already written most of the code that we need for our video calling app. All that's left to do is to just put them all together so we see the overall code and where each code snippet above falls into. To spare us from a very long code snippet in this post, please allow me to link to the file in the GitHub repository containing the complete code.

One thing that made it challenging to understand the complete flow in the beginning was the fact that we use the same JavaScript file in the caller and callee peers, which means that file contains the code for handling the signaling and event handling for both peers (sometimes if I read the file it looks like a peer is trying to connect to itself). For me what helped make it less confusing is to keep in mind in which peer (the caller or the callee, or both) a code snippet is meant to run for.

Conclusion

Congratulations, we just built a very simple video calling app! At this point it is very minimal and has several limitations. One limitation is that since we are using the BroadcastChannel API for our signaling mechanism, it means that only two browser tabs on our local computers can call each other for now. Our video calling app also currently doesn't support having multiple participants within the same video call. We will address all of these limitations in the next article where we will change our signaling mechanism to use Firebase Cloud Firestore.

In the meantime, thank you for reading this article. If you have any feedback, comments, thoughts, etc. please let me know!

Async Clipboard API: Accessing the clipboard using JavaScript

Arnelle Balane — Fri, 06 Sep 2019 00:00:00 +0000

Accessing the user's clipboard has not been the nicest thing to do for a longtime. We had to use the document.execCommandAPI to copy and paste text to and from the user's clipboard, which involves the following steps:

// #1. Use an input element
const input = document.querySelector('input');

// #2. Set the input's value to the text we want to copy to clipboard
input.value = 'hello there!';

// #3. Highlight the input's value
input.select();

// #4. Copy the highlighted text
document.execCommand('copy');

The input element can be dynamically created and removed during the process, or styled to not be visible to the user. During the times that I got to use this approach before, I always thought that it looks ugly and not very elegant.Fortunately, a new Web API is here to make this a lot easier!

Async Clipboard API

The Async Clipboard APIprovides Web apps with the ability to programmatically read from and write to the system clipboard easily. A few notes about the API:

It can be accessed at navigator.clipboard
The site needs to be served over HTTPS or localhost
Only works when the page is the active browser tab

Now let's see how simple it actually is, compared to the old way of doing it.

Writing to the clipboard

async function writeToClipboard(text) {
    try {
        await navigator.clipboard.writeText(text);
    } catch (error) {
        console.error(error);
    }
}

This method returns a Promise, which we can wait to resolve by chaining a.then() or using async/await. With that single, short line of code, we've just written our text into the clipboard!

Note: In Firefox, the text gets written to the clipboard only when calling writeText() in response to a user gesture, otherwise it throws an exception.Chrome writes the text to the clipboard regardless of the user gesture. Both allows writing to the clipboard without having to request for permission.

Reading from the clipboard

async function readFromClipboard() {
    try {
        const text = await navigator.clipboard.readText();
        console.log(text);
    } catch (error) {
        console.error(error);
    }
}

This method also returns a Promise, and is as straightforward as writing to the clipboard. The first time a site tries to read the contents of the clipboard, the browser prompts the user whether they want to allow the requestor not:

Note: In Chrome, permission to read the clipboard is automatically denied when the user has dismissed it several times (~3 times from my observation).

Note: At the time of writing, Firefox (version 68) doesn't support the readText() method yet, with the MDN docs stating that it is only supported in browser extensions.

Checking clipboard access permissions

We can check if we have permission to access the clipboard using thePermissions API:

await navigator.permissions.query({name: 'clipboard-read'});
// or 'clipboard-write' for permission to write

// sample result: {state: 'granted'}

We can use this result for example to display some UI letting the user know whether we have access to the clipboard or not.

Clipboard Events

Aside from allowing us to easily write to and read from the clipboard, theAsync Clipboard API also gives us clipboard events. We can know when the user performs a clipboard-related action such as copy, cut, or paste by listening for the copy, cut, and paste events, respectively.

document.addEventListener('copy', event => {});
document.addEventListener('cut', event => {});
document.addEventListener('paste', event => {});

These events don't fire when accessing the clipboard using the Async ClipboardAPI (i.e. through writeText() or readText()), but they do when calling their corresponding document.execCommand commands. Calling event.preventDefault() cancels the action and maintains the current state of the clipboard.

These events only fire when the action was performed on the page, and not when performed in other pages or apps.

The clipboard event objects have a clipboardData property which is aDataTransferobject. This allows us to overwrite the data that will be written to the clipboard, giving us the opportunity to write data in other formats, such as text/html:

document.addEventListener('copy', event => {
    event.preventDefault();
    event.clipboardData.setData('text/plain', 'COPY ME!!!');
    event.clipboardData.setData('text/html', '<p>COPY ME!!!</p>');
});

When doing this, we need to call event.preventDefault() so that our custom data is written to the clipboard instead of the original. For cut and paste events, we need to handle removing/inserting the content in the document ourselves.

Image Support

So far we've only seen the version of the Async Clipboard API which only supports reading/writing text, and it already looks cool! A recent addition to the API is support for images, making it easy to programatically read and write images to the clipboard!

Note: For the meantime, only PNG images are supported, but support for other image formats (and maybe files in general) will be added in the future.

Write an image to the clipboard

Before we can write an image to the clipboard, we first need to get aBlob of the image.There are several ways to obtain an image blob:

Ask the user to select the image using a file input
fetch() the image from the network as a blob (with response.blob())
Draw the image to a canvas and call canvas.toBlob()

Once we have an image blob (let's call it imageBlob), we need to create an instance of ClipboardItem containing our image Blob:

new ClipboardItem({ 'image/png': imageBlob})

The ClipboardItem constructor accepts an object whose keys are the MIME types and the values are the actual blobs themselves. We can provide multiple MIME type and blob pairs, giving different representations of the data using different types.

Now we can write our image to the clipboard using navigator.clipboard.write():

async function writeToClipboard(imageBlob) {
    try {
        await navigator.clipboard.write([
            new ClipboardItem({
                'image/png': imageBlob
            })
        ]);
    } catch (error) {
        console.error(error);
    }
}

navigator.clipboard.write() accepts an array of ClipboardItems, but at the time of writing only supports a single item. This will most likely change int he future.

Reading an image from the clipboard

Reading items (not just text) from the clipboard can be done using navigator.clipboard.read():

async function readFromClipboard() {
    try {
        const items = await navigator.clipboard.read();
    } catch (error) {
        console.error(error);
    }
}

It returns an array of ClipboardItems that mirrors the contents of the system clipboard, although currently in Chrome it only returns the latest item in the clipboard.

We can loop over this array to get each item. We can get all available MIME types in a ClipboardItem through its items property, and get the actual blob data for a specific type using its asynchronous getType() method:

for (let item of items) {
    console.log(item.types); // e.g. ['image/png']

    for (let type of item.types) {
        const blob = await item.getType(type);
    }
}

After we get the blob, we can now do anything we want with it. We can use theFileReader APIto convert the blob to appropriate formats that we want:

const reader = new FileReader();
reader.onload = () => {
    const data = reader.result;
    // e.g. 'data:image/png;base64,...'
};

reader.readAsDataURL(blob);

The Async Clipboard API's write() and read() methods provide generic ways for accessing the clipboard. In fact, the writeText() and readText()methods discussed earlier are just convenience methods for them, and can otherwise be done using write()/read() by using blobs with type text/plain.

async function writeToClipboard(text) {
    try {
        await navigator.clipboard.write([
            new ClipboardItem({
                'text/plain': new Blob([text], {type: 'text/plain'})
            })
        ]);
    } catch (error) {
        console.error(error);
    }
}

async function readFromClipboard() {
    try {
        const items = await navigator.clipboard.read();
        for (let item of items) {
            const data = item.getType('text/plain');
            // convert `data` to string using FileReader API's
            // `.readAsText(data)` method
        }
    } catch (error) {
        console.error(error);
    }
}

Browser Support & Feature Detection

The Async Clipboard API with text support shipped in Chrome 66 and FireFox 63 (with readText() not yet available for Web apps). For PNG image support, only Chrome supports it at the time of writing, shipping it in Chrome 76. See this browser compatibility tablefor more info.

We can take advantage of this API already on browsers that support through feature detection, by checking if navigator.clipboard is present.

if (navigator.clipboard) {
    // Safe to use Async Clipboard API!
} else {
    // Use document.execCommand() instead
}

Resources

Thanks for reading this article, I hope you enjoyed it and learned something from it. Here are more resources to learn more about the Async Clipboard API:

Playing the T-Rex Runner game inside a Picture-in-Picture window

Arnelle Balane — Mon, 22 Jul 2019 14:45:32 +0000

The Picture-in-Picture API is a new Web platform API that allows websites to play videos in a little floating window that stays on top of other windows even when the browser is not visible, allowing us to continue watching these videos while we interact with other websites or applications.

The API is currently limited to video elements only. Fortunately, we can also create video streams from canvas elements. This means that we can draw anything on a canvas and have it appear on a Picture-in-Picture window. Experimenting with this eventually led me into trying to make Chrome's offline T-Rex runner game playable inside a Picture-in-Picture window (click here to play it now).

// Detect dark theme var iframe = document.getElementById('tweet-1134838128127012864-376'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1134838128127012864&theme=dark" }

Get the game's source code

We can get the game's source code from the Chromium repository. We copy the contents in that location, rename some of the files, and clean up the HTML to only contain this necessary markup inside the <body>:

<!-- This will contain the canvas element -->
<div class="interstitial-wrapper"></div>

<!-- Game assets: sprits + audio -->
<div id="offline-resources">
  <img id="offline-resources-1x" src="images/100-percent/100-offline-sprite.png">
  <img id="offline-resources-2x" src="images/200-percent/200-offline-sprite.png">

  <div id="audio-resources">
    <audio id="offline-sound-press" src="sounds/button-press.mp3"></audio>
    <audio id="offline-sound-hit" src="sounds/hit.mp3"></audio>
    <audio id="offline-sound-reached" src="sounds/score-reached.mp3"></audio>
  </div>
</div>

<!-- The main game script -->
<script src="offline.js"></script>

<!-- Initialize the canvas and the game. -->
<!-- Originally performed inside neterror.js -->
<script>
  const runner = new Runner('.interstitial-wrapper');
</script>

This gives us an exact replica of the offline T-Rex runner game which we can already play with:

Implement programmatic T-Rex jump

The T-Rex jumps every time we press on the spacebar. Let's add a way to programatically make our T-Rex jump without us having to actually press the spacebar.

Digging through the game's code, we will find two methods on the runner object that handles the spacebar key:

onKeyDown, makes the T-Rex jump while the game is running.
onKeyUp, restarts the game when the T-Rex crashed into an obstacle.

Let's write a method that calls either of these based on the state of the game, passing in a dummy keyboard event.

function simulateSpacebar() {
  const keyboardEventOptions = {
    code: 'Space',
    keyCode: 32,
  };

  if (runner.crashed) {
    const event = new KeyboardEvent('keyup', keyboardEventOptions);
    runner.onKeyUp(event);
  } else {
    const event = new KeyboardEvent('keydown', keyboardEventOptions);
    runner.onKeyDown(event);
  }
}

Capture video stream of canvas contents

Calling new Runner('...') creates a canvas element and inserts it into the page. We need to get a reference to that canvas element, and then capture its contents as a video stream:

const canvas = document.querySelector('canvas');
const videoStream = canvas.captureStream();

We then create a video element with the video stream as the source:

const video = new Video();
video.srcObject = videoStream;

video.muted = true;
video.play();

Here we're also muting the video so we can autoplay it (see Chrome's autoplay policy).

Show the Picture-in-Picture window

When using new Web APIs like Picture-in-Picture, always feature-detect if they are available before trying to use them. This makes sure that our apps don't break when the API is not available, and only progressively enhance the experience when it is available. For Picture-in-Picture, it can be done by checking the document.pictureInPictureEnabled property:

const button = document.querySelector('button');

if (document.pictureInPictureEnabled) {

  // Picture-in-Picture is available!
  // Subsequent code snippets will be place inside this block.

} else {
  // Picture-in-Picture is not available. User can still play the game normally in the page.

  button.textContent = 'Picture-in-Picture is not available';
  button.disabled = true;
}

We also add a <button> element to the page, which the user can click to
enter Picture-in-Picture. We want to give this control to our users, usually through a Picture-in-Picture icon in the UI, so that they can decide when they want to view our content in the Picture-in-Picture window.

Now to the fun part, let's show our video stream in the Picture-in-Picture window when the button is clicked!

button.addEventListener('click', async () => {
  simulateSpacebar();
  await video.requestPictureInPicture();
});

The result looks like this:

Implement game controls

The Picture-in-Picture window can stay on top of other application windows, and in that case we won't be able to press the spacebar key on the page to make the T-Rex jump, so we need another way to make it do that.

Media Session API to the rescue!

The Media Session API allows websites to customize media notifications, as well as define event handlers for playback controls (e.g. play, pause, etc.). We can make our T-Rex jump whenever we press the play/pause buttons on our keyboards (or other devices that can control media playback) by defining play and pause event handlers.

navigator.mediaSession.setActionHandler('play', simulateSpacebar);
navigator.mediaSession.setActionHandler('pause', simulateSpacebar);

The Picture-in-Picture API integrates well with the Media Session API. When we define the playback event handlers, the Picture-in-Picture window will also display their corresponding action buttons.

Let's play!

With all of those changes in place, we've now made the T-Rex Runner game
playable inside a Picture-in-Picture window, using our play/pause media buttons to make the T-Rex jump!

You can find the live demo of this project, as well as the complete source code, in the following links:

Conclusion

In this article we were able to use the Picture-in-Picture API and Media
Session API to build something silly. There are more serious and useful uses for these APIs- Youtube has a hidden Picture-in-Picture button in their player controls, and before I worked on this experiment, I also built a demo on displaying audio visualization inside a Picture-in-Picture window using the same techniques in this article.

If you've built / are currently working on something that uses these APIs, or see some really amazing uses of them in the wild, please share them with us in the comments, I'd love to hear about them!

Resources

The Picture-in-Picture API
Watch video using Picture-in-Picture by François Beaufort
Customize Media Notifications and Handle Playlists (Media Session API) by François Beaufort

Setting up multiple GitHub accounts, the nicer way

Arnelle Balane — Fri, 08 Mar 2019 10:43:53 +0000

This article was originally published in our UncaughtException blog. Read this article in there by following this link.

Cover photo by Brina Blum on Unsplash.

I previously wrote about setting up multiple GitLab accounts, which works by relying mostly on SSH config files and using custom hosts as remotes for git repositories. You can read it by following this link: Setting up multiple GitLab accounts

The previous setup works, but I find it inconvenient having to use custom hosts when cloning repositories or for git remotes all the time, like:

$ git clone git@gitlab.com-work:work/repository.git

I recently (almost) had to deal with the same scenario, on GitHub this time (hence the title), so I decided to check if I can achieve a better setup. Turns out that there’s a better way to do it — one that does not rely on custom hosts or modifying any SSH config file.

In this article, we are going to assume that we have two GitHub accounts, personal@email.com and work@email.com. When working on work-related projects we want to commit and push our changes using our work account, while using our personal account for everything else.

We’re also going to assume that we’ve already defined global configs telling git to use our personal account. See this guide on how to do that.

Generate SSH keys for each user

GitHub does not allow us to use the same SSH key in multiple accounts, so we’ll have to create separate keys for each account. We can create SSH keys and add them to our SSH agent by following this guide from the GitHub Documentation.

Once we’re done, we will have two sets of SSH keys, e.g.:

~/.ssh/id_rsa and ~/.ssh/id_rsa.pub (let’s use this for our personal account)
~/.ssh/work_key and ~/.ssh/work_key.pub

Now add the SSH keys to their corresponding GitHub accounts. This guide shows us how to do that.

Committing as the correct user

We then modify our ~/.gitconfig file to use our work account when we are inside a work-related repository. We are going to use “Conditional Includes” for that, which dynamically adds/removes configuration based on certain conditions. Let’s add the following snippet to our ~/.gitconfig file:

[includeIf "gitdir:/path/to/work/repository/"]
    [user]
        email = "work@email.com"

This makes git override our global configs and use work@email.com when committing changes inside the work repository. When in any other repository, our personal account will be used.

Do this for all work repositories

The way I personally structure my repositories on my file system is I place all work-related repositories inside the ~/work directory. This makes it easier to enable the configs above for all work-related repositories by just using the work directory in the includeIf directive:

[includeIf "gitdir:~/work/"]

Pushing as the correct user

When we push our commits now, they will be pushed to our personal GitHub account, even though we’ve already committed them using our work email. That’s because under the hood, git uses SSH, and SSH uses the id_rsa key by default. So when GitHub receives the push, it will try to determine which user pushed the commits based on the SSH key used, which will turn out to be our personal account.

We can resolve this by using the work_key SSH key when pushing to our work repositories. In SSH, this can be done through the -i flag, like so:

$ ssh -i ~/.ssh/work_key ...

We can configure git to use a custom SSH command like above by setting core.sshCommand. And since we only want to set it when we’re inside any work repository, we also conditionally include it inside the includeIf directive:

[includeIf "gitdir:/path/to/work/repository/"]
    [user]
        email = "work@email.com"
    [core]
        sshCommand = "ssh -i ~/.ssh/work_key"

These configs now make git commit using our work email, as well as make GitHub associate commits with our work account when we push them 🎉.

Summary

We’ve managed to configure git to use different configs when committing and associating these commits to different GitHub accounts when pushing, based on what repository we are performing those changes — and we used conditional includes in git’s config file to achieve that.

I like this way better than the previous setup since it’s more convenient to use (no more custom hosts) and all the config changes only affect git (no more changing of SSH config files).

If you deal with these multiple GitHub accounts in a different way, please feel free to share with us how you do it as well!

Building a site navigation menu using React Hooks

Arnelle Balane — Sun, 20 Jan 2019 13:23:22 +0000

I’m currently learning React, and since I learn better by building stuff, I decided to rebuild my personal website with it. It’s still a work in progress but there is one component which I found interesting to build: the site’s navigation menu.

It’s just a simple menu, and I only have two requirements for it:

The user needs to be able to toggle its state to open or close
It should close when the user navigates to a different page

Initial setup

I initially built a static version of the site, composed of the top-level App component, a Header component, and a Menu component. The App component looks like this:

// App.jsx

import Header from './Header.jsx';

function App(props) {
    const isMenuOpen = false;

    return (
        <div>
            <Header isMenuOpen={isMenuOpen} />

            {/\* Other stuff \*/}
        </div>
    );
}

As shown in the code snippet, the App component has an isMenuOpen variable which it passes to Header as the isMenuOpen prop. The Header in turn passes the same isMenuOpen prop to Menu. The value of this variable controls whether Menu should be shown or hidden.

`isMenuOpen` component state

Initially, isMenuOpen is just a variable whose value I manually change to update the UI. This is okay for the initial static version of the app, but I don’t really want that on the actual app. I want the component to keep track of this variable, modify its value in response to a user action (e.g. a click on the toggle menu button), and re-render the UI based on its new value.

To achieve that, I need to make isMenuOpen an actual state on the App component. Normally this would be done by converting App from a functional component into a class component. This is because functional components can’t have state while class components can. If I follow this approach, the App component will become:

// App.jsx

class App extends React.Components {
    constructor(props) {
        super(props);
        this.state = {
            isMenuOpen: false
        };
        this.handleToggleMenu = this.handleToggleMenu.bind(this);
    }

    handleToggleMenu() {
        this.setState(state => ({
            isMenuOpen: !state.isMenuOpen
        }));
    }

    render() {
        return (
            <div>
                <Header 
                    isMenuOpen={this.state.isMenuOpen} 
                    onToggleMenu={this.handleToggleMenu}
                />

                {/\* Other stuff \*/}
            </div>
        );
    }
}

I would have done it this way, but then it just happened that I just recently read about React Hooks from the docs.

React Hooks gives us access to features such as states and lifecycle methods without having to use class components (in fact, they should only be used in functional components). It seemed like I had an opportunity to use React Hooks for my navigation menu so I decided to try it out.

Make sure to use the right React version

At the time of writing, React Hooks is still on preview, and is only available in React v16.8.0-alpha.0. I had to update the corresponding packages to use the right versions:

npm install react@16.8.0-alpha.0 react-dom@16.8.0-alpha.0

Using the `useState` hook

With the correct versions of react and react-dom installed, I can now start using React Hooks. Since I want to use states in my functional App component, I used React’s built-in useState hook.

import {useState} from react;

Then used it to initialize the isMenuOpen state:

function App(props) {
    const [isMenuOpen, setIsMenuOpen] = useState(false);
}

The useState hook accepts one argument which is the initial value to set the state to, and returns an array of two things: the current state value and a function used to update the state value.

And just like that, I now have a reactive isMenuOpen state with just very minimal changes in the code. I was able to use state in my component while keeping it as a functional component. In fact, to me it still kinda looks like I’m just declaring the isMenuOpen variable from the static version of the component. The complete App component now looks like:

// App.jsx

function App(props) {
    const [isMenuOpen, setIsMenuOpen] = useState(false);

    return (
        <div className={style.wrapper}>
            <Header
                isMenuOpen={isMenuOpen}
                onToggleMenu={() => setIsMenuOpen(!isMenuOpen)}
            />

            {/\* Other stuff \*/}
        </div>
    );
}

Detecting page navigations

At this point the navigation menu already opens and closes when clicking on the menu button inside the Header component. The next thing that I needed to do was to make sure to close it when a menu item gets clicked. Otherwise, the menu will continue covering the page even after navigating to the next page.

I am using React Router to route URLs to specific page components. To detect page navigations, I first needed access to the history object being used by React Router from the App component. This was achieved by wrapping App inside the withRouter higher-order component, which passed history as one of App’s props.

// App.jsx

import {withRouter} from 'react-router-dom';

function App(props) {
    const history = props.history;

    // Other stuff
}

export default withRouter(App);

The history object has a .listen() method which accepts a callback function that it will call every time the current location changes. Subscribing to these changes is usually done in the component’s componentDidMount lifecycle method (and unsubscribing in componentWillUnmount), which requires a class component and will make App look like this:

// App.jsx

class App extends React.Component {
    // constructor(props)
    // handleToggleMenu()

    componentDidMount() {
        this.unlisten = this.props.history.listen(() => {
            this.setState({
                isMenuOpen: false
            });
        });
    }

    componentWillUnmount() {
        this.unlisten();
    }

    // render()
}

But again, I didn’t want to convert my App component into a class component. And also I just read that there is a built-in React Hook for doing exactly this pattern, so I decided to use it instead.

Using the `useEffect` hook

The pattern of registering something in a component’s componentDidMount and unregistering it in componentWillUnmount is apparently very common that it got abstracted into its own React Hook, the useEffect hook.

import {useEffect} from 'react';

The useEffect hook accepts a function containing code that will normally run inside the componentDidMount (and componentDidUpdate) lifecycle method; in my case, that would be code to listen to changes to the current history location and closing the menu when it does.

// App.jsx

function App(props) {
    useEffect(() => {
        props.history.listen(() => {
            setIsMenuOpen(false);
        });
    });

    // Other stuff
}

We can also return a function containing code that will normally run inside the componentWillUnmount lifecycle method; in my case, stop listening for changes to the current history location. Calling history.listen() already returns such function so I can just return it right away.

// App.jsx

function App(props) {
    useEffect(() => {
        return props.history.listen(() => {
            setIsMenuOpen(false);
        });
    });

    // Other stuff
}

And these are all the changes needed to make the App component close the navigation menu on page navigations. No need to convert it to a class component and setup lifecycle methods. All the related code are located in close proximity to each other instead of being separated in different places in the component code.

Final `App` component

After applying all these changes, the App component, complete with the stateful navigation menu which closes on page navigation, now looks like this:

// App.jsx

import {useState, useEffect} from 'react';
import {withRouter} from 'react-router-dom';
import Header from './Header.jsx';

function App(props) {
    const [isMenuOpen, setIsMenuOpen] = useState(false);

    useEffect(() => props.history.listen(() => {
        setIsMenuOpen(false);
    }));

    return (
        <div>
            <Header
                isMenuOpen={isMenuOpen}
                onToggleMenu={() => setIsMenuOpen(!isMenuOpen)}
            />

            {/\* Other stuff \*/}
        </div>
    );
}

export default withRouter(App);

I could have gone even further by making a generic React Hook for such functionality, in case I need to use it again somewhere else. We can use these built-in React Hooks to build more hooks. But I guess I’ll just reserve that for another day when I actually need to.

Summary

In this article I walked through how I made my site’s navigation menu using React Hooks. We used the built-in useState hook to keep track of the menu’s open/close state, and the built-in useEffect hook to listen to changes in the current location (and cleanup after when the component is going to be removed). After applying the changes, we end up with a functional component that has its own state.

This is the first time that I’ve used React Hooks on something and so far I totally enjoyed the experience. I think the code is more readable and easier to figure out compared to using class components with lots of lifecycle methods, since I didn’t need to look in multiple separate places to understand a component’s functionality. Instead, all the related functionality are defined in one place. Also, we are able to build custom, more complex hooks out of the built-in ones if we want to, and reuse these functionalities all over our application. I’m really looking forward to using React Hooks more in the future.

Resources

Introducing Hooks

Thanks for reading this article! Feel free to leave your comments and let me know what you think. I also write other articles and make demos about cool Web stuff. You can check them out on my blog and on my GitHub profile. Have a great day! 🦔

Using lazysizes to lazyload images on the Web

Arnelle Balane — Sun, 13 Jan 2019 03:17:07 +0000

Images constitute a large portion of the page weights of a lot of Web pages. They make our pages more enjoyable to look at, but can also hurt our page load times especially when they have large file sizes or when there are a lot of them on the page, not to mention that they can unnecessarily consume our users’ bandwidths. One way to deal with these concerns is to lazyload our images.

Lazyloading means that we load our images only when they are going to be visible in the page. For example, an image located somewhere near the end of a long article doesn’t need to be loaded until the user actually scrolls it into view.

We can implement our own lazyloading functionality with JavaScript using some modern Web APIs (such as the Intersection Observer API), or use an image lazyloading library like lazysizes by aFarkas. What I like about this library is that it is very easy to start using it, and is performant while being packed with tons of amazing features.

Example Web page

Let’s create an example Web page with an image which we want to lazyload later on with lazysizes. Our example page is going to have the following markup:

<p>Lorem ipsum dolor sit amet...</p>
<!-- 9 more lorem ipsum paragraphs -->

<img src="image.jpg" alt="Sample Image">

<!-- 3 more lorem ipsum paragraphs -->

Notice from DevTools’ Network tab that the image has already been loaded since the page has loaded. Let’s try to change that to only load it when it is about to scroll into view using lazysizes.

Basic usage

To start using lazysizes, the first thing we need to do is load the lazysizes JavaScript file:

<script src="http://afarkas.github.io/lazysizes/lazysizes.min.js"></script>

It can also be installed via npm (npm install lazysizes) or bower (bower install lazysizes).

Second, we modify the img tags that we want to lazyload to:

Use data-src attribute instead of src, and
Add lazyload as one of its classes

<!-- Before: -->
<img src="image.jpg" alt="Sample Image">

<!-- After: -->
<img data-src="image.jpg" class="lazyload" alt="Sample Image">

And the result?

Notice from DevTools’ Network tab how the image is only loaded once it’s about to be scrolled into view. Awesome!

Lazyload responsive images

We can serve responsive images on the Web using the srcset attribute on the img element or using the picture element. This allows us to serve an appropriately-sized image to our users depending on their screen sizes.

One really cool feature of lazysizes that I really like is its ability to lazyload these responsive images without any additional configuration. All we have to do is to replace the srcset attribute with data-srcset.

<!-- Before: -->
<img
    srcset="
        image-100.jpg 100w,
        image-300.jpg 300w"
    src="image.jpg"
    alt="Responsive Image"
>

<-- After: -->
<img 
    data-srcset="
        image-100.jpg 100w,
        image-300.jpg 300w"
    data-src="image.jpg"
    alt="Responsive Image"
>

Use a low-quality image placeholder

One potential concern with our lazyloaded images at this point is that when the user’s network connection is slow, it will still take a long time for the images to show up, leaving us with a blank space in our page.

We can address this by displaying a small-sized, low-quality image as a placeholder until the actual image finishes loading. Because of its very small size, this placeholder image will load very quickly, giving the user an idea of what the actual image is going to look like. To do that, we add the placeholder image as the src attribute of the img tag.

<img 
    data-srcset="
        image-100.jpg 100w,
        image-300.jpg 300w"
    data-src="image.jpg"
    src="image-lqip.jpg"  <!-- eg a 20px-wide image -->
    width="300"
    height="200"
    alt="Responsive Image"
>

Adding the width and height attributes makes the small-sized image take up the space of the actual image. This can also be done via CSS.

Summary

So far we’ve looked at how to use lazysizes to lazyload images for basic use cases. With the changes we’ve made, users will get to download the images in our page only when necessary. As a result, our page loads faster since it doesn’t need to wait for the large image to finish loading. We also keep the users from unnecessarily consuming their bandwidths on images that they’re not going to see yet.

However, we’ve only scratched the surface of what lazysizes offers. It offers much more features and different configurations and other patterns of how it can be used for more advanced use cases, so definitely check out the library’s documentation for those, and let’s all make our Web experiences better together by lazyloading our images.

This article was originally posted on my personal blog. View original article.

DEV Community: Arnelle Balane

Let's build a video calling app! - Part One

Introduction

Concepts

MediaStream API

RTCPeerConnection API

Interactive Connectivity Establishment (ICE)

Session Description Protocol (SDP)

Signaling

Events

Implementation

Conclusion

Async Clipboard API: Accessing the clipboard using JavaScript

Async Clipboard API

Writing to the clipboard

Reading from the clipboard

Checking clipboard access permissions

Clipboard Events

Image Support

Write an image to the clipboard

Reading an image from the clipboard

Browser Support & Feature Detection

Resources

Playing the T-Rex Runner game inside a Picture-in-Picture window

Get the game's source code

Implement programmatic T-Rex jump

Capture video stream of canvas contents

Show the Picture-in-Picture window

Implement game controls

Let's play!

Conclusion

Resources

Setting up multiple GitHub accounts, the nicer way

Generate SSH keys for each user

Committing as the correct user

Do this for all work repositories

Pushing as the correct user

Summary

Building a site navigation menu using React Hooks

Initial setup

isMenuOpen component state

Make sure to use the right React version

Using the useState hook

Detecting page navigations

Using the useEffect hook

Final App component

Summary

Resources

Using lazysizes to lazyload images on the Web

Example Web page

Basic usage

Lazyload responsive images

Use a low-quality image placeholder

Summary

`isMenuOpen` component state

Using the `useState` hook

Using the `useEffect` hook

Final `App` component