DEV Community: Kimberlee Johnson

What changelogs do you follow, and why?

Kimberlee Johnson — Wed, 19 Jan 2022 21:18:51 +0000

Add flying emoji reactions to a React video chat app

Kimberlee Johnson — Thu, 02 Dec 2021 17:26:38 +0000

If a picture is worth a thousand words, what does that mean for emoji? For decades they’ve been used to add color to all sorts of written communications, from text messages to entire translations of Moby Dick to — most relevant to this blog post — video calls.

We build developer tools at Daily that enable new ways to communicate online. Adding emoji reactions to video chats gives participants a familiar (and fun!) way to express themselves.

In this tutorial we’ll add a set of flying emoji reactions to a custom video call built on the Daily call object.

You might’ve seen similar emoji overlays in Instagram Live broadcasts, Twitter Periscope live streams, or Twitch “emote walls” that fill an entire screen on a live stream, for example. We’ll be making a similar wall of reaction emoji for our WebRTC video calls with a little React, CSS, and some Daily methods.

To achieve this we will:

Create a button that sends an emoji of our choice flying on click
Send our emoji reaction to all other participants using the Daily sendAppMessage() method
Render the emoji for both the local participant who sent it and the remote participants who receive it

We will do all of these things in a Next.js demo app that we built in a previous post. Reference that tutorial for details on the foundation of the app, like participant, device, and track management. This post just focuses on the emoji 😎

The demo and code snippets are React-based, but you can still apply the essence of the steps outlined in the tutorial to work with other frameworks.

To run the demo locally:

Fork and clone the daily-demos/examples repository
cd examples/custom/flying-emojis
Set your DAILY_API_KEY and DAILY_DOMAIN env variables (see env.example)
yarn
yarn workspace @custom/flying-emojis dev

With that, our emoji are ready to fly.

Create a button that sends an emoji flying

The star icon, labeled "Emoji" in the call tray component, (Tray.js), reveals available emoji reactions, and allows participants to pick one to send.

Here’s that component’s structure, with tangential elements removed:



// Tray.js 

<div>
  {showEmojis && (
    <div className="emojis">
      <Button
        onClick={() => sendEmoji('fire')}
      >
        🔥
      </Button>
      <Button
        onClick={() => sendEmoji('squid')}
      >
        🦑
      </Button>
      <Button
        onClick={() => sendEmoji('laugh')}
      >
        🤣
      </Button>
    </div>
  )}
  <TrayButton
    label="Emoji"
    onClick={() => setShowEmojis(!showEmojis)}
  >
    <IconStar />
  </TrayButton>
</div>

When the star icon is clicked, it displays the available emoji. When a participant selects an emoji, the component calls sendEmoji() and passes a string representing the selection. For example, after clicking on "🦑" onClick={() => sendEmoji('squid')} is called.

Let’s look at sendEmoji().



// Tray.js 

function sendEmoji(emoji) {
  window.dispatchEvent(
   new CustomEvent('reaction_added', { detail: { emoji } })
 );
 setShowEmojis(false);
}

sendEmoji() triggers a CustomEvent that we named reaction_added. The string representing the emoji is reaction_added’s CustomEvent.detail.

We’ll listen for the reaction_added event in FlyingEmojisOverlay.js, via window.addEventListener('reaction_added', handleSendFlyingEmoji);.

Use `sendAppMessage()` to broadcast the emoji to other call participants

handleSendFlyingEmoji() gets the string representing the emoji from CustomEvent.detail, and broadcasts it to all other call participants using the Daily sendAppMessage() method:



// FlyingEmojiOverlay.js

function handleSendFlyingEmoji(e) {
   const { emoji } = e.detail;

   if (emoji) {
     callObject.sendAppMessage({ message: `${emoji}` }, '*');
     handleDisplayFlyingEmoji(emoji);
   }
}

sendAppMessage() emits a corresponding app-message event that all remote participants receive. The <FlyingEmojiOverlay /> component listens for the event and calls handleReceiveFlyingEmoji() when a message is received: callObject.on('app-message', handleReceiveFlyingEmoji);.



// FlyingEmojisOverlay.js 

const handleReceiveFlyingEmoji = useCallback(
   (e) => {
     if (!overlayRef.current) {
       return;
     }
     handleDisplayFlyingEmoji(e.data.message);
   },
   [handleDisplayFlyingEmoji]
);

handleReceiveFlyingEmoji() passes the message data from e.data.message along to handleDisplayFlyingEmoji().

Render the emoji for both the local sender and the remote recipient

handleDisplayFlyingEmoji() is called both on sending, in handleSendFlyingEmoji() and upon receiving in handleReceiveFlyingEmoji(). That’s because app-message only fires for remote participants, but we want the local participant to see their own emoji reaction as well.

The handleDisplayFlyingEmoji() function takes a string as a parameter. handleSendFlyingEmoji() passes the display handler a string from the CustomEvent.detail from the window event, while handleReceiveFlyingEmoji() passes a string from the app-message event object, e.data.message.

Now that we know how and when handleDisplayFlyingEmoji() is executed, let’s have a look at its definition:



// FlyingEmojisOverlay.js 

const handleDisplayFlyingEmoji = useCallback(
  (emoji) => {
    if (!overlayRef.current) {
      return;
    }

    const node = document.createElement('div');
    node.appendChild(document.createTextNode(EMOJI_MAP[emoji]));
    node.className =
      Math.random() * 1 > 0.5 ? 'emoji wiggle-1' : 'emoji wiggle-2';
    node.style.transform = `rotate(${-30 + Math.random() * 60}deg)`;
    node.style.left = `${Math.random() * 100}%`;
    node.src = '';
    overlayRef.current.appendChild(node);

    node.addEventListener('animationend', (e) =>
      handleRemoveFlyingEmoji(e.target)
    );
  },
  [handleRemoveFlyingEmoji]
);

Let’s break it all down.

First, it creates a new <div>, and appends the selected emoji in a text node to that div.



// FlyingEmojiOverlay.js 

const node = document.createElement('div');
node.appendChild(document.createTextNode(EMOJI_MAP[emoji]));

It gets the emoji by referencing a CONSTANT EMOJI_MAP object whose keys map to emoji:



// FlyingEmojisOverlay.js 

const EMOJI_MAP = {
 fire: '🔥',
 squid: '🦑',
 laugh: '🤣',
};

Once the emoji is added, the function applies styles. Math.random() sets the className to either 'emoji wiggle-1' or 'emoji wiggle-2'.



// FlyingEmojisOverlay.js

@keyframes wiggle-1 {
  from {
    margin-left: -50px;
  }
  to {
    margin-left: 50px;
  }
}

@keyframes wiggle-2 {
  from {
    margin-left: 50px;
  }
  to {
    margin-left: -50px;
  }
}

These classes determine where the emoji starts wiggling on the screen. Math.random() also determines the degree to which the emoji rotates, and its left position.



 // FlyingEmojiOverlay.js 

node.className =
       Math.random() * 1 > 0.5 ? 'emoji wiggle-1' : 'emoji wiggle-2';
 node.style.transform = `rotate(${-30 + Math.random() * 60}deg)`;
 node.style.left = `${Math.random() * 100}%`;

With styling set, the emoji is ready to be added to overlayRef:



// FlyingEmojisOverlay.js

overlayRef.current.appendChild(node);

Finally, handleDisplayFlyingEmoji() listens for the emoji animation to end, node.addEventListener('animationend', (e) => handleRemoveFlyingEmoji(e.target)); and then removes the appended child:



const handleRemoveFlyingEmoji = useCallback((node) => {

   if (!overlayRef.current) return;

   overlayRef.current.removeChild(node);

 }, []);

What’s next ❓

We hope this tutorial has helped you add personality to your video calls. To build on this demo, you could: experiment with emoji that multiply and burst more quickly in a “particle effect” (instead of a gentle float, maybe they bounce around the video window); generate random emoji; add reactions to a webinar app, or explore libraries like confetti.

To keep reading for more inspiration, Butter, a web events facilitation platform, has a writeup on on their engineering blog on how they implemented floating emojis for their video chats with Framer Motion and Lottie Web.

The world is your oyster, 🌍🦪.

Add video chat to a Next.js app in 30 minutes with Daily Prebuilt

Kimberlee Johnson — Thu, 07 Oct 2021 19:52:08 +0000

With the launch of our new docs site, we’ve been spending a lot of time in Next.js. We even got a little meta and embedded a Daily Prebuilt demo, built on Next, into the docs site, also built on Next.

The demo lets readers quickly test out Daily calls, and get a sense of what Daily Prebuilt would look like embedded in their own app, right on the docs site. Our docs use Next API routes to create temporary Daily rooms dynamically server-side.

Since our docs codebase isn’t currently public, this post uses our /examples/prebuilt/basic-embed repository as a template to show how you can do the same in any Next app. We’ll cover:

Setting up the repository locally
Using Next API routes to create Daily rooms dynamically server-side
Creating a Daily callframe and joining a call once we have a room

You’ll need a Daily account if you don’t have one already.

Skip to API routes if you already have a Next project that you want to add video chat to, or if you’d rather run create-next-app to start a new app from scratch.

Set up the demo repository

Clone the /examples repo, and cd examples/prebuilt/basic-embed.

Create an .env based on .env.example, adding your Daily domain (you set this up when you created an account) and API key (you can find this in the "Developers" tab in the Daily dashboard):

DAILY_DOMAIN="your-domain"
DAILY_API_KEY="Daily API Key"

Once you’ve added your own values, run the following from inside /basic-embed to install dependencies and start the server:

yarn 
yarn workspace @prebuilt/basic-embed dev

You should now be able to click "Create room and start" and jump into a Daily Prebuilt call:

Let’s look at how that all works.

Use API routes to create Daily video rooms dynamically server-side

Our /pages directory is where most of the fun happens. Next pages are React components. They’re associated with routes based on their file names, and come with some other neat built-in features.

For example, files inside pages/api are treated like API endpoints. These are Next API routes. In development, they are served by your dev servers, but in prod in our demo app they’ll get converted into Vercel functions, technically making them serverless.

In our app, we use a Next API route to create Daily rooms:

// pages/api/room/index.js 

export default async function handler(req, res) {
 if (req.method === 'POST') {
   const options = {
     method: 'POST',
     headers: {
       Accept: 'application/json',
       'Content-Type': 'application/json',
       Authorization: `Bearer ${process.env.DAILY_API_KEY}`,
     },
     body: JSON.stringify({
       properties: {
         enable_prejoin_ui: true,
         enable_network_ui: true,
         enable_screenshare: true,
         enable_chat: true,
         exp: Math.round(Date.now() / 1000) + 300,
         eject_at_room_exp: true,
       },
     }),
   };

   const dailyRes = await fetch(
     `${process.env.DAILY_REST_DOMAIN}/rooms`,
     options
   );

   const response = await dailyRes.json();

   if (response.error) {
     return res.status(500).json(response.error);
   }

   return res.status(200).json(response);
 }

 return res.status(500);
}

All requests to /room are handled here, and we’re specifically adding a case to handle a POST request. The request references both the Daily API key and base REST domain in the .env.

We send this request in the <CreateRoomButton /> component. This component is a button that onClick creates a room:

// components/CreateRoom.js

return (
     <Button onClick={createRoom} disabled={isValidRoom}>
       Create room and start
     </Button>
 );

createRoom() sends a request to the Next /api/room endpoint, which makes the Daily endpoint POST request in api/room/index described above:

// components/CreateRoom.js

const createRoom = async () => {
   try {
     const res = await fetch('/api/room', {
       method: 'POST',
       headers: {
         'Content-Type': 'application/json',
       },
     });
    // Abridged snippet 
 };

When that request resolves, it returns the Daily response object, including a url value. createRoom() sets the room value stored in local state to the response object’s url:

// components/CreateRoom.js

const resJson = await res.json();
setRoom(resJson.url);

Now that we have a room, we’re ready for a callframe.

Create a Daily callframe and join a call

Our <Call /> component not only renders <CreateRoom />, but also initializes the callframe with a useEffect hook:

// components/Call.js

useEffect(() => {
   if (callFrame) return;

   createAndJoinCall();
 }, [callFrame, createAndJoinCall]);

The hook calls createAndJoinCall(), a function that:

Creates a new Daily callframe, embedding it in the ref we identified <div ref={callRef} className="call" /> and passing along some properties we stored in the CALL_OPTIONS constant
Joins the Daily room using the room value stored in local state
- Listens for the 'left-meeting' event so it can reset app state when the local participant leaves the call

// components/Call.js 

const createAndJoinCall = useCallback(() => {
   const newCallFrame = DailyIframe.createFrame(
     callRef?.current,
     CALL_OPTIONS
   );

   setCallFrame(newCallFrame);

   newCallFrame.join({ url: room });

   const leaveCall = () => {
     setRoom(null);
     setCallFrame(null);
     callFrame.destroy();
   };

   newCallFrame.on('left-meeting', leaveCall);
 }, [room, setCallFrame]);

createAndJoinCall() is invoked whether a room is created dynamically in real-time, as we walked through in the <CreateRoom /> component, or a room is submitted through the input rendered in <Home />:

// components/Home.js 

<Field label="Or enter room to join">
    <TextInput
        ref={roomRef}
        type="text"
        placeholder="Enter room URL..."
        pattern="^(https:\/\/)?[\w.-]+(\.(daily\.(co)))+[\/\/]+[\w.-]+$"
        onChange={checkValidity}
     />
</Field>

The input calls checkValidity() as its values change. This function makes sure that the entered text is a valid Daily room URL based on the pattern value, and sets the local state value isValidRoom to true if so:

// components/Home.js 

const checkValidity = useCallback(
   (e) => {
     if (e?.target?.checkValidity()) {
       setIsValidRoom(true);
     }
   },
   [isValidRoom]
 );

This enables the "Join room" button:

// components/Home.js 

<Button onClick={joinCall} disabled={!isValidRoom}>
     Join room
</Button>

Clicking the button calls joinCall(), which sets the room value stored in local state to the input:

// components/Home.js

const joinCall = useCallback(() => {
   const roomUrl = roomRef?.current?.value;
   setRoom(roomUrl);
 }, [roomRef]);

The room value in local state triggers the callframe creation in <Call /> in the same way it did when we created a room dynamically. In both cases a room value also instructs index.js to display the <Call /> instead of the <Home /> component, according to this ternary statement:

// pages/index.js      

<main>
       {room ? (
         <Call
           room={room}
           expiry={expiry}
           setRoom={setRoom}
           setCallFrame={setCallFrame}
           callFrame={callFrame}
         />
       ) : (
         <Home
           setRoom={setRoom}
           setExpiry={setExpiry}
           isConfigured={isConfigured}
         />
       )}
     </main>

thank u, Next.js

That’s the core of the app! There are a few other tangential things in the codebase that we didn’t get into, like the <ExpiryTimer />component and how we put [getStaticProps()`](https://nextjs.org/docs/basic-features/data-fetching#getstaticprops-static-generation) to work checking for env variables, but we welcome you to explore those things yourself and ping us with questions. Or, if you’d rather build your own video chat interface with Next.js, check out our post using Next with the Daily call object.

More resources

Did Dolly Parton’s ice cream really break the internet?

Kimberlee Johnson — Mon, 21 Jun 2021 04:36:09 +0000

Like many people, I’m a Dolly Parton fan. I’ve been wearing a mask with her face on it all year (thanks, Chloe!); I celebrated her birthday in January with a Chrome DevTools script.

I’m also an ice cream fan.

There are lots of us who really like both of these things! Jeni’s Splendid Ice Creams combined two crowd pleasers when they collaborated with Dolly on a limited edition Strawberry Pretzel Pie pint to benefit her Imagination Library.

When the pints went on sale online, Jeni's website crashed, fast:

// Detect dark theme var iframe = document.getElementById('tweet-1380189252311474182-443'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1380189252311474182&theme=dark" }

This Jeni’s Splendid Schadenfreude inspired me to ask: What actually happens when a website crashes? My Daily colleague Brian was kind enough to entertain my curiosity. Read on for what I learned from our conversation, including a review of what happens when you visit a URL in your browser, an intro to CDNs, and a few possible failure modes. To be clear, that last part will just be some Splendid Speculation.

We all scream for ice cream

It’s a question almost as popular during tech interviews as Dolly is all the time: What happens when you visit a URL in your browser?

I wrote about what happens when you visit a video chat URL in your browser, and at the highest level it’s largely the same for any link. The browser starts the Domain Name System (DNS) lookup process, which matches the URL to the IP address of the server hosting the site. Assuming the lookup is successful, the browser initiates a TCP connection with that IP, and sends a request for the site’s resources. Back and forth negotiation ensues, and if all goes well, the requested resource is returned.

That’s a lot of back and forth for a single request. Multiply that by the number of people who scream for both Jeni’s ice cream and Dolly Parton, and very quickly there are a lot of requests and negotiations for a single server to handle.

Unlike novelty ice cream flavors, managing web traffic is not a new and limited edition problem. Content Delivery Networks (CDNs) have been around for years to help businesses manage it.

CDN: a coat of many servers

(Okay, a CDN is really more like a network of many servers, but I had to try to make the song work).

When a website uses a CDN, the CDN makes its resources, like images and JavaScript, available across the CDN’s server network. There are lots of advantages to this. If resources were instead concentrated at a single server location, then it matters where site visitors physically are. The farther away a visitor is from the server, the longer it would take for the server to return the resources they request from it. Websites get around this problem by using a CDN. The CDN is configured to know where to find all of the website’s resources, and the website then points its DNS records at the CDN. Then, when a visitor looks up the website and their browser initiates the DNS process, it finds the addresses at the CDN, and the nearest server in the network returns the resource.

Unfortunately, even with a CDN, lots of things can still go wrong. I was one of many Jeni's customers who saw this message from CDN provider Cloudflare when trying to get my hands on a pint of Dolly's ice cream.

While it’s impossible to know what exactly happened, we can think of a few things.

Failure modes: the salt in my tears

A “thundering herd” took down the host

Like many hyped limited product releases, Jeni’s launched the Dolly pints for sale online at a specific time: 12pm ET on April 8, 2021.

Many fans across the country flocked to the website on the dot, flooding the CDN with requests for resources. If the CDN servers all across the network don’t have the resources stored locally (stay tuned for more on caching configuration!), they’ll need to request them from the host site. All of those geographically distributed servers making the same request at the same time from the same host may have quickly overwhelmed it.

Depending on different servers’ configurations across the network, some servers at some locations could be requesting different resources, or be closer to the host. That explains why some customers, though very few, were able to successfully buy ice cream, and why so many different people saw so many different screens.

CDN configuration conflicted with site traffic needs

When a website sets up a CDN, it configures how long the CDN holds onto cached resources. The CDN can update resources every time a visitor requests them (generally leading to slower responses and not ideal!), or to only refresh a resource every X or so minutes, depending on how okay the host is with a page or image being stale when somebody visits the site.

If lots of updates are happening, let’s say the number of available pints in a database is decreasing, but a CDN only checks for that update every X minutes, somebody making a request could be on the receiving end of stale data, and then unable to complete a checkout.

The way CDN settings interact with host settings, especially under a large amount of traffic, can quickly get complicated and break things by sending too many requests.

Host-specific processes got overwhelmed

While a CDN can cache static resources like images on a marketing page, it can’t actually help with some of the more interactive parts of a website that involve sharing unique customer information, like entering a credit card or billing address. It’s possible that the origin server could’ve been overwhelmed with people trying to actually complete purchases. CDNs improve overall capacity with lots and lots of servers in lots and lots of places, but CDN resource caching doesn’t necessarily help the website process individual purchases. The error screen I saw on jenis.com did indeed look different from the error screen I saw on shop.jenis.com.

It’s all wrong, but it’s all right

Jeni’s specifically called out website traffic testing in their statement about the Dolly debacle:

Of course, it’s always possible to make a mistake in preparedness, like just not having enough servers, etc. But, it’s also really, really hard to simulate a Dolly Parton level of website traffic! Predicting the exact requests to be made in the exact configuration is impossible to simulate to a 100% degree of accuracy. And, while there are lots of advanced website testing services today, nothing exactly replicates real requests from real customers in real time. It’s kind of like how Dolly herself lost a Dolly Parton lookalike contest. Imitation and reality are not the same.

Dolly’s ice cream didn’t break the internet. It just overloaded Jeni’s host server in some way, the details of which are impossible to pinpoint from the outside. But, like a country song, it gives us some things to think about and stories to tell.

Build a real time video chat app with Next.js and Daily

Kimberlee Johnson — Fri, 18 Jun 2021 21:50:46 +0000

We built one of our first Daily demos with React, because we like working with the framework. We’re not alone. More developers expressed interest in learning React than in picking up any other web framework in the 2020 Stack Overflow Developer Survey.

Meta frameworks for React like Next.js are also gaining traction, so we built a basic video call demo app using Next.js and the Daily call object.

The demo draws inspiration from the new Daily Prebuilt (We’ll eventually open source Daily Prebuilt’s components, stay tuned!), using shared contexts and custom hooks that we hope help get your own apps up and running ASAP. Dive right into the repository or read on for a sneak peek at some of the most foundational pieces, like the core call loop (shared contexts and hooks) and generating meeting tokens.

Run the demo locally

You can find our basic Next.js and Daily video chat demo in our ✨ new ✨ daily-demos/examples repository. This is a living repo. It’ll grow and evolve as Daily does and as we receive feedback. Poke around and you might notice a few other demos in progress. To hop right into the basic Next.js and Daily app:

Fork and clone the repository
cd examples/dailyjs/basic-call
Set your DAILY_API_KEY and DAILY_DOMAIN environment variables (see env.example)
yarn
yarn workspace @dailyjs/basic-call dev

The core call loop: shared contexts and hooks

As you’re probably well aware in the year 2021, lots of things can happen on video calls. Participants join and leave, mute and unmute their devices, not to mention the funny things networks can decide to do. Application state can get unwieldy quickly, so we make use of the Context API to avoid passing ever-changing props to all the different components that need to know about the many states.

Six contexts make up what we refer to as our call loop. They handle four different sets of state: devices, tracks, participants, and call state, in addition to a waiting room experience and the overall user interface.

// pages/index.js

  return (
    <UIStateProvider>
      <CallProvider domain={domain} room={roomName} token={token}>
        <ParticipantsProvider>
          <TracksProvider>
            <MediaDeviceProvider>
              <WaitingRoomProvider>
                <App />
              </WaitingRoomProvider>
            </MediaDeviceProvider>
          </TracksProvider>
        </ParticipantsProvider>
      </CallProvider>
    </UIStateProvider>
  );

Some of the contexts also make use of custom hooks that abstract some complexity, depending on the, well, context.

With that pun out of the way, let’s dive into each of the contexts except for <WaitingRoomProvider>, You’ll have to...wait for a post on that one.

Okay, really, we’re ready now.

Managing devices

The <MediaDeviceProvider> grants the entire app access to the cams and mics used during the call.

// MediaDeviceProvider.js

return (
   <MediaDeviceContext.Provider
     value={{
       cams,
       mics,
       speakers,
       camError,
       micError,
       currentDevices,
       deviceState,
       setMicDevice,
       setCamDevice,
       setSpeakersDevice,
     }}
   >
     {children}
   </MediaDeviceContext.Provider>
 );

<MediaDeviceProvider> relies on a useDevices hook to listen for changes to the call object to make sure the app has an up to date list of the devices on the call and each device’s state.

// useDevices.js

const updateDeviceState = useCallback(async () => {

   try {
     const { devices } = await callObject.enumerateDevices();

     const { camera, mic, speaker } = await callObject.getInputDevices();

     const [defaultCam, ...videoDevices] = devices.filter(
       (d) => d.kind === 'videoinput' && d.deviceId !== ''
     );
     setCams(
       [
         defaultCam,
         ...videoDevices.sort((a, b) => sortByKey(a, b, 'label', false)),
       ].filter(Boolean)
     );
     const [defaultMic, ...micDevices] = devices.filter(
       (d) => d.kind === 'audioinput' && d.deviceId !== ''
     );
     setMics(
       [
         defaultMic,
         ...micDevices.sort((a, b) => sortByKey(a, b, 'label', false)),
       ].filter(Boolean)
     );
     const [defaultSpeaker, ...speakerDevices] = devices.filter(
       (d) => d.kind === 'audiooutput' && d.deviceId !== ''
     );
     setSpeakers(
       [
         defaultSpeaker,
         ...speakerDevices.sort((a, b) => sortByKey(a, b, 'label', false)),
       ].filter(Boolean)
     );

     setCurrentDevices({
       camera,
       mic,
       speaker,
     });

   } catch (e) {
     setDeviceState(DEVICE_STATE_NOT_SUPPORTED);
   }
 }, [callObject]);

useDevices also handles device errors, like if a cam or mic is blocked, and updates a device’s state when something changes for the participant using the device, like if their tracks change.

Keeping track of tracks

Different devices share different kinds of tracks. A microphone shares an audio type track; a camera shares video. Each track contains its own state: playable, loading, off, etc. <TracksProvider> simplifies keeping track of all those tracks as the number of call participants grows. This context listens for changes in track state and dispatches updates. One type of change, for example, could be when a participant’s tracks start or stop.

// TracksProvider.js

export const TracksProvider = ({ children }) => {
 const { callObject } = useCallState();
 const [state, dispatch] = useReducer(tracksReducer, initialTracksState);

 useEffect(() => {
   if (!callObject) return false;

   const handleTrackStarted = ({ participant, track }) => {
     dispatch({
       type: TRACK_STARTED,
       participant,
       track,
     });
   };
   const handleTrackStopped = ({ participant, track }) => {
     if (participant) {
       dispatch({
         type: TRACK_STOPPED,
         participant,
         track,
       });
     }
   };

   /** Other things happen here **/

   callObject.on('track-started', handleTrackStarted);
   callObject.on('track-stopped', handleTrackStopped);
   }, [callObject];

Handling participants

<ParticipantsProvider> makes sure any and all participant updates are available across the app. It listens for participant events:

// ParticipantsProvider.js

 useEffect(() => {
   if (!callObject) return false;

   const events = [
     'joined-meeting',
     'participant-joined',
     'participant-updated',
     'participant-left',
   ];

   // Listen for changes in state
   events.forEach((event) => callObject.on(event, handleNewParticipantsState));

   // Stop listening for changes in state
   return () =>
     events.forEach((event) =>
       callObject.off(event, handleNewParticipantsState)
     );
 }, [callObject, handleNewParticipantsState]);

And dispatches state updates depending on the event:

// ParticipantsProvider.js

const handleNewParticipantsState = useCallback(
   (event = null) => {
     switch (event?.action) {
       case 'participant-joined':
         dispatch({
           type: PARTICIPANT_JOINED,
           participant: event.participant,
         });
         break;
       case 'participant-updated':
         dispatch({
           type: PARTICIPANT_UPDATED,
           participant: event.participant,
         });
         break;
       case 'participant-left':
         dispatch({
           type: PARTICIPANT_LEFT,
           participant: event.participant,
         });
         break;
       default:
         break;
     }
   },
   [dispatch]
 );

<ParticipantsProvider> also calls on use-deep-compare to memoize expensive calculations, like all of the participants on the call:

// ParticipantsProvider.js

const allParticipants = useDeepCompareMemo(
   () => Object.values(state.participants),
   [state?.participants]
 );

Managing room and call state

<CallProvider> handles configuration and state for the room where the call happens, where all those devices, participants, and tracks interact.

<CallProvider> imports the abstraction hook useCallMachine to manage call state.

// CallProvider.js

 const { daily, leave, join, state } = useCallMachine({
   domain,
   room,
   token,
 });

useCallMachine listens for changes in call access, for example, and updates overall call state accordingly:

// useCallMachine.js

useEffect(() => {
   if (!daily) return false;

   daily.on('access-state-updated', handleAccessStateUpdated);
   return () => daily.off('access-state-updated', handleAccessStateUpdated);
 }, [daily, handleAccessStateUpdated]);

// Other things happen here

 const handleAccessStateUpdated = useCallback(
   async ({ access }) => {

     if (
       [CALL_STATE_ENDED, CALL_STATE_AWAITING_ARGS, CALL_STATE_READY].includes(
         state
       )
     ) {
       return;
     }

     if (
       access === ACCESS_STATE_UNKNOWN ||
       access?.level === ACCESS_STATE_NONE
     ) {
       setState(CALL_STATE_NOT_ALLOWED);
       return;
     }

     const meetingState = daily.meetingState();
     if (
       access?.level === ACCESS_STATE_LOBBY &&
       meetingState === MEETING_STATE_JOINED
     ) {
       return;
     }
     join();
   },
   [daily, state, join]
 );

<CallProvider> then uses that information, to do things like verify a participant’s access to a room, and whether or not they’re permitted to join the call:

// CallProvider.js

useEffect(() => {
   if (!daily) return;

   const { access } = daily.accessState();
   if (access === ACCESS_STATE_UNKNOWN) return;

   const requiresPermission = access?.level === ACCESS_STATE_LOBBY;
   setPreJoinNonAuthorized(requiresPermission && !token);
 }, [state, daily, token]);

If the participant requires permission to join, and they’re not joining with a token, then the participant will not be allowed into the call.

Generating Daily meeting tokens with Next.js

Meeting tokens control room access and session configuration on a per-user basis. They’re also a great use case for Next API routes.

API routes let us query endpoints directly within our app, so we don’t have to maintain a separate server. We call the Daily /meeting-tokens endpoint in /pages/api/token.js:

// pages/api/token.js

export default async function handler(req, res) {
 const { roomName, isOwner } = req.body;

 if (req.method === 'POST' && roomName) {

   const options = {
     method: 'POST',
     headers: {
       'Content-Type': 'application/json',
       Authorization: `Bearer ${process.env.DAILY_API_KEY}`,
     },
     body: JSON.stringify({
       properties: { room_name: roomName, is_owner: isOwner },
     }),
   };

   const dailyRes = await fetch(
     `${process.env.DAILY_REST_DOMAIN}/meeting-tokens`,
     options
   );

   const { token, error } = await dailyRes.json();

   if (error) {
     return res.status(500).json({ error });
   }

   return res.status(200).json({ token, domain: process.env.DAILY_DOMAIN });
 }

 return res.status(500);
}

In index.js, we fetch the endpoint:

// pages/index.js

const res = await fetch('/api/token', {
     method: 'POST',
     headers: {
       'Content-Type': 'application/json',
     },
     body: JSON.stringify({ roomName: room, isOwner }),
   });
   const resJson = await res.json();

What’s Next.js?

Please fork, clone, and hack away! There are lots of ways you could start building on top of this demo: adding custom user authentication, building a chat component, or pretty much anything that springs to mind.

We’d appreciate hearing what you think about the demo, especially how we could improve it. We’re also curious about other framework and meta-framework specific sample code that you’d find useful.

If you’re hoping for more Daily and Next.js sample code, we’ve got you covered. Come back soon!

Optional chaining in the ~real world (React video chat app)

Kimberlee Johnson — Thu, 29 Apr 2021 03:33:14 +0000

I started learning JavaScript in 2019, around the time that optional chaining became a thing.

The optional chaining operator (?.) enables you to read the value of a property located deep within a chain of connected objects without having to check that each reference in the chain is valid. (MDN)

I remember hearing the hearsay about why this was awesome, but, at the time, the above explanation and others and any conversations about a question mark in javascript that isn't the ternary operator still went a bit over my head. Fast forward two years, and I've finally run into optional chaining in the ~real world.

This post shares that encounter! I'll go over video chat participant "tracks" at the highest level, and then walk through why optional chaining makes sense in this use case.

Before diving in, a quick thank you to my colleague Jess for the lesson that became this post. I'm lucky to learn from her at work everyday and on Twitter too!

Video chat participant tracks

Like a lot of people, I've been on a lot of video calls this year. I also work at Daily, where my colleagues build real-time audio and video APIs. I write documentation for the tools they build and prototype demo apps, so I'm learning a fair amount about the different moving parts behind video and audio-only calls, things I didn't really think about before.

Take, for example, tracks!

When I join a video call with someone else, I and that other person or people trade audio, video, and sometimes screen media tracks back and forth.

As you've probably experienced, participants' tracks can go through many states. Tracks load as participants join, and then they're playable; they can be muted intentionally or because of a disruption. The Daily API accounts for the following participant track states, for example:

blocked
off
sendable
loading
playable
interrupted

We can find a track's state on the Daily participants object. The object's keys are session id's for each participant, and the corresponding values include lots of details about the participant. For example, here's the participant object for a session_id "e20b7ead-54c3-459e-800a-ca4f21882f2f":

"e20b7ead-54c3-459e-800a-ca4f21882f2f": {
    user_id: "e20b7ead-54c3-459e-800a-ca4f21882f2f",
    audio: true,
    video: false,
    screen: false,
    joined_at: Date(2019-04-30T00:06:32.485Z),
    local: false,
    owner: false,
    session_id: "e20b7ead-54c3-459e-800a-ca4f21882f2f",
    user_name: ""
    tracks: {
      audio: {
        subscribed: boolean,
        state: 'playable',
        blocked?: {
          byDeviceMissing?: boolean,
          byPermissions?: boolean
        },
        off?: {
          byUser?: boolean,
          byBandwidth?: boolean 
        },
        track?: <MediaStreamTrack>
      }
      video: { /* same as above */ },
      screenAudio: { /* same as above */ },
      screenVideo: { /* same as above */ },
    }
  }
}

The track's state is deeply nested at participant.tracks.track.state, where track stands for the kind of track (audio, video, screenAudio or screenVideo).

And this is where optional chaining comes in.

Opting into optional chaining

In JavaScript, if an object doesn't exist, trying to access values on that object throws an error.

This can be inconvenient when a value we need is deeply nested, like the participant's video/audio track state. Let's look at an example.

When a participant leaves a call, their audio/video tracks stop. When their audio/video tracks stop, we want to remove their participant tile from the call.

We handle this update the same way we handle all participant updates. I wrote a longer post about how React hooks help us manage state in this video chat app, but tl; dr: the useEffect hook listens for changes to participantUpdated state, and on that change updates the rendered participants list.

participantUpdated stores a string including the name of the event, that participant's session id, and the time the event happened. When a participant's tracks stop, as for other events, we call setParticipantUpdated to change the string. Here's how that looks without optional chaining:

const handleTrackStopped = useCallback((e) => {
  logDailyEvent(e);
  setParticipantUpdated(
`track-stopped-${e.participant.user_id}-${Date.now()}`
  );
}, []);

Can you guess why this might cause a problem?

Because when a participant leaves a call and their tracks stop, they're no longer a meeting participant. They can't be found on the Daily participants object. .participant does not exist. The console throws an error, Cannot read property 'user_id' of null:

From a UI perspective, a black, trackless tile remains even after the participant leaves. This is because setParticipantUpdated can't fire, so the hook listening for the change doesn't update the rendered participant list to remove the absent participant, even though their tracks disappear.

Optional chaining helps us avoid this. Let's add the syntax to handleTrackStopped:

const handleTrackStopped = useCallback((e) => {
  logDailyEvent(e);
  setParticipantUpdated(
`track-stopped-${e?.participant?.user_id}-${Date.now()}`
  );
}, []);

Now, those .? evaluate the missing .participant as undefined. If I add a console.log() to handleTrackStopped to see the string passed to state, that's confirmed:

With this successful change to participantUpdated state, our hook can register the change, update the participant list, and be sure to remove any trackless tiles.

Remember, it's all optional

Optional chaining makes sense in this demo video chat app for a few reasons. For one thing, our track state data was pretty deeply nested. For another, it's okay if the .participant doesn't exist in our app after they leave (we won't be trying to access their data again once they're gone).

We didn't use optional chaining as our default syntax for every nested object in our app, and it's unlikely that would ever be a good idea. If you're using this syntax in the ~real world, be sure to be explicit about it.

And, if you are using optional chaining, please tell me about it! When have you opted for it recently? Let me know in the comments or over on Twitter.

How to stream a local file into a real-time video call

Kimberlee Johnson — Fri, 16 Apr 2021 21:41:39 +0000

We've joined a lot of video calls over the last year here at Daily. While it’s easy to chat in real-time with someone on the other side of the world, streaming a video to watch simultaneously as part of that conversation quickly complicates things. We’ve all been in meetings, training sessions, discussion groups, and even catch ups with friends where lags and buggy streams make it impossible to watch a video at the same time.

With great frustration comes great experimentation, and we’ve come up with a quick hack for streaming local video files during our calls: using the Daily API startScreenShare() method.

This tutorial covers our workaround! We’ll go over uploading the local file and creating a stream from it, sharing that stream, and making sure that our hack is working.

If you’re looking to implement this feature in your own embedded prebuilt UI or Daily call object video app, you can add to your own code as we go. Or, if you want to fiddle with a working prototype, download our gist and create a Daily room if you don’t have one already.

Before we get started

We put it in the title and we mean it: this is a hack. It’s a workaround, not meant for production, but for low stakes, internal use cases. Somebody may or may not have debuted it by sneaking mom’s spaghetti into a Daily all hands.

As with any hack, it also comes with a few caveats:

If you’ll be using Chrome, you’ll need to disable hardware acceleration before you share a video stream.
Unfortunately, this solution doesn’t work for streaming video from Safari, because Safari doesn’t support captureStream().

With that, here we go here’s our shot, feet fail us not.

Upload the local video file and create a stream

We need an <input> element with three attributes to upload the video:

id: so that we can use document.getElementById
type: to specify that this input field is for a file upload.
accept: to make sure the uploaded file is a video.

<input id="vid-file-picker" type="file" accept="video/*"" />

We add an event listener to the <input>. On 'change', we call our function playLocalVideoFile():

let videoInput = document.getElementById('vid-file-picker'), callObject;
videoInput.addEventListener('change', playLocalVideoFile, false);

playLocalVideoFile() creates a video stream from the file using captureStream() or mozCaptureStream() if you’re using FireFox.

function playLocalVideoFile(evt) {
   let videoEl = document.getElementById('local-vid');
   let file = this.files[0];
   let type = file.type;
   if (!videoEl.canPlayType(type)) {
     alert('cannot play that file');
     return;
   }
   videoEl.src = URL.createObjectURL(file);
   videoEl.play().then(() => {
   // Mozilla currently prefixes the function name, so we have to check for either
   if (typeof videoEl.mozCaptureStream == 'function') { 
      window.localVideoStream = videoEl.mozCaptureStream();
      } else if (typeof videoEl.captureStream == 'function') {
          window.localVideoStream = videoEl.captureStream();
      }
    });
}

Now that we have the stream, we can share it.

Share the stream with call participants

We add a <button> to shareVideo() through the screen share stream.

<button onclick="shareVideo()" />share video through screenshare stream</button>

shareVideo() calls the Daily startScreenShare() method on the Daily call object, passing the localVideoStream created via playLocalVideo() as the media stream to be shared:

async function shareVideo() {
    callObject.startScreenShare({ mediaStream: localVideoStream });
}

Test the stream

Make sure you have a Daily room URL ready. We hardcoded ours as a const in our html file because, again, this is a hack. Relatedly, if you’re using Chrome, don’t forget to disable hardware acceleration.

Next, open the file in your browser. In a new tab, visit your Daily room URL. You should see two participants in the room tab, but just you in the file tab.

Click "Choose File" and upload the local video of your choice. If you need inspiration for what file to choose, try searching on Pexels for puppies (that’s what I did). Finally, click “share video through screen share stream” and in the tab with the room URL you should see the video coming through.

Snap back to reality (what’s next)

That’s it! Thanks for reading. We hope this makes sharing videos for everyone to watch on your calls easier. Let us know in the comments!

And, if you’re interested in building more apps that are all about sharing and streaming content, our webinar series might spark some ideas.

Learn React Hooks by looking under the hood of a video chat app

Kimberlee Johnson — Thu, 01 Apr 2021 05:48:22 +0000

Like most developers I know, I learn a lot every day at work. That's part of why I wanted to make programming part of my job!

I really like getting to immediately apply new things, and that helps me learn better, too. Despite reading lots of explainers and a few tutorials, I didn't really start to grok React Hooks until I needed to dig into the Daily React video chat demo.

When I think about the roles useEffect, useMemo, and useCallback play in a video chat app, something a lot of us use every day, I better remember how each hook works and recognize other opportunities to use them.

In case reading about that practical application could help you too, I decided to write about it! After a quick Hooks refresher, we'll look at an example of each these Hooks in the Daily demo app, and why we decided to use each.

I mentioned in my last post about avoiding global variables that I'm trying to get better about sharing what my colleagues teach me. This round is brought to us by the inimitable Jess Mitchell, who I am so grateful to work with and learn from!

Before we get hook'd

I really liked Ali Spittel's definition of Hooks on the latest Ladybug podcast:

"Hooks are a special type of function that allow you to hook into the component lifecycle of react components. So they allow you to do special things that a normal function wouldn't be able to do."

This means we can do unique things when components mount, update, and unmount. Like the docs say, we can take advantage of state and other features without having to write class components.

With that overview in mind, let's look at three Hooks in our video chat app: useEffect, useMemo, and useCallback.

`useEffect` to manage participant updates in state

With useEffect, we can, well, perform side effects in function components, based on state or prop changes.

In a video chat app, lots of things happen! Participants join and leave calls, start and stop their audio and video tracks, and then some. Our UI needs to update along with these changes. For example, it needs to add and remove video tracks as participants come and go.

The Daily API fires corresponding events as these things happen, e.g. 'participant-joined', 'track-stopped', etc. In our video chat app, we listen for these events and their handlers set our particpantUpdated state in response.

Here's where useEffect comes in! We only need to update the UI when a change has happened, when participantUpdated is set. We pass participantUpdated as a dependency (along with the call object that contains the updated participant data) to a useEffect hook, so we only update our participant list when something has changed.

useEffect(() => {
    if (participantUpdated) {
      const list = Object.values(callObject?.participants());
      setParticipants(list);
    }
  }, [participantUpdated, callObject]);

That covers storing the participant list, but what about displaying participants, rendering their video and audio tracks? That's where our next hook comes in.

`useMemo` to re-render videos only when we have to

useMemo returns a memoized value. Memoized means a value that's the result of an expensive function call.

There are lots of expensive calculations in a video chat app. Each participant's audio and video track alone contains loads of data, and computing that on every render would be a lot.

Instead, we pass our participants state value as a dependency to the useMemo hook that displays our tiles.

const displayLargeTiles = useMemo(() => {
    const isLarge = true;
    const tiles = participants?.filter((p) => !p.local);
     return (
        <div className="large-tiles">
          {tiles?.map((t, i) => (
            <Tile
              key={`large-${i}`}
              videoTrackState={t?.tracks?.video}
              audioTrackState={t?.tracks?.audio}
              isLarge={isLarge}
              disableCornerMessage={isScreenShare}
              onClick={
                t.local
                  ? null
                  : () => {
                      sendHello(t.id);
                    }
              }
            />
          ))}
        </div>
      );
    }
  }, [participants]);

The useMemo hook lets us only change the videos displayed when the participants have changed, instead of recalculating on every render.

Note: Because we built a video chat app with participant changes that happen often, the performance benefits fromuseMemo are significant; it saves us many recalculations. In a lot of cases, though, this hook might be more than your application needs. useMemo has a reputation of being overused, so tread with caution.

`useCallback` to re-render `startLeavingCall()` function only when we have to

Just like useMemo prevents us from re-calculating values that haven't changed, useCallback lets us stop specific functions from re-rendering.

In our App.js component, lots of things can trigger a re-render. But our startLeavingCall function, for example, only needs to re-render if the callObject, which stores data about our call, or our appState changes. This is because the function does different things depending on those values.

We pass callObject and appState as our dependencies.

/**
   * Starts leaving the current call.
   */
  const startLeavingCall = useCallback(() => {
    if (!callObject) return;
    // If we're in the error state, we've already "left", so just clean up
    if (appState === STATE_ERROR) {
      callObject.destroy().then(() => {
        setRoomUrl(null);
        setCallObject(null);
        setAppState(STATE_IDLE);
      });
    } else {
      setAppState(STATE_LEAVING);
      callObject.leave();
    }
  }, [callObject, appState]);

Hook'd and wanting more?

I hope this helped make Hooks feel a little more applicable! Can you think of any new ways to apply useEffect or useMemo, or useCallback in apps that you're building? Tell me in the comments! Especially tell me if you'll be building any video (or audio!) apps. You can give me a shout over on Twitter too!

How I learned to avoid implied globals (and why)

Kimberlee Johnson — Tue, 02 Mar 2021 18:19:54 +0000

Before I started drafting pull requests, I used to draft press releases. My public relations background comes in handy in my DevRel role today, and it helps me keep learning, too. I’m a community-taught developer, picking up a lot of my technical skills from people I meet in communities like Girl Geek Dinner, the CS department at CCSF, and, of course, DEV.

Today, I’m lucky and grateful to also learn on the job, from colleagues patient enough to teach me best practices. In the spirit of Laurie’s tweet, I’m going to try to do a better job of sharing what they teach me.

// Detect dark theme var iframe = document.getElementById('tweet-1337179838818492417-999'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1337179838818492417&theme=dark" }

This post is my first pass at that! Read on to learn how I learned to be less afraid of spooky JavaScript Promises, to avoid implied global variables, and to get better at bridging the gap between what I know and what my colleagues can teach me.

Spooky Promises from scary code

When I built a Halloween themed video call demo to prank the team, in addition to setting up the video call elements my main run() function needed to retrieve a list of gifs from the Giphy API, and then to plop a random gif on the page.

Here’s the original code I wrote to do that:

async function run() {     
   getGifs();
        setInterval(() => {
          try {
            let url =
              window.giphs.data[Math.floor(Math.random() * 50)].images.original
                .url;
            document.getElementById('bday').src = url;
          } catch (e) {
            console.error(e);
          }
        }, 20 * 1000);
// Some other things happen here too 
}

While this code worked, you might’ve noticed the same thing Phil did:

If you’re at a similar spot in your programming journey to where I was before I wrote this post, your first thought reading his comment might’ve been, "Oh! I just need to store the return value of getGifs inside a variable."

This first attempt led to bad news bears, or, a lot of pending Promises in my spooky.html:

Oh no. Promises. They’re on almost every interview questions list, but I somehow got this job even though I’m still a bit scared seeing these errors, what am I even doing?!?

Better stop that narrative and take a breath. And then take a Google.

Promises and async/await

There are loads of fantastic articles about JavaScript Promises and async/await out there. The part I needed to understand to fix my code, the part that Phil helped surface from the noise, is that the async/await pattern is syntactic sugar on top of Promises.

While I got the async part of the pattern ahead of my async function run(), I forgot the await. Await, well, tells a function to wait on the next step until a Promise resolves. I saw all those {<pending>} Promises because await was missing.

With that fixed, I could focus on specifying return values and replacing implied global variables.

Variable scope and unpredictable consequences

It’s helpful for me to trace back every step a function makes, so I went back to my getGifs() function:

async function getGifs() {
        try {
          const token = '<INSERT_GIPHY_API_KEY_HERE>';
          const giphyEndpoint = `https://api.giphy.com/v1/gifs/search?api_key=${token}&q=halloween&rating=pg`;
          let response = await fetch(giphyEndpoint);
          gifs = await response.json();
          return gifs;
        } catch (e) {
          console.error(e);
        }
      }

It’s not just my run() function, that was missing variable declarations. gifs = await response.json() in getGifs() is missing one too.

When I called getGifs() in run(), I was telling the function to create a side effect and change the state of a global variable on the window object. If someone else wrote gifs = somewhere else, that could override the values I actually wanted.

See what I mean in this codepen.

"Color circles" fills the initial circle colors. Since we didn’t scope the colors as variables within the colorCircles() function, they became global variables on the window object. That means we can "accidentally" override() them in the next function, and reset() them too.

While that side effect works for the purposes of an example codepen, keeping track of the colors as they get swapped is still hard enough to follow. It's like Elle Woods said:

Consequences of implied globals can be greater in larger applications, or even when it comes to picking gifs to prank your colleagues.

Final code and final takeaways

let gifSearchResults = await getGifs();
          setInterval(() => {
            try {
              let url =
                gifSearchResults.data[Math.floor(Math.random() * 50)].images.original.url;
              document.getElementById('gifs').src = url;
            } catch (error) {
              console.error(error);
            }
          }, 20 * 1000);
        );

In the final code, I’m using the actual response object from my call to getGifs(). Now, if I want, I can reuse the function in other places, pass in specific search parameters, and use multiple instances of the return object instead of just one globally. Best of all, the state outside of the object won’t get accidentally mutated.

After this code review, I know a bit more about how async/await works and principles of good functional programming. Beyond that, I also learned:

Digging around before asking other devs for help can lead to better debugging and faster learning (Julia Evans’ post describes this well!).
That said, sometimes sharing learnings in progress can be good too! When I shared my first pass at what I thought I learned with Phil, he helped point out the most important parts.
Even "silly" projects can teach you useful things. Because I built an app that picked random Halloween gifs, I now better understand why mutating state outside of a function itself is Bad Functional Programming.

Follow your heart! Build what’s fun! Like my friend Chloe says, it’s all digital crafting.

Let me know what things you’re excited about building over @kimeejohnson, and especially let me know if you’ll be building something with video chat.

Add text chat to a video call built on React and daily-js

Kimberlee Johnson — Wed, 09 Dec 2020 18:34:03 +0000

When Paul built a custom video chat app using React and the daily-js library, he used the Daily call object.

The Daily call object is like a direct line to the Daily API. It gives us the finest-grained control over a video call, letting us access its lowest level foundations, like participant video and audio tracks. With that great access comes great flexibility! But, it also means that features like text chat need to be added from scratch.

This post walks you through how to do that!

What we'll build

We're adding to Paul's demo React video app. When a user clicks to start a call, the app passes a Daily room URL to a new Daily call object, then joins the call. The call object tracks important information about the meeting, like other participants (including their audio and video tracks) and the things they do on the call (e.g. muting their mic or leaving), and provides methods for interacting with the meeting.

We'll use the sendAppMessage method and corresponding "app-message" event to add text chat to the app.

What you'll need to build it

Daily account: Sign up for an account if you don’t have one already.
Cloned daily-demos/call-object-react Github repository: The fastest way to follow along with this tutorial and get a demo app up and running is to clone this repo.
Familiarity with React: In this post we skip over a lot of the code that isn’t related to Daily, so it could be worth brushing up on React and/or hooks [0].

With that, we're ready to start!

Add text chat functionality

Our <Chat> is a form that does a few things with the participant <input>, meaning their chat message. First, via React’s useState hook and our handleChange function, it tracks their input in the component state inputValue.

const [inputValue, setInputValue] = useState('');

const handleChange = (event) => {
   setInputValue(event.target.value);
 };

When a participant sends a message, they call handleSubmit, which broadcasts the message, stored in inputValue to other participants on the call using Daily’s sendAppMessage method.

callObject.sendAppMessage({ message: inputValue }, '*');

handleSubmit also updates the sender’s chatHistory state.

const name = callObject.participants().local.user_name
     ? callObject.participants().local.user_name
     : 'Guest';
   setChatHistory([
     ...chatHistory,
     {
       sender: name,
       message: inputValue,
     },
   ]);
   setInputValue('');

In the return statement, <Chat> maps over every value in chatHistory to display the messages.

{chatHistory.map((entry, index) => (
       <div key={`entry-${index}`} className="messageList">
         <b>{entry.sender}</b>: {entry.message}
       </div>
 ))}

That covers sending messages. Now, we're ready to tackle making sure the app notifies recipients and displays their messages.

Alert participants to new messages

After the Daily sendAppMessage fires, it triggers the Daily “app-message” event. Our handleAppMessage event listener reacts to every "app-message." The handler gets the name of the sender from the Daily call object, and updates recipients’ chat histories using setChatHistory.

const participants = callObject.participants();
     const name = participants[event.fromId].user_name
       ? participants[event.fromId].user_name
       : 'Guest';
     setChatHistory([
       ...chatHistory,
       {
         sender: name,
         message: event.data.message,
       },
     ]);

After updating state, the function calls the props.notification() passed down from the parent Tray.js component to highlight the recipient’s chat icon.

We added that chat icon in Tray.js, where we pass state booleans and controller functions as props to Chat.js that determine whether the icon is highlighted or the chat window should be displayed (if it’s been clicked).

const [displayChat, setChatDisplay] = useState(false);
const [highlightedChat, setChatHighlight] = useState(false);

function toggleChat() {
   setChatDisplay(!displayChat);
   if (highlightedChat) {
     setChatHighlight(!highlightedChat);
   }
 }

function handleNewChat() {
   setChatHighlight(!highlightedChat);
 }

// Relevant component snippets 
<TrayButton
       type={TYPE_CHAT}
       disabled={props.disabled}
       highlighted={highlightedChat}
       onClick={toggleChat}
     />
     <Chat onClickDisplay={displayChat} notification={handleNewChat} />

Want to chat more?

There are so many ways to keep customizing Daily calls. While we hope this post is a good springboard for adding text chat to your video apps, you might be interested in more advanced functionality, like enabling file exchange or a chat history that persists once a call ends. A third-party chat integration would be the best way to do that. Stay tuned for more!

For now, experiment with our repository, and please send feedback our way any time at help@daily.co.

[0] If you’d like to get familiar with React and come back later, check out the many great resources on DEV (like Ali Spittel’s intro), or the React docs for more on hooks.

Add a text or image overlay to a video element

Kimberlee Johnson — Thu, 03 Dec 2020 18:44:48 +0000

We talk to a lot of developers building video calls at Daily, and one thing they often want to do is overlay text (like a participant’s name) or small images (muted state indicators or logos) on top of a video element. This post walks through how to do that!

It took me more time to take this screenshot with one hand than it took me to add my name.

First, we’ll cover the foundational CSS for positioning one element on top of another. Then, we’ll apply that CSS and build on top of Paul’s React video chat app tutorial.

Arrange and stack elements with CSS

We’ll set position and z-index properties to arrange our elements.

position gives us control of how an element will sit in the overall layout of the page. When the property is not set, every block-level HTML element appears on a new line [0]. We don’t want that! We specifically want our name tag directly on top of and overlapping our video container. Where the name tag goes depends on the video's position.

To set up this dependent relationship, we set our video's position property to relative. Then, we can arrange any child elements, in our case our name tag, in relation to it by setting their position property to absolute.

To see this in action, experiment with removing position:relative from the .parent-container class in this codepen:

Our boxes’ top, bottom, right, and left properties offset them relative to .parent-container.

With the dependent relationship established, it's time to move on to stacking elements. To do that, we'll need the z-index property. Because we set position properties, we can make use of z-index to stack our elements. The higher the z-index number, the closer to the screen the element will be. Swap the .red-box and .green-box z-index values in the codepen to see what I mean.

We now know how to arrange child elements in relation to their parents using position, and how to stack them with z-index. We’re ready to take those concepts over to our React video chat app, but first let’s look at how we can get participant names from the Daily call object.

Passing participant names as props in React

The Daily call object keeps track of our call state, meaning important information about the meeting. This includes details like other participants (e.g. their audio and video tracks and user_name) and the things they do on the call (e.g. muting their mic or leaving)[1]. The call object also provides methods for interacting with the meeting.

In our demo app, we map the Daily call object state to a corresponding component state called callItems in callState.js. Each call item represents a participant, and contains their audio and video tracks, along with a boolean state indicator about whether or not their call is loading. To also track participant names, we'll add participantName to each call item.

const initialCallState = {
 callItems: {
   local: {
     isLoading: true,
     audioTrack: null,
     videoTrack: null,
     participantName: '',
   },
 },
 clickAllowTimeoutFired: false,
 camOrMicError: null,
 fatalError: null,
};

We need to add participantName to our getCallItems function as well. This function loops over the call object to populate our callItems.

function getCallItems(participants, prevCallItems) {
 let callItems = { ...initialCallState.callItems }; // Ensure we *always* have a local participant
 for (const [id, participant] of Object.entries(participants)) {
   // Here we assume that a participant will join with audio/video enabled.
   // This assumption lets us show a "loading" state before we receive audio/video tracks.
   // This may not be true for all apps, but the call object doesn't yet support distinguishing
   // between cases where audio/video are missing because they're still loading or muted.
   const hasLoaded = prevCallItems[id] && !prevCallItems[id].isLoading;
   const missingTracks = !(participant.audioTrack || participant.videoTrack);
   callItems[id] = {
     isLoading: !hasLoaded && missingTracks,
     audioTrack: participant.audioTrack,
     videoTrack: participant.videoTrack,
     participantName: participant.user_name ? participant.user_name : 'Guest',
   };
   if (participant.screenVideoTrack || participant.screenAudioTrack) {
     callItems[id + '-screen'] = {
       isLoading: false,
       videoTrack: participant.screenVideoTrack,
       audioTrack: participant.screenAudioTrack,
     };
   }
 }
 return callItems;
}

getCallItems gets called in Call.js [2]. It then passes the callItems as props via the getTiles function to <Tile>, the component that displays each participant. We’ll add participantName to the list of props:

export default function Call() {
// Lots of other things happen here! See our demo for full code.
//
function getTiles() {
   let largeTiles = [];
   let smallTiles = [];
   Object.entries(callState.callItems).forEach(([id, callItem]) => {
     const isLarge =
       isScreenShare(id) ||
       (!isLocal(id) && !containsScreenShare(callState.callItems));
     const tile = (
       <Tile
         key={id}
         videoTrack={callItem.videoTrack}
         audioTrack={callItem.audioTrack}
         isLocalPerson={isLocal(id)}
         isLarge={isLarge}
         isLoading={callItem.isLoading}
         participantName={callItem.participantName}
         onClick={
           isLocal(id)
             ? null
             : () => {
                 sendHello(id);
               }
         }
       />
     );
     if (isLarge) {
       largeTiles.push(tile);
     } else {
       smallTiles.push(tile);
     }
   });
   return [largeTiles, smallTiles];
 }

 const [largeTiles, smallTiles] = getTiles();

return (
   <div className="call">
     <div className="large-tiles">
       {
         !message
           ? largeTiles
           : null /* Avoid showing large tiles to make room for the message */
       }
     </div>
     <div className="small-tiles">{smallTiles}</div>
     {message && (
       <CallMessage
         header={message.header}
         detail={message.detail}
         isError={message.isError}
       />
     )}
   </div>
 );
}

Now, in Tile.js, we display the name:

export default function Tile(props) {
// More code
function getParticipantName() {
   return (
     props.participantName && (
       <div className="participant-name">{props.participantName}</div>
     )
   );
 }

 return (
   <div>
     <div className={getClassNames()} onClick={props.onClick}>
       <div className="background" />
       {getLoadingComponent()}
       {getVideoComponent()}
       {getAudioComponent()}
       {getParticipantName()}
     </div>
   </div>
 );
}

And style it using familiar CSS in Tile.css, with our container tiles set to relative positioning and our video streams and name tags set to absolute:

.tile.small {
 width: 200px;
 margin: 0 10px;
 position: relative;
}

.tile.large {
 position: relative;
 margin: 2px;
}

.tile video {
 width: 100%;
 position: absolute;
 top: 0px;
 z-index: 1;
}

.participant-name {
 padding: 5px 5px;
 position: absolute;
 background: #ffffff;
 font-family: 'Helvetica Neue';
 font-style: normal;
 font-weight: normal;
 font-size: 1rem;
 line-height: 13px;
 text-align: center;
 color: #4a4a4a;
 top: 0;
 left: 0;
 z-index: 10;
}

And there you have it!

If you have any questions or feedback about this post, please email me any time at kimberlee@daily.co. Or, if you’re looking to explore even more ways to customize Daily calls, explore our docs.

[0] This is not the case for inline elements.

[1] A participant’s user_name can be set in a few different ways. It can be passed as a property to the DailyIframe, or set with a meeting token.

[2] More specifically, any time there’s a change to participants on the call, Call.js dispatches an action to a reducer that updates state via getCallItems.

Monitor how code performance impacts video call quality

Kimberlee Johnson — Mon, 16 Nov 2020 18:43:09 +0000

Add a stats.js bookmarklet to your video app for immediate insights

If you’re reading this post in 2020, then the odds are high that you’ve been on a lot of video calls this year. And that means that more than once you’ve probably experienced the frustrations that accompany low quality video calls: delayed audio or video, blurry screens, patchy connections, to name a few. In order to avoid these hiccups, it’s useful to monitor call performance to improve video apps.

Developers building with our Daily video chat APIs can use our prebuilt UI or build their own custom video chat layout on top of the Daily call object. Both options support ways to gain insights into call performance.

We built a network tab in the Daily prebuilt UI to show how the video chat adapts to the local network conditions. Daily calls work well even on networks with limited bandwidth and high packet loss, and the network tab gives users some insight into what's happening under the hood. Call participants can see packet loss, download rate, and upload rate.

If you’re a developer building on top of the Daily call object, you can monitor for several network events and build your own monitoring panel.

While we built these tools for monitoring network performance, we’re also often curious about how the performance of the code in the demo apps we build might impact the quality of our calls. We realized other developers building on Daily might also be asking this question.

Enter stats.js!

Stats.js is a JavaScript performance monitor that tracks:

Frames per second (FPS): a low number means there’s something we should change.
Milliseconds needed to render a frame (MS): a high number signals we should make some updates.
MBytes of allocated memory (MB): Run Chrome with --enable-precise-memory-info.

To use, create a new bookmark, saving this bookmarklet as the URL:

javascript:(function(){var script=document.createElement('script');script.onload=function(){var stats=new Stats();document.body.appendChild(stats.dom);requestAnimationFrame(function loop(){stats.update();requestAnimationFrame(loop)});};script.src='//mrdoob.github.io/stats.js/build/stats.min.js';document.head.appendChild(script);})()

Then, navigate to your web app, and click on the bookmark. When you do, the panes will display. You can click through to see the different stats.

If you test this out in your projects, let us know what you think! In the meantime, stay tuned for more updates about monitoring Daily call performance.