DEV Community: Dylan Jhaveri

Mux is the video API for the JAMstack

Dylan Jhaveri — Wed, 08 Apr 2020 16:28:01 +0000

What is the JAMstack?

The JAMstack is a term popularized in the last year, largely by the React community and companies like Netlify and Zeit. Specifically, JAMstack stands for "Javascript", "APIs" and "Markup". These terms don't exactly describe what the JAMstack is in a clear way, but the name itself has a nice ring to it so it seems to have stuck.

Here is a breakdown of all the pieces for a "JAMstack" application and what some of the popular options are. For a more exhaustive list you might check out awesome-jamstack on Github.

Static content frameworks

This covers the "Javascript" and "Markup" part of the stack.

Next.js: Open source, write everything with React and the framework gives you automatic code splitting and a server-side rendered web application.
Gatsby: Also open source and you write everything with React components. The Gatsby framework handles code splitting and lazy loading resources. Gatsby also has a concept of “sources” where you can write GraphQL queries to pull in data from 3rd party sources via a plugin.
11ty: A static site generator that works with all kinds of templates: markdown, liquid templates, nunjucks, handlebars, mustache, ejs, haml, pug and Javascript template literals

Deploy

These are platforms that can host your statically built application. With common JAMstack frameworks you end up with static files that can be hosted by a static file server and delivered over a CDN.

Cloud Functions (“Serverless”)

All of these services, in one way or another, allow you to write code in javascript that handles an API request and returns a response. This, along with other 3rd party APIs is the "API" part of the stack. The serverless part is that you don’t have to worry about the details on how or where that code gets run. These platforms will handle the server configuration and the deployment of your API endpoints as “cloud functions” or “lambdas”. In your client side application, you can make requests to these functions the same way you would make requests to API endpoints that you would have deployed to your own traditional server.

Headless CMS

A “headless” CMS is a CMS that gives you and your team an interface to log in, edit content, add new content, upload assets and the “publish” data that makes it into your website or application.

There are many headless CMSes. We are a little biased, so these are the one ones that work with Mux and these are the ones that we have used. Look around for what works for you. And if you have one that you want to use with Mux, let us know and we can build an integration.

Authentication (advanced)

If you’re building a static marketing site you probably will not need to deal with authentication. However, for a more advanced application you will need to have users login, reset passwords and do all the pieces of authentication.

Database (advanced)

If you are authenticating users and dealing with logged in sessions, you probably need a database. These are commonly used for JAMstack applications.

How did we get here?

Before these tools gained popularity the answer to “What stack should I use for my marketing site?” might have been “use Rails” and that is a clear answer. But now if someone says “use the JAMstack” well, that is a complicated answer. It’s a little misleading to call the “JAMstack” a specific stack, because as you can see from above, even if you decided to use the JAMstack, you still have a lot of choices to make.

Before the JAMstack was popularized, we have had a long history of static site generators. You may remember Jekyl or Middleman from the Ruby community. These tools allowed you to write Markdown, Liquid or Ruby’s ERB templates and generate a static site that you could host somewhere like s3 to host your blog. These tools are great and they are still widely used.

These static site generators were great for developers that wanted to make something like a blog or a simple marketing website. Someone non-technical might reach for a tool like Wordpress or Squarespace, whereas a hacker would turn to a static site generator.

For more advanced applications that went beyond statically rendered HTML, we had to switch gears away from static site generators and into a web framework like Rails.

Then advanced frontend frameworks for building interactive single page applications became popular: Angular, Ember and React. Suddenly, frontend developers had all these tools and got comfortable writing React code for their applications. But for static marketing sites we couldn’t write React or Angular code because we still needed static HTML for SEO purposes and fast initial load times. Developers were stuck in a world where we wrote what we were comfortable with for our application frontend but then for our marketing site had to switch back to some ad-hoc cobbled together jQuery functions.

The biggest feature that made the JAMstack popular is that you get the best of both worlds: server-side rendered HTML plus interactive React components that you can do whatever you want with. This is the big innovation and the first “oh wow” moment I had using both Next.js and Gatsby. You write normal React like you’re used to, run the build process and then all of a sudden you end up with static HTML returned by the server and all your interactive React code works as you would expect.

Video for the JAMstack

Mux is the video API for the JAMstack. The philosophy behind Mux and how we approach video fits in neatly with the JAMstack philosophy. Mux will act as your video infrastructure by handling the storage, hosting and delivery of your video without getting in the way or being opinionated about the presentation.

In fact, Mux does not even give you a video player. You have to bring your own player to the party. The entire “frontend” of the video experience is up to you, Mux is focused on handling the backend or the “serverless” part of your video stack. Think of Mux as the headless video platform. You control every bit of the user experience while Mux does the heavy lifting behind the scenes.

JAMstack at Mux

In addition to providing APIs that you can use for your JAMstack website, Mux also uses the JAMstack ourselves to power our marketing site (mux.com) and the Mux blog.

A couple of months ago we finished the process of moving the Mux Blog to the JAMstack. Before this project, the Mux blog was hosted and deployed separately from mux.com. The blog was powered by an old version of Ghost, using the default Casper theme. Our marketing site is a Gatsby site that uses gatsby-source-filesystem to create some pages from markdown and gatsby-source-airtable to pull in some data from Airtable.

The main issue with our existing blog that we wanted to address was that since we were using a Ghost theme, not only was the design of the blog completely different from the design of the rest of our marketing website, but it was an entirely different application with a different structure, hosting and deploy process.

As a result, visitors that landed on a blog post didn’t have an easy way to get back to the main marketing site and since the look and feel didn’t exactly line up, the experience was too disconnected. We decided that we wanted to move everything to a headless CMS so that we could make the blog part of our existing Gatsby marketing site for consistency.

Migrating to a headless CMS

There are pre-built Mux integrations for Sanity, Contentful, and Cosmic. All of these options allow you to bring your own Mux account. Alternatively, Dato is a headless CMS that offers native video built into the product that is powered by Mux.

We ended up choosing Sanity as our headless CMS. We loved that Sanity felt like an open-ended developer product that could grow with our needs past just the blog today. Calling Sanity a headless CMS sells it short from what it really is: it’s more akin to a structured, real-time database. The CMS part is all open source and in your control for how you want things to look and work. The way to think about it is that Sanity provides a real-time database along with some low-level primitives to define your data model, then from there, you build your own CMS.

As a part of this project of moving the blog to a new CMS, we wanted to set ourselves up with a headless CMS that could be used beyond just the blog and could also create a variety of pages on mux.com and allow us to move existing content like the video glossary.

For a more technical in-depth read about how we did this, check out this Sanity Guide we wrote How to migrate your HTML blog-content from Ghost and the blog post Moving the Mux blog to the JAMstack.

How to Host Your Own Online Conference

Dylan Jhaveri — Tue, 03 Mar 2020 00:01:30 +0000

Online conferences seem to have gained in popularity the past year. We’re seeing more and more folks reach out with questions around best practices and how to pull off their own remote conference. Sometimes, people will announce a conference, get thousands of sign ups and then one or two weeks before it’s set to go live they will be figuring out how to do it; how much different from a normal video conference call could it be? For our purposes let’s say you want to build a custom experience and host an online conference on your own website.

Use this guide as your playbook. This is what we will cover:

You have multiple presenters that will be broadcasting live from different locations.
Your team has no video expertise at all, but you do have a technical team that is capable of building a functioning web application.
The experience you want to provide to your audience is something custom that you control. Your brand is important for you and you want to control the look, feel and user experience of the conference.
To broaden the reach of your conference you simultaneously want to broadcast out the video feed to social channels like Youtube Live, Facebook Live and Periscope.
If possible, you would really like to have a branded overlay with your logo on the video.
In addition to streaming live, you want to record the broadcast so that people who did not attend live are able to view the recordings on-demand as soon as each session is over.

The basic structure of your setup is going to be a live conversation that is broadcast to a larger group of live viewers. The live conversation could be something like one person presenting with a screen share, or one person interviewing someone else, or a panel discussion among a group of experts.

A really simple way to do this live conversation is to use Zoom. Most people are familiar with Zoom. It is one of the most stable, reliable and high quality pieces of meeting software around and it runs natively on your desktop. What’s really cool about Zoom is that if you enable live streaming for meetings then you can set up an RTMP output for your Zoom call to any arbitrary RTMP endpoint (this is where Mux comes in).

Adding Mux in the middle is how you can broadcast your Zoom call to an audience of thousands on your own website. The live audience does not have to download Zoom, they do not interact with Zoom at all. All they do is see a video player that you make on your website.

Let’s break down the steps and API calls:

1) Make sure in Zoom settings you have allowed meetings to be live streamed.

2) Create a Mux live stream - this is one API call and for every live stream you create you will get back a unique stream key also make sure you save a playback_id for this live stream - you will need this to play the live stream.

3) Set up a Zoom call like you normally would. From the call, click the 3 dots at the bottom where it says "More" and click "Live on Custom Live Streaming Service".

4) From here, enter the RTMP server details for Mux.

5) When you’re ready to stream click “Go Live!” from Zoom. Now your Zoom call will be live and Mux will start receiving the video and audio. To confirm that this part is working, navigate to the live stream in your Mux dashboard and you should see video and audio coming in. Later you can set up Webhooks so that you can be notified when every live stream is connected, active, completed, etc.

6) Next is the player side, you have the playback_id from step 3, right? You will need to take that ID and form a URL like this: https://stream.mux.com/{playback-id}.m3u8. This is an “m3u8” URL which is a URL for streaming video over HLS. HLS is a standard streaming format for both live and on-demand video. You will need to use this HLS URL in a video player. Which player you choose is entirely up to you. Here’s two free ones you can check out to get started: videojs and plyr. With whichever player you choose, follow the instructions for streaming a HLS video.

Note that HLS streaming will come with some latency. Expect for 15-20 seconds, there is a reduced_latency flag that you can use which will bring that number down closer to 8 or 10 seconds (with some tradeoffs). Check the docs here for more information.

Live chat

After you get the live stream of your conference working on your webpage, you will almost certainly want to add a live chat component. Live chat is outside the scope of what Mux offers, but we’ve seen this done in a few different ways:

Use your own database and push real time changes to your clients with something like Pusher, PubNub or Socket.IO.
Use a realtime database like Firebase and build your own chat experience.
Try something like Stream which offers real time APIs specifically around creating chat experience. Fully featured with things like uploading images to chat, emoji reactions, typing notifications and all the bells and whistles are built in.
Skip the step of adding chat on your webpage and create a Slack community where everyone can chat. The benefit of having a slack community is that it’s free and you don’t have to go through the steps of building chat onto your website. Most people are already familiar with Slack, so they likely have it downloaded already. You can also create channels for specific topics and allow attendees to DM each other. This has the added benefit of allowing for the kind of attendee networking that happens at in-person conferences.

Everything is recorded

Every live stream that you do will be recorded by Mux and the asset will be available for on-demand playback. After each live stream is over, you can use the Mux asset to give attendees who were not present live the ability to view the recording.

Bonus: simulcast to the socials

When you create a Mux live stream you can optionally add simulcast_targets which are arbitrary RTMP endpoints that Mux will push out your stream to. It’s fairly straightforward and we have some guides on how to do this. Read more in the announcement blog post and the guide. All you really have to do is track down the RTMP server URL and stream keys for each of the social networks you want to broadcast to and add them to the Mux live stream with an API call.

Bonus: add a watermark

In the new_asset_settings parameter when you create the live stream, you have the option to specify a watermark. You give Mux the URL to an image you want to use as a watermark and some details about where to place it and how to align it. When this is configured, Mux will add it to the stream that comes from Zoom and the watermark will appear on the HLS stream that you show on your website.

You don’t have to use Zoom

Zoom is the simple example I used here because many people are familiar with it and know how it works. But the reality is you can use any software that allows you to RTMP out to Mux. To name a few other options: OBS, Wirecast, ecamm live, all of these products are built to compose a single video stream and RTMP out to somewhere. All of these will work with Mux.

Final Setup

Here’s what your final setup might look like. If you are going to host an online conference with Mux, please reach out! We would love to talk to you and help you make sure it’s successful.

Extra details if you’re curious

Mux supports live streams for up to 12 hours. If you have one single stream of an all day conference then this should be enough.
There are two options for playing the live stream on your website, you can either use a playback_id associated directly with the live stream, OR you can use the playback_id associated with the active_asset that is associated with the live stream. The former will not allow seeking backwards in the stream, the latter will allow your attendees (if they want) to seek all the way back to the beginning. This is a subtle detail but it might be something you want to consider.

If you’re doing an online conference, please get in touch!

Dylan Jhaveri — Thu, 23 Jan 2020 17:19:28 +0000

If you’re trying to autoplay videos on the web, you might be tempted to reach for the HTML5 autoplay attribute. This sounds exactly like what you’re looking for, right? Well, not exactly. Let’s talk about why that’s probably not what you’re looking for and what the better option is.

Browsers will block your autoplay attempts

Over the last few years, all major browser vendors have taken steps to aggressively block autoplaying videos on webpages. Safari announced some policy changes in June 2017 and Chrome followed suit shortly after and Firefox after that.

In summary: all these browsers will aggressively block videos from autoplaying on webpages. Each browser has slightly different rules around how it makes this decision. It’s a huge black box and browsers will not tell you what their exact rules are. The default behavior is block most autoplay attempts.

There are however some conditions that make it more likely for autoplay to work:

Your video is muted with the muted attribute.
The user has interacted with the page with a click or a tap.
(Chrome - desktop) The user’s Media Engagement Index threshold has been crossed. Chrome keeps track of how often a user consumes media on a site and if a user has played a lot of media on this site then Chrome will probably allow autoplay.
(Chrome - mobile) The user has added the site to their home screen.
(Safari) Device is not in power-saving mode.

These conditions only make autoplay more likely, but remember that aside from these conditions, the user can override the browser’s default setting on a per-domain basis. This basically means that you can never rely on autoplay actually working.

Autoplay will probably work for you, but it will break for your users

Even if you try to follow the rules above, autoplay is still a finicky beast. One thing to keep in mind (for Chrome at least) is that due to Chrome’s Media Engagement Index. When you are testing autoplay on your own site it will probably work for you (because you visit your site often and play content, your MEI score is high). But then when new users come to your site, it is likely to fail (because their MEI score is low). As a developer, this is incredibly frustrating and another reason to always avoid the autoplay attribute.

What should I do instead?
I’m not suggesting that you avoid autoplaying videos, but I am suggesting that you always avoid the autoplay attribute. There is a better way.

Use video.play() in javascript world, which returns a promise. If the promise resolves, then autoplay worked, if the promise rejects then autoplay was blocked.
If the promise returned from video.play() rejects, then show a play button in the UI so that the user can click to play (the default video controls attribute will work just fine). If you are using your own custom controls and your javascript calls video.play() again as the result of an event that bubbled up from a user click, then it will work.
Consider starting with the video muted, this gives you a much lower chance of your video.play() call rejecting. You will want to show some kind of “muted” icon in the UI that the user can click to unmute (again, the default video controls attribute works great for that). You may notice that twitter and a lot of sites start videos in the muted state.
Have I mentioned showing controls for your player? Always make sure controls for your player are accessible. We have seen sites try to get fancy and be too minimalist by hiding controls. Inevitably, they run into situations where autoplay fails and the user has no way of clicking to make the video play. Make sure you do not fall into this trap.

Here’s an example with vanilla javascript

// get a reference to a <video> element
const videoEl = document.querySelector('video');

// attempt to call play() and catch if it fails
videoEl.play().then(() => {
  console.log('Autoplay success!');
}).catch((error) => {
  console.warning('Autoplay error', error);
});

Here’s an example with React

If you look at mux.com you’ll see that we autoplay a video on the top of the home page. I copied over how we did that and set up a demo here: https://o9s4w.csb.app/. The code is copied below and you can fork it and play around with the Code Sandbox.

Notice that we’re doing a few things here:

Try to call video.play() when the component loads.
Show the user a play/pause state in the UI by using the default controls attribute.
Start with the video in the muted state. Our video does not have audio, but if it did you would still want to start off in the muted state with the muted attribute, and show a mute/unmute icon in the UI.

import React, { useEffect, useRef } from "react";
import "./styles.css";

export default function App() {
  const videoEl = useRef(null);

  const attemptPlay = () => {
    videoEl &&
      videoEl.current &&
      videoEl.current.play().catch(error => {
        console.error("Error attempting to play", error);
      });
  };

  useEffect(() => {
    attemptPlay();
  }, []);

  return (
    <div className="App">
      <h1>Autoplay example</h1>
      <div>
        <video
          style={{ maxWidth: "100%", width: "800px", margin: "0 auto" }}
          playsInline
          loop
          muted
          controls
          alt="All the devices"
          src="https://stream.mux.com/6fiGM5ChLz8T66ZZiuzk1KZuIKX8zJz00/medium.mp4"
          ref={videoEl}
        />
      </div>
    </div>
  );
}

No BART terminals were hacked in the making of this ad

Dylan Jhaveri — Tue, 21 Jan 2020 22:55:36 +0000

Originally posted by my colleague Bonnie Pecevich on mux.com/blog

In the process of creating a BART ad for the first time, we had some learnings that we thought we would share that could hopefully help someone else on their out-of-home ad buying journey. (We’ll also remember to follow our own advice for next time.)

Our learnings:

Ask upfront for explicit restrictions on creative.
Build in extra time for more than one round of feedback and time to iterate on design.
Be realistic about what’s feasible, especially with an aggressive timeline.
Submit a draft of the concept and see if they’ll approve it before you spend extra time finalizing the details (and telling everyone about it at the company all hands meeting.)

All of these learnings actually stemmed from one preeminent learning:

BART doesn’t allow any code on their ads. 😲

Why put code in an ad?

First, an introduction–Mux is a startup that does video, and one of our aspirational goals is for every developer to know that. With our headquarters located in San Francisco, we’re aware that our city has a great supply of developers so we thought we’d try advertising in some well-traveled, public spaces.

Doing ads in a BART station (the underground transit system in the Bay Area) is generally assumed to be expensive, maybe even beyond the reach of a startup which is what we thought, too. But we learned doing an ad could fit in our budget if we were flexible on timing–we were able to sign up for a single digital display at the Montgomery BART station with a 12/30/19 start date. Even though that only gave us about 2 weeks to create an ad (ignore the wailing coming from our one and only in-house designer), we were excited!

Since the ad is just :15 seconds long with a not-so-captive audience, we wanted to create something that quickly caught the attention of developers. We thought we could achieve this by showing a terminal with a blinking cursor and then typed code to show use of our API. Sure, it crossed our minds that a blank screen with a blinking cursor might look like the screen is broken (which adds to the eye-catching-ness), so we added browsers to frame the terminal and added our logo to the top left corner. Our hope was that someone would take away that Mux is for developers and, if we were lucky, that we do something with video.

Insert wrench here

The process was to submit a final file at least a week in advance of the live date to include time to get BART’s approval. There weren’t any specific guidelines beforehand on what’s allowed and what’s not but we assumed some common sense restrictions would apply like no explicit/harmful language imagery, etc. We figured getting BART’s approval would be relatively simple, like checking a box.

Wrong. Our ad was rejected! We received feedback that the beginning of the ad that showed the terminal could give the impression that “the screen is malfunctioning or has been hacked into.”

Busted. Turns out they also thought having a terminal on the screen would be eye-catching but not in a good way. We did feel a bit deflated, though, as we were all ready for our BART debut.

We went through the five stages of grief and settled on “Bargaining.” We tried to come up with a creative solution where we could still use the same ad. Hey, what if we could add a persistent banner to the ad that said something like “Don’t worry, no BART terminals were hacked in the making of this ad.”?

Or what if we stylized the terminal so it looked more illustrated and cartoon-y?

Alas, BART held firm and said, in no uncertain terms, “Nothing involving coding.” Since we couldn’t come up with a brand new design in 48 hours, our plans for a BART ad had to be put on hold.

Silver lining

All is not lost! We used the final video for our homepage and are genuinely excited at how it came out.

Although the BART approval process is still a bit of a black box, we're excited to continue to work with the same ad agency and pursue our out-of-home ad dreams. We’re looking forward to iterating our design and hopefully making a public appearance at CalTrain in the very near future. And if you see our ad, you’ll know the journey it took to get that little video up on those screens.

In defense of 'flicks' (or how I learned to stop worrying and love 705600000)

Dylan Jhaveri — Tue, 26 Nov 2019 18:26:56 +0000

Note originally posted by my colleague on the mux blog

About 2 years ago, the Oculus VR division of THE FACEBOOK created a project they called 'flicks'. Essentially a flick is just a really big number, specifically the number 705,600,000. This project was picked up by some news outlets like TechCrunch, The Verge, and the BBC and seems to cause some confusion and even some ridicule. To be fair, a news article about a number is a bit odd. If you’re not an engineer working with digital media, the idea behind this number is difficult to grasp. And to those who do work in digital media, the number seems to not offer anything new. It purports to solve a problem that nobody in the industry actually has. So where did it come from, and why does it exist? Let’s back up…

Time is a surprisingly difficult concept in digital media. For starters, we are dealing with time values that are very small and difficult to imagine. Recently I saw the movie Gemini Man at the AMC Metreon here in San Francisco. It was one of the few theaters capable of playing the 120 frames per second version, whereas most films are 24 frames per second. At 120fps, every frame is projected for just over 0.00833 seconds before flashing the next one– a very short period of time. But compared to digital audio, this is an eternity. Audio recorded at 44100hz has a sample every 0.000022675 seconds. That is 367.5 times more audio samples than video frames.

Example audio visual timeline

These numbers with fractional components (a decimal point) are known as “floating point” numbers in computer science and computers are surprisingly bad at dealing with them. Above when I said every video frame was on the screen for 0.00833 seconds, that was not exactly true. When you divide 1 by 120 The number result is 0.008333333333 with the 3 repeating forever. For a computer to store a number that repeats forever would require an infinite amount of memory, so the number is approximated. The difference between the approximation and the actual number results in tiny errors in the math. These small errors can add up over time and become big errors and could result in problems such as audio and video becoming out of sync. Using a unit of time like milliseconds or nanoseconds would help, but would only delay the problem and not solve it.

The ultimate solution is to not record time in seconds but instead as an integer number of fractional units. For example 1000 x 1 ÷ 120 is the 1000th frame of a 120fps video. Converting to seconds we still end up with a floating point number but as long as we count frames as integers the error does not accumulate over time. If you’re following the math closely you may have noticed that while solving this problem we have created another one.

What frame should we render first? The video frame at 1000 x 1 ÷ 120, or the audio sample at 367500 x 1 ÷ 44100? We need to convert to a common time base to know for sure. We could convert to seconds then compare, but that brings us back again to the floating point problem. By using the “least common multiple” or LCM, of the ratios, 88,200 in this case, we can convert these fractions to a common time base at which point we can compare them. 88,200 ÷ 44100 x 367500 = 735,000, and 88,200 ÷ 120 x 1000 = 735,000. These time stamps are exactly the same and should be rendered together to ensure sync. At no time did we need to use floating point math, which may have given us a slightly different answer.

In the world of digital media there are some time bases that come up very frequently. As stated, film commonly uses 24 fps, European television (and other PAL countries) use 25 fps, and for obscure reasons, American television (and other NTSC countries) use 29.97 fps. Wait! Floating point numbers again? Actually no, because it’s not really 29.97fps. It’s actually 30000 ÷ 1001 fps.

Here is where flicks come in. You see 705600000 ÷ 44100 = 16000 EXACTLY, 705600000 ÷ 120 = 5,880,000 EXACTLY, and even 1001 x 705600000 ÷ 30000 = 23,543,520 EXACTLY. This is why the flick is interesting, it has a special property of being the least common multiple of many of the commonly used timebases in digital media.

We now know what the flick is, Buy why? We have established that if we record every time stamp as 3 integers, a numerator, a denominator and a multiplier, we don’t need a common base since we can convert between them as needed. There are two primary reasons. First is efficiency. If we know we will need to compare a lot of time stamps in different time bases, or compare the same timestamp multiple times, converting them to a common base can be faster for a computer. Once converted to a common base, comparing two numbers is probably the fastest operation a computer can do. Whereas comparing two fractions requires an algorithm to find a common base, convert the value, and then compare. But the motivation cited in the flicks github page is slightly different, and caused by a design decision in the C++ programing language.

Computers are pretty good at dealing with time, but humans are really bad at it. In most parts of the United States, once a year for daylight saving, we have a 25 hour, only to have a 23 hour day a few months later. Every 4 years, we get an extra day at the end of February, unless the year is divisible by 100; except when the year is also evenly divisible by 400 then it is a leap year (there was no leap day in the year 1900). We even have leap seconds, where we add an extra second to a year whenever we notice that the atomic clocks don't quite agree with astronomical observations. Meanwhile a computer's clock needs to keep moving forward one second per second otherwise bad things happen.

To help with this human nonsense and standardize how to manage time, C++11 added a new package called chrono to its standard library. Because the language designers were smart, the std::chrono::duration time type included support for the time as a ratio technique we have established. Perfect! Well... not so fast. Because the language designers were unwilling to give up more cpu cycles and slow down programs (C++’s defining feature is speed after all), it was decided that the time base must be known in advance while writing the program (compile time). This allows for fast running programs because the fractions can be ignored when they are known to be equal, but it sacrifices automatic time base conversions when they are not. Herein lies the problem. When playing back a media file we can’t know the time base in advance, we haven't seen the file yet. What we need is a time base that can support any media we are likely to encounter. Enter flicks. An elegant solution to a problem only a handful of media engineers will ever encounter that happened to be announced on a slow news day.

Flicks does not seem to be in wide use. I used it at Mux in one specific place in our transcoding pipeline with its intended purpose. I used it with a C++11 program where utilizing std::chrono made things a bit easier to standardize on. But searching GitHub and Google, I could only find a handful of places where it is used in the wild, and I really don't expect that to change.

Using Netlify Functions to Create Signing Tokens

Dylan Jhaveri — Thu, 07 Nov 2019 20:38:31 +0000

Have you used cloud functions yet? They come in many flavors: Amazon Lambda, Cloudflare Workers, Zeit Serverless Functions, and the one we’re using here: Netlify Functions.

Cloud functions are an essential component of the JAMstack. I had not heard the name JAMstack until recently. For the uninitiated (like me) it stands for Javascript, APIs and Markup. You may have seen JAMstack technologies like Gatsby, Next.js and tools of this nature that focus on performance, new developer tooling, and leveraging CDNs to serve pre-compiled HTML pages. I will be at JAMstack Conf 2019 in SF, if you will be there too, then come find me and say hi!

All the code in this post is open source here on GitHub in our examples repo: muxinc/examples/signed-playback-netlify.

The Main Benefits of Cloud Functions

Run code close to your clients (clients can be browsers, mobile apps, internet-of-things devices, self-driving cars, drones, anything that is talking to your server). Like a CDN, cloud functions are deployed to edge data centers to minimize latency between your clients and the server that runs their code.
Protect your origin servers from being flooded with traffic. Cloud functions are a good way to cache data and intercept requests and respond to your users before they reach your origin servers. This means less bandwidth and CPU that your origin servers have to process.

Cloud functions, like Netlify Functions, might be a good option for you if you are using Mux’s Signed URLs feature.

A Little Background About Signed Urls

When you create a video asset via Mux’s POST /video API you can also create a Playback ID (Mux API docs) and specify the playback_policy as either "public" or "signed".

A “public” playback policy can be played back on any site, in any player and does not have an expiration. A “signed” playback policy requires that when the playback URL is requested from a player, it has to be accompanied by a “token” param that is generated and signed on your server.

This is how it looks:

public playback URL:
https://stream.mux.com/${playbackId}.m3u8

signed playback URL:
https://stream.mux.com/${playbackId}.m3u8?token=${token}

The token param is what you need to create on your server in order for the playback URL to work.

Create a Mux Asset

Sign up for a mux.com account (free account comes with $20 credit)
Go to settings/access-tokens and click “Generate new token” to create a token you can use for API calls
Copy your token id (we'll call this MUX_TOKEN_ID) and secret (MUX_TOKEN_SECRET). You will need these to make two api calls.
Create a Mux video asset with a “signed” playback policy

curl https://api.mux.com/video/v1/assets \
  -X POST \
  -H "Content-Type: application/json" \
  -u ${MUX_TOKEN_ID}:${MUX_TOKEN_SECRET} \
  -d '{ "input": "https://storage.googleapis.com/muxdemofiles/mux-video-intro.mp4", "playback_policy": "signed" }'

Copy the playback_id from the response, you will need this later.

Create a URL Signing Key

curl https://api.mux.com/video/v1/signing-keys \
  -X POST \
  -H "Content-Type: application/json" \
  -u ${MUX_TOKEN_ID}:${MUX_TOKEN_SECRET}

Copy the id (MUX_TOKEN_ID) and the private_key (MUX_PRIVATE_KEY) from the response, you will need these later. These are the keys you will need to create signed urls for playback.

Setup a Netlify project

Create a new directory for your project mkdir netlify-mux-signing && cd netlify-mux-signing
Install the Netlify CLI and run netlify init to create a new project. You can choose to connect Netlify to a GitHub repository.
If you’re starting from scratch, run yarn init to create an empty package.json and git init to make this a git repository.
Now you have a barebones project that is connected to Netlify, but nothing is in it yet (you can see there is a hidden and gitignored directory called .netlify which Netlify uses to handle deploys and Netlify commands
Run yarn add netlify-lambda to install the netlify-lambda package into your project (it’s recommended to install this locally instead of globally).
Run yarn add @mux/mux-node to add the Mux node SDK to your project

Step 1: Create a module to generate a signing token

Create a src/ folder in your project and let’s create a small module called mux_signatures.js. It will export one function called signPlaybackId which takes a playback id and returns a token that is generated with Mux.JWT.sign:

// ./src/mux_signatures

import Mux from '@mux/mux-node';

export const signPlaybackId = function (playbackId) {
  return Mux.JWT.sign(playbackId, {
    keyId: process.env.MUX_SIGNING_KEY,
    keySecret: process.env.MUX_PRIVATE_KEY
  })
}

Our lambda function is going to use this module in Step 2.

Step 2: Create a `sign_playback_id` cloud function

Create a Netlify Function entry point. This is the single function that will handle one request. The idomatic pattern for creating cloud functions is to do one file and one javascript function per route. We will create a directory called functions/ and add a file called sign_playback_id.js.

// ./functions/sign_playback_ids.js

const keySecret = process.env.MUX_PRIVATE_KEY
import { signPlaybackId } from './src/mux_signatures';

exports.handler = async (event, context) => {
  try {
    const { queryStringParameters } = event;
    const { playbackId } = queryStringParameters;
    if (!playbackId) {
      return { statusCode: 400, body: JSON.stringify({errors: [{message: 'Missing playbackId in query string'}]}) };
    }
    const token = await signPlaybackId(playbackId);
    return  {
      statusCode: 302,
      headers: {
        'Access-Control-Allow-Origin': '*',
        location: `https://stream.mux.com/${playbackId}.m3u8?token=${token}`
      },
      body: '',
    }
  } catch (e) {
    console.error(e);
    return { statusCode: 500, body: JSON.stringify({ errors: [{message: 'Server Error'}] }) };
  }
}

Step 3: Add netlify.toml

Add a netlify.toml file to the root directory and tell Netlify where your functions will live. This tells Netlify that before we deploy we are going to build our functions into the ./.netlify/functions directory

    [build]
      functions = "./.netlify/functions"

Step 4: Connect to git

In order to use Netlify Functions you will now need to commit your code and push it up to a git repository like GitHub. Do that next and in Netlify’s dashboard connect your git repository to the Netlify project that you created. After connecting you git repository then

Step 5: Set your environment variables

In your Netlify project dashboard, naviate to "Settings" > "Deploys" > “Environment” to set your environment variables. Enter the MUX_SIGNING_KEY and MUX_PRIVATE_KEY from the Create a URL Signing Key step above.

Step 6: Test in development

Open one terminal and run netlify dev this will start a local Netlify dev server
Open another terminal window and run netlify-lambda serve ./functions this will build your functions/, get them ready to handle requests and watch the filesystem for changes.
In a third terminal window, curl your endpoint to test out the function (replace <netlify-port> and <playback-id> with your values.

curl -I 'http://localhost:<netlify-port>/.netlify/functions/sign_playback_id?playbackId=<playback-id>'

You should see a 302 (redirect) response with a location header for the signed url.

When you make any changes to your source files, netlify-lambda serve will pick up on the changes and recompile the functions into ./.netlify/functions.

Deploy

When you’re ready to deploy, you can deploy from the command line with netlify-lambda build ./functions && netlify deploy --prod. This will build the functions and then push up the changes to Netlify.

Try making a POST request to your cloud function on Netlify:

curl -I 'https://<your-netlify-app>.netlify.com/.netlify/functions/sign_playback_id?playbackId=<playback-id>'

Just like in dev, you should get back a 302 response with a location header that points to the signed playback URL.

https://stream.mux.com/${playbackId}.m3u8?token=${token}

This is what your response should look like:

HTTP/2 302
access-control-allow-origin: *
cache-control: no-cache
location: https://stream.mux.com/jqi1UtiO3gccQ019UcYjGJTLO9Ee00TLMY.m3u8?token=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IkFsVFZncktBVTYzVldIdVplcDEwMVhZUk5mbHozeDIxRiJ9.eyJleHAiOjE1NzE3NjE0NzMsImF1ZCI6InYiLCJzdWIiOiJqcWkxVXRpTzNnY2NRMDE5VWNZakdKVExPOUVlMDBUTE1ZIn0.i7oANZ6inmwmGVQjon4WEv_gKcqQ2v8GuQA8xuCBdT0Reegkm6WyTdU-VloZvAt7duaRR3-T8dt147vUQjM1n70CLi0996pwMejYWIbRHUMqrDBtsENHG8T9jtz-EJcBGONSzgs7fBQIVQx8xJvPuX4YqpylDK_lNX0-RDqfhz5THAfuyxzePJod709msD8kbHAqnIke5lHzbQNHuO2ecNFVCb2ZozW7XkIEctyLxrDAK1ITtQV8iHek3whwO9S05kM-5bQzomJEliN3mXBqCwMBmyIp8l88YKl59tVXDdU-l-cZvZjt1GYKv0J7shO-oBYcr00NmVKkP7bie_w50w
date: Tue, 15 Oct 2019 16:24:33 GMT
age: 0
server: Netlify
x-nf-request-id: 80484951-e7ff-46f3-b78e-1349b8514bec-1426623

Now, in your player, you can use your netlify function as the URL src. Here's an example for a web player (note that in order to get HLS in a <video> tag to work outside of Safari you will need to use another library like [Video.js](https://videojs.com" target="_blank" or HLS.js:

<video src="https://<your-netlify-project>.netlify.com/.netlify/functions/sign_playback_id?playbackId=<playback-id>"></video>

And here's an example on iOS with AVPlayer in Swift:

let url = URL(string: "https://<your-netlify-project>.netlify.com/.netlify/functions/sign_playback_id?playbackId=<playback-id>")

player = AVPlayer(url: url!)
player!.play()

The player will load the netlify URL, get the 302 redirect to the signed Mux URL and load the HLS manifest from stream.mux.com.

Restrict who can access your function

Now that your cloud function is working, you can add some security around it to make sure you only allow authorized users to generate signed urls.

For web players you will want to change this line in the sign_playback_id.js function:

'Access-Control-Allow-Origin': '*',

You can use the Access-Control-Allow-Origin header to control the CORS rules for the resource.

Phoenix LiveView: Build Twitch Without Writing JavaScript

Dylan Jhaveri — Thu, 31 Oct 2019 20:21:14 +0000

Phoenix LiveView is a new experiment that allows developers to build rich, real-time user experiences with server-rendered HTML. If you’re not familiar with Phoenix, it’s the fully-featured web framework for the Elixir programming language. At Mux we use Phoenix and Elixir to power our API. I decided to start playing around with LiveView to see what it’s capable of. The idea I had for an example app is Snitch, it’s like “Twitch,” but for snitches (put away your checkbooks potential investors). Under the hood, of course, we’re using Mux Live Streaming.

From the user’s perspective, first you create a “channel”. When that channel is created, Snitch will give you RTMP streaming credentials (just like Twitch does). As the user, you enter those streaming credentials into your mobile app or broadcast software and start streaming.

Right here we have the perfect test case for LiveView. In the UI we show the user the streaming credentials and are now waiting for them to start streaming. Mux is going to send webhooks to our server when relevant events happen. For example:

video.live_stream.connected
video.live_stream.recording
video.live_stream.active
video.live_stream.disconnected

In a typical web application, without LiveView, common solutions are to either use websockets to push new data to client applications or have those applications poll the server. About every second or so the browser would send a request to the server to get the updated data. But now with LiveView when a webhook hits the server we can re-render on the server-side and push those changes to the client.

Using LiveView to Handle Webhooks

The first step is to follow the instructions on the Installation page to add LiveView to your Phoenix application. This includes adding the dependency, exposing the WebSocket route and the phoenix_live_view JavaScript package for the client-side.

After following the installation instructions, let’s add a route for the Mux Webhooks:

    scope "/", SnitchWeb do
      pipe_through :api
      post "/webhooks/mux", WebhookController, :mux
    end

Then, in the Mux UI we can add this as our webhooks route. For local development I’m using ngrok to receive webhooks on my localhost server.

The webhook controller is going to receive the payload and update the “channel” in our database by calling Snitch.Channels.update_channel . Let’s look at the update_channel/2 function:

# lib/snitch/channels.ex 
def update_channel(%Channel{} = channel, attrs) do
  channel
  |> Channel.changeset(attrs)
  |> Repo.update()
  |> notify_subs()
end

notify_subs/1 is the new function we are going to call when a channel gets updated. This is where the LiveView magic happens.

# lib/snitch/channels.ex 
def notify_subs({:ok, channel}) do
  Phoenix.PubSub.broadcast(Snitch.PubSub, "channel-updated:#{channel.id}", channel)
  {:ok, channel}
end

This function is going to broadcast a message so that subscribers can react to this change. More on that shortly.

Now let’s update the controller and tell the controller to render with LiveView:

# lib/snitch_web/controllers/channel_controller.ex 
Phoenix.LiveView.Controller.live_render(conn, SnitchWeb.LiveChannelView,
  session: %{channel: channel}
)

And let’s create SnitchWeb.LiveChannelView. When we call notify_subs() up above, this LiveChannelView is the code that needs to subscribe and push an update to the client.

defmodule SnitchWeb.LiveChannelView do
  use Phoenix.LiveView

  #
  # When the controller calls live_render/3 this mount/2 function will get called
  # after the mount/2 function finishes then the render/1 function will get called
  # with the assigns
  #
  def mount(session, socket) do
    channel = session[:channel]
    if connected?(socket), do: SnitchWeb.Endpoint.subscribe("channel-updated:#{channel.id}")

    {
      :ok,
      set_assigns(channel, socket)
    }
  end

  def render(%{playback_url: nil} = assigns), 
    do: SnitchWeb.ChannelView.render("show.html", assigns)

  def render(assigns), do: SnitchWeb.ChannelView.render("show_active.html", assigns)

  #
  # Since the mount/2 function called "subscribe" to with the identifier
  # "channel-updated:#{channel.id}" then anytime data is broadcast this
  # handle_info/2 function will run and we have the power to set new values
  # with set_assigns/2
  #
  # After we assign new values, the render/1 function will get called with the
  # new assigns
  #
  def handle_info(channel, socket) do
    {
      :noreply,
      set_assigns(channel, socket)
    }
  end

  def set_assigns(channel, socket) do
    playback_url = Snitch.Channels.playback_url_for_channel(channel)

    socket
    |> assign(name: channel.name)
    |> assign(status: channel.mux_resource["status"])
    |> assign(connected: channel.mux_resource["connected"])
    |> assign(stream_key: channel.stream_key)
    |> assign(playback_url: playback_url)
  end
end

To summarize what’s happening above:

live_render/3 will invoke mount/2
mount/2 will subscribe using an identifier ("channel-updated:#{channel.id}") and set_assigns for the view
render/1 will get called with the assigns
anytime somewhere else in the app broadcasts to ("channel-updated:#{channel.id}"), this view is going to call handle_info/2 and that gives us the opportunity to use set_assigns again to update the assigns and re-render the template
re-renders auto-magically get pushed to the client over a websocket and the client updates the dom

The only difference in the show and show_active templates that we use in LiveChannelView is that instead of eex extension we use the leex extension which stands for live embedded elixir.

Here is a webapp where this is currently deployed at snitch.world The full code is up here on github. You can clone it and run it yourself. You’ll also need to sign up for a free account with Mux to get an API key. Feel free to reach out if you have any questions!

DEV Community: Dylan Jhaveri

Mux is the video API for the JAMstack

What is the JAMstack?

Static content frameworks

Deploy

Cloud Functions (“Serverless”)

Headless CMS

Authentication (advanced)

Database (advanced)

How did we get here?

Video for the JAMstack

JAMstack at Mux

Migrating to a headless CMS

How to Host Your Own Online Conference

Let’s break down the steps and API calls:

Live chat

Everything is recorded

Bonus: simulcast to the socials

Bonus: add a watermark

You don’t have to use Zoom

Final Setup

Extra details if you’re curious

Considered Harmful

Browsers will block your autoplay attempts

Autoplay will probably work for you, but it will break for your users

Here’s an example with vanilla javascript

Here’s an example with React

No BART terminals were hacked in the making of this ad

Why put code in an ad?

Insert wrench here

Silver lining

In defense of 'flicks' (or how I learned to stop worrying and love 705600000)

Using Netlify Functions to Create Signing Tokens

The Main Benefits of Cloud Functions

A Little Background About Signed Urls

Create a Mux Asset

Create a URL Signing Key

Setup a Netlify project

Step 1: Create a module to generate a signing token

Step 2: Create a sign_playback_id cloud function

Step 3: Add netlify.toml

Step 4: Connect to git

Step 5: Set your environment variables

Step 6: Test in development

Deploy

Restrict who can access your function

Phoenix LiveView: Build Twitch Without Writing JavaScript

Using LiveView to Handle Webhooks

Demo

Step 2: Create a `sign_playback_id` cloud function