DEV Community: Mux

Kafka Connect: The Magic Behind Mux Data Realtime Exports

Scott Kidder — Fri, 15 Jan 2021 23:36:38 +0000

At Mux we’ve seen an increasing demand for access to the raw & enriched video QoS data processed by Mux Data. Moreover, it’s not sufficient for these exports to be available on a daily or even hourly basis; many customers want access to low-latency, real-time data streams with a latency of one minute or less, glass-to-glass. We’ve made this possible for Mux Data Enterprise customers through our real-time metric and event-stream exports.

Mux Data customers are building their own impressive applications that take action on the realtime streams, such as:

Monitoring the performance of CDNs and networks to enable CDN switching & troubleshooting
Identifying popular videos as they become viral to increase promotion
Joining Mux Data metrics with their own internal metrics to speed up troubleshooting and root-cause analysis

The number of uses is limited only by one’s imagination!

However, this presents an obvious challenge for us in managing the numerous streams destined for customer streaming systems.

Mux uses Apache Kafka as our internal streaming platform. Mux customers might use AWS Kinesis, Google Cloud PubSub, Kafka, or some other streaming service. When faced with the problem of bridging data in our Kafka clusters to numerous external streaming services, we chose to “stand on the shoulders of giants” and use battle-tested open-source software to power the streaming export system as much as possible.

Enter, Kafka Connect

The Kafka Connect project is part of the Apache Kafka ecosystem. To quote Confluent:

“Kafka Connect is a tool for scalably and reliably streaming data between Apache Kafka and other data systems. It makes it simple to quickly define connectors that move large data sets into and out of Kafka.”

Anyone looking to stream data between Kafka and other data systems should first look to Kafka Connect.

Kafka Connect can run in either a standalone or distributed mode.

The standalone mode uses a maximum of one server, and the connector configuration is not fault-tolerant. This should only be used for testing or development.

Most production deployments of Kafka Connect run in a distributed mode. In this mode, a Kafka Connect connector (analogous to a job), can run across multiple Kafka Connect instances that make up a cluster, allowing for horizontal scalability & fault-tolerance.

Kafka Connect connector configurations are stored in an Apache Kafka topic, ensuring durability. Connector configurations are managed using the Kafka Connect REST API which can be accessed via any of the Kafka Connect instances in the cluster. A connector configuration describes the source (e.g. Kafka cluster & topic), sink (e.g. external AWS Kinesis stream), and any transformations to be applied along the way.

One of the best parts about Kafka Connect is that sources & sinks must only implement a single Kafka Connect Java Interface, making them extremely easy to write. This permits your source or sink to focus on reading or writing from the remote system. It's possible to write a source or sink for just about any system imaginable! All other concerns like offset management, consumer rebalancing, error-handling, and more are handled by Kafka Connect itself. Simply package your source/sink code in a JAR, include it on the Kafka Connect classpath, and you’re ready to reference it from a connector.

As an engineer working on streaming data systems, I’m certain I’ve written software equivalent to Kafka Connect several times during my career. It’s a tedious and error-prone process. Being able to delegate the majority of the work to a popular, well-supported open-source tool is a tremendous relief.

Mux uses Kafka Connect to manage the realtime event-stream exports to external streaming services. In the 9 months that we’ve been using Kafka Connect in production, it’s been extremely reliable, scalable, and easy to extend & customize.

Getting Started with Kafka Connect

Let’s walk through the process of deploying a Kafka Connect cluster on Kubernetes.

Step 1: Build your Kafka Connect Docker Image

A lot of open-source software is designed to be run as-is, without modifications. I was surprised to learn that this does not apply to Kafka Connect. Expect to add connectors to Kafka Connect.

If you’re deploying Kafka Connect as a Docker container, then this involves adding image layers with source or sink JARs (Java Archives of compiled code) that provide the extra or customized functionality you need.

In our case, we add sink connectors for AWS Kinesis and Google Cloud PubSub. We also add a Prometheus exporter JAR that scrapes the Kafka Connect JMX metrics and exposes them as Prometheus metrics.

Here's a sample Dockerfile that pulls adds these dependencies to the Confluent Kafka Connect base-image:

We compile the JARs, place them in the same directory as the Dockerfile shown above, and run docker build. The resulting Docker image is pushed to a Docker image repository where it can be pulled and run.

Step 2: Start up your Kafka Connect Cluster

Nearly all of the infrastructure at Mux runs in Kubernetes clusters. Kafka Connect is no exception. The following Kubernetes manifest shows how you might run the Kafka Connect Docker image in a Kubernetes cluster:

At a minimum, you’ll need to change the Docker image URI to point at your new image, and change the imagePullSecrets name if you’re storing the image in a private repository.

Credentials for AWS and GCP are stored as Kubernetes secrets. The AWS credentials are set as environment variables on the container, which are automatically recognized and used by the AWS Kinesis sink for authentication. The GCP credentials are stored in a Kubernetes secret that is accessible from a file on the container filesystem; the GOOGLE_APPLICATION_CREDENTIALS environment variable is used by the GCP PubSub sink to determine the location of the file containing application credentials.

After deploying the Kafka Connect manifest to Kubernetes, you should see your Kafka Connect pod(s) running as part of a cluster. They’re not doing much at this point, but that’s fine!

Step 3: Run some Connectors!

As mentioned earlier, connectors are administered through the Kafka Connect REST API. You can access this API via any of the Kafka Connect pods. This can be accomplished by sending REST calls to TCP port 8083 on a Kafka Connect pod.

Example: GCP PubSub Sink

In this example, we’re reading from the Kafka topic demo-event-stream-export and writing to the GCP PubSub topic mux-demo-event-stream-export in the project external-mux using the GCP PubSub sink implementation. The connector will run 6 Kafka Connector tasks; each task works as a Kafka consumer, so it makes sense for the number of tasks to not exceed the number of partitions on the source topic (e.g. demo-event-stream-export). The connector sink is identified by the connector.class setting, which references the GCP sink implementation.

Example: AWS Kinesis Sink

This example shows a connector that writes to a Kinesis sink using the AWS Kinesis sink implementation. Again, it’s reading from the Kafka topic demo-event-stream-export. There are Kafka Connect standard configuration keys (e.g. name, topics, connector.class, tasks.max) intermingled with the sink configuration keys (e.g. streamName, roleARN, ttl).

This example shows how you can use IAM authentication and external-id’s to authenticate to a restricted role when accessing external resources, such as a Kinesis stream owned by another organization. It also shows how you can use the kinesisEndpoint key to use an alternate endpoint, which is a popular way of using a VPC endpoint to avoid having your Kinesis-writes egress through a NAT, which can become quite costly.

Lessons We’ve Learned

Make sure ‘CONNECT_REST_ADVERTISED_HOST_NAME’ is set

The Kafka Connect pods communicate with each other via the REST API. This requires them to advertise themselves using a hostname or address that’s accessible by all of the other Kafka Connect pods.

In Kubernetes deployments you should probably set the ‘CONNECT_REST_ADVERTISED_HOST_NAME’ environment variable to the pod IP. This is done in the Kubernetes deployment example above.

Failure to set this correctly will yield errors any time you try to modify the connector configurations. Other Kafka Connect community blog posts have addressed this specific issue, it’s a painful one.

Set the ‘errors.retry.timeout’ value on your sinks

Kafka Connect sinks can be configured to retry failed writes for a configurable duration with exponential backoff. However, the default behavior in Kafka Connect is to not retry writes on a worker task and immediately mark the worker task as failed. Once a task is failed, its source topic partition claims are reassigned to other tasks where they’ll be retried. This can lead to an imbalance of load across Kafka Connect pods. The only way to resolve this is by restarting the individual failed tasks.

Here's an example of an imbalance of network & CPU load across Kafka Connect instances due to task failures. Nothing is wrong, per se, but the uneven load leads to inefficiency that could later result in lag:

Ideally, the load should be even across all Kafka Connect instances:

For this reason, it’s highly recommended that you set the ‘errors.retry.timeout’ setting on each sink. In the case of AWS Kinesis sinks, it’s common to get spurious write-failures that immediately succeed upon retry. Setting the ‘errors.retry.timeout’ ensures that you’ll get the benefit of a retry while keeping the workload balanced. You should also monitor task failures as an early indicator of load imbalances.

Autoscaling is tricky

We try to autoscale all of the services we run at Mux, adding or removing servers based on some observed set of metrics. For instance, when CPU or memory utilization rises above some predetermined threshold, the cluster should add enough servers to bring the load back down below the threshold.

In our experience, Kafka Connect resource utilization has typically been constrained by CPU usage. Adding a server will cause a cluster to rebalance which, on its own, will temporarily drive up CPU utilization.

If the autoscaling service (e.g. Kubernetes Horizontal Pod Autoscaler, or HPA) is not configured to support a cooldown or delay, then the addition of servers could cause the HPA to think that even more servers are needed to deal with the temporary spike in CPU utilization. This can lead to massive overprovisioning and thrashing in response to a moderate increase in load. In our experience, this is a general problem for Kafka consumers that are subject to stop-the-world Kafka consumer rebalances.

One approach to this problem is to configure the autoscaler to use a stabilization-window during which no scaling decisions are made. Support for stabilization windows is present in the HPA in Kubernetes 1.18 or greater.

Monitoring

We use Grafana to visualize Prometheus metrics for all services at Mux. The JMX Exporter plugin that we bundle with Kafka Connect is responsible for exposing Kafka Connect JMX metrics for scraping with Prometheus. We’ve created a Grafana Dashboard to present significant metrics in a single-page dashboard.

Some of the useful metrics are:

CPU utilization
Memory utilization
Topic consumer lag
Records consumed by connector
Sink batch average put-time (e.g. time to write)
Task errors
Task failures

We have alerts on the following metrics:

CPU utilization
Memory utilization
Kafka consumer fetch-record lag
Task failures

Conclusion

Kafka Connect has been an extremely useful tool for distributing Mux Data streams to customer stream-processing systems. The simplicity of deployment and the broad array of community-supported open-source integrations is unmatched!

If you’re a Mux Data Enterprise customer, we encourage you to contact us to set up a data stream that can be used to react to viewer activity and trends in realtime!

A Beginner's Guide to Video File Formats: MP4s

Bonnie Pecevich — Tue, 12 Jan 2021 19:50:47 +0000

Note: Originally posted by my colleague on the Mux blog.

One of my first projects as an engineer at Mux was to add a bit of validation logic to stream.new to block uploads of videos with extremely long durations (1+ hours). stream.new is a simple web application the team built to both serve as an example application for users and to help us dogfood our own product. The site allows users to upload an input video file to Mux, where it is processed and encoded, then made easily streamable and shareable (you can also record directly from your camera or screen -- check it out!) Adding a client-side duration check is a commonly requested feature: imagine building an application where users upload videos up to a minute long (think TikTok). It would be a huge bummer to spend a bunch of time uploading a file just to be told at the end that your video was too long!

As an uninitiated beginner in the world of video, I did some preliminary research into the canonical method of determining a video file’s duration. After skimming through some disappointingly sparse search results, I posed the “simple” question in the team Slack channel. I then proceeded to do my best to follow a 45 minute minute conversation between six of my coworkers (with dozens of years of video experience between them) demonstrating just how non-trivial the problem actually is.

As it turns out, video file formats are a bit more complex than I had previously thought. Fun! Accordingly, in the rest of this post I’ll be doing my best to provide a beginner’s introduction into how video file formats work -- no prior video experience required.

How the sausage gets made

Video files are generally serialized into files with a container format: multiple streams of data are encoded and multiplexed (or “muxed”, if you will) together in separate containers within a single file. These files hold multiple streams of data because the visual, audio, captions, etc. components are all stored separately and tied together by a shared playback timeline. Let’s discuss one of the most common and well-known file formats, MP4.

Each container is called a box (or atom) and is a serialized array of bytes formatted with a prefix that specifies (1) the box type with a canonical four character label and (2) the serialized box’s length so a parser can know how far into the box to read. Boxes are hierarchical, meaning there can be multiple boxes within a box.

So mp4s are just a collection of boxes within boxes… what do they all mean?

By convention, the first box in an mp4 file is an ftyp box. This contains some high level metadata about the file, including the types of decoders that can be used on the rest of the file. “Decoders” are code that transforms serialized data into a signal that humans can understand, while codecs do the exact opposite.

After the ftyp box usually comes a moov box. This is where, in my opinion, things start getting more interesting. The moov box generally contains a few trak boxes within it which provide reference information for interpreting the encoded data streams. For example, in a normal MP4 there might be a trak for video (visual) and a trak for audio. In addition to describing more details about the appropriate decoders for each stream, the trak includes offset information into the rest of the file (basically serving like pointers to a video player) about where the encoded streams can be found. The actual encoded bitstreams are, in turn, contained within the following mdat (Media Data) box! So to play back a video, the player would need to first load the moov box to find the relevant offsets into the mdat box for the audio and visual streams to start decoding them for a device’s physical output.

Streaming Video

Things get fun when we start thinking about fragmented mp4s (fMP4s), which are used for streaming video. After all, if you’re serving a large video file you wouldn’t want to make viewers download the whole thing before they can start watching. Or, let’s say you want to seek to a specific location in the video (say, 11:42). If the video and audio streams were stored in two large, contiguous ranges of bytes then you would need to iterate through the whole thing to find your desired seek location. Instead, in a fMP4 file the mdat box is segmented into pieces (conceptually, think of the video being chopped up every few seconds), and a sidx box provides indexing information so a player knows which timestamps in the video correspond to which segmented mdat boxes.

Segmentation of the mdat box also unlocks the benefit of adaptive bitrates. Since all client bandwidths and user devices are not created equal, you might want to seamlessly deliver versions of the video with a higher or lower bitrate (think 240p, 480p, 1080p…) in different situations. With segmented video files, your player can detect bandwidth changes or slowdowns and switch bitrate accordingly. The original content is encoded into multiple versions with different bitrates when it is ingested, and the player can decide which one to iteratively download during playback. For example, if your user is watching something on a mobile device and they walk out of range of wifi, then you may want to switch from 1080p to 480p rather than subject the user to a poor viewing experience when their device begins to struggle.

Returning to the original problem at hand… so how do we find the duration of a video file? The short answer is that (at least for MP4s) you should be able to just inspect the header values stored in the file’s moov box. However, there is no guarantee that the header metadata is even accurate! Check out my co-worker Phil Cluff’s (aka “other Phil”) talk on this topic at Demuxed -- it turns out you can pretty easily munge/delete/edit boxes within the file even while [certain players] continue to play them normally! Therefore, to derive an authoritative answer for what the duration of a video is, one would actually need to parse and iterate through the entire contents of the file. (And that’s not even taking alternative formats into account!) That being said, in general, the header information should serve as a reasonable signal of the video’s length. For our use case of a simple preliminary check, we made do in stream.new with loading the file as a video element and checking its metadata -- a simple change of just a few lines of code.

Hopefully this was a helpful introduction to the world of video formats! If you’re interested in learning more about how playback works, check out howvideo.works. If you’re itching to just get started with streaming video, go to mux.com. Thanks for reading!

The state of going live from a browser

Matt McClure — Mon, 20 Apr 2020 16:30:35 +0000

Publishing a live stream directly from a browser feels like it must be one of those solved problems. Watching live video in a browser is so common these days it's hard to imagine a time when it required proprietary plugins to even have a chance of working. Even video communication feels trivial now thanks to modern browser features like WebRTC. The "trivial" part is only really true if you're using two browser windows on the same machine, but still, it's you on video! Twice!

So as a web developer looking at all this video successfully being sent and played back by the browser, it's totally reasonable to think that publishing a live broadcast directly from a browser would be easy. All the building blocks are here, there's surely an npm package that ties it all together for publishing to sources like Mux, Facebook, YouTube Live, Twitch, etc...

That's gonna be a no from browsers, dawg.

Unfortunately that's simply not the case. There's no reasonable way to publish a live broadcast directly from a browser. It's possible to capture the video and eventually get it there, but you're almost always going to need to get a server involved.

One of the big reasons for this is that the industry standard for publishing live streams is RTMP, which is a protocol browsers simply aren't able to natively speak. We've written about the options out there for native mobile applications, and the desktop has fantastic, open tools like the OBS project.

Why go live from the browser?

One of the most common reasons is simply due to friction. If you're building a live streaming solution and you want your customers to be able to go live as easily as possible, asking them to leave your service to go figure out some other piece of desktop software is a big ask.

On top of that, the tools out there for live streaming are complex in their own right. OBS Studio, for example, is an incredibly powerful and flexible tool, but that comes with the cost of being a daunting piece of software for the unfamiliar. Even with guides and tools out there to help users get set up, you're now supporting not only your service, but whatever tools your streamers end up using.

If you're already building a web app there's a good chance your team is good at...well building web apps. Building your go-live dashboard directly into your browser application would allow you to continue to utilize the expertise of your team, giving end-users a low-friction, branded experience that doesn't require them to learn anything but your application.

Before we go on...

Yes, for all of the reasons just mentioned, it's easy to see why it's so tempting, but going live directly from the browser is almost certainly going to be a worse experience for everyone involved. The quality will be worse, the stream less reliable, and the tooling more limited. Your streamers and your viewers are all probably better off if the broadcast is done from a native application.

Ok cool, now let's talk about our options.

We're going to talk about 3 high-level approaches to going live from the browser. By "going live," what we're specifically referring to is getting video from a streamer's browser to a broadcast endpoint via RTMP. Spoiler alert: all three of the approaches we're going to discuss are related, and two of them are essentially the same workflow with a twist. There are probably other options out there, but these are the closest to production ready you'll find.

WebRTC rebroadcasting

Most commonly, WebRTC is known as the technology that lets web developers to build live video chat into the browser. That's true, but it actually goes much further than that. WebRTC is made up of standards that allow for peer-to-peer Web applications that can transmit audio, video, or even just arbitrary data without the need for plug-ins or technically even servers[1].

A quick aside, a fellow Muxologist, Nick Chadwick, gave a talk on WebRTC → RTMP at AllThingsRTC in 2019. He goes much deeper into the underlying protocols in that talk than we are here, so if you're interested in the nitty gritty details, that one's highly recommended.

Given the well-documented path to video teleconferencing that WebRTC provides, the most common solution that people immediately gravitate towards is what's called "rebroadcasting." A server implements the WebRTC API to become a peer, then takes the video feed and publishes it via RTMP.

This approach is, to put it simply, difficult. The good news is, that path has gotten a little easier in recent months, with projects like Pion maturing and higher level tools like node-webrtc adding support for accessing actual video frames.

Broadcasting headless Chrome

Nick also mentions this approach in his talk (and built an example), but another approach is to simply bypass server-side implementations altogether and use the one that's arguably the most battle-tested and has a wide selection of open-source tooling: Chrome. Yes, that one, the browser.

Thanks to projects like Puppeteer, the process of programmatically interacting with a headless Chrome instance is pretty straightforward. From there you can build a normal WebRTC experience and use ffmpeg to broadcast whatever's in your headless Chrome instance via RTMP.

The huge benefit of this approach is that it allows the developer to effectively build any experience in the user interface. Stream overlays, multiple speakers on a call, video effects, whatever you could build with canvas or the DOM would Just Work™ since it's...well, it's a browser. It's also not that much additional work on top of building out normal, peer-to-peer chat for that reason.

The downside of this approach is that you need to have a Chrome instance for every streamer. If you're just looking to stream yourself this isn't a huge issue, but if you're looking to support an arbitrary number of streamers this could become problematic.

Video over WebSockets

This one is the simplest and, in my opinion, the most fun to hack around on. Yes, as promised, this solution also uses at least one piece of the WebRTC toolchain, getUserMedia() (the way you request access to the browser's mic and camera). However, once you have the media, instead of delivering the media via WebRTC's protocols, you use the MediaRecorder API.

This allows for similar flexibility to the headless Chrome example: you can render the user's camera to a canvas element and manipulate the video however you'd like there. The MediaRecorder will fire an event every time it has a "chunk" of video data ready, at which point you send it to the server via the websocket as a binary blob. The server then listens for these data chunks and pipes them into a running ffmpeg command as they're received.

The benefit to this approach is that it's much closer to "traditional" applications in terms of running and scaling. You need a persistent WebSocket connection with each streamer, yes, but the requirements of each stream are actually pretty low since we've got ffmpeg doing as little as possible before publishing the RTMP stream. In fact, this example application using Next.js runs just fine on a Glitch server. Let's talk about how it works.

The Client

For the example we used a React framework called Next.js with a custom Node.js server.

Before the client can do anything, it needs to request access to the user's camera and microphone by calling getUserMedia with the requested constraints. Calling this function will prompt the browser to ask the end-user if they'd like to share the requested resources.

// This would just ask for access to audio and video, but you can also specify
// what resolution you want from the video if you'd like.
const cameraStream = await navigator.mediaDevices.getUserMedia({
  audio: true,
  video: true
});

The call to getUserMedia returns a promise. which (if the user agrees) will resolve and return the camera stream. That camera stream can then be set as the srcObject of a video tag, at which point you've got the webcam playing back in the browser window!

From here, what we're doing in the demo is rendering that video stream to a canvas element using a very similar technique to what we described in our blog post on manipulating video via the canvas element. Once we're copying the video over to the canvas element, we can capture that stream, and initialize a new MediaRecorder instance.

const mediaStream = canvasEl.captureStream(30); // 30 frames per second
const mediaRecorder = new MediaRecorder(mediaStream, {
  mimeType: 'video/webm',
  videoBitsPerSecond: 3000000
});

The new MediaRecorder object will fire an event every time a blob is ready (ondataavailable). We can listen for that event, and when we receive it send the data blob right down an open WebSocket connection.

// Listen for the dataavailable event on our mediaRecorder instance

mediaRecorder.addEventListener('dataavailable', e => {

  ws.send(e.data); // Then send the binary data down the website!

});

The Server

The server listens for incoming WebSocket connections, and when a new one is created it initializes a new ffmpeg process that's streaming to the specified RTMP endpoint. Whenever a new chunk of video comes in via a message, the server pipes that received data to the ffmpeg process, which in turn broadcasts it via RTMP.

webSocketServer.on('connection', (ws) => {;

  // When a new connection comes in, spawn a new ffmpeg process

  const ffmpeg = child_process.spawn('ffmpeg', [

    // ... ffmpeg settings ...
// final argument should be the output, 
// which in this case is our RTMP endpoint
`rtmps://global-live.mux.com/app/${STREAM_KEY}`,

]);

// If our ffmpeg process goes away, end the WebSocket connection

  ffmpeg.on('close', (code, signal) => {

    ws.terminate();

  });

ws.on('message', (msg) => {

    // If we're using this WebSocket for other messages, check 

    // and make sure before piping it to our ffmpeg process

    if (Buffer.isBuffer(msg)) {

      ffmpeg.stdin.write(msg);

    }

  });

// If the WebSocket connection goes away, clean up the ffmpeg process

  ws.on('close', (e) => {

    ffmpeg.kill('SIGINT');

  });

});

Profit! Kinda.

It works! It's fun and fairly simple, with both code and client coming in at < 300 lines of code. It's got the advantage of being easy to interact with the outgoing stream, and it's quick and easy to hack on. You can give it a try now, just go remix the Glitch, specify your own Mux stream key, and try it out.

However, there are huge drawbacks to the Javascript side of things. For example, modern browsers will de-prioritize the timers on a tab that isn't front-and-center, meaning if the streamer switches to a different tab, the streaming page won't send chunks of video fast enough and eventually the stream will stall. There are ways to ensure that doesn't happen, but most of them will require at least some participation from your streamer.

Let us help your users go live!

Unless you have a lot of resources to devote for building out an application around going live from the browser we suggest providing your users other tried and true native options or pointing them towards one of the fantastic paid browser options. That being said, we're here to help! If you want help figuring out the best way to let users go live in your application, please reach out.

[1]: Yes, in practice most applications would want a server for connection negotiation and more, but technically a simple application could allow users to share the required details via another method.

Mux is the video API for the JAMstack

Dylan Jhaveri — Wed, 08 Apr 2020 16:28:01 +0000

What is the JAMstack?

The JAMstack is a term popularized in the last year, largely by the React community and companies like Netlify and Zeit. Specifically, JAMstack stands for "Javascript", "APIs" and "Markup". These terms don't exactly describe what the JAMstack is in a clear way, but the name itself has a nice ring to it so it seems to have stuck.

Here is a breakdown of all the pieces for a "JAMstack" application and what some of the popular options are. For a more exhaustive list you might check out awesome-jamstack on Github.

Static content frameworks

This covers the "Javascript" and "Markup" part of the stack.

Next.js: Open source, write everything with React and the framework gives you automatic code splitting and a server-side rendered web application.
Gatsby: Also open source and you write everything with React components. The Gatsby framework handles code splitting and lazy loading resources. Gatsby also has a concept of “sources” where you can write GraphQL queries to pull in data from 3rd party sources via a plugin.
11ty: A static site generator that works with all kinds of templates: markdown, liquid templates, nunjucks, handlebars, mustache, ejs, haml, pug and Javascript template literals

Deploy

These are platforms that can host your statically built application. With common JAMstack frameworks you end up with static files that can be hosted by a static file server and delivered over a CDN.

Cloud Functions (“Serverless”)

All of these services, in one way or another, allow you to write code in javascript that handles an API request and returns a response. This, along with other 3rd party APIs is the "API" part of the stack. The serverless part is that you don’t have to worry about the details on how or where that code gets run. These platforms will handle the server configuration and the deployment of your API endpoints as “cloud functions” or “lambdas”. In your client side application, you can make requests to these functions the same way you would make requests to API endpoints that you would have deployed to your own traditional server.

Headless CMS

A “headless” CMS is a CMS that gives you and your team an interface to log in, edit content, add new content, upload assets and the “publish” data that makes it into your website or application.

There are many headless CMSes. We are a little biased, so these are the one ones that work with Mux and these are the ones that we have used. Look around for what works for you. And if you have one that you want to use with Mux, let us know and we can build an integration.

Authentication (advanced)

If you’re building a static marketing site you probably will not need to deal with authentication. However, for a more advanced application you will need to have users login, reset passwords and do all the pieces of authentication.

Database (advanced)

If you are authenticating users and dealing with logged in sessions, you probably need a database. These are commonly used for JAMstack applications.

How did we get here?

Before these tools gained popularity the answer to “What stack should I use for my marketing site?” might have been “use Rails” and that is a clear answer. But now if someone says “use the JAMstack” well, that is a complicated answer. It’s a little misleading to call the “JAMstack” a specific stack, because as you can see from above, even if you decided to use the JAMstack, you still have a lot of choices to make.

Before the JAMstack was popularized, we have had a long history of static site generators. You may remember Jekyl or Middleman from the Ruby community. These tools allowed you to write Markdown, Liquid or Ruby’s ERB templates and generate a static site that you could host somewhere like s3 to host your blog. These tools are great and they are still widely used.

These static site generators were great for developers that wanted to make something like a blog or a simple marketing website. Someone non-technical might reach for a tool like Wordpress or Squarespace, whereas a hacker would turn to a static site generator.

For more advanced applications that went beyond statically rendered HTML, we had to switch gears away from static site generators and into a web framework like Rails.

Then advanced frontend frameworks for building interactive single page applications became popular: Angular, Ember and React. Suddenly, frontend developers had all these tools and got comfortable writing React code for their applications. But for static marketing sites we couldn’t write React or Angular code because we still needed static HTML for SEO purposes and fast initial load times. Developers were stuck in a world where we wrote what we were comfortable with for our application frontend but then for our marketing site had to switch back to some ad-hoc cobbled together jQuery functions.

The biggest feature that made the JAMstack popular is that you get the best of both worlds: server-side rendered HTML plus interactive React components that you can do whatever you want with. This is the big innovation and the first “oh wow” moment I had using both Next.js and Gatsby. You write normal React like you’re used to, run the build process and then all of a sudden you end up with static HTML returned by the server and all your interactive React code works as you would expect.

Video for the JAMstack

Mux is the video API for the JAMstack. The philosophy behind Mux and how we approach video fits in neatly with the JAMstack philosophy. Mux will act as your video infrastructure by handling the storage, hosting and delivery of your video without getting in the way or being opinionated about the presentation.

In fact, Mux does not even give you a video player. You have to bring your own player to the party. The entire “frontend” of the video experience is up to you, Mux is focused on handling the backend or the “serverless” part of your video stack. Think of Mux as the headless video platform. You control every bit of the user experience while Mux does the heavy lifting behind the scenes.

JAMstack at Mux

In addition to providing APIs that you can use for your JAMstack website, Mux also uses the JAMstack ourselves to power our marketing site (mux.com) and the Mux blog.

A couple of months ago we finished the process of moving the Mux Blog to the JAMstack. Before this project, the Mux blog was hosted and deployed separately from mux.com. The blog was powered by an old version of Ghost, using the default Casper theme. Our marketing site is a Gatsby site that uses gatsby-source-filesystem to create some pages from markdown and gatsby-source-airtable to pull in some data from Airtable.

The main issue with our existing blog that we wanted to address was that since we were using a Ghost theme, not only was the design of the blog completely different from the design of the rest of our marketing website, but it was an entirely different application with a different structure, hosting and deploy process.

As a result, visitors that landed on a blog post didn’t have an easy way to get back to the main marketing site and since the look and feel didn’t exactly line up, the experience was too disconnected. We decided that we wanted to move everything to a headless CMS so that we could make the blog part of our existing Gatsby marketing site for consistency.

Migrating to a headless CMS

There are pre-built Mux integrations for Sanity, Contentful, and Cosmic. All of these options allow you to bring your own Mux account. Alternatively, Dato is a headless CMS that offers native video built into the product that is powered by Mux.

We ended up choosing Sanity as our headless CMS. We loved that Sanity felt like an open-ended developer product that could grow with our needs past just the blog today. Calling Sanity a headless CMS sells it short from what it really is: it’s more akin to a structured, real-time database. The CMS part is all open source and in your control for how you want things to look and work. The way to think about it is that Sanity provides a real-time database along with some low-level primitives to define your data model, then from there, you build your own CMS.

As a part of this project of moving the blog to a new CMS, we wanted to set ourselves up with a headless CMS that could be used beyond just the blog and could also create a variety of pages on mux.com and allow us to move existing content like the video glossary.

For a more technical in-depth read about how we did this, check out this Sanity Guide we wrote How to migrate your HTML blog-content from Ghost and the blog post Moving the Mux blog to the JAMstack.

How to Host Your Own Online Conference

Dylan Jhaveri — Tue, 03 Mar 2020 00:01:30 +0000

Online conferences seem to have gained in popularity the past year. We’re seeing more and more folks reach out with questions around best practices and how to pull off their own remote conference. Sometimes, people will announce a conference, get thousands of sign ups and then one or two weeks before it’s set to go live they will be figuring out how to do it; how much different from a normal video conference call could it be? For our purposes let’s say you want to build a custom experience and host an online conference on your own website.

Use this guide as your playbook. This is what we will cover:

You have multiple presenters that will be broadcasting live from different locations.
Your team has no video expertise at all, but you do have a technical team that is capable of building a functioning web application.
The experience you want to provide to your audience is something custom that you control. Your brand is important for you and you want to control the look, feel and user experience of the conference.
To broaden the reach of your conference you simultaneously want to broadcast out the video feed to social channels like Youtube Live, Facebook Live and Periscope.
If possible, you would really like to have a branded overlay with your logo on the video.
In addition to streaming live, you want to record the broadcast so that people who did not attend live are able to view the recordings on-demand as soon as each session is over.

The basic structure of your setup is going to be a live conversation that is broadcast to a larger group of live viewers. The live conversation could be something like one person presenting with a screen share, or one person interviewing someone else, or a panel discussion among a group of experts.

A really simple way to do this live conversation is to use Zoom. Most people are familiar with Zoom. It is one of the most stable, reliable and high quality pieces of meeting software around and it runs natively on your desktop. What’s really cool about Zoom is that if you enable live streaming for meetings then you can set up an RTMP output for your Zoom call to any arbitrary RTMP endpoint (this is where Mux comes in).

Adding Mux in the middle is how you can broadcast your Zoom call to an audience of thousands on your own website. The live audience does not have to download Zoom, they do not interact with Zoom at all. All they do is see a video player that you make on your website.

Let’s break down the steps and API calls:

1) Make sure in Zoom settings you have allowed meetings to be live streamed.

2) Create a Mux live stream - this is one API call and for every live stream you create you will get back a unique stream key also make sure you save a playback_id for this live stream - you will need this to play the live stream.

3) Set up a Zoom call like you normally would. From the call, click the 3 dots at the bottom where it says "More" and click "Live on Custom Live Streaming Service".

4) From here, enter the RTMP server details for Mux.

5) When you’re ready to stream click “Go Live!” from Zoom. Now your Zoom call will be live and Mux will start receiving the video and audio. To confirm that this part is working, navigate to the live stream in your Mux dashboard and you should see video and audio coming in. Later you can set up Webhooks so that you can be notified when every live stream is connected, active, completed, etc.

6) Next is the player side, you have the playback_id from step 3, right? You will need to take that ID and form a URL like this: https://stream.mux.com/{playback-id}.m3u8. This is an “m3u8” URL which is a URL for streaming video over HLS. HLS is a standard streaming format for both live and on-demand video. You will need to use this HLS URL in a video player. Which player you choose is entirely up to you. Here’s two free ones you can check out to get started: videojs and plyr. With whichever player you choose, follow the instructions for streaming a HLS video.

Note that HLS streaming will come with some latency. Expect for 15-20 seconds, there is a reduced_latency flag that you can use which will bring that number down closer to 8 or 10 seconds (with some tradeoffs). Check the docs here for more information.

Live chat

After you get the live stream of your conference working on your webpage, you will almost certainly want to add a live chat component. Live chat is outside the scope of what Mux offers, but we’ve seen this done in a few different ways:

Use your own database and push real time changes to your clients with something like Pusher, PubNub or Socket.IO.
Use a realtime database like Firebase and build your own chat experience.
Try something like Stream which offers real time APIs specifically around creating chat experience. Fully featured with things like uploading images to chat, emoji reactions, typing notifications and all the bells and whistles are built in.
Skip the step of adding chat on your webpage and create a Slack community where everyone can chat. The benefit of having a slack community is that it’s free and you don’t have to go through the steps of building chat onto your website. Most people are already familiar with Slack, so they likely have it downloaded already. You can also create channels for specific topics and allow attendees to DM each other. This has the added benefit of allowing for the kind of attendee networking that happens at in-person conferences.

Everything is recorded

Every live stream that you do will be recorded by Mux and the asset will be available for on-demand playback. After each live stream is over, you can use the Mux asset to give attendees who were not present live the ability to view the recording.

Bonus: simulcast to the socials

When you create a Mux live stream you can optionally add simulcast_targets which are arbitrary RTMP endpoints that Mux will push out your stream to. It’s fairly straightforward and we have some guides on how to do this. Read more in the announcement blog post and the guide. All you really have to do is track down the RTMP server URL and stream keys for each of the social networks you want to broadcast to and add them to the Mux live stream with an API call.

Bonus: add a watermark

In the new_asset_settings parameter when you create the live stream, you have the option to specify a watermark. You give Mux the URL to an image you want to use as a watermark and some details about where to place it and how to align it. When this is configured, Mux will add it to the stream that comes from Zoom and the watermark will appear on the HLS stream that you show on your website.

You don’t have to use Zoom

Zoom is the simple example I used here because many people are familiar with it and know how it works. But the reality is you can use any software that allows you to RTMP out to Mux. To name a few other options: OBS, Wirecast, ecamm live, all of these products are built to compose a single video stream and RTMP out to somewhere. All of these will work with Mux.

Final Setup

Here’s what your final setup might look like. If you are going to host an online conference with Mux, please reach out! We would love to talk to you and help you make sure it’s successful.

Extra details if you’re curious

Mux supports live streams for up to 12 hours. If you have one single stream of an all day conference then this should be enough.
There are two options for playing the live stream on your website, you can either use a playback_id associated directly with the live stream, OR you can use the playback_id associated with the active_asset that is associated with the live stream. The former will not allow seeking backwards in the stream, the latter will allow your attendees (if they want) to seek all the way back to the beginning. This is a subtle detail but it might be something you want to consider.

If you’re doing an online conference, please get in touch!

Dylan Jhaveri — Thu, 23 Jan 2020 17:19:28 +0000

If you’re trying to autoplay videos on the web, you might be tempted to reach for the HTML5 autoplay attribute. This sounds exactly like what you’re looking for, right? Well, not exactly. Let’s talk about why that’s probably not what you’re looking for and what the better option is.

Browsers will block your autoplay attempts

Over the last few years, all major browser vendors have taken steps to aggressively block autoplaying videos on webpages. Safari announced some policy changes in June 2017 and Chrome followed suit shortly after and Firefox after that.

In summary: all these browsers will aggressively block videos from autoplaying on webpages. Each browser has slightly different rules around how it makes this decision. It’s a huge black box and browsers will not tell you what their exact rules are. The default behavior is block most autoplay attempts.

There are however some conditions that make it more likely for autoplay to work:

Your video is muted with the muted attribute.
The user has interacted with the page with a click or a tap.
(Chrome - desktop) The user’s Media Engagement Index threshold has been crossed. Chrome keeps track of how often a user consumes media on a site and if a user has played a lot of media on this site then Chrome will probably allow autoplay.
(Chrome - mobile) The user has added the site to their home screen.
(Safari) Device is not in power-saving mode.

These conditions only make autoplay more likely, but remember that aside from these conditions, the user can override the browser’s default setting on a per-domain basis. This basically means that you can never rely on autoplay actually working.

Autoplay will probably work for you, but it will break for your users

Even if you try to follow the rules above, autoplay is still a finicky beast. One thing to keep in mind (for Chrome at least) is that due to Chrome’s Media Engagement Index. When you are testing autoplay on your own site it will probably work for you (because you visit your site often and play content, your MEI score is high). But then when new users come to your site, it is likely to fail (because their MEI score is low). As a developer, this is incredibly frustrating and another reason to always avoid the autoplay attribute.

What should I do instead?
I’m not suggesting that you avoid autoplaying videos, but I am suggesting that you always avoid the autoplay attribute. There is a better way.

Use video.play() in javascript world, which returns a promise. If the promise resolves, then autoplay worked, if the promise rejects then autoplay was blocked.
If the promise returned from video.play() rejects, then show a play button in the UI so that the user can click to play (the default video controls attribute will work just fine). If you are using your own custom controls and your javascript calls video.play() again as the result of an event that bubbled up from a user click, then it will work.
Consider starting with the video muted, this gives you a much lower chance of your video.play() call rejecting. You will want to show some kind of “muted” icon in the UI that the user can click to unmute (again, the default video controls attribute works great for that). You may notice that twitter and a lot of sites start videos in the muted state.
Have I mentioned showing controls for your player? Always make sure controls for your player are accessible. We have seen sites try to get fancy and be too minimalist by hiding controls. Inevitably, they run into situations where autoplay fails and the user has no way of clicking to make the video play. Make sure you do not fall into this trap.

Here’s an example with vanilla javascript

// get a reference to a <video> element
const videoEl = document.querySelector('video');

// attempt to call play() and catch if it fails
videoEl.play().then(() => {
  console.log('Autoplay success!');
}).catch((error) => {
  console.warning('Autoplay error', error);
});

Here’s an example with React

If you look at mux.com you’ll see that we autoplay a video on the top of the home page. I copied over how we did that and set up a demo here: https://o9s4w.csb.app/. The code is copied below and you can fork it and play around with the Code Sandbox.

Notice that we’re doing a few things here:

Try to call video.play() when the component loads.
Show the user a play/pause state in the UI by using the default controls attribute.
Start with the video in the muted state. Our video does not have audio, but if it did you would still want to start off in the muted state with the muted attribute, and show a mute/unmute icon in the UI.

import React, { useEffect, useRef } from "react";
import "./styles.css";

export default function App() {
  const videoEl = useRef(null);

  const attemptPlay = () => {
    videoEl &&
      videoEl.current &&
      videoEl.current.play().catch(error => {
        console.error("Error attempting to play", error);
      });
  };

  useEffect(() => {
    attemptPlay();
  }, []);

  return (
    <div className="App">
      <h1>Autoplay example</h1>
      <div>
        <video
          style={{ maxWidth: "100%", width: "800px", margin: "0 auto" }}
          playsInline
          loop
          muted
          controls
          alt="All the devices"
          src="https://stream.mux.com/6fiGM5ChLz8T66ZZiuzk1KZuIKX8zJz00/medium.mp4"
          ref={videoEl}
        />
      </div>
    </div>
  );
}

No BART terminals were hacked in the making of this ad

Dylan Jhaveri — Tue, 21 Jan 2020 22:55:36 +0000

Originally posted by my colleague Bonnie Pecevich on mux.com/blog

In the process of creating a BART ad for the first time, we had some learnings that we thought we would share that could hopefully help someone else on their out-of-home ad buying journey. (We’ll also remember to follow our own advice for next time.)

Our learnings:

Ask upfront for explicit restrictions on creative.
Build in extra time for more than one round of feedback and time to iterate on design.
Be realistic about what’s feasible, especially with an aggressive timeline.
Submit a draft of the concept and see if they’ll approve it before you spend extra time finalizing the details (and telling everyone about it at the company all hands meeting.)

All of these learnings actually stemmed from one preeminent learning:

BART doesn’t allow any code on their ads. 😲

Why put code in an ad?

First, an introduction–Mux is a startup that does video, and one of our aspirational goals is for every developer to know that. With our headquarters located in San Francisco, we’re aware that our city has a great supply of developers so we thought we’d try advertising in some well-traveled, public spaces.

Doing ads in a BART station (the underground transit system in the Bay Area) is generally assumed to be expensive, maybe even beyond the reach of a startup which is what we thought, too. But we learned doing an ad could fit in our budget if we were flexible on timing–we were able to sign up for a single digital display at the Montgomery BART station with a 12/30/19 start date. Even though that only gave us about 2 weeks to create an ad (ignore the wailing coming from our one and only in-house designer), we were excited!

Since the ad is just :15 seconds long with a not-so-captive audience, we wanted to create something that quickly caught the attention of developers. We thought we could achieve this by showing a terminal with a blinking cursor and then typed code to show use of our API. Sure, it crossed our minds that a blank screen with a blinking cursor might look like the screen is broken (which adds to the eye-catching-ness), so we added browsers to frame the terminal and added our logo to the top left corner. Our hope was that someone would take away that Mux is for developers and, if we were lucky, that we do something with video.

Insert wrench here

The process was to submit a final file at least a week in advance of the live date to include time to get BART’s approval. There weren’t any specific guidelines beforehand on what’s allowed and what’s not but we assumed some common sense restrictions would apply like no explicit/harmful language imagery, etc. We figured getting BART’s approval would be relatively simple, like checking a box.

Wrong. Our ad was rejected! We received feedback that the beginning of the ad that showed the terminal could give the impression that “the screen is malfunctioning or has been hacked into.”

Busted. Turns out they also thought having a terminal on the screen would be eye-catching but not in a good way. We did feel a bit deflated, though, as we were all ready for our BART debut.

We went through the five stages of grief and settled on “Bargaining.” We tried to come up with a creative solution where we could still use the same ad. Hey, what if we could add a persistent banner to the ad that said something like “Don’t worry, no BART terminals were hacked in the making of this ad.”?

Or what if we stylized the terminal so it looked more illustrated and cartoon-y?

Alas, BART held firm and said, in no uncertain terms, “Nothing involving coding.” Since we couldn’t come up with a brand new design in 48 hours, our plans for a BART ad had to be put on hold.

Silver lining

All is not lost! We used the final video for our homepage and are genuinely excited at how it came out.

Although the BART approval process is still a bit of a black box, we're excited to continue to work with the same ad agency and pursue our out-of-home ad dreams. We’re looking forward to iterating our design and hopefully making a public appearance at CalTrain in the very near future. And if you see our ad, you’ll know the journey it took to get that little video up on those screens.

In defense of 'flicks' (or how I learned to stop worrying and love 705600000)

Dylan Jhaveri — Tue, 26 Nov 2019 18:26:56 +0000

Note originally posted by my colleague on the mux blog

About 2 years ago, the Oculus VR division of THE FACEBOOK created a project they called 'flicks'. Essentially a flick is just a really big number, specifically the number 705,600,000. This project was picked up by some news outlets like TechCrunch, The Verge, and the BBC and seems to cause some confusion and even some ridicule. To be fair, a news article about a number is a bit odd. If you’re not an engineer working with digital media, the idea behind this number is difficult to grasp. And to those who do work in digital media, the number seems to not offer anything new. It purports to solve a problem that nobody in the industry actually has. So where did it come from, and why does it exist? Let’s back up…

Time is a surprisingly difficult concept in digital media. For starters, we are dealing with time values that are very small and difficult to imagine. Recently I saw the movie Gemini Man at the AMC Metreon here in San Francisco. It was one of the few theaters capable of playing the 120 frames per second version, whereas most films are 24 frames per second. At 120fps, every frame is projected for just over 0.00833 seconds before flashing the next one– a very short period of time. But compared to digital audio, this is an eternity. Audio recorded at 44100hz has a sample every 0.000022675 seconds. That is 367.5 times more audio samples than video frames.

Example audio visual timeline

These numbers with fractional components (a decimal point) are known as “floating point” numbers in computer science and computers are surprisingly bad at dealing with them. Above when I said every video frame was on the screen for 0.00833 seconds, that was not exactly true. When you divide 1 by 120 The number result is 0.008333333333 with the 3 repeating forever. For a computer to store a number that repeats forever would require an infinite amount of memory, so the number is approximated. The difference between the approximation and the actual number results in tiny errors in the math. These small errors can add up over time and become big errors and could result in problems such as audio and video becoming out of sync. Using a unit of time like milliseconds or nanoseconds would help, but would only delay the problem and not solve it.

The ultimate solution is to not record time in seconds but instead as an integer number of fractional units. For example 1000 x 1 ÷ 120 is the 1000th frame of a 120fps video. Converting to seconds we still end up with a floating point number but as long as we count frames as integers the error does not accumulate over time. If you’re following the math closely you may have noticed that while solving this problem we have created another one.

What frame should we render first? The video frame at 1000 x 1 ÷ 120, or the audio sample at 367500 x 1 ÷ 44100? We need to convert to a common time base to know for sure. We could convert to seconds then compare, but that brings us back again to the floating point problem. By using the “least common multiple” or LCM, of the ratios, 88,200 in this case, we can convert these fractions to a common time base at which point we can compare them. 88,200 ÷ 44100 x 367500 = 735,000, and 88,200 ÷ 120 x 1000 = 735,000. These time stamps are exactly the same and should be rendered together to ensure sync. At no time did we need to use floating point math, which may have given us a slightly different answer.

In the world of digital media there are some time bases that come up very frequently. As stated, film commonly uses 24 fps, European television (and other PAL countries) use 25 fps, and for obscure reasons, American television (and other NTSC countries) use 29.97 fps. Wait! Floating point numbers again? Actually no, because it’s not really 29.97fps. It’s actually 30000 ÷ 1001 fps.

Here is where flicks come in. You see 705600000 ÷ 44100 = 16000 EXACTLY, 705600000 ÷ 120 = 5,880,000 EXACTLY, and even 1001 x 705600000 ÷ 30000 = 23,543,520 EXACTLY. This is why the flick is interesting, it has a special property of being the least common multiple of many of the commonly used timebases in digital media.

We now know what the flick is, Buy why? We have established that if we record every time stamp as 3 integers, a numerator, a denominator and a multiplier, we don’t need a common base since we can convert between them as needed. There are two primary reasons. First is efficiency. If we know we will need to compare a lot of time stamps in different time bases, or compare the same timestamp multiple times, converting them to a common base can be faster for a computer. Once converted to a common base, comparing two numbers is probably the fastest operation a computer can do. Whereas comparing two fractions requires an algorithm to find a common base, convert the value, and then compare. But the motivation cited in the flicks github page is slightly different, and caused by a design decision in the C++ programing language.

Computers are pretty good at dealing with time, but humans are really bad at it. In most parts of the United States, once a year for daylight saving, we have a 25 hour, only to have a 23 hour day a few months later. Every 4 years, we get an extra day at the end of February, unless the year is divisible by 100; except when the year is also evenly divisible by 400 then it is a leap year (there was no leap day in the year 1900). We even have leap seconds, where we add an extra second to a year whenever we notice that the atomic clocks don't quite agree with astronomical observations. Meanwhile a computer's clock needs to keep moving forward one second per second otherwise bad things happen.

To help with this human nonsense and standardize how to manage time, C++11 added a new package called chrono to its standard library. Because the language designers were smart, the std::chrono::duration time type included support for the time as a ratio technique we have established. Perfect! Well... not so fast. Because the language designers were unwilling to give up more cpu cycles and slow down programs (C++’s defining feature is speed after all), it was decided that the time base must be known in advance while writing the program (compile time). This allows for fast running programs because the fractions can be ignored when they are known to be equal, but it sacrifices automatic time base conversions when they are not. Herein lies the problem. When playing back a media file we can’t know the time base in advance, we haven't seen the file yet. What we need is a time base that can support any media we are likely to encounter. Enter flicks. An elegant solution to a problem only a handful of media engineers will ever encounter that happened to be announced on a slow news day.

Flicks does not seem to be in wide use. I used it at Mux in one specific place in our transcoding pipeline with its intended purpose. I used it with a C++11 program where utilizing std::chrono made things a bit easier to standardize on. But searching GitHub and Google, I could only find a handful of places where it is used in the wild, and I really don't expect that to change.

Using Netlify Functions to Create Signing Tokens

Dylan Jhaveri — Thu, 07 Nov 2019 20:38:31 +0000

Have you used cloud functions yet? They come in many flavors: Amazon Lambda, Cloudflare Workers, Zeit Serverless Functions, and the one we’re using here: Netlify Functions.

Cloud functions are an essential component of the JAMstack. I had not heard the name JAMstack until recently. For the uninitiated (like me) it stands for Javascript, APIs and Markup. You may have seen JAMstack technologies like Gatsby, Next.js and tools of this nature that focus on performance, new developer tooling, and leveraging CDNs to serve pre-compiled HTML pages. I will be at JAMstack Conf 2019 in SF, if you will be there too, then come find me and say hi!

All the code in this post is open source here on GitHub in our examples repo: muxinc/examples/signed-playback-netlify.

The Main Benefits of Cloud Functions

Run code close to your clients (clients can be browsers, mobile apps, internet-of-things devices, self-driving cars, drones, anything that is talking to your server). Like a CDN, cloud functions are deployed to edge data centers to minimize latency between your clients and the server that runs their code.
Protect your origin servers from being flooded with traffic. Cloud functions are a good way to cache data and intercept requests and respond to your users before they reach your origin servers. This means less bandwidth and CPU that your origin servers have to process.

Cloud functions, like Netlify Functions, might be a good option for you if you are using Mux’s Signed URLs feature.

A Little Background About Signed Urls

When you create a video asset via Mux’s POST /video API you can also create a Playback ID (Mux API docs) and specify the playback_policy as either "public" or "signed".

A “public” playback policy can be played back on any site, in any player and does not have an expiration. A “signed” playback policy requires that when the playback URL is requested from a player, it has to be accompanied by a “token” param that is generated and signed on your server.

This is how it looks:

public playback URL:
https://stream.mux.com/${playbackId}.m3u8

signed playback URL:
https://stream.mux.com/${playbackId}.m3u8?token=${token}

The token param is what you need to create on your server in order for the playback URL to work.

Create a Mux Asset

Sign up for a mux.com account (free account comes with $20 credit)
Go to settings/access-tokens and click “Generate new token” to create a token you can use for API calls
Copy your token id (we'll call this MUX_TOKEN_ID) and secret (MUX_TOKEN_SECRET). You will need these to make two api calls.
Create a Mux video asset with a “signed” playback policy

curl https://api.mux.com/video/v1/assets \
  -X POST \
  -H "Content-Type: application/json" \
  -u ${MUX_TOKEN_ID}:${MUX_TOKEN_SECRET} \
  -d '{ "input": "https://storage.googleapis.com/muxdemofiles/mux-video-intro.mp4", "playback_policy": "signed" }'

Copy the playback_id from the response, you will need this later.

Create a URL Signing Key

curl https://api.mux.com/video/v1/signing-keys \
  -X POST \
  -H "Content-Type: application/json" \
  -u ${MUX_TOKEN_ID}:${MUX_TOKEN_SECRET}

Copy the id (MUX_TOKEN_ID) and the private_key (MUX_PRIVATE_KEY) from the response, you will need these later. These are the keys you will need to create signed urls for playback.

Setup a Netlify project

Create a new directory for your project mkdir netlify-mux-signing && cd netlify-mux-signing
Install the Netlify CLI and run netlify init to create a new project. You can choose to connect Netlify to a GitHub repository.
If you’re starting from scratch, run yarn init to create an empty package.json and git init to make this a git repository.
Now you have a barebones project that is connected to Netlify, but nothing is in it yet (you can see there is a hidden and gitignored directory called .netlify which Netlify uses to handle deploys and Netlify commands
Run yarn add netlify-lambda to install the netlify-lambda package into your project (it’s recommended to install this locally instead of globally).
Run yarn add @mux/mux-node to add the Mux node SDK to your project

Step 1: Create a module to generate a signing token

Create a src/ folder in your project and let’s create a small module called mux_signatures.js. It will export one function called signPlaybackId which takes a playback id and returns a token that is generated with Mux.JWT.sign:

// ./src/mux_signatures

import Mux from '@mux/mux-node';

export const signPlaybackId = function (playbackId) {
  return Mux.JWT.sign(playbackId, {
    keyId: process.env.MUX_SIGNING_KEY,
    keySecret: process.env.MUX_PRIVATE_KEY
  })
}

Our lambda function is going to use this module in Step 2.

Step 2: Create a `sign_playback_id` cloud function

Create a Netlify Function entry point. This is the single function that will handle one request. The idomatic pattern for creating cloud functions is to do one file and one javascript function per route. We will create a directory called functions/ and add a file called sign_playback_id.js.

// ./functions/sign_playback_ids.js

const keySecret = process.env.MUX_PRIVATE_KEY
import { signPlaybackId } from './src/mux_signatures';

exports.handler = async (event, context) => {
  try {
    const { queryStringParameters } = event;
    const { playbackId } = queryStringParameters;
    if (!playbackId) {
      return { statusCode: 400, body: JSON.stringify({errors: [{message: 'Missing playbackId in query string'}]}) };
    }
    const token = await signPlaybackId(playbackId);
    return  {
      statusCode: 302,
      headers: {
        'Access-Control-Allow-Origin': '*',
        location: `https://stream.mux.com/${playbackId}.m3u8?token=${token}`
      },
      body: '',
    }
  } catch (e) {
    console.error(e);
    return { statusCode: 500, body: JSON.stringify({ errors: [{message: 'Server Error'}] }) };
  }
}

Step 3: Add netlify.toml

Add a netlify.toml file to the root directory and tell Netlify where your functions will live. This tells Netlify that before we deploy we are going to build our functions into the ./.netlify/functions directory

    [build]
      functions = "./.netlify/functions"

Step 4: Connect to git

In order to use Netlify Functions you will now need to commit your code and push it up to a git repository like GitHub. Do that next and in Netlify’s dashboard connect your git repository to the Netlify project that you created. After connecting you git repository then

Step 5: Set your environment variables

In your Netlify project dashboard, naviate to "Settings" > "Deploys" > “Environment” to set your environment variables. Enter the MUX_SIGNING_KEY and MUX_PRIVATE_KEY from the Create a URL Signing Key step above.

Step 6: Test in development

Open one terminal and run netlify dev this will start a local Netlify dev server
Open another terminal window and run netlify-lambda serve ./functions this will build your functions/, get them ready to handle requests and watch the filesystem for changes.
In a third terminal window, curl your endpoint to test out the function (replace <netlify-port> and <playback-id> with your values.

curl -I 'http://localhost:<netlify-port>/.netlify/functions/sign_playback_id?playbackId=<playback-id>'

You should see a 302 (redirect) response with a location header for the signed url.

When you make any changes to your source files, netlify-lambda serve will pick up on the changes and recompile the functions into ./.netlify/functions.

Deploy

When you’re ready to deploy, you can deploy from the command line with netlify-lambda build ./functions && netlify deploy --prod. This will build the functions and then push up the changes to Netlify.

Try making a POST request to your cloud function on Netlify:

curl -I 'https://<your-netlify-app>.netlify.com/.netlify/functions/sign_playback_id?playbackId=<playback-id>'

Just like in dev, you should get back a 302 response with a location header that points to the signed playback URL.

https://stream.mux.com/${playbackId}.m3u8?token=${token}

This is what your response should look like:

HTTP/2 302
access-control-allow-origin: *
cache-control: no-cache
location: https://stream.mux.com/jqi1UtiO3gccQ019UcYjGJTLO9Ee00TLMY.m3u8?token=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IkFsVFZncktBVTYzVldIdVplcDEwMVhZUk5mbHozeDIxRiJ9.eyJleHAiOjE1NzE3NjE0NzMsImF1ZCI6InYiLCJzdWIiOiJqcWkxVXRpTzNnY2NRMDE5VWNZakdKVExPOUVlMDBUTE1ZIn0.i7oANZ6inmwmGVQjon4WEv_gKcqQ2v8GuQA8xuCBdT0Reegkm6WyTdU-VloZvAt7duaRR3-T8dt147vUQjM1n70CLi0996pwMejYWIbRHUMqrDBtsENHG8T9jtz-EJcBGONSzgs7fBQIVQx8xJvPuX4YqpylDK_lNX0-RDqfhz5THAfuyxzePJod709msD8kbHAqnIke5lHzbQNHuO2ecNFVCb2ZozW7XkIEctyLxrDAK1ITtQV8iHek3whwO9S05kM-5bQzomJEliN3mXBqCwMBmyIp8l88YKl59tVXDdU-l-cZvZjt1GYKv0J7shO-oBYcr00NmVKkP7bie_w50w
date: Tue, 15 Oct 2019 16:24:33 GMT
age: 0
server: Netlify
x-nf-request-id: 80484951-e7ff-46f3-b78e-1349b8514bec-1426623

Now, in your player, you can use your netlify function as the URL src. Here's an example for a web player (note that in order to get HLS in a <video> tag to work outside of Safari you will need to use another library like [Video.js](https://videojs.com" target="_blank" or HLS.js:

<video src="https://<your-netlify-project>.netlify.com/.netlify/functions/sign_playback_id?playbackId=<playback-id>"></video>

And here's an example on iOS with AVPlayer in Swift:

let url = URL(string: "https://<your-netlify-project>.netlify.com/.netlify/functions/sign_playback_id?playbackId=<playback-id>")

player = AVPlayer(url: url!)
player!.play()

The player will load the netlify URL, get the 302 redirect to the signed Mux URL and load the HLS manifest from stream.mux.com.

Restrict who can access your function

Now that your cloud function is working, you can add some security around it to make sure you only allow authorized users to generate signed urls.

For web players you will want to change this line in the sign_playback_id.js function:

'Access-Control-Allow-Origin': '*',

You can use the Access-Control-Allow-Origin header to control the CORS rules for the resource.

Phoenix LiveView: Build Twitch Without Writing JavaScript

Dylan Jhaveri — Thu, 31 Oct 2019 20:21:14 +0000

Phoenix LiveView is a new experiment that allows developers to build rich, real-time user experiences with server-rendered HTML. If you’re not familiar with Phoenix, it’s the fully-featured web framework for the Elixir programming language. At Mux we use Phoenix and Elixir to power our API. I decided to start playing around with LiveView to see what it’s capable of. The idea I had for an example app is Snitch, it’s like “Twitch,” but for snitches (put away your checkbooks potential investors). Under the hood, of course, we’re using Mux Live Streaming.

From the user’s perspective, first you create a “channel”. When that channel is created, Snitch will give you RTMP streaming credentials (just like Twitch does). As the user, you enter those streaming credentials into your mobile app or broadcast software and start streaming.

Right here we have the perfect test case for LiveView. In the UI we show the user the streaming credentials and are now waiting for them to start streaming. Mux is going to send webhooks to our server when relevant events happen. For example:

video.live_stream.connected
video.live_stream.recording
video.live_stream.active
video.live_stream.disconnected

In a typical web application, without LiveView, common solutions are to either use websockets to push new data to client applications or have those applications poll the server. About every second or so the browser would send a request to the server to get the updated data. But now with LiveView when a webhook hits the server we can re-render on the server-side and push those changes to the client.

Using LiveView to Handle Webhooks

The first step is to follow the instructions on the Installation page to add LiveView to your Phoenix application. This includes adding the dependency, exposing the WebSocket route and the phoenix_live_view JavaScript package for the client-side.

After following the installation instructions, let’s add a route for the Mux Webhooks:

    scope "/", SnitchWeb do
      pipe_through :api
      post "/webhooks/mux", WebhookController, :mux
    end

Then, in the Mux UI we can add this as our webhooks route. For local development I’m using ngrok to receive webhooks on my localhost server.

The webhook controller is going to receive the payload and update the “channel” in our database by calling Snitch.Channels.update_channel . Let’s look at the update_channel/2 function:

# lib/snitch/channels.ex 
def update_channel(%Channel{} = channel, attrs) do
  channel
  |> Channel.changeset(attrs)
  |> Repo.update()
  |> notify_subs()
end

notify_subs/1 is the new function we are going to call when a channel gets updated. This is where the LiveView magic happens.

# lib/snitch/channels.ex 
def notify_subs({:ok, channel}) do
  Phoenix.PubSub.broadcast(Snitch.PubSub, "channel-updated:#{channel.id}", channel)
  {:ok, channel}
end

This function is going to broadcast a message so that subscribers can react to this change. More on that shortly.

Now let’s update the controller and tell the controller to render with LiveView:

# lib/snitch_web/controllers/channel_controller.ex 
Phoenix.LiveView.Controller.live_render(conn, SnitchWeb.LiveChannelView,
  session: %{channel: channel}
)

And let’s create SnitchWeb.LiveChannelView. When we call notify_subs() up above, this LiveChannelView is the code that needs to subscribe and push an update to the client.

defmodule SnitchWeb.LiveChannelView do
  use Phoenix.LiveView

  #
  # When the controller calls live_render/3 this mount/2 function will get called
  # after the mount/2 function finishes then the render/1 function will get called
  # with the assigns
  #
  def mount(session, socket) do
    channel = session[:channel]
    if connected?(socket), do: SnitchWeb.Endpoint.subscribe("channel-updated:#{channel.id}")

    {
      :ok,
      set_assigns(channel, socket)
    }
  end

  def render(%{playback_url: nil} = assigns), 
    do: SnitchWeb.ChannelView.render("show.html", assigns)

  def render(assigns), do: SnitchWeb.ChannelView.render("show_active.html", assigns)

  #
  # Since the mount/2 function called "subscribe" to with the identifier
  # "channel-updated:#{channel.id}" then anytime data is broadcast this
  # handle_info/2 function will run and we have the power to set new values
  # with set_assigns/2
  #
  # After we assign new values, the render/1 function will get called with the
  # new assigns
  #
  def handle_info(channel, socket) do
    {
      :noreply,
      set_assigns(channel, socket)
    }
  end

  def set_assigns(channel, socket) do
    playback_url = Snitch.Channels.playback_url_for_channel(channel)

    socket
    |> assign(name: channel.name)
    |> assign(status: channel.mux_resource["status"])
    |> assign(connected: channel.mux_resource["connected"])
    |> assign(stream_key: channel.stream_key)
    |> assign(playback_url: playback_url)
  end
end

To summarize what’s happening above:

live_render/3 will invoke mount/2
mount/2 will subscribe using an identifier ("channel-updated:#{channel.id}") and set_assigns for the view
render/1 will get called with the assigns
anytime somewhere else in the app broadcasts to ("channel-updated:#{channel.id}"), this view is going to call handle_info/2 and that gives us the opportunity to use set_assigns again to update the assigns and re-render the template
re-renders auto-magically get pushed to the client over a websocket and the client updates the dom

The only difference in the show and show_active templates that we use in LiveChannelView is that instead of eex extension we use the leex extension which stands for live embedded elixir.

Here is a webapp where this is currently deployed at snitch.world The full code is up here on github. You can clone it and run it yourself. You’ll also need to sign up for a free account with Mux to get an API key. Feel free to reach out if you have any questions!

DEV Community: Mux

Kafka Connect: The Magic Behind Mux Data Realtime Exports

Enter, Kafka Connect

Getting Started with Kafka Connect

Step 1: Build your Kafka Connect Docker Image

Step 2: Start up your Kafka Connect Cluster

Step 3: Run some Connectors!

Example: GCP PubSub Sink

Example: AWS Kinesis Sink

Lessons We’ve Learned

Make sure ‘CONNECT_REST_ADVERTISED_HOST_NAME’ is set

Set the ‘errors.retry.timeout’ value on your sinks

Autoscaling is tricky

Monitoring

Conclusion

A Beginner's Guide to Video File Formats: MP4s

How the sausage gets made

Streaming Video

The state of going live from a browser

That's gonna be a no from browsers, dawg.

Why go live from the browser?

Before we go on...

Ok cool, now let's talk about our options.

WebRTC rebroadcasting

Broadcasting headless Chrome

Video over WebSockets

The Client

The Server

Profit! Kinda.

Let us help your users go live!

Mux is the video API for the JAMstack

What is the JAMstack?

Static content frameworks

Deploy

Cloud Functions (“Serverless”)

Headless CMS

Authentication (advanced)

Database (advanced)

How did we get here?

Video for the JAMstack

JAMstack at Mux

Migrating to a headless CMS

How to Host Your Own Online Conference

Let’s break down the steps and API calls:

Live chat

Everything is recorded

Bonus: simulcast to the socials

Bonus: add a watermark

You don’t have to use Zoom

Final Setup

Extra details if you’re curious

Considered Harmful

Browsers will block your autoplay attempts

Autoplay will probably work for you, but it will break for your users

Here’s an example with vanilla javascript

Here’s an example with React

No BART terminals were hacked in the making of this ad

Why put code in an ad?

Insert wrench here

Silver lining

In defense of 'flicks' (or how I learned to stop worrying and love 705600000)

Using Netlify Functions to Create Signing Tokens

The Main Benefits of Cloud Functions

A Little Background About Signed Urls

Create a Mux Asset

Create a URL Signing Key

Setup a Netlify project

Step 1: Create a module to generate a signing token

Step 2: Create a sign_playback_id cloud function

Step 3: Add netlify.toml

Step 4: Connect to git

Step 5: Set your environment variables

Step 6: Test in development

Deploy

Restrict who can access your function

Phoenix LiveView: Build Twitch Without Writing JavaScript

Using LiveView to Handle Webhooks

Demo

Step 2: Create a `sign_playback_id` cloud function