DEV Community: Preet Shihn

Autoplaying back-to-back videos on the web using a Media Pool

Preet Shihn — Fri, 14 Aug 2020 18:09:19 +0000

No one likes it when you go to a web page and it automatically starts playing a video loudly. Browsers have changed their auto-play policies to let JavaScript auto-play a video only if it's inline and muted. (See posts for Safari and Chrome).

Problem: Playing a video queue

This change in policy is great, but it may restrict some use cases. Say, you have created a queue of videos (or audios). You'd want the next one to play as soon as the first one finishes. Instagram/Snapchat like stories is a good example for this.

Normally one would implement this by listening to your video's progress and as soon as it ends, call play on the next video. But, if the video is not muted, it will not play according to the browser's media policy.

Unmuted videos can be played if they were triggered by a user action. But, when playing one video after another, asking the user to tap to play again doesn't feel like a great user experience.

Reusing the media element

Once a media element (<video> or <audio>) has been triggered by a user action, it gets cleared by the browser for future use. i.e. Later on, some script can play the video unmuted.

With that in mind, the very basic solution is to create a single media element, and re-use it for every video. This, however, may not be a great user experience. Loading the next video may take time, so transitioning from one video to another will not be a smooth one.

Media Pool

In an ideal world where resources and browser limitations do not exist, one could load every video in memory. But it's not good practice, and some browsers (like iOS Safari) limit the number of media elements in memory.

So we create a Media Pool. This is similar to thread pools in multi-threaded languages where you allocate a thread to a task and when the task is done, the thread is released to be used by another task.

Applying this model to media elements, we can keep a few videos in memory for a smooth experience, and release them back to the pool when they are done playing. As new videos get queued up to play, they request a media element from the pool. The pool will provide an unused or a recycled media element.

In the following diagram, we have a pool of seven media elements ♦. We keep the last video in memory, and preload the next one. So there are only 3 allocated media elements at a time. As the videos leave this three-video-window, the media element is released back into the pool.

When video-3 finishes, video-4 starts playing. video-2 is outside the window so it releases the media element back into the queue. video5 gets a new element allocated from the queue.

Blessing the Media

A media pool sounds like a great way to reuse video elements but we have not addressed the original problem - playing unmuted videos automatically. I discovered that Google's AMP project had the same problem as this, and they use a term called blessing a video element.

When the user takes their first action to unmute a video or to start an unmuted video; in the same event handler, the media-pool will unmute all the videos in the pool (allocated, unallocated, and released ones). It will then mute the ones that are not active right away. By doing this, all videos have been cleared by the browser. If we use AMP's terminology, these videos have now been blessed.

As one video finishes, the script can now call play on a blessed video element to start playing it even when it's unmuted.

There are a couple of things to keep in mind when implementing a blessing based media pool. Firstly, one cannot invoke operations like mute, unmute, play on a media element if it doesn't have any media loaded. That's why the unallocated media-elements should be loaded with an empty video file. Secondly, in some browsers operations of media elements can be interrupted when invoked in a certain way, for example calling load after calling play can reject the play promise. The AMP project creates a queue of operations per media element. I decided to do the same in my case.

Media Pool implementation

I have created a small framework independent media-pool implementation that you can use if you like, or feel free to look at the code to create a new one. AMP's version of media pool is available here but it may be a bit AMP specific.

I have further optimized my media-pool implementation so it lazy-releases a media-element; which means if the video is active again it can reuse its old media-element even though it was released. This prevents re-attaching to DOM and re-loading videos. In the case where the released media-element has been allocated to some other video player, the video player will get a new one from the pool.

Need for Seed: Taming Randomness in a Pseudorandom World

Preet Shihn — Tue, 19 May 2020 20:39:54 +0000

Computers are inherently not good at randomness. The thing is, you can program a machine to generate numbers which may be considered random, but the machine is at the mercy of its programming. You cannot really call something truly random if it follows an algorithm to generate the random numbers.

Truly random numbers can be generated but they usually rely on unpredictable physical processes which are not defined by man-made patterns, e.g. based on thermal noise or nuclear decay of a particle. Random.org, for example, generates true random numbers from atmospheric noise. These are used in casino games, and lotteries to make them unpredictable.

Pseudorandom numbers

Programming languages and libraries provide procedures to generate what we call pseudorandom numbers. These algorithms generate a sequence of numbers approximating a truly random sequence. For most applications, these pseudorandom number generators (PRNGs) are good enough.

Typically, a PRNG would start with a seed — a number that forms the starting point of the random sequence, and subsequent numbers are generated from there. So if you always start from the same seed, you will always get the same sequence of numbers.

Auto seeding pseudorandom number generators

Most libraries and languages will provide an interface for you to specify the seed to generate a sequence, but they will usually default to automatically selecting a seed.

JavaScript for example, does not let you specify a seed for Math.random() and crypto.getRandomValues(). Each invocation of any of these methods produces a fresh unpredictable random number which is not reproducible across runs. This is great, right? Yes, but there are cases where controlling the seed is desirable.

Need for Seed 🏎

Being able to specify a seed for a PRNG has several use cases, but they could be generally described as cases where the state is tied to the random sequence. Here are some examples:

A generated graphics object or a data structure. There may be several cases where the object instance needs to be recreated (or repainted in the graphics world). You want the object to be the same every time the function is invoked for that instance. e.g. Consider this sketchy circle which is generated using a set or random points. If this instance of the circle is drawn on a canvas n times, each rendering should look the same.

Simulations & Testing frameworks — sometimes you want to replicate a test in different environments and you may want to replicate the exact test run in those environments. Same goes for simulations - simulating certain physics systems or processes repeatedly.
Gaming — The environment in a game (world/scene) is often generated with some randomness, be it terrain artifacts or locations of hidden gems. When a player returns to a game after saving it, you would want the generated world to be the same as when they left it.

Seeding in JavaScript

When the language or the API does not give you a way to seed your random numbers, as in JavaScript, you need to create your own PRNG. There are many out there. Here's a very simple one, even though it may not be the best one:

class SeededRandom {
  constructor(seed) {
    this.seed = seed;
  }

  next() {
    return ((2 ** 31 - 1) & (this.seed = Math.imul(48271, this.seed))) / 2 ** 31;
  }
}

// To create a random seed:
function createNewSeed() {
  return Math.floor(Math.random() * 2 ** 31);
}

There's a TC39 proposal to add seed based random number generation in JavaScript, but there doesn't seem to be much activity going on 🙁

Things to consider when using seeds

When seeding, you are usually creating some artifacts using the generated random numbers.
These artifacts should be the same for the same seed value.

Maintain the order

Let's say you need to create three artifacts from a seed: A, B, C and each requires two random numbers.

The sequence of numbers will always be deterministic, so if the artifacts are generated in a different order, you will not maintain the consistency you need.

Constant bits and Dynamic bits

There are times when you use a seed to create an object, but the next time around you may want the same object but with slightly different attributes. This may sound vague, so let's look at a specific example: You use random points to create the edges of a polygon and fill the polygon with more random lines. Now you want to scale this shape up. When doing so, you can use the same seed to maintain the general shape of the polygon; but to maintain the density of the filled lines, you need more lines.

The simple rule here is to figure out which bits of your object remain constant, and which ones don't. Always create the constant ones first so you always use the same sequence of numbers for the constant bits. This scenario is not specific to graphic shapes of course.

Back to the polygon example, consider a sequence of random numbers for a specified seed N1, N2, N3, ..... Say you need k random numbers to draw the outline of the shape. Then you use more numbers unti Nl to fill the shape.

When you scale this shape up, the value of k remains constant, and you may now have to go upto Nm to fill the larger shape.

Sub-seeding

Consider an object that needs to maintain two parallel sets of artifacts. A1, B1, C1 and A2, B2, C2, each artifact requiring a random number. These can be generated in two ways. A1, A2, B1, B2, C1, C2 (interleaved method) or one segment at a time A1, B1, C1, A2, B2, C2 (sequential method).

Both these methods will generate the same object every time because the assignment of the random numbers is fixed. But let's say you want to add a fourth artifact while maintaining the values of the previous artifacts.

Only the first method of interleaving parallel artifacts maintains the values of A, B, C when D is added. (More specifically, notice that A2 is always associated with n2 in the interleaved method, and gets associated with n4 or n5 in the sequential method).

So you might say, always interleave in situations like this. Which is great, but there are cases where it may not be possible. Limitations of the design may lead to situations where computing A2 may not be possible before computing A1, B1, C1, D1.

One strategy for dealing with this is creating a new seed that is a deterministic value derived from the original seed. I'm going to call this method sub-seeding. With this new sub-seed, you create a new sequence of numbers that is still determined by the original seed.

Creating a sub-seed could be a very simple function, e.g. subSeed = seed + 1. Now you have two independent sequences of numbers determined by a single seed associated with the object.

Announcing Wired-Elements 2.0

Preet Shihn — Thu, 24 Oct 2019 20:36:55 +0000

I'm happy to announce the 2.0 release of wired-elements. It started of, and still is, a fun project to work on. In this era of really slick and buttery smooth design systems, a brutal and rough beacon of light 😉

In this release, the underlying base code for all components was refactored to create smaller and more efficient components. By more efficient, I mean fewer re-renders of the sketchy shapes. This was done mainly by not re-rendering shapes if the dimensions do not change, and also by utilizing the Resize Observer where it's available.

View all components in action in the wired-elements showcase.

New components in 2.0 👇

wired-calendar

This is the first fully community contributed component. Eduardo Martinez created this. The calendar is customizable in many ways, for details check out the wired-calendar readme.

wired-video

A simple video player that shows a sketchy progress and hand-drawn controls for play/pause and volume.

wired-image

An image component akin the the <img> tag, but it frames the image in a hand-drawn box. The elevation of the box is configurable.

wired-dialog

A modal dialog implementation. This contents of the dialog look like paper cutouts around a hand drawn box.

wired-divider

A hand-drawn horizontal line that can be used to divide two sections.

wired-link

Akin to <a> tag, a link with href, and a sketchy underline.

wired-search

A search input control

Try wired-elements out on Glitch playground.

Checkout wiredjs.com for more.

Why I use Web Components - My use cases

Preet Shihn — Fri, 21 Jun 2019 19:53:14 +0000

First of all, I am not a blogger or advocate of any technology, or speak at events/conferences. So mind my first attempt at writing. I hope I am clear in expressing myself.

I want to talk about some practical use cases and scenarios where I have found Web Components to be the best solution available.

Whenever I have seen discourse on Web Components, it usually devolves to API design and comparison to frameworks. I will not defend the API as I did not write it. I will not criticize it, others have done a better job at it.

Solved Problem?

It is often stated the components are a solved problem - look at all these frameworks with religious following!

Now, if you're developing an app, say a book recommendation app - you should write this in any framework you are comfortable with. Or write in only web components or write in HTML + Javascript circa 2000. All good. You should use whatever component system you want to, as long as it meets your goals.

But, if you want to write a rich component that is truly shareable, there is no other model available. You cannot use React components in Svelte or Svelte modules in React or in Vue or in Nimbus3000.

My Use Cases

I'm going to talk about real scenarios where I think using WebComponents seems like the right choice. The central theme here is shareable and cross-framework.

1. Embeddable Frontends (MicroFrontends?)

I help lots of websites add engagement and all sorts of bells and whistles to their sites. These sites are not maintained by fancy developers - think bloggers, artists, content creators, small businesses. Some of them get huge amounts of traffic (100M pages per month). We have a data engine that scans and monitors data on their sites and feeds them to embeddable frontends that add engagement to sites. Some examples:

Real time search that is customized to the user
Launch a dynamic game related to the content the user is interested in
Infinite list of related content

Most of these sites are on WordPress, Squarespace, Framework rendered using React, Vue. Our frontends get injected as WebComponents.
These publishers add the component wherever they want. No npm or fancy build process here.

Being in a web component shields the content from whichever theme they are using or whichever other frameworks that are running on the system. Some of these components interact with other content on the host page. They have to be highly performant and small in size.

These publishers usually have basic HTML knowledge and they add them to their content just like they would be adding images or videos.

2. Complex Widgets

The above example was a very customized solution, but then there are developers who create general widgets.

<model-viewer> is a perfect example of this one. It's been used all over the place like in NASA, and Shopify. I have no clue what framework Shopify uses (and it's probably not the same as NASA), but with the model-viewer available as a Web Component, they had a solution.

// Detect dark theme var iframe = document.getElementById('tweet-1141363065901977601-189'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1141363065901977601&theme=dark" }

Same applies to so called 'design systems'. Ionic's components for example, developed as Web Components and then also wrapped for different frameworks like React, Vue.

3. Evolving HTML

HTML5 added all sorts of new tags to the spec like <video>. Will it never add any new tags? One argument is that DOM sucks and all new elements should be defined a completely new component format. Or more realistically, it will probably add more tags - people know how to use tags already.

One of the new tags being considered is a switch. To test out possible implementations, Chrome is shipping it as web component <std-switch>, because it essentially extends the built in elements. When not on Chrome, the web component could be loaded as a fallback module. Extending existing underlying element model has its values.

Personal story in this category: A friend of mine in grad school was trying to use MathML on some web page. (They are NOT a developer.) Chrome does not support MathML at the moment. They could have used some sort of library to render it. I implemented MathML as web components as a fun project, and they could use it in chrome with minimal changes.

Styling

In some ways there is no such thing as Web Components in the platform. There are a few separate specs, like Custom Elements, and ShadowDom, being the main ones. More under development for importing CSS, HTML, JSON.

All of these individual spec have merit on their own, and their own design flaws, in the eyes of the developer. These can be used individually without being all web componenty.

Parts of the component that are not inside a ShadowDOM can be styled using your global stylesheet, where parts inside the ShadowDOM are isolated. This is particularly useful in the shareable component case which I focus on.

Developer Experience

A common complaint people have with WCs is that they are too verbose to code. It doesn't have bindings, or whatever. Like I said earlier, I'm not going to debate the merits and faults of the existing API and DX.

I do think that it is fair to use frameworks and libraries if you want to. I mean, you already do and you even compile at times. Some believe they should use no external libs at all, and that is a fine goal to strive for. But the reality is that it is much easier for most devs to use libs. So stop comparing DOM api to a framework API which could be compiled to DOM api. I think helper libraries are excellent. We already use it for so many other web tech like Web RTC, Workers, etc.

There are some great helper libs that will help you with WCs should you wish to. A few: Lit Element, Stencil, Haunted if you like hooks.

Example using LitElement:

@customElement('my-counter')
class Counter extends LitElement {
  @property({ type: Number }) count = 0;

  render() {
    return html`
      <div>${this.count}</div>
      <button @click="${() => this.count++}">Increment</button>
    `;
  }
}

Example using Haunted:

function Counter() {
  const [count, setCount] = useState(0);
  return html`
      <div id="count">${count}</div>
      <button type="button" @click=${() => setCount(count + 1)}>Increment</button>
  `;
}
customElements.define('my-counter', component(Counter));

Not your use case

My use case is not your use case. This may all mean nothing to you, but I thought you may be interested in hearing about a point of view from a different plane.