DEV Community: Vitalii Bobrov

Web Audio for Electric Guitar: Cabinet Emulation

Vitalii Bobrov — Mon, 17 Jun 2019 19:22:36 +0000

It is hard to imagine a guitarist without any amplifier or guitar cabinet. Even more without any speakers, an electric guitar is hard to hear even at home. Today I want to show how to emulate guitar cabinet using Web Audio API. It is possible to recreate a sound made by top brand amp without the investment of thousands of dollars using convolution.

In previous post, I described the process of getting sound from an electric guitar in a browser using Web Audio. Today I'm continuing that series with guitar cabinet emulation.

This post is a part of "Web Audio for Electric Guitar" series, check out other posts as well!

Web Audio for Electric Guitar Series:

How to Connect Instrument
Cabinet Emulation - Current

While playing on electric guitar at home, it usually connected to an individual device - amplifier or head with a cabinet. There are a lot of famous amp producers, and each of them has its own "firm" sound. However, using digital sound processing, it is possible to emulate them programmatically using a process called convolution. For simplicity, you can imagine convolution as a multiplication of two functions. When we are speaking about sound convolution mean the application of some sample to the live input. To simulate a guitar cabinet, we need to apply such a short sample recorded from real device to guitar sound. That samples called impulse response (IR).

Impulse Response

An impulse response is a recording of an amp impulse characteristics such as amplitude or frequency. For example, a photo is a snapshot of a light that the camera got on a film or digital matrix at some point. You can think about impulse response in the same way. It is a snapshot of a live speaker.

Unfortunately, that snapshot is limited to only one particular setting is recorded. What I mean by this is that on the real amplifier, you have a bunch of controls like volume or equalizer, and depending on amp configuration, you get different impulse response. So you can only simulate a device with a particular configuration. However, we can implement equalizer using Web Audio as well. Equalization gives us some flexibility to get the sound shape we want.

Convolver Node

Now we have an idea what we want to perform, and it is time to learn how to implement that using Web Audio API. The API hides much heavy math behind nodes it provides. Convolution is not an exception - we have a Convolver Node to apply impulse response to audio stream:

const context = new AudioContext();
const convolver = new ConvolverNode(context);

The convolver node has a buffer option that is used to pass an impulse response audio buffer. You must load an audio file with the IR in a format browser understands. Modern browsers support different formats that happened because of those formats licensing. Modern browsers have excellent support of WAV (all except IE, that doesn't support Web Audio as well), AAC (Firefox supports it only in an MP4 container), MP3, FLAC, and OGG (all except IE and Safari). I considered staying with WAV as it is well-supported and is a lossless audio format. The quality is essential here because we are using a very short - just a few bytes recordings and the compression could create artifacts in the output.

If you want to provide various file formats depending on the current browser you can check the support:

const audio = document.createElement('audio');
console.log(audio.canPlayType('audio/wav')); // "maybe"

if (!audio.canPlayType('audio/wav')) {
  console.log('The format is not supported!');
}

All browsers, including IE 9+ support 'canPlayType' method. It returns a string with 3 possible values: 'maybe', 'probably', and ''. Yes, it couldn't give you an exact answer 🤣, only probabilistic one. The empty string means that the format is not supported. 'maybe' - can't answer without starting playback, and 'probably' - seems to be possible.

You might have a reasonable question where you can get those cabinets to impulse responses? You can "do it your self"™️ - but this variant requires you to have a cabinet itself and a bunch of additional equipment, like capacitor microphone, professional audio card, and so far and so forth. Luckily there are a lot of free high-quality impulse responses made by professional studios and enthusiasts. Just google for "free cabinet impulse response" to find and download one. If you are too lazy you can check the impulses I'm using for a "JS Rocks" app.

After we have an IR to work with, we can load and apply it to the Convolver Node using audio context and Fetch API:

const convolver = new ConvolverNode(context);

fetch('impulse.wav')
  .then(response => response.arrayBuffer())
  .then(buffer => {
    context.decodeAudioData(buffer, decoded => {
    convolver.buffer = decoded;
  })
  .catch((err) => console.error(err));
});

Note: it is essential to disconnect/connect Convolver Node after a new buffer set if you want to re-use the same node few times. If you set a new buffer on the connected node the old buffer will be used, and possibly you get audio glitches.

We fetched the impulse response file, then transform the response into array buffer. We can't apply that buffer directly on convolver, before that we need to decode it using context. When convolver configured you can chain it in your audio processing graph:

guitarInput.connect(convolver).connect(context.destination);

Some IR could have a low amplitude, so after you apply it, they might reduce the overall volume. In this case, you can boost it using a Gain Node:

const makeUpGain = new GainNode(context, {
  // Need to be adjusted to a particular IR.
  gain: 5
});

guitarInput
  .connect(convolver)
  .connect(makeUpGain)
  .connect(context.destination);

The gain value needs to be adjusted manually for a particular impulse response. It is also good to expose that configuration in the user interface with some control like range input.

Three-band Equalizer

The last feature I want to add to cabinet emulation is a three-band equalizer. It gives us the tone control of the output. We want to create a configurable filter for bass, middle and treble frequencies. All below 500 Hz will is related to bass, between 500 Hz and 3 KHz to the middle, and above 3000 Hz to treble.

How can we increase the output of particular frequencies using Web Audio? We have a great node for that - Biquad Filter Node. It is an implementation of a group of filters, that could be specified by provided type value. For bass control, we pick a 'lowshelf' filter. It will increase the level of all frequencies below passed one. For treble, we use the opposite type - 'highshelf'. It boosts everything above the passed value. Moreover, for middle, we choose 'peaking' type. It boosts frequencies around passed value - the range of the band controlled by the Q (filter quality) parameter. Attenuation or boost for each band changed by gain parameter. It is an amount of level change in dB applied to the frequencies, using negative values we attenuate it, using positive - boosting.

const bassNode = new BiquadFilterNode(context, {
  type: 'lowshelf',
  frequency: 500
});

const midNode = new BiquadFilterNode(context, {
  type: 'peaking',
  Q: Math.SQRT1_2,
  frequency: 1500
});

const trebleNode = new BiquadFilterNode(context, {
  type: 'highshelf',
  frequency: 3000
});

// Decreasing bass level by 10 dB.
bassNode.gain.value = -10;

// Increasing middle level by 15 dB.
midNode.gain.value = 15;

// No boost.
trebleNode.gain.value = 0;

To prevent click on gain value updates it is possible to use setTargetAtTime update method:

const level = 5;
const duration = 0.01;

midNode.gain.setTargetAtTime(level, context.currentTime, duration);

Now we can connect all the nodes to have a flexible guitar cabinet emulation:

guitarInput
  .connect(convolver)
  .connect(makeUpGain)
  .connect(bassNode)
  .connect(midNode)
  .connect(trebleNode)
  .connect(context.destination);

I've recorded an example video showing how convolution affects the output sound. It might not have a significant impact on a clean signal, but if some distortion effect applied, it is much more noticeable.

Recap

If you read that post to that point, you learned how to emulate a guitar amp using convolution, and create a three-band equalizer to configure tone. If you are playing guitar, you can check out the "JS Rocks" Angular app I've built. It has 9‼️ of ready-to-use cabinets and 7 effects for electric guitar created with Web Audio. Moreover, it sounds well 😎. Stay tuned in the next post I'm going to make a deep dive into distortion effects. Rock it with Web Audio 🤘!

Web Audio for Electric Guitar: How to Connect Instrument

Vitalii Bobrov — Tue, 11 Jun 2019 06:49:16 +0000

Modern Web becomes more and more powerful, especially with APIs giving developers access to hardware. One of such API - Web Audio. It gives you a set of low-level features to generate or process music right in your browser. Today I want to show you how to connect your electric guitar 🎸 (or any other electric instrument with wire connection, ex. bass) in a browser and control its volume level.

This post is a part of "Web Audio for Electric Guitar" series, check out other posts as well!

Web Audio for Electric Guitar:

How to Connect Instrument - Current
Cabinet Emulation

I am playing a bunch of instruments, including some very unusual, like bandura 🤯, but my favorite one is a guitar. At my teenage years, I've used to play at punk-rock bands 🤘 at school and university. I'm not a professional guitarist, and it is just my hobby that helps to relax and clean my mind after work. Web Audio allowed me to combine programming and music, so I've started experimenting with it. The very first task to do that was how to connect my instrument to the laptop.

Hardware

Before writing any code, you need a few things: a laptop, an instrument (electric guitar in my case) and an audio interface. The last part is crucial. Of course, you can plug your instrument directly to the audio input of your computer, but it has a bunch of downsides. First, it might require an adapter. Usually, laptops have only 3.5mm jack, but 6.4mm cables used for instruments. Second, the quality of built-in audio cards is not usually suitable for playing music - in most cases, producers think that users use it for music, movies, and games. You should expect a high latency of an audio stream. Moreover, the last argument in favor of functional interface, that build-in solutions might be damaged on hard input levels. I did that when I was young 😱.

Fortunately, you can buy a cheap and good-to-start device just for 20-25$. I'm using the most popular and quite cheap one - Behringer UCG-102 Guitar Link (this is not an advertisement!). It gives a low latency, has 6.4mm jack input and output, connects via USB and doesn't require any particular setup to work on Mac (on Windows you might need to install asio4all driver to achieve good results). You can see it among my devices on the photo below.

You can buy something more fancy and better, but this audio interface is good to start with, you can always upgrade it if you ever need it.

You need to connect your device to the computer, connect the instrument to the interface. Then you need to make sure that your system audio input and output setup is correct. You must select an external audio card as input and select built-in one as an output.

Note: if you want to use Bluetooth headphones to play, I'd like to recommend to use an only wired connection, at least when I used Marshall MID the latency was so huge that I couldn't play anything, they worked fine for me to listen to the music. It might be only my specific problem, but when trying to use Web Audio at first time try not to use wireless speakers or headphones, as they might add latency and make you think Web Audio API to be slow.

Web Audio Context

Before you request user audio input, you need to create a Web Audio context - the main point to create any nodes and work with APIs. Web Audio, in general, is a uni-directed graph of individual audio nodes. Connecting and changing their parameters, you could create effects pipeline or generate sound. That graph should have some input, in our case audio stream from the guitar signal.

To produce any noise or sound out, the input should be connected to the output. Usually, it is a context destination - the output device configured in the system. You can imagine it as a standard guitar stack. We have a guitar that could be connected to a line of stompboxes (guitar pedals), and at the end, you connect it to an amplifier or cabinet.

Sometimes, for ease of use, it is helpful to draw a graph before implementing any complex effect.

Let's create the audio context, to do so we need to use a constructor that returns context:

const context = new AudioContext();

The friend of mine Reactive Fox pointed me that you might have a problem with an audio context that created without any user interaction, ex. click. Browsers could set such context in a suspended state; you can read about it in details here. To prevent such issues, you need to make sure that the context is in the active state using the following code:

if (context.state === 'suspended') {
  await context.resume();
}

Side note: I've used TypeScript while working with Web Audio and it saved me much time of searching through the documentation. It is not required to use TypeScript, but I can say it might make your life way easier.

Request audio stream

After we created an audio context, we are ready to request user input with getUserMedia API. In the past, this method was located on navigator object, but the specification was changed, and now it is on navigator.mediaDevices. Keep this in mind if you're going to support legacy browsers.

const stream = await navigator.mediaDevices
  .getUserMedia({audio: true});

By default, browsers might apply sound optimization to the stream - echo cancellation, noise suppression, and auto gain control. Those optimizations are suitable for a microphone but not for a live instrument. To disable them, you need to pass an object with audio constraints instead of true:

const stream = await navigator.mediaDevices
  .getUserMedia({
    audio: {
      echoCancellation: false,
      autoGainControl: false,
      noiseSuppression: false,
      latency: 0
    }
  });

After the call of the method, the user will be asked for permission to allow audio input and output. You can request those permissions manually using Permissions API, but that feature is not fully supported right now. For now, you're allowed to check permission status and reset it.

// Getting permission status.
const micStatus = await navigator.permissions.query({name: 'microphone'});

console.log(micStatus); // state: "prompt"

// Reset permission to initial state.
await navigator.permissions.revoke({name: 'microphone'});

Note: getUserMedia user media requires your app to be hosted via secure connection - HTTPS. If your local or deployed app is running using HTTP, you might need to grant permissions using web site settings in a browser manually.

Ok, we requested an audio stream, what next? We need to create a media stream source and pass that stream to the audio context. However, to get any sound from the speakers, we must connect our source to destination node:

const lineInSource = context.createMediaStreamSource(stream);

lineInSource.connect(context.destination);

And 🥁... now you should hear sound from the guitar in your browser - congratulations 🎉! Let's put it all together:

const context = new AudioContext();

if (context.state === 'suspended') {
  await context.resume();
}

const stream = await navigator.mediaDevices
  .getUserMedia({
    audio: {
      echoCancellation: false,
      autoGainControl: false,
      noiseSuppression: false,
      latency: 0
    }
  });
const lineInSource = context.createMediaStreamSource(stream);

lineInSource.connect(context.destination);

To disconnect your guitar, you need to call disconnect method on your source node:

lineInSource.disconnect();

Volume control

The last basic thing I want to show in this post - volume control. Web Audio provides us with a gain node. It has only one parameter - gain. This parameter accepts any numeric value. The zero gain means muted sound, 1 means normal, the same volume level. You could use values bigger than 1 to amplify original sound; for example, the value of 2 will boost volume in two times. You can create a gain node using a factory method on audio context or using constructor. The main difference between those methods that constructor allows you to pass initial configuration parameters, at the same time, you create an instance. Let's see it in code:

// Create a gain node and set the initial value to 0.5
// that means that volume will be haft of the original.
const gainNode = new GainNode(context, {gain: 0.5});

// Disconnect source before constructing a new graph.
lineInSource.disconnect();

// Connect nodes
lineInSource.connect(gainNode).connect(context.destination);

// Increasing volume.
gainNode.gain.value = 2;

Few points to explain here, you can chain connect methods, as they are returning the node you connected to during the previous call. You can assign value to gain node, but you might notice clicking on value changes. Those "clicks" are artifacts of discrete gain changes. Fortunately, Web Audio nodes have a bunch of methods to set values smoothly:

// Setting target value (1st argument) starting from
// the current time in 0.01 second period
gainNode.gain.setTargetAtTime(2, context.currentTime, 0.01);

// Doing the same but exponentially.
gainNode.gain.exponentialRampToValueAtTime(gain, context.currentTime + 0.01);

Using those methods, you will avoid sound glitches. To update a value of the volume in a user interface you need some control, the most suitable one for that purpose is a range input:

<input
    class="gain-control"
    type="range"
    min="0"
    max="1"
    step="0.01"
    value="0.5">

You can listen to the changes in the input to update the gain value. Note, that you will need to validate (at least clamp) and parse the value, cause inputs value is always a string.

const control = document.querySelector('.gain-control');

control.addEventListener('change', (event) => {
  const parsed = parseFloat(event.target.value);
  const value = Number.isNaN(parsed) ? 1 : parsed;
  const clamped = clamp(value);

  gainNode.gain.setTargetAtTime(clamped, context.currentTime, 0.01);
});

function clamp(min, max, value) {
  return Math.min(Math.max(value, min), max);
}

Recap

If you read that post to that point, you learned how to create an audio context, request media stream from a browser, connect it to the output, and control its volume. If you are playing guitar, you can check out the "JS Rocks" Angular app I've built. It has a bunch of ready-to-use cabinets and effects for electric guitar created with Web Audio. Moreover, it sounds well 😎. Stay tuned in the next post I'm going to show how to emulate a real guitar cabinet. Rock it with Web Audio 🤘!

Exploring the CSS Paint API

Vitalii Bobrov — Mon, 10 Jun 2019 14:45:26 +0000

CSS Paint API is the first part of the Houdini project that is available in the stable version of the browser. It Google Chrome team added it to Chrome 65 on March 6th. That is why it is an excellent time to try it out and start experimenting. I want you to get started and start own experimenting with it.

What is Houdini?

Before we start the exploration of CSS Paint API, let me made a short intro to the Houdini project. I will not get into too many details but will provide you with links to resources to learn more if you want.

So Houdini is the set of APIs that allows you to interact with CSS engine internals. Unfortunately, until this time we are limited to works with some part of CSSOM (CSS Object Model) via JavaScript. So we can try polyfill CSS with JS but after browser renders stage. And after changes browser needs to perform rendering of the screen again. But with Houdini, we can extend styles in the same way we do with JS.

Houdini APIs are here to work with CSS parser, CSSOM, cascade, layout, paint, and composite rendering stages. There are two main groups of the API: CSS properties & values and worklets. And worklets cover renders states access: layout, paint, and composite. While properties and values focused on parser extension, work with CSSOM and cascade. You can check browsers implementation status for each API here and more information about it here.

CSS Paint Worklet

CSS Paint API is the kind of worklets and how you could understand the name it works with paint rendering process. What does it do? It allows you to create custom CSS function to draw an image as background with JavaScript. And then use this function for any CSS property that expects image. For example, you can use it for background-image, border-image or list-style-image. But more exciting that it also could be used for custom CSS property, we will come back to them later.

For drawing pictures with JavaScript, you allowed using the limited version of Canvas API. Why limited? For security reasons, you are not able to read pixels from an image or render text. But you can draw arcs, rectangles, paths, etc.

Why we need CSS Paint API?

There are a few use cases that I have in my mind for now:

CSS polyfills - of course, we could write a polyfill for CSS with JavaScript, but it is not a good idea in case of usability and performance. You can read some thoughts about that here. But CSS Paint is a good candidate for that, for example, take a look on conic-gradient polyfill example.
Reduce DOM nodes number - sometimes we need to add dummy DOM nodes, like span just for visuals. Also, some of the animations may require additional elements. Take a look at the painter that implements Material Design "ripple" animation. In original Material Design library, it creates two additional span elements for that animation and with worklet no need to do so. Now imagine you have ten buttons with "ripple" effect on the page, and CSS paint saves you twenty DOM nodes for that.
Fancy backgrounds - you can create some kind of new experience for end users with unusual patterns and backgrounds. And the good thing here that they will not affect performance and could be used as a part of progressive enhancement.

How to use CSS paint in styles?

To use custom paint in your stylesheets you need to use paint function, and pass your paint name, as well as any required arguments next. Here the example how it could look like:

div {
  background-image: paint(my-custom-paint);
}

In the case above we are using custom paint with name my-custom-paint, next let's imagine that it allows us to pass additional arguments inside, like color:

div {
  background-image: paint(my-custom-paint, #fff);
}

Looks similar too what we have with some CSS built-in functions, like linear-gradient:

div {
  background-image: linear-gradient(to bottom, #fff, #000);
}

As paint is just a value of CSS declaration it will be easy to fallback it for old browsers with solid color or image:

.paint-with-fallback {
  background-image: url('./my-paint-fallback.jpg');
  background-image: paint(my-custom-paint);
}

Or we can check browser support in CSS:

.paint-with-fallback {
  background-image: url('./my-paint-fallback.jpg');
}

@supports(background-image: paint(id)) {
  .paint-with-fallback {
    background-image: paint(my-custom-paint);
  }
}

In this case, browsers that don't support the CSS Paint API will ignore last background-image declaration and use some static image instead.

But if we will create some wide-used painter, it will be great to automate fallback insertion, as humans could forget about it. And I have great news for you, PostCSS could do it for us with a plugin. To write such a plugin, we don't need a lot of lines of code. PostCSS provide us with a bunch of handy helper tools to iterate through CSS AST (Abstract Syntax Tree) and manipulate it. Below is the example of such plugin that replaces custom paint with a static fallback value passed as a fallbackValue option:

const postcss = require('postcss');

module.exports = postcss.plugin(
  'postcss-fallback-my-paint',
  options => {
    return css => {
      css.walkRules(rule => {
        rule.walkDecls(decl => {
          const value = decl.value;

          if (value.includes('my-custom-paint')) {
            decl.cloneBefore({value: options.fallbackValue});
          }
        });
      });
    };
  }
);

This plugin will walk through all CSS rules and then all declarations inside them. The look for my-css-paint calls and insert clones declaration before with value replaced to fallback. Not the 🚀 science, isn't it?

How to create custom CSS paint?

So how to create custom CSS paint? It is just three steps:

Declare a custom paint class
Register paint
Load worklet

So, first of all, we want to declare CSS Paint class. It should be a JavaScript class with paint method. We will explore this method and its arguments later, for now just look at the underlying implementation:

class MyCustomPainter {
  paint(ctx, geom, props, args) {
    // paint implementation.
  }
}

After that we need to register the newly defined painter:

registerPaint('my-custom-paint', MyCustomPainter);

We were using registerPaint function and pass paint name as the first argument and our class reference as the second. Here I want to notice that our paint module file with class and registration call has a separate context. That means that we can't access any function or variable available in global browser scope or even load any dependency script.

Next, the last step is to load worklet, so after that, you will be able to use it in your stylesheets:

if ('paintWorklet' in CSS) {
  CSS.paintWorklet.addModule('my-custom-paint.js');
}

First, we are checking if paintWorklet available in the browser and then register our custom paint calling the only available method on CSS.paintWorklet called addModule. It accepts one parameter - path to our worklet JavaScript file. Here you can also opt-in with JavaScript-based fallback for CSS Paint with additional else statement.

Here how should look the final result:

// my-custom-paint.js

class MyCustomPainter {
  paint(ctx, geom, props, args) {
    // paint implementation.
  }
}

registerPaint('my-custom-paint', MyCustomPainter);

// script loaded on page - script.js

if ('paintWorklet' in CSS) {
  CSS.paintWorklet.addModule('my-custom-paint.js');
}

Practice

After the introduction to Paint API, the best idea is to try it. Let's start with the primary example - create painter that will draw few circles as background. To get started let's define a class for paint and register it:

// paint.js

class CirclesPainter {
  paint(ctx, geom) {
    const offset = 10;
    const size = Math.min(geom.width, geom.height);
    const radius = (size / 4) - offset;
    const point = radius + offset;

    for (let i = 0; i < 2; i++) {
      for (let j = 0; j < 2; j++) {
        ctx.fillStyle = `rgb(0, ${Math.floor(255 - 42.5 * i)}, ${Math.floor(255 - 42.5 * j)})`;

        ctx.beginPath();
        ctx.arc(point + (i * (point * 2)), point + (j * (point * 2)), radius, 0, 2 * Math.PI);
        ctx.fill();
      }
    }
  }
}

registerPaint('circles', CirclesPainter);

We created CirclesPainter class with the paint method. This method accepting two arguments: ctx which is our canvas context and geom object that consists of 2 properties. geom contains the width and height of our canvas surface. Then using our context, we draw four circles inside loops and fill them with some shades of blue. And finally we load our worklet on the page:

if ('paintWorklet' in CSS) {
  CSS.paintWorklet.addModule('paint.js');
}

To use it we created simple div with class name circles and added next rules to our stylesheet:

.circles {
  overflow: hidden;
  height: 0;
  padding-top: 50%;
  background: #000;
  background: paint(circles);
}

So we make it square and add black color as a fallback for old browsers. That is it! You can check result and code on GitHub. Here is the demo:

One thing I want to mention now, we haven't added any resize event listener, but browser calls paint method automatically on any layout changes. Current Chrome implementation uses main UI thread for paint rendering, but in the future, it will use a separate thread. You can imagine some heavy animations or backgrounds that have zero effect on the main thread. It will be an enormous performance boost!

Your backgrounds could be responsive, and this responsiveness depends on element size itself without any listeners on resize events. Until element queries are still the proposal, you can generate different picture depending on element size. Try out this example with source code. When the element changes its size, we fill our circles with another color.

All this nice, but next I want to make our paint configurable. So let me introduce a few CSS variables:

--circles-offset - to control the distance between circles
--circles-count - for the number of circles to render
--circles-opacity - to change circles opacity

With those variables, we can create some kind of pattern that could change over time. To access CSS variables in our painter class, we need to define the static property called inputProperties. It should be an Array of CSS properties and variables we want to access in the paint method. On every property from the array change browser will call render for us without any additional line of code. Below is the updated CirclesPainter:

class CirclesPainter {
  static get inputProperties() {
    return [
      '--circles-offset',
      '--circles-count',
      '--circles-opacity'
    ];
  }

  paint(ctx, geom, props) {
    const offset = parseInt(props.get('--circles-offset').toString(), 10) || 0;
    const count = parseInt(props.get('--circles-count').toString(), 10) || 2;
    const opacity = parseFloat(props.get('--circles-opacity').toString()) || 1;
    const size = Math.min(geom.width, geom.height);
    const radius = Math.max(Math.round(((size / count) - offset * 2) / 2), 10);
    const point = radius + offset;

    for (let i = 0; i < count; i++) {
      for (let j = 0; j < count; j++) {
        ctx.fillStyle = `rgba(0,
          ${Math.floor(255 - 42.5 * i)},
          ${Math.floor(255 - 42.5 * j)},
          ${opacity})`;

        ctx.beginPath();
        ctx.arc(point + (i * (point * 2)), point + (j * (point * 2)), radius, 0, 2 * Math.PI);
        ctx.fill();
      }
    }
  }
}

registerPaint('circles', CirclesPainter);

So we added the getter for inputProperties, and it returns the list of CSS variable we want to use. Then we get access to all properties we subscribed with props argument. It contains a CSS map of properties with values. We call get method with variable name to get its value as a string. Also, we define some defaults to have values even if they are not specified in styles. And we used this value to make circles rendering dynamic.

We should update styles with our variables:

.circles {
  --circles-count: 2;
  --circles-offset: 10;
  --circles-opacity: 1;
  overflow: hidden;
  height: 0;
  padding-top: 50%;
  background: #000;
  background: paint(circles);
}

Check code and result here.

For GitHub demos I added a simple script to connect CSS variables with input controls, you can find it here.

So now we can modify our rendering parameters in the runtime just updating variables with CSS or JavaScript. But if we look on built-in CSS functions like linear-gradient, they accepted to pass additional parameters to a function itself:

.gradient-bg {
  background: linear-gradient(to top, #fff, #000);
}

Could we also achieve the same behavior for custom paint? And the answer is - YES! For that, we need to use another Houdini API that still exists as an experiment in Chrome. It called CSS Typed OM, and it allows you to use built-in CSS engine types like colors, images, length, etc. And then we can pass them as arguments to the paint function.

As feature still experimental now, we need to enable "Experimental Web Platform features" flag in Chrome. Go to chrome://flags/#enable-experimental-web-platform-features and enable it.

After that, we should add a new static property to our paint class - inputArguments. Like inputProperties it subscribes to any changes to listed arguments but should contain an array of CSS types. Let replace CSS variable with arguments:

class CirclesPainter {
  static get inputArguments() {
    return [
      '<number>',    // offset
      '<number>',    // number of circles
      '<percentage>' // opacity
    ];
  }

  paint(ctx, geom, props, args) {
    const offset = args[0].value;
    const count = args[1].value;
    const opacity = args[2].value / 100;
    const size = Math.min(geom.width, geom.height);
    const radius = Math.max(Math.round(((size / count) - offset * 2) / 2), 10);
    const point = radius + offset;

    for (let i = 0; i < count; i++) {
      for (let j = 0; j < count; j++) {
        ctx.fillStyle = `rgba(0,
          ${Math.floor(255 - 42.5 * i)},
          ${Math.floor(255 - 42.5 * j)},
          ${opacity})`;

        ctx.beginPath();
        ctx.arc(point + (i * (point * 2)), point + (j * (point * 2)), radius, 0, 2 * Math.PI);
        ctx.fill();
      }
    }
  }
}

So our CirclesPainter now includes inputArguments list that contains three parameters: <number>, <number> and <percentage>. You might mention syntax for CSS types - <type name>. Also, you can accept types union similar way as it possible in TypeScript - <number | percentage>. You can find all the available types in "CSS Values and Units" specification draft.

Then in the paint method, we got args parameter that similar to JavaScript function arguments object, but it is just Array that contains CSSUnitValue objects. Each of unit objects consists of two properties value and unit. So in our example, we accessed all the value of each argument. And we should modify our CSS to use it:

.circles {
  overflow: hidden;
  height: 0;
  padding-top: 50%;
  background: #000;
  background: paint(circles, 2, 10, 100%);
}

Check out code and result here.

Animations

This was cool, but how we can create animations for a painter? As I told before, our worklets execute in a separate context, and there is no requestAnimationFrame or even setTimeout functions. How to implement animations? The first solution is to use CSS variables. Let's try to animate them with CSS:

.circles {
  --circles-count: 2;
  --circles-offset: 10;
  --circles-opacity: 1;
  overflow: hidden;
  height: 0;
  padding-top: 50%;
  background: #000;
  background: paint(circles);
}

.circles:hover {
  animation: opacify 0.3s;
}

@keyframes opacify {
  from {
    --circles-opacity: 1;
  }

  to {
    --circles-opacity: 0;
  }
}

And... this solution won't work as we expect. It just switches opacity from one to zero at 50% point. But why? And the answer is simple, all CSS variable are just strings, they are similar to variables in SASS or LESS - variables are replaced with values with simple string interpolation. To animate some CSS property browser need to apply interpolation function, but it doesn't know how to interpolate one string to another. It has the only built-in functionality to animate colors, length, numbers, but not for strings. That is why it just switch value at 50%. In this case, we can animate variables with JavaScript using requestAnimationFrame. Such a script could look like this:

const canvas = document.querySelector('.circles');
let start = performance.now();

canvas.addEventListener('mouseenter', event => {
  canvas.classList.add('animating');
  start = performance.now();

  requestAnimationFrame(function raf(now) {
    const count = Math.floor(now - start);
    const rawValue = canvas.style.getPropertyValue('--circles-opacity').trim();
    const value = (parseFloat(rawValue) * 100 - 3) / 100;

    canvas.style.setProperty('--circles-opacity', value);

    if(count > 300) {
      canvas.classList.remove('animating');
      canvas.style.setProperty('--circles-opacity', 0);

      return;
    }

    requestAnimationFrame(raf);
  });
});

Not so good, could we do it better? Yes, with Custom Properties API. This API also under the flag in Chrome now, so don't forget to enable it. It allows us to register custom CSS property with syntax similar to variables, but this time with CSS type assigned to it. So browser will have an idea about how to animate it, and we can use CSS animations and transition for that!

So to register custom property, we need to call CSS.registerProperty and pass options object:

CSS.registerProperty({
  name: '--circles-opacity',
  syntax: '<percentage>',
  inherits: false,
  initialValue: '100%'
});

As you can see we need to give the property the name with name option. Then we specify its type with syntax property, and besides, we say that it won't be inherited by children nodes and initial value as 100%.

After that we can use our newly created custom property in the stylesheet:

.circles {
  --circles-count: 2;
  --circles-offset: 10;
  --circles-opacity: 100%;
  overflow: hidden;
  height: 0;
  padding-top: 50%;
  background: #000;
  background: paint(circles);
  transition: --circles-opacity 0.3s ease;
}

.circles:hover {
  --circles-opacity: 0%;
}

Now we can use a transition to change circles opacity smoothly. Check out code and result here.

Conclusion

Today to just got started with CSS Paint API exploring how to create own one, how to use input properties and arguments, CSS variables and custom properties, and how to animate it. In the next article, I'm going to implement more production ready examples using the knowledge we got with that article. If you are reading this article using latest Chrome you might mention that I am using custom paint to make Material Design background, you can check it here and take a look at the code. Try to experiment with CSS Paint API yourself!

DEV Community: Vitalii Bobrov

Web Audio for Electric Guitar: Cabinet Emulation

Web Audio for Electric Guitar Series:

Impulse Response

Convolver Node

Three-band Equalizer

Recap

Web Audio for Electric Guitar: How to Connect Instrument

Web Audio for Electric Guitar:

Hardware

Web Audio Context

Request audio stream

Volume control

Recap

Exploring the CSS Paint API

What is Houdini?

CSS Paint Worklet

Why we need CSS Paint API?

How to use CSS paint in styles?

How to create custom CSS paint?

Practice

Animations

Conclusion

Resources