Optimizing graphics when working with Joint JS

#javascript #showdev #webdev #typescript

What do I do if my Voximplant Kit scenario is pretty neat, but has so many elements it becomes slow? That is the question our developers asked themselves and answered with an idea to achieve optimization.

Continuing the series of articles about Kit and Joint JS, I’m going to tell you how optimization made large scenarios super fast and what problems our dev team encountered while creating this feature.

Why optimize

Many of us are familiar with the performance issue caused by a large number of elements on the page. What does it mean? In our case, the more elements in the Voximplant Kit scenario, the more they affect the rendering speed of the elements on the canvas when we drag them across it, as well as the speed of scaling and rendering the canvas itself.

We wanted to use the will-change CSS property that allows informing the browser about changes to be applied to a certain element. It means one can set optimization up beforehand to avoid launching the operations that negatively affect the responsiveness of the page.

However, in the JointJS library that we use in Voximplant Kit, transformations related to dragging and scaling the canvas are applied to the child group of the SVG element, not to the entire canvas. We didn’t manage to shift the calculations to the video card; browsers just ignored this property and redrew all the group elements with each movement, which caused a delay.

<svg ... > <!-- Canvas -->
  <g transform="matrix(1,0,0,1,224,444)"> <!-- Group of elements inside svg -->
    <rect>
    <rect>

Implementation

Our developers decided that if they wrap the SVG element in a div element they would be able to apply all transformations first to the div element, and only if necessary, to the SVG element with the canvas. Once the transformations started to apply to the div, the time came to use will-change:transform to keep track of them:

<div> <!-- div-wrapper to which we apply optimization and transformation -->
  <svg> <!-- Canvas -->
    <g> <!-- Group of elements inside svg -->
      <rect>
      <rect>

But here we face another challenge – the use of will-change initiates the creation of an additional layer. And the greater the width and height of the element to which we apply this property, the more RAM we need to store this layer. We solved this problem by reducing the scale of SVG by 10 times. For example, when the scale of the canvas is 200%, the will-change layer requires 300 megabytes of RAM. After scaling down, about 3 megabytes are required.

To achieve that, we set zoom = 0.1 and shrink the SVG canvas using the transformToCenterViewport method. Then we apply the same transformations to the div element:

if (isPerfMode) {
  this.el.classList.add('perf-mode');
  // Change scaling before enabling the performance mode
  const prevScale = this._viewportMatrix.a;
  const point = this.getViewPortCenter();
  const zoom = 0.1;
  // Shrink the original svg so that will-change uses less RAM
  this.transformToCenterViewport(point, zoom, true, false, true);
  this.initScale = this._viewportMatrix.a;
  this.createMatrix();
  this.isPerfMode = true;
  // Apply transformations to the wrapper-element
  this.startPerformance();
  this.transformToCenterViewport(point, prevScale, false, false, true);
}

When enabling optimization mode, we shrink the SVG and the canvas gets really small and hard to work with. To fix it we'll apply inverse scaling directly to the div element:

public startPerformance(force = false) {
  ...
  this.isPerformance = true;
  // Get the size of the area with blocks and the indent from the left corner of the viewport
  const { x, y, width, height } = this.layers.getBBox();
  const initScale = this.initScale;
  // Width and height for the wrapper and the x- and y-axis offsets for the area with blocks
  const wrapW = Math.floor(width * initScale) + 2;
  const wrapH = Math.floor(height * initScale) + 2;
  const layerX = -x * initScale;
  const layerY = -y * initScale;
  // this.wrapMatrix - div-element matrix containing the canvas
  this.wrapMatrix.e = +(this._viewportMatrix.e + x * this._viewportMatrix.a);
  this.wrapMatrix.f = +(this._viewportMatrix.f + y * this._viewportMatrix.d);
  this.svgWrapper.style.width = wrapW + 'px';
  this.svgWrapper.style.height = wrapH + 'px';
  this.svgWrapper.style.transform = this.wrapMatrix.toString();
  this.svgWrapper.style.willChange = 'transform';
  this.layers.style.transform = `matrix(${initScale},0,0,${initScale},${layerX} ,${layerY} )`;
}

We solved the speed problem, but it didn’t end there: when scaling down the canvas, the image details started to disappear. So when scaling up, it became blurry. We found the solution with the help of the article about re-rastering composited layers on scale change.

After scaling is stopped (the scroll event), we remove the will-change property for 0.1 seconds and then set it again. This causes the browser to re-raster the layer and return the missing image details:

// Add a 3d transformation so that the layer is not deleted
this.svgWrapper.style.transform = this.wrapMatrix.toString() + ' translateZ(0)';
this.transformFrameId = requestAnimationFrame(() => {
  // Set the will-change property to apply in the next frame
  this.svgWrapper.style.willChange = '';
  this.transformFrameId = requestAnimationFrame(() => {
    this.svgWrapper.style.willChange = 'transform';
    this.svgWrapper.style.transform = this.wrapMatrix.toString();
  });
});

The last thing we should do is always display the moved block on top of other blocks. In JointJS, there are toFront and toBack methods (analog of z-index in HTML) to move blocks and links along the Z-axis. But they have to sort elements and redraw blocks and links, thus causing delays.

Our developers came up with the following: the block we interact with is temporarily put at the end of the SVG element tree (the element with the highest z-index is at the end) on the mousedown event, and then returns to its previous position on the mouseup event.

How it works

One can test optimization mode in all Chromium-based browsers (Chrome, Opera, Edge, Yandex Browser, etc.) and the Safari browser. If scenarios contain 50 blocks or more, the optimization mode is enabled automatically. You can enable or disable it yourself by switching to the editor settings in the upper right corner:

Once you enable or disable the optimization mode, you’ll see a notification at the top of the scenario window:

Check the GIFs below to see the difference between scenarios with the optimization mode on and off. But since it's always more interesting to try it yourself, feel free to go to your Voximplant Kit scenario or, if you don't have an account yet – to the registration page.

Working with the canvas and its elements when the optimization is off looks more or less like this (computers with different characteristics may show different results):

We enable optimization and voila!

This is how we made the canvas moving and scaling smoother and faster, as well as increased the rendering speed of dragging and dropping blocks with links.

I hope you liked this article! We will continue to improve the product, so be ready for me sharing more updates and tricks with you! :)