【You don't know about canvas】Replace the green screen video background

wangzc — Thu, 13 May 2021 08:21:13 +0000

canvas Api Brief introduction

The ImageData object represents the underlying pixel data of an area of a canvas object. It contains the following read-only attributes:：

width：The width of the image in pixels.
height：The height of the image in pixels.
data：A Uint8ClampedArrayrepresenting a one-dimensional array containing the data in the RGBA order, with integer values between 0 and 255 (included).

The data property returns aUint8ClampedArray which can be accessed to look at the raw pixel data; each pixel is represented by four one-byte values (red, green, blue, and alpha, in that order; that is, "RGBA" format). Each color component is represented by an integer between 0 and 255. Each component is assigned a consecutive index within the array, with the top left pixel's red component being at index 0 within the array. Pixels then proceed from left to right, then downward, throughout the array.

In short, we need to group every four pixels to get the rgba value of each pixel

Then we combine the features of 'Canvas' to operate the video to make green screen matting and change the background

First, the renderings：

Code address：canvas-viideo
preview address：githubPage、阿里云

Realization idea
Video '= = >'video screenshot' = = >'process green pixels as transparent '= = >'map to the top of the background image
Take a screenshot of the video, and then make the green pixel block transparent
Then put the processed image above the prepared background image

Implementation

1. Prepare videos and canvases

<body onload="processor.doLoad()">
    <div>
      <video id="video" src="./q.mp4" width="350" controls="true"></video>
    </div>
    <div>
      <!-- Video capture -->
      <canvas id="c1" width="260" height="190"></canvas>
      <!-- Processing green pixels as transparent -->
      <canvas id="c2" width="260" height="190"></canvas>
      <!-- Map to top of background map -->
      <canvas id="c3" width="260" height="190"></canvas>
    </div>
</body>

2.Add video playback monitor

doLoad: function doLoad() {
  this.video = document.getElementById("video");
  this.c1 = document.getElementById("c1");
  this.ctx1 = this.c1.getContext("2d");
  this.c2 = document.getElementById("c2");
  this.ctx2 = this.c2.getContext("2d");
  this.c3 = document.getElementById("c3");
  this.ctx3 = this.c3.getContext("2d");
  let self = this;
  this.video.addEventListener(
    "play",
    function() {
      self.width = self.video.videoWidth / 5;
      self.height = self.video.videoHeight / 3;
      self.timerCallback();
    },
    false
  );
}

3.Add timer

Call after video playback, and capture screenshot of each frame

timerCallback: function timerCallback() {
  if (this.video.paused || this.video.ended) {
    return;
  }
  this.computeFrame();
  let self = this;
  setTimeout(function () {
    self.timerCallback();
  }, 0);
}

4.Video frame operation

Set the green background to transparent and map it to the custom background image

computeFrame: function computeFrame() {
  this.ctx1.drawImage(this.video, 0, 0, this.width, this.height);
  let frame = this.ctx1.getImageData(0, 0, this.width, this.height);
  let l = frame.data.length / 4;

  for (let i = 0; i < l; i++) {
    let r = frame.data[i * 4 + 0];
    let g = frame.data[i * 4 + 1];
    let b = frame.data[i * 4 + 2];
    //rgb(8 204 4)
    if (r > 4 && g > 100 && b < 100) {
      frame.data[i * 4 + 3] = 0;
    }
  }
  this.ctx2.putImageData(frame, 0, 0);
  this.ctx3.putImageData(frame, 0, 0);
  return;
}

5.Fine tuning

    //rgb(8 204 4)
    The green video color is not pure, not always RGB (82004), so a simple fine-tuning has been made..
    if (r > 4 && g > 100 && b < 100) {
      frame.data[i * 4 + 3] = 0;
    }

END

Thank you for reading
My English is not good. I want to share this good technology with you，This is my Chinese blog address:segmentfault

Code address：canvas-viideo
preview addrss：阿里云、githubPage
green video download：pixabay

DEV Community: wangzc