DEV Community

Xiao Ling
Xiao Ling

Posted on • Edited on • Originally published at dynamsoft.com

How to Detect and Rectify Documents in Angular Web Applications

#webdev #angular #programming #typescript

Dynamsoft Document Normalizer JavaScript SDK features document detection and rectification. It can be easily integrated into web applications with a few lines of code. In this article, we will create an Angular web application to demonstrate how to use the SDK to detect and rectify documents.

Prerequisites

  • Angular CLI: A command-line interface tool for Angular development. 
  npm install -g @angular/cli
Enter fullscreen mode Exit fullscreen mode

Initialize Angular Project and Install Dependencies

In your terminal, run the following command to create a new Angular project.

ng new angular-document-edge-detection
Enter fullscreen mode Exit fullscreen mode

Then, navigate to the project folder and install the dependencies.

  • Dynamsoft Document Normalizer: A JavaScript SDK for document detection and rectification. 
  npm i dynamsoft-document-normalizer
Enter fullscreen mode Exit fullscreen mode
  • Dynamsoft Camera Enhancer: A JavaScript SDK for camera control, image capture and video streaming. 
  npm i dynamsoft-camera-enhancer
Enter fullscreen mode Exit fullscreen mode

Next, open angular.json file to configure the asset path for Dynamsoft Document Normalizer:

"assets": [
  "src/favicon.ico",
  "src/assets",
  {
    "glob": "**/*",
    "input": "./node_modules/dynamsoft-document-normalizer/dist",
    "output": "assets/dynamsoft-document-normalizer"
  }
],
Enter fullscreen mode Exit fullscreen mode

The output path should be the same as the one set in code. We create a dynamsoft.service.ts file to set the license key and the asset path:

import { Injectable, Optional } from '@angular/core';
import { DocumentNormalizer} from 'dynamsoft-document-normalizer';

@Injectable({
  providedIn: 'root'
})
export class DynamsoftService {

  constructor() {
    DocumentNormalizer.license = "DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ==";
    DocumentNormalizer.engineResourcePath = "assets/dynamsoft-document-normalizer";
  }
}
Enter fullscreen mode Exit fullscreen mode

Algorithm Parameters for Document Detection and Rectification

Dynamsoft Document Normalizer provides a set of parameters to control the detection and rectification process. You can refer to the online documentation for more details.

Here we create some simple parameter templates with TypeScript multiline strings:

export class Template {
    static binary = `
      {
          "GlobalParameter":{
              "Name":"GP"
          },
          "ImageParameterArray":[
              {
                  "Name":"IP-1",
                  "NormalizerParameterName":"NP-1",
                  "BinarizationModes":[{"Mode":"BM_LOCAL_BLOCK", "ThresholdCompensation":9}],
                  "ScaleDownThreshold":2300
              }
          ],
          "NormalizerParameterArray":[
              {
                  "Name":"NP-1",
                  "ColourMode": "ICM_BINARY" 
              }
          ]
      }
      `;

    static color = `
      {
          "GlobalParameter":{
              "Name":"GP"
          },
          "ImageParameterArray":[
              {
                  "Name":"IP-1",
                  "NormalizerParameterName":"NP-1",
                  "BinarizationModes":[{"Mode":"BM_LOCAL_BLOCK", "ThresholdCompensation":9}],
                  "ScaleDownThreshold":2300
              }
          ],
          "NormalizerParameterArray":[
              {
                  "Name":"NP-1",
                  "ColourMode": "ICM_COLOUR" 
              }
          ]
      }
      `;

    static grayscale = `
      {
          "GlobalParameter":{
              "Name":"GP"
          },
          "ImageParameterArray":[
              {
                  "Name":"IP-1",
                  "NormalizerParameterName":"NP-1",
                  "BinarizationModes":[{"Mode":"BM_LOCAL_BLOCK", "ThresholdCompensation":9}],
                  "ScaleDownThreshold":2300
              }
          ],
          "NormalizerParameterArray":[
              {
                  "Name":"NP-1",
                  "ColourMode": "ICM_GRAYSCALE"
              }
          ]
      }
      `;
}
Enter fullscreen mode Exit fullscreen mode

The only difference between these three templates is the ColourMode parameter. The ColourMode parameter controls the color mode of the output image. It can be set to ICM_BINARY, ICM_COLOUR or ICM_GRAYSCALE.

Angular Components for Detecting and Rectifying Documents

We are going to create two Angular components: file and camera. The file component is responsible for detecting and rectifying documents from image files. The camera component is responsible for detecting and rectifying documents from camera stream.

ng generate component file-detection
ng generate component camera-detection
Enter fullscreen mode Exit fullscreen mode

File Detection Component

The file detection page consists of five parts: a radio button group, a file input button, an image element, a canvas and a div element.

<span id="loading-status" style="font-size:x-large" [hidden]="isLoaded">Loading Library...</span>
<br />

<div class="row">

    <label for="binary"> <input type="radio" name="templates" value="binary" (change)="onRadioChange($event)" />Black &
        White </label>


    <label for="grayscale"><input type="radio" name="templates" value="grayscale" (change)="onRadioChange($event)" />
        Grayscale </label>

    <label for="color"><input type="radio" name="templates" value="color" [checked]="true"
            (change)="onRadioChange($event)" /> Color </label>
</div>

<input type="file" title="file" id="file" accept="image/*" (change)="onChange($event)" />

<div class="container">
    <div id="imageview">
        <img id="image" alt="" />
        <canvas id="overlay"></canvas>
    </div>

    <div id="resultview">
        <canvas id="normalizedImage"></canvas>
    </div>

</div>
Enter fullscreen mode Exit fullscreen mode

In TypeScript, we first inject the DynamsoftService in the constructor and initialize the Dynamsoft Document Normalizer object in the ngOnInit method.

import { Component, OnInit } from '@angular/core';
import { DocumentNormalizer } from 'dynamsoft-document-normalizer';
import { DynamsoftService } from '../dynamsoft.service';
import { OverlayManager } from '../overlay';
import { Template } from '../template';

@Component({
  selector: 'app-file-detection',
  templateUrl: './file-detection.component.html',
  styleUrls: ['./file-detection.component.css']
})
export class FileDetectionComponent implements OnInit {
  isLoaded = false;
  overlay: HTMLCanvasElement | undefined;
  context: CanvasRenderingContext2D | undefined;
  normalizer: DocumentNormalizer | undefined;
  overlayManager: OverlayManager;
  points: any[] = [];
  currentFile: File | undefined;

  constructor(private dynamsoftService: DynamsoftService) {
    this.overlayManager = new OverlayManager();
  }

  ngOnInit(): void {
    this.overlayManager.initOverlay(document.getElementById('overlay') as HTMLCanvasElement);
    (async () => {
      this.normalizer = await DocumentNormalizer.createInstance();
      this.isLoaded = true;
      await this.normalizer.setRuntimeSettings(Template.color);
    })();
  }
}
Enter fullscreen mode Exit fullscreen mode

Then add implementations for the UI elements.

  • The radio button group is used to switch the parameter template.

    onRadioChange(event: Event) {
  if (!this.normalizer) {
    return;
  }

  let target = event.target as HTMLInputElement;
  let template = Template.binary;
  if (target.value === 'binary') {
    template = Template.binary;
  } else if (target.value === 'grayscale') {
    template = Template.grayscale;
  } else if (target.value === 'color') {
    template = Template.color;
  }
  (async () => {
    await this.normalizer!.setRuntimeSettings(template);
    this.normalize(this.currentFile!, this.points);
  })();
}

  • The file input button is used to select the image file.

    onChange(event: Event) {
  const element = event.currentTarget as HTMLInputElement;
  let fileList: FileList | null = element.files;
  if (fileList) {
    let file = fileList.item(0) as any;

  }
}

  • The image element is used to display the loaded image file.

    if (file) {
  this.currentFile = file;
  let fr = new FileReader();
  fr.onload = (event: any) => {
    let image = document.getElementById('image') as HTMLImageElement;
    if (image) {
      image.src = event.target.result;
      const img = new Image();

      img.onload = (event: any) => {

      };
      img.src = event.target.result;
    }
  };
  fr.readAsDataURL(file);
}

    As the image is loaded， invoke the detectQuad method to detect the document edges.

    this.normalizer.detectQuad(file).then((results: any) => {
  try {
    if (results.length > 0) {

    } 
  } catch (e) {
    alert(e);
  }
});

  • The canvas is used to display the detected document edges.

    try {
  if (results.length > 0) {
    let result = results[0];
      this.points = result['location']['points'];
      this.overlayManager.drawOverlay(
        this.points,
      );
  } 
} catch (e) {
  alert(e);
}

  • The div element is used to display the rectified document.

    this.normalize(file, this.points);

normalize(file: File, points: any) {
  if (this.normalizer) {
    this.normalizer.normalize(file, points).then((result: any) => {
      let image = document.getElementById('normalizedImage') as HTMLCanvasElement;
      if (image) {
        image.width = result.image.width;
        image.height = result.image.height;
        let ctx = image.getContext('2d') as CanvasRenderingContext2D;

        var imgdata = ctx.createImageData(image.width, image.height);
        var imgdatalen = result.image.data.length;
        for(var i=0; i<imgdatalen; i++) {
            imgdata.data[i] = result.image.data[i];
        }
        ctx.putImageData(imgdata, 0, 0);
      }
    });
  }
}

Angular Document Edge Detection from image file

Camera Detection Component

The UI of the camera detection component is similar to the file detection component. The difference is the image element is replaced by the video element.

<div id="document-scanner">
    <span id="loading-status" style="font-size:x-large" [hidden]="isLoaded">Loading Library...</span>
    <br />
    <div class="row">

        <label for="binary"> <input type="radio" name="templates" value="binary" (change)="onRadioChange($event)" />Black &
            White </label>


        <label for="grayscale"><input type="radio" name="templates" value="grayscale" (change)="onRadioChange($event)" />
            Grayscale </label>

        <label for="color"><input type="radio" name="templates" value="color" [checked]="true"
                (change)="onRadioChange($event)" /> Color </label>
    </div>
    <label for="threshold"><input id="thredshold" (change)="updateThresholdCompensation($event)" type="range" min="0" max="10" value="9" step="1">Edge Detection Threshold: <span id="ThresholdCompensationval" style="width: 12px;" [textContent]="9"></span></label>
    <div>
        <label for="videoSource">Video Source: 
        <select id="videoSource" (change)="openCamera()"></select></label>
        <button id="detectButton" (click)="detectDocument()">Start Detection</button>
        <button id="captureButton" (click)="captureDocument()">Capture Document</button>
    </div>

    <div id="videoview">
        <div class="dce-video-container" id="videoContainer"></div>
        <canvas id="overlay"></canvas>
    </div>

    <div class="container">
        <div id="resultview">
            <canvas id="normalizedImage"></canvas>
        </div>
    </div>
</div>
Enter fullscreen mode Exit fullscreen mode

It is known that the getUserMedia method is the only API that can access the camera stream. With the getUserMedia method, we still have lots of work to do in camera programming. To simplify the coding, we use Dynamsoft Camera Enhancer instead. The JavaScript camera SDK encapsulates the getUserMedia method and provides some useful APIs.

Here is the initialization code for the camera detection component.

import { Component, OnInit } from '@angular/core';
import { DocumentNormalizer } from 'dynamsoft-document-normalizer';
import { CameraEnhancer } from 'dynamsoft-camera-enhancer';
import { DynamsoftService } from '../dynamsoft.service';
import { OverlayManager } from '../overlay';
import { Template } from '../template';

@Component({
  selector: 'app-camera-detection',
  templateUrl: './camera-detection.component.html',
  styleUrls: ['./camera-detection.component.css']
})
export class CameraDetectionComponent implements OnInit {
  isLoaded = false;
  overlay: HTMLCanvasElement | undefined;
  context: CanvasRenderingContext2D | undefined;
  normalizer: DocumentNormalizer | undefined;
  overlayManager: OverlayManager;
  currentData: any;
  cameraInfo: any = {};
  videoSelect: HTMLSelectElement | undefined;
  enhancer: CameraEnhancer | undefined;
  isDetecting = false;
  captured: any[] = [];

  constructor(private dynamsoftService: DynamsoftService) {
    this.overlayManager = new OverlayManager();
  }

  ngOnDestroy() {
    this.normalizer?.dispose();
    this.normalizer = undefined;

    this.enhancer?.dispose(true);
    this.enhancer = undefined;
  }

  ngOnInit(): void {
    this.videoSelect = document.querySelector('select#videoSource') as HTMLSelectElement;
    this.overlayManager.initOverlay(document.getElementById('overlay') as HTMLCanvasElement);
    (async () => {
      this.normalizer = await DocumentNormalizer.createInstance();
      this.enhancer = await CameraEnhancer.createInstance();
      this.enhancer.on("cameraOpen", (playCallBackInfo: any) => {
        this.overlayManager.updateOverlay(playCallBackInfo.width, playCallBackInfo.height);
      });
      this.enhancer.on("cameraClose", (playCallBackInfo: any) => {
        console.log(playCallBackInfo.deviceId);
      });


      this.isLoaded = true;
      await this.normalizer.setRuntimeSettings(Template.color);
    })();
  }
}
Enter fullscreen mode Exit fullscreen mode

After instantiating the Dynamsoft Camera Enhancer, we need to bind it to a div element for displaying the camera stream:

let uiElement = document.getElementById('videoContainer');
if (uiElement) {
  await this.enhancer.setUIElement(uiElement);
}
Enter fullscreen mode Exit fullscreen mode

If you have multiple cameras, you can call the getAllCameras method to get the camera list:

let cameras = await this.enhancer.getAllCameras();
this.listCameras(cameras);
Enter fullscreen mode Exit fullscreen mode

Then select a camera and open it:

async openCamera(): Promise<void> {
  if (this.videoSelect) {
    let deviceId = this.videoSelect.value;
    if (this.enhancer) {
      await this.enhancer.selectCamera(this.cameraInfo[deviceId]);
      await this.enhancer.open()
    }
  }

}
Enter fullscreen mode Exit fullscreen mode

The detectQuad method supports different input types. In the above section, we used the detectQuad method to detect the document from an image file. Now we use it to detect the document from frames of the camera stream.

detect(): void {
  if (this.normalizer && this.enhancer && this.isDetecting) {

    let data = this.enhancer.getFrame().toCanvas();
    this.normalizer.detectQuad(data).then((results: any) => {
      this.overlayManager.clearOverlay();
      if (!this.isDetecting) return;
      try {
        if (results.length > 0) {
          if (this.captured.length > 0) {
            this.captured.pop();
          }

          let result = results[0];
          let points = result['location']['points'];
          this.captured.push({ 'image': data, 'points': points });
          this.overlayManager.drawOverlay(
            points,
          );
        }
      } catch (e) {
        alert(e);
      }
      this.detect();
    });
  }
}
Enter fullscreen mode Exit fullscreen mode

The document detection job runs in a web worker. Thus, don't call it continuously. You must wait for the previous detection job to finish before calling it again.

Run ng serve to have fun with the web document detection and recification app.

Angular Document Edge Detection from camera stream

Source Code

https://github.com/yushulx/angular-barcode-mrz-document-scanner

Top comments (0)

Read next

gauravadhikari1997 profile image

How to customize Strapi Dashboard / Home Page: The Right Way

Gaurav Adhikari -

peterdanwan profile image

Coming Full Circle: It starts and ends with Git and GitHub

Peter Wan -

jeferson0993 profile image

Why Monorepo Projects Sucks: Performance Considerations with Nx

Jeferson F Silva -

alpira profile image

🚀 Introducing Odysseus-CLI: Simplified Deployment for Laravel, React and other Applications

alpira -