DEV Community

Xiao Ling
Xiao Ling

Posted on • Originally published at dynamsoft.com

How to Build a Razor Class Library for Document Detection and Rectification

#dotnet #blazor #csharp #webdev

Document detection and rectification are crucial in document management, with widespread applications across various industries, including banking, insurance, and healthcare. In a previous project, we developed a Blazor document rectification app using both C# and JavaScript, which can be found here: https://github.com/yushulx/dotnet-blazor-document-rectification. To further streamline the development process using only C#, this article will guide you through creating a Razor Class Library. This library will be based on the Dynamsoft Document Normalizer SDK.

Online Document Detection Demo

https://yushulx.me/Razor-Document-Library/

NuGet Package

https://www.nuget.org/packages/RazorDocumentLibrary

Download the JavaScript Version of Dynamsoft Document Normalizer

The Dynamsoft Document Normalizer is designed to detect quadrilateral objects in images and supports various document types, such as passports, driver's licenses, and ID cards. The JavaScript version of the SDK can be downloaded from npm using the following command:

npm i dynamsoft-document-normalizer@1.0.12
Enter fullscreen mode Exit fullscreen mode

Note: We opted for version 1.0.12 instead of the latest version 2.0.11 due to changes in SDK architecture in version 2.x. The version 2.x divides the original package into several smaller packages. Despite these structural differences, the underlying algorithm remains unchanged. We chose the 1.x package for its simplicity in integrating with the Razor Class Library

After downloading the npm package, the SDK files are located in the node_modules/dynamsoft-document-normalizer/dist folder. The essential files for constructing the Razor Document Library include:

  • core.js
  • core.wasm
  • ddn.js
  • ddn.wasm
  • ddn_wasm_glue.js
  • ddn-1.0.12.browser.worker.js
  • dls.license.dialog.html
  • image-process.wasm
  • intermediate-result.wasm

Starting a Razor Class Library Project for Document Detection and Rectification

  1. Open Visual Studio and create a new Razor Class Library project. Name the project RazorDocumentLibrary.
  2. Copy the files listed above (core.js, core.wasm, etc.) into the wwwroot folder of your new project.
  3. Within the wwwroot folder, create a new file named documentJsInterop.js. This file will be used to store JavaScript functions that are callable from C#.

Integrating the JavaScript Document Normalizer SDK

  1. Open the documentJsInterop.js file and insert the necessary code to load the Dynamsoft Document Normalizer library.

    export function init() {
    return new Promise((resolve, reject) => {
        let script = document.createElement('script');
        script.type = 'text/javascript';
        script.src = '_content/RazorDocumentLibrary/ddn.js';
        script.onload = async () => {
            resolve();
        };
        script.onerror = () => {
            reject();
        };
        document.head.appendChild(script);
    });
}

  2. In the root folder of your project, create a file named DocumentJsInterop.cs. Then, add the following C# code to this file.

    using Microsoft.JSInterop;

namespace RazorDocumentLibrary
{
    public class DocumentJsInterop : IAsyncDisposable
    {
        private readonly Lazy<Task<IJSObjectReference>> moduleTask;

        public DocumentJsInterop(IJSRuntime jsRuntime)
        {
            moduleTask = new(() => jsRuntime.InvokeAsync<IJSObjectReference>(
                "import", "./_content/RazorDocumentLibrary/documentJsInterop.js").AsTask());
        }

        public async ValueTask DisposeAsync()
        {
            if (moduleTask.IsValueCreated)
            {
                var module = await moduleTask.Value;
                await module.DisposeAsync();
            }
        }

        public async Task LoadJS()
        {
            var module = await moduleTask.Value;
            await module.InvokeAsync<object>("init");
        }
    }
}

    Note:

    • LoadJS(): The method is designed to call the init() function in the JavaScript file.

Initializing the Dynamsoft Document Normalizer

The JavaScript SDK is comprised of wasm and js files, and it requires a license key. To initialize the SDK, follow these steps:

  1. Set the the license key provided by Dynamsoft. This key is essential for activating the SDK's features.

    JavaScript

    // documentJsInterop.js
export function setLicense(license) {
    if (!Dynamsoft) return;
    try {
        Dynamsoft.DDN.DocumentNormalizer.license = license;
    }
    catch (ex) {
        console.error(ex);
    }
}

    C#

    // DocumentJsInterop.cs
public async Task SetLicense(string license)
{
    var module = await moduleTask.Value;
    await module.InvokeVoidAsync("setLicense", license);
}

  2. Load the WebAssembly module. This step is crucial as it prepares the WebAssembly environment necessary for the SDK.

    JavaScript

    // documentJsInterop.js
export async function loadWasm() {
    if (!Dynamsoft) return;
    try {
        await Dynamsoft.DDN.DocumentNormalizer.loadWasm();
    }
    catch (ex) {
        console.error(ex);
    }
}

    C#

    // DocumentJsInterop.cs
public async Task LoadWasm()
{
    var module = await moduleTask.Value;
    await module.InvokeVoidAsync("loadWasm");
}

  3. Once the wasm module is loaded, instantiate the Dynamsoft Document Normalizer object.

    JavaScript

    // documentJsInterop.js
export async function createDocumentNormalizer() {
    if (!Dynamsoft) return;

    try {
        let normalizer = await Dynamsoft.DDN.DocumentNormalizer.createInstance();
        return normalizer;
    }
    catch (ex) {
        console.error(ex);
    }
    return null;
}

    C#

    // DocumentJsInterop.cs
public async Task<DocumentNormalizer> CreateDocumentNormalizer()
{
    var module = await moduleTask.Value;
    IJSObjectReference jsObjectReference = await module.InvokeAsync<IJSObjectReference>("createDocumentNormalizer");
    DocumentNormalizer recognizer = new DocumentNormalizer(module, jsObjectReference);
    return recognizer;
}

    The DocumentNormalizer class is defined as follows:

    using Microsoft.JSInterop;
using System.Text.Json;

namespace RazorDocumentLibrary
{
    public class DocumentNormalizer
    {
        private IJSObjectReference _module;
        private IJSObjectReference _jsObjectReference;
        private DotNetObjectReference<DocumentNormalizer> objRef;

        public DocumentNormalizer(IJSObjectReference module, IJSObjectReference normalizer)
        {
            _module = module;
            _jsObjectReference = normalizer;
            objRef = DotNetObjectReference.Create(this);
        }
    }
}

Implementing Document Edge Detection

To enable document edge detection in your Razor Class Library, you need to update the documentJsInterop.js file and the corresponding C# file.

  1. Update the JavaScript file by adding a new JavaScript method that detects document edges in a canvas element.

    export async function detectCanvas(normalizer, canvas) {
    if (!Dynamsoft) return;

    try {
        let quads = await normalizer.detectQuad(canvas);
        return quads;
    }
    catch (ex) {
        console.error(ex);
    }
    return null;
}

  2. Define the corresponding C# method in the DocumentNormalizer.cs file. The JavaScript function may return multiple detected quadrilaterals, but for our purposes, we will only consider the first detected quadrilateral as the relevant one.

    public async Task<Quadrilateral?> DetectCanvas(IJSObjectReference canvas)
{
    JsonElement? quads = await _module.InvokeAsync<JsonElement>("detectCanvas", _jsObjectReference, canvas);
    List<Quadrilateral> all = Quadrilateral.WrapQuads(quads);
    return all.Count > 0 ? all[0] : null;
}

    Note:

    • canvas: This is the JavaScript canvas element that holds the image in which you want to detect document edges.

    • Quadrilateral: Define a C# class named Quadrilateral. This class should be structured to hold the coordinates of the detected document edges. It deserialize the JSON string returned by the JavaScript function.

      public class Quadrilateral
{
    public int[] Points { get; set; } = new int[8];
    public string location;

    public Quadrilateral(string location)
    {
        this.location = location;
    }

    public static List<Quadrilateral> WrapQuads(JsonElement? result)
    {
        List<Quadrilateral> results = new List<Quadrilateral>();
        if (result != null)
        {
            JsonElement element = result.Value;

            if (element.ValueKind == JsonValueKind.Array)
            {
                foreach (JsonElement item in element.EnumerateArray())
                {
                    if (item.TryGetProperty("location", out JsonElement locationValue))
                    {
                        Quadrilateral? quadrilateral = WrapQuad(item);
                        if (quadrilateral != null)
                        {
                            results.Add(quadrilateral);
                        }
                    }
                }
            }
        }
        return results;
    }

    public static Quadrilateral? WrapQuad(JsonElement result)
    {
        Quadrilateral? quadrilateral = null;
        if (result.TryGetProperty("location", out JsonElement locationValue))
        {
            quadrilateral = new Quadrilateral(locationValue.ToString());
            if (locationValue.TryGetProperty("points", out JsonElement pointsValue))
            {
                int index = 0;
                if (pointsValue.ValueKind == JsonValueKind.Array)
                {
                    foreach (JsonElement point in pointsValue.EnumerateArray())
                    {
                        if (point.TryGetProperty("x", out JsonElement xValue))
                        {
                            int intValue = xValue.GetInt32();
                            quadrilateral.Points[index++] = intValue;
                        }

                        if (point.TryGetProperty("y", out JsonElement yValue))
                        {
                            int intValue = yValue.GetInt32();
                            quadrilateral.Points[index++] = intValue;
                        }
                    }
                }
            }
        }

        return quadrilateral;
    }
}

Implementing Document Rectification

In the documentJsInterop.js file, add a new JavaScript method that performs document rectification. This method should manipulate the image present in a canvas element based on specified edge coordinates.

export async function rectifyCanvas(normalizer, canvas, location) {
    if (!Dynamsoft) return;

    try {
        let points = JSON.parse(location);
        let result = await normalizer.normalize(canvas, { quad: points });
        if (result.image) {
            return result.image.toCanvas();
        }
        else {
            return null;
        }
    }
    catch (ex) {
        console.error(ex);
    }
    return null;
}
Enter fullscreen mode Exit fullscreen mode

In the DocumentNormalizer.cs file, define an interop method that corresponds to the JavaScript rectification method.

public async Task<IJSObjectReference> RectifyCanvas(IJSObjectReference canvas, string location)
{
    IJSObjectReference? rectifiedDocument = await _module.InvokeAsync<IJSObjectReference>("rectifyCanvas", _jsObjectReference, canvas, location);
    return rectifiedDocument;
}
Enter fullscreen mode Exit fullscreen mode
  • canvas: This parameter refers to the JavaScript canvas element containing the image you intend to rectify.
  • location: A JSON string representing the coordinates of the document edges.

Adding Color Filter Options

  1. Modify the documentJsInterop.js file by adding the necessary JavaScript code to apply a color filter to the rectified images.

    export async function setFilter(normalizer, filter) {
    if (!Dynamsoft) return;

    try {
        let settings = await normalizer.getRuntimeSettings();
        settings.ImageParameterArray[0].BinarizationModes[0].ThresholdCompensation = 10;
        settings.NormalizerParameterArray[0].ColourMode = filter;
        await normalizer.setRuntimeSettings(settings);
    }
    catch (ex) {
        console.error(ex);
    }
    return null;
}

  2. In the DocumentNormalizer.cs file, define a Filter class that contains the color filter options including BlackAndWhite, Gray, and Colorful.

    public class Filter
{
    public static string BlackAndWhite = "ICM_BINARY";
    public static string Gray = "ICM_GRAYSCALE";
    public static string Colorful = "ICM_COLOUR";
}

  3. Add a SetFilter() method to the DocumentNormalizer class.

    public async Task SetFilter(string filter)
{
    await _module.InvokeVoidAsync("setFilter", _jsObjectReference, filter);
}

Developing a Document Edge Editor

document edge editor

Implementing an edge editor for fine-tuning document edges is a valuable feature for your application. We can create an overlay for the canvas element and draw the document edges on it. The user can then drag the corners to adjust the document's boundaries.

  1. In JavaScript, dynamically create a new canvas object over the image canvas and add a mouse movement event listener to it. The size of the overlay canvas is the same as that of the image canvas.

    export async function showDocumentEditor(dotNetHelper, cbName, elementId, canvas, location) {
    if (!Dynamsoft) return;
    try {
        let parent = document.getElementById(elementId);
        parent.innerHTML = '';
        parent.appendChild(canvas);

        let overlayCanvas = document.createElement('canvas');
        overlayCanvas.id = 'overlayCanvas';
        overlayCanvas.width = canvas.width; 
        overlayCanvas.height = canvas.height; 
        overlayCanvas.style.position = 'absolute';
        overlayCanvas.style.left = canvas.offsetLeft + 'px';
        overlayCanvas.style.top = canvas.offsetTop + 'px';
        parent.appendChild(overlayCanvas);
        let overlayContext = overlayCanvas.getContext("2d");

        let data = JSON.parse(location);
        overlayCanvas.addEventListener("mousedown", (event) => updatePoint(event, dotNetHelper, cbName, data.points, overlayContext, overlayCanvas));
        overlayCanvas.addEventListener("touchstart", (event) => updatePoint(event, dotNetHelper, cbName, data.points, overlayContext, overlayCanvas));
        drawQuad(dotNetHelper, cbName, data.points, overlayContext, overlayCanvas);
    }
    catch (ex) {
        console.error(ex);
    }
    return null;
}

  2. Draw the quadrilateral on the overlay canvas and send the updated coordinates to C#.

    function drawQuad(dotNetHelper, cbName, points, overlayContext, overlayCanvas) {
    overlayContext.clearRect(0, 0, overlayCanvas.width, overlayCanvas.height);
    overlayContext.strokeStyle = "red";
    for (let i = 0; i < points.length; i++) {
        overlayContext.beginPath();
        overlayContext.arc(points[i].x, points[i].y, 5, 0, 2 * Math.PI);
        overlayContext.stroke();
    }
    overlayContext.beginPath();
    overlayContext.moveTo(points[0].x, points[0].y);
    overlayContext.lineTo(points[1].x, points[1].y);
    overlayContext.lineTo(points[2].x, points[2].y);
    overlayContext.lineTo(points[3].x, points[3].y);
    overlayContext.lineTo(points[0].x, points[0].y);
    overlayContext.stroke();
    dotNetHelper.invokeMethodAsync(cbName, { "location": { "points": points } });
}

  3. Implement the updatePoint() function to update the coordinates of the document edges when the user drags them.

    function updatePoint(e, dotNetHelper, cbName, points, overlayContext, overlayCanvas) {
    let rect = overlayCanvas.getBoundingClientRect();

    let scaleX = overlayCanvas.clientWidth / overlayCanvas.width;
    let scaleY = overlayCanvas.clientHeight / overlayCanvas.height;
    let mouseX = (e.clientX - rect.left) / scaleX;
    let mouseY = (e.clientY - rect.top) / scaleY;

    let delta = 10;
    for (let i = 0; i < points.length; i++) {
        if (Math.abs(points[i].x - mouseX) < delta && Math.abs(points[i].y - mouseY) < delta) {
            overlayCanvas.addEventListener("mousemove", dragPoint);
            overlayCanvas.addEventListener("mouseup", releasePoint);
            overlayCanvas.addEventListener("touchmove", dragPoint);
            overlayCanvas.addEventListener("touchend", releasePoint);
            function dragPoint(e) {
                let rect = overlayCanvas.getBoundingClientRect();
                let mouseX = e.clientX || e.touches[0].clientX;
                let mouseY = e.clientY || e.touches[0].clientY;
                points[i].x = Math.round((mouseX - rect.left) / scaleX);
                points[i].y = Math.round((mouseY - rect.top) / scaleY);
                drawQuad(dotNetHelper, cbName, points, overlayContext, overlayCanvas);
            }
            function releasePoint() {
                overlayCanvas.removeEventListener("mousemove", dragPoint);
                overlayCanvas.removeEventListener("mouseup", releasePoint);
                overlayCanvas.removeEventListener("touchmove", dragPoint);
                overlayCanvas.removeEventListener("touchend", releasePoint);
            }
            break;
        }
    }
}

  4. In C#, add the ShowDocumentEditor() method to the DocumentNormalizer class.

    public async Task ShowDocumentEditor(string elementId, IJSObjectReference imageCanvas, string location)
{
    await _module.InvokeVoidAsync("showDocumentEditor", objRef, "OnQuadChanged", elementId, imageCanvas, location);
}

    Note:

    • elementId: The ID of the HTML element that contains the image canvas.
    • imageCanvas: A JavaScript canvas element that contains the original image.
    • location: A JSON string that contains the coordinates of the document edges.

  5. Define a callback interface and trigger it in the OnQuadChanged() method.

    private ICallback? _callback;

public interface ICallback
{
    Task OnCallback(Quadrilateral quad);
}

public void RegisterCallback(ICallback callback)
{
    _callback = callback;
}

[JSInvokable]
public Task OnQuadChanged(JsonElement quad)
{
    if (_callback != null)
    {
        Quadrilateral? q = Quadrilateral.WrapQuad(quad);
        if (q != null)
        {
            _callback.OnCallback(q);
        }
    }
    return Task.CompletedTask;
}

Building a Blazor Document Scanner App

  1. In Visual Studio, create a new Blazor WebAssembly application.
  2. Install the RazorCameraLibrary and RazorDocumentLibrary NuGet packages into your project.

  3. Open the Pages/Index.razor file and add the following layout code.

    @page "/"

@inject IJSRuntime JSRuntime
@using System.Text.Json
@using RazorCameraLibrary
@using Camera = RazorCameraLibrary.Camera
@using RazorDocumentLibrary
@implements DocumentNormalizer.ICallback

<PageTitle>Index</PageTitle>

<div id="loading-indicator" class="loading-indicator" style="@(isLoading ? "display: flex;" : "display: none;")">
    <div class="spinner"></div>
</div>

<div class="container">
    <div>
        <input type="radio" name="format" value="grayscale" @onchange="HandleInputChange">Grayscale
        <input type="radio" name="format" value="color" checked @onchange="HandleInputChange">Color
        <input type="radio" name="format" value="binary" @onchange="HandleInputChange">B&W
    </div>
    <div class="row">
        <label>
            Get a License key from <a href="https://www.dynamsoft.com/customer/license/trialLicense?product=ddn"
                target="_blank">here</a>
        </label>
        <div class="filler"></div>
        <input type="text" placeholder="@licenseKey" @bind="licenseKey">
        <button @onclick="Activate">Activate SDK</button>
    </div>

    <div>
        <button @onclick="GetCameras">Get Cameras</button>
        <select id="sources" @onchange="e => OnChange(e)">
            @foreach (var camera in cameras)
            {
                <option value="@camera.DeviceId">@camera.Label</option>
            }
        </select>
        <button @onclick="Capture">@buttonText</button>
        <button @onclick="Edit">Edit</button>
        <button @onclick="Rectify">Rectify</button>
    </div>

    <div id="videoview">
        <div id="videoContainer"></div>
    </div>
    <div id="rectified-document"></div>
</div>

  4. Load both the camera enhancer and document detection libraries into your application.

    @code {
    private string licenseKey =
    "LICENSE-KEY";
    private bool isLoading = false;
    private List<Camera> cameras = new List<Camera>();
    private CameraJsInterop? cameraJsInterop;
    private CameraEnhancer? cameraEnhancer;
    private DocumentNormalizer? normalizer;
    private DocumentJsInterop? documentJsInterop;
    private string selectedValue = string.Empty;
    private bool _isCapturing = false;
    private string buttonText = "Start";
    private bool _detectEnabled = false;
    private string? inputValue;
    private IJSObjectReference? savedCanvas = null;
    private Quadrilateral? savedLocation = null;

    protected override async Task OnInitializedAsync()
    {
        documentJsInterop = new DocumentJsInterop(JSRuntime);
        await documentJsInterop.LoadJS();

        cameraJsInterop = new CameraJsInterop(JSRuntime);
        await cameraJsInterop.LoadJS();

        cameraEnhancer = await cameraJsInterop.CreateCameraEnhancer();
        await cameraEnhancer.SetVideoElement("videoContainer");
    }
}

  5. Initialize the document normalizer object using a license key:

    public async Task Activate()
{
    if (documentJsInterop == null) return;
    isLoading = true;
    await documentJsInterop.SetLicense(licenseKey);
    await documentJsInterop.LoadWasm();
    normalizer = await documentJsInterop.CreateDocumentNormalizer();
    normalizer.RegisterCallback(this);
    isLoading = false;
}

public void OnCallback(Quadrilateral quad)
{
    savedLocation = quad;
}

  6. Retrieve all available cameras and open the first one.

    public async Task GetCameras()
{
    if (cameraEnhancer == null) return;
    try
    {
        cameras = await cameraEnhancer.GetCameras();
        if (cameras.Count >= 0)
        {
            selectedValue = cameras[0].DeviceId;
            await OpenCamera();
        }
    }
    catch (Exception ex)
    {
        Console.WriteLine(ex.Message);
    }
}

public async Task OpenCamera()
{
    if (cameraEnhancer == null) return;
    try
    {
        int selectedIndex = cameras.FindIndex(camera => camera.DeviceId == selectedValue);
        await cameraEnhancer.SetResolution(640, 480);
        await cameraEnhancer.OpenCamera(cameras[selectedIndex]);

    }
    catch (Exception ex)
    {
        Console.WriteLine(ex.Message);
    }
}

  7. Implement a button click event to start the document scanning. Set up a loop to continuously capture frames from the camera feed and detect document edges from these frames.

    public async Task Capture()
{
    if (cameraEnhancer == null || recognizer == null) return;

    if (!_isCapturing)
    {
        buttonText = "Stop";
        _isCapturing = true;
        _ = WorkLoop();
    }
    else
    {
        buttonText = "Start";
        _isCapturing = false;
    }
}

private async Task WorkLoop()
{

    if (normalizer == null || cameraEnhancer == null) return;
    Quadrilateral? result;
    while (_isCapturing)
    {
        try
        {
            IJSObjectReference canvas = await cameraEnhancer.AcquireCameraFrame();
            result = await normalizer.DetectCanvas(canvas);
            await cameraEnhancer.ClearOverlay();
            if (result != null)
            {
                if (_detectEnabled)
                {
                    _detectEnabled = false;
                    savedCanvas = canvas;
                    savedLocation = result;
                    await normalizer.ShowDocumentEditor("rectified-document", canvas, savedLocation.location);
                }

                await cameraEnhancer.DrawLine(result.Points[0], result.Points[1], result.Points[2], result.Points[3]);
                await cameraEnhancer.DrawLine(result.Points[2], result.Points[3], result.Points[4], result.Points[5]);
                await cameraEnhancer.DrawLine(result.Points[4], result.Points[5], result.Points[6], result.Points[7]);
                await cameraEnhancer.DrawLine(result.Points[6], result.Points[7], result.Points[0], result.Points[1]);
            }

        }
        catch (Exception ex)
        {
            Console.WriteLine(ex.Message);
        }
    }

    await cameraEnhancer.ClearOverlay();
}

  8. Implement a button click event to display the document edge editor.

    public void Edit()
{
    _detectEnabled = true;
}

  9. Implement a button click event to rectify the document based on the coordinates of the detected document edges.

    public async Task Rectify()
{
    if (normalizer == null || cameraEnhancer == null) return;

    if (savedCanvas != null && savedLocation != null)
    {
        IJSObjectReference rectifiedDocument = await normalizer.RectifyCanvas(savedCanvas, savedLocation.location);
        if (rectifiedDocument != null)
        {
            await normalizer.ShowRectifiedDocument("rectified-document", rectifiedDocument);
        }

        savedCanvas = null;
        savedLocation = null;
    }
    else
    {
        IJSObjectReference canvas = await cameraEnhancer.AcquireCameraFrame();
        Quadrilateral? result = await normalizer.DetectCanvas(canvas);
        if (result != null)
        {
            IJSObjectReference rectifiedDocument = await normalizer.RectifyCanvas(canvas, result.location);
            if (rectifiedDocument != null)
            {
                await normalizer.ShowRectifiedDocument("rectified-document", rectifiedDocument);
            }
        }
    }
}

  10. Run your Blazor document scanner in a browser.

    blazor document scanner

Source Code

https://github.com/yushulx/Razor-Document-Library

Top comments (0)

Read next

anmolbaranwal profile image

🚀 21 Tools to take your dev skills to the moon 🌝

Anmol Baranwal -

oatula profile image

Docker

Atul Anand Oraon -

kashifsoofi profile image

GTK4 `dotnet new` Project Template

Kashif Soofi -

thetombomb profile image

How Well Does Azure Static Web Apps (SWA) Support Next.js? | Pages Architecture

Thomas Desmond -