DEV Community

Xiao Ling
Xiao Ling

Posted on • Originally published at dynamsoft.com

How to Build a Razor Class Library for Passport MRZ Recognition

#webdev #dotnet #csharp #razor

We have written a tutorial demonstrating how to build a web-based passport MRZ scanner app using .NET Blazor WebAssembly. This tutorial requires developers to use both C# and JavaScript. To streamline the development process and allow .NET developers to focus exclusively on C#, we plan to create a Razor class library. This library will encapsulate the Dynamsoft Label Recognizer SDK and be distributed as a NuGet package. It will enable the rapid development of Blazor passport MRZ recognition apps with just a few lines of C# code.

Online Passport MRZ Recognition Demo

https://yushulx.me/Razor-MRZ-Library/

NuGet Package

https://www.nuget.org/packages/RazorMrzLibrary

Download the JavaScript Version of Dynamsoft Label Recognizer

The Dynamsoft Label Recognizer is an OCR library designed to extract text from images. It supports a variety of documents, such as passports, driver's licenses, and ID cards. The JavaScript version of the SDK can be downloaded from npm.

npm i dynamsoft-label-recognizer
Enter fullscreen mode Exit fullscreen mode

After downloading the npm package, the SDK files can be found in the node_modules/dynamsoft-label-recognizer/dist folder. This folder contains several models tailored for different OCR scenarios. The specific files required to build the Razor MRZ recognition library include:

  • MRZ/MRZ.data
  • dlr.js
  • dlr-2.2.31.browser.worker.js
  • dlr-2.2.31.wasm
  • dlr-2.2.31.wasm.js
  • dls.license.dialog.html

dynamsoft label recognizer js sdk

Initiating a Razor Class Library Project for MRZ Recognition

  1. In Visual Studio, create a new Razor class library project and name it RazorMrzLibrary.
  2. Copy the previously listed files into the wwwroot folder of your project.
  3. In the wwwroot folder, create a file named mrzJsInterop.js. This file will contain JavaScript functions that can be invoked from C#.

Loading the JavaScript OCR SDK

  1. Open the mrzJsInterop.js file and add code to load the Dynamsoft Label Recognizer library.

    export function init() {
    return new Promise((resolve, reject) => {
        let script = document.createElement('script');
        script.type = 'text/javascript';
        script.src = '_content/RazorMrzLibrary/dlr.js';
        script.onload = async () => {
            resolve();
        };
        script.onerror = () => {
            reject();
        };
        document.head.appendChild(script);
    });
}

  2. In the project's root folder, create a file named MrzJsInterop.cs. Add the specified C# code to this file.

    using Microsoft.JSInterop;

namespace RazorMrzLibrary
{
    public class MrzJsInterop : IAsyncDisposable
    {
        private readonly Lazy<Task<IJSObjectReference>> moduleTask;

        public MrzJsInterop(IJSRuntime jsRuntime)
        {
            moduleTask = new(() => jsRuntime.InvokeAsync<IJSObjectReference>(
                "import", "./_content/RazorMrzLibrary/mrzJsInterop.js").AsTask());
        }

        public async ValueTask DisposeAsync()
        {
            if (moduleTask.IsValueCreated)
            {
                var module = await moduleTask.Value;
                await module.DisposeAsync();
            }
        }

        public async Task LoadJS()
        {
            var module = await moduleTask.Value;
            await module.InvokeAsync<object>("init");
        }
    }
}

    Note: The LoadJS() method in MrzJsInterop.cs invokes the init() function in JavaScript.

Initializing the Dynamsoft Label Recognizer

The JavaScript SDK is comprised of wasm and js files, and it requires a license key to activate the OCR engine. To initialize the SDK, follow these steps:

  1. Set the license key for the SDK.

    JavaScript

    // mrzJsInterop.js
export function setLicense(license) {
    if (!Dynamsoft) return;
    try {
        Dynamsoft.DLR.LabelRecognizer.license = license;
    }
    catch (ex) {
        console.error(ex);
    }
}

    C#

    // MrzJsInterop.cs

public async Task SetLicense(string license)
{
    var module = await moduleTask.Value;
    await module.InvokeVoidAsync("setLicense", license);
}

  2. Load the wasm (WebAssembly) module.

    JavaScript

    // mrzJsInterop.js
export async function loadWasm() {
    if (!Dynamsoft) return;
    try {
        await Dynamsoft.DLR.LabelRecognizer.loadWasm();
    }
    catch (ex) {
        console.error(ex);
    }
}

    C#

    // MrzJsInterop.cs

public async Task LoadWasm()
{
    var module = await moduleTask.Value;
    await module.InvokeVoidAsync("loadWasm");
}

  3. Instantiate the Dynamsoft Label Recognizer object.

    JavaScript

    // mrzJsInterop.js
export async function createMrzRecognizer() {
    if (!Dynamsoft) return;

    try {
        let recognizer = await Dynamsoft.DLR.LabelRecognizer.createInstance();
        recognizer.ifSaveOriginalImageInACanvas = true;
        await recognizer.updateRuntimeSettingsFromString("MRZ");
        return recognizer;
    }
    catch (ex) {
        console.error(ex);
    }
    return null;
}

    C#

    // MrzJsInterop.cs

public async Task<MrzRecognizer> CreateMrzRecognizer()
{
    var module = await moduleTask.Value;
    IJSObjectReference jsObjectReference = await module.InvokeAsync<IJSObjectReference>("createMrzRecognizer");
    MrzRecognizer recognizer = new MrzRecognizer(module, jsObjectReference);
    return recognizer;
}

The MrzRecognizer class is defined as follows:

using Microsoft.JSInterop;
using System.Text.Json;

namespace RazorMrzLibrary
{
    public class MrzRecognizer
    {
        private IJSObjectReference _module;
        private IJSObjectReference _jsObjectReference;

        public int SourceWidth, SourceHeight;
        public MrzRecognizer(IJSObjectReference module, IJSObjectReference recognizer)
        {
            _module = module;
            _jsObjectReference = recognizer;
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

Recognizing MRZ Text from a Canvas

Within the MrzRecognizer class, the DecodeCanvas() method is added to facilitate the recognition of MRZ (Machine Readable Zone) text from a canvas object.

public async Task<List<OcrResult>> DecodeCanvas(IJSObjectReference canvas)
{
    JsonElement? result = await _module.InvokeAsync<JsonElement>("recognizeCanvas", _jsObjectReference, canvas);
    SourceWidth = await _module.InvokeAsync<int>("getSourceWidth", _jsObjectReference);
    SourceHeight = await _module.InvokeAsync<int>("getSourceHeight", _jsObjectReference);
    return OcrResult.WrapResult(result);
}
Enter fullscreen mode Exit fullscreen mode

The DecodeCanvas() method calls the recognizeCanvas() function in JavaScript to recognize MRZ (Machine Readable Zone) text from the canvas object. Additionally, it utilizes the getSourceWidth() and getSourceHeight() functions to obtain the width and height of the source image.

export async function recognizeCanvas(recognizer, canvas) {
    if (!Dynamsoft) return;

    try {
        let results = await recognizer.recognize(canvas);
        return results;
    }
    catch (ex) {
        console.error(ex);
    }
    return null;
}

export function getSourceWidth(recognizer) {
    let canvas = recognizer.getOriginalImageInACanvas();
    return canvas.width;
}

export function getSourceHeight(recognizer) {
    let canvas = recognizer.getOriginalImageInACanvas();
    return canvas.height;
}
Enter fullscreen mode Exit fullscreen mode

The OcrResult class is designed to deserialize the JSON object returned from the JavaScript function. It includes the following properties: Confidence, Text, and Points. The Confidence property indicates the OCR result's reliability. Text holds the recognized MRZ (Machine Readable Zone) text. Points is an array consisting of eight integers, representing the coordinates of the four corners of the MRZ text area.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Text.Json;
using System.Threading.Tasks;

namespace RazorMrzLibrary
{
    public class OcrResult
    {
        public int Confidence { get; set; }
        public string Text { get; set; }
        public int[] Points { get; set; }

        public OcrResult()
        {
            Points = new int[8];
            Text = "";
        }

        public static List<OcrResult> WrapResult(JsonElement? result)
        {
            List<OcrResult> results = new List<OcrResult>();
            if (result != null)
            {
                JsonElement element = result.Value;

                if (element.ValueKind == JsonValueKind.Array)
                {
                    foreach (JsonElement item in element.EnumerateArray())
                    {
                        if (item.TryGetProperty("lineResults", out JsonElement lineResults))
                        {
                            foreach (JsonElement line in lineResults.EnumerateArray())
                            {
                                OcrResult mrzResult = new OcrResult();

                                if (line.TryGetProperty("confidence", out JsonElement confidenceValue))
                                {
                                    int value = confidenceValue.GetInt32();
                                    mrzResult.Confidence = value;
                                }

                                if (line.TryGetProperty("location", out JsonElement locationValue))
                                {
                                    if (locationValue.TryGetProperty("points", out JsonElement pointsValue))
                                    {
                                        int index = 0;
                                        if (pointsValue.ValueKind == JsonValueKind.Array)
                                        {
                                            foreach (JsonElement point in pointsValue.EnumerateArray())
                                            {
                                                if (point.TryGetProperty("x", out JsonElement xValue))
                                                {
                                                    int intValue = xValue.GetInt32();
                                                    mrzResult.Points[index++] = intValue;
                                                }

                                                if (point.TryGetProperty("y", out JsonElement yValue))
                                                {
                                                    int intValue = yValue.GetInt32();
                                                    mrzResult.Points[index++] = intValue;
                                                }
                                            }
                                        }
                                    }
                                }

                                if (line.TryGetProperty("text", out JsonElement textValue))
                                {
                                    string? value = textValue.GetString();
                                    if (value != null)
                                    {
                                        mrzResult.Text = value;
                                    }

                                }

                                results.Add(mrzResult);
                            }
                        }
                    }
                }
            }
            return results;
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

Parsing MRZ Text to Extract Passport Information

The final step involves parsing the MRZ text to extract passport information. This can be accomplished by using the existing MrzResult.cs and MrzParser.cs source files, which are part of the .NET MRZ SDK project. We can add them to the Razor library project directly. For those looking to develop a .NET desktop application, the necessary package can be installed from NuGet using the following command:

dotnet add package MrzScannerSDK
Enter fullscreen mode Exit fullscreen mode

Building a Blazor Passport MRZ Scanner App

Now, let's build a Blazor passport MRZ scanner app using the RazorMrzLibrary and the RazorCameraLibrary.

  1. In Visual Studio, create a new Blazor WebAssembly application.
  2. Install the RazorCameraLibrary and RazorMrzLibrary NuGet packages into your project.

  3. Open the Pages/Index.razor file and insert the specified layout code.

    @page "/"
@inject IJSRuntime JSRuntime
@using RazorCameraLibrary
@using RazorMrzLibrary
@using Camera = RazorCameraLibrary.Camera

<PageTitle>Index</PageTitle>

<div id="loading-indicator" class="loading-indicator" style="@(isLoading ? "display: flex;" : "display: none;")">
    <div class="spinner"></div>
</div>

<div class="container">
    <div class="row">
        <label>Get a License key from <a href="https://www.dynamsoft.com/customer/license/trialLicense?product=dlr"
                target="_blank">here</a> </label>
        <div class="filler"></div>
        <input type="text" placeholder="@licenseKey" @bind="licenseKey">
        <button @onclick="Activate">Activate SDK</button>
    </div>

    <div>
        <button @onclick="GetCameras">Get Cameras</button>
        <select id="sources" @onchange="e => OnChange(e)">
            @foreach (var camera in cameras)
            {
                <option value="@camera.DeviceId">@camera.Label</option>
            }
        </select>
        <button @onclick="Capture">@buttonText</button>
    </div>

    <div id="videoview">
        <div id="videoContainer"></div>
    </div>

</div>

<div class="mrz-result">@result</div>

  4. Load both the camera enhancer and MRZ recognition libraries into your application.

    @code {
    private string licenseKey =
    "LICENSE-KEY";
    private MarkupString result = new MarkupString("");
    private bool isLoading = false;
    private List<Camera> cameras = new List<Camera>();
    private CameraJsInterop? cameraJsInterop;
    private CameraEnhancer? cameraEnhancer;
    private MrzRecognizer? recognizer;
    private MrzJsInterop? mrzJsInterop;
    private string selectedValue = string.Empty;
    private bool _isCapturing = false;
    private string buttonText = "Start";

    protected override async Task OnInitializedAsync()
    {
        mrzJsInterop = new MrzJsInterop(JSRuntime);
        await mrzJsInterop.LoadJS();

        cameraJsInterop = new CameraJsInterop(JSRuntime);
        await cameraJsInterop.LoadJS();

        cameraEnhancer = await cameraJsInterop.CreateCameraEnhancer();
        await cameraEnhancer.SetVideoElement("videoContainer");
    }
}

  5. Initialize the MRZ recognizer object using a license key:

    public async Task Activate()
{
    if (mrzJsInterop == null) return;
    isLoading = true;
    await mrzJsInterop.SetLicense(licenseKey);
    await mrzJsInterop.LoadWasm();
    recognizer = await mrzJsInterop.CreateMrzRecognizer();
    isLoading = false;
}

  6. Retrieve all available cameras and open the first one.

    public async Task GetCameras()
{
    if (cameraEnhancer == null) return;
    try
    {
        cameras = await cameraEnhancer.GetCameras();
        if (cameras.Count >= 0)
        {
            selectedValue = cameras[0].DeviceId;
            await OpenCamera();
        }
    }
    catch (Exception ex)
    {
        Console.WriteLine(ex.Message);
    }
}

public async Task OpenCamera()
{
    if (cameraEnhancer == null) return;
    try
    {
        int selectedIndex = cameras.FindIndex(camera => camera.DeviceId == selectedValue);
        await cameraEnhancer.SetResolution(640, 480);
        await cameraEnhancer.OpenCamera(cameras[selectedIndex]);

    }
    catch (Exception ex)
    {
        Console.WriteLine(ex.Message);
    }
}

  7. Implement a button click event handler to start the MRZ scanning process. Set up a loop to continuously capture frames from the camera feed and extract MRZ information from these frames.

    public async Task Capture()
{
    if (cameraEnhancer == null || recognizer == null) return;

    if (!_isCapturing)
    {
        buttonText = "Stop";
        _isCapturing = true;
        _ = WorkLoop();
    }
    else
    {
        buttonText = "Start";
        _isCapturing = false;
    }
}

private async Task WorkLoop()
{
    List<OcrResult> results = new List<OcrResult>();
    if (recognizer == null || cameraEnhancer == null) return;

    while (_isCapturing)
    {
        try
        {
            IJSObjectReference canvas = await cameraEnhancer.AcquireCameraFrame();
            results = await recognizer.DecodeCanvas(canvas);
            await cameraEnhancer.ClearOverlay();
            string[] lines = new string[results.Count];
            for (int i = 0; i < results.Count; i++)
            {
                OcrResult result = results[i];
                int minX = result.Points[0];
                int minY = result.Points[1];

                await cameraEnhancer.DrawLine(result.Points[0], result.Points[1], result.Points[2], result.Points[3]);
                minX = minX < result.Points[2] ? minX : result.Points[2];
                minY = minY < result.Points[3] ? minY : result.Points[3];
                await cameraEnhancer.DrawLine(result.Points[2], result.Points[3], result.Points[4], result.Points[5]);
                minX = minX < result.Points[4] ? minX : result.Points[4];
                minY = minY < result.Points[5] ? minY : result.Points[5];
                await cameraEnhancer.DrawLine(result.Points[4], result.Points[5], result.Points[6], result.Points[7]);
                minX = minX < result.Points[6] ? minX : result.Points[6];
                minY = minY < result.Points[7] ? minY : result.Points[7];
                await cameraEnhancer.DrawLine(result.Points[6], result.Points[7], result.Points[0], result.Points[1]);

                await cameraEnhancer.DrawText(result.Text, minX, minY);
                lines[i] = result.Text;
            }

            if (lines.Length > 0)
            {
                MrzResult mrzResult = MrzParser.Parse(lines);
                string text = $"Type: {mrzResult.Type}<br>" +
               $"Nationality: {mrzResult.Nationality}<br>" +
               $"Surname: {mrzResult.Surname}<br>" +
               $"Given name: {mrzResult.GivenName}<br>" +
               $"Passport Number: {mrzResult.PassportNumber}<br>" +
               $"Issue Country: {mrzResult.IssuingCountry}<br>" +
               $"Date of birth: {mrzResult.BirthDate}<br>" +
               $"Gender: {mrzResult.Gender}<br>" +
               $"Expiration: {mrzResult.Expiration}<br>";
                result = new MarkupString(text);
                StateHasChanged();
            }

        }
        catch (Exception ex)
        {
            Console.WriteLine(ex.Message);
        }
    }

    await cameraEnhancer.ClearOverlay();
}

  8. Execute the app to run your Blazor passport MRZ scanner.

    blazor passport MRZ scanner

Source Code

https://github.com/yushulx/Razor-MRZ-Library

Top comments (0)

Read next

jamcha123 profile image

My first full stack app

james -

aaronksaunders profile image

Quick REST API with Hono JS and Drizzle ORM

Aaron K Saunders -

aaravjoshi profile image

7 Essential Accessibility Testing Tools for Web Developers: Ensure Inclusive Design

Aarav Joshi -

abhivyaktii profile image

🌟 NestJS + Databases: Making Snake Case Seamless!🐍

Abhinav -