Xiao Ling

Posted on Dec 15, 2023 • Originally published at dynamsoft.com

How to Build a Razor Class Library for Web Camera Access

#webdev #dotnet #csharp #blazor

In our previous article, we mentioned how to create a Web Barcode Scanner using RazorBarcodeLibrary. The library is built on top of the Dynamsoft JavaScript Barcode SDK, which encapsulates the camera access and barcode reading logic in JavaScript. In fact, Dynamsoft also provides an independent JavaScript camera library, named Dynamsoft Camera Enhancer, for FREE use. In this article, we will demonstrate how to build a Razor class library that features camera access, utilizing the Dynamsoft JavaScript Camera SDK. You can combine this library with other image processing libraries, such as RazorBarcodeLibrary, to create a more powerful web camera application.

Demo: Integrating RazorBarcodeLibrary and RazorCameraLibrary

https://yushulx.me/Razor-Camera-Library/

NuGet Package

https://www.nuget.org/packages/RazorCameraLibrary

Initiating a Razor Class Library Project for Camera Access

Create a new Razor class library project named RazorCameraLibrary in Visual Studio.
Download the JavaScript Camera Enhancer library via NPM:
```
npm i dynamsoft-camera-enhancer
```
Copy the dce.js file from the node_modules/dynamsoft-camera-enhancer/dist folder to the wwwroot folder of your project.
Create a cameraJsInterop.js file in the wwwroot folder. This file will export JavaScript functions that can be invoked in C#.

Loading the JavaScript Camera SDK

In the cameraJsInterop.js file, add the following code to load the Dynamsoft Camera Enhancer library:

export function init() {
    return new Promise((resolve, reject) => {
        let script = document.createElement('script');
        script.type = 'text/javascript';
        script.src = '_content/RazorCameraLibrary/dce.js';
        script.onload = async () => {
            resolve();
        };
        script.onerror = () => {
            reject();
        };
        document.head.appendChild(script);
    });
}

Create a CameraJsInterop.cs file in the project root folder and add the following C# code:

using Microsoft.JSInterop;

namespace RazorCameraLibrary
{
    public class CameraJsInterop : IAsyncDisposable
    {
        private readonly Lazy<Task<IJSObjectReference>> moduleTask;

        public CameraJsInterop(IJSRuntime jsRuntime)
        {
            moduleTask = new(() => jsRuntime.InvokeAsync<IJSObjectReference>(
                "import", "./_content/RazorCameraLibrary/cameraJsInterop.js").AsTask());
        }

        public async ValueTask DisposeAsync()
        {
            if (moduleTask.IsValueCreated)
            {
                var module = await moduleTask.Value;
                await module.DisposeAsync();
            }
        }

        public async Task LoadJS()
        {
            var module = await moduleTask.Value;
            await module.InvokeAsync<object>("init");
        }
    }
}

As the LoadJS() method is called, the JavaScript library will be loaded dynamically.

Initializing Dynamsoft Camera Enhancer

Add a createCameraEnhancer() function to the cameraJsInterop.js file. This function creates a JavaScript camera object and returns it to the C# environment:

export async function createCameraEnhancer() {
    if (!Dynamsoft) return;

    try {
        let cameraEnhancer = await Dynamsoft.DCE.CameraEnhancer.createInstance();
        return cameraEnhancer;
    }
    catch (ex) {
        console.error(ex);
    }
    return null;
}

In the CameraJsInterop.cs file, add a CreateCameraEnhancer() function that invokes the createCameraEnhancer() function in JavaScript:

public async Task<CameraEnhancer> CreateCameraEnhancer()
{
    var module = await moduleTask.Value;
    IJSObjectReference jsObjectReference = await module.InvokeAsync<IJSObjectReference>("createCameraEnhancer");
    CameraEnhancer cameraEnhancer = new CameraEnhancer(module, jsObjectReference);
    return cameraEnhancer;
}

The CameraEnhancer class is defined as follows:

using Microsoft.JSInterop;
using System.Text.Json;

namespace RazorCameraLibrary
{
    public class CameraEnhancer : IDisposable
    {
        private IJSObjectReference _module;
        private IJSObjectReference _jsObjectReference;
        private List<Camera> _cameras = new List<Camera>();
        private DotNetObjectReference<CameraEnhancer> objRef;
        private bool _disposed = false;
        public int SourceWidth, SourceHeight;

        public CameraEnhancer(IJSObjectReference module, IJSObjectReference cameraEnhancer)
        {
            _module = module;
            _jsObjectReference = cameraEnhancer;
            objRef = DotNetObjectReference.Create(this);
        }
    }
}

We will add more functionalities to this class later.

Getting All Available Cameras

Create a Camera class in C# to manage camera properties like deviceId and label. This class can be structured as follows:

{
    public string DeviceId { get; set; } = string.Empty;
    public string Label { get; set; } = string.Empty;
}

Add a getCameras(cameraEnhancer) function to the cameraJsInterop.js file. This function retrieves a list of available cameras and returns it to the C# environment:

export async function getCameras(cameraEnhancer) {
    if (!Dynamsoft) return;

    try {
        return await cameraEnhancer.getAllCameras();
    }
    catch (ex) {
        console.error(ex);
    }
}

In the CameraEnhancer.cs file, add a GetCameras() function that invokes the getCameras() function in JavaScript. This function returns a list of Camera objects:

public async Task<List<Camera>> GetCameras()
{
    _cameras.Clear();
    JsonElement? result = await _module.InvokeAsync<JsonElement>("getCameras", _jsObjectReference);

    if (result != null)
    {
        JsonElement element = result.Value;

        if (element.ValueKind == JsonValueKind.Array)
        {
            foreach (JsonElement item in element.EnumerateArray())
            {
                Camera camera = new Camera();
                if (item.TryGetProperty("deviceId", out JsonElement devideIdValue))
                {
                    string? value = devideIdValue.GetString();
                    if (value != null)
                    {
                        camera.DeviceId = value;
                    }
                }

                if (item.TryGetProperty("label", out JsonElement labelValue))
                {
                    string? value = labelValue.GetString();
                    if (value != null)
                    {
                        camera.Label = value;
                    }
                }
                _cameras.Add(camera);
            }
        }
    }

    return _cameras;
}

The TryGetProperty() method is used to deserialize the JSON object returned from JavaScript.

Opening and Closing a Camera

Opening a camera takes two steps:

Setting an HTML Div element as the video container for displaying the camera feed.

JavaScript

export async function setVideoElement(cameraEnhancer, elementId) {
    if (!Dynamsoft) return;

    try {
        let element = document.getElementById(elementId);
        element.className = "dce-video-container";
        await cameraEnhancer.setUIElement(element);
    }
    catch (ex) {
        console.error(ex);
    }
}

public async Task SetVideoElement(string elementId)
{
    await _module.InvokeVoidAsync("setVideoElement", _jsObjectReference, elementId);
}

Open a specific camera.

JavaScript

export async function openCamera(cameraEnhancer, cameraInfo, dotNetHelper, callback) {
    if (!Dynamsoft) return;

    try {
        await cameraEnhancer.selectCamera(cameraInfo);
        cameraEnhancer.on("played", function () {
            let resolution = cameraEnhancer.getResolution();
            dotNetHelper.invokeMethodAsync(callback, resolution[0], resolution[1]);
        });
        await cameraEnhancer.open();
    }
    catch (ex) {
        console.error(ex);
    }
}

public async Task OpenCamera(Camera camera)
{
    await _module.InvokeVoidAsync("openCamera", _jsObjectReference, camera, objRef, "OnSizeChanged");
}

Closing a camera is a straightforward process. Here's how you can implement it in both JavaScript and C#:

JavaScript

export async function closeCamera(cameraEnhancer) {
    if (!Dynamsoft) return;

    try {
        await cameraEnhancer.close();
    }
    catch (ex) {
        console.error(ex);
    }
}

public async Task CloseCamera()
{
    await _module.InvokeVoidAsync("closeCamera", _jsObjectReference);
}

Capturing a Camera Frame

In this section, we'll implement a function to capture a frame from the camera feed. This function will return a JavaScript canvas object, which can be utilized for various image processing tasks. For instance, the canvas can be used to read barcodes from the captured image. This integration allows for versatile applications within ASP.NET and Blazor environments, where you might need to process or analyze the camera feed in real-time.

JavaScript

export function acquireCameraFrame(cameraEnhancer) {
    if (!Dynamsoft) return;

    try {
        let img = cameraEnhancer.getFrame().toCanvas();
        return img;
    }
    catch (ex) {
        console.error(ex);
    }
}

public async Task<IJSObjectReference> AcquireCameraFrame()
{
    IJSObjectReference jsObjectReference = await _module.InvokeAsync<IJSObjectReference>("acquireCameraFrame", _jsObjectReference);
    return jsObjectReference;
}

Drawing Graphics over the Camera Feed

In many camera applications, an overlay is essential for displaying additional information or results directly on the camera feed. To facilitate this, we will create two key functions within our Razor class library:

DrawText(): This function will draw text onto the camera feed, which can be used to display scanning results or other relevant information.

JavaScript

export function drawText(cameraEnhancer, text, x, y) {
    if (!Dynamsoft) return;

    try {
        let drawingLayers = cameraEnhancer.getDrawingLayers();
        let drawingLayer;
        let drawingItems = new Array(
            new Dynamsoft.DCE.DrawingItem.DT_Text(text, x, y, 1),
        )
        if (drawingLayers.length > 0) {
            drawingLayer = drawingLayers[0];
        }
        else {
            drawingLayer = cameraEnhancer.createDrawingLayer();
        }
        drawingLayer.addDrawingItems(drawingItems);
    }
    catch (ex) {
        console.error(ex);
    }
}

public async Task DrawText(string text, int x, int y)
{
    await _module.InvokeVoidAsync("drawText", _jsObjectReference, text, x, y);
}

DrawLine(): This function will be responsible for drawing lines on the camera feed. It can be used to highlight or outline specific areas of interest.

JavaScript

export function drawLine(cameraEnhancer, x1, y1, x2, y2) {
    if (!Dynamsoft) return;

    try {
        let drawingLayers = cameraEnhancer.getDrawingLayers();
        let drawingLayer;
        let drawingItems = new Array(
            new Dynamsoft.DCE.DrawingItem.DT_Line({
                x: x1,
                y: y1
            }, {
                x: x2,
                y: y2
            }, 1)
        )
        if (drawingLayers.length > 0) {
            drawingLayer = drawingLayers[0];
        }
        else {
            drawingLayer = cameraEnhancer.createDrawingLayer();
        }
        drawingLayer.addDrawingItems(drawingItems);
    }
    catch (ex) {
        console.error(ex);
    }
}

public async Task DrawLine(int x1, int y1, int x2, int y2)
{
    await _module.InvokeVoidAsync("drawLine", _jsObjectReference, x1, y1, x2, y2);
}

Testing the Razor Camera Library

To test the Razor camera library, we'll integrate it with the RazorBarcodeLibrary to create a QR code scanner in a Blazor WebAssembly app.

Create a new Blazor WebAssembly app in Visual Studio.
Install the RazorCameraLibrary and RazorBarcodeLibrary NuGet packages.

In the Pages/Index.razor file, add the following code for layout:

@page "/"
@inject IJSRuntime JSRuntime
@using RazorCameraLibrary
@using RazorBarcodeLibrary
@using Camera = RazorCameraLibrary.Camera

<PageTitle>Index</PageTitle>    
<div class="container">
    <div>
        <button @onclick="GetCameras">Get Cameras</button>
        <select id="sources" @onchange="e => OnChange(e)">
            @foreach (var camera in cameras)
            {
                <option value="@camera.DeviceId">@camera.Label</option>
            }
        </select>
        <button @onclick="Capture">@buttonText</button>
    </div>

    <div id="videoview">
        <div id="videoContainer"></div>
    </div>
</div>

Initialize the camera enhancer object:

@code {
    private bool isLoading = false;
    private List<Camera> cameras = new List<Camera>();
    private CameraJsInterop? cameraJsInterop;
    private CameraEnhancer? cameraEnhancer;
    private BarcodeReader? reader;
    private BarcodeJsInterop? barcodeJsInterop;
    private string selectedValue = string.Empty;
    private bool _isCapturing = false;
    private string buttonText = "Start";

    protected override async Task OnInitializedAsync()
    {
        cameraJsInterop = new CameraJsInterop(JSRuntime);
        await cameraJsInterop.LoadJS();

        cameraEnhancer = await cameraJsInterop.CreateCameraEnhancer();
        await cameraEnhancer.SetVideoElement("videoContainer");
    }
}

Get all available cameras and open the first one:

public async Task GetCameras()
{
    if (cameraEnhancer == null) return;
    try
    {
        cameras = await cameraEnhancer.GetCameras();
        if (cameras.Count >= 0)
        {
            selectedValue = cameras[0].DeviceId;
            await OpenCamera();
        }
    }
    catch (Exception ex)
    {
        Console.WriteLine(ex.Message);
    }
}

public async Task OpenCamera()
{
    if (cameraEnhancer == null) return;
    try
    {
        int selectedIndex = cameras.FindIndex(camera => camera.DeviceId == selectedValue);
        await cameraEnhancer.SetResolution(640, 480);
        await cameraEnhancer.OpenCamera(cameras[selectedIndex]);

    }
    catch (Exception ex)
    {
        Console.WriteLine(ex.Message);
    }
}

Add a button click event handler to trigger barcode scanning. Implement a loop to capture frames from the camera feed and read barcodes from the captured frames:

public async Task Capture()
{
    if (barcodeJsInterop == null || reader == null)
    {
        isLoading = true;
        barcodeJsInterop = new BarcodeJsInterop(JSRuntime);
        await barcodeJsInterop.LoadJS();

        reader = await barcodeJsInterop.CreateBarcodeReader();
        isLoading = false;
    }

    if (cameraEnhancer == null) return;

    if (!_isCapturing)
    {
        buttonText = "Stop";
        _isCapturing = true;
        _ = WorkLoop();
    }
    else
    {
        buttonText = "Start";
        _isCapturing = false;
    }
}

private async Task WorkLoop()
{
    List<BarcodeResult> results = new List<BarcodeResult>();
    if (barcodeJsInterop == null || cameraEnhancer == null || reader == null) return;

    while (_isCapturing)
    {
        try
        {
            IJSObjectReference canvas = await cameraEnhancer.AcquireCameraFrame();
            results = await reader.DecodeCanvas(canvas);
            await cameraEnhancer.ClearOverlay();
            for (int i = 0; i < results.Count; i++)
            {
                BarcodeResult result = results[i];
                int minX = result.X1;
                int minY = result.Y1;

                await cameraEnhancer.DrawLine(result.X1, result.Y1, result.X2, result.Y2);
                minX = minX < result.X2 ? minX : result.X2;
                minY = minY < result.Y2 ? minY : result.Y2;
                await cameraEnhancer.DrawLine(result.X2, result.Y2, result.X3, result.Y3);
                minX = minX < result.X3 ? minX : result.X3;
                minY = minY < result.Y3 ? minY : result.Y3;
                await cameraEnhancer.DrawLine(result.X3, result.Y3, result.X4, result.Y4);
                minX = minX < result.X4 ? minX : result.X4;
                minY = minY < result.Y4 ? minY : result.Y4;
                await cameraEnhancer.DrawLine(result.X4, result.Y4, result.X1, result.Y1);

                await cameraEnhancer.DrawText(result.Text, minX, minY);
            }
        }
        catch (Exception ex)
        {
            Console.WriteLine(ex.Message);
        }
    }

    await cameraEnhancer.ClearOverlay();
}