How to Build a Windows Desktop App for Document, Barcode, and MRZ Detection with C# and .NET WinForms

Dynamsoft's Capture Vision SDKs consist of the Dynamsoft Document Normalizer, Dynamsoft Barcode Reader, and Dynamsoft Label Recognizer. All of these SDKs are cross-platform, supporting Windows, Linux, Android, iOS and web. Each vision SDK can be used independently or in combination with others. In this article, we'll show you how to build a Windows desktop app that integrates Dynamsoft's vision APIs for document rectification, barcode scanning, and MRZ detection using C# and .NET WinForms.

Development Environment

• Visual Studio 2022
• Visual Studio Code
• .NET 6.0 SDK or later

NuGet Packages

OpenCV

• OpenCvSharp4
• OpenCvSharp4.Extensions
• OpenCvSharp4.runtime.win

We use OpenCV to access the camera and display the video stream.

Dynamsoft Vision SDKs

• BarcodeQRCodeSDK: Used to detect 1D and 2D barcodes.
• DocumentScannerSDK: Used to detect document edges and rectify documents.
• MrzScannerSDK: Used to detect MRZ (Machine Readable Zone) on passport, Visa, ID card and other travel documents.

To utilize the Dynamsoft Vision SDKs, you need a valid license. A free trial license is available from the Dynamsoft Customer Portal. You have the option to request individual licenses for each of the three SDKs, or a unified license that covers all SDKs.

Create a Windows Forms Project

In Visual Studio, create a new Windows Forms project for .NET, not for the .NET Framework.

In the Solution Explorer, right-click the project and select Manage NuGet Packages. Search for the above NuGet packages and install them.

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>WinExe</OutputType>
    <TargetFramework>net6.0-windows</TargetFramework>
    <Nullable>enable</Nullable>
    <UseWindowsForms>true</UseWindowsForms>
    <ImplicitUsings>enable</ImplicitUsings>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="BarcodeQRCodeSDK " Version="2.3.4" />
    <PackageReference Include="DocumentScannerSDK " Version="1.1.0" />
    <PackageReference Include="MrzScannerSDK" Version="1.2.0" />
    <PackageReference Include="OpenCvSharp4" Version="4.6.0.20220608" />
  </ItemGroup>
  <ItemGroup>
    <PackageReference Include="OpenCvSharp4.Extensions" Version="4.5.5.20211231" />
    <PackageReference Include="OpenCvSharp4.runtime.win" Version="4.6.0.20220608" />
  </ItemGroup>
</Project>

Windows Form UI Design

The UI contains some basic controls, such as ToolStripStatusLabel, ToolStripMenuItem, PictureBox, Button, CheckBox, and RichTextBox.

• There are two PictureBox controls. One is used to display the input image or video stream. The other is used to display the captured image with the detected results.
• The ToolStripMenuItem allows you to enter the license key and SDK-relevant templates.
• The CheckBox is used to enable or disable the corresponding SDK. You will see the corresponding vision effect in the PictureBox control.
• The Button provides the operations of loading an image, toggling the camera, as well as saving the captured image.
• The ToolStripStatusLabel makes you know the status of the SDKs.
• The RichTextBox displays the detected results.

How to Initialize Dynamsoft Vision SDKs

Press F7 to view the code behind the form. Within the constructor, initialize the SDKs and set the license keys.

using Dynamsoft;
using static Dynamsoft.MrzScanner;
using static Dynamsoft.DocumentScanner;
using static Dynamsoft.BarcodeQRCodeReader;

using MrzResult = Dynamsoft.MrzScanner.Result;
using DocResult = Dynamsoft.DocumentScanner.Result;
using BarcodeResult = Dynamsoft.BarcodeQRCodeReader.Result;

public Form1()
{
    InitializeComponent();
    FormClosing += new FormClosingEventHandler(Form1_Closing);
    string license = "DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ==";

    ActivateLicense(license);

    // Initialize camera
    capture = new VideoCapture(0);
    isCapturing = false;

    // Initialize MRZ scanner
    mrzScanner = MrzScanner.Create();
    mrzScanner.LoadModel();

    // Initialize document scanner
    documentScanner = DocumentScanner.Create();
    documentScanner.SetParameters(DocumentScanner.Templates.color);

    // Initialize barcode scanner
    barcodeScanner = BarcodeQRCodeReader.Create();
}

private void ActivateLicense(string license)
{
    int ret = MrzScanner.InitLicense(license);
    ret = DocumentScanner.InitLicense(license);
    BarcodeQRCodeReader.InitLicense(license);
    if (ret != 0)
    {
        toolStripStatusLabel1.Text = "License is invalid.";
    }
    else
    {
        toolStripStatusLabel1.Text = "License is activated successfully.";
    }
}

The MRZ model can be found in the NuGet package.

While developing the .NET application, the model will be automatically sourced from the NuGet package path. However, for the distribution of the application, it is necessary to copy the model to the directory where the executable file resides and explicitly specify the path to the model.

string? assemblyPath = System.IO.Path.GetDirectoryName(
        System.Reflection.Assembly.GetExecutingAssembly().Location
    );

string modelPath = assemblyPath == null ? "" : Path.Join(assemblyPath, "model");

mrzScanner.LoadModel(modelPath);

Menu Items and Input Dialogs

When you click on the menu item, an input box will appear, allowing you to enter the license key or template string. This box can be created by customizing a form:

public static string InputBox(string title, string promptText, string value)
{
    Form form = new Form();
    TextBox textBox = new TextBox();
    Button buttonOk = new Button();
    Button buttonCancel = new Button();

    form.Text = title;
    textBox.Text = value;

    buttonOk.Text = "OK";
    buttonCancel.Text = "Cancel";
    buttonOk.DialogResult = DialogResult.OK;
    buttonCancel.DialogResult = DialogResult.Cancel;

    textBox.SetBounds(12, 36, 372, 20);
    buttonOk.SetBounds(60, 72, 80, 30);
    buttonCancel.SetBounds(260, 72, 80, 30);

    form.ClientSize = new System.Drawing.Size(400, 120);
    form.Controls.AddRange(new Control[] { textBox, buttonOk, buttonCancel });
    form.FormBorderStyle = FormBorderStyle.FixedDialog;
    form.StartPosition = FormStartPosition.CenterScreen;
    form.MinimizeBox = false;
    form.MaximizeBox = false;
    form.AcceptButton = buttonOk;
    form.CancelButton = buttonCancel;

    DialogResult dialogResult = form.ShowDialog();
    return textBox.Text;
}

The click event handler of the menu item is as follows:

enterLicenseKeyToolStripMenuItem.Click += enterLicenseKeyToolStripMenuItem_Click;
dDNToolStripMenuItem.Click += dDNToolStripMenuItem_Click;
dBRToolStripMenuItem.Click += dBRToolStripMenuItem_Click;

private void enterLicenseKeyToolStripMenuItem_Click(object sender, EventArgs e)
{
    string license = InputBox("Enter License Key", "", "");
    if (license != null && license != "")
    {
        ActivateLicense(license);
    }
}

private void dDNToolStripMenuItem_Click(object sender, EventArgs e)
{
    string template = InputBox("Set DDN Template", "", "");
    if (template != null && template != "")
    {
        documentScanner.SetParameters(template);
    }
}

private void dBRToolStripMenuItem_Click(object sender, EventArgs e)
{
    string template = InputBox("Set DBR Template", "", "");
    if (template != null && template != "")
    {
        barcodeScanner.SetParameters(template);
    }
}

Load Image Files from the Local Disk

To load an image file from the local disk, you can use the OpenFileDialog class. The ListBox control can be used to store the history of loaded images.

private void buttonFile_Click(object sender, EventArgs e)
{
    StopScan();
    using (OpenFileDialog dlg = new OpenFileDialog())
    {
        dlg.Title = "Open Image";
        dlg.Filter = "Image files (*.bmp, *.jpg, *.png) | *.bmp; *.jpg; *.png";

        if (dlg.ShowDialog() == DialogResult.OK)
        {
            listBox1.Items.Add(dlg.FileName);
        }
    }
}

Show Camera Video Stream

The camera video stream is implemented using OpenCV's VideoCapture class. A worker thread is created, running an infinite loop, to constantly capture the video stream and display frames in the PictureBox control.

private void StartScan()
{
    buttonCamera.Text = "Stop";
    isCapturing = true;
    thread = new Thread(new ThreadStart(FrameCallback));
    thread.Start();
}

private void StopScan()
{
    buttonCamera.Text = "Camera Scan";
    isCapturing = false;
    if (thread != null) thread.Join();
}

private void FrameCallback()
{
    while (isCapturing)
    {
        capture.Read(_mat);
        Mat copy = new Mat(_mat.Rows, _mat.Cols, MatType.CV_8UC3);
        _mat.CopyTo(copy);

        ...

        pictureBoxSrc.Image = BitmapConverter.ToBitmap(copy);
    }
    ...
}

Detect and Rectify Documents

When a frame is captured as a Mat object, we initially create a duplicate of it. We use one frame buffer as an input for the DetectBuffer() method, while the other is employed to display the detection results using the DrawContours() method.

private Mat DetectDocument(Mat mat, Mat canvas)
{
    int length = mat.Cols * mat.Rows * mat.ElemSize();
    byte[] bytes = new byte[length];
    Marshal.Copy(mat.Data, bytes, 0, length);

    _docResults = documentScanner.DetectBuffer(bytes, mat.Cols, mat.Rows, (int)mat.Step(), DocumentScanner.ImagePixelFormat.IPF_RGB_888);
    if (_docResults != null)
    {
        DocResult result = _docResults[0];
        if (result.Points != null)
        {
            Point[] points = new Point[4];
            for (int i = 0; i < 4; i++)
            {
                points[i] = new Point(result.Points[i * 2], result.Points[i * 2 + 1]);
            }
            Cv2.DrawContours(canvas, new Point[][] { points }, 0, Scalar.Blue, 2);
        }
    }
    return canvas;
}

Once the document is detected, the NormalizeBuffer() method can be employed to crop and rectify the document. This method returns a NormalizedImage object. To display the NormalizedImage object in the PictureBox control, you need to convert it to a Mat object first. Then use BitmapConverter class to convert the Mat object to a Bitmap object:

private void PreviewNormalizedImage()
{
    if (_docResults != null)
    {
        DocResult result = _docResults[0];
        int length = _mat.Cols * _mat.Rows * _mat.ElemSize();
        byte[] bytes = new byte[length];
        Marshal.Copy(_mat.Data, bytes, 0, length);

        NormalizedImage image = documentScanner.NormalizeBuffer(bytes, _mat.Cols, _mat.Rows, (int)_mat.Step(), DocumentScanner.ImagePixelFormat.IPF_RGB_888, result.Points);
        if (image != null && image.Data != null)
        {
            Mat newMat;
            if (image.Stride < image.Width)
            {
                // binary
                byte[] data = image.Binary2Grayscale();
                newMat = new Mat(image.Height, image.Width, MatType.CV_8UC1, data);
            }
            else if (image.Stride >= image.Width * 3)
            {
                // color
                newMat = new Mat(image.Height, image.Width, MatType.CV_8UC3, image.Data);
            }
            else
            {
                // grayscale
                newMat = new Mat(image.Height, image.Stride, MatType.CV_8UC1, image.Data);
            }

            Mat copy = new Mat(_mat.Rows, _mat.Cols, MatType.CV_8UC3);
            newMat.CopyTo(copy);

            pictureBoxDest.Image = BitmapConverter.ToBitmap(copy);
        }
    }
}

Read Barcode and QR Code

Similar to the document detection process, we use the DecodeBuffer() method to detect barcodes and QR codes:

private Mat DetectBarcode(Mat mat, Mat canvas)
{
    int length = mat.Cols * mat.Rows * mat.ElemSize();
    byte[] bytes = new byte[length];
    Marshal.Copy(mat.Data, bytes, 0, length);

    BarcodeResult[]? results = barcodeScanner.DecodeBuffer(bytes, mat.Cols, mat.Rows, (int)mat.Step(), BarcodeQRCodeReader.ImagePixelFormat.IPF_RGB_888);
    if (results != null)
    {
        foreach (BarcodeResult result in results)
        {
            string output = "Text: " + result.Text + Environment.NewLine + "Format: " + result.Format1 + Environment.NewLine;
            this.BeginInvoke((MethodInvoker)delegate {
                richTextBoxInfo.AppendText(output);
                richTextBoxInfo.AppendText(Environment.NewLine);
            });

            int[]? points = result.Points;
            if (points != null)
            {
                OpenCvSharp.Point[] all = new OpenCvSharp.Point[4];
                int xMin = points[0], yMax = points[1];
                all[0] = new OpenCvSharp.Point(xMin, yMax);
                for (int i = 2; i < 7; i += 2)
                {
                    int x = points[i];
                    int y = points[i + 1];
                    OpenCvSharp.Point p = new OpenCvSharp.Point(x, y);
                    xMin = x < xMin ? x : xMin;
                    yMax = y > yMax ? y : yMax;
                    all[i / 2] = p;
                }
                OpenCvSharp.Point[][] contours = new OpenCvSharp.Point[][] { all };
                Cv2.DrawContours(canvas, contours, 0, new Scalar(0, 255, 0), 2);
                if (result.Text != null) Cv2.PutText(canvas, result.Text, new OpenCvSharp.Point(xMin, yMax), HersheyFonts.HersheySimplex, 1, new Scalar(0, 0, 255), 2);
            }
        }
    }

    return canvas;
}

The BeginInvoke() method is used to update the UI from a non-UI thread.

Enhance MRZ Detection with Document Rectification

MRZ detection relies heavily on the alignment of characters. It is better to make the MRZ area horizontal for more accurate detection. Although the MRZ SDK does not feature quadrilateral distortion correction, it can employ the document scanner SDK to achieve the preprocessing. By combining these two SDKs, MRZ detection from the document image can be significantly optimized.

private Mat DetectMrz(Mat mat, Mat canvas)
{
    int length = mat.Cols * mat.Rows * mat.ElemSize();
    byte[] bytes = new byte[length];
    Marshal.Copy(mat.Data, bytes, 0, length);
    _mrzResults = mrzScanner.DetectBuffer(bytes, mat.Cols, mat.Rows, (int)mat.Step(), MrzScanner.ImagePixelFormat.IPF_RGB_888);
    if (_mrzResults != null)
    {
        string[] lines = new string[_mrzResults.Length];
        var index = 0;
        foreach (MrzResult result in _mrzResults)
        {
            lines[index++] = result.Text;
            this.BeginInvoke((MethodInvoker)delegate {
                richTextBoxInfo.Text += result.Text + Environment.NewLine;
            });

            if (result.Points != null)
            {
                Point[] points = new Point[4];
                for (int i = 0; i < 4; i++)
                {
                    points[i] = new Point(result.Points[i * 2], result.Points[i * 2 + 1]);
                }
                Cv2.DrawContours(canvas, new Point[][] { points }, 0, Scalar.Red, 2);
            }
        }

        JsonNode? info = Parse(lines);
        if (info != null)
        {
            this.BeginInvoke((MethodInvoker)delegate {
                richTextBoxInfo.Text = info.ToString();
            });

        }
    }

    return canvas;
}

private void PreviewNormalizedImage()
{
    if (_docResults != null)
    {
        ...
        if (checkBoxMrz.Checked)
        {
            copy = DetectMrz(newMat, copy);
        }
        pictureBoxDest.Image = BitmapConverter.ToBitmap(copy);
    }
}

MRZ SDK only

Combine document detection and MRZ detection

Source Code

https://github.com/yushulx/dotnet-barcode-qr-code-sdk/tree/main/example/barcode_document_mrz