Developing .NET MAUI iOS Apps to Scan Barcode, Document and MRZ

In the previous article, we demonstrated how to incorporate iOS MRZ detection frameworks into the MrzScannerSDK NuGet package. Having mastered the integration of iOS frameworks for a standalone NuGet package, we now turn our attention to integrating iOS SDKs for barcode, MRZ, and document detection into the comprehensive .NET MAUI library: Capture.Vision.Maui. Our ultimate goal is to enable the development of cross-platform .NET MAUI applications for Windows, Android and iOS using a single codebase.

Integrating iOS Frameworks into Document Detection NuGet Package

Based on our experience with integrating the MRZ detection SDK, we can apply similar steps to incorporate the iOS Document Detection SDK into the DocumentScannerSDK NuGet package.

Step 1: Add iOS Frameworks to the NuGet Package

The Dynamsoft Document Normalizer SDK offers a suite of frameworks for iOS:

• DynamsoftCore
• DynamsoftDocumentNormalizer
• DynamsoftImageProcessing
• DynamsoftIntermediateResult

First, we utilize Objective Sharpie to create the C# bindings for these frameworks. Following this, we add all iOS frameworks and generated files to an iOS binding project.

The csproj file for the binding project is structured as follows:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net7.0-ios</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>true</ImplicitUsings>
    <AssemblyName>DocumentScannerSDK</AssemblyName>
    <IsBindingProject>true</IsBindingProject>
    <GeneratePackageOnBuild>true</GeneratePackageOnBuild>
  </PropertyGroup>

  <ItemGroup>
    <ObjcBindingApiDefinition Include="DynamsoftCore/ApiDefinitions.cs" />
    <ObjcBindingCoreSource Include="DynamsoftCore/StructsAndEnums.cs" />
    <NativeReference Include="DynamsoftCore.framework">
      <Kind>Framework</Kind>
      <Frameworks></Frameworks>
    </NativeReference>
  </ItemGroup>

  <ItemGroup>
    <ObjcBindingApiDefinition Include="DynamsoftDocumentNormalizer/ApiDefinitions.cs" />
    <ObjcBindingCoreSource Include="DynamsoftDocumentNormalizer/StructsAndEnums.cs" />
    <NativeReference Include="DynamsoftDocumentNormalizer.framework">
      <Kind>Framework</Kind>
      <Frameworks></Frameworks>
    </NativeReference>
  </ItemGroup>

  <ItemGroup>
    <ObjcBindingApiDefinition Include="DynamsoftImageProcessing/ApiDefinitions.cs" />
    <NativeReference Include="DynamsoftImageProcessing.framework">
      <Kind>Framework</Kind>
      <Frameworks></Frameworks>
    </NativeReference>
  </ItemGroup>

  <ItemGroup>
    <ObjcBindingApiDefinition Include="DynamsoftIntermediateResult/ApiDefinitions.cs" />
    <NativeReference Include="DynamsoftIntermediateResult.framework">
      <Kind>Framework</Kind>
      <Frameworks></Frameworks>
    </NativeReference>
  </ItemGroup>

</Project>

Step 2: Align the C# API with Android's Implementation

Copy the DocumentScanner.cs file from the Android binding project into the iOS binding project. Modify the code as necessary to ensure it functions correctly on iOS.

• InitLicense:
  

  public class LicenseVerification : LicenseVerificationListener
  {
      public override void LicenseVerificationCallback(bool isSuccess, NSError error)
      {
          if (!isSuccess)
          {
              System.Console.WriteLine(error.UserInfo);
          }
      }
  }
  
  public static void InitLicense(string license, object? context = null)
  {
      DynamsoftLicenseManager.InitLicense(license, new LicenseVerification());
  }
  

• SetParameters：
  

  public void SetParameters(string parameters)
  {
      try
      {
          NSError error;
          normalizer.InitRuntimeSettingsFromString(parameters, out error);
      }
      catch (Exception ex)
      {
          Console.WriteLine(ex.ToString());
      }
  }
  

• DetectFile and DetectBuffer:
  

  public Result[]? DetectFile(string filename)
  {
      NSError error;
      iDetectedQuadResult[]? results = normalizer.DetectQuadFromFile(filename, out error);
      return GetResults(results);
  }
  
  private Result[]? GetResults(iDetectedQuadResult[]? results)
  {
      if (results == null) return null;
  
      var result = new Result[results.Length];
      for (int i = 0; i < results.Length; i++)
      {
          iDetectedQuadResult tmp = results[i];
          iQuadrilateral quad = tmp.Location;
          result[i] = new Result()
          {
              Confidence = (int)tmp.ConfidenceAsDocumentBoundary,
              Points = new int[8]
              {
                  (int)((NSValue)quad.Points[0]).CGPointValue.X, (int)((NSValue)quad.Points[0]).CGPointValue.Y,
                  (int)((NSValue)quad.Points[1]).CGPointValue.X, (int)((NSValue)quad.Points[1]).CGPointValue.Y,
                  (int)((NSValue)quad.Points[2]).CGPointValue.X, (int)((NSValue)quad.Points[2]).CGPointValue.Y,
                  (int)((NSValue)quad.Points[3]).CGPointValue.X, (int)((NSValue)quad.Points[3]).CGPointValue.Y,
              }
          };
      }
  
      return result;
  }
  
  public Result[]? DetectBuffer(byte[] buffer, int width, int height, int stride, ImagePixelFormat format)
  {
      NSData converted = NSData.FromArray(buffer);
  
      iImageData imageData = new iImageData()
      {
          Bytes = converted,
          Width = width,
          Height = height,
          Stride = stride,
          Format = (EnumImagePixelFormat)format,
      };
      NSError error;
      iDetectedQuadResult[]? results = normalizer.DetectQuadFromBuffer(imageData, out error);
      return GetResults(results);
  }
  

• NormalizeFile and NormalizeBuffer:
  

  public NormalizedImage NormalizeFile(string filename, int[] points)
  {
      iQuadrilateral quad = new iQuadrilateral();
      quad.Points = new NSObject[4];
      quad.Points[0] = NSValue.FromCGPoint(new CGPoint(points[0], points[1]));
      quad.Points[1] = NSValue.FromCGPoint(new CGPoint(points[2], points[3]));
      quad.Points[2] = NSValue.FromCGPoint(new CGPoint(points[4], points[5]));
      quad.Points[3] = NSValue.FromCGPoint(new CGPoint(points[6], points[7]));
  
      NSError error;
      iNormalizedImageResult? result = normalizer.NormalizeFile(filename, quad, out error);
      return GetNormalizedImage(result);
  }
  
  public NormalizedImage NormalizeBuffer(byte[] buffer, int width, int height, int stride, ImagePixelFormat format, int[] points)
  {
      iImageData imageData = new iImageData()
      {
          Bytes = NSData.FromArray(buffer),
          Width = width,
          Height = height,
          Stride = stride,
          Format = (EnumImagePixelFormat)format,
      };
  
      iQuadrilateral quad = new iQuadrilateral();
      quad.Points = new NSObject[4];
      quad.Points[0] = NSValue.FromCGPoint(new CGPoint(points[0], points[1]));
      quad.Points[1] = NSValue.FromCGPoint(new CGPoint(points[2], points[3]));
      quad.Points[2] = NSValue.FromCGPoint(new CGPoint(points[4], points[5]));
      quad.Points[3] = NSValue.FromCGPoint(new CGPoint(points[6], points[7]));
      NSError error;
      iNormalizedImageResult? result = normalizer.NormalizeBuffer(imageData, quad, out error);
      return GetNormalizedImage(result);
  }
  
  private NormalizedImage GetNormalizedImage(iNormalizedImageResult? result)
  {
      NormalizedImage normalizedImage = new NormalizedImage();
      if (result != null)
      {
          iImageData imageData = result.Image;
          normalizedImage.Width = (int)imageData.Width;
          normalizedImage.Height = (int)imageData.Height;
          normalizedImage.Stride = (int)imageData.Stride;
          normalizedImage.Format = (ImagePixelFormat)imageData.Format;
          normalizedImage.Data = imageData.Bytes.ToArray();
      }
      return normalizedImage;
  }
  

Step 3: Resolve Build Errors

The C# binding code generated may lead to build errors. These errors must be addressed and corrected manually.

Step 4: Compile the NuGet Package

# build dll for desktop
cd desktop
dotnet build --configuration Release

# build dll for android
cd android
dotnet build --configuration Release

# build dll for iOS
cd ios
dotnet build --configuration Release

# build nuget package
nuget pack .\DocumentScannerSDK.nuspec

Merging DocumentScannerSDK and MrzScannerSDK into a Single NuGet Package

The DynamsoftCore framework serves as a crucial dependency for both the MrzScannerSDK and DocumentScannerSDK NuGet packages. Utilizing both packages within a .NET MAUI project leads to a build conflict due to this shared dependency. To overcome this challenge, we introduced a new NuGet package named CaptureVision, which consolidates all frameworks and C# files from both MrzScannerSDK and DocumentScannerSDK for compilation.

Resolving Build Conflicts between BarcodeQRCodeSDK and CaptureVision

The BarcodeQRCodeSDK NuGet package and the CaptureVision NuGet package are compatible within the same .NET MAUI project for Windows and Android. However, a build conflict arises when attempting to compile our demo application for iOS. This conflict is due to certain data structures, such as iImageData and iQuadrilateral, being defined in both the DynamsoftCore and BarcodeQRCodeSDK. To address this issue, a viable solution involves removing the redundant data structures from the BarcodeQRCodeSDK NuGet package.

Completing the Capture.Vision.Maui NuGet Package with iOS Support

Utilizing CaptureVision 2.0.1 and BarcodeQRCodeSDK 2.3.7, we are now able to finalize the Capture.Vision.Maui NuGet package by incorporating iOS-specific code within Platforms/iOS/NativeCameraView.cs. The code snippet below illustrates the process of detecting documents and MRZ from camera images on iOS.

if (cameraView.EnableDocumentDetect)
{
    DocumentScanner.Result[] results = documentScanner.DetectBuffer(bytearray,
                            (int)width,
                            (int)height,
                            (int)bpr,
                            DocumentScanner.ImagePixelFormat.IPF_ARGB_8888);
    DocumentResult documentResults = new DocumentResult();
    if (results != null && results.Length > 0)
    {
        documentResults = new DocumentResult
        {
            Confidence = results[0].Confidence,
            Points = results[0].Points
        };

        if (cameraView.EnableDocumentRectify)
        {
            NormalizedImage normalizedImage = documentScanner.NormalizeBuffer(bytearray,
                            (int)width,
                            (int)height,
                            (int)bpr,
                            DocumentScanner.ImagePixelFormat.IPF_ARGB_8888, documentResults.Points);
            documentResults.Width = normalizedImage.Width;
            documentResults.Height = normalizedImage.Height;
            documentResults.Stride = normalizedImage.Stride;
            documentResults.Format = normalizedImage.Format;
            documentResults.Data = normalizedImage.Data;
        }
    }

    cameraView.NotifyResultReady(documentResults, (int)width, (int)height);

}

if (cameraView.EnableMrz)
{
    MrzResult mrzResults = new MrzResult();
    try
    {
        MrzScanner.Result[] results = mrzScanner.DetectBuffer(bytearray,
                            (int)width,
                            (int)height,
                            (int)bpr,
                            MrzScanner.ImagePixelFormat.IPF_ARGB_8888);


        if (results != null && results.Length > 0)
        {
            Line[] rawData = new Line[results.Length];
            string[] lines = new string[results.Length];

            for (int i = 0; i < results.Length; i++)
            {
                rawData[i] = new Line()
                {
                    Confidence = results[i].Confidence,
                    Text = results[i].Text,
                    Points = results[i].Points,
                };
                lines[i] = results[i].Text;
            }


            Dynamsoft.MrzResult info = MrzParser.Parse(lines);
            mrzResults = new MrzResult()
            {
                RawData = rawData,
                Type = info.Type,
                Nationality = info.Nationality,
                Surname = info.Surname,
                GivenName = info.GivenName,
                PassportNumber = info.PassportNumber,
                IssuingCountry = info.IssuingCountry,
                BirthDate = info.BirthDate,
                Gender = info.Gender,
                Expiration = info.Expiration,
                Lines = info.Lines
            };

        }
    }

    catch (Exception ex)
    {
        System.Diagnostics.Debug.WriteLine(ex.Message);
    }

    cameraView.NotifyResultReady(mrzResults, (int)width, (int)height);
}

Developing a .NET MAUI iOS App for Barcode, Document, and MRZ Detection

To adapt the existing demo app for iOS, simply replicate the license initialization code from the Android section and incorporate it into Platforms/iOS/Program.cs:

using ObjCRuntime;
using UIKit;
using Dynamsoft;

namespace Capture.Vision.Maui.Example
{
    public class Program
    {
        static void Main(string[] args)
        {
            DocumentScanner.InitLicense("LICENSE-KEY");
            BarcodeQRCodeReader.InitLicense("LICENSE-KEY");
            MrzScanner.InitLicense("LICENSE-KEY");

            UIApplication.Main(args, null, typeof(AppDelegate));
        }
    }
}

Source Code

https://github.com/yushulx/Capture-Vision-Maui

Comments

[jhonwills]

Exciting series! Looking forward to learning more about developing .NET MAUI iOS apps with Prince Of persia barcode scanning and document features. Thanks for sharing your expertise in this area.