How to Create a .NET MAUI Plugin for Camera Barcode Qr Code Scanning

As .NET MAUI continues to gain traction, numerous .NET developers are exploring ways to transition their existing desktop and mobile applications to this new framework. Previously, I developed a .NET MAUI application capable of scanning barcodes and QR codes from camera frames across Windows, Android, and iOS platforms. For reuse, I decided to isolate the camera and barcode scanning functionality and encapsulate them into a .NET MAUI plugin. In this article, I will guide you through the creation of a .NET MAUI plugin, featuring a custom camera view and the integration of the Dynamsoft Barcode Reader SDK.

Demo Video

Try Capture.Vision.Maui

https://www.nuget.org/packages/Capture.Vision.Maui/

The Shortcomings of My Current .NET MAUI Application

Check out the source code: https://github.com/yushulx/Capture-Vision-Maui. Before getting started with the plugin, let's take a look at the shortcomings of my current .NET MAUI application:

• Windows: The camera stream is captured using OpenCVSharp. You need to install OpenCVSharp4 and OpenCV Windows runtime to make it work. Since the camera stream is displayed by rendering the OpenCVSharp Mat object to a SkiaSharp SKCanvas, the UI code cannot be shared across Android and iOS platforms.
• Android and iOS: The code for Android and iOS is ported from my Xamarin.Forms Qr code scanner application. To make the code compatible with .NET MAUI, the AddCompatibilityRenderer() method is used. However, within the .NET MAUI environment, the AddHandler() method is recommended.

What's the difference between AddCompatibilityRenderer() and AddHandler()?

• AddCompatibilityRenderer: This method is used to map Xamarin.Forms renderers to .NET MAUI handlers. It's a way of maintaining compatibility with existing Xamarin.Forms custom renderers as you transition your apps to .NET MAUI.

• AddHandler: This is a method used in the context of the new architecture in .NET MAUI. Handlers are a more lightweight, efficient replacement for the old Xamarin.Forms renderers, and they allow greater control and easier customization of your controls.

Our objective is to develop a unified camera view equipped with barcode scanning capabilities, compatible with Windows, Android, and iOS platforms. The following resources will be helpful:

• Camera: https://github.com/hjam40/Camera.MAUI
• Barcode: https://github.com/yushulx/dotnet-barcode-qr-code-sdk

Initiate a .NET MAUI Plugin Project

Let's set up a .NET MAUI plugin project and configure the dependencies:

1. Create a .NET MAUI class library project in Visual Studio 2022.

   

2. Double-click the project file and then add BarcodeQRCodeSDK as a dependency. The package is a .NET wrapper for the Dynamsoft Barcode Reader SDK, supporting Windows, Linux, macOS, Android, and iOS platforms. A valid license key is required to use the package. Note that the BarcodeQRCodeSDK package does not support Maccatalyst framework.
   

   <ItemGroup>
       <PackageReference Include="BarcodeQRCodeSDK" Version="2.3.4" />     
   </ItemGroup>
   

3. In order to test the plugin, add a new .NET MAUI app project to the solution.

   

   After that, add a reference to the class library project in the app project.

   

.NET MAUI Custom View with Platform-Specific Code

In .NET MAUI, a custom view is created by extending the View class. Simultaneously, an associated ViewHandler must be established to handle platform-specific rendering.

A view is responsible for defining what elements are present on the user interface and their properties. In our case, the camera view should be able to display the camera stream, retrieve camera frames and return barcode results.

using System.Collections.ObjectModel;
using static Capture.Vision.Maui.CameraInfo;

namespace Capture.Vision.Maui
{
    public class ResultReadyEventArgs : EventArgs
    {
        public ResultReadyEventArgs(object result, int previewWidth, int previewHeight)
        {
            Result = result;
            PreviewWidth = previewWidth;
            PreviewHeight = previewHeight;
        }

        public object Result { get; private set; }
        public int PreviewWidth { get; private set; }
        public int PreviewHeight { get; private set; }

    }

    public class FrameReadyEventArgs : EventArgs
    { 
        public enum PixelFormat
        {
            GRAYSCALE,
            RGB888,
            BGR888,
            RGBA8888,
            BGRA8888,
        }
        public FrameReadyEventArgs(byte[] buffer, int width, int height, int stride, PixelFormat pixelFormat)
        {
            Buffer = buffer;
            Width = width;
            Height = height;
            Stride = stride;
            Format = pixelFormat;
        }

        public byte[] Buffer { get; private set; }
        public int Width { get; private set; }
        public int Height { get; private set; }
        public int Stride { get; private set; }
        public PixelFormat Format { get; private set; }
    }

    public class CameraView : View
    {
        public static readonly BindableProperty CamerasProperty = BindableProperty.Create(nameof(Cameras), typeof(ObservableCollection<CameraInfo>), typeof(CameraView), new ObservableCollection<CameraInfo>());
        public static readonly BindableProperty CameraProperty = BindableProperty.Create(nameof(Camera), typeof(CameraInfo), typeof(CameraView), null);
        public static readonly BindableProperty EnableBarcodeProperty = BindableProperty.Create(nameof(EnableBarcode), typeof(bool), typeof(CameraView), false);
        public static readonly BindableProperty ShowCameraViewProperty = BindableProperty.Create(nameof(ShowCameraView), typeof(bool), typeof(CameraView), false, propertyChanged: ShowCameraViewChanged);
        public event EventHandler<ResultReadyEventArgs> ResultReady;
        public event EventHandler<FrameReadyEventArgs> FrameReady;
    }
}

A handler is responsible for taking the view's definitions and translating them into platform-specific code that can be rendered on the screen.

using Microsoft.Maui.Handlers;
using static Capture.Vision.Maui.CameraInfo;
#if IOS
using PlatformView = Capture.Vision.Maui.Platforms.iOS.NativeCameraView;
#elif ANDROID
using PlatformView = Capture.Vision.Maui.Platforms.Android.NativeCameraView;
#elif WINDOWS
using PlatformView = Capture.Vision.Maui.Platforms.Windows.NativeCameraView;
#else
using PlatformView = System.Object;
#endif

namespace Capture.Vision.Maui
{
    internal partial class CameraViewHandler : ViewHandler<CameraView, PlatformView>
    {
        public static IPropertyMapper<CameraView, CameraViewHandler> PropertyMapper = new PropertyMapper<CameraView, CameraViewHandler>(ViewMapper)
        {
        };
        public static CommandMapper<CameraView, CameraViewHandler> CommandMapper = new(ViewCommandMapper)
        {
        };
        public CameraViewHandler() : base(PropertyMapper, CommandMapper)
        {
        }

#if ANDROID
    protected override PlatformView CreatePlatformView() => new(Context, VirtualView);
#elif IOS || WINDOWS
        protected override PlatformView CreatePlatformView() => new(VirtualView);
#else
    protected override PlatformView CreatePlatformView() => new();
#endif
        protected override void ConnectHandler(PlatformView platformView)
        {
            base.ConnectHandler(platformView);
        }

        protected override void DisconnectHandler(PlatformView platformView)
        {
#if WINDOWS || IOS || ANDROID
            platformView.DisposeControl();
#endif
            base.DisconnectHandler(platformView);
        }

        public Task<Status> StartCameraAsync()
        {
            if (PlatformView != null)
            {
#if WINDOWS || ANDROID || IOS
                return PlatformView.StartCameraAsync();
#endif
            }
            return Task.Run(() => { return Status.Unavailable; });
        }

        public Task<Status> StopCameraAsync()
        {
            if (PlatformView != null)
            {
#if WINDOWS
            return PlatformView.StopCameraAsync();
#elif ANDROID || IOS
                var task = new Task<Status>(() => { return PlatformView.StopCamera(); });
                task.Start();
                return task;
#endif
            }
            return Task.Run(() => { return Status.Unavailable; });
        }
    }

}

The code snippet below demonstrates the platform-relevant classes used to display video across various platforms.

• Windows: MediaPlayerElement
  

  public sealed partial class NativeCameraView : UserControl, IDisposable
  {
      private readonly MediaPlayerElement mediaElement;
      private readonly CameraView cameraView;
      ...
  
      public NativeCameraView(CameraView cameraView)
      {
          this.cameraView = cameraView;
          mediaElement = new MediaPlayerElement
          {
              HorizontalAlignment = Microsoft.UI.Xaml.HorizontalAlignment.Stretch,
              VerticalAlignment = Microsoft.UI.Xaml.VerticalAlignment.Stretch
          };
          Content = mediaElement;
          ...
      }
      ...
  }
  
  

• Android: TextureView
  

  internal class NativeCameraView : FrameLayout
  {
      private readonly CameraView cameraView;
      private readonly Context context;
      private readonly TextureView textureView;
      ...
  
      public NativeCameraView(Context context, CameraView cameraView) : base(context)
      {
          this.context = context;
          this.cameraView = cameraView;
  
          textureView = new(context);
  
          AddView(textureView);
  
          ...
      }
      ...
  }
  

• iOS: AVCaptureVideoPreviewLayer
  

  internal class NativeCameraView : UIView, IAVCaptureVideoDataOutputSampleBufferDelegate, IAVCapturePhotoCaptureDelegate
  {
      private readonly CameraView cameraView;
      private readonly AVCaptureVideoPreviewLayer PreviewLayer;
      ...
  
      public NativeCameraView(CameraView cameraView)
      {
          this.cameraView = cameraView;
  
          captureSession = new AVCaptureSession
          {
              SessionPreset = AVCaptureSession.PresetPhoto
          };
          PreviewLayer = new(captureSession)
          {
              VideoGravity = AVLayerVideoGravity.ResizeAspectFill
          };
          Layer.AddSublayer(PreviewLayer);
          ...
      }
      ...
  }
  

Our primary concern is determining how to obtain camera frames for image processing across different platforms.

Windows

1. Create a MediaFrameReader:
   

   MediaFrameSource frameSource;
   MediaFrameReader frameReader = await mediaCapture.CreateFrameReaderAsync(frameSource);
   frameReader.AcquisitionMode = MediaFrameReaderAcquisitionMode.Realtime;
   if (frameReader != null)
   {
       frameReader.FrameArrived += OnFrameAvailable;
       var status = await frameReader.StartAsync();
   }
   
   

2. Retrieve the latest frame via the OnFrameAvailable event handler.
   

   private void OnFrameAvailable(MediaFrameReader sender, MediaFrameArrivedEventArgs args)
       {
           var frame = sender.TryAcquireLatestFrame();
           if (frame == null) return;
   
           SoftwareBitmap bitmap = frame.VideoMediaFrame.SoftwareBitmap;
           // process image
           bitmap.Dispose();
       }
   
   

Android

1. Create an ImageReader and bind it to a background thread:
   

   private ImageReader imageReader;
   imageReader = ImageReader.NewInstance(videoSize.Width, videoSize.Height, ImageFormatType.Yuv420888, 1);
   backgroundThread = new HandlerThread("CameraBackground");
   backgroundThread.Start();
   backgroundHandler = new Handler(backgroundThread.Looper);
   frameListener = new ImageAvailableListener(cameraView, barcodeReader);
   imageReader.SetOnImageAvailableListener(frameListener, backgroundHandler);
   surfaces.Add(new OutputConfiguration(imageReader.Surface));
   previewBuilder.AddTarget(imageReader.Surface);
   

2. Fetch the latest frame via the OnImageAvailable event handler.
   

   class ImageAvailableListener : Java.Lang.Object, ImageReader.IOnImageAvailableListener
   {
       ...
   
       public void OnImageAvailable(ImageReader reader)
       {
           try
           {
               var image = reader?.AcquireLatestImage();
               if (image == null)
                   return;
   
               Image.Plane[] planes = image.GetPlanes();
               if (planes == null) return;
   
               int width = image.Width;
               int height = image.Height;
               ByteBuffer buffer = planes[0].Buffer;
               byte[] bytes = new byte[buffer.Remaining()];
               buffer.Get(bytes);
               int nRowStride = planes[0].RowStride;
               int nPixelStride = planes[0].PixelStride;
               image.Close();
   
               // process image
           }
           catch (Exception ex)
           {
           }
       }
   }
   

iOS

1. Create an AVCaptureVideoDataOutput and set the SampleBufferDelegate to the current view.
   

   AVCaptureVideoDataOutput videoDataOutput = new AVCaptureVideoDataOutput();
           var videoSettings = NSDictionary.FromObjectAndKey(
               new NSNumber((int)CVPixelFormatType.CV32BGRA),
               CVPixelBuffer.PixelFormatTypeKey);
   videoDataOutput.WeakVideoSettings = videoSettings;
   videoDataOutput.AlwaysDiscardsLateVideoFrames = true;
   cameraDispacher = new DispatchQueue("CameraDispacher");
   
   videoDataOutput.SetSampleBufferDelegate(this, cameraDispacher);
   

2. Fetch the latest frame via the DidOutputSampleBuffer event handler.
   

   [Export("captureOutput:didOutputSampleBuffer:fromConnection:")]
   public void DidOutputSampleBuffer(AVCaptureOutput captureOutput, CMSampleBuffer sampleBuffer, AVCaptureConnection connection)
   {
       CVPixelBuffer cVPixelBuffer = (CVPixelBuffer)sampleBuffer.GetImageBuffer();
       cVPixelBuffer.Lock(CVPixelBufferLock.ReadOnly);
       nint dataSize = cVPixelBuffer.DataSize;
       width = cVPixelBuffer.Width;
       height = cVPixelBuffer.Height;
       IntPtr baseAddress = cVPixelBuffer.BaseAddress;
       bpr = cVPixelBuffer.BytesPerRow;
       cVPixelBuffer.Unlock(CVPixelBufferLock.ReadOnly);
       buffer = NSData.FromBytes(baseAddress, (nuint)dataSize);
       // process image
   }
   

How to Scan Barcodes from Camera Frames

As the camera frames are retrieved, we can now use the Dynamsoft Barcode Reader SDK to scan barcodes from the frames.

1. Create a BarcodeReader instance:
   

   BarcodeQRCodeReader barcodeReader = BarcodeQRCodeReader.Create();
   

2. To decode barcodes from the camera frame, you're required to retrieve the byte data, frame width, height, stride, and pixel format.
   

   // Windows
   Result[] results = barcodeReader.DecodeBuffer(buffer, bitmap.PixelWidth, bitmap.PixelHeight, bitmap.PixelWidth, BarcodeQRCodeReader.ImagePixelFormat.IPF_GRAYSCALED);
   // Android
   Result[] results = barcodeReader.DecodeBuffer(bytes, width, height, nPixelStride * nRowStride, BarcodeQRCodeReader.ImagePixelFormat.IPF_GRAYSCALED);
   // iOS
   Result[] results = barcodeReader.DecodeBuffer(bytearray, (int)width, (int)height, (int)bpr, BarcodeQRCodeReader.ImagePixelFormat.IPF_ARGB_8888);
   

How to Use the Plugin in a .NET MAUI Application

Now that we have a .NET MAUI plugin in place, let's see how to use it in a .NET MAUI application.

1. Add the plugin to the .NET MAUI app project via NuGet Manager.

   

2. In MauiProgram.cs, add the following code to register the plugin:
   

   using Microsoft.Extensions.Logging;
   
   namespace Capture.Vision.Maui.Example
   {
       public static class MauiProgram
       {
           public static MauiApp CreateMauiApp()
           {
               var builder = MauiApp.CreateBuilder();
               builder.UseNativeCameraView()
                   .UseMauiApp<App>()
                   .ConfigureFonts(fonts =>
                   {
                       fonts.AddFont("OpenSans-Regular.ttf", "OpenSansRegular");
                       fonts.AddFont("OpenSans-Semibold.ttf", "OpenSansSemibold");
                   });
   
   #if DEBUG
           builder.Logging.AddDebug();
   #endif
   
               return builder.Build();
           }
       }
   }
   

3. Apply for a trial license and then activate Dynamsoft Barcode Reader in MainPage.xaml.cs:
   

   using Dynamsoft;
   
   namespace Capture.Vision.Maui.Example
   {
       public partial class MainPage : ContentPage
       {
           public MainPage()
           {
               InitializeComponent();
               InitService();
           }
   
           private async void InitService()
           {
               await Task.Run(() =>
               {
                   BarcodeQRCodeReader.InitLicense("LICENSE-KEY");
   
                   return Task.CompletedTask;
               });
           }
       }
   }
   

4. Create a .NET MAUI content page and add a CameraView to it:
   

   <?xml version="1.0" encoding="utf-8" ?>
   <ContentPage xmlns="http://schemas.microsoft.com/dotnet/2021/maui"
               xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
               xmlns:skia="clr-namespace:SkiaSharp.Views.Maui.Controls;assembly=SkiaSharp.Views.Maui.Controls"
               xmlns:cv="clr-namespace:Capture.Vision.Maui;assembly=Capture.Vision.Maui"
               x:Class="Capture.Vision.Maui.Example.CameraPage"
               Title="CameraPage">
       <ScrollView>
           <Grid>
               <cv:CameraView x:Name="cameraView" HorizontalOptions="FillAndExpand"
               VerticalOptions="FillAndExpand" EnableBarcode="True" 
                                   ResultReady="cameraView_ResultReady" FrameReady="cameraView_FrameReady"
                               />
               <skia:SKCanvasView x:Name="canvasView" 
                           Margin="0"
                           HorizontalOptions="FillAndExpand" VerticalOptions="FillAndExpand"
                           PaintSurface="OnCanvasViewPaintSurface" />
           </Grid>
       </ScrollView>
   </ContentPage>
   

   The cameraView_ResultReady event handler is used to display the barcode result:
   

   private void cameraView_ResultReady(object sender, ResultReadyEventArgs e)
   {
       if (e.Result != null)
       {
           Result[] results = (Result[])e.Result;
           foreach (Result result in results)
           {
               System.Diagnostics.Debug.WriteLine(result.Text);
           }
       }
   }
   

   The cameraView_FrameReady event handler is used to retrieve the camera frame for image processing:
   

   private void cameraView_FrameReady(object sender, FrameReadyEventArgs e)
   {
       // process image
   }
   

   

Source Code

https://github.com/yushulx/Capture-Vision-Maui