DEV Community

Xiao Ling
Xiao Ling

Posted on • Originally published at dynamsoft.com

Creating a Flutter App to Scan MRZ, QR Codes, and Barcodes

#dart #flutter #qr #mrz

flutter_ocr_sdk and flutter_barcode_sdk are two Flutter plugins built on top of the Dynamsoft Capture Vision SDK. They provide easy-to-use, cross-platform APIs for adding MRZ recognition and barcode scanning capabilities to Flutter apps. In this article, you'll learn how to create a Flutter app that integrates both plugins to scan machine-readable zones (MRZ) and 1D/2D barcodes. Instead of starting from scratch, we'll combine two existing example projects into a unified solution.

Demo Video: Flutter MRZ and Barcode Scanner

Prerequisites

Steps to Build an MRZ and Barcode Scanner App in Flutter

The app will include:

  • A toggle button to switch between MRZ and Barcode scanning modes
  • A button to load an image file for detection
  • A button to open the camera for live detection
  • A camera view for real-time MRZ and 1D/2D barcode scanning
  • A result view to display scan results from camera streams and images

We'll start by modifying the source code from the Flutter MRZ Scanner project to add UI and functionality.

Step 1: Install Dependencies

Add the following dependencies to your pubspec.yaml file:

flutter_barcode_sdk: ^3.0.4
flutter_ocr_sdk: ^2.0.4
Enter fullscreen mode Exit fullscreen mode

Step 2: Initialize the SDKs

Since both plugins wrap the same Dynamsoft SDK, you can initialize them with a shared license key in global.dart.

FlutterOcrSdk detector = FlutterOcrSdk();
FlutterBarcodeSdk barcodeReader = FlutterBarcodeSdk();
bool isLicenseValid = false;

Future<int> initSDK() async {
  String licenseKey =
      "LICENSE-KEY";
  int? ret = await detector.init(licenseKey);
  ret = await barcodeReader.init();
  ret = await barcodeReader.setLicense(licenseKey);

  if (ret == 0) isLicenseValid = true;
  return await detector.loadModel(modelType: ModelType.mrz) ?? -1;
}
Enter fullscreen mode Exit fullscreen mode

Step 3: Toggle MRZ and Barcode Modes

In home_page.dart, update the toggle button to switch between MRZ and Barcode modes:

ToggleButtons(
  borderRadius: BorderRadius.circular(10),
  isSelected: [isMrzSelected, !isMrzSelected],
  selectedColor: Colors.white,
  fillColor: Colors.orange,
  color: Colors.grey,
  children: const [
    Padding(
      padding: EdgeInsets.symmetric(horizontal: 20),
      child: Text('MRZ'),
    ),
    Padding(
      padding: EdgeInsets.symmetric(horizontal: 20),
      child: Text('Barcode'),
    ),
  ],
  onPressed: (index) {
    setState(() {
      isMrzSelected = (index == 0);
    });
  },
)
Enter fullscreen mode Exit fullscreen mode

The global boolean variable isMrzSelected determines which scanning mode is active.

Step 4: Scan from Image File

Update the scanImage() method in home_page.dart to perform MRZ or barcode recognition depending on the selected mode:

void openMrzResultPage(OcrLine information) {
  Navigator.push(
    context,
    MaterialPageRoute(
      builder: (context) => MrzResultPage(information: information),
    ),
  );
}

void openBarcodeResultPage(List<BarcodeResult> barcodeResults) {
  Navigator.push(
    context,
    MaterialPageRoute(
      builder: (context) => BarcodeResultPage(barcodeResults: barcodeResults),
    ),
  );
}


void scanImage() async {
  XFile? photo = await picker.pickImage(source: ImageSource.gallery);

  if (photo == null) {
    return;
  }

  if (!kIsWeb && (Platform.isAndroid || Platform.isIOS)) {
    File rotatedImage = await FlutterExifRotation.rotateImage(
      path: photo.path,
    );
    photo = XFile(rotatedImage.path);
  }

  Uint8List fileBytes = await photo.readAsBytes();

  ui.Image image = await decodeImageFromList(fileBytes);

  ByteData? byteData = await image.toByteData(
    format: ui.ImageByteFormat.rawRgba,
  );
  if (byteData != null) {
    if (isMrzSelected) {
      List<List<OcrLine>>? results = await detector.recognizeBuffer(
        byteData.buffer.asUint8List(),
        image.width,
        image.height,
        byteData.lengthInBytes ~/ image.height,
        ImagePixelFormat.IPF_ARGB_8888.index,
        ImageRotation.rotation0.value,
      );

      if (results != null && results[0].isNotEmpty) {
        openMrzResultPage(results[0][0]);
      } else {
        showAlert(context, "OCR Result", "Recognition Failed!");
      }
    } else {
      List<BarcodeResult>? results = await barcodeReader.decodeImageBuffer(
        byteData.buffer.asUint8List(),
        image.width,
        image.height,
        byteData.lengthInBytes ~/ image.height,
        ImagePixelFormat.IPF_ARGB_8888.index,
      );

      if (results.isNotEmpty) {
        openBarcodeResultPage(results);
      } else {
        showAlert(context, "Barcode Result", "Detection Failed!");
      }
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Both result pages already exist in the sample projects. Rename them to avoid conflicts.

Step 5: Real-time Scanning

In camera_manager.dart, update functions like webCamera(), processId(), and _decodeFrame() to call the appropriate APIs based on isMrzSelected. This enables real-time MRZ and barcode scanning across platforms.

Future<void> webCamera() async {
  _isWebFrameStarted = true;
  try {
    while (!(controller == null || isFinished || cbIsMounted() == false)) {
      XFile file = await controller!.takePicture();
      dynamic results;
      if (isMrzSelected) {
        results = await detector.recognizeFile(file.path);
        ocrLines = results;
      } else {
        results = await barcodeReader.decodeFile(file.path);
        barcodeResults = results;
      }

      if (results == null || !cbIsMounted()) return;

      cbRefreshUi();
      if (isReadyToGo && results != null) {
        handleResults(results);
      }
    }
  } catch (e) {
    print(e);
  }
  _isWebFrameStarted = false;
}

Future<void> processId(
      Uint8List bytes, int width, int height, int stride, int format) async {
  int rotation = 0;
  bool isAndroidPortrait = false;
  if (MediaQuery.of(context).size.width <
      MediaQuery.of(context).size.height) {
    if (Platform.isAndroid) {
      rotation = OCR.ImageRotation.rotation90.value;
      isAndroidPortrait = true;
    }
  }

  dynamic results;

  if (isMrzSelected) {
    ocrLines = await detector.recognizeBuffer(
        bytes, width, height, stride, format, rotation);
    results = ocrLines;
  } else {
    barcodeResults = await barcodeReader.decodeImageBuffer(
        bytes, width, height, stride, format);

    if (isAndroidPortrait &&
        barcodeResults != null &&
        barcodeResults!.isNotEmpty) {
      barcodeResults =
          rotate90barcode(barcodeResults!, previewSize!.height.toInt());
    }
    results = barcodeResults;
  }
  _isScanAvailable = true;
  if (results == null || !cbIsMounted()) return;

  cbRefreshUi();
  if (isReadyToGo && results != null) {
    handleResults(results!);
  }
}

Future<void> _decodeFrame(Uint8List rgb, int width, int height) async {
  if (isDecoding) return;

  isDecoding = true;
  dynamic results;
  if (isMrzSelected) {
    ocrLines = await detector.recognizeBuffer(
        rgb,
        width,
        height,
        width * 3,
        ImagePixelFormat.IPF_RGB_888.index,
        OCR.ImageRotation.rotation0.value);

    results = ocrLines;
  } else {
    barcodeResults = await barcodeReader.decodeImageBuffer(
        rgb, width, height, width * 3, ImagePixelFormat.IPF_RGB_888.index);
    results = barcodeResults;
  }

  if (cbIsMounted()) {
    cbRefreshUi();
    if (isReadyToGo && results != null) {
      handleResults(results!);
    }
  }

  isDecoding = false;
}
Enter fullscreen mode Exit fullscreen mode

Step 6: Display Scan Overlays

To render overlays for MRZ and barcode results:

  1. Update createCameraPreview() in camera_page.dart:

    List<Widget> createCameraPreview() {
    Positioned widget;
    if (isMrzSelected) {
      widget = Positioned(
        top: 0.0,
        right: 0.0,
        bottom: 0,
        left: 0.0,
        child: createOverlay(_cameraManager.ocrLines),
      );
    } else {
      widget = Positioned(
        top: 0.0,
        right: 0.0,
        bottom: 0,
        left: 0.0,
        child: createOverlay(_cameraManager.barcodeResults),
      );
    }

    if (!kIsWeb &&
        (Platform.isWindows || Platform.isLinux || Platform.isMacOS)) {
      return [
        SizedBox(width: 640, height: 480, child: _cameraManager.getPreview()),
        widget,
      ];
    } else {
      if (_cameraManager.controller != null &&
          _cameraManager.previewSize != null) {
        double width = _cameraManager.previewSize!.width;
        double height = _cameraManager.previewSize!.height;
        if (!kIsWeb && (Platform.isAndroid || Platform.isIOS)) {
          if (MediaQuery.of(context).size.width <
              MediaQuery.of(context).size.height) {
            width = _cameraManager.previewSize!.height;
            height = _cameraManager.previewSize!.width;
          }
        }

        return [
          SizedBox(
            width: width,
            height: height,
            child: _cameraManager.getPreview(),
          ),
          widget,
        ];
      } else {
        return [const CircularProgressIndicator()];
      }
    }
  }

  2. Add overlay rendering logic for barcodes in global.dart:

    class OverlayPainter extends CustomPainter {
  List<List<OcrLine>>? ocrResults;
  List<BarcodeResult>? barcodeResults;

  OverlayPainter(dynamic results) {
    if (results is List<List<OcrLine>>) {
      ocrResults = results;
    } else if (results is List<BarcodeResult>) {
      barcodeResults = results;
    }
  }

  @override
  void paint(Canvas canvas, Size size) {
    final paint = Paint()
      ..color = Colors.blue
      ..strokeWidth = 5
      ..style = PaintingStyle.stroke;

    final textStyle = TextStyle(
      color: Colors.red, 
      fontSize: 16, 
    );

    if (ocrResults != null) {
      for (List<OcrLine> area in ocrResults!) {
        for (OcrLine line in area) {
          canvas.drawLine(Offset(line.x1.toDouble(), line.y1.toDouble()),
              Offset(line.x2.toDouble(), line.y2.toDouble()), paint);
          canvas.drawLine(Offset(line.x2.toDouble(), line.y2.toDouble()),
              Offset(line.x3.toDouble(), line.y3.toDouble()), paint);
          canvas.drawLine(Offset(line.x3.toDouble(), line.y3.toDouble()),
              Offset(line.x4.toDouble(), line.y4.toDouble()), paint);
          canvas.drawLine(Offset(line.x4.toDouble(), line.y4.toDouble()),
              Offset(line.x1.toDouble(), line.y1.toDouble()), paint);

          final textSpan = TextSpan(
            text: line.text,
            style: textStyle,
          );

          final textPainter = TextPainter(
            text: textSpan,
            textAlign: TextAlign.left,
            textDirection: TextDirection.ltr,
          );

          textPainter.layout();

          final offset = Offset(
            line.x1.toDouble(),
            line.y1.toDouble(),
          );

          textPainter.paint(canvas, offset);
        }
      }
    }

    if (barcodeResults != null) {
      for (var result in barcodeResults!) {
        double minX = result.x1.toDouble();
        double minY = result.y1.toDouble();
        if (result.x2 < minX) minX = result.x2.toDouble();
        if (result.x3 < minX) minX = result.x3.toDouble();
        if (result.x4 < minX) minX = result.x4.toDouble();
        if (result.y2 < minY) minY = result.y2.toDouble();
        if (result.y3 < minY) minY = result.y3.toDouble();
        if (result.y4 < minY) minY = result.y4.toDouble();

        canvas.drawLine(Offset(result.x1.toDouble(), result.y1.toDouble()),
            Offset(result.x2.toDouble(), result.y2.toDouble()), paint);
        canvas.drawLine(Offset(result.x2.toDouble(), result.y2.toDouble()),
            Offset(result.x3.toDouble(), result.y3.toDouble()), paint);
        canvas.drawLine(Offset(result.x3.toDouble(), result.y3.toDouble()),
            Offset(result.x4.toDouble(), result.y4.toDouble()), paint);
        canvas.drawLine(Offset(result.x4.toDouble(), result.y4.toDouble()),
            Offset(result.x1.toDouble(), result.y1.toDouble()), paint);

        TextPainter textPainter = TextPainter(
          text: TextSpan(
            text: result.text,
            style: const TextStyle(
              color: Colors.yellow,
              fontSize: 22.0,
            ),
          ),
          textAlign: TextAlign.center,
          textDirection: TextDirection.ltr,
        );
        textPainter.layout(minWidth: 0, maxWidth: size.width);
        textPainter.paint(canvas, Offset(minX, minY));
      }
    }
  }

  @override
  bool shouldRepaint(OverlayPainter oldDelegate) => true;
}

Running the Flutter MRZ/Barcode Scanner App

Run the app on Windows, Linux, Android, iOS, and Web:

flutter run -d chrome    # Web
flutter run -d linux     # Linux
flutter run -d windows   # Windows
flutter run              # Android or iOS
Enter fullscreen mode Exit fullscreen mode

Flutter Windows MRZ and VIN scanner

Source Code

https://github.com/yushulx/flutter-barcode-mrz-document-scanner/tree/main/examples/barcode_mrz

Top comments (0)