Xiao Ling

Posted on Apr 28 • Originally published at dynamsoft.com

Implementing Flutter Barcode Scanner with Kotlin and CameraX for Android

#flutter #android #kotlin #camera

When considering the creation of a Flutter barcode scanner application, your first thought might be to search for an existing Flutter plugin. One option is to combine the camera plugin with a barcode scanning plugin, or to utilize a barcode scanning plugin that includes a camera preview feature. However, the camera plugin may not deliver optimal performance for image processing due to the memory copy process between native and Dart code. On the other hand, a barcode scanning plugin with a camera preview may lack the flexibility needed to access camera preview data for other purposes.

Therefore, to achieve both optimal performance and flexibility in camera-related applications, it may be preferable to incorporate native code for easy customization within a Flutter project. In this article, I will guide you through the step-by-step process of implementing a Flutter barcode scanner using Kotlin, CameraX, and a third-party image processing SDK—Dynamsoft Barcode Reader—for Android.

Prerequisites

Dynamsoft Barcode Reader Trial license

Step 1: Scaffolding a Flutter Project

We use the Flutter CLI to create a new Flutter project:

flutter create <app_name>

To enable camera access, update the AndroidManifest.xml file with the required permissions and features:

<manifest xmlns:android="http://schemas.android.com/apk/res/android">
    <uses-permission android:name="android.permission.CAMERA" />
    <uses-feature android:name="android.hardware.camera.any" />
    <uses-feature android:name="android.hardware.camera.autofocus" />

    <application>
        ...
    </application>
</manifest>

In this project, we will utilize CameraX for handling the camera preview and Dynamsoft Barcode Reader for decoding barcodes. To integrate these dependencies:

Set up the self-hosted Maven repository for Dynamsoft Barcode Reader in the android/build.gradle file:

allprojects {
    repositories {
        google()
        mavenCentral()
        maven {
            url "https://download2.dynamsoft.com/maven/aar"
        }
    }
}

Include the dependencies in the android/app/build.gradle file:

dependencies {
    def camerax_version = "1.2.2"
    implementation "androidx.camera:camera-core:${camerax_version}"
    implementation "androidx.camera:camera-camera2:${camerax_version}"
    implementation "androidx.camera:camera-lifecycle:${camerax_version}"
    implementation "androidx.camera:camera-video:${camerax_version}"

    implementation "androidx.camera:camera-view:${camerax_version}"
    implementation "androidx.camera:camera-extensions:${camerax_version}"

    implementation 'com.dynamsoft:dynamsoftbarcodereader:9.6.40'
}

Note: The CameraX version may affect the compatibility with other dependencies. You can check the latest version on the CameraX release page.

Step2: Establishing Communication Between Dart and Kotlin

The MethodChannel class is used to facilitate interoperability between Dart and Kotlin. After instantiating the channel with a specific name in both Dart and Kotlin, messages can be sent and received between the two languages using invokeMethod and setMethodCallHandler methods.

CameraPreviewScreen.dart

class _CameraPreviewScreenState extends State<CameraPreviewScreen> {
  static const platform = MethodChannel('barcode_scan');

  @override
  void initState() {
    super.initState();
    platform.setMethodCallHandler(_handleMethod);
  }

  Future<dynamic> _handleMethod(MethodCall call) async {

  }
}

MainActivity.kt

class MainActivity : FlutterActivity(), ActivityAware {
    private val CHANNEL = "barcode_scan"
    private lateinit var channel: MethodChannel

    override fun configureFlutterEngine(flutterEngine: FlutterEngine) {
        channel = MethodChannel(flutterEngine.dartExecutor.binaryMessenger, CHANNEL)

        channel.setMethodCallHandler { call, result ->

        }
    }
}

Step 3: Integrating CameraX into Flutter Android Project

To get started with CameraX in Android, you can refer to the following sample code:

In Flutter, there are two options for displaying a camera preview: using a Texture through a TextureId or embedding native views directly using registerViewFactory with a platform view.

Using a TextureId

The native code renders the camera preview (or other content) to a SurfaceTexture allocated in Android.
The ID of this SurfaceTexture is passed to Flutter, which uses it to create a Texture widget.
Flutter then displays the Texture widget, showing whatever is rendered on the associated SurfaceTexture.

Using a Platform View

Define a native view in Android, such as a CameraView or any custom view.
Register this view with Flutter's platform view system by creating a PlatformViewFactory and linking it via registerViewFactory.
In Flutter, use a PlatformView (like AndroidView or UiKitView for iOS) to directly embed the native view within the Flutter widget tree.

For high-performance rendering, the Texture approach is recommended for displaying the camera preview in Flutter. Below is an outline of how to render the camera preview to a Texture.

MainActivity.kt

Define class variables for the camera preview and image analyzer:

private val CAMERA_REQUEST_CODE = 101
private val CHANNEL = "barcode_scan"
private lateinit var channel: MethodChannel
private lateinit var flutterTextureEntry: SurfaceTextureEntry
private lateinit var flutterEngine: FlutterEngine
private var lensFacing: Int = CameraSelector.LENS_FACING_BACK
private var preview: Preview? = null
private var imageAnalyzer: ImageAnalysis? = null
private lateinit var cameraExecutor: ExecutorService
private var camera: Camera? = null
private var previewWidth = 1280
private var previewHeight = 720

Request camera permission when configuring the Flutter engine:


override fun configureFlutterEngine(flutterEngine: FlutterEngine) {
    ...

    requestCameraPermission()

    ...
}

private fun requestCameraPermission() {
    if (ContextCompat.checkSelfPermission(this, android.Manifest.permission.CAMERA) !=
                    PackageManager.PERMISSION_GRANTED
    ) {
        ActivityCompat.requestPermissions(
                this,
                arrayOf(android.Manifest.permission.CAMERA),
                CAMERA_REQUEST_CODE
        )
    }
}

Define an ImageAnalyzer class for processing images on a separate thread:

private class ImageAnalyzer(listener: ResultListener? = null) : ImageAnalysis.Analyzer {
    private val listeners = ArrayList<ResultListener>().apply { listener?.let { add(it) } }

    private fun ByteBuffer.toByteArray(): ByteArray {
        rewind() 
        val data = ByteArray(remaining())
        get(data) 
        return data 
    }

    override fun analyze(image: ImageProxy) {

        if (listeners.isEmpty()) {
            image.close()
            return
        }

        val buffer = image.planes[0].buffer

        // Image Processing

        listeners.forEach { it() }

        image.close()
    }
}

Create a startCamera() method to initialize the camera:

override fun configureFlutterEngine(flutterEngine: FlutterEngine) {
    ...

    requestCameraPermission()

    channel = MethodChannel(flutterEngine.dartExecutor.binaryMessenger, CHANNEL)

    channel.setMethodCallHandler { call, result ->
        if (call.method == "startCamera") {
            startCamera(result)
        }
    }
}

private fun startCamera(result: MethodChannel.Result) {
    val cameraProviderFuture = ProcessCameraProvider.getInstance(this)
    cameraProviderFuture.addListener(
            {
                val cameraProvider: ProcessCameraProvider = cameraProviderFuture.get()
                bindCamera(cameraProvider, result)
            },
            ContextCompat.getMainExecutor(this)
    )
}

private fun bindCamera(provider: ProcessCameraProvider, result: MethodChannel.Result) {
    val metrics = windowManager.getCurrentWindowMetrics().bounds

    val screenAspectRatio = aspectRatio(metrics.width(), metrics.height())

    val rotation = display!!.rotation

    var resolutionSize = Size(previewWidth, previewHeight)

    if (rotation == ROTATION_0 || rotation == Surface.ROTATION_180) {
        resolutionSize = Size(previewHeight, previewWidth)
    }

    val cameraProvider =
            provider ?: throw IllegalStateException("Camera initialization failed.")

    val cameraSelector = CameraSelector.Builder().requireLensFacing(lensFacing).build()

    flutterTextureEntry = flutterEngine.renderer.createSurfaceTexture()

    // Preview
    preview =
            Preview.Builder()
                    .setTargetResolution(resolutionSize)
                     .setTargetRotation(rotation)
                    .build()
                    .also {
                        it.setSurfaceProvider { request ->
                            val surfaceTexture =
                                    flutterTextureEntry?.surfaceTexture().apply {
                                        this?.setDefaultBufferSize(request.resolution.width, request.resolution.height)
                                    }
                            val surface = Surface(surfaceTexture)

                            request.provideSurface(
                                    surface,
                                    ContextCompat.getMainExecutor(this)
                            ) {}
                        }
                    }

    imageAnalyzer =
            ImageAnalysis.Builder()
                    .setTargetResolution(resolutionSize)
                     .setTargetRotation(rotation)
                    .build()
                    .also {
                        it.setAnalyzer(
                                cameraExecutor,
                                ImageAnalyzer { results ->

                                }
                        )
                    }

    if (camera != null) {
        removeCameraStateObservers(camera!!.cameraInfo)
    }

    try {
        cameraProvider.unbindAll() // Unbind use cases before rebinding
        camera = cameraProvider.bindToLifecycle(this, cameraSelector, preview, imageAnalyzer)
        observeCameraState(camera?.cameraInfo!!)
        result.success(flutterTextureEntry?.id())
    } catch (e: Exception) {
        result.error("CAMERA_INIT_FAILED", "Failed to initialize camera: ${e.message}", null)
    }
}

The preview and imageAnalyzer are created with the target resolution and rotation. The setSurfaceProvider method is used to provide the camera preview to the Flutter engine. The camera preview is displayed on the Flutter screen using the flutterTextureEntry ID.

Define methods getPreviewWidth() and getPreviewHeight() to fetch the camera preview resolution:

private fun getPreviewWidth(): Double {
    return previewWidth.toDouble()
}

private fun getPreviewHeight(): Double {
    return previewHeight.toDouble()
}

CameraPreviewScreen.dart

Initialize the camera preview:

class _CameraPreviewScreenState extends State<CameraPreviewScreen> {
    static const platform = MethodChannel('barcode_scan');
    int? _textureId;
    double _previewWidth = 0.0;
    double _previewHeight = 0.0;
    bool isPortrait = false;

    @override
    void initState() {
        super.initState();
        platform.setMethodCallHandler(_handleMethod);
        _initializeCamera();
    }

    Future<void> _initializeCamera() async {
        try {
        final int textureId = await platform.invokeMethod('startCamera');
        final double previewWidth =
            await platform.invokeMethod('getPreviewWidth');
        final double previewHeight =
            await platform.invokeMethod('getPreviewHeight');
        setState(() {
            _textureId = textureId;
            _previewWidth = previewWidth;
            _previewHeight = previewHeight;
        });
        } catch (e) {
        print(e);
        }
    }
}

Display the camera preview using the Texture widget:

@override
Widget build(BuildContext context) {
    final screenWidth = MediaQuery.of(context).size.width;
    final screenHeight = MediaQuery.of(context).size.height;
    var orientation = MediaQuery.of(context).orientation;
    isPortrait = orientation == Orientation.portrait;
    return Scaffold(
    body: _textureId == null
        ? const Center(child: CircularProgressIndicator())
        : SizedBox(
            width: screenWidth,
            height: screenHeight,
            child: FittedBox(
                fit: BoxFit.cover,
                child: Stack(
                children: [
                    SizedBox(
                    width: isPortrait ? _previewHeight : _previewWidth,
                    height: isPortrait ? _previewWidth : _previewHeight,
                    child: Texture(textureId: _textureId!),
                    ),
                ],
                ),
            ),
            ),
    );
}

The Texture is wrapped in a SizedBox to maintain the correct aspect ratio of the camera preview. Without the SizedBox, the preview might be stretched or distorted. The FittedBox widget is used to scale the camera preview to fit the screen size. The Stack widget is used to overlay additional widgets on the camera preview. The SizedBox widget, set with screen width and height, is used to display the camera preview in full screen.

Step 4: Integrating Dynamsoft Barcode Reader SDK

The Dynamsoft Barcode Reader SDK offers a straightforward API for decoding barcodes from images. Just a few lines of code can equip your Flutter application with robust barcode scanning capabilities.

MainActivity.kt

Activate the SDK with a valid license key:

fun setLicense(license: String?) {
    BarcodeReader.initLicense(license) { isSuccessful, e ->
        if (isSuccessful) {
            // The license verification was successful.
        } else {
            // The license verification failed. e contains the error information.
        }
    }
}

override fun configureFlutterEngine(flutterEngine: FlutterEngine) {
    setLicens("LICENSE-KEY")
    ...
}

Create a barcode reader instance in the ImageAnalyzer class:

private class ImageAnalyzer(listener: ResultListener? = null) : ImageAnalysis.Analyzer {
    private val mBarcodeReader: BarcodeReader = BarcodeReader()

    ...
}

Decode the barcode from the image and pass the results through listeners:

typealias ResultListener = (results: List<Map<String, Any>>) -> Unit

private class ImageAnalyzer(listener: ResultListener? = null) : ImageAnalysis.Analyzer {
    private val mBarcodeReader: BarcodeReader = BarcodeReader()
    private val listeners = ArrayList<ResultListener>().apply { listener?.let { add(it) } }

    private fun ByteBuffer.toByteArray(): ByteArray {
        rewind() 
        val data = ByteArray(remaining())
        get(data) 
        return data 
    }

    override fun analyze(image: ImageProxy) {
        if (listeners.isEmpty()) {
            image.close()
            return
        }

        val buffer = image.planes[0].buffer
        val stride = image.planes[0].rowStride

        val data = buffer.toByteArray()

        val results =
                mBarcodeReader?.decodeBuffer(
                        data,
                        image.width,
                        image.height,
                        stride,
                        EnumImagePixelFormat.IPF_NV21
                )
        listeners.forEach { it(wrapResults(results)) }

        image.close()
    }

    private fun wrapResults(results: Array<TextResult>?): List<Map<String, Any>> {
        val out = mutableListOf<Map<String, Any>>()
        if (results != null) {
            for (result in results) {
                val data: MutableMap<String, Any> = HashMap()
                data["format"] = result.barcodeFormatString
                data["x1"] = result.localizationResult.resultPoints[0].x
                data["y1"] = result.localizationResult.resultPoints[0].y
                data["x2"] = result.localizationResult.resultPoints[1].x
                data["y2"] = result.localizationResult.resultPoints[1].y
                data["x3"] = result.localizationResult.resultPoints[2].x
                data["y3"] = result.localizationResult.resultPoints[2].y
                data["x4"] = result.localizationResult.resultPoints[3].x
                data["y4"] = result.localizationResult.resultPoints[3].y
                data["angle"] = result.localizationResult.angle
                data["barcodeBytes"] = result.barcodeBytes
                out.add(data)
            }
        }
        return out
    }
}

Send the barcode results to the Dart side:

imageAnalyzer =
            ImageAnalysis.Builder()
                    .setTargetResolution(resolutionSize)
                     .setTargetRotation(rotation)
                    .build()
                    .also {
                        it.setAnalyzer(
                                cameraExecutor,
                                ImageAnalyzer { results ->
                                    Handler(Looper.getMainLooper()).post {
                                        channel.invokeMethod("onBarcodeDetected", results)
                                    }
                                }
                        )
                    }

Note: The callback is executed on a background thread. To update the UI, use Handler to switch to the main thread.

CameraPreviewScreen.dart

Define a BarcodeResult class based on https://github.com/yushulx/flutter_barcode_sdk/blob/main/lib/dynamsoft_barcode.dart.

Retrieve and construct the barcode results. Rotate the coordinates of the barcode results by 90 degrees for portrait mode on Android:

List<BarcodeResult> rotate90barcode(List<BarcodeResult> input, int height) {
  List<BarcodeResult> output = [];
  for (BarcodeResult result in input) {
    int x1 = result.x1;
    int x2 = result.x2;
    int x3 = result.x3;
    int x4 = result.x4;
    int y1 = result.y1;
    int y2 = result.y2;
    int y3 = result.y3;
    int y4 = result.y4;

    BarcodeResult newResult = BarcodeResult(
        result.format,
        result.text,
        height - y1,
        x1,
        height - y2,
        x2,
        height - y3,
        x3,
        height - y4,
        x4,
        result.angle,
        result.barcodeBytes);

    output.add(newResult);
  }

  return output;
}

Future<dynamic> _handleMethod(MethodCall call) async {
    if (call.method == "onBarcodeDetected") {
      barcodeResults =
          convertResults(List<Map<dynamic, dynamic>>.from(call.arguments));
      if (Platform.isAndroid && isPortrait && barcodeResults != null) {
        barcodeResults =
            rotate90barcode(barcodeResults!, _previewHeight.toInt());
      }

      setState(() {});
    }
  }

Draw the barcode results on the camera preview using the CustomPainter class:

@override
Widget build(BuildContext context) {
    final screenWidth = MediaQuery.of(context).size.width;
    final screenHeight = MediaQuery.of(context).size.height;
    var orientation = MediaQuery.of(context).orientation;
    isPortrait = orientation == Orientation.portrait;
    return Scaffold(
    body: _textureId == null
        ? const Center(child: CircularProgressIndicator())
        : SizedBox(
            width: screenWidth,
            height: screenHeight,
            child: FittedBox(
                fit: BoxFit.cover,
                child: Stack(
                children: [
                    SizedBox(
                    width: isPortrait ? _previewHeight : _previewWidth,
                    height: isPortrait ? _previewWidth : _previewHeight,
                    child: Texture(textureId: _textureId!),
                    ),
                    Positioned(
                        left: 0,
                        top: 0,
                        right: 0,
                        bottom: 0,
                        child: createOverlay(
                        barcodeResults,
                        ))
                ],
                ),
            ),
            ),
    );
}

Widget createOverlay(
    List<BarcodeResult>? barcodeResults,
    ) {
    return CustomPaint(
        painter: OverlayPainter(barcodeResults),
    );
    }

    class OverlayPainter extends CustomPainter {
    List<BarcodeResult>? barcodeResults;

    OverlayPainter(this.barcodeResults);

    @override
    void paint(Canvas canvas, Size size) {
        final paint = Paint()
        ..color = Colors.blue
        ..strokeWidth = 5
        ..style = PaintingStyle.stroke;

        if (barcodeResults != null) {
        for (var result in barcodeResults!) {
            double minX = result.x1.toDouble();
            double minY = result.y1.toDouble();
            if (result.x2 < minX) minX = result.x2.toDouble();
            if (result.x3 < minX) minX = result.x3.toDouble();
            if (result.x4 < minX) minX = result.x4.toDouble();
            if (result.y2 < minY) minY = result.y2.toDouble();
            if (result.y3 < minY) minY = result.y3.toDouble();
            if (result.y4 < minY) minY = result.y4.toDouble();

            canvas.drawLine(Offset(result.x1.toDouble(), result.y1.toDouble()),
                Offset(result.x2.toDouble(), result.y2.toDouble()), paint);
            canvas.drawLine(Offset(result.x2.toDouble(), result.y2.toDouble()),
                Offset(result.x3.toDouble(), result.y3.toDouble()), paint);
            canvas.drawLine(Offset(result.x3.toDouble(), result.y3.toDouble()),
                Offset(result.x4.toDouble(), result.y4.toDouble()), paint);
            canvas.drawLine(Offset(result.x4.toDouble(), result.y4.toDouble()),
                Offset(result.x1.toDouble(), result.y1.toDouble()), paint);

            TextPainter textPainter = TextPainter(
            text: TextSpan(
                text: result.text,
                style: const TextStyle(
                color: Colors.yellow,
                fontSize: 22.0,
                ),
            ),
            textAlign: TextAlign.center,
            textDirection: TextDirection.ltr,
            );
            textPainter.layout(minWidth: 0, maxWidth: size.width);
            textPainter.paint(canvas, Offset(minX, minY));
        }
        }
    }

    @override
    bool shouldRepaint(OverlayPainter oldDelegate) => true;
    }

Source Code

https://github.com/yushulx/flutter-barcode-scanner/tree/main/examples/native_camera

DEV Community

Implementing Flutter Barcode Scanner with Kotlin and CameraX for Android

Prerequisites

Step 1: Scaffolding a Flutter Project

Step2: Establishing Communication Between Dart and Kotlin

Step 3: Integrating CameraX into Flutter Android Project

Step 4: Integrating Dynamsoft Barcode Reader SDK

Source Code

Top comments (0)

Read next

Ulyp: Recording Java code execution for faster debugging (Part 1)

Mobile Security Tools part 3: Objection

Override java version on flutter

Kotlin Null Safety vs. Java: A Comedy of Errors (But Mostly in Java)