DEV Community

Xiao Ling
Xiao Ling

Posted on • Originally published at dynamsoft.com

How to Build Flutter Document Scanning Plugin for Windows and Linux

#flutter #dart #cpp #programming

Last week, we finished a Flutter document scanning plugin for web, which is just a start. This week, we continue to empower the plugin for desktop development using Dynamsoft Document Normalizer C++ SDK. The plugin will support Windows and Linux. In this article, you will see the steps to build the Flutter desktop document scanning plugin and the differences of Flutter C++ code for Windows and Linux.

Flutter Document Scan SDK

https://pub.dev/packages/flutter_document_scan_sdk

Development Environment

  • Flutter 3.3.9
  • Ubuntu 22.04
  • Windows 10

Add Support for Windows and Linux in Flutter Plugin

To support Windows and Linux, run the following command in the project root directory:

flutter create --org com.dynamsoft --template=plugin --platforms=windows,linux .
Enter fullscreen mode Exit fullscreen mode

After generating the platform-specific code, we update the pubspec.yaml file:

plugin:
    platforms:
      linux:
        pluginClass: FlutterDocumentScanSdkPlugin
      windows:
        pluginClass: FlutterDocumentScanSdkPluginCApi
      web:
        pluginClass: FlutterDocumentScanSdkWeb
        fileName: flutter_document_scan_sdk_web.dart
Enter fullscreen mode Exit fullscreen mode

You should have noticed that the Flutter plugin project contains flutter_document_scan_sdk_web.dart and flutter_document_scan_sdk_method_channel.dart files. The former is used for web. We had added implementations last week. The latter is used for mobile and desktop. It uses method channel to interact with the native platform:

class MethodChannelFlutterDocumentScanSdk
    extends FlutterDocumentScanSdkPlatform {
  @visibleForTesting
  final methodChannel = const MethodChannel('flutter_document_scan_sdk');

  @override
  Future<String?> getPlatformVersion() async {
    final version =
        await methodChannel.invokeMethod<String>('getPlatformVersion');
    return version;
  }

  @override
  Future<int?> init(String path, String key) async {
    return await methodChannel
        .invokeMethod<int>('init', {'path': path, 'key': key});
  }

  @override
  Future<int?> setParameters(String params) async {
    return await methodChannel
        .invokeMethod<int>('setParameters', {'params': params});
  }

  @override
  Future<String?> getParameters() async {
    return await methodChannel.invokeMethod<String>('getParameters');
  }

  @override
  Future<List<DocumentResult>> detect(String file) async {
    List? results = await methodChannel.invokeListMethod<dynamic>(
      'detect',
      {'file': file},
    );

    return _resultWrapper(results);
  }

  List<DocumentResult> _resultWrapper(List<dynamic>? results) {
    List<DocumentResult> output = [];

    if (results != null) {
      for (var result in results) {
        int confidence = result['confidence'];
        List<Offset> offsets = [];
        int x1 = result['x1'];
        int y1 = result['y1'];
        int x2 = result['x2'];
        int y2 = result['y2'];
        int x3 = result['x3'];
        int y3 = result['y3'];
        int x4 = result['x4'];
        int y4 = result['y4'];
        offsets.add(Offset(x1.toDouble(), y1.toDouble()));
        offsets.add(Offset(x2.toDouble(), y2.toDouble()));
        offsets.add(Offset(x3.toDouble(), y3.toDouble()));
        offsets.add(Offset(x4.toDouble(), y4.toDouble()));
        DocumentResult documentResult = DocumentResult(confidence, offsets, []);
        output.add(documentResult);
      }
    }

    return output;
  }

  @override
  Future<NormalizedImage?> normalize(String file, dynamic points) async {
    Offset offset = points[0];
    int x1 = offset.dx.toInt();
    int y1 = offset.dy.toInt();

    offset = points[1];
    int x2 = offset.dx.toInt();
    int y2 = offset.dy.toInt();

    offset = points[2];
    int x3 = offset.dx.toInt();
    int y3 = offset.dy.toInt();

    offset = points[3];
    int x4 = offset.dx.toInt();
    int y4 = offset.dy.toInt();
    Map? result = await methodChannel.invokeMapMethod<String, dynamic>(
      'normalize',
      {
        'file': file,
        'x1': x1,
        'y1': y1,
        'x2': x2,
        'y2': y2,
        'x3': x3,
        'y3': y3,
        'x4': x4,
        'y4': y4
      },
    );

    if (result != null) {
      return NormalizedImage(
        result['data'],
        result['width'],
        result['height'],
      );
    }

    return null;
  }

  @override
  Future<int?> save(String filename) async {
    return await methodChannel
        .invokeMethod<int>('save', {'filename': filename});
  }
}
Enter fullscreen mode Exit fullscreen mode

Linking Libraries and Writing C/C++ Code

Linking Shared Libraries in Flutter Plugin

Flutter uses CMake to build the Windows and Linux plugins. So the first step is to setting up the CMakeLists.txt file. We need to configure link_directories, target_link_libraries, and flutter_document_scan_sdk_bundled_libraries.

Windows

link_directories("${PROJECT_SOURCE_DIR}/lib/") 

target_link_libraries(${PLUGIN_NAME} PRIVATE flutter flutter_wrapper_plugin "DynamsoftCorex64" "DynamsoftDocumentNormalizerx64")

set(flutter_document_scan_sdk_bundled_libraries
  "${PROJECT_SOURCE_DIR}/bin/"
  PARENT_SCOPE
)
Enter fullscreen mode Exit fullscreen mode

Linux

link_directories("${PROJECT_SOURCE_DIR}/lib/") 

target_link_libraries(${PLUGIN_NAME} PRIVATE flutter "DynamsoftCore" "DynamsoftDocumentNormalizer")

set(flutter_document_scan_sdk_bundled_libraries

  PARENT_SCOPE
)
Enter fullscreen mode Exit fullscreen mode

The linking library files of Dynamsoft Document Normalizer SDK are DynamsoftCore and DynamsoftDocumentNormalizer. On Windows, they are named DynamsoftCorex64.lib and DynamsoftDocumentNormalizerx64.lib, whereas on Linux, they are named libDynamsoftCore.so and libDynamsoftDocumentNormalizer.so. To make Windows plugin work, take one more step to bundle the corresponding dll files.

Implementing Flutter C/C++ Code Logic for Windows and Linux

The native entry points of the plugin are FlutterDocumentScanSdkPlugin::HandleMethodCall and flutter_document_scan_sdk_plugin_handle_method_call respectively for Windows and Linux. Because the Flutter header files and data types are different between the two platforms, you have to write code accordingly.

Windows

#include "flutter_document_scan_sdk_plugin.h"

#include <windows.h>

#include <VersionHelpers.h>

#include <flutter/method_channel.h>
#include <flutter/plugin_registrar_windows.h>
#include <flutter/standard_method_codec.h>

#include <memory>
#include <sstream>

void FlutterDocumentScanSdkPlugin::HandleMethodCall(
      const flutter::MethodCall<flutter::EncodableValue> &method_call,
      std::unique_ptr<flutter::MethodResult<flutter::EncodableValue>> result){}
Enter fullscreen mode Exit fullscreen mode

Linux

#include "include/flutter_document_scan_sdk/flutter_document_scan_sdk_plugin.h"

#include <flutter_linux/flutter_linux.h>
#include <gtk/gtk.h>
#include <sys/utsname.h>

#include <cstring>

static void flutter_document_scan_sdk_plugin_handle_method_call(
    FlutterDocumentScanSdkPlugin *self,
    FlMethodCall *method_call){}
Enter fullscreen mode Exit fullscreen mode

To handle the method call, we need to parse the method name and arguments firstly.

Windows

const auto *arguments = std::get_if<EncodableMap>(method_call.arguments());

if (method_call.method_name().compare("init") == 0)
{
  std::string license;
  int ret = 0;

  if (arguments)
  {
    auto license_it = arguments->find(EncodableValue("key"));
    if (license_it != arguments->end())
    {
      license = std::get<std::string>(license_it->second);
    }
    ret = DocumentManager::SetLicense(license.c_str());
  }

  result->Success(EncodableValue(ret));
}
...
Enter fullscreen mode Exit fullscreen mode

Linux

if (strcmp(method, "init") == 0)
{
  if (fl_value_get_type(args) != FL_VALUE_TYPE_MAP)
  {
    return;
  }

  FlValue *value = fl_value_lookup_string(args, "key");
  if (value == nullptr)
  {
    return;
  }
  const char *license = fl_value_get_string(value);

  int ret = DocumentManager::SetLicense(license);
  g_autoptr(FlValue) result = fl_value_new_int(ret);
  response = FL_METHOD_RESPONSE(fl_method_success_response_new(result));
}
...
Enter fullscreen mode Exit fullscreen mode

As you can see, not only the APIs of parsing the method name are different, but also the data type of the arguments. For example, the license argument is std::string on Windows, whereas it is const char* on Linux.

Now, we create a document_manager.h file to implement the methods that calls Dynamsoft Document Normalizer API. Since Dynamsoft Document Normalizer allows you to write code in the same way on Windows and Linux, most of the code can be shared except for wrapping the return values as Flutter data type.

  • Import Dynamsoft Document Normalizer SDK header files: 
  #include "DynamsoftCore.h"
  #include "DynamsoftDocumentNormalizer.h"

  using namespace std;
  using namespace dynamsoft::ddn;
  using namespace dynamsoft::core;
Enter fullscreen mode Exit fullscreen mode
  • Initialize and destroy Dynamsoft Document Normalizer: 
  class DocumentManager
  {
  public:
      ~DocumentManager()
      {
          if (normalizer != NULL)
          {
              DDN_DestroyInstance(normalizer);
              normalizer = NULL;
          }

          FreeImage();
      };

      const char *GetVersion()
      {
          return DDN_GetVersion();
    }

    void Init()
    {
        normalizer = DDN_CreateInstance();
        imageResult = NULL;
    }
    private:
      void *normalizer;
      NormalizedImageResult *imageResult;

      void FreeImage()
      {
          if (imageResult != NULL)
          {
              DDN_FreeNormalizedImageResult(&imageResult);
              imageResult = NULL;
          }
      }
  };
Enter fullscreen mode Exit fullscreen mode 
  static int SetLicense(const char *license)
  {
      char errorMsgBuffer[512];
      // Click https://www.dynamsoft.com/customer/license/trialLicense/?product=ddn to get a trial license.
      int ret = DC_InitLicense(license, errorMsgBuffer, 512);
      if (ret != DM_OK)
      {
          cout << errorMsgBuffer << endl;
      }
      return ret;
  }
Enter fullscreen mode Exit fullscreen mode
  • Detect document edges by calling DDN_DetectQuadFromFile(). On Windows, the detected information is stored in EncodableMap, whereas on Linux, it is stored in FlValue created by fl_value_new_map().

Windows

  EncodableList Detect(const char *filename)
  {
      EncodableList out;
      if (normalizer == NULL)
          return out;

      DetectedQuadResultArray *pResults = NULL;

      int ret = DDN_DetectQuadFromFile(normalizer, filename, "", &pResults);
      if (ret)
      {
          printf("Detection error: %s\n", DC_GetErrorString(ret));
      }

      if (pResults)
      {
          int count = pResults->resultsCount;

          for (int i = 0; i < count; i++)
          {
              EncodableMap map;

              DetectedQuadResult *quadResult = pResults->detectedQuadResults[i];
              int confidence = quadResult->confidenceAsDocumentBoundary;
              DM_Point *points = quadResult->location->points;
              int x1 = points[0].coordinate[0];
              int y1 = points[0].coordinate[1];
              int x2 = points[1].coordinate[0];
              int y2 = points[1].coordinate[1];
              int x3 = points[2].coordinate[0];
              int y3 = points[2].coordinate[1];
              int x4 = points[3].coordinate[0];
              int y4 = points[3].coordinate[1];

              map[EncodableValue("confidence")] = EncodableValue(confidence);
              map[EncodableValue("x1")] = EncodableValue(x1);
              map[EncodableValue("y1")] = EncodableValue(y1);
              map[EncodableValue("x2")] = EncodableValue(x2);
              map[EncodableValue("y2")] = EncodableValue(y2);
              map[EncodableValue("x3")] = EncodableValue(x3);
              map[EncodableValue("y3")] = EncodableValue(y3);
              map[EncodableValue("x4")] = EncodableValue(x4);
              map[EncodableValue("y4")] = EncodableValue(y4);
              out.push_back(map);
          }
      }

      if (pResults != NULL)
          DDN_FreeDetectedQuadResultArray(&pResults);

      return out;
  }
Enter fullscreen mode Exit fullscreen mode

Linux

  FlValue* Detect(const char *filename)
  {
      FlValue* out = fl_value_new_list();
      if (normalizer == NULL)
          return out;

      DetectedQuadResultArray *pResults = NULL;

      int ret = DDN_DetectQuadFromFile(normalizer, filename, "", &pResults);
      if (ret)
      {
          printf("Detection error: %s\n", DC_GetErrorString(ret));
      }

      if (pResults)
      {
          int count = pResults->resultsCount;

          for (int i = 0; i < count; i++)
          {
              FlValue* result = fl_value_new_map ();

              DetectedQuadResult *quadResult = pResults->detectedQuadResults[i];
              int confidence = quadResult->confidenceAsDocumentBoundary;
              DM_Point *points = quadResult->location->points;
              int x1 = points[0].coordinate[0];
              int y1 = points[0].coordinate[1];
              int x2 = points[1].coordinate[0];
              int y2 = points[1].coordinate[1];
              int x3 = points[2].coordinate[0];
              int y3 = points[2].coordinate[1];
              int x4 = points[3].coordinate[0];
              int y4 = points[3].coordinate[1];

              fl_value_set_string_take (result, "confidence", fl_value_new_int(confidence));
              fl_value_set_string_take (result, "x1", fl_value_new_int(x1));
              fl_value_set_string_take (result, "y1", fl_value_new_int(y1));
              fl_value_set_string_take (result, "x2", fl_value_new_int(x2));
              fl_value_set_string_take (result, "y2", fl_value_new_int(y2));
              fl_value_set_string_take (result, "x3", fl_value_new_int(x3));
              fl_value_set_string_take (result, "y3", fl_value_new_int(y3));
              fl_value_set_string_take (result, "x4", fl_value_new_int(x4));
              fl_value_set_string_take (result, "y4", fl_value_new_int(y4));

              fl_value_append_take (out, result);
          }
      }

      if (pResults != NULL)
          DDN_FreeDetectedQuadResultArray(&pResults);

      return out;
  }
Enter fullscreen mode Exit fullscreen mode
  • Call DDN_NormalizeFile() to normalize the document based on detected corner coordinates: 
  Quadrilateral quad;
  quad.points[0].coordinate[0] = x1;
  quad.points[0].coordinate[1] = y1;
  quad.points[1].coordinate[0] = x2;
  quad.points[1].coordinate[1] = y2;
  quad.points[2].coordinate[0] = x3;
  quad.points[2].coordinate[1] = y3;
  quad.points[3].coordinate[0] = x4;
  quad.points[3].coordinate[1] = y4;

  int errorCode = DDN_NormalizeFile(normalizer, filename, "", &quad, &imageResult);
  if (errorCode != DM_OK)
      printf("%s\r\n", DC_GetErrorString(errorCode));

  if (imageResult)
  {
      ImageData *imageData = imageResult->image;
      int width = imageData->width;
      int height = imageData->height;
      int stride = imageData->stride;
      int format = (int)imageData->format;
      unsigned char* data = imageData->bytes;
      int orientation = imageData->orientation;
      int length = imageData->bytesLength;
  }
Enter fullscreen mode Exit fullscreen mode
  • Create a Flutter C++ map object to pass the normalized image data to Flutter:

Windows:

  EncodableMap map;
  map[EncodableValue("width")] = EncodableValue(width);
  map[EncodableValue("height")] = EncodableValue(height);
  map[EncodableValue("stride")] = EncodableValue(stride);
  map[EncodableValue("format")] = EncodableValue(format);
  map[EncodableValue("orientation")] = EncodableValue(orientation);
  map[EncodableValue("length")] = EncodableValue(length);
  std::vector<uint8_t> rawBytes(rgba, rgba + width * height * 4);
  map[EncodableValue("data")] = rawBytes;
Enter fullscreen mode Exit fullscreen mode

Linux:

  FlValue* result = fl_value_new_map ();
  fl_value_set_string_take (result, "width", fl_value_new_int(width));
  fl_value_set_string_take (result, "height", fl_value_new_int(height));
  fl_value_set_string_take (result, "stride", fl_value_new_int(stride));
  fl_value_set_string_take (result, "format", fl_value_new_int(format));
  fl_value_set_string_take (result, "orientation", fl_value_new_int(orientation));
  fl_value_set_string_take (result, "length", fl_value_new_int(length));
  fl_value_set_string_take (result, "data", fl_value_new_uint8_list(rgba, width * height * 4));
Enter fullscreen mode Exit fullscreen mode

Note: According to the image color format, we need to do the following conversion before wrapping the data:

  • If the output image is black and white, each byte represents 8 pixels. Each bit of a byte needs to be converted to a byte as 0 or 255. The final length of the image bytes should be the same as the grayscale image.

    unsigned char *grayscale = new unsigned char[width * height];
binary2grayscale(data, grayscale, width, height, stride, length);

void binary2grayscale(unsigned char* data, unsigned char* output, int width, int height, int stride, int length) 
{
    int index = 0;

    int skip = stride * 8 - width;
    int shift = 0;
    int n = 1;

    for (int i = 0; i < length; ++i)
    {
        unsigned char b = data[i];
        int byteCount = 7;
        while (byteCount >= 0)
        {
            int tmp = (b & (1 << byteCount)) >> byteCount;

            if (shift < stride * 8 * n - skip) {
                if (tmp == 1)
                    output[index] = 255;
                else
                    output[index] = 0;
                index += 1;
            }

            byteCount -= 1;
            shift += 1;
        }

        if (shift == stride * 8 * n) {
            n += 1;
        }
    }
}

  • The format of pixel data given to the Flutter decodeImageFromPixels() method only supports rgba8888 and bgra8888. So, we need to construct the pixel data in the format of rgba8888 or bgra8888 based on the output image color format.

    unsigned char *rgba = new unsigned char[width * height * 4];
memset(rgba, 0, width * height * 4);
if (format == IPF_RGB_888)
{
    int dataIndex = 0;
    for (int i = 0; i < height; i++)
    {
        for (int j = 0; j < width; j++)
        {
            int index = i * width + j;

            rgba[index * 4] = data[dataIndex + 2];     // red
            rgba[index * 4 + 1] = data[dataIndex + 1]; // green
            rgba[index * 4 + 2] = data[dataIndex];     // blue
            rgba[index * 4 + 3] = 255;                 // alpha
            dataIndex += 3;
        }
    }
}
else if (format == IPF_GRAYSCALED)
{
    int dataIndex = 0;
    for (int i = 0; i < height; i++)
    {
        for (int j = 0; j < width; j++)
        {
            int index = i * width + j;
            rgba[index * 4] = data[dataIndex];
            rgba[index * 4 + 1] = data[dataIndex];
            rgba[index * 4 + 2] = data[dataIndex];
            rgba[index * 4 + 3] = 255;
            dataIndex += 1;
        }
    }
}
else if (format == IPF_BINARY)
{
    unsigned char *grayscale = new unsigned char[width * height];
    binary2grayscale(data, grayscale, width, height, stride, length);

    int dataIndex = 0;
    for (int i = 0; i < height; i++)
    {
        for (int j = 0; j < width; j++)
        {
            int index = i * width + j;
            rgba[index * 4] = grayscale[dataIndex];
            rgba[index * 4 + 1] = grayscale[dataIndex];
            rgba[index * 4 + 2] = grayscale[dataIndex];
            rgba[index * 4 + 3] = 255;
            dataIndex += 1;
        }
    }

    free(grayscale);
}

So far, we have completed the implementation of the Flutter Document Scan SDK plugin for Windows and Linux. The next step is to test the plugin on desktop platforms.

Test Flutter Document Scan SDK Plugin on Windows and Linux

Since the example code we implemented for web is shared with Windows and Linux, there is no code change required. We can directly run the example code on Windows and Linux.

Windows

flutter run -d windows
Enter fullscreen mode Exit fullscreen mode

Flutter windows document edge detection and normalization

Linux

flutter run -d linux
Enter fullscreen mode Exit fullscreen mode

Flutter Linux document edge detection and normalization

Source Code

https://github.com/yushulx/flutter_document_scan_sdk

Top comments (0)

Read next

techpulse55 profile image

A Beginner’s Guide to Machine Learning

Techpulse -

jennife05918349 profile image

Guide To Cloud Computing in Banking

Jennifer Smith -

skytin1004 profile image

Automating Markdown and Image Translations with Co-op Translator

송민석 -

guybuildingai profile image

🚨🏆 Top 5 Open-source Alternatives for LLM Development You Must Know About 💥

Jeffrey Ip -