How to Upload Images in Power Apps for Barcode, MRZ, and Document Detection

In our previous article, we explained how to develop API endpoints for reading barcodes, recognizing MRZ, and rectifying document images in Node.js. In this article, we will integrate these REST API endpoints with the low-code platform Power Apps.

Loading and Displaying Images within Power Apps

1. Click Insert and search for Add picture control, then drag it onto the canvas. This control allows you to upload images from your local disk or capture photos using a camera.

   

   The control is comprised of an AddMediaButton and an image display component.

   

2. Set the Image property of the image control using this formula:
   

   If(IsBlank(AddMediaButton.Media), SampleImage, AddMediaButton.Media)
   

Now, we can upload an image and it will be displayed in the image control. You may need to adjust the position and size of both the button and the image for optimal layout.

Uploading Images from Power Apps to a Self-hosted Node.js Server

Uploading images from Power Apps to a self-hosted Node.js server involves several steps:

1. Convert the image to a base64-encoded string.
2. Establish an endpoint on the Node.js server for receiving the base64-encoded string and saving it as an image file.
3. Develop a Power Automate flow to transmit the base64 string to the server.

How to Convert an Image to a Base64-Encoded String in Power Apps

In Power Apps, the JSON function can be used to convert an image to a base64-encoded string. This is achieved by utilizing the IncludeBinaryData parameter. To perform this conversion and store the result in a variable named ImageJSON, use the following formula:

Set(ImageJSON, JSON(UploadedImage.Image, JSONFormat.IncludeBinaryData));

You can view the generated base64 string by navigating to Variables > ImageJSON:

How to Create an Endpoint for Uploading Images in Node.js

1. Install the Express package:
   

   npm install express
   

2. Create a web server using Express. Here's an example of basic server setup code:
   

   const express = require('express');
   const app = express();
   
   const port = process.env.PORT || 3000;
   
   server.listen(port, '0.0.0.0', () => {
       host = server.address().address;
       console.log(`Server running at http://0.0.0.0:${port}/`);
   });
   

3. Create an endpoint to receive a base64-encoded string and save it as an image file on the server. This typically involves parsing the base64 string and writing it to a file. Here's a basic structure of such an endpoint:
   

   app.post('/dynamsoft/dlr/DetectMrz/base64', (req, res) => {
   let jsonObject = req.body;
   let size = Object.keys(jsonObject).length;
   if (size == 0) {
       return res.status(400).send('No file uploaded.');
   }
   
   Object.keys(jsonObject).forEach(key => {
       let base64Image = jsonObject[key].split(';base64,').pop();
       const imageBuffer = Buffer.from(base64Image, 'base64');
   
       // Write buffer to a file
       const timestamp = Date.now();
       let filePath = 'uploads/' + timestamp;
       fs.writeFile(filePath, imageBuffer, async (err) => {
               if (err) {
                   console.error('Error writing file:', err);
               } else {
                   console.log('File saved:', filePath);
               }
           });
       });
   });
   

   The endpoint is named /dynamsoft/dlr/DetectMrz/base64. This can be modified for various image processing tasks. It's designed to accept a POST request that includes a JSON object in the request body. This JSON object should contain the base64-encoded string of the image. The provided code will save the image in the uploads directory on the server, naming the file with a timestamp.

4. For a quick test, we can use Ngrok to expose the local server to the Internet.
   

   ngrok http 3000
   

   

How to Upload a Base64-Encoded String via Power Automate

1. Open Power Automate and create a new flow.

2. Select Text as the input type.

   

3. Add an HTTP action. Set the Method to POST. For the URI, use the endpoint we created previously. The Body should contain the input text.

   

4. Incorporate a Response action to display the server's response.

   

   Note: The JSON schema for the response needs to be defined according to the server's response format. This will be covered in the following section.

Reading Barcodes, Recognizing MRZ, and Rectifying Documents in Power Apps

First, let's finalize the server-side code by defining three API endpoints for barcode reading, MRZ recognition, and document rectification:

app.post('/dynamsoft/dbr/DecodeBarcode/base64', (req, res) => {
    // ...
});

app.post('/dynamsoft/dlr/DetectMrz/base64', (req, res) => {
    // ...
});

app.post('/dynamsoft/dlr/DetectMrz/base64', (req, res) => {
    // ...
});

For barcode processing: Utilize the decodeBase64Async method to decode the base64-encoded string directly:

let base64Image = jsonObject[key].split(';base64,').pop();
try {
    barcode4nodejs.decodeBase64Async(base64Image, barcode4nodejs.barcodeTypes).then((result) => {
        console.log(result);
        res.status(200).send(result);
    }).catch((err) => {
        console.error(err);
        return res.status(500).send('An error occurred while processing the image.');
    });
}
catch (err) {
    console.error(err);
    return res.status(500).send('An error occurred while processing the image.');
}

For MRZ and document processing: Since the SDKs for MRZ and documents do not have methods to decode base64 strings, first save the base64 string as a file. Then, pass the file path to the respective SDK methods for further processing.

• MRZ Recognition. Recognize the machine-readable zone (MRZ) from an image file and return the parsed results as a JSON string.
  

  try {
      let result = await mrzScanner.decodeFileAsync(filePath);
      let output = "";
      if (result.length == 2) {
          output = mrzScanner.parseTwoLines(result[0].text, result[1].text);
      }
      else if (result.length == 3) {
          output = mrzScanner.parseThreeLines(result[0].text, result[1].text, result[2].text);
      }
      console.log(output);
      res.status(200).send(output);
  }
  catch (err) {
      console.error(err);
      return res.status(500).send('An error occurred while processing the image.');
  }
  

• Document Rectification. Rectify a document image and save it as a JPEG file. Then, return the file path to display the rectified image in Power Apps.
  

  try {
      let results = await docRectifier.detectFileAsync(filePath);
      let result = results[0];
      result = await docRectifier.normalizeFileAsync(filePath, result['x1'], result['y1'], result['x2'], result['y2'], result['x3'], result['y3'], result['x4'], result['y4']);
      let data = result['data']
      let width = result['width']
      let height = result['height']
      for (let i = 0; i < data.length; i += 4) {
          const red = data[i];
          const blue = data[i + 2];
          data[i] = blue;
          data[i + 2] = red;
      }
  
      timestamp = Date.now();
  
      sharp(data, {
          raw: {
              width: width,
              height: height,
              channels: 4
          }
      }).toFile('uploads/' + timestamp + '.jpeg', (err, info) => {
          if (err) {
              console.error('Error:', err);
          } else {
              res.send(JSON.stringify({
                  'image': 'uploads/' + timestamp + '.jpeg'
              }));
          }
      });
  }
  catch (err) {
      console.error(err);
      return res.status(500).send('An error occurred while processing the image.');
  }
  

We create three Power Automate Flows for the three endpoints. While these flows are similar in structure, they differ in their response actions.

• The JSON schema of the barcode reading response is:
  

  {
      "type": "array",
      "items": {
          "type": "object",
          "properties": {
              "format": {
                  "type": "string"
              },
              "value": {
                  "type": "string"
              },
              "x1": {
                  "type": "integer"
              },
              "y1": {
                  "type": "integer"
              },
              "x2": {
                  "type": "integer"
              },
              "y2": {
                  "type": "integer"
              },
              "x3": {
                  "type": "integer"
              },
              "y3": {
                  "type": "integer"
              },
              "x4": {
                  "type": "integer"
              },
              "y4": {
                  "type": "integer"
              },
              "page": {
                  "type": "integer"
              },
              "time": {
                  "type": "integer"
              }
          },
          "required": [
              "format",
              "value",
              "x1",
              "y1",
              "x2",
              "y2",
              "x3",
              "y3",
              "x4",
              "y4",
              "page",
              "time"
          ]
      }
  }
  

• The JSON schema of the MRZ recognition response is:
  

  {
      "type": "object",
      "properties": {
          "type": {
              "type": "string"
          },
          "nationality": {
              "type": "string"
          },
          "surname": {
              "type": "string"
          },
          "givenname": {
              "type": "string"
          },
          "passportnumber": {
              "type": "string"
          },
          "issuecountry": {
              "type": "string"
          },
          "birth": {
              "type": "string"
          },
          "gender": {
              "type": "string"
          },
          "expiry": {
              "type": "string"
          }
      }
  }
  

• The JSON schema of the document rectification response is:
  

  {
      "type": "object",
      "properties": {
          "image": {
              "type": "string"
          }
      }
  }
  

Finally, to complete the app, add more controls to the canvas:

1. Insert a Drop down control and set the Items property to ["Dynamsoft Barcode Reader", "Dynamsoft Label Recognizer", "Dynamsoft Document Normalizer"].

2. Add a Button control and set the OnSelect property to the following formula:
   

   Set(ImageJSON, JSON(UploadedImage.Image, JSONFormat.IncludeBinaryData));
   Switch(DropDown.SelectedText.Value, "Dynamsoft Barcode Reader", ClearCollect(Results, BarcodeUpload.Run(ImageJSON)), "Dynamsoft Label Recognizer", ClearCollect(Results, MrzUpload.Run(ImageJSON)), "Dynamsoft Document Normalizer", Set(URL, DocumentUpload.Run(ImageJSON)))
   

   This formula will call the corresponding Power Automate flow based on the selection in the dropdown control, utilizing the Switch function. The ClearCollect function will store the response in the Results collection, while the Set function will assign the response to the URL variable.

3. Add a Text input control and set the Visible property to the following formula:
   

   If(DropDown.SelectedText.Value = "Dynamsoft Barcode Reader" Or DropDown.SelectedText.Value = "Dynamsoft Label Recognizer", true, false)
   

   Only the results from barcode and MRZ recognition will be displayed in the Text input control. The Default property is set to JSON(Results).

4. Add an Image control and set the Visible property to the following formula:
   

   If(DropDown.SelectedText.Value = "Dynamsoft Document Normalizer", true, false)
   

   The Image property is set to Concatenate("https://563f-98-96-222-66.ngrok-free.app/", URL.image).

Test the Power App

Barcode Detection

MRZ Recognition

Document Rectification

Source Code

https://github.com/yushulx/dynamsoft-sdk-nodejs-REST-API.git