Cropping Text Areas from Images After Text Recognition

Read the original article：Cropping Text Areas from Images After Text Recognition

Context

After using textRecognition to identify text content in images, there's often a need to extract or crop the specific areas containing the recognized text from the original image for further processing or display purposes.

Description

The process involves two main steps:

1. Using textRecognition API to identify text and obtain corner point coordinates
2. Using the crop functionality to extract the text area from the original pixelMap

The textRecognition API returns corner points in pixel coordinates that define the bounding box of recognized text areas, which can then be used to crop the corresponding regions from the original image.

Solution

Step 1: Text Recognition and Corner Point Extraction

textRecognition.recognizeText(visionInfo, textConfiguration)
  .then((data: textRecognition.TextRecognitionResult) => {
    // Recognition successful, get the corresponding result 
    let recognitionString = JSON.stringify(data);
    hilog.info(0x0000, 'OCRDemo', `Succeeded in recognizing text: ${recognitionString}`);

    // Update the result to Text to display
    this.dataValues = data.value;

    // Get the corner points of the recognized text
    let cornerPoints = data.blocks[0].lines[0].cornerPoints;

    // Extract coordinates for cropping
    let pointx = cornerPoints[0].x;
    let pointy = cornerPoints[0].y;
    let width = cornerPoints[2].x - cornerPoints[0].x;
    let height = cornerPoints[3].y - cornerPoints[0].y;

    // Proceed to crop the image
    this.cropTextArea(pointx, pointy, width, height);
  })
  .catch((error: BusinessError) => {
    hilog.error(0x0000, 'OCRDemo', `Failed to recognize text. Code: ${error.code}, message: ${error.message}`);
    this.dataValues = `Error: ${error.message}`;
  });

Step 2: Image Cropping Using Corner Points

async function cropTextArea(x: number, y: number, width: number, height: number) {
  let region: image.Region = { 
    x: x, 
    y: y, 
    size: { height: height, width: width } 
  };

  if (pixelMap != undefined) {
    pixelMap.crop(region, (err: BusinessError) => {
      if (err) {
        console.error(`Failed to crop pixelmap. code is ${err.code}, message is ${err.message}`);
        return;
      } else {
        console.info("Succeeded in cropping pixelmap.");
        // The pixelMap now contains only the cropped text area
      }
    });
  }
}

Complete Integration Example

import { BusinessError } from '@kit.BasicServicesKit';

async function processTextRecognitionAndCrop() {
  try {
    const data = await textRecognition.recognizeText(visionInfo, textConfiguration);

    // Process each recognized text block
    for (let block of data.blocks) {
      for (let line of block.lines) {
        let cornerPoints = line.cornerPoints;

        // Calculate region coordinates
        let region: image.Region = {
          x: cornerPoints[0].x,
          y: cornerPoints[0].y,
          size: {
            width: cornerPoints[2].x - cornerPoints[0].x,
            height: cornerPoints[3].y - cornerPoints[0].y
          }
        };

        // Crop the text area
        if (pixelMap) {
          await new Promise<void>((resolve, reject) => {
            pixelMap.crop(region, (err: BusinessError) => {
              if (err) {
                reject(err);
              } else {
                resolve();
              }
            });
          });
        }
      }
    }
  } catch (error) {
    console.error('Text recognition or cropping failed:', error);
  }
}

Key Takeaways

• Text recognition returns corner points in pixel coordinates that define text bounding boxes
• The crop function requires a Region object with x, y coordinates and size dimensions
• Corner points array provides four coordinates: [top-left, top-right, bottom-right, bottom-left]
• Width is calculated as: cornerPoints[2].x - cornerPoints[0].x
• Height is calculated as: cornerPoints[3].y - cornerPoints[0].y
• Handle multiple text blocks and lines if your use case requires processing all recognized text areas
• Always implement proper error handling for both text recognition and cropping operations

Written by Emincan Ozcan