Learning OCR Process from Scratch – 004: Parsing OCR Detection Output

In the previous post, I ran the PP-OCRv4 detection model and obtained its raw probability map.
But this probability map is not yet a list of text boxes.
Now we need to turn that prediction map into actual detection regions.

The basic flow is:
Probability Map → Thresholding → Contours → Box Filtering → Box Expansion → Cropped Text Region

1. Filtering Low-Confidence Regions

The first step is to convert the probability map into a binary image.

cv::Mat bitmap;
cv::threshold(pred,bitmap,thresh,255,cv::THRESH_BINARY);

Here, thresh is the threshold used to separate likely text regions from the background.

Pixels above the threshold become 255, while the remaining pixels become 0.

This gives us a binary mask of the regions where the model believes text exists.

2. Finding the Contours

The binary image can now be used to find connected regions:

bitmap.convertTo(bitmap, CV_8U);
std::vector<std::vector<cv::Point>> contours;
cv::findContours(bitmap * 255,contours,cv::RETR_LIST,cv::CHAIN_APPROX_SIMPLE);

The contour representation is:

vector<vector<Point>>

Each vector<Point> represents one contour, while each Point represents a coordinate along that contour.

There are two parameters here that are important.

cv::RETR_LIST tells OpenCV to retrieve all contours without building a parent-child hierarchy between them.

cv::CHAIN_APPROX_SIMPLE compresses the contour representation by removing redundant points. For example, points along a straight line do not all need to be stored.

At this point, the probability map has been transformed into a collection of geometric regions.

3. Filtering Small and Low-Confidence Regions

Not every detected region should become a text box.

I first filter out regions that are too small, such as contours with a width or height below 3 pixels.

I also calculate the confidence score of each region and discard boxes with a score below my threshold of 0.7.

This helps remove noise and tiny regions that are unlikely to represent actual text.

4. Ordering the Box Points

After extracting a rectangular detection region, I need to make sure its four corner points are in a consistent order.
The points are first sorted by their x-coordinate:

std::sort(
    boxPoints.begin(),
    boxPoints.end(),
    [](const cv::Point2f &a, const cv::Point2f &b) {
        return a.x < b.x;
    }
);

From there, I determine which points correspond to the top-left, top-right, bottom-left, and bottom-right corners. Which is important for detection regions to be cropped later on.

std::vector<cv::Point2f> box = {
    boxPoints[index1],
    boxPoints[index2],
    boxPoints[index3],
    boxPoints[index4]
};

5. Calculating the Detection Confidence

The contour itself does not directly tell me how confident the model is about the detected region.

Instead, I calculate the average probability inside the detected polygon.

First, I create a mask and fill the detected polygon:

cv::fillPoly(
    mask,
    std::vector<std::vector<cv::Point>>{polygon},
    cv::Scalar(255)
);

Then I calculate the mean value of the original probability map inside that mask:

float score = static_cast<float>(cv::mean(pred, mask)[0]);
if (score < boxThresh)
{continue;}

This gives me an average confidence score for the detected region.

If the score is too low, the detection is discarded.

This is different from the thresholding step earlier.

The first threshold creates candidate regions from the probability map. This second score checks how confident the model is about the entire candidate region.

6. Expanding the Detection Box

The detected contour usually does not perfectly cover the complete text.
It can be slightly smaller than the actual text region, so RapidOCR expands the box before cropping it.
The expansion distance is calculated from the rectangle's area and perimeter:

float area = rect.size.width * rect.size.height;

float perimeter = 2.0f * (rect.size.width +rect.size.height);

float distance = area * unclip_ratio / perimeter;

The rectangle is then expanded:

rect.size.width += 2.0f * distance;
rect.size.height += 2.0f * distance;

Finally, the expanded rectangle is converted back into four cornerpoints:

cv::Point2f expandedPoints[4];

rect.points(expandedPoints);

std::vector<cv::Point2f> expandedBoxPoints(
    expandedPoints,
    expandedPoints + 4
);

The points are then sorted again into a consistent order.

This expansion is useful because a detection that is too tightly cropped can cut off parts of the characters, which can negatively affect the recognition model.

7. From Detection to Recognition

At this point, the detection stage is essentially complete.
These cropped regions can now be passed to the next stages of the OCR pipeline, such as orientation classification and text recognition.

What I Learned

This was another step where I realised that the neural network is only part of the OCR system.

The detection model gives us a probability map, but it does not directly give us the final bounding boxes.

The rest is classical computer vision and geometry processing:

Thresholding → Contour Extraction → Filtering → Polygon Scoring → Box Ordering → Box Expansion

So the OCR detection stage is really a combination of deep learning and traditional computer vision.

The next step is to take these cropped regions and understand how the recognition model turns them into actual characters.

Learning OCR Process from Scratch – 004