Learning OCR Process from Scratch – 002: From Image to ONNX Tensor

In the previous post, I explored the basic OCR workflow and discovered that OCR is not simply a single model inference call.

There are multiple stages involved, from text detection and orientation classification to text recognition and decoding.

This time, I wanted to go one level deeper:

What actually happens to the image before it reaches the OCR model?

1. Loading the Image

Since I am implementing the inference pipeline in C++, the first step is loading the image into memory using OpenCV:

cv::Mat image = cv::imread(filename, cv::IMREAD_COLOR);

At this point, we simply have an OpenCV cv::Mat.
However, the ONNX model cannot directly consume this object.
We need to transform the image into the exact tensor representation expected by the model.

2. Resizing the Image

Before inference, the image dimensions also need to be checked.
For the detection model, images that are too large or too small need to be resized according to the preprocessing logic used by the original OCR pipeline.
This is important because the model is trained within a particular input distribution. Feeding images with dimensions that are significantly different from what the preprocessing pipeline expects can affect detection accuracy.
So the image first goes through the appropriate resizing step before being passed to the model.

3. Converting and Normalising the Image

Next, the image needs to be converted from its original data type into float32.
The pixel values are also normalised by dividing them by 255:

cv::Mat floatImage;
image.convertTo(floatImage, CV_32F, 1.0 / 255);

Now the pixel values are represented as floating-point numbers in approximately the range:

[0, 1]

But there is another problem.
The image is still represented as an OpenCV cv::Mat, while the ONNX model expects a tensor.

4. Understanding the ONNX Input

I inspected the detection model using netron:

ch_PP-OCRv4_det_infer.onnx

and found that its input shape is:

[-1, 3, -1, -1]

The -1 dimensions are dynamic, while the 3 represents the colour channels.
For inference, I provide the concrete shape:

[1, 3, H, W]

This follows the common NCHW tensor layout:

N = batch
C = channels
H = height
W = width

So I need to transform the OpenCV image from its image representation into this tensor layout.

5. Converting HWC to CHW

OpenCV stores the image spatially as:

H × W × C

But the ONNX model expects:

C × H × W

Therefore, the channels need to be separated first:

std::vector<cv::Mat> channels;
cv::split(floatImage, channels);

I then allocate a contiguous block of memory for the tensor:

std::vector<float> inputTensorValues(3 * srcH * srcW);

Instead of copying every pixel individually, I can copy each channel directly into the appropriate region of the tensor using std::memcpy and ptr<float>(), which gives me a pointer to the first float element of a specified row. When called without a row index, it points to the first element of the matrix.

std::memcpy(
    inputTensorValues.data(),
    channels[0].ptr<float>(),
    srcH * srcW * sizeof(float)
);

std::memcpy(
    inputTensorValues.data() + srcH * srcW,
    channels[1].ptr<float>(),
    srcH * srcW * sizeof(float)
);

std::memcpy(
    inputTensorValues.data() + 2 * srcH * srcW,
    channels[2].ptr<float>(),
    srcH * srcW * sizeof(float)
);

The offsets are important here.
Each channel contains:

H × W

floating-point values.
Therefore:

• Channel 0 starts at offset 0
• Channel 1 starts at offset H × W
• Channel 2 starts at offset 2 × H × W This gives us a contiguous CHW representation in memory.

6. Defining the Tensor Shape

The data itself is not enough.
ONNX Runtime also needs to know how that block of memory should be interpreted.
So I define the tensor shape:

std::vector<int64_t> inputShape = {1, 3, srcH, srcW};

The shape tells the runtime that the memory represents:

Batch × Channel × Height × Width

7. Creating the ONNX Runtime Tensor

Finally, I need to tell ONNX Runtime how the memory should be handled.
First, I create the memory information:

Ort::MemoryInfo memoryInfo(
    "Cpu",
    OrtDeviceAllocator,
    0,
    OrtMemTypeDefault
);

Then I create the actual tensor:

Ort::Value inputTensor = Ort::Value::CreateTensor<float>(
    memoryInfo,
    inputTensorValues.data(),
    inputTensorValues.size(),
    inputShape.data(),
    inputShape.size()
);

At this point, the original image has gone through the following transformation:

The Interesting Part

What surprised me here was how much work is hidden behind a simple Python inference call.

In Python, the framework handles most of this conversion for me.

In C++, I have to explicitly deal with:

• Image preprocessing
• Data types
• Tensor shapes
• Channel ordering
• Memory layout
• Memory allocation
• Tensor construction

This is also where I started to appreciate the difference between running a model and building an inference pipeline.

The model itself is only one part of the system.

The preprocessing and memory representation have to match what the model expects as well.

In the next step, I can finally pass this tensor into ONNX Runtime and start looking at what actually comes out of the detection model.

Learning OCR Process from Scratch – 002