A research-based learning journey exploring how computers process and interpret images, from pixels and RGB colour channels to mathematical kernels, image filters, object classification, detection, and segmentation. In this blog, I share the concepts I researched, the mathematical foundations I explored, and the key insights I gained while learning how computer vision works.
1. Introduction
Computer vision is a field of artificial intelligence that enables computers to extract useful information from images and videos. A computer does not see an image as a person does; instead, it receives numerical values arranged in a grid.
Image processing applies mathematical operations to these values to improve an image or reveal features such as edges and textures. Deep-learning models can then use these features for tasks such as image classification, object detection, and image segmentation (Gonzalez & Woods, 2018; Szeliski, 2022).
2. Understanding Pixels and RGB Images
2.1 What is a pixel?
A pixel, meaning picture element, is the smallest individual sample in a digital image. An image consists of many pixels arranged in rows and columns.
In an 8-bit grayscale image, each pixel usually stores an intensity value between 0 and 255:
0: Black
255: White
Values between 0 and 255: Different shades of gray
A grayscale image can be represented mathematically as: I(x,y)
Where:
(I) represents the image.
(x) represents the row position.
(y) represents the column position.
(I(x,y)) represents the intensity of the pixel at that position.
2.2 Understanding RGB images
RGB stands for Red, Green, and Blue. It is a colour model that represents colours by combining three colour channels.
Each pixel in a standard 8-bit RGB image contains three values:
I(x,y)= [R(x,y), G(x,y), B(x,y)]
Where:
R(x,y) represents the red intensity.
G(x,y) represents the green intensity.
B(x,y) represents the blue intensity.
Each channel usually ranges from 0 to 255.
Examples:
RGB value Colour
(255, 0, 0) Red
(0, 255, 0) Green
(0, 0, 255) Blue
(255, 255, 255) White
(0, 0, 0) Black
(255, 255, 0) Yellow
An RGB image can therefore be represented as a three-dimensional array with dimensions: H×W×3
Where H is the image height, W is its width, and 3 represents the RGB channels.
2.3 Converting RGB to grayscale
Sometimes, image processing algorithms operate on grayscale images because they need intensity information rather than colour information.
A common luminance-weighted conversion formula is:
(Gray=0.299R+0.587G+0.114B)
Where (R), (G), and (B) are the red, green, and blue channel values.
Example:
Suppose a pixel has RGB values:
R=100, G=150, B=200
Substituting these values:
Gray = 0.299(100) + 0.587(150) + 0.114(200)
Gray = 29.9 + 88.05 + 22.8
Gray = 140.75 ~ 141
The resulting grayscale pixel has an intensity of approximately 141.
This weighted formula is associated with standard video luminance calculations, including ITU-R BT.601 (International Telecommunication Union, 2011).
3. Understanding Kernels and Filters
3.1 What is a kernel?
A kernel, also called a filter, is a small matrix of numerical values applied to an image to modify its appearance or extract useful features.
For example, a 3×3 kernel contains nine values:
K = [ 1 1 1
1 1 1
1 1 1 ]
The kernel moves across the image. At each position, its values are multiplied by the corresponding neighbouring pixel values, and the results are added together.
This operation is used for blurring, sharpening, edge detection, and other image-processing tasks.
3.2 General mathematical formula for convolution
A two-dimensional convolution can be expressed as:
O(x, y) = Σᵢ Σⱼ K(i, j) I(x − i, y − j)
Where:
O(x,y) is the output pixel.
I(x,y) **is the input image.
**K(i,j) represents the kernel values.
i,j represent positions within the kernel.
∑ indicates summation over the kernel positions.
Strict mathematical convolution flips the kernel before applying it. Some image-processing libraries instead perform cross-correlation, which uses the kernel without flipping it.
3.3 Common image kernels and filters
A. Average blur filter
Kernel
** [ 1 1 1 ]
K = 1/9 [ 1 1 1 ]
[ 1 1 1 ]**
This filter calculates the average of a pixel and its eight neighbours. It smooths an image and reduces small details and noise.
The output pixel is:
O(x, y) = (1/9) Σᵢ₌₋₁¹ Σⱼ₌₋₁¹ I(x + i, y + j)
B. Large blur filter (15 × 15)
Kernel
K = (1/225) × a 15 × 15 matrix in which every entry is 1
This filter averages 225 neighbouring pixels. It produces stronger smoothing than a 3×3 blur and removes more fine details.
The output pixel is:
O(x, y) = (1/225) Σᵢ₌₋₇⁷ Σⱼ₌₋₇⁷ I(x + i, y + j)
C. Sharpening filter
Kernel
K = [ 0 -1 0
-1 5 -1
0 -1 0 ]
The sharpening filter strengthens the central pixel and subtracts the four directly adjacent pixels. This emphasizes local changes and makes details appear sharper.
The output is:
O(x, y) = 5I(x, y)
− I(x − 1, y) − I(x + 1, y)
− I(x, y − 1) − I(x, y + 1)
D. Sobel X filter
Kernel
Gₓ = [ -1 0 1
-2 0 2
-1 0 1 ]
The Sobel X filter estimates intensity changes along the horizontal direction. It commonly highlights vertical edges in an image.
The response is:
Gₓ = Kₓ * I
Here, ***** denotes convolution.
E. Sobel Y filter
Kernel
Gᵧ = [ -1 -2 -1
0 0 0
1 2 1 ]
The Sobel Y filter estimates intensity changes along the vertical direction. It commonly highlights horizontal edges.
The response is:
Gᵧ = Kᵧ * I
F. Gradient magnitude filter
After calculating the Sobel X and Sobel Y responses, they can be combined to estimate the strength of an edge.
G = √(Gₓ² + Gᵧ²) **
Where:
**Gₓ is the horizontal gradient response.
Gᵧ is the vertical gradient response.
** G **is the gradient magnitude.
Larger gradient magnitudes indicate stronger intensity changes, which often correspond to image edges.
3.4 Summary of common filters
Filter Main purpose
Average blur Smooths an image and reduces small
details.
Large blur Produces stronger smoothing.
Sharpening Emphasizes details and local
changes.
Sobel X Highlights predominantly vertical
edges.
Sobel Y Highlights predominantly horizontal
edges.
Gradient magnitude Combines edge responses to estimate
edge strength.
4. From Image Processing to Computer Vision
Traditional image-processing filters operate directly on pixel values. Modern computer-vision systems often use convolutional neural networks (CNNs), which learn useful filters from training data.
Early network layers may learn edges and textures, while deeper layers combine these features into more complex patterns.
Three important computer-vision tasks are image classification, object detection, and image segmentation.
4.1 Image classification
Question: What is in the image?
Image classification assigns one or more labels to an entire image.
For example, a model might classify an image as:
A dog
A car
A building
A landscape
Classification generally does not identify the precise location of each object.
4.2 Object detection
Question: What objects are present, and where are they located?
Object detection identifies objects and estimates their locations, commonly using bounding boxes.
For example, a traffic image might contain:
Three cars
Two pedestrians
One bicycle
The model returns the object classes and their corresponding locations.
4.3 Image segmentation
Question: Which pixels belong to each object or region?
Image segmentation assigns labels to individual pixels, allowing a computer to identify the precise regions occupied by objects.
Two common forms are:
Semantic segmentation: Assigns a class to every pixel, such as road, building, person, or vehicle.
Instance segmentation: Separates individual objects, including different objects belonging to the same class.
For example, in a street photograph, segmentation can label the pixels belonging to roads, buildings, vehicles, and pedestrians.
4.4 Comparing the three tasks
Task Output Example
Classification Image-level label "This is a street
scene."
Object detection Labels and bounding "There is a car at this
boxes location."
Segmentation Pixel-level labels or "These pixels belong to
masks the car."
The main difference is the level of spatial detail: classification identifies the overall content, detection locates objects, and segmentation identifies their regions at pixel level.
5. What I Learned
All of these things I learned about computer vision helped me understand how computers process and interpret images, from analyzing individual pixels and RGB colour channels to applying mathematical kernels and filters to extract useful visual information.
Through these concepts and practical experiments, I learned how image filters can smooth images, sharpen details, and detect edges. I also understood the differences between image classification, object detection, and image segmentation, and how each technique helps computers understand visual information at a different level.
Working with pretrained models also helped me recognize their limitations. I learned that models can produce incorrect predictions when an image contains information that differs from the patterns or categories learned during training. For this reason, confidence scores should not automatically be considered proof that a prediction is correct.
Most importantly, this learning experience helped me connect mathematical concepts with practical applications. I realized that understanding computer vision requires not only learning the theory but also experimenting with real images, analyzing model outputs, and critically evaluating the results.
6. Conclusion
Pixels and colour channels form the numerical foundation of digital images. RGB represents colour using red, green, and blue intensity values, while grayscale represents image intensity using a single channel.
Kernels and filters apply mathematical operations to local groups of pixels. Blur filters smooth images, sharpening filters emphasize details, and Sobel filters estimate edge information.
These fundamental operations help explain how computers process visual information. Building on this foundation, image classification identifies image content, object detection locates objects, and image segmentation assigns labels to individual pixels.
Understanding these concepts provides a strong foundation for studying image processing, convolutional neural networks, and modern computer-vision applications.
7. References
Gonzalez, R. C., & Woods, R. E. (2018). Digital image processing (4th ed.). Pearson.
International Telecommunication Union. (2011). Studio encoding parameters of digital television for standard 4:3 and wide screen 16:9 aspect ratios (ITU-R Recommendation BT.601). https://www.itu.int/rec/R-REC-BT.601
Sobel, I., & Feldman, G. (1968). A 3×3 isotropic gradient operator for image processing. Stanford Artificial Intelligence Project.
Szeliski, R. (2022). Computer vision: Algorithms and applications (2nd ed.). Springer. https://szeliski.org/Book/
Top comments (0)