From Pixels to Predictions: Exploring Computer Vision with Python

When I first started learning computer vision, I thought it was simply about teaching computers to recognize images. However, as I explored the concepts and experimented with Python, I discovered that there is much more happening behind the scenes.

How does a computer represent an image? How can it detect the edges of an object? How does a model recognize a person or a vehicle? And how can it identify the exact pixels belonging to an object rather than just drawing a rectangle around it?

In this article, I will share what I have learned about computer vision so far, including image representation, filters, image classification, object detection, and segmentation. I will also share some practical experiments using Python, NumPy, Matplotlib, and YOLO.

My goal is to explain these concepts in a way that is easy for other beginners to understand.

1. What Is Computer Vision?

Computer vision is a field of artificial intelligence (AI) that enables computers to analyze and interpret visual information, such as images and videos.

Humans can look at a photograph and immediately recognize people, vehicles, buildings, or animals. Computers do not naturally understand images in the same way. They process numerical representations of visual information and use algorithms or trained models to identify patterns and make predictions.

Computer vision has many real-world applications, including:

• Self-driving cars that detect pedestrians, traffic signs, and other vehicles.
• Medical imaging systems that help identify abnormalities.
• Security systems that recognize or track objects.
• Agriculture systems that detect crop diseases.
• Satellite imagery systems that identify buildings and roads.
• Retail systems that recognize and count products.

Understanding computer vision starts with a fundamental question: How does a computer see an image?

2. Understanding Pixels: How Computers Represent Images

An image may look like a continuous photograph to us, but a computer represents a digital image as a grid of tiny elements called pixels.

Each pixel stores numerical information about the color or brightness at a particular position in the image.

For example, suppose we have an image that is 400 pixels high and 600 pixels wide.

The total number of pixels is:

400 × 600 = 240,000 pixels.

If the image uses RGB color, each pixel has three color channels:

• Red (R)
• Green (G)
• Blue (B)

Each channel stores a value representing the intensity of that color. In a standard 8-bit image, these values usually range from 0 to 255.

For example:

• (255, 0, 0) represents red.
• (0, 255, 0) represents green.
• (0, 0, 255) represents blue.
• (255, 255, 255) represents white.
• (0, 0, 0) represents black.

Since our example image contains 240,000 pixels and each pixel has three color-channel values, it contains:

240,000 × 3 = 720,000 channel values.

The important distinction is that the image still contains 240,000 pixels, not 720,000 pixels. The larger number represents the individual red, green, and blue values stored across those pixels.

This helped me understand that computers process images as numerical data rather than as pictures in the human sense.

3. Exploring Images with Python

Python provides useful libraries for working with images. Two important tools I explored are NumPy and Matplotlib.

NumPy allows us to manipulate numerical arrays, while Matplotlib helps us display images and visualize results.

For example, an RGB image can be represented as a three-dimensional array:

import numpy as np
import matplotlib.pyplot as plt
from PIL import Image

# Load an image
img = Image.open("image.jpg").convert("RGB")

# Convert the image into a NumPy array
img_array = np.array(img)

# Display image information
print("Image shape:", img_array.shape)
print("Total pixels:", img_array.shape[0] * img_array.shape[1])

# Display the image
plt.imshow(img_array)
plt.axis("off")
plt.show()

If the image is 400 pixels high and 600 pixels wide, its shape will be:

(400, 600, 3)

The first number represents the height, the second represents the width, and the third represents the three RGB channels.

I also learned that we can manipulate individual color channels. For example:

red_removed = img_array.copy()
red_removed[:, :, 0] = 0

plt.imshow(red_removed)
plt.axis("off")
plt.show()

Here, [:, :, 0] selects the red channel, and assigning zero removes the red contribution from every pixel.

The resulting image looks different because we have changed the numerical information used to represent its colors.

This experiment demonstrated how simple array operations can directly change an image's appearance.

4. Image Filters: How Computers Extract Visual Information

After understanding pixels, I moved on to image filters.

An image filter is a mathematical operation applied to an image to change its appearance or emphasize certain visual features.

Many filters use a small matrix called a kernel or filter. This kernel moves across the image, examining small neighborhoods of pixels and calculating new values.

Depending on the kernel, the operation can blur an image, sharpen details, or highlight edges.

4.1 Blurring an image

Blurring reduces fine details and makes an image look smoother.

One way to do this is to calculate the average of the pixel values in a small neighborhood.

For example:

blur_kernel = np.ones((5, 5)) / 25

print(blur_kernel)

This creates a 5 × 5 matrix containing 25 equal weights, each equal to 0.04.

When applied as a filter, each output pixel is calculated from the average of the corresponding 25-pixel neighborhood, assuming the usual uniform averaging operation.

The effect is that sudden changes in brightness and small details become less pronounced.

4.2 Sharpening an image

Sharpening emphasizes differences between a pixel and its neighbors, making edges and details appear more distinct.

For example:

sharpen_kernel = np.array([
    [ 0, -1,  0],
    [-1,  5, -1],
    [ 0, -1,  0]
])

This kernel gives the center pixel a weight of 5 and subtracts the values of its four direct neighbors.

Conceptually, the calculation is:

New pixel value = 5 × center − left − right − top − bottom.

This can emphasize details, although it can also amplify noise. The resulting values may fall outside the normal 0–255 range, so they may need to be clipped or otherwise converted appropriately.

4.3 Detecting edges with Sobel filters

Edges are important because they often indicate boundaries between objects or regions with different brightness levels.

I also learned about Sobel filters, which help detect brightness changes in particular directions.

A Sobel Y kernel can be written as:

sobel_y = np.array([
    [-1, -2, -1],
    [ 0,  0,  0],
    [ 1,  2,  1]
])

This kernel measures brightness changes from the top toward the bottom of an image. As a result, it emphasizes horizontal edges.

A Sobel X kernel measures changes from left to right and emphasizes vertical edges.

These operations helped me understand that computers can extract useful visual patterns from images through mathematical calculations.

It is also important to note that traditional filters and modern deep learning models are not the same thing. Filters such as Sobel use predefined mathematical operations, whereas a convolutional neural network can learn useful filters from training data.

5. Understanding Convolutional Neural Networks (CNNs)

As I continued learning, I encountered convolutional neural networks, commonly known as CNNs.

A CNN is a type of neural network widely used in image-related tasks. It learns patterns from image data, often starting with simple features such as edges and textures and building toward more complex patterns.

For example, a CNN might learn to recognize:

1. Simple visual patterns, such as edges and lines.
2. More complex features, such as shapes and textures.
3. Combinations of features associated with objects, such as faces, vehicles, or animals.

Unlike a manually designed Sobel kernel, the filters in a CNN are typically learned during training.

CNNs are used in tasks such as image classification, object detection, and segmentation.

This introduced an important distinction in my learning: traditional image processing applies operations that people explicitly define, while deep learning can learn useful visual patterns from examples.

6. Image Classification: What Is in This Image?

My first practical experiment with YOLO focused on image classification.

Image classification answers a simple question:

What does this image contain?

For example, given a photograph of a dog, a classification model might predict:

dog             92%
wolf             5%
fox              3%

These values are illustrative. They represent possible model scores, not results from a particular experiment.

For my experiment, I used a pretrained YOLO classification model with the Ultralytics Python library.

from ultralytics import YOLO

# Load a pretrained classification model
classifier = YOLO("yolo11n-cls.pt")

# Classify an image
results = classifier("image.jpg", verbose=False)

# Get the first result
r = results[0]

# Display the top five predictions
for idx in r.probs.top5[:5]:
    name = r.names[idx]
    confidence = r.probs.data[idx].item()

    print(f"{name:<25} {confidence:.1%}")

The filename yolo11n-cls.pt identifies a YOLO11 nano classification model. The -cls suffix indicates that it is designed for classification.

In this example:

• classifier(...) runs the model on the image.
• r.probs contains the classification scores.
• r.probs.top5 provides the indices of the five highest-scoring classes.
• r.names[idx] retrieves the corresponding class name.

The pretrained classification model used in this experiment is trained on ImageNet classes, so its predictions are limited to the categories it knows.

One important lesson was that a model's prediction is not always correct. A model may assign a high score to an incorrect class, especially when an image contains unfamiliar objects or differs from the data used during training.

Classification is useful when we need an overall label for an image, but it does not tell us where an object is located.

7. Object Detection: What Is in the Image, and Where Is It?

Object detection goes one step further than classification.

Instead of only predicting which objects are present, a detection model identifies individual objects, predicts their classes, and estimates their locations using bounding boxes.

A bounding box is a rectangle drawn around a detected object.

For example, a street photograph may contain a bus, two people, and a car. A detection model can draw separate boxes around these objects and provide a confidence score for each detection.

I used the following YOLO model:

from ultralytics import YOLO

# Load the pretrained object detection model
detector = YOLO("yolo11n.pt")

# Detect objects in an image
results = detector("image.jpg", conf=0.4)

# Display the annotated image
results[0].show()

Unlike the classification model, yolo11n.pt is the object detection version.

The conf=0.4 argument sets a confidence threshold of 40%. Detections below the configured threshold are generally filtered out.

I also explored how to retrieve information about each detected object:

r = results[0]

for box in r.boxes:
    class_id = int(box.cls[0])
    confidence = float(box.conf[0])
    label = r.names[class_id]

    x1, y1, x2, y2 = box.xyxy[0].int().tolist()

    print(
        f"{label}: {confidence:.1%} "
        f"box=({x1}, {y1})-({x2}, {y2})"
    )

The output provides three important pieces of information:

• Class: the predicted object category.
• Confidence: the model's confidence score for that detection.
• Coordinates: the estimated location of the bounding box.

The coordinates are represented as (x1, y1, x2, y2), where the first pair identifies the top-left corner and the second pair identifies the bottom-right corner. The x-coordinate increases from left to right, while the y-coordinate increases from top to bottom.

The width and height can be calculated as:

Width = x2 − x1

Height = y2 − y1

Experimenting with confidence thresholds

I also explored how changing the confidence threshold affects the detections.

# Lower threshold
results_low = detector("image.jpg", conf=0.1)

# Higher threshold
results_high = detector("image.jpg", conf=0.9)

A lower threshold can retain more detections, including uncertain or incorrect predictions. A higher threshold generally retains fewer detections, but the remaining detections have higher model scores.

This is a trade-off between retaining potential objects and filtering out uncertain predictions. A high confidence score does not guarantee that a detection is correct.

The YOLO model used here is pretrained on the COCO dataset, which contains 80 object categories. It cannot automatically recognize every possible object in the world.

8. Image Segmentation: Which Pixels Belong to Each Object?

The next step in my learning journey was segmentation.

Segmentation provides more detailed information than a bounding box. It identifies the pixels associated with an object, allowing the model to follow its shape more closely.

For example, a bounding box around a person may also include some of the surrounding road or background. A segmentation mask can outline the person more precisely.

Segmentation is useful in applications such as:

• Medical imaging, where models can outline organs or potential abnormalities.
• Satellite imagery, where models can identify buildings, roads, or agricultural areas.
• Autonomous driving, where systems can distinguish roads, vehicles, pedestrians, and other scene elements.
• Image editing, where a foreground object needs to be separated from its background.

Running a segmentation model

I used the YOLO11 nano segmentation model:

from ultralytics import YOLO
import matplotlib.pyplot as plt

# Load the segmentation model
segmenter = YOLO("yolo11n-seg.pt")

# Run segmentation
results = segmenter("image.jpg", conf=0.4)
r = results[0]

# Display the segmentation result
r.show()

The -seg suffix indicates that the model is designed for segmentation.

For each detected object, the model can provide a class, a confidence score, a bounding box, and a segmentation mask.

Understanding segmentation masks

A mask represents the pixels associated with an object. Conceptually, a binary mask can contain:

• 1: the pixel belongs to the object.
• 0: the pixel does not belong to the object.

The following example shows how to access the masks and count their marked pixels:

if r.masks is not None:
    for mask, class_id in zip(r.masks.data, r.boxes.cls):
        name = r.names[int(class_id)]
        pixel_count = int(mask.sum())

        print(f"{name}: {pixel_count:,} mask pixels")
else:
    print("No objects were detected.")

The exact mask representation depends on the model and library. In this example, the mask data is represented numerically, and summing the mask values gives an approximate count of the marked pixels.

These counts are based on the model's mask resolution, which may be smaller than the original image. They are useful for comparing mask areas, but they are not automatically exact measurements of real-world object sizes.

Combining masks

I also explored how multiple object masks can be combined into one grid.

if r.masks is not None:
    combined = r.masks.data.cpu().numpy().max(axis=0)

    plt.imshow(combined, cmap="gray")
    plt.title("Combined object masks")
    plt.axis("off")
    plt.show()

    print(
        f"Object pixels cover approximately "
        f"{combined.mean():.0%} of the mask grid"
    )

The operation .max(axis=0) combines the masks by taking the maximum value at each corresponding pixel position. For binary masks, a pixel is marked if at least one mask marks it as belonging to an object.

The mean of a binary mask represents the fraction of its values that are marked as 1. Multiplying this fraction by 100 gives a percentage.

This is a useful introduction to how pixel-level information can be used to estimate the area covered by detected objects.

9. Classification vs. Detection vs. Segmentation

After experimenting with these approaches, I found it helpful to compare them directly.

Feature	Classification	Object detection	Segmentation
Main question	What is in the image?	What objects are present, and where?	Which pixels belong to each object?
Identifies classes	Yes	Yes	Yes
Locates individual objects	Not generally	Yes	Yes
Produces bounding boxes	No	Yes	Usually available with instance segmentation
Produces pixel-level masks	No	No	Yes
Example use	Categorizing product images	Counting vehicles in a street image	Outlining buildings in satellite imagery

The main difference is the level of detail each task provides.

Classification predicts categories for an image. Object detection identifies and locates individual objects. Segmentation goes further by identifying the pixels associated with objects or regions.

One additional distinction is worth knowing: instance segmentation identifies separate objects individually, while semantic segmentation assigns a category to each pixel. For example, instance segmentation can distinguish one person from another, whereas semantic segmentation may label both as the category person without separating their individual identities.

10. My Main Lessons and Takeaways

Working through these experiments helped me connect the theory of computer vision with practical Python code.

Here are the key lessons I have learned so far:

1. Images are numerical data.

A computer represents a digital image using pixel values arranged in arrays. Understanding image dimensions and color channels makes image manipulation easier to understand.

2. Filters can reveal visual features.

Blurring, sharpening, and edge detection demonstrate how mathematical operations can change images and emphasize useful information.

3. CNNs can learn visual patterns.

Rather than relying only on manually designed filters, convolutional neural networks learn useful representations from training examples.

4. Classification, detection, and segmentation solve different problems.

Classification identifies categories, detection locates objects, and segmentation identifies object pixels.

5. Confidence scores need careful interpretation.

A model's score is not a guarantee of correctness. Changing a detection threshold affects which predictions are retained.

6. Pretrained models have limitations.

A model can only reliably recognize categories it has learned to represent, and its performance depends on the quality and suitability of its training data.

7. Practical experimentation makes concepts clearer.

Running models on different images, inspecting predictions, and changing parameters helped me understand the concepts better than reading definitions alone.

11. What I Want to Explore Next

This is only the beginning of my computer vision journey. There are several areas I would like to explore next:

• Understanding convolutional neural networks in greater depth.
• Learning how YOLO detects objects internally.
• Exploring how models are trained and evaluated.
• Comparing pretrained models with models trained on custom datasets.
• Testing computer vision applications using local images and real-world problems.
• Exploring how computer vision can contribute to agriculture, healthcare, and other challenges in Rwanda and beyond.

I am particularly interested in understanding not only how to use pretrained models, but also how they work and how to evaluate their limitations.

Conclusion

Computer vision has helped me see images differently. What looks like a simple photograph to a human is a structured collection of numerical values that a computer can process to identify patterns, recognize objects, and understand visual scenes.

Starting with pixels and filters gave me the foundation to understand how images are represented and processed. Experimenting with YOLO then showed me how pretrained models can perform classification, object detection, and segmentation.

I am still learning, but these practical experiments have given me a stronger foundation and a clearer picture of how computer vision works.

If you are also learning computer vision, I encourage you to start with a simple image, experiment with it, and observe how different techniques interpret the same scene.

You might be surprised by how much you can learn by looking beyond the picture itself.

---

This article reflects my learning and practical experiments with computer vision so far. I welcome feedback, corrections, and suggestions from others who are exploring this field.

Comments

[Irakoze Deborah]

Great article! I liked how you explained computer vision concepts in a simple way for beginners. Adding more visual results from your experiments and a link to a runnable notebook would make it even better. Keep learning and sharing!