Computer Vision for Beginners: From Pixels to Predictions

What I learned in my first computer vision : how images become numbers, how filters find edges, and how classification, detection and segmentation differ. With my own results, including where the models failed.

I used to think computer vision meant a computer "seeing" a picture the way I do. It doesn't. After my first hands-on session in the kLab TekHer AI Program (Cohort 2), I understand why, and I learned it best by running the code and testing my own photos.

This post follows the same path as the session: pixels → filters → classification → detection → segmentation → failure cases. Everything here comes from a Python notebook I ran in VS Code, with the "Try it yourself" exercises done by hand.

What you will learn

• How a computer stores an image
• How a filter finds edges, and why that matters for CNNs
• The difference between classification, detection and segmentation
• Where pretrained models fail, with my own results

What is computer vision?

Computer vision lets computers get meaning from images and video. Almost every vision system answers one or more of three questions:

Question Example
What is in the image? A cat, a cassava leaf
Where is it? Which pixels, which part
What is happening? A person crossing the road

Humans and computers are good at different things. We learn from years of seeing and need few examples. A computer needs millions of labelled images, but it then works at huge speed and scale. Its main weakness is anything unlike its training data, and we will see that later.

1. An image is a grid of numbers

A computer never sees "a bus". It receives a grid of rows × columns × 3 colour values (red, green, blue), each from 0 to 255. Why 255? Each value uses 8 bits, and 8 bits give 2⁸ = 256 possible values.

python
import numpy as np
from PIL import Image

img = np.array(Image.open("photo.jpg").convert("RGB"))
print(img.shape)       # (height, width, 3): height first, then width
print(img.dtype)       # uint8: whole numbers from 0 to 255
print(img[100, 400])   # one pixel: [R, G, B]

markdown
A small 400 × 600 photo holds 400 × 600 × 3 = 720,000 numbers. A phone photo holds more than 30 million. The core challenge of computer vision is going from these numbers to meaning.

A colour image is three grids

Each pixel has a red, a green and a blue value. Showing each channel on its own makes this visible:

Strong colour means a high value (near 255) and pale means a low value (near 0).

I also removed the red channel to see what happens:

no_red = img.copy()      # copy, so the original is not damaged
no_red[:, :, 0] = 0      # set every red value to 0

Pure red areas became black, because (255, 0, 0) turns into (0, 0, 0). White areas became cyan, because white without red is green plus blue.

1. Filters: finding edges by hand

A filter(or kernel) is a small grid of weights. For every pixel, you multiply it and its neighbours by the weights and add the results. That sum is the new pixel value. Then the filter slides across the whole image.

Here are three filters:

blur    = np.ones((5, 5)) / 25                              # average of 25 neighbours
sobel_x = np.array([[-1, 0, 1], [-2, 0, 2], [-1, 0, 1]])    # right minus left: vertical edges
sobel_y = sobel_x.T                                         # bottom minus top: horizontal edges

Why does sobel_x find vertical edges? Look at two 3×3 patches:

Patch	Left side	Right side	Result
Flat: every pixel is 100	100 × (−1 −2 −1) = −400	100 × (1 + 2 + 1) = +400	0, no edge
Edge: left is 0, right is 200	0	200 × 4 = +800	800, an edge

So flat areas become black and edges become bright. The weights of a Sobel filter add up to 0, which is why flat areas disappear. Blur and sharpen weights add up to 1, so the overall brightness stays the same.

I also tried my own kernels. My first sharpen result looked flat grey because the values went outside 0–255. Clipping them with np.clip(result, 0, 255) fixed it.

The big idea: a CNN learns its own filters

A convolutional neural network (CNN) does exactly this multiply-and-add. The only difference is that nobody chooses the weights. Training learns them from labelled data. Researchers who visualise trained CNNs find a clear pattern:

Pixels → edges → textures → objects

Early layers learn edges and colours, middle layers learn textures and parts like eyes or wheels, and the last layers respond to whole objects.

1. Classification: one label for the whole image

I loaded a small pretrained model that picks from 1,000 ImageNet classes and returns a confidence for each.

from ultralytics import YOLO

classifier = YOLO("yolo11n-cls.pt")
r = classifier("photo.jpg", verbose=False)[0]
for idx in r.probs.top5:
    print(r.names[idx], f"{r.probs.data[idx].item():.1%}")

I tested it on photos from my own context in Rwanda, and the results taught me more than the sample images did:

Photo Top label Confidence
agaseke(woven basket) vase 35.1%
umukenyero (wrapped cloth) sarong 78.7%
A photo of me next to a car maillot (tights 17.3%

Three lessons:

1. The model always picks something. Its final step, softmax, forces the 1,000 probabilities to add up to 100%, so it can never say "I don't know." 2.High confidence is not correctness. The sarong answer was 78.7% sure and only roughly right. The model does not know the Rwandan name or meaning.

2. There is no "person" class in ImageNet.** On my photo, the model looked at the dress and returned clothing labels.

   1. Object detection: what and where

A classifier gives one label. A detector draws a box around each object it knows (80 everyday categories from the COCO dataset) and gives a confidence for each.

detector = YOLO("yolo11n.pt")
r = detector("photo.jpg", conf=0.4, verbose=False)[0]
for box in r.boxes:
    x1, y1, x2, y2 = box.xyxy[0].int().tolist()
    print(r.names[int(box.cls)], f"{float(box.conf):.0%}", (x1, y1), (x2, y2))

Each box has a corner position, where x counts from the left and y counts from the top.
Detection on the sample bus photo: a box, a label and a confidence for each object.

On my own photo the detector found person (94%) and car (70%). The car was less certain because I was standing in front of it and hiding part of it.

The confidence threshold is a choice you make

The conf setting is the minimum confidence a box needs to be shown.

Threshold Misses real objects False alarms

Low (0.1) fewer more
High (0.9) more fewer

This is the same trade-off as in medical screening: a low threshold misses fewer real cases but raises more false alarms.

Why is YOLO fast?

YOLO stands for You Only Look Once. Older detectors first proposed many regions and then classified each one, which was accurate but slow. YOLO looks at the whole imag once and predicts every box, label and confidence in a single pass. That makes it fast enough for live video and small devices.

1. Segmentation: which pixels belong to which object

Segmentation goes one step further and gives every object a mask, a grid of 0s and 1s where 1 means "this pixel belongs to the object."

segmenter = YOLO("yolo11n-seg.pt")
r = segmenter("photo.jpg", conf=0.4, verbose=False)[0]

for mask, c in zip(r.masks.data, r.boxes.cls):
    print(r.names[int(c)], int(mask.sum()), "pixels")

Every pixel of each object is labelled, not just a box around it.

There is a neat trick here: the mean of a grid of 0s and 1s is the fraction of 1s. One line of code tells you what share of the image an object covers. The same idea measures building areas in satellite images, or a tumour in a scan.

One photo, three tasks

Task Question it answers Output

Classification What is this picture about? One label
Detection What is here, and where? A box per object
Segmentation Which exact pixels? A mask per object

More detailed outputs need more detailed training labels, and those cost more to create.

1. When models fail

This was my favourite part. A model only knows what it was trained on, so I made harder versions of my photo and ran the detector on each.

These are my own results on my photo:

Version Objects found What happened
Original 2 person 94%, car 70%
Very dark 2 person 95%, car 80%: no damage
Upside down 1 person 86%, car lost
Low resolution (48 px wide) 0 found nothing
Blurry 2 person 94%, car 69% almost no damage

What stood out:
Low resolution broke the detector completely. The tiny, blocky image removed almost all detail.
The dark photo did not hurt the detector, but the classifier on the same dark photo changed its top answer to "crutch" (11.9%). Different models cope with different problems.
Rotation hurt. When I turned the photo sideways, confidence fell, the same person appeared in two boxes, and the more confident box was the less accurate one.

The pattern is simple: models fail on conditions they rarely saw in training.

Why this matters in Africa

These models were trained mostly on photos from Europe, North America and Asia. My agaseke became a "vase". Other risks I would check are local clothing, unpaved roads, moto-taxis, strong sunlight and low-quality phone photos. I have not tested all of these, so I treat them as risks, not proven results.

The same technology is already used across the continent. The session slides listed examples such as:

Agriculture: a farmer photographs a cassava leaf and an app names the disease, even offline
-Health: software scores chest X-rays to decide who gets a TB lab test

• Conservation:** smart camera traps alert rangers within minutes
• Mapping: models outline every building from satellite images -Accessibility:a wearable vision assistant built in Kigali that describes objects and text

The fix for the failures above is local data plus transfer learning: start from a pretrained model and train it a little more on local images, so it can learn to say "agaseke" instead of "vase".

Key takeaways

1. An image is numbers: height × width × 3.
2. Filters find patterns, and CNNs learn the filters themselves.
3. Classification gives one label, detection adds location, and segmentation labels every pixel.
4. Confidence is not correctness.
5. Models fail on what they have not seen, so local data matters.

What is next for me: building my own image classifier on local data, with an honest test set.

If you are learning computer vision too, try the exercises yourself. Predicting the result before running the code taught me the most. Thanks for reading!