How Machines Learn to See: A Beginner's Intro to Computer Vision (with Python)

u look at a photo of a street and instantly see a bus, some people, maybe a lamp post. It takes you a fraction of a second.

A computer gets something very different: a grid of numbers.

In this post I'll walk through the core ideas of computer vision (CV) from the first day of the kLab TekHer AI Program (Cohort 2). We follow one photo from raw numbers all the way to a model that can say what is in it, and where. Everything runs in Google Colab, and a CPU is enough.

What does the computer actually see?

Let's load a photo and look at what the computer receives:

import numpy as np
from PIL import Image
from ultralytics.utils import ASSETS

img = np.array(Image.open(ASSETS / "bus.jpg").convert("RGB"))

print("Shape (height, width, channels):", img.shape)
print("Total numbers in this image:", f"{img.size:,}")

The photo is 1080 rows x 810 columns of pixels, and each pixel has 3 numbers (red, green, blue). That is 1080 x 810 x 3 = 2,624,400 numbers. The computer never sees "a bus", only these numbers.

A few things worth remembering:

• Each value is a uint8: a whole number from 0 to 255 (8 bits give 2^8 = 256 values). 0 means none of that colour, 255 means full.
• In shape, height comes first, then width. Rows go down (y), columns go across (x).
• A colour image is really three grids stacked together: one each for red, green and blue. That shape, height x width x 3, is called a tensor.

The three questions computer vision answers

1. What is in the image? A cat, a cassava leaf.
2. Where is it? Which pixels, which part of the image.
3. What is happening? A person crossing the road.

These map to five core tasks:

Task	What it does
Classification	One label for the whole image
Detection	A box around each object
Segmentation	A label for every pixel
OCR	Reads text in images
Tracking	Follows objects across video

Filters: finding edges by hand

Before deep learning, we could already find patterns in images using a filter (or kernel): a small grid of weights that slides across the image. Each output pixel is a weighted sum of its neighbours.

blur    = np.ones((5, 5)) / 25
sobel_x = np.array([[-1, 0, 1],
                    [-2, 0, 2],
                    [-1, 0, 1]])
sobel_y = sobel_x.T

• blur averages 25 neighbours, so fine detail fades.
• sobel_x computes right side minus left side, so it finds vertical edges.
• sobel_y (the same filter flipped with .T) computes bottom minus top, so it finds horizontal edges.

Why does sobel_x work? Try it on two small patches:

Patch	Left side	Right side	Result
Flat (every pixel = 100)	100 x (-1 -2 -1) = -400	100 x (1 + 2 + 1) = +400	0, no edge
Edge (left = 0, right = 200)	0	200 x (1 + 2 + 1) = +800	800, an edge

Same brightness on both sides gives zero. Dark on the left and bright on the right gives a big number. Big result = brightness changes = edge.

Here is the function that slides the kernel over a grayscale image:

def convolve(gray, kernel):
    k = kernel.shape[0]
    pad = k // 2
    padded = np.pad(gray, pad, mode="edge")
    out = np.zeros_like(gray, dtype=float)
    for dy in range(k):
        for dx in range(k):
            out += kernel[dy, dx] * padded[dy:dy + gray.shape[0], dx:dx + gray.shape[1]]
    return out

gray = img.mean(axis=2)
edges = np.hypot(convolve(gray, sobel_x), convolve(gray, sobel_y))

np.hypot combines both directions like Pythagoras, so we get edge strength whichever way the edge points.

So what does a CNN do?

A Convolutional Neural Network (CNN) does exactly this, with one big difference: we don't choose the filters. Training learns them from thousands of labelled images. Early layers learn edges, later layers learn textures, and the last layers learn whole objects:

Pixels -> Edges -> Textures -> Objects

Classify, detect, segment in a few lines

We use Ultralytics YOLO, which ships small pretrained models.

!pip install -q ultralytics
from ultralytics import YOLO

1. Classification: one label for the whole image

classifier = YOLO("yolo11n-cls.pt")
r = classifier(ASSETS / "bus.jpg", verbose=False)[0]

for idx in r.probs.top5:
    print(f"{r.names[idx]:<25} {r.probs.data[idx].item():6.1%}")

The model answers minibus (57%), with police_van (34%) in second place. It was trained on ImageNet's 1,000 categories, and there is no class called "street scene", so it picks the closest thing it knows. Notice that high confidence does not mean correct. It only means the model is sure.

2. Detection: what and where

detector = YOLO("yolo11n.pt")
r = detector(ASSETS / "bus.jpg", conf=0.4, verbose=False)[0]

for box in r.boxes:
    x1, y1, x2, y2 = box.xyxy[0].int().tolist()
    print(r.names[int(box.cls)], f"{float(box.conf):.0%}", (x1, y1), (x2, y2))

Now we get a bounding box for each object: one bus (94% confidence) and four people. The conf threshold is your choice. Lower it and you get more boxes but more mistakes. Raise it and you get fewer boxes you can trust more. It is the same trade-off as medical screening, where a low threshold misses fewer real cases but raises more false alarms.

Why is YOLO fast? YOLO stands for You Only Look Once. Older detectors first proposed many regions and then classified each one separately. YOLO makes one pass over the whole image and predicts every box and label at once, which is fast enough for live video.

3. Segmentation: which pixels belong to which object

segmenter = YOLO("yolo11n-seg.pt")
r = segmenter(ASSETS / "bus.jpg", conf=0.4, verbose=False)[0]

combined = r.masks.data.cpu().numpy().max(axis=0)
print(f"Object pixels cover {combined.mean():.0%} of the image")

Each object gets a mask: a grid of 0s and 1s where 1 means "this pixel belongs to the object". The mean of a grid of 0s and 1s is the fraction of 1s, so one line tells us objects cover 44% of the image. The same trick measures building area in satellite images or a tumour in a scan.

When models fail

A model only knows what it was trained on. I made the same photo harder in three ways that are common with cheap phones, poor lighting or dusty lenses:

Change	What happened
Original	1 bus, 4 people
Very dark	Adds a fake skateboard
Upside down	Misses the bus
Low resolution	Finds nothing

Same scene, same model, very different answers. Most of these models were trained mostly on photos from Europe, North America and Asia, so it is worth asking where they might fail on African images: crops, clothing, roads, skin tones, lighting. Local data matters.

Computer vision across Africa

This is not just a lab exercise. CV is already being used here:

• Agriculture: Nuru diagnoses crop disease from a photo of a cassava leaf, offline, on a basic phone (East Africa).
• Health: Software scored X-rays to find TB in 5,298 people screened in Gombe and Adamawa, Nigeria. High scores got a lab test, and 85 people with TB were found.
• Conservation: Smart camera traps in Gabon spot elephants and people, then alert rangers by satellite within minutes.
• Finance: Smile Identity matches a selfie to an ID photo and checks it is a real, live person, with over 1 million identity checks a month across Africa (2021).
• Mapping: Google's Open Buildings project outlined 1.8 billion building footprints from satellite images, used for vaccination planning and disaster response.
• Accessibility: BLAIVA, built by FabLab Rwanda in Kigali and in beta testing, is a wearable camera that describes objects, text and faces through earphones.

For each one, ask three questions: What goes in? What comes out? Who is harmed if it fails?

Why now?

CV has taken off because several things arrived together: cameras everywhere (every phone is a camera and a computer), huge image datasets, cheap cloud GPUs, pretrained models you can adapt with a few thousand local photos, and multimodal AI that reads images as well as text. There is even a link to NLP: a Vision Transformer cuts an image into 16 x 16 patches and treats each patch like a word.

Tools to start with

• PyTorch: build and train your own networks
• TensorFlow / Keras: Google's alternative, common in mobile apps
• Hugging Face: download pretrained models and fine-tune them
• Ultralytics YOLO: ready-made detection, used in this post

Recap

• Images are numbers: height x width x 3.
• Filters find patterns, and CNNs learn them.
• Classify, detect, segment: three different questions about the same image.
• Models fail on data unlike their training data, so local data matters.

Your turn: try a photo of a street, a market or a crop leaf. Find one hit and one miss, and think about what data you would need to fix the miss.

Next up for me: datasets, CNNs in PyTorch, augmentation and metrics, and building an image classifier with an honest test set. I'll share what I learn.

If you've tried computer vision on local images, I'd love to hear how it went in the comments!

You can find more of my front-end and portfolio projects on GitHub.

Comments

[Tuyishimire claudine]

Great post, Claire! I liked how you followed one photo from raw numbers all the way to detection and segmentation, and the point that “high confidence does not mean correct” is so true. I tried the same models on my own photos, and the classifier called a jerrycan a candle, while the detector found nothing at all when I raised the threshold to 0.9. My favourite part was the Africa examples, especially BLAIVA from Kigali, and the question “Who is harmed if it fails?” is one I will remember. Thanks for sharing!

[Cynthia Umutoniwase]

This is such an impressive and insightful blog! I love how you used practical Python examples with YOLO to explain classification, detection, and segmentation, while connecting computer vision to real-world applications across Africa. You made a complex topic easy to understand. Thank you for sharing your knowledge!