Before my first computer vision class, I thought a computer looked at a photo the way I do. I was wrong. A computer has never seen a photo, not once.
This week we started computer vision, the part of AI that teaches machines to "see". Here is what I learned, in simple words, for anyone starting from zero like me.
A photo is just numbers
When I look at a street photo, I instantly see a bus and some people. A computer gets a grid of numbers instead. Every tiny square (a pixel) has three numbers: how much red, green and blue it has, each from 0 to 255. Why 255? Each value uses 8 bits, and 8 bits can hold 256 different values, from 0 to 255.
In class we opened a sample photo with Python:
import numpy as np
from PIL import Image
img = np.array(Image.open("bus.jpg").convert("RGB"))
print(img.shape) # (1080, 810, 3) for our sample photo
print(img.size) # 2624400 numbers in total
That one photo is 1080 rows × 810 columns × 3 colours, which is 2,624,400 numbers. The computer never "sees a bus". It only sees those numbers. The big challenge of computer vision is getting from the numbers to the meaning.
What can a computer do with a photo?
Almost every vision system answers one of three questions: What is in the image? Where is it? What is happening? We ran three pretrained models on the same street photo, and each one answered differently:
- Classification gives one label for the whole image. It said "minibus" with 57.1% confidence and "police van" with 33.7%. It was unsure between two vehicles that look alike, and one label is not very useful for a whole street scene.
- Detection draws a box around each object. It found 1 bus (94% confident) and 4 people.
- Segmentation labels every pixel. It marked exactly which pixels belong to each object. In our photo, the objects covered 44% of the image.
More detailed answers need more detailed training labels, and those cost more to create.
How does a computer find an edge?
This was my favourite part, because it is only multiplication and addition. A filter is a small grid of weights that slides over the image. At each spot, you multiply each pixel by its weight and add everything up.
sobel_x = np.array([[-1, 0, 1],
[-2, 0, 2],
[-1, 0, 1]])
This filter computes "right side minus left side". On a flat patch where every pixel is 100, you get −400 + 400 = 0, so there is no edge. On a patch that goes from dark (0) to bright (200), you get 0 + 800 = 800, a big number, so there is an edge.
A CNN (convolutional neural network) does exactly this, with one difference: nobody chooses the weights. Training learns them from thousands of labelled images. The first layers learn edges and colours, the middle layers learn textures and parts like eyes or wheels, and the last layers recognise whole objects.
Why YOLO is fast
For detection we used YOLO, which stands for "You Only Look Once". Older detectors first proposed possible regions and then classified each one, which was accurate but slow. YOLO looks at the whole image one time and predicts every box and label together. That is fast enough for live video and small devices. It was introduced in 2015 by Joseph Redmon and colleagues.
from ultralytics import YOLO
detector = YOLO("yolo11n.pt") # small pretrained detection model
r = detector("street.jpg", conf=0.4)[0] # only keep boxes it is 40%+ sure about
for box in r.boxes:
print(r.names[int(box.cls)], f"{float(box.conf):.0%}")
When models fail
This part surprised me most. We made the same photo harder, the way a cheap phone or bad lighting would:
- Very dark: the model invented a skateboard that was not there.
- Upside down: it lost the bus.
- Low resolution: it found nothing.
A model only knows what it was trained on. Many of these models were trained mostly on photos from Europe, North America and Asia. So where might they fail here? On crops, clothing, road scenes, skin tones and lighting. The fix is local data.
Why this matters for us
Our course looked at real examples from Africa. Nuru helps farmers diagnose cassava disease from a leaf photo, and it works offline on a basic phone, so it works in fields with no internet. BLAIVA, built by FabLab Rwanda in Kigali, is a wearable camera that describes objects, text and faces through earphones. It is still in beta testing. These systems were built by people like us, which makes this field feel close and possible.
What I'm taking away
- Images are numbers: height × width × 3.
- Filters find patterns, and CNNs learn those filters on their own.
- Classification, detection and segmentation answer different questions.
- Models fail on what they have not seen, so local data matters.
Next, I'll build my own image classifier on photos from around me, with a separate test set so I can check it honestly.
Sources
- kLab TekHer AI Program, "Introduction to Computer Vision" class slides and demo notebook
- Ultralytics YOLO11 pretrained models; ImageNet and COCO datasets
- YOLO: Joseph Redmon et al., 2015
Top comments (1)
This was so amazing as you showed very well how you were interested in learning and how you added the codes showing how well you got the realtionship. Please keep up the spirit because there is still alot to learn and dive in