Before the first day of computer vision, I thought it meant "teaching a computer to look at pictures, the way we do." By the end of the day I realised that sentence hides about six different questions. These are the ones I wrote down, with the answers that finally made sense to me.
1. Why is "seeing" hard for a computer?
Because a computer doesn't receive a picture. It receives numbers.
A photo is a grid of pixels, and each pixel is three numbers: how much red, green and blue light it has, each from 0 to 255. Our sample street photo is 1080 rows by 810 columns, so that is 1080 x 810 x 3 = 2,624,400 numbers. Somewhere in those numbers is a bus, but nothing in the file says "bus".
Think of it like being handed a spreadsheet of 2.6 million cells and being asked what it shows. Computer vision is the work of getting from the spreadsheet to the meaning.
2. What can computer vision actually do?
I found it easier to remember the field as five jobs, from simplest to most detailed:
- Classification: one label for the whole picture ("minibus").
- Detection: a box around each object ("1 bus, 4 people").
- Segmentation: a label for every single pixel.
- OCR: reading text inside an image.
- Tracking: following an object from frame to frame in a video.
The same street photo can feed the first three, and each gives a different kind of answer. Choosing the right job matters. If you want to count motorcycles at a junction, a single label for the whole photo will not help you. You need detection.
3. How does a computer find an edge?
An edge is just a place where brightness changes quickly. To find one, the computer slides a small grid of weights, called a filter, across the image, multiplying and adding as it goes.
You can see the whole idea in a few lines of NumPy, with no model and no download. Here is a tiny 6x6 "image" that is dark on the left and bright on the right:
import numpy as np
image = np.array([[0, 0, 0, 200, 200, 200]] * 6)
sobel_x = np.array([[-1, 0, 1],
[-2, 0, 2],
[-1, 0, 1]])
flat_patch = image[1:4, 0:3] # all zeros: no change in brightness
edge_patch = image[1:4, 2:5] # dark -> bright: an edge
print((flat_patch * sobel_x).sum()) # 0
print((edge_patch * sobel_x).sum()) # 800
Where nothing changes, the answer is 0. Where the brightness jumps from left to right, the answer is 800. That big number is the computer's way of saying "something is happening here". Run the same filter over the whole photo and every edge lights up.
4. So what does a CNN add?
In that example I chose the weights. A Convolutional Neural Network (CNN) uses the same sliding-filter idea, but it learns the weights by itself from thousands of labelled images.
Early layers end up finding edges. Middle layers combine edges into textures and shapes. Later layers combine those into whole objects. I found this the most surprising part of the day: nobody tells the network what an edge is. It works that out because edges turn out to be useful.
5. How can a detector find objects so quickly?
Older detectors worked in two steps: first propose regions that might hold an object, then classify each region. That was accurate, but slow.
YOLO ("You Only Look Once") does it in one pass. The whole image goes through the network once, and out come every box, label and confidence score together. It is fast enough for live video, and the small version we used is only about 5 MB.
from ultralytics import YOLO
detector = YOLO("yolo11n.pt")
result = detector("bus.jpg", conf=0.4, verbose=False)[0]
for box in result.boxes:
print(result.names[int(box.cls)], f"{float(box.conf):.0%}")
On the sample photo it finds the bus (94% sure) and the people standing in front of it. The conf value is a setting you choose, and it changes the result: lower it and you get more boxes with more mistakes, raise it and you get fewer but surer ones.
6. Does it work everywhere? What can go wrong?
This was the question that stayed with me, because the answer is: not automatically.
We made the same photo harder on purpose. In the dark, the model invented a skateboard. Upside down, it missed the bus. At very low resolution, it found nothing. The pixels were the same photo, but the conditions were different from what the model was trained on.
That matters here, because many of these models were trained mostly on photos from other parts of the world. And yet computer vision is already being used in Africa:
- Nuru helps farmers diagnose cassava disease from a leaf photo, offline, on a basic phone.
- In Nigeria, X-ray screening software helped find TB: 5,298 people were screened, and 85 with TB were found.
- BLAIVA, built by FabLab Rwanda in Kigali, is a wearable camera that describes objects, text and faces for people with low vision.
For each of these, the useful question is not "does it work?" but "what happens to the person if it fails?" A wrong answer on a leaf photo and a wrong answer on an X-ray are not the same kind of mistake.
What I take away
- A photo is height x width x 3 numbers.
- Filters find patterns, and CNNs learn their own filters.
- Classification, detection and segmentation answer different questions.
- A model is only as good as the data it has seen, so local data matters.
Next I want to learn how datasets are built, how to train my own small CNN in PyTorch, and how to test a model honestly. If you have tried a computer vision model on your own photos, I would love to hear what it got right and what it got wrong.
Top comments (1)
this is a very good explanation.I especially liked how you explained the difference between classification, object detection, and segmentation using simple examples. The part about local data and how models can fail in different conditions was also very important. As someone learning Machine Learning and Deep Learning, this gives me a clearer understanding of how computers move from pixels to meaningful information. Great work, Aline