This was the first time I ever studied computer vision. I expected it to be simple: you show a computer a photo, and it tells you what is in it. I was so wrong, and honestly, I loved finding out why.
I spent hours testing real models on market photos. The mistakes taught me far more than the right answers.
1. A photo is only numbers
I started with a photo of a vegetable stall. To me it was tomatoes, onions and little bowls. To the computer it was a grid of 400 rows and 600 columns, with three numbers (red, green and blue) in every square. That is 720,000 numbers for one photo. I stared at that number for a moment. It looks like a picture to us, and to the computer it is a very long list.
One pixel I printed was [70 71 37]. Low numbers mean dark, so it was dark olive. Then I switched off the red channel, and the white parts turned cyan while the red tomatoes almost vanished. That was my "ohhh" moment. Colour is just numbers, and you can change them.
2. How a computer finds edges
An edge is a place where brightness changes quickly. A small grid called a filter slides across the image and measures that change. Some filters find vertical edges, some find horizontal ones, and some only blur.
What impressed me most: in deep learning, nobody designs these filters by hand. The network learns them from data.
3. Sure and wrong at the same time
Then I tried the classifier. It looked at my market photo and said "banana" with 38.3% confidence. The long green vegetables had fooled it. "Grocery store" came second at 33.3%, and that was the better answer.
Next I tried something from home: a jerrycan. I searched the full list of 1,000 classes the model knows, and a jerrycan is not on it. I tried "jerrycan", then "jug", then "bucket", and I was curious what it would do. It said "candle". Its second guess, "water jug", was much closer, and I laughed out loud at the first one.
The lesson I will keep: being confident is not the same as being right. The model has to pick something from its list, even when the right answer is not there.
4. Detection, and the number you choose
A detector draws boxes around objects. At a confidence threshold of 0.4, it found 12 bowls and 1 person in my photo.
When I lowered the threshold to 0.1, it found 41 objects, and many were wrong. It saw bananas on the green vegetables, and even a chair.
When I raised it to 0.9, it found nothing at all.
So the number you choose changes what the model "sees". It is the same trade-off as in medical screening: catch more cases and accept false alarms, or be strict and miss real ones. The detector also ignored the big green vegetables completely, because it only knows 80 everyday classes.
5. What the model outlines, and what it ignores
Segmentation outlines the exact pixels of each object. In my photo, the objects covered only 7% of the image. The other 93% counted as background, because the model outlines only what it recognises.
6. When the photo is not perfect
I took a photo with five people and made it harder. In the dark, the detector found 3 people. Blurry, it found 1. Upside down, 1. Shrunk to a tiny size, none. Many of us take photos with phones in poor light, so this is not a rare case.
7.My own small mistakes
I also made a few mistakes of my own. For a while my results kept showing the bus from the example, because an earlier cell still held the old image. Waiting for cells to run felt endless some minutes, and once the notebook even refused to save. It was frustrating, but fixing each one taught me something. Now I always check which photo I am really working on.
8. What this means for us in Rwanda
Models like these learn mostly from photos from Europe, North America and Asia. Mine mistook a jerrycan for a candle. If we want tools that recognise our crops, our containers and our streets, we need local photos, labelled by people who know them. I find that exciting, because it means we can help build that data.
9. What I still don't know
I don't know yet how to train a model on my own photos, or how many photos are enough. I don't know how to tell whether a model is truly good, beyond checking a few examples. And I still can't see why it chose "banana", because I cannot look inside it.
10. Where I go next
I want to start thinking about which local photos to collect, and teach a model to recognise them.
If you have tried this too, I would love to hear what your model got wrong.
Top comments (2)
Great post, Claudine! The jerrycan being called a candle made me laugh, but it also shows the real problem: the model has to pick something from its list, even when the right answer isn't there. I also liked that you tried the threshold at 0.1 and 0.9 and saw how the number you choose changes what the model "sees". Thanks for sharing your mistakes too, it makes the post feel honest and easy to learn from.
I really enjoyed reading this! Your real-life experiments, especially the jerrycan being mistaken for a candle, made computer vision both relatable and fun to learn. I also love how you connected your findings to the need for local data in Rwanda. Great work!