My first hands-on computer vision experiment, where pixels became predictions, handbags disappeared in the dark, and turning a photo upside down confused AI completely.
You know that feeling when you look at a picture and immediately understand what's happening?
A crowded street? Easy.
People crossing the road? Obviously.
A dog standing next to a bicycle? No problem.
Now imagine showing the same picture to an AI model and having it confidently suggest that you're looking at a scoreboard.
Yes. A scoreboard.
That was one of the first surprises I encountered during the course of computer vision.
The goal was simple: understand how computers process images, how models recognize objects, and what happens when those models encounter something unexpected.
But instead of just running the provided examples and calling it a day, I decided to experiment with my own photographs.
And that's when things got interesting.
1. Before AI Can See Anything, It Sees Numbers
Let's start with something that sounds obvious but becomes fascinating when you actually experiment with it.
A computer doesn't see a photograph the way we do.
When I look at my first image, I see a crowded pedestrian crossing, buildings, advertisements, and people going about their day.
When Python looks at the same image, it sees a NumPy array.
My photograph had:
- Width: 1,280 pixels
- Height: 1,920 pixels
- Colour channels: 3 (Red, Green, Blue)
That gives us:
1,280 × 1,920 × 3 = 7,372,800 numbers.
More than seven million numbers representing one photograph!
Each colour value ranges from 0 to 255.
For example, one pixel in my image had the RGB values:
[1, 202, 228]
This represents a bright cyan colour because there is almost no red, but plenty of green and blue.
I used Pillow and NumPy to inspect the image, separate its RGB channels, and explore how the pixels were represented.
One interesting experiment involved finding the 6 × 6 pixel region with the highest colour variation.
I expected something like a person's face or the boundary between two objects.
Instead, the notebook selected a bright cyan sign.
Why?
Because my calculation measured differences in colour values, not necessarily meaningful objects or edges.
Lesson learned: What a computer considers interesting depends entirely on what we ask it to measure.
And that distinction becomes very important in machine learning.
2. I Built My Own Image Filter (Before Letting AI Do the Work)
Next came convolution.
If you've studied Convolutional Neural Networks (CNNs), you've probably heard about kernels, filters, and feature maps.
But understanding the theory and implementing it yourself are two different experiences.
I wrote my own convolution operation using NumPy.
The idea is surprisingly straightforward.
Take a small matrix of numbers, slide it across an image, multiply the overlapping values, and add the results.
Different matrices produce different effects.
For example:
Blur filters smooth out details and reduce sharp variations.
Sobel filters highlight changes in brightness, helping reveal edges.
When I applied the Sobel filter to my crowded street photograph, the buildings, pedestrian crossing, and outlines of people became more visible.
It was fascinating to see how much information could be extracted using relatively simple mathematics.
And then came the connection to deep learning.
In a traditional image-processing experiment, I decide which filter to use.
In a CNN, the model learns useful filter weights during training.
Early layers can learn simple patterns such as edges, while deeper layers combine features into more complex representations.
So essentially, I was manually experimenting with one of the fundamental ideas behind CNNs.
Except CNNs don't need me sitting there choosing filters one by one. Thankfully.
3. I Asked AI What Was in My Picture. It Said "Scoreboard."
Now for the fun part.
I used YOLO11n-cls, a lightweight image classification model, to predict what was in my photograph.
The model had been trained to classify images into 1,000 ImageNet categories.
My expectation?
Something related to people, streets, or cities.
The actual result?
| Prediction | Confidence |
|---|---|
| Scoreboard | 22.6% |
Other predictions included street sign, unicycle, cinema, and grocery store.
I looked at my photograph again.
A scoreboard? Really?
But after thinking about it, the result started making sense.
My photograph contained large signs, advertisements, buildings, and many overlapping objects.
The classifier had to choose a single label from the categories it knew.
It wasn't designed to describe an entire busy street scene.
And importantly, 22.6% confidence is not a particularly strong prediction.
The model wasn't really telling me, "I'm absolutely sure this is a scoreboard."
It was telling me that, among its available categories, scoreboard received the highest score.
I repeated the experiment with a second photograph containing people, a bicycle, and a dog.
This time, the model predicted:
Miniskirt — 25.1% confidence.
At this point, I started wondering whether my AI was more interested in fashion than computer vision.
But the experiment revealed an important limitation.
Image classification assigns a label to the whole image. It doesn't tell us everything happening inside it.
For a complex scene containing multiple objects, that can be a serious limitation.
Which brought me to the next experiment.
4. Object Detection: Finally, AI Started Seeing People
Instead of asking the model to give my entire image one label, I switched to YOLO11n, an object detection model.
Object detection answers two questions:
What objects are present, and where are they located?
Unlike classification, detection can return multiple labels and bounding boxes.
This sounded perfect for my crowded street photograph.
I ran the model.
And the result was...
11 objects detected.
Specifically:
- 7 people
- 4 handbags
Now here's the problem.
My photograph contained well over 100 people.
The model found seven.
SEVEN.
Imagine using that system to count attendance at a crowded event.
The organizers would probably think nobody showed up!
But the model's behaviour revealed something important.
It was much better at detecting larger people near the camera than people farther away.
The distant pedestrians were small, densely packed, and often partially hidden behind one another.
The model also resized the image for inference, reducing the detail available for detecting tiny objects.
And because I used a confidence threshold of 40%, less certain predictions were filtered out.
This wasn't necessarily a case of the model being completely useless.
It was a case of using a lightweight general-purpose detector on a scene that is particularly challenging for object detection.
To investigate further, I tested my second photograph.
This one had fewer objects and a less crowded environment.
The model detected:
4 people, 1 bicycle, and 1 dog.
All six detections matched objects in the photograph.
Even the small dog and the bicycle partially hidden behind a person were identified.
Same model. Different image. Very different results.
That was probably one of my biggest takeaways from the entire exercise:
A model performing well on one image doesn't mean it will perform equally well on every image.
The environment, object size, visibility, and image quality all matter.
5. Segmentation: Going Beyond Bounding Boxes
Object detection was interesting, but I wanted to understand how computers could identify the actual shape of an object.
That's where segmentation comes in.
I used YOLO11n-seg, which predicts pixel-level masks for detected objects.
Instead of simply drawing a rectangle around a person, the model identifies the pixels belonging to that person's visible shape.
In my crowded photograph, I used these masks to estimate how much of the image was occupied by detected people.
The result was approximately 13% of the image.
One detail I found particularly interesting was how simple the calculation could be.
A binary mask contains values of 0 and 1.
Zero means the pixel is not part of the object.
One means it is.
By calculating the mean of a binary mask, we can determine the proportion of pixels covered by that mask.
Of course, for multiple people, overlapping masks must be handled carefully to avoid counting the same pixels more than once.
What made this experiment exciting was realizing how the same concept can be useful in completely different industries.
In agriculture, segmentation can help estimate crop coverage.
In healthcare, it can help measure regions of interest in medical images.
In satellite imagery, it can help identify buildings, roads, and land-cover areas.
The mathematics may look simple, but the applications are powerful.
There was still one limitation, though.
Just like the detector, the segmentation model mostly identified larger people near the camera.
It couldn't magically recover people the underlying model had failed to detect.
6. Then I Decided to Make the Model's Life Difficult
After seeing the models work on normal photographs, I had another question:
What happens when the image conditions are no longer ideal?
Because real-world photographs are rarely perfect.
People take pictures at night.
Phones have different camera qualities.
Images get compressed.
And sometimes a camera captures something from a very unusual angle.
So I created three modified versions of my crowded street photograph:
- A very dark version at 15% brightness.
- An upside-down version.
- A very low-resolution version, reduced to 48 pixels wide.
Then I ran the same object detector on each version.
Here is what happened:
| Image condition | Total objects | People detected |
|---|---|---|
| Original | 11 | 7 |
| Very dark | 7 | 7 |
| Upside down | 0 | 0 |
| Low resolution | 0 | 0 |
Let's talk about these results.
Experiment A: Darkness made the handbags disappear
When I reduced the brightness to 15%, the model still detected seven people.
But the four handbags?
Gone.
Apparently, the people survived the darkness, but their accessories didn't.
A possible explanation is that the handbags had less visible detail and were harder to distinguish from their surroundings.
The larger people remained recognizable enough for the detector.
This demonstrated that different object categories and sizes can respond differently to poor lighting.
Experiment B: Turning the image upside down broke everything
Next, I rotated the photograph by 180 degrees.
The model detected:
Zero objects.
Not one person.
Not one handbag.
Nothing.
Meanwhile, I could still recognize that the photograph showed a crowded street.
This was a useful reminder that AI models don't automatically possess the same visual understanding or robustness as humans.
The detector had likely learned many patterns from photographs in more typical orientations.
Turning the entire scene upside down changed the appearance and spatial arrangement of those patterns.
The model struggled to generalize.
Experiment C: Low resolution finished the job
Finally, I reduced the image to just 48 pixels wide.
Again:
Zero detections.
At such a small resolution, individual people lost most of their distinguishing features.
When the image was enlarged again, the missing details didn't magically return.
The model was left trying to identify objects from heavily degraded visual information.
And it failed.
These experiments showed me why testing AI under different conditions is so important.
A model can perform reasonably well during a demonstration and still struggle when deployed in a real environment.
7. What This Experiment Changed About How I Think About AI
Before this exercise, it was easy to think about computer vision as a model that takes an image and tells us what's inside it.
But working through the process from pixels to predictions made me appreciate how many decisions and limitations exist along the way.
Here are five things I'm taking away from this experience.
First, images are data before they are meaning.
Everything starts with numbers. How we transform and represent those numbers determines what patterns become available to a model.
Second, different computer vision tasks solve different problems.
Classification answers what an image might represent.
Detection identifies objects and their locations.
Segmentation identifies the pixels belonging to objects.
Choosing the right approach matters just as much as choosing the model.
Third, small objects and crowded scenes are difficult.
My detector finding seven people in a crowd of more than 100 was a practical demonstration of why object size, occlusion, and image resolution matter.
Fourth, model predictions need to be questioned.
A prediction is not automatically correct because it came from a trained model.
Confidence scores, missing detections, and unexpected outputs all deserve investigation.
Finally, real-world testing matters.
If a computer vision application is going to be used on nighttime photographs, low-quality phone images, or crowded environments, it needs to be evaluated under those conditions.
Otherwise, we risk building systems that look impressive in a notebook but fail when people actually need them.
Final Thoughts: AI Can See, But Does It Understand?
My first day exploring computer vision started with a simple question:
How does a computer turn pixels into predictions?
By the end, I had implemented convolution filters, experimented with image classification, detected objects, explored segmentation, and tested how models behave under difficult conditions.
I also discovered that an AI model could mistake a crowded street for a scoreboard, overlook most of a crowd, and completely lose its ability to detect people when an image was flipped upside down.
And strangely enough, those failures were the most valuable part of the experience.
Because learning AI isn't just about getting models to produce predictions.
It's about understanding why they make those predictions, when we can trust them, and what happens when the real world doesn't look like the training data.
This was Day 1 of my computer vision journey and I'm looking forward to exploring what comes next.
Hopefully, the next model will be better at counting people.
Top comments (2)
I really enjoyed this! The title is a great hook, and the “scoreboard” and “miniskirt” moments are funny while still teaching something. I especially liked that you explained your results instead of just listing them, like why the detector found only 7 people, and comparing the crowded photo with the simpler one.
A few suggestions:
Some images seem missing. There are blank spaces after key results, like “11 objects detected.” The crowd with detection boxes would be your strongest picture.
Test your explanations. Try conf=0.1 or a bigger image size (imgsz=1280) on the crowd to check whether the threshold and resizing really caused the missed people.
Great work overall. It reads like a real experiment!
Great article, Anne Marie! I really enjoyed reading about your hands-on computer vision experiments, especially how the model detected only 7 people out of a crowd of more than 100. It shows that AI models have limitations and that image quality, object size, and lighting can significantly affect their predictions. I also liked how you compared image classification, object detection, and segmentation using real examples. This is a great reminder that understanding why a model fails is just as important as understanding how it works. Keep sharing us your learning journey! and good lucky