Have you ever looked at a picture and wondered how your brain recognizes what's inside it so quickly?
Imagine looking at a picture of a bus. You immediately notice its shape, colors, and windows, and your brain says, "That's a bus!"
But what happens when we give that same picture to a computer?
Does a computer actually see a bus, or does it see something completely different from what we see?
That question got me curious. I wanted to understand how a computer goes from processing numbers to identifying objects in an image.
I didn't want to just run an AI model, watch it draw boxes around objects, and say, "Wow, AI is amazing!" I wanted to understand what was happening behind the results.
So, I started exploring computer vision using Python, experimenting with image pixels, filters, classification, object detection, and segmentation.
And honestly, the more I learned, the more questions I had.
I discovered that an image, which looks so simple to us, is represented by numbers inside a computer. Wait... how can numbers represent a bus, a face, or even a tiny detail in a picture? And what happens when we change those numbers?
Then I started wondering how AI models recognize objects. How can a model identify a bus in one image but struggle with another because of lighting, different angles, or objects being partially hidden?
And here's another question that caught my attention: What's the difference between recognizing an object and identifying the exact pixels that belong to it? That's where I started exploring the difference between object detection and image segmentation.
I'm still learning, experimenting, and connecting these concepts. I don't want to know how to run a model; I want to understand why it works and what happens behind the scenes.
In this post, I'll share some of what I'm discovering as I explore how computers process images and learn to recognize what's inside them.
Sometimes, learning something new starts with one simple question you just can't stop thinking about.
Let's explore it together!
1. Wait, an image is just numbers?
One of the first things I explored was how images are represented in Python.
When we look at a picture, we see colors, people, objects, and backgrounds. But a computer represents a color image using numerical values called pixels.
Using NumPy, we can inspect those values:
pixel = img[100, 400]
print(pixel)
This gives us the RGB values of the pixel at row 100 and column 400.
RGB means Red, Green, and Blue. Each channel usually has a value between 0 and 255.
For example:
[255, 0, 0] # Red
[0, 255, 0] # Green
[0, 0, 255] # Blue
When I think about it, it is interesting that something we see as a complete picture can be represented by numbers that we can access and change using code.
I also explored how to extract a small part of an image:
patch = img[100:106, 100:106]
This selects a 6 × 6 section of the image.
My takeaway: Before trying to understand how AI recognizes an image, I need to understand what an image looks like from the computer's point of view. It starts with data.
2. What happens if we remove one color?
This was another interesting experiment because it showed me that I can change an image by modifying its pixel values.
no_red = img.copy()
no_red[:, :, 0] = 0
show(no_red, "No red")
plt.show()
Let me explain what is happening here.
First, img.copy() creates a copy of the original image. Then [:, :, 0] selects the red channel, and setting it to zero removes the red contribution from every pixel.
For example, a white pixel has the values [255, 255, 255]. After removing red, it becomes [0, 255, 255], which is cyan.
This helped me understand that the colors we see depend on the values stored in the different channels.
I like experiments like this because instead of only reading a definition, I can change something in the code and understand the effect by looking at the result.
3. Image filters: why do we need them?
I also explored image filters, including blur, sharpening, and edge detection.
At first, these might look like simple image effects, but they help us understand how image processing can emphasize or reduce certain features.
A filter often uses a small matrix called a kernel. The kernel moves across an image and calculates new pixel values using nearby pixels.
For example, this is a sharpening kernel:
sharpen = np.array([
[0, -1, 0],
[-1, 5, -1],
[0, -1, 0]
])
The values in this matrix affect how the output image is calculated. This particular kernel emphasizes differences between a pixel and its neighbors, which can make details appear sharper.
A blur filter works differently. It averages nearby pixel values, making the image smoother and reducing small details.
Edge detection helps highlight places where brightness changes, such as the boundaries between objects and their backgrounds.
What I find interesting is that we can use mathematical operations to make different features of an image easier to see.
My takeaway: Image processing is not magic. There are operations happening behind the output, and understanding those operations helps me understand the bigger picture of computer vision.
4. Image classification: how does a model guess what it sees?
Next, I explored image classification using a pretrained YOLO classification model.
classify(ASSETS / "zidane.jpg")
Classification means predicting what category an image belongs to. Depending on the model and its training classes, it might predict categories such as a person, an animal, or another object.
The model gives predictions with confidence scores, but something I want to remember is that a prediction is not automatically correct just because the model produced it.
We still need to look at the image and ask whether the prediction makes sense.
This also made me think about how much a model depends on the examples it learned from during training. If the category we want is not represented well in its training data, the prediction may not be what we expect.
I am beginning to see why learning about models also means learning about their limitations.
5. Object detection: what changes when we change confidence?
Object detection is different from classification. Instead of only predicting a category, a detection model can identify objects and draw bounding boxes around them.
One experiment I explored was changing the confidence threshold.
_ = detect(IMAGE_PATH, conf=0.1)
Then comparing it with:
_ = detect(IMAGE_PATH, conf=0.9)
The conf value sets the minimum confidence required for a detection to be kept.
With a lower threshold, the model can keep more predictions, including uncertain ones. With a higher threshold, it generally keeps fewer predictions, but it can also miss real objects.
This is where I started thinking beyond just getting an output.
What if we want a model to detect every possible object? What if false detections are a serious problem? Would we always choose a high confidence threshold?
I learned that the answer depends on what we are trying to achieve. There is a balance between detecting more objects and trusting the predictions we keep.
My takeaway: A model's output depends not only on the model itself, but also on the settings we choose and the problem we want to solve.
6. Object detection vs. image segmentation
I also explored image segmentation, which gave me another way to think about how AI understands images.
With object detection, a model draws a bounding box around an object. With segmentation, it can identify the pixels that belong to the object, giving us a more detailed outline.
For example, after running a segmentation model, we can inspect the masks:
r = segmenter(IMAGE_PATH, conf=0.4, verbose=False)[0]
if r.masks is not None:
for mask, c in zip(r.masks.data, r.boxes.cls):
print(r.names[int(c)], int(mask.sum()), "pixels")
else:
print("No segmentation masks were found.")
This code checks whether segmentation masks exist and counts the nonzero values in each mask. The counts represent the mask data at the model's output resolution, not necessarily exact pixel counts in the original image.
The difference is becoming clearer to me:
Classification: What category is in the image?
Object detection: What objects are present, and where are they?
Segmentation: Which pixels belong to each detected object?
These tasks may seem similar at first, but they answer different questions.
7. What happens when the image is not perfect?
Something else I want to explore is how well a model performs when an image changes.
Think about a picture taken in poor lighting, a blurry photo, or an image that has been rotated. As humans, we can sometimes recognize an object even when the image is not perfect. But will a model always recognize it?
One way to investigate this is to create a blurry version of an image:
from PIL import ImageFilter
blurry = original.filter(ImageFilter.GaussianBlur(8))
Then we can compare the model's predictions on the original image and the blurry version.
We can also experiment with darker images, lower resolutions, and different orientations.
I think this is an important part of learning AI because getting a model to work on one image does not mean it will perform equally well in every situation.
I want to understand not only when a model works, but also when it struggles and why.
What I am taking from this learning experience
The biggest thing I am learning is that understanding AI takes more than knowing which library to import or which function to call.
I need to understand the data going into the model, the operations being performed, the output coming back, and whether that output actually makes sense.
I am also learning to question results instead of accepting them immediately. If I change a value, what happens? If I change the image, does the prediction change? If the result looks wrong, what could be causing it?
These are the kinds of questions I want to keep asking as I continue learning.
I know I still have a lot to learn about computer vision and machine learning, but I am trying to build my understanding step by step rather than rushing through concepts without understanding them.
What is next for me?
I want to complete the hands-on exercises, compare the results, and get more comfortable experimenting with the code myself.
After that, I want to go deeper into how these models work internally and how they learn from data.
My goal is not just to use AI tools. I want to understand how they work well enough to eventually build things of my own.
I would love your feedback
Since I am still learning, I would really appreciate feedback from people with more experience in computer vision and AI.
What beginner-friendly experiments helped you understand computer vision better? Is there something in this blog that I should explain differently? And what would you recommend I learn next after understanding the basics of pixels, filters, classification, detection, and segmentation?
I am sharing this to document my learning and improve through feedback. If you are also learning AI, I hope we can learn from each other.
Thank you for reading!
Top comments (1)
I really liked how curious this post feels. The questions in the intro pull the reader in, and your explanations are clear and accurate, especially the white → cyan example and your note that segmentation pixel counts are at the model’s resolution, not the original photo’s. Ending by asking readers for feedback is a nice touch too.
A few suggestions:
Share your actual results. The post explains the code well but doesn’t show what happened. What did the classifier say for zidane.jpg? How many boxes appeared at conf=0.1 vs 0.9? Adding your outputs (and a screenshot or two) would make the post much stronger.
Finish the “imperfect image” experiment. Section 7 says you want to explore it. Running it on the blurry, dark and upside-down versions and sharing what the model found would answer your own question.
Try your own photo. Using a photo you took, instead of the sample images, would make the experiments more personal and interesting.
Fix the code formatting. The indentation in the segmentation snippet got lost, so the if/for lines won’t run as shown. Putting code in code blocks keeps it intact.