DEV Community: CLAIRE

How Machines Learn to See: A Beginner's Intro to Computer Vision (with Python)

CLAIRE — Tue, 13 Oct 2026 11:42:40 +0000

u look at a photo of a street and instantly see a bus, some people, maybe a lamp post. It takes you a fraction of a second.

A computer gets something very different: a grid of numbers.

In this post I'll walk through the core ideas of computer vision (CV) from the first day of the kLab TekHer AI Program (Cohort 2). We follow one photo from raw numbers all the way to a model that can say what is in it, and where. Everything runs in Google Colab, and a CPU is enough.

What does the computer actually see?

Let's load a photo and look at what the computer receives:

import numpy as np
from PIL import Image
from ultralytics.utils import ASSETS

img = np.array(Image.open(ASSETS / "bus.jpg").convert("RGB"))

print("Shape (height, width, channels):", img.shape)
print("Total numbers in this image:", f"{img.size:,}")

The photo is 1080 rows x 810 columns of pixels, and each pixel has 3 numbers (red, green, blue). That is 1080 x 810 x 3 = 2,624,400 numbers. The computer never sees "a bus", only these numbers.

A few things worth remembering:

Each value is a uint8: a whole number from 0 to 255 (8 bits give 2^8 = 256 values). 0 means none of that colour, 255 means full.
In shape, height comes first, then width. Rows go down (y), columns go across (x).
A colour image is really three grids stacked together: one each for red, green and blue. That shape, height x width x 3, is called a tensor.

The three questions computer vision answers

What is in the image? A cat, a cassava leaf.
Where is it? Which pixels, which part of the image.
What is happening? A person crossing the road.

These map to five core tasks:

Task	What it does
Classification	One label for the whole image
Detection	A box around each object
Segmentation	A label for every pixel
OCR	Reads text in images
Tracking	Follows objects across video

Filters: finding edges by hand

Before deep learning, we could already find patterns in images using a filter (or kernel): a small grid of weights that slides across the image. Each output pixel is a weighted sum of its neighbours.

blur    = np.ones((5, 5)) / 25
sobel_x = np.array([[-1, 0, 1],
                    [-2, 0, 2],
                    [-1, 0, 1]])
sobel_y = sobel_x.T

blur averages 25 neighbours, so fine detail fades.
sobel_x computes right side minus left side, so it finds vertical edges.
sobel_y (the same filter flipped with .T) computes bottom minus top, so it finds horizontal edges.

Why does sobel_x work? Try it on two small patches:

Patch	Left side	Right side	Result
Flat (every pixel = 100)	100 x (-1 -2 -1) = -400	100 x (1 + 2 + 1) = +400	0, no edge
Edge (left = 0, right = 200)	0	200 x (1 + 2 + 1) = +800	800, an edge

Same brightness on both sides gives zero. Dark on the left and bright on the right gives a big number. Big result = brightness changes = edge.

Here is the function that slides the kernel over a grayscale image:

def convolve(gray, kernel):
    k = kernel.shape[0]
    pad = k // 2
    padded = np.pad(gray, pad, mode="edge")
    out = np.zeros_like(gray, dtype=float)
    for dy in range(k):
        for dx in range(k):
            out += kernel[dy, dx] * padded[dy:dy + gray.shape[0], dx:dx + gray.shape[1]]
    return out

gray = img.mean(axis=2)
edges = np.hypot(convolve(gray, sobel_x), convolve(gray, sobel_y))

np.hypot combines both directions like Pythagoras, so we get edge strength whichever way the edge points.

So what does a CNN do?

A Convolutional Neural Network (CNN) does exactly this, with one big difference: we don't choose the filters. Training learns them from thousands of labelled images. Early layers learn edges, later layers learn textures, and the last layers learn whole objects:

Pixels -> Edges -> Textures -> Objects

Classify, detect, segment in a few lines

We use Ultralytics YOLO, which ships small pretrained models.

!pip install -q ultralytics
from ultralytics import YOLO

1. Classification: one label for the whole image

classifier = YOLO("yolo11n-cls.pt")
r = classifier(ASSETS / "bus.jpg", verbose=False)[0]

for idx in r.probs.top5:
    print(f"{r.names[idx]:<25} {r.probs.data[idx].item():6.1%}")

The model answers minibus (57%), with police_van (34%) in second place. It was trained on ImageNet's 1,000 categories, and there is no class called "street scene", so it picks the closest thing it knows. Notice that high confidence does not mean correct. It only means the model is sure.

2. Detection: what and where

detector = YOLO("yolo11n.pt")
r = detector(ASSETS / "bus.jpg", conf=0.4, verbose=False)[0]

for box in r.boxes:
    x1, y1, x2, y2 = box.xyxy[0].int().tolist()
    print(r.names[int(box.cls)], f"{float(box.conf):.0%}", (x1, y1), (x2, y2))

Now we get a bounding box for each object: one bus (94% confidence) and four people. The conf threshold is your choice. Lower it and you get more boxes but more mistakes. Raise it and you get fewer boxes you can trust more. It is the same trade-off as medical screening, where a low threshold misses fewer real cases but raises more false alarms.

Why is YOLO fast? YOLO stands for You Only Look Once. Older detectors first proposed many regions and then classified each one separately. YOLO makes one pass over the whole image and predicts every box and label at once, which is fast enough for live video.

3. Segmentation: which pixels belong to which object

segmenter = YOLO("yolo11n-seg.pt")
r = segmenter(ASSETS / "bus.jpg", conf=0.4, verbose=False)[0]

combined = r.masks.data.cpu().numpy().max(axis=0)
print(f"Object pixels cover {combined.mean():.0%} of the image")

Each object gets a mask: a grid of 0s and 1s where 1 means "this pixel belongs to the object". The mean of a grid of 0s and 1s is the fraction of 1s, so one line tells us objects cover 44% of the image. The same trick measures building area in satellite images or a tumour in a scan.

When models fail

A model only knows what it was trained on. I made the same photo harder in three ways that are common with cheap phones, poor lighting or dusty lenses:

Change	What happened
Original	1 bus, 4 people
Very dark	Adds a fake skateboard
Upside down	Misses the bus
Low resolution	Finds nothing

Same scene, same model, very different answers. Most of these models were trained mostly on photos from Europe, North America and Asia, so it is worth asking where they might fail on African images: crops, clothing, roads, skin tones, lighting. Local data matters.

Computer vision across Africa

This is not just a lab exercise. CV is already being used here:

Agriculture: Nuru diagnoses crop disease from a photo of a cassava leaf, offline, on a basic phone (East Africa).
Health: Software scored X-rays to find TB in 5,298 people screened in Gombe and Adamawa, Nigeria. High scores got a lab test, and 85 people with TB were found.
Conservation: Smart camera traps in Gabon spot elephants and people, then alert rangers by satellite within minutes.
Finance: Smile Identity matches a selfie to an ID photo and checks it is a real, live person, with over 1 million identity checks a month across Africa (2021).
Mapping: Google's Open Buildings project outlined 1.8 billion building footprints from satellite images, used for vaccination planning and disaster response.
Accessibility: BLAIVA, built by FabLab Rwanda in Kigali and in beta testing, is a wearable camera that describes objects, text and faces through earphones.

For each one, ask three questions: What goes in? What comes out? Who is harmed if it fails?

Why now?

CV has taken off because several things arrived together: cameras everywhere (every phone is a camera and a computer), huge image datasets, cheap cloud GPUs, pretrained models you can adapt with a few thousand local photos, and multimodal AI that reads images as well as text. There is even a link to NLP: a Vision Transformer cuts an image into 16 x 16 patches and treats each patch like a word.

Tools to start with

PyTorch: build and train your own networks
TensorFlow / Keras: Google's alternative, common in mobile apps
Hugging Face: download pretrained models and fine-tune them
Ultralytics YOLO: ready-made detection, used in this post

Recap

Images are numbers: height x width x 3.
Filters find patterns, and CNNs learn them.
Classify, detect, segment: three different questions about the same image.
Models fail on data unlike their training data, so local data matters.

Your turn: try a photo of a street, a market or a crop leaf. Find one hit and one miss, and think about what data you would need to fix the miss.

Next up for me: datasets, CNNs in PyTorch, augmentation and metrics, and building an image classifier with an honest test set. I'll share what I learn.

If you've tried computer vision on local images, I'd love to hear how it went in the comments!

You can find more of my front-end and portfolio projects on GitHub.

Cracking Word Embeddings: Why One-Hot Fails and How Word2Vec Actually Works

CLAIRE — Wed, 30 Sep 2026 13:03:43 +0000

Building natural language processing models always brings you face-to-face with a core challenge: computers have no clue what words actually mean. Feed a machine a sentence like "the programmer writes code," and it sees nothing. Computers don't process text natively; they need numbers.

For a long time, traditional strategies like bag-of-words or TF-IDF handled this conversion. But when you look closely at how they represent text, you realize they hit a hard ceiling when it comes to capturing actual meaning. That's where word embeddings and the word2vec framework change the game.

Let's break down why traditional approaches fall short and how self-supervised vector representations solve the problem.

Why One-Hot Vectors Fall Short

A straightforward way to numericize text is using one-hot encoding. If your vocabulary contains a specific number of unique words, say a dictionary size of $\vert{}V\vert{}$, you assign each word an integer index from $0$ to $\vert{}V\vert{}-1$. To represent any specific word, you build a vector of length $\vert{}V\vert{}$ filled entirely with zeros, except for a single 1 placed at that word's index.

While these vectors are simple to set up, they are a terrible choice for machine learning models. The biggest issue is that they cannot capture semantic similarity. If you measure the cosine similarity between two different one-hot vectors, the result is always zero.

Because every orthogonal vector is completely independent, a one-hot encoder treats words like "cat" and "dog" as just as distant from each other as "cat" and "refrigerator." They completely fail to encode relationships or shared meanings.

Self-Supervised Word2Vec

To fix this, researchers introduced word2vec, mapping each word to a dense, fixed-length vector that captures semantic similarity and analogies.

The clever part about word2vec is that it relies on self-supervised learning. Instead of manual data labeling, the model extracts supervision straight from raw text by predicting words based on their surrounding context. It splits into two primary model architectures: Skip-Gram and Continuous Bag of Words (CBOW).

The Skip-Gram Model

The skip-gram architecture operates on the assumption that a central target word can generate its surrounding context words in a sequence.

Take the text sequence: "the", "man", "loves", "his", "son". If we select "loves" as our center word and set a context window size of 2, skip-gram looks at the conditional probability of generating the words that appear no more than two steps away: "the", "man", "his", and "son".

In this model, every word gets two separate vector representations: one used when it acts as a center word, and another used when it acts as a context word. During training, we maximize the likelihood function using stochastic gradient descent. Once trained, the center word vectors are typically pulled out and used as the final word embeddings for downstream tasks.

The Continuous Bag of Words (CBOW) Model

The continuous bag of words (CBOW) model flips the script. Instead of predicting context from a center word, CBOW assumes that a center word is generated based on its surrounding context words.

Using that same sequence ("the", "man", "loves", "his", "son"), with "loves" as the center word and a window size of 2, CBOW takes the surrounding context words ("the", "man", "his", "son") and averages their vectors to predict the target center word.

While training follows a similar gradient optimization process to skip-gram, CBOW typically uses the context word vectors as the final word representations.

Wrapping Up

Moving away from sparse frequency matrices to dense vector representations completely transforms how text data is handled in machine learning. By mapping words into a continuous vector space where geometric distance reflects semantic meaning, models can finally understand relationships, synonyms, and context.

References & Further Reading

Dive into Deep Learning (D2L.ai): Chapter 15.1 – Word Embedding (word2vec). Available at d2l.ai.
Mikolov, T., et al. (2013): Efficient Estimation of Word Representations in Vector Space. arXiv preprint arXiv:1301.3781.
Mikolov, T., et al. (2013): Distributed Representations of Words and Phrases and their Compositionality. Advances in Neural Information Processing Systems (NeurIPS).