How Machines Learned to See 👀: From CNNs to Vision Transformers

Janani — Tue, 01 Sep 2026 12:50:22 +0000

Imagine showing a computer a picture of a dog.

You see a dog.

The computer sees thousands or even millions of numbers representing pixels.

So how does it go from pixels → patterns → “that's a dog!”?

That's the fascinating story of Computer Vision.

👁️ Teaching Machines to See

Computer Vision is a branch of Artificial Intelligence that enables machines to understand images and videos.

But computers don't naturally understand objects the way humans do. They need to learn visual patterns such as edges, textures, shapes, and eventually entire objects.

Early computer vision systems relied heavily on handcrafted features. Humans had to decide what features the machine should look for.

Then came a game-changing idea:

What if the machine could learn the features by itself?

Enter Convolutional Neural Networks (CNNs).

🧠 The CNN Revolution

CNNs became one of the most important architectures in Computer Vision because they could automatically learn visual features directly from images.

And CNNs didn't become powerful overnight.

They evolved.

Year	Architecture	Big Idea
1998	LeNet-5	Early successful CNN
2012	AlexNet	Deep learning breakthrough
2014	VGGNet	Deeper networks
2014	GoogLeNet	Multi-scale feature extraction
2015	ResNet	Skip connections
2017	MobileNet	Efficient CNNs
2019	EfficientNet	Accuracy-efficiency scaling
2022	ConvNeXt	Modern CNN design

The journey looked like:

LeNet → AlexNet → VGG → GoogLeNet → ResNet → Modern CNNs

CNNs became exceptionally good at recognizing local patterns.

But that raised another question...

🤔 Can CNNs See the Bigger Picture?

Imagine an image of someone riding a bicycle.

A CNN can learn:

Edges → wheels → body parts → bicycle → person

But understanding the complete scene requires connecting information from different regions of the image.

A face might be in one part.

A hand might be somewhere else.

The bicycle could occupy another region.

CNNs can learn these relationships through deeper layers, but researchers wanted a mechanism that could directly model relationships between different parts of an image.

And that's where Transformers entered Computer Vision.

🤖 From Words to Images

Transformers originally became famous in Natural Language Processing.

Their secret weapon?

Self-attention.

Attention allows a model to determine which parts of an input are important to one another.

Researchers asked:

If attention works for words, could it work for image patches too?

The answer was yes.

In 2020, researchers introduced the Vision Transformer (ViT).

And the idea was surprisingly simple:

Turn an image into a sequence.

🧩 An Image Becomes a Sentence

A Vision Transformer divides an image into small patches.

For example:

224 × 224 image

with

16 × 16 patches

produces:

196 image patches

Each patch is converted into a numerical representation called an embedding.

The model then adds positional information so it knows where each patch belongs.

The patches can now be processed like a sequence of tokens.

Think of it as:

Image → Patches → Embeddings → Transformer → Prediction

Pretty clever, right? 👀

🔎 The Magic: Self-Attention

Here's where ViT becomes really interesting.

Instead of only asking:

“What's happening around this pixel?”

self-attention allows the model to ask:

“Which other parts of this image are important to understanding this part?”

A patch containing a person's face can relate to patches containing their body.

A wheel can relate to the bicycle frame.

A person can relate to the object they're holding.

The model learns these relationships using three components:

Query (Q)
Key (K)
Value (V)

Together, they allow the Transformer to determine where attention should be focused.

This gives Vision Transformers a powerful ability to capture global context.

⚔️ CNN vs Vision Transformer

CNN	Vision Transformer
Uses convolution	Uses self-attention
Strong at local features	Strong at global relationships
Processes feature maps	Processes image patches
Spatial structure is built into convolution	Uses positional information
Generally computationally efficient	Can be computationally expensive
Often works well with less data	Benefits from large-scale pretraining

So...

Is ViT replacing CNN?

Not exactly.

And that's actually the most interesting part.

🔥 The Best of Both Worlds?

CNNs are excellent at extracting local features.

Transformers are excellent at modeling relationships between distant regions.

So researchers started asking:

Why choose one when we can combine both?

This led to CNN–Transformer hybrid architectures.

Other important developments followed.

🎓 DeiT

Data-efficient Image Transformers explored ways to train Vision Transformers more efficiently using knowledge distillation.

🪟 Swin Transformer

Swin Transformer introduced shifted-window attention, allowing attention to be calculated within smaller windows instead of across the entire image at once.

This made Transformers more practical for tasks such as object detection and image segmentation.

🌍 Where Can We Use Vision Transformers?

Vision Transformers are being explored across many areas:

🏥 Medical Imaging
Analyzing medical scans and detecting visual patterns.

🚗 Autonomous Vehicles
Understanding roads, vehicles, pedestrians, and surroundings.

🛰️ Satellite Imaging
Analyzing large geographical regions.

🎯 Object Detection
Finding and identifying objects in complex scenes.

🎥 Video Understanding
Learning relationships across both space and time.

🤖 Robotics
Helping robots understand their visual environment.

🚀 Where Are We Going Next?

The future of Computer Vision isn't simply:

CNN ❌ → Transformer ✅

It's moving toward:

CNN + Transformer + Vision-Language Models + Multimodal AI

Imagine an AI system that can:

See an image → understand it → read a question → reason about it → explain its answer.

That's a much bigger goal than simply recognizing an object.

💡 The Bigger Picture

The evolution of Computer Vision can be summarized in one journey:

Handcrafted Features
↓
CNNs — learn visual features
↓
Deep CNNs — learn increasingly complex representations
↓
Vision Transformers — model relationships between image patches
↓
Hybrid & Multimodal Models — combine different forms of intelligence

The biggest change isn't just a new architecture.

It's a change in how we think about images.

CNNs taught machines to recognize patterns.

Transformers taught them to focus on relationships.

And the next generation of AI aims to understand the whole picture.

👁️ From pixels to patterns.

🧠 From patterns to relationships.

🤖 From relationships to understanding.

That's how machines are learning to see.

🔗 Want to Go Deeper?

The original Vision Transformer paper is a great place to start:

An Image is Worth 16×16 Words: Transformers for Image Recognition at Scale

Read the original ViT paper on arXiv

For another major development in Vision Transformers:

Read the Swin Transformer paper

💬 What do you think?

Will Transformers completely replace CNNs someday or will the future belong to hybrid models?

DEV Community: Janani