Introduction to Bert : Part One
Have you ever wondered how Siri understands your voice, or how Google Translate can switch between English and Spanish in seconds? Behind these everyday miracles lies the fascinating field of Natural Language Processing (NLP) — the bridge between human language and machines.
In this post, we’ll explore how NLP evolved, why it was challenging, and how BERT (Bidirectional Encoder Representations from Transformers) changed the game forever.
What is NLP?
In simple terms, NLP means teaching computers to understand and generate human language — just like how we talk or write.
Think of NLP as giving machines the ability to “read” and “respond” like a human.
Everyday examples:
Siri or Alexa understanding our voice commands.
Google Translate converting English to Spanish.
Spam filters catching junk emails.
NLP turns human language into actionable data. For developers, it’s the key to smarter apps, better UX, automated tasks, and AI-powered innovations.
Why is NLP Hard?
Language is messy!
Here’s why it’s so difficult for machines:
Words can have multiple meanings:
“bank” could mean a riverbank or a financial institution.
Context matters:
“I’m feeling blue” doesn’t mean someone is literally blue — it means sad. Sentences can be long and complex, with lots of dependencies between words. Machines needed to learn not just words, but meanings in context
The Evolution of NLP
Before BERT, NLP went through a few big phases:
- Rule-Based Systems (Pre-2010)
Humans wrote grammar rules for computers — accurate but painfully slow and rigid.
- Statistical Models
These used probabilities to predict words (like autocomplete), but couldn’t understand deep meaning.
- Word Embeddings (2013–2017)
We turned words into vectors — numbers that capture meaning.
But here’s the catch: the word “bank” always had the same number, even if used in different contexts.
- Transformers (2017)
Then came the Transformer model, a true revolution.
It allowed machines to read entire sentences at once instead of one word at a time — understanding meaning from context.
Enter BERT (2018)
BERT, created by Google in 2018, stands for:
Bidirectional Encoder Representations from Transformers
It’s a model that helps computers understand the meaning of words in a sentence by looking at both directions — left and right.
This is what makes it bidirectional
Why Bidirectional Matters
Unidirectional model (Left-to-right):
How Unidirectional Model (e.g., Left-to-Right) Reads It:
“He”,
“sat”
“on”
“the”
“bank” -- Here's the tricky part. The model sees: "He sat on the bank"
But it has NOT yet seen “of the river.” At this point, the model doesn’t know if “bank” means a riverbank or financial institution.
Problem: Unidirectional model made its guess at word (bank) without knowing the true meaning.
Now read the full sentence:
“He sat on the bank of the river.”
Bidirectional:
BERT sees all words at once, including “river,” it understands "bank" = riverbank from the start.
Only after seeing river does the real meaning appear — but traditional models can’t look ahead
Top comments (0)