Highlighting the important words
Day 20 of 149
👉 Full deep-dive with code examples
The Highlighter Example
You're studying for an exam. The textbook says:
"The mitochondria is the powerhouse of the cell, and cells need energy to do work..."
You highlight: "mitochondria is the powerhouse of the cell"
You ignore: birth year, university (not important for the exam)
Attention is AI's highlighter!
The Problem AI Had
Sentence: "The bank by the river had no money"
What does "bank" mean?
- 💰 Bank (financial) ?
- 🏞️ Bank (riverbank) ?
Old AI: "Uh... 50/50?"
With Attention:
- "bank" looks at "river" → Strong connection!
- "bank" looks at "money" → Weaker (it says "no money")
- Answer: riverbank! 🏞️
How It Works
For each word, AI asks:
"How relevant is every other word to understanding me?"
"The cat sat because it was tired"
For "it":
"cat" → high (very relevant)
"sat" → low
"tired" → medium
Result: AI knows "it" = "the cat"
Self-Attention
The word literally "attends" to other words:
The cat sat on mat it was tired
it: low high low - - - - medium
Higher score = More attention = More relevant!
Why It's Revolutionary
Before: AI read one word at a time, forgot earlier context
After: AI sees all words, decides what matters
This is the key to:
- Translating languages
- Understanding questions
- Writing paragraphs
- Coding assistants
In One Sentence
Attention lets AI focus on the most relevant parts of text, understanding context like a human highlighting important passages.
🔗 Enjoying these? Follow for daily ELI5 explanations!
Making complex tech concepts simple, one day at a time.
Top comments (0)