DEV Community: Javier Aguirre

Bag of Words & TF-IDF: How NLP Turns Text into Numbers

Javier Aguirre — Mon, 14 Sep 2026 11:00:00 +0000

If you’ve been following this series, you’ve already seen NLP do some impressive things with language.

You’ve seen how tokenization breaks raw text into individual pieces a program can work with. You’ve seen POS tagging assign grammatical labels to words. You’ve seen dependency parsing map the relationships between words in a sentence. And you’ve seen word sense disambiguation use context to determine whether “bank” refers to a financial institution or the side of a river.

Those techniques are far from academic curiosities. They’re useful in their own right and continue to play important roles in many real-world systems. Search engines, information extraction pipelines, document processing systems, and domain-specific NLP applications often rely on linguistic analysis to improve accuracy and relevance.

But there is another challenge we haven’t addressed yet.

Many of the algorithms used for search, ranking, classification, clustering, and machine learning operate on numerical representations rather than words and linguistic labels alone. To compare documents mathematically, measure similarity, train models, or rank search results, we need a way to convert text into vectors of numbers.

This process is known as text vectorization.

Bag of Words and TF-IDF are two of the foundational approaches to solving that problem. They don’t replace tokenization, POS tagging, dependency parsing, or other linguistic techniques. Instead, they provide a numerical representation that can be used alongside them in larger NLP systems.

These methods are simple enough to understand in an afternoon, powerful enough to have driven production systems for decades, and still widely used today. Modern search engines, retrieval systems, and machine learning pipelines often combine classical techniques such as TF-IDF and BM25 with linguistic analysis and neural models rather than choosing one approach over another.

In this post, we’ll explore Bag of Words and TF-IDF from first principles: the intuition behind them, the mathematics that makes them work, and practical Python examples you can run yourself.

The Core Problem: Computers Can’t Read Text

Imagine you’re building a spam filter. You want your program to look at an email and decide: spam or not spam?

To do that, you need to compare emails somehow. You need to ask questions like: does this email look more like the spam emails I’ve seen, or more like the legitimate ones?

But you can’t compare raw text directly in any meaningful mathematical way. What would it even mean to subtract one sentence from another? How do you calculate the “distance” between “Win a free prize!” and “Your invoice is attached”?

What you need is a way to represent each document as a set of numbers (a vector) so that you can apply all the usual mathematical tools to them: measuring distance, finding similarities, feeding them into classifiers.

This transformation from raw text into numerical vectors is called text vectorization. It’s the foundation that everything else in NLP sits on.

One important thing to set expectations correctly before we dive in: the classical methods we’re covering today (Bag of Words and TF-IDF) capture shared vocabulary, not meaning. Two documents get similar vectors if they use many of the same words, not simply because they express the same idea. A document about “automobiles” and one about “cars” could look quite different to these methods, even though they cover the same subject. That’s one of the limitations we’ll come back to, and a big part of why word embeddings and transformers were developed later.

Bag of Words and TF-IDF are the two classic approaches to text vectorization.

Bag of Words: Counting What’s There

The Basic Idea

The bag of words model is built on a beautifully simple premise: a document can be represented by the words it contains, and how often they appear.

That’s it. You throw away sentence structure, word order, grammar: everything except the raw inventory of words and their counts.

Why “bag”? Because imagine tipping all the words out of a document into a bag and shaking it. The structure is gone. The order is gone. What’s left is just a jumbled collection of words. That jumbled collection is your representation.

Here’s a concrete example. Suppose you have three short documents:

Document 1: “The cat sat on the mat” Document 2:”The cat sat on the hat” Document 3:”The dog lay on the mat”

The first step is to build a vocabulary: the complete list of unique words across all documents.

Vocabulary: [the, cat, sat, on, mat, hat, dog, lay]

Now you represent each document as a vector of word counts — one number per word in the vocabulary:

Each row is now a vector of numbers. Documents 1 and 2 share most of the same words, so their vectors look similar. Document 3 shares fewer words with the others, which shows up in the numbers too.

You’ve just turned text into maths.

Notice something about these vectors: most of the entries are zero. Each document only contains a handful of the words in the full vocabulary, so most positions are empty. This is called a sparse vector, and it’s a defining characteristic of both Bag of Words and TF-IDF. In a real corpus with a vocabulary of tens or hundreds of thousands of words, a typical document might use only a few hundred of them, meaning the vast majority of entries in its vector are zero.

This sparsity has practical consequences. Sparse vectors can be stored and computed with efficiently using specialised data structures, but they’re very high-dimensional, and all those zeros mean the vectors don’t capture much about what words mean, only which words appear. Later in this series we’ll look at dense embeddings (compact vectors where every entry carries meaning) which take a fundamentally different approach to representing text. But sparse vectors remain useful and widely deployed, as we’ll see.

Bag of Words (BoW) — Sparse Vector Example

What Bag of Words Is Actually Good For

Once your documents are vectors, you can do real computation on them. You can measure how similar two documents are by comparing their vectors. You can feed them into a machine learning classifier. You can cluster them, search them, rank them.

The bag of words model works surprisingly well for many tasks:

Text classification: Is this review positive or negative? The presence of words like “great,” “loved,” “excellent” versus “terrible,” “disappointed,” “broken” carries a lot of signal.
Spam detection: Certain words and phrases appear much more often in spam than in legitimate email.
Document similarity: Two documents covering the same topic will tend to use many of the same words.

The key insight is that for a lot of real-world tasks, what matters most is which words are present, not the exact order they appear in.

The Obvious Limitations

Bag of words is simple by design, and that simplicity comes with real costs.

Word order is lost entirely. “The dog bit the man” and “The man bit the dog” produce identical bag of words vectors, even though they mean opposite things.

Common words drown out meaningful ones. Words like “the,” “a,” “is,” “and” appear in almost every document. They dominate the word counts but carry almost no useful information. You can partially fix this with a stop word list (a predefined list of common words to ignore) but the underlying problem remains.

Every word is treated as equally important. A document about machine learning that mentions “neural” twice gets the same weight as one that mentions it twenty times. And a rare technical term that appears once gets the same weight as “the” appearing fifty times.

That last problem is exactly what TF-IDF was designed to solve.

TF-IDF: Not All Words Are Created Equal

The Intuition

Think about the word “machine” in a collection of technology articles. It probably appears in a lot of documents. It’s somewhat useful for identifying tech content, but it’s not very distinctive.

Now think about the word “backpropagation.” If that word appears in a document, you know a lot about what that document is about. It’s rare in most text, but highly specific and meaningful in context.

The core intuition behind TF-IDF is: a word is important if it appears a lot in this specific document, but not in many other documents.

Words that appear everywhere are not very useful for distinguishing one document from another. Words that appear in only a few documents are highly distinctive. A good word representation should reward distinctive words and discount common ones.

TF-IDF (Term Frequency-Inverse Document Frequency) formalises this intuition with a simple formula.

Term Frequency

Term frequency measures how often a word appears in a document.

The simplest version is a raw count: if “neural” appears 4 times in a document, its term frequency is 4. In practice, implementations vary — some use the raw count, some normalise by dividing by the total number of words in the document (so longer documents don’t automatically produce higher scores), and some apply logarithmic scaling to reduce the impact of words that appear very many times. Scikit-learn’s TfidfVectorizer, for instance, uses a specific variant under the hood. The core idea is the same across all of them: measure how prominent this word is within this document.

The normalised version looks like this:

TF(word, document) = (number of times word appears in document) / (total words in document)

For example, if “neural” appears 4 times in a 200-word document:

TF = 4 / 200 = 0.02

This measures local importance: how much does this word dominate this particular document?

Inverse Document Frequency

Inverse document frequency measures how rare or common a word is across the entire collection.

IDF(word) = log( total number of documents / number of documents containing word )

If a word appears in every document, the fraction inside the log becomes 1, and log(1) = 0. The IDF is zero. That word gets no weight at all.

If a word appears in only 1 out of 1,000 documents, the fraction is 1,000, and log(1,000) is a large number. That word gets a high IDF score: it’s highly distinctive.

The logarithm is there to compress the scale. Without it, a word appearing in 1 out of 1,000,000 documents would get a weight a thousand times higher than a word appearing in 1 out of 1,000. The log smooths that out.

Putting Them Together

The TF-IDF score for a word in a document is simply:

TF-IDF(word, document) = TF(word, document) × IDF(word)

Words that are frequent in this document and rare across the collection get high scores. Words that are common everywhere get scores near zero.

Let’s work through a simple example. Suppose you have a collection of 1,000 news articles.

Even though “the” has the highest term frequency, it gets a TF-IDF of zero because it appears in every document. “Candidacy” is rarer and more distinctive, so it gets the highest score despite appearing less often in this specific article.

This is exactly what you’d want. If you’re trying to understand what makes this article distinct, “the” tells you nothing. “Candidacy” tells you a lot.

TF-IDF Scoring VIsualizaion

The TF-IDF Vectorizer: From Concept to Tool

In practice, you rarely compute TF-IDF by hand. You use a TF-IDF vectorizer: a ready-made tool that takes a collection of documents and transforms each one into a TF-IDF vector automatically.

The vectorizer handles all the bookkeeping: building the vocabulary, computing term frequencies, calculating IDF scores across the entire corpus, and assembling the final document-term matrix.

The output is the same shape as a bag of words matrix (rows are documents, columns are words) but now each cell contains a TF-IDF score instead of a raw count.

TF-IDF in Python

Python’s scikit-learn library includes a TfidfVectorizer that makes this trivially easy.

from sklearn.feature_extraction.text import TfidfVectorizer  

# Your documents  
documents = [  
    "The cat sat on the mat",  
    "The cat sat on the hat",  
    "The dog lay on the mat"  
]  

# Create and fit the vectorizer  
vectorizer = TfidfVectorizer()  
tfidf_matrix = vectorizer.fit_transform(documents)  

# See the vocabulary  
print(vectorizer.get_feature_names_out())  
# ['cat', 'dog', 'hat', 'lay', 'mat', 'on', 'sat', 'the']  

# See the TF-IDF scores for document 1  
print(tfidf_matrix[0].toarray())

The fit_transform call does everything at once: it learns the vocabulary and IDF weights from your documents (fit), then transforms each document into a TF-IDF vector (transform).

If you later want to transform new documents using the same vocabulary and IDF weights (which you almost always do when building a real system) you call transform alone:

new_docs = ["The cat lay on the hat"]  
new_vectors = vectorizer.transform(new_docs)

You can also use CountVectorizer if you just want raw word counts (the bag of words approach):

from sklearn.feature_extraction.text import CountVectorizer  

bow_vectorizer = CountVectorizer()  
bow_matrix = bow_vectorizer.fit_transform(documents)

Both vectorizers support useful parameters for cleaning your text automatically:

vectorizer = TfidfVectorizer(  
    stop_words='english',       # Remove common English stop words  
    max_features=10000,         # Only keep the 10,000 most frequent words  
    ngram_range=(1, 2),         # Include both single words and two-word phrases  
    min_df=2                    # Ignore words that appear in fewer than 2 documents  
)

A Practical Example: Finding Similar Documents

Here’s a slightly more realistic use case: using TF-IDF vectors to find which documents are most similar to a query.

from sklearn.feature_extraction.text import TfidfVectorizer  
from sklearn.metrics.pairwise import cosine_similarity  
import numpy as np  

# A small collection of descriptions  
documents = [  
    "Machine learning algorithms learn patterns from data",  
    "Deep learning uses neural networks with many layers",  
    "Natural language processing handles text and speech",  
    "Computer vision processes images and video",  
    "Neural networks are inspired by the human brain"  
]  

# Fit and transform  
vectorizer = TfidfVectorizer()  
tfidf_matrix = vectorizer.fit_transform(documents)  

# A search query  
query = ["neural networks and deep learning"]  
query_vector = vectorizer.transform(query)  

# Calculate similarity between query and all documents  
similarities = cosine_similarity(query_vector, tfidf_matrix).flatten()  

# Rank documents by similarity  
ranked = np.argsort(similarities)[::-1]  
print("Most similar documents:")  
for i in ranked:  
    print(f"  {similarities[i]:.3f} — {documents[i]}")

Output:

Most similar documents:  
  0.712 — Neural networks are inspired by the human brain  
  0.634 — Deep learning uses neural networks with many layers  
  0.000 — Machine learning algorithms learn patterns from data  
  0.000 — Natural language processing handles text and speech  
  0.000 — Computer vision processes images and video

The query “neural networks and deep learning” correctly identifies the two most relevant documents. Documents that don’t mention neural networks at all score zero.

Notice what TF-IDF is actually doing here: it’s matching shared terms, not understanding concepts. It found the right documents because the query and those documents happen to use the same words. If one of our documents had said “artificial neurons arranged in layers” instead of “neural networks,” TF-IDF would have given it a score of zero. The terminology doesn’t overlap, so the vectors don’t either. This is the core limitation of all sparse, vocabulary-based approaches. It works well when related documents use consistent terminology, which is true in many real-world settings. When it breaks down, that’s where dense embeddings and semantic search come in.

This term-matching approach is essentially how document search worked for decades — and in many systems, it still does.

Bag of Words vs TF-IDF: When to Use Which

Both approaches transform text into vectors. The difference is what those numbers mean and how much signal they carry.

Use Bag of Words (raw counts) when: — Your task is simple and you want maximum interpretability — You’re working with very short texts where document length doesn’t vary much — You’re doing a quick baseline before trying something more sophisticated

Use TF-IDF when: — Your documents vary in length (longer documents shouldn’t automatically dominate) — You want distinctive terms to be weighted more heavily than ubiquitous ones — You’re doing search, document similarity, or text classification where specificity matters

In practice, TF-IDF almost always outperforms raw bag of words for text classification and search tasks. It’s the default choice when you want a classical vectorization approach.

Where These Methods Fit in Modern NLP

If TF-IDF has been around since the 1970s, you might wonder whether it’s still worth learning.

The honest answer: yes, absolutely. And not just for historical context.

Transformer models like BERT and GPT brought contextual embeddings to NLP, which capture meaning, sentence structure, and semantic relationships in ways sparse methods can’t. That was a genuine step forward for many tasks. But it didn’t make sparse retrieval disappear. The story is more nuanced than “new methods replaced old ones.”

In practice, many production systems today use hybrid architectures: they run a sparse retrieval step (BM25, TF-IDF) alongside a dense neural retrieval step, then combine the results. Each component handles what it’s good at. Sparse methods are fast, interpretable, and excellent at exact keyword matching. Dense methods are better at semantic similarity: finding documents that mean the same thing even when they use different words. Neither has made the other obsolete. Classical NLP techniques — tokenization, lemmatization, POS tagging, dependency parsing, TF-IDF, and the retrieval pipelines built around them — remain important parts of real production systems.

NLP Vectoriation Progression

TF-IDF’s direct successor, BM25, is one of the most widely deployed ranking algorithms in the world right now. BM25 takes the core TF-IDF idea and fixes two specific weaknesses: it handles diminishing returns on term frequency (the 50th mention of a word shouldn’t count as much as the 5th), and it normalises more gracefully for document length. Elasticsearch, OpenSearch, and most modern search engines use BM25 as their default relevance model. It’s also the sparse retrieval component in many of those hybrid systems mentioned above. We’ll cover BM25 in the next post. Once you’ve understood TF-IDF, it clicks almost immediately.

Understanding Bag of Words and TF-IDF gives you the conceptual foundation to understand why every subsequent approach works differently, and what specific problem each one was built to solve. Dense embeddings exist because sparse methods can’t capture meaning across different vocabulary. BM25 exists because raw TF-IDF has specific edge cases in term weighting and length normalisation. Each step in the progression was motivated by a concrete limitation of what came before — not a wholesale replacement of it.

Every serious NLP practitioner knows these methods. They’re the starting point for a reason.

What Comes Next

We’ve now covered how text gets turned into numbers — the prerequisite for almost every NLP task that follows.

The immediate next step is BM25: the direct evolution of TF-IDF that fixes its two most significant weaknesses and has become the default ranking algorithm inside most modern search engines. If you search anything on Elasticsearch or use a hybrid retrieval system, BM25 is almost certainly involved. Because it builds directly on what you’ve just learned, it’s the natural next post.

After that, we’ll go deeper into what you can do with these vectors — the machine learning models that sit on top of text vectorization, and the neural architectures that eventually pushed beyond classical methods altogether. That includes word embeddings like Word2Vec and GloVe, which represent words as dense vectors rather than sparse counts, and eventually the contextual embeddings that power modern AI.

Learn This at Fondra Labs

This post is part of our NLP Foundations series, where we build up practical AI knowledge one concept at a time, from text processing basics all the way to the systems powering modern AI.

At Fondra Labs, we teach AI from production reality, not hype. Every topic in this series is here because it genuinely matters when you sit down to build something real.

If this was useful, explore the rest of the blog. We cover machine learning, deep learning, NLP, and the practical skills that turn understanding into building.

Originally published at https://fondralabs.com.

Word Sense Disambiguation: How NLP Understands Word Meaning in Context

Javier Aguirre — Fri, 11 Sep 2026 11:00:00 +0000

Here is a sentence that will break most computers.

“I went to the bank.”

Did you go to a financial institution to withdraw money? Or did you sit down by the bank of a river? Both readings are perfectly valid. As a human, you’d resolve this instantly, probably without even noticing there was ambiguity to resolve. You’d use context, common sense, and everything else you know about the world to pick the right meaning without thinking twice.

For a computer, this is a genuinely hard problem. And it turns out that language is absolutely full of it.

Word sense disambiguation (WSD) is the NLP term for solving this: given a word that could mean multiple things, figure out which meaning the writer actually intended. As a concept, it sits at the heart of what makes language understanding difficult. As a standalone task with its own dedicated pipeline stage, it has a more specific and nuanced role in modern systems, which we’ll get into shortly.

Why Language Is So Ambiguous

Before diving into how WSD works, it helps to appreciate just how widespread the problem is.

Most common words in English carry more than one meaning. The word “bank” has over ten distinct senses in major dictionaries. “Run” has over thirty. Even simple words like “bright” (intelligent? luminous? cheerful?) or “cold” (temperature? illness? emotional distance?) carry multiple meanings that shift depending on context.

This is called lexical ambiguity: a single word form with multiple possible meanings. It’s not an edge case or a quirk of English. It’s a fundamental property of how human language works. Words evolve over time, get borrowed across domains, and accumulate meanings. The result is a vocabulary where almost every high-frequency word is ambiguous to some degree.

There’s a related but slightly different phenomenon worth knowing: semantic ambiguity. This is where the ambiguity doesn’t come from a single word but from the structure of a phrase or sentence. “I saw the man with the telescope” is semantically ambiguous because the phrase “with the telescope” could modify either “saw” or “the man.” Both the word-level and phrase-level forms of ambiguity fall under the broader challenge WSD is designed to address.

Common words with multiple meanings illustrating lexical ambiguity in natural language

What Is Word Sense Disambiguation?

Word sense disambiguation is the task of automatically identifying which sense of a word is being used in a given context.

In the traditional framing, the inputs are a piece of text and a target word within it. The output is a label identifying which meaning the word carries in that specific context. Those labels typically come from a dictionary or lexical resource like WordNet, which organises words into sets of synonyms (called synsets) and defines each sense separately.

For the sentence “The surgeon operated on the patient,” a WSD system would correctly identify that “operated” refers to performing surgery, not running a machine or managing a business. For “The technician operated the crane,” it would pick the machinery sense. Same word. Different contexts. Different meanings.

It’s worth noting that this traditional framing assumes word meaning can be mapped onto a fixed inventory of discrete dictionary senses. That’s a useful engineering abstraction, but it’s not the only way to think about meaning. Modern NLP often represents meaning as continuous vectors in high-dimensional space rather than selecting from a predefined list of senses. Some researchers view dictionary sense inventories as practical tools for structured tasks rather than fundamental representations of how meaning works. Both views have merit, and understanding the distinction helps you choose the right approach for a given problem.

What makes this genuinely interesting is that humans do it effortlessly and continuously. Every sentence you read contains multiple potentially ambiguous words, and you resolve all of them simultaneously without effort. A computational system has to learn to replicate that, either by selecting from a sense inventory or by building rich contextual representations that capture meaning without explicit labels.

How WSD Systems Work

There are several approaches to word sense disambiguation, and they’ve evolved significantly over the decades.

The earliest systems were knowledge-based: they used structured resources like WordNet to compare the definitions of candidate senses against the surrounding context. If you’re trying to disambiguate “bank” and the surrounding words include “river,” “water,” and “fish,” the riverbank sense scores higher because its definition overlaps more with those words. This approach is interpretable and requires no training data, but it struggles when context is sparse or unusual.

Supervised machine learning approaches treat WSD as a classification problem. Given a word and its surrounding context (the words before and after it), a classifier learns to predict the correct sense from labelled training examples. These methods work well when you have enough annotated data for the specific words you care about, which is often the bottleneck. Building a labelled WSD dataset is slow and expensive because it requires human annotators to read each sentence and select the intended sense.

Neural approaches using contextual word embeddings changed the game significantly. Models like BERT produce a different vector representation of a word depending on the context it appears in. The word “bank” in a sentence about finance gets a different embedding than “bank” in a sentence about rivers, even though the surface form is identical. This means disambiguation happens inside the model’s representations rather than through an explicit sense-selection step.

This is the key shift in modern NLP. Transformer models and large language models perform sense disambiguation implicitly, as an emergent property of how they process context, rather than through a dedicated WSD component. When you ask ChatGPT a question containing an ambiguous word, it almost always resolves the ambiguity correctly from context without any explicit disambiguation step. That capability emerged from training on vast amounts of text. There is no separate WSD module running underneath. The disambiguation is baked into the contextual representations the model builds for every word it processes.

This means that in most modern NLP pipelines, you won’t find a dedicated WSD stage. The problem hasn’t gone away. It’s been absorbed.

Three approaches to word sense disambiguation: knowledge-based, supervised, and neural contextual embeddings

Lexical Semantics: The Bigger Picture

Word sense disambiguation doesn’t exist in isolation. It’s part of a broader field called lexical semantics: the study of word meaning and how words relate to each other.

Lexical semantics is interested in questions like: what does it mean for two words to be synonyms? How do words like “dog” and “animal” relate (one is a type of the other)? Why do “hot” and “cold” feel like opposites while “hot” and “warm” feel like a scale?

These relationships between words are encoded in resources like WordNet, a large lexical database where words are organised into synonym sets and connected by semantic relationships: synonymy (same meaning), antonymy (opposite meaning), hypernymy (broader category), and hyponymy (narrower category). “Dog” is a hyponym of “animal.” “Animal” is a hypernym of “dog.”

Understanding these relationships matters because WSD is not just about picking the right dictionary entry. It’s about understanding how word meaning is structured in language. A system that understands that “operated” in a medical context belongs to the same semantic neighbourhood as “surgery,” “incision,” and “patient” is doing something more interesting than lookup. It’s reasoning about meaning.

Where Explicit WSD Still Has a Role

Most modern NLP systems don’t include a dedicated WSD stage. But that doesn’t mean the problem is solved or that explicit disambiguation is never useful. There are specific settings where it still earns its place, and understanding them helps you know when to reach for it.

In biomedical NLP, the word “discharge” can mean a patient leaving hospital, a fluid emission, or an electrical release. For systems that extract structured information from clinical notes and feed it into downstream databases or alerts, explicit WSD with a domain-specific sense inventory gives teams control and auditability that a black-box language model doesn’t. When a decision needs to be explainable and traceable, knowing exactly which sense was assigned and why matters.

In legal document analysis, words like “party,” “consideration,” and “execution” carry technical legal senses that differ sharply from their everyday meanings. Explicit terminology mapping using legal ontologies and controlled vocabularies is a common pattern in legal NLP, and WSD is the mechanism that connects surface word forms to those structured definitions.

Knowledge graph construction and ontology mapping are probably where explicit WSD remains most actively used in practice. When you’re building a structured knowledge base from unstructured text, you need to map entity mentions and relation phrases to specific nodes in a taxonomy. That requires knowing exactly which sense of a word is intended, and the output needs to be a discrete label, not a vector. Contextual embeddings alone don’t give you that.

In machine translation, it’s worth being accurate about what’s changed. Explicit WSD was historically important in rule-based and early statistical translation systems, where the correct translation of a word genuinely depended on resolving its sense first. Modern neural machine translation systems handle disambiguation implicitly through contextual modelling, just as LLMs do, so dedicated WSD modules are not a standard part of current MT pipelines.

The honest summary: explicit WSD survives in specialised settings where you need structured, auditable, ontology-aligned outputs. For general language understanding in most production systems, transformer models and LLMs have absorbed the disambiguation problem into their contextual representations, making a dedicated WSD stage unnecessary.

Language models don’t read words. They read words in context, and context is everything.

The State of WSD Today

It’s worth stepping back and being clear about where this all stands.

Resolving the meaning of words from context is not optional in language understanding. It’s fundamental. Every NLP system that processes real text is dealing with ambiguity constantly, whether it acknowledges it explicitly or not.

What has changed is how that disambiguation happens. Explicit WSD, where a system selects from a fixed sense inventory using rules or a dedicated classifier, was the dominant approach before transformer models arrived. Today it’s a specialist tool. Most production NLP systems, from search engines to document classifiers to LLM-powered assistants, perform disambiguation implicitly through the contextual representations learned by transformer models. There is no dedicated WSD stage. The problem is handled as a side effect of how modern models process language.

Explicit WSD survives where structured outputs are required: ontology mapping, knowledge graph construction, terminology alignment in biomedical and legal domains. In those settings, the ability to produce a named, auditable sense label still has genuine value.

For everyone else, understanding WSD is valuable not because you’ll necessarily build a WSD system, but because it helps you understand what language models are actually doing and why context matters so deeply to how language works.

What Comes Next

Word sense disambiguation is the point in this series where we move from structure to meaning. We’ve covered how text is broken into tokens, how words get grammatical labels, and how sentences get mapped into dependency trees. Now we understand how the meaning of individual words gets resolved from context.

The next step is understanding how that meaning gets represented as numbers that computers can actually work with. That leads us to Bag of Words and TF-IDF: the foundational techniques for turning text into numerical representations, and the starting point for almost every classical NLP system ever built.

Learn This at Fondra Labs

This post is part of our NLP Foundations series, where we build up practical AI knowledge one concept at a time, from text processing basics all the way to the systems powering modern AI.

At Fondra Labs, we teach AI from production reality, not hype. Every topic in this series is here because it genuinely matters when you sit down to build something real.

If this was useful, explore the rest of the blog. We cover machine learning, deep learning, NLP, and the practical skills that turn understanding into building.

Originally published at https://fondralabs.com.

Dependency Parsing: How NLP Understands Sentence Structure

Javier Aguirre — Wed, 09 Sep 2026 11:00:00 +0000

You now know what each word is. But do you know what each word does?

In the previous post, we covered Part of Speech Tagging: the process of labelling each word as a noun, verb, adjective, and so on. That tells you the category of every word in a sentence. But it doesn’t tell you how those words connect to each other.

Take this sentence: “The dog bit the man.”

POS tagging tells you that “dog” is a noun and “bit” is a verb. But it doesn’t tell you that “dog” is the one doing the biting, or that “man” is the one receiving it. For a lot of NLP tasks, that distinction matters enormously.

Dependency parsing is how NLP systems figure out those relationships. It maps the grammatical connections between words, revealing who did what to whom and how every part of a sentence fits together.

What is dependency parsing?

Before we get into how computers do this, it helps to understand the underlying idea.

Dependency grammar is a theory of language structure that describes sentences as networks of relationships between words. Instead of breaking a sentence into nested phrases (like traditional grammar diagrams), dependency grammar draws direct links between individual words.

Every word in a sentence (except one) depends on another word. The word it depends on is called its head. The relationship between them is called a dependency relation, and these relations have names that describe the grammatical role: subject, object, modifier, determiner, and so on. The one word that depends on nothing is the root of the sentence, usually the main verb.

In “The dog bit the man,” the structure looks like this:

“bit” is the root (nothing governs it)
“dog” depends on “bit” as the subject (nsubj)
“man” depends on “bit” as the object (obj)
“The” (first one) depends on “dog” as a determiner (det)
“the” (second one) depends on “man” as a determiner (det)

Every word is accounted for. Every relationship is named. The sentence has been turned into a map.

When you draw out all those relationships visually, you get what’s called a dependency tree. It’s called a tree because the structure branches outward from a single root, just like a real tree. The main verb sits at the top. Subjects, objects, and modifiers hang below it. Determiners and other function words hang below those. Every node is a word, every edge is a labelled relationship, and every word appears exactly once with no circular chains.

Here’s what the dependency tree for “The quick brown fox jumped over the fence” looks like conceptually:

The tree makes the structure of the sentence explicit and machine-readable. A system can now navigate it programmatically: find the subject of the main verb, find all modifiers of a given noun, find the object of any action.

Dependency grammar maps every word to its head with a named relationship label

Parse Trees: Two Flavours

You’ll often hear the term parse tree used when people talk about dependency parsing, and it’s worth understanding what it means and how it relates to what we’ve just described.

A dependency parse tree (what we’ve been describing) connects individual words to each other with typed relationship labels. The structure is flat in the sense that it connects words directly, without grouping them into phrases.

A constituency parse tree (also called a phrase structure tree) takes a different approach. Instead of connecting words to words, it groups words into nested phrases. A noun phrase contains a determiner and a noun. A verb phrase contains a verb and its noun phrase. Those phrases nest inside a sentence node at the top. The two approaches reveal the same underlying sentence structure but from different angles.

In practice the choice between them comes down to the task:

Dependency trees are word-to-word. Better for tasks that need to know relationships between specific words: who did what to whom.
Constituency trees are phrase-based. Better for tasks that need to understand the hierarchical structure of a sentence: how clauses nest inside each other.

In modern NLP, dependency parsing is more widely used in production systems. It’s faster to compute, easier to work with programmatically, and sufficient for most downstream tasks.

Constituency parse tree vs dependency parse tree for the same sentence showing the structural difference

Both dependency parsing and constituency parsing are forms of syntactic analysis: the broader process of examining the grammatical structure of a sentence. POS tagging is also a form of syntactic analysis. They’re all part of the same family of techniques, each revealing a different layer of grammatical structure. In NLP pipelines, syntactic analysis typically sits between basic text processing (tokenization, POS tagging) and higher-level understanding (semantic analysis, information extraction). It provides the structural backbone that makes deeper analysis possible.

How Dependency Parsing Works

Modern dependency parsers use neural networks, but the core task is always the same: for each word in a sentence, find its head and label the relationship.

There are two main algorithmic approaches. Transition-based parsing works through the sentence from left to right, maintaining a stack of words currently being processed. At each step, a classifier decides between a small set of actions: shift the next word onto the stack, create a dependency between two words, or move on. Because decisions are made one step at a time, this approach runs in linear time and is fast enough for production use.

Graph-based parsing takes a different route. Instead of building the tree step by step, it scores all possible edges between all pairs of words and then finds the tree that maximises the total score. This is slower but tends to be more accurate, especially for longer sentences with complex structure. Most state-of-the-art parsers today combine neural networks with graph-based approaches, using Transformer-based representations to score edges with rich contextual information.

Why It Matters in Practice

Like POS tagging, dependency parsing is rarely the end goal. It’s a layer in a pipeline that makes other things possible.

Information extraction uses dependency trees to find subject-verb-object triples at scale. Given a corpus of news articles, you can extract every event of the form “company acquired company” or “person said statement” by querying the parse tree structure. This is faster and more precise than hoping a language model happens to extract the right thing.

Question answering systems use dependency parsing to understand what a question is actually asking. “Who did the dog bite?” and “Who bit the dog?” have identical words but opposite meanings. The dependency tree makes that difference explicit.

Machine translation relies on syntactic structure to produce grammatically correct output in the target language. Word order differs between languages, and dependency relations provide language-agnostic structure that translation systems can map across.

Clinical NLP uses dependency parsing to extract medication dosages, symptoms, and diagnoses from clinical notes. A rule like “find the noun that is the object of the verb ‘prescribed’” is far more reliable than keyword matching when clinical language is ambiguous and variable.

Coreference resolution (figuring out that “he” in one sentence refers to “John” in a previous one) uses dependency structure to track entities across sentences. The grammatical role of a pronoun is a strong signal for what it refers to.

The honest take on modern NLP: large language models have reduced how much explicit dependency parsing is done in cutting-edge research. But in production systems where interpretability, speed, and precision matter, dependency parsing remains a practical and widely used tool. spaCy runs a dependency parser by default on every document you process. You might not use the output directly, but it’s there, and the components that do use it quietly improve everything downstream.

The structure of a sentence is not decoration. It is the scaffolding that holds meaning together.

Dependency Parsing in Python with spaCy

spaCy makes dependency parsing straightforward. It runs automatically when you process text:

import spacy

nlp = spacy.load("en_core_web_sm")
doc = nlp("The dog bit the man.")

for token in doc:
    print(f"{token.text} --[{token.dep_}]--> {token.head.text}")

Output:

The --[det]--> dog
dog --[nsubj]--> bit
bit --[ROOT]--> bit
the --[det]--> man
man --[dobj]--> bit
. --[punct]--> bit

Every token shows its dependency label and the head it points to. The root points to itself. From this output, a downstream system can immediately answer: what is the subject of “bit”? Dog. What is the object? Man.

You can also visualise the tree directly in spaCy using its built-in displaCy renderer, which produces a clean arc diagram in the browser.

A dependency tree visualised as an arc diagram showing labelled relationships between words

Is Dependency Parsing Still Relevant?

Short answer: yes, especially in production.

Large language models are impressive at understanding language holistically, but they’re opaque. You can’t inspect why they interpreted a sentence a certain way. Dependency parsing gives you an explicit, interpretable representation of sentence structure that you can query, validate, and debug.

For teams building NLP pipelines in healthcare, legal tech, or finance, where decisions need to be explainable and auditable, dependency parsing is not a relic of the past. It’s a practical engineering choice that trades some accuracy for a lot of transparency and control.

Even in LLM-powered systems, dependency parsing often runs in the preprocessing layer: cleaning up the input, extracting structured signals, or filtering out irrelevant sentences before the expensive model ever sees them. It earns its place not by being the most powerful tool, but by being fast, reliable, and interpretable in a world where those qualities are often more valuable than raw performance.

What Comes Next

Dependency parsing gives you the grammatical structure of a sentence. You now know what each word is (POS tags) and how each word connects to the others (dependency relations).

The next step is moving from structure to meaning. That’s where Word Sense Disambiguation comes in: the process of figuring out which meaning a word carries when it could mean several different things. It’s where NLP starts crossing from grammar into genuine language understanding.

Learn This at Fondra Labs

This post is part of our NLP Foundations series, where we build up practical AI knowledge one concept at a time, from tokenization and POS tagging all the way to retrieval systems and production pipelines.

At Fondra Labs, we teach AI from production reality, not hype. Every topic in this series is here because it genuinely matters when you sit down to build something real.

If this was useful, explore the rest of the blog. We cover machine learning, deep learning, NLP, and the practical skills that turn understanding into building.

Originally published at https://fondralabs.com.

Part of Speech Tagging: How NLP Understands Grammar

Javier Aguirre — Mon, 07 Sep 2026 11:00:00 +0000

You learned this in school, even if you’ve forgotten most of it.

Nouns are things. Verbs are actions. Adjectives describe. Adverbs modify. At some point a teacher made you underline the subject of a sentence or circle the verb, and then you moved on with your life.

But here’s the thing: that grammatical knowledge you picked up in school is exactly what NLP systems need to understand language. Before a model can extract meaning from text, it needs to know what role each word is playing. Is “bank” a noun or a verb? Is “fast” an adjective or an adverb? The same word can be both, depending on context.

Part of speech tagging is how NLP systems figure that out. It’s one of the foundational steps in language understanding, and it quietly powers a huge number of the NLP applications you use every day.

What Is Part of Speech?

A part of speech is a grammatical category that tells you what role a word plays in a sentence.

Every word in a sentence belongs to at least one category. The main ones you’ll encounter in NLP are:

Noun (NN): a person, place, thing, or concept. “dog,” “London,” “happiness”
Verb (VB): an action or state. “run,” “is,” “understand”
Adjective (JJ): describes a noun. “fast,” “blue,” “complicated”
Adverb (RB): modifies a verb, adjective, or other adverb. “quickly,” “very,” “not”
Pronoun (PRP): replaces a noun. “he,” “she,” “it,” “they”
Preposition (IN): shows relationships between words. “in,” “on,” “at,” “between”
Conjunction (CC): connects words or clauses. “and,” “but,” “or”
Determiner (DT): introduces a noun. “the,” “a,” “this,” “some”

In NLP, these categories are assigned short codes called Penn Treebank tags, which is why you’ll often see labels like NN, VBZ, or JJ rather than full words. Different frameworks use slightly different tag sets, but the underlying categories are consistent.

Common part of speech tags used in NLP with their Penn Treebank codes and examples

What Is Part of Speech Tagging?

Part of speech tagging (also called POS tagging) is the process of automatically assigning a grammatical label to every word in a sentence.

Given the sentence:

“The quick brown fox jumps over the lazy dog.”

A POS tagger produces:

The/DT quick/JJ brown/JJ fox/NN jumps/VBZ over/IN the/DT lazy/JJ dog/NN

Every word gets its tag. The model has now turned an unstructured sentence into structured grammatical information that downstream systems can reason about.

This sounds straightforward, but the hard part is ambiguity. The same word can be different parts of speech depending on context, and it happens constantly in real language.

Take the word “run”:

A POS tagger has to resolve this ambiguity correctly, using the surrounding words as context. That’s where the actual intelligence lies.

Why Does POS Tagging Matter?

On its own, knowing that a word is a noun or a verb might seem like a small thing. But POS tags unlock a surprisingly wide range of downstream capabilities.

Named Entity Recognition relies on POS tags to know where to look. Proper nouns (NNP) are strong signals for names of people, places, and organisations. A system that already knows which tokens are proper nouns has a massive head start on finding entities.

Dependency parsing (figuring out the grammatical relationships between words) depends on POS tags. You can’t determine whether a word is the subject or object of a verb without first knowing which words are verbs and which are nouns.

Word sense disambiguation uses POS tags to narrow down meaning. “Bank” as a noun has different senses than “bank” as a verb. Knowing the part of speech cuts the ambiguity in half before any deeper analysis begins.

Information extraction uses POS patterns to find specific structures. A rule like “find all noun phrases followed by a verb” can extract subject-action pairs from text at scale. POS tags make those patterns possible.

Search and indexing systems use POS tags to focus on content words (nouns, verbs, adjectives) and skip function words (the, a, in, of) that carry little meaning on their own.

How POS Tagging Works

Modern POS taggers use machine learning, but it helps to understand the intuition first.

The Rule-Based Approach

Early POS taggers were rule-based. Linguists wrote explicit rules: if a word ends in “-ing” and follows a modal verb like “will” or “can,” tag it as a verb. These rules worked surprisingly well for formal text but broke down on informal language, new vocabulary, and edge cases.

Statistical Approaches

Statistical taggers replaced hand-written rules with probabilities learned from annotated data. Given a large corpus of text where humans have manually tagged every word, the model learns: how likely is this word to be a verb? Given the previous word was a determiner, how likely is the next word to be a noun?

Hidden Markov Models (HMMs) were the dominant approach for years. They model POS tagging as a sequence problem: what’s the most likely sequence of tags given this sequence of words?

Neural Approaches

Today, most production POS taggers use neural networks, typically the same Transformer-based models used for other NLP tasks. These models learn rich contextual representations of each word and can resolve ambiguity that simpler models miss.

The key advantage is context. A neural tagger doesn’t just look at a word and its immediate neighbours. It considers the entire sentence, which is exactly what’s needed to correctly tag words like “run” or “fast” that change meaning depending on what surrounds them.

How a POS tagger processes a sentence from raw text to tagged output

Syntactic Parsing: The Next Level

POS tagging tells you what each word is. Syntactic parsing goes a step further and tells you how the words relate to each other.

In the sentence “The cat chased the mouse,” POS tagging tells you that “cat” is a noun and “chased” is a verb. Syntactic parsing tells you that “cat” is the subject of “chased” and “mouse” is the object.

There are two main types of syntactic parsing:

Dependency parsing maps out the grammatical relationships between individual words. Each word gets connected to a “head” word that it depends on, with a label describing the relationship: subject, object, modifier, and so on. This produces a tree structure called a dependency tree.

Constituency parsing breaks a sentence into nested phrases: a noun phrase, a verb phrase, a prepositional phrase. Each phrase can contain smaller phrases, producing a hierarchical tree that represents the full grammatical structure of the sentence.

Both forms of parsing build directly on POS tags. They’re the next layer of structural understanding above tagging, and they’re used in more complex NLP tasks like question answering, machine translation, and relation extraction.

Dependency parsing and constituency parsing both build on POS tags to reveal sentence structure

POS Tagging in Practice with NLTK

If you want to try POS tagging yourself, NLTK is the easiest starting point in Python.

import nltk  
nltk.download('averaged_perceptron_tagger')  
nltk.download('punkt')  

from nltk.tokenize import word_tokenize  
from nltk import pos_tag  

sentence = "The quick brown fox jumps over the lazy dog."  
tokens = word_tokenize(sentence)  
tags = pos_tag(tokens)  

print(tags)  
# [('The', 'DT'), ('quick', 'JJ'), ('brown', 'JJ'), ('fox', 'NN'),  
#  ('jumps', 'VBZ'), ('over', 'IN'), ('the', 'DT'), ('lazy', 'JJ'), ('dog', 'NN'), ('.', '.')]

Four lines of meaningful code. Every token gets its tag.

For production use, spaCy is the better choice. It’s faster, more accurate, and built for real-world text:

import spacy  

nlp = spacy.load("en_core_web_sm")  
doc = nlp("The quick brown fox jumps over the lazy dog.")  

for token in doc:  
    print(token.text, token.pos_, token.tag_)

spaCy gives you two tag levels: pos is a simplified universal tag (NOUN, VERB, ADJ), and tag is the fine-grained Penn Treebank tag (NN, VBZ, JJ). For most applications, the simplified tags are enough.

Is POS Tagging Still Used Today?

It’s a fair question. With large language models able to understand language end-to-end, does anyone still bother with POS tagging?

The answer is yes, and more than you might expect.

In production NLP systems, POS tagging is rarely the star of the show. But it quietly sits inside many of the pipelines that power real applications. Here’s where it still earns its place.

Clinical and legal NLP relies heavily on POS tagging. These domains often use rule-based extraction pipelines alongside machine learning, because rules are auditable and explainable in ways that a neural network isn’t. When a hospital system needs to extract medication names and dosages from clinical notes, a pipeline that uses POS tags to identify noun phrases and filter by surrounding context is fast, transparent, and reliable.

Information extraction at scale still uses POS-based patterns extensively. If you need to extract all “company acquired company” events from a corpus of ten million news articles, a POS-aware pattern matcher is often faster and more controllable than a fine-tuned language model, especially when your compute budget is limited.

Lightweight NLP on constrained hardware uses POS tagging as a preprocessing step to reduce the amount of text that needs to go through expensive models. Filter out irrelevant sentences using POS patterns first, then run your heavy model only on what’s left.

spaCy pipelines in production almost always include a POS tagger as a standard component. When you load a spaCy model, POS tagging happens automatically as part of the pipeline. Even if you never explicitly use the tags yourself, the downstream components like the dependency parser and named entity recogniser depend on them under the hood.

Multilingual systems often lean on POS tagging more heavily than English-only ones. For languages with complex morphology or less training data available for large models, grammatical structure provides signal that pure statistical learning can miss.

The broader point: POS tagging is a foundational layer, not a product in itself. It doesn’t do anything impressive on its own. But remove it from a serious NLP pipeline and you’ll quickly feel the gaps it was quietly filling.

The best NLP systems are not built from one powerful model. They are built from layers of structured understanding, each one making the next more accurate.

What Comes Next

POS tagging gives you the grammatical skeleton of a sentence. But grammar alone doesn’t tell you what a sentence means.

The next step is understanding how words relate to each other in terms of meaning, not just structure. That leads us to Dependency Parsing, where we look at how NLP models map out the relationships between words: who did what to whom, what modifies what, and how the parts of a sentence connect into a coherent whole.

Learn This at Fondra Labs

This post is part of our NLP Foundations series, where we build up practical AI knowledge one concept at a time, starting from the basics and going all the way to production-grade systems.

At Fondra Labs, we teach AI from production reality, not hype. Every topic in this series is chosen because it genuinely matters when you sit down to build something real.

If this was useful, explore the rest of the blog. We cover everything from machine learning fundamentals to deep learning architectures, NLP pipelines, and the practical skills that turn understanding into building.

Originally published at https://fondralabs.com.

Tokenization in NLP: How Machines Break Down Text

Javier Aguirre — Fri, 04 Sep 2026 11:00:00 +0000

Before a computer can understand a single word you write, it has to do something much more basic.

It has to figure out where one piece of text ends and the next begins.

That process is called tokenization, and it’s the first step in almost every NLP system ever built. Whether you’re using a spam filter, a search engine, or ChatGPT, tokenization happened before anything else. It’s the foundation everything else is built on.

This post explains what tokenization is, why it matters, and how it has evolved from simple word splitting into the sophisticated subword methods that power modern AI. Along the way, we’ll also cover the related concepts you’ll encounter whenever you work with text: stemming, lemmatization, and n-grams.

What Is Tokenization?

Tokenization is the process of breaking text into smaller units called tokens.

A token is just a piece of text. It might be a word, a punctuation mark, a part of a word, or even a single character. The important thing is that tokens are the units a model actually works with. Before any analysis, translation, or generation can happen, the raw text needs to be split into these pieces.

Take this sentence:

“NLP is surprisingly powerful.”

After word tokenization, it becomes:

[“NLP”, “is”, “surprisingly”, “powerful”, “.”]

That’s it. Five tokens. The model now has something it can process.

Simple in concept. But the details turn out to matter enormously.

Word tokenization splits a sentence into individual tokens including punctuation

Why Tokenization Is Harder Than It Looks

You might be thinking: can’t you just split on spaces?

For English, that gets you most of the way there. But language is messier than that.

What do you do with “don’t”? Is it one token or two (“do” and “n’t”)? What about “New York”? Splitting on spaces gives you two tokens, but it’s one concept. What about emojis, URLs, email addresses, or code snippets mixed into text?

And that’s just English. Languages like Chinese and Japanese have no spaces between words at all. Arabic and Hebrew write without vowels. German creates long compound words that are really several concepts joined together.

Tokenization that works well across all of these is a genuinely hard problem. And how you solve it has downstream consequences for everything the model learns.

Word Tokenization

The most intuitive approach is word tokenization: split the text into individual words.

This works well for clean, formal text. It’s fast, easy to understand, and produces tokens that map directly to human intuitions about language.

The limitation is vocabulary size. If you train a model on word tokens, it can only handle words it has seen before. Give it “unimaginably” when it’s only ever seen “imagine” and “imaginable”, and it’s stuck. This is called the out-of-vocabulary (OOV) problem, and it’s one of the main reasons word tokenization has been largely superseded for modern AI systems.

Stemming and Lemmatization: Reducing Words to Their Core

Before diving into more advanced tokenization, it’s worth understanding two related text processing techniques you’ll encounter constantly: stemming and lemmatization.

Both answer the same question: when should “running”, “runs”, and “ran” be treated as the same word?

Stemming

Stemming is the blunter approach. It chops off the ends of words using simple rules until you’re left with a root form called a stem.

“Running” becomes “run”. “Fishing” becomes “fish”. “Better” becomes “better” (stemming can’t handle irregular forms).

It’s fast and simple, but it’s crude. The stem doesn’t have to be a real word. “Arguing” might become “argu”. “Studies” might become “studi”. That’s fine for some applications, like topic modelling or search indexing, where you just need to group related words together. It falls apart when you need the actual meaning of a word.

Lemmatization

Lemmatization is the smarter approach. Instead of blindly chopping characters, it uses vocabulary and grammar rules to find the true base form of a word, called the lemma.

Lemmatization understands that “better” is a form of “good”, something stemming can never figure out. It’s slower and requires more linguistic knowledge, but the output is always a real, meaningful word.

The Key Difference:

“Stemming is fast and crude. Lemmatization is slower and correct. Choose based on what your task actually needs.”

When to use each:

Stemming: search indexing, topic modelling, situations where speed matters and rough grouping is enough
Lemmatization: sentiment analysis, question answering, any task where word meaning matters

Subword Tokenization: The Modern Solution

Neither word tokenization nor character tokenization is quite right for modern AI. Word tokenization has the OOV problem. Character tokenization produces sequences so long they’re computationally impractical.

The solution the field landed on is subword tokenization: split words into pieces, but not all the way down to individual characters.

The idea is elegant. Common words stay as single tokens. Rare or unknown words get split into recognisable subword pieces. “Tokenization” might become “token” + “ization”. “Unbelievable” might become “un” + “believ” + “able”.

This gives the model the best of both worlds. It can handle any word, including ones it has never seen, because it can always fall back to subword pieces. And it keeps vocabulary size manageable.

Subword tokenization splits rare words into recognisable pieces while keeping common words intact

Byte Pair Encoding (BPE)

Byte Pair Encoding (BPE) is the most widely used subword tokenization algorithm. It powers GPT, GPT-2, GPT-3, GPT-4, and many other major models.

The algorithm works like this:

Start with every character as its own token. Then repeatedly find the most common pair of adjacent tokens and merge them into a single new token. Repeat until you reach your desired vocabulary size.

So if “lo” and “w” appear together constantly, they get merged into “low”. If “low” and “er” appear together constantly, they get merged into “lower”. The algorithm keeps building up frequent sequences into single tokens.

The result is a vocabulary that naturally captures common words and common word fragments, while still being able to handle anything new by falling back to shorter pieces.

WordPiece

WordPiece is a similar approach used by BERT and many Google models. The main difference is in how it decides what to merge. BPE merges the most frequent pairs. WordPiece merges the pairs that increase the likelihood of the training data the most.

In practice, the results look similar. Subword pieces in WordPiece are often prefixed with ## to indicate they’re continuations of a word. “playing” might become “play” + “##ing”.

SentencePiece

SentencePiece takes a slightly different approach: it treats the input as a raw stream of characters (including spaces) rather than pre-tokenized words. This makes it language-agnostic. It works equally well on English, Japanese, Chinese, or any other language without needing language-specific rules. It’s used in models like T5 and many multilingual systems.

N-grams: Capturing Context Between Tokens

Once you have tokens, a natural next question is: does the order matter?

Almost always, yes. “Not good” means something very different from “good”. “Bank robbery” means something different from “robbery bank”.

N-grams are a simple way to capture this. An n-gram is a sequence of n consecutive tokens.

By looking at n-grams instead of individual tokens, models can capture short-range context. “New York” as a bigram carries different meaning than “New” and “York” separately.

N-grams were the backbone of language modelling before neural networks. They’re still used in text classification, spam detection, and search systems where simplicity and speed matter more than capturing long-range context.

The limitation is obvious: n-grams only capture context within a short window. A trigram can’t know that the word at the start of a paragraph shapes the meaning of a word at the end. That’s what neural networks, and ultimately the Transformer, were built to solve.

N-grams capture sequences of consecutive tokens: unigrams, bigrams, and trigrams

How Modern LLMs Tokenize Text

If you’ve ever used ChatGPT or Claude and noticed that pricing is measured in “tokens,” this is where that comes from.

Modern large language models use BPE or similar subword methods. Every piece of text you send gets broken into tokens before the model ever sees it. The model generates its response one token at a time.

On average, one token is roughly four characters in English, or about three-quarters of a word. “Tokenization” is two tokens. “Hello” is one. A typical paragraph is around 100 tokens.

This is also why LLMs sometimes handle unusual words or names strangely. If a word gets split into many subword pieces, the model has to work harder to reason about it as a coherent unit. The tokenization shapes what the model finds easy or difficult.

Putting It Together

Here’s a quick summary of the landscape:

Word tokenization: simple, intuitive, but can’t handle unknown words
Stemming: fast root-finding, crude, sometimes produces non-words
Lemmatization: accurate root-finding, slower, always produces real words
Subword tokenization (BPE, WordPiece, SentencePiece): the modern standard, handles any word, used in all major LLMs
N-grams: sequences of tokens that capture short-range context, still used in classical NLP

Each approach has its place. For production NLP systems today, subword tokenization is almost always the right choice. For understanding older systems or building lightweight classifiers, word tokens and n-grams still have their role.

What Comes Next

Tokenization turns raw text into something a model can process. But once you have tokens, the next question is: what role does each one play in the sentence?

That’s what Part of Speech Tagging covers. It’s the process of labelling each token with its grammatical role: noun, verb, adjective, and so on. It sounds simple, but it’s one of the foundational steps that allows NLP systems to understand the structure of language, not just the words themselves.

In the next post in this series, we’ll break down how POS tagging works, why it matters, and where it shows up in real NLP pipelines.

Learn This at Fondra Labs

This post is part of our NLP Foundations series, where we cover everything from the basics of text processing to the architectures powering modern AI systems.

At Fondra Labs, we teach AI from production reality, not hype. Every concept in this series is chosen because it matters in practice, not just in theory. Tokenization, embeddings, retrieval, evaluation: these are the building blocks that engineers actually use when they build real systems.

If you found this useful, explore the rest of the blog. We cover machine learning, deep learning, NLP, and the practical skills that bridge the gap between understanding AI and building with it.

Originally published at https://fondralabs.com.

NLP Meaning: What is Natural Language Processing & How Does It Work?

Javier Aguirre — Wed, 02 Sep 2026 06:07:02 +0000

NLP Meaning: What is Natural Language Processing & How Does It Work?

You’ve used it today. Probably without noticing.

Every time your phone autocompletes a sentence, every time Gmail moves a suspicious email to spam, every time you ask Google a question and it actually understands what you meant: that’s natural language processing at work.

And yet most people couldn’t explain what it actually is.

This post fixes that. By the end, you’ll have a clear, honest understanding of what NLP is, how it works, where it shows up in the real world, and how to get started with it. No jargon. No unnecessary complexity. Just the real thing, explained so it actually makes sense.

What Is NLP?

Natural language processing (NLP) is the field of AI that teaches computers to understand human language.

That’s it at its core. But let’s unpack why that’s harder than it sounds.

Computers are designed for precision. They’re comfortable with numbers, rules, and structured data. Tell a computer that 2 + 2 = 4 and it will never be confused. But human language is the opposite of precise. It’s full of ambiguity, context, and meaning that shifts depending on who’s speaking, how they’re saying it, and what came before.

Take this sentence: “I saw the man with the telescope.”

Did you use a telescope to see the man? Or did you see a man who was holding a telescope? Both are valid readings of the exact same sentence. A human picks the right one instantly using context. A computer has to be taught how to do that.

Or take the word “sick.” Depending on who says it and where, it can mean ill, disgusting, or (if you’re under 30) genuinely impressive.

NLP is the field dedicated to teaching computers to navigate all of that.

Why Does This Matter?

Think about how much of the world runs on language.

Emails. Contracts. Medical records. Customer reviews. News articles. Social media. Search queries. Support tickets. Code documentation.

Most of the information that matters to businesses, governments, hospitals, and individuals exists as unstructured text. Nobody sat down and put it into tidy rows and columns. It’s just words.

For decades, this information was largely unprocessable at scale. You couldn’t automatically read a million customer reviews to understand what people loved or hated about your product. You couldn’t scan thousands of legal contracts to flag the risky ones. You couldn’t build a system that actually understood what a user was asking for.

NLP changed that. It turned language into something computers can work with.

How Does NLP Work?

Here’s the honest answer: NLP works by converting text into numbers, finding patterns in those numbers, and using those patterns to make predictions or generate new text.

Let’s break that down into something concrete.

Step 1: Breaking Text Apart

Before a computer can understand a sentence, it needs to split it into manageable pieces. This is called tokenization.

The sentence “NLP is fascinating” becomes something like: [“NLP”, “is”, “fascinating”]. These pieces (tokens) are the units the model actually works with.

Simple enough. But tokenization gets interesting fast. Modern AI models don’t always split on words. They often split on parts of words. The word “fascinating” might become [“fascin”, “ating”]. This is called subword tokenization, and it’s one of the reasons modern language models can handle words they’ve never seen before.

Step 2: Turning Words into Numbers

Computers can’t process words directly. Everything needs to become a number.

The clever solution NLP researchers found is called word embeddings. Each word gets converted into a list of numbers (a vector) that captures its meaning. Words with similar meanings end up with similar vectors. “Dog” and “puppy” are close together. “Dog” and “aeroplane” are far apart.

This is how meaning gets encoded into something a computer can actually compute with.

Step 3: Finding Patterns

Once text is in numerical form, machine learning models can find patterns in it.

Think of it like this: if you show a model millions of sentences and tell it which ones are positive and which are negative, it learns to recognize the patterns that signal each. It starts noticing that words like “love,” “amazing,” and “perfect” tend to appear in positive sentences. Words like “terrible,” “waste,” and “broken” tend to appear in negative ones.

That’s sentiment analysis, one of the most widely used NLP tasks, and it works through the same basic loop: show the model many examples, let it find the patterns, use those patterns to make predictions on new text.

The Transformer: Why Modern NLP Is So Powerful

For a long time, NLP models had a fundamental limitation: they processed language one word at a time, left to right. They’d often “forget” what was said at the start of a long sentence by the time they reached the end.

In 2017, a team at Google introduced a new architecture called the Transformer. The key idea was simple but powerful: instead of reading words one by one, process all of them at once, and let every word pay attention to every other word simultaneously.

This meant the model could understand that the word “it” at the end of a paragraph referred to something mentioned at the beginning. It could track relationships across long stretches of text without losing track.

“Attention is all you need.”

- Vaswani et al., Google Brain, 2017

Every major AI language system you’ve heard of (ChatGPT, Claude, Gemini, BERT) is built on the Transformer. It was the breakthrough that made modern NLP possible.

NLP in Artificial Intelligence: Where It Fits

If you’re trying to understand how NLP relates to AI more broadly, here’s a simple way to think about it.

Artificial intelligence is the big umbrella. It’s the general goal of making machines that can do things that normally require human intelligence.

Under that umbrella, there are different areas depending on what type of intelligence you’re trying to replicate:

Computer Vision: understanding images and video (what’s in this photo?)
Speech Recognition: understanding spoken audio (what did that person just say?)
Natural Language Processing: understanding and generating text (what does this sentence mean?)

NLP is the branch of AI focused specifically on language. And increasingly, these areas are merging. Modern AI systems handle text, images, and audio together. But NLP remains the backbone. Even in multimodal systems, language is how instructions are given and how outputs are delivered.

NLP and Machine Learning: What’s the Relationship?

This trips a lot of beginners up, so let’s clear it up once and for all.

Think of it as nested circles:

Artificial Intelligence is the outermost circle: the broad goal of intelligent machines
Machine Learning sits inside AI: it’s the approach of teaching machines by showing them data rather than writing explicit rules
Deep Learning sits inside ML: it’s machine learning using large neural networks
NLP is an application: it’s ML and deep learning applied to the specific problem of language

So NLP isn’t a competitor to machine learning or deep learning. It uses them. It’s what you get when you take those tools and point them at language.

NLP Examples: Where You’ve Already Seen It

The best way to make NLP concrete is to look at where you already use it every day.

Spam filters read your incoming email and decide whether it’s legitimate or junk. They learned to do this from millions of examples of spam and non-spam emails. Now they make that judgment in milliseconds, invisibly, on every email you receive.

Search engines don’t just match keywords anymore. When you type “best way to cook chicken without drying it out,” Google doesn’t look for pages containing those exact words. It understands the intent (you want a moist chicken) and surfaces pages that address that, even if they use completely different words.

Virtual assistants (Siri, Alexa, Google Assistant) listen to your voice, convert it to text, figure out what you’re asking for, retrieve the answer, and speak it back to you. NLP is running at every single step of that process.

Machine translation tools like Google Translate take text in one language and produce text in another, handling grammar, idioms, and context that would have seemed like science fiction twenty years ago.

Autocomplete on your phone has learned from how you type. It predicts what you’re about to say based on patterns it has observed across millions of conversations.

ChatGPT, Claude, Gemini (the AI assistants you may already use) are the most advanced NLP systems in existence. They can write, explain, summarise, translate, code, and reason in natural language.

NLP Applications Across Industries

NLP isn’t just for consumer apps. It’s quietly transforming how whole industries operate.

Healthcare: Doctors write clinical notes in natural language, unstructured text that’s hard to search or analyse. NLP systems can read these notes and extract structured information: what diagnosis was made, what medication was prescribed, what symptoms were mentioned. This makes patient records usable in ways they previously weren’t.

Finance: Analysts use NLP to monitor news and earnings call transcripts for signals. When a CEO says “we’re cautiously optimistic” versus “we’re very confident,” the difference matters. NLP systems can track these signals across thousands of documents simultaneously.

Legal: Lawyers spend enormous amounts of time reading contracts. NLP tools can scan a contract, flag unusual clauses, compare it against standard templates, and summarise the key terms. Tasks that previously took hours now take minutes.

Customer service: When you submit a support ticket or chat with a bot, NLP is classifying your request, routing it to the right team, and often generating the first response. The better the NLP, the less you notice you’re not talking to a human.

The common thread: wherever there’s unstructured text that matters, NLP creates value.

Getting Started with NLP in Python

If you want to go beyond understanding NLP and actually start building with it, Python is where you start. It’s the language the entire NLP community uses, and the tools are excellent.

Three libraries worth knowing about:

NLTK (Natural Language Toolkit) is the classic beginner library. It’s great for learning the basics: tokenization, stemming, simple text analysis. Not what you’d use in production, but a solid starting point for understanding concepts.

spaCy is the professional-grade library. Fast, reliable, and built for real-world use. If you’re building something that needs to run in production, spaCy is usually where you land.

Hugging Face Transformers is where modern NLP lives. It gives you access to thousands of pretrained models (including the same architectures behind ChatGPT and BERT) with just a few lines of code.

Here’s what that looks like in practice:

from transformers 
import pipeline 
classifier = pipeline("sentiment-analysis") 
result = classifier("I absolutely loved this product.") 
print(result) # [{'label': 'POSITIVE', 'score': 0.9998}]

Four lines of code. State-of-the-art sentiment analysis. That’s the power of the current ecosystem.

A sensible learning path:

Start with NLTK to get comfortable with the basics
Move to spaCy when you want to build something real
Learn the concepts (tokens, embeddings, attention) before jumping to Hugging Face
Use Hugging Face to fine-tune pretrained models on your own data

The concepts are more important than the tools. Tools change. The ideas behind them don’t.

What Comes Next

You now understand what NLP is, why it matters, how it works at a high level, and where it shows up in the world.

This is the foundation. The rest of this series builds on it, one concept at a time:

Tokenization: how text gets split into the pieces models actually process, including the subword methods that power every modern LLM
Word Embeddings: how words become numbers that carry meaning, from Word2Vec to BERT
Named Entity Recognition: teaching models to find people, places, and organisations in text
Sentiment Analysis: detecting opinion and emotion at scale
Semantic Search & RAG: how NLP powers modern search and AI assistants

Each post goes deeper on one piece of the picture. If you follow the series in order, you’ll build a solid, practical understanding of NLP from the ground up, the kind that actually helps when you sit down to build something.

Originally published at https://fondralabs.com.

Deep Learning for Beginners: A Complete Guide

Javier Aguirre — Fri, 29 May 2026 03:31:41 +0000

You've heard the words. Neural networks. Deep learning AI. Transformers. Maybe you've even heard that deep learning is what powers ChatGPT, image generators, voice assistants, and self-driving cars. And now you're wondering: what actually is it?

This guide is the honest, comprehensive answer. Not watered down. Not padded with fluff. Just the core ideas, the key architectures, and the intuition you need to actually understand what's going on under the hood, explained clearly enough that a beginner can follow it, and thoroughly enough that it stays useful as you go deeper.

If you haven't read our post on machine learning basics yet, I'd recommend starting there. Deep learning builds directly on top of machine learning, and a few concepts from that post will make this one click much faster. That said, let's get into it.

What Is Deep Learning?

Deep learning is a branch of machine learning that uses artificial neural networks (systems loosely inspired by the structure of the human brain) to learn patterns from data.

The word "deep" refers to depth: the number of layers stacked inside the network. A shallow network might have one or two layers. A deep network might have dozens, hundreds, or in the case of modern large language models, thousands. Each layer transforms the data slightly, learning increasingly abstract representations as you move through the stack.

Here's the key insight that separates deep learning from classical machine learning: traditional ML algorithms need humans to engineer features manually. You decide what inputs matter. In deep learning, the network learns its own features directly from raw data, pixels, waveforms, text characters, without being told what to look for.

That's what makes it so powerful. And that's what makes it so data-hungry.

"Deep learning is not magic. It's a very large function with millions of parameters, trained on enormous amounts of data, that has learned to approximate patterns no human could write by hand."

What Is Deep Learning vs Machine Learning?

It's a nested relationship, not a competition. All deep learning is machine learning, but not all machine learning is deep learning.

Classical machine learning (decision trees, random forests, gradient boosting) works well on structured, tabular data. It's interpretable, efficient, and still dominant across most real-world business applications.

Deep learning takes over when the data is unstructured: images, audio, video, raw text. These formats have too many dimensions and too much complexity for traditional algorithms to handle well. Neural networks, with their layered feature learning, are built exactly for this.

Use classical ML for credit scoring, demand forecasting, fraud detection, churn prediction: structured tables, interpretability required
Use deep learning for image recognition, speech, language understanding, video: unstructured data, pattern complexity at scale

The honest take: deep learning is not always better. It requires significantly more data and compute. When you have a clean tabular dataset and a clear prediction task, XGBoost will often beat a neural network and train in seconds rather than hours.

Neural Network Basics

Before we get into specific architectures, you need to understand the building blocks. Every deep learning model, regardless of how complex, is built from the same core components.

Neurons, Layers, and Activation Functions

A neuron is the basic unit. It takes a set of inputs, multiplies each by a learned weight, sums everything up, and passes the result through an activation function. That output feeds into the next layer.

Here I attach an image of the comparison of a real neural network and an artificial one (on the right). You do not need to understand the formula, but just understand the following: 3 weights are coming to the neuron (these are the values of the previous neurons of a neural network). Inside there is a scary formula. However, trust me, is not that scary, it is basically is a fancy way of saying... Let's sum them all up! and just a little thing more I will describe soon, which is, we add an activation function to it.

That's all!

So... what is an activation function? the activation function is what gives neural networks their power. Without it, stacking layers would be mathematically equivalent to having just one layer: you'd just be doing linear transformations. Activation functions introduce non-linearity, which is what allows the network to learn complex patterns. If you did not completely get it, it is okay, this is the biggest mathematical part of the explanation!

The three you'll see everywhere:

ReLU (Rectified Linear Unit): outputs zero for negative inputs, passes positive inputs through unchanged. Simple, fast, and the default choice for hidden layers in most networks. The fact that something this simple works so well is one of the quiet surprises of deep learning.
Sigmoid: squashes output to a value between 0 and 1. Used in binary classification output layers where you want a probability.
Softmax: extends sigmoid to multiple classes. Takes a vector of raw scores and converts them into probabilities that sum to 1. Used in the final layer of any multi-class classifier.

So basically, you can see the activation functions as a filter to decide what value will the neuron have after we summed up previous neurons values. And we can do it in different ways. If you want to understand it a bit deeper here is a deconstruction on how it exactly works:

Feedforward Networks: How Information Flows

Now you know how an artificial neural netwok works, great! The next step is stacking them up and we would get a neural network. Feedforward neural networks are the simplest neural network, information travels in one direction only: input → hidden layers → output. No loops, no memory, no feedback. Each layer is fully connected to the next, every neuron in layer N connects to every neuron in layer N+1.

This is called a feedforward network, and it's the foundation that every other architecture builds on top of or departs from. (yes, including chatGPT, Claude and other transformer based models, here is where it all starts.)

Backpropagation: How the Network Actually Learns

Training a neural network means finding the right weights. You do this by:

Making a prediction with the current weights
Measuring how wrong it was (the loss, remember this term)
Computing how much each weight contributed to that error
Nudging every weight slightly in the direction that reduces the loss

Step 3 is backpropagation, the algorithm that efficiently computes the gradient of the loss with respect to every weight in the network, propagating the error signal backward from the output layer to the input. Step 4 is gradient descent, the optimiser that uses those gradients to update the weights.

This loop (forward pass, compute loss, backward pass, update weights) repeats for millions or billions of iterations during training. That's how a network goes from random noise to something that can recognise faces, translate languages, or generate code.

Important note: How the neuron, backpropagation, gradient descent, losses work is important to understand. You will not need to perform any math in practice but this theory can help you grasp what you are doing better. Do not get stuck on it, but if you can learn it, I highly recommend it.

I decided to write it in this blog so you know it exists but this part and the next two (weight initialisation and batch normalization) could be skipped since they are not that much oriented towards the practice but more towards foundational knowledge.

Give it a quick read, if you do not fully get it, keep on going and dont worry!

Weight Initialisation

How you set the initial weights before training matters more than most tutorials admit. Start them all at zero and the network won't learn: every neuron computes the same thing and the gradients are identical. Start them too large and training becomes unstable. Smart initialisation schemes (Xavier, He initialisation) are designed to keep signal flowing cleanly through the network from the start.

Batch Normalisation

As networks get deeper, a problem emerges: the distribution of activations shifts during training, making learning unstable and slow. Batch normalisation addresses this by normalising the inputs to each layer across a mini-batch, keeping activations in a stable range. It's one of those techniques that felt like a trick when it was introduced and turned out to be foundational, it's now standard in almost every deep architecture.

Key Architectures

Now for the interesting part. If you made it till here, congratulations! now its the fun part, so, keep on reading!

Deep learning is not one thing, it's a family of architectures, each designed for a different kind of data and a different kind of problem. Here are the ones worth knowing.

Feedforward Neural Networks (FFNN)

The simplest deep network. Fully connected layers, information flows in one direction, no special structure. This is the architecture that introduces every concept (neurons, activations, backpropagation) in its clearest form.

In practice, pure FFNNs are rarely used for complex tasks. Images have spatial structure that FFNNs ignore. Sequences have temporal dependencies that FFNNs can't capture. But understanding the FFNN deeply is non-negotiable before moving to anything else.

When you'd use it: tabular data, simple classification and regression tasks, as a component inside larger architectures.

Convolutional Neural Networks (CNNs)

CNNs are the architecture that put deep learning on the map. In 2012, a CNN called AlexNet won the ImageNet competition by a margin so large it ended the debate about whether deep learning worked. It did.

The key idea: instead of connecting every neuron to every pixel (computationally insane for large images), CNNs apply small filters that slide across the input, detecting local patterns. Early layers learn to detect edges and textures. Later layers combine those into shapes, objects, faces.

This design is efficient, spatially aware, and extraordinarily effective on anything that has grid-like structure: images, video frames, certain kinds of audio.

When you'd use it: image classification, object detection, medical imaging, video analysis, any problem where spatial patterns matter.

Recurrent Neural Networks (RNNs) and LSTMs

What if your data is a sequence (a sentence, a time series, an audio clip) where the order of elements matters?

FFNNs and CNNs don't have memory. They process each input independently. RNNs fix this by feeding the hidden state from the previous step into the current step, giving the network a form of short-term memory.

In theory, this lets RNNs capture long-range dependencies. In practice, they struggle: gradients either explode or vanish as they travel through many time steps, making it hard to learn patterns that span long sequences.

LSTMs (Long Short-Term Memory networks) solve this with a more sophisticated memory mechanism, gates that control what information to keep, what to forget, and what to output. LSTMs were the state of the art for language and sequence tasks for years before Transformers arrived.

The following is a illustration of its internal construction, this is how one LSTM cell looks like:

Do not worry as you will never have implement this from scratch :)

Again, is just good to know this exists for the future usage.

They're not obsolete, they're still used in production systems where efficiency matters and sequences are modest in length. But for most language tasks, Transformers have superseded them.

The following is an illustration of how an entire LSTM looks like:

When you'd use them: time series forecasting, speech recognition (in resource-constrained settings), sensor data, any ordered sequence where Transformers would be overkill.

Autoencoders and VAEs

An autoencoder is trained to compress an input into a smaller representation (the latent space) and then reconstruct it back to the original. The bottleneck forces the network to learn the most essential features of the data.

Variational Autoencoders (VAEs) extend this by learning a probability distribution over the latent space rather than a fixed point. This makes the latent space continuous and structured, which means you can sample from it to generate new data, not just reconstruct existing inputs.

VAEs were an early serious approach to generative modelling and introduced ideas (latent space, encoder-decoder structure) that appear throughout modern AI.

When you'd use them: anomaly detection, data compression, generative modelling, representation learning, synthetic data generation.

GANs (Generative Adversarial Networks)

GANs are one of the most creative ideas in all of deep learning. The setup: two networks trained in opposition.

The generator produces fake data (images, audio, whatever the domain). The discriminator tries to tell real from fake. As training progresses, the generator gets better at fooling the discriminator, and the discriminator gets better at detecting fakes. Each improves in response to the other.

When it works, GANs produce extraordinarily realistic outputs. They dominated image synthesis for years and the photorealistic faces you may have seen on sites like "This Person Does Not Exist" are GAN-generated.

They're notoriously difficult to train, mode collapse, training instability, and sensitivity to hyperparameters make them frustrating in practice. Diffusion models have largely superseded them for image generation, but the adversarial training concept remains influential.

When you'd use them: image synthesis, data augmentation, style transfer, domain adaptation.

Diffusion Models

Diffusion models are the architecture behind Stable Diffusion, DALL-E, and most of the state-of-the-art image generators you've seen. The idea is elegant and counterintuitive.

Training: take real images and gradually add Gaussian noise until they're pure static. Teach the network to reverse this process, to predict and remove the noise at each step.

Generation: start with pure random noise and run the learned denoising process repeatedly until a coherent image emerges.

Diffusion models produce higher quality, more diverse outputs than GANs and train more stably. They're computationally heavier at inference time (many denoising steps required), but the quality improvement has made the trade-off worth it for most applications.

When you'd use them: image generation, video generation, audio synthesis, any high-quality generative task.

Personal Note

The latest three (VAEs, GANs and Difussion Models) are great for generative related tasks including synthetic data generation. Currently diffusion models have taken the space due to their high accuracy, efficiency and deployability.

We published in 2023 a research between Samsung Advanced Institute of Health Science and Technology (SAIHST), Samsung Medical Center (SMC), Yonsei Severance Hospital and Google Cloud (USA) comparing the use of the three of them for synthetic data generation on healthcare settings. If you are interested in the topic click here.

The Transformer

Everything changed in 2017 when a Google paper titled "Attention Is All You Need" introduced the Transformer architecture. GPT, BERT, DALL-E, Whisper, Stable Diffusion, every major model of the last several years is built on top of it or derives from it directly.

The core innovation: self-attention. Instead of processing a sequence step by step (like an RNN), the Transformer processes all positions simultaneously and lets each position directly attend to every other position. This solves the long-range dependency problem completely, and critically, allows full parallelisation during training.

Self-Attention and Multi-Head Attention

Self-attention allows the model to weigh how relevant each word (or token) is to every other word when building a representation. In the sentence "The bank by the river was steep," the word "bank" needs to attend strongly to "river" to resolve its meaning correctly. Self-attention learns to do this.

Multi-head attention runs several self-attention operations in parallel, each learning to attend to different kinds of relationships simultaneously. One head might track syntactic structure; another might track semantic similarity. The outputs are combined and projected forward.

Positional Encoding

Transformers have no built-in sense of order, self-attention is permutation-invariant. Positional encoding fixes this by adding information about each token's position in the sequence before it enters the network. The model learns to use this position signal to understand order, proximity, and structure.

Encoder vs. Decoder vs. Encoder-Decoder

Not all Transformers are the same. There are three architectural variants:

Encoder-only (e.g. BERT): reads the full sequence bidirectionally, building rich contextual representations. Best for tasks that require understanding: classification, named entity recognition, semantic search.
Decoder-only (e.g. GPT): generates tokens one at a time, each attending only to previous tokens. Best for generation: writing, code, conversation.
Encoder-decoder (e.g. T5, original Transformer): encodes an input sequence, then decodes an output sequence. Best for transformation tasks: translation, summarisation, question answering.

Understanding which variant you're working with (and why it was chosen) is one of the most practically useful things you can know when working with modern AI.

What Comes Next

You now have a map of the deep learning landscape: the building blocks, the key architectures, when to use each, and why they exist. That's the conceptual foundation.

The practical path from here:

Get hands-on with PyTorch or TensorFlow: implement a simple FFNN, then a CNN on image data. Seeing the training loop in code cements everything.
Work through a sequence task: build or use an LSTM on a real time series dataset.
Study the Transformer in depth: read "Attention Is All You Need" after you've built intuition. It will make sense now in a way it wouldn't have before.
Explore modern applications: fine-tune a pretrained model, experiment with diffusion pipelines, build something that uses what you've learned.

If you're wondering where deep learning fits in the bigger picture (how it relates to machine learning and where Generative AI comes in) check out our AI learning roadmap for the full view.

The architecture names will start feeling familiar quickly. Build things. Break them. Figure out why. That's the actual learning.

If you want to learn more, we have more content in our blog here!

Machine Learning Basics: Core Concepts Explained Simply

Javier Aguirre — Fri, 29 May 2026 03:22:57 +0000

If you've heard the term "machine learning" thrown around but still aren't sure what it actually means, you're in the right place. This isn't a roadmap for learning machine learning (we covered that here). This is the conceptual foundation: the ideas, the vocabulary, and the mental models you need so that everything else clicks.

Think of it as the "what" before the "how."

Let's get into it!

What Is Machine Learning?

Machine learning is a way of building systems that learns from data rather than following hand-written rules.

In traditional programming, a developer writes explicit instructions: if X, do Y. Every scenario must be anticipated and coded manually. Machine learning flips this model entirely. Instead of writing the rules yourself, you feed the system a large collection of examples (data where you already know the outcome) and the algorithm figures out the rules on its own.

A concrete way to think about it: imagine you want a computer to recognise photos of cats. You could try to write rules: "look for pointy ears, whiskers, fur." But edge cases multiply fast. What about a cartoon cat? A sleeping cat? A hairless breed?

Machine learning sidesteps the rule-writing problem entirely. You show the model thousands of labelled photos ("cat" / "not a cat"), it extracts the underlying patterns, and it generalises that knowledge to photos it's never seen before.

That pattern-recognition loop (examples in, predictions out) is what machine learning is, at its core.

Machine learning, in simple words:
"a system that gets smarter the more data it sees, rather than following a fixed set of rules."

What Is Machine Learning Used For?

Machine learning is already embedded in most of the software you use daily, though the reality is more layered than the usual list of examples suggests.

Credit risk and underwriting: banks use gradient boosted trees and logistic regression to assess lending decisions, because income, debt, history, and geography interact in ways too complex for manual rules.

Fraud detection: modern fraud systems combine anomaly detection, graph ML (to surface fraud rings across networks), and rule-based filters working in tandem. Patterns are adversarial and constantly shifting, which is exactly where ML earns its place.

Search ranking: retrieval uses indexing and heuristics, but ranking is heavily learned. Models predict which result a specific user is most likely to find useful based on signals from billions of past interactions.

Advertising and recommendations: arguably the largest economic application of ML on earth. Predicting click-through rate, conversion probability, and long-term user value drives enormous commercial value across every major platform.

Demand forecasting: retailers, energy grids, and supply chains use ML to predict inventory needs, consumption patterns, and logistics requirements. Gradient boosted trees and hybrid statistical models dominate here.

Anomaly detection: server monitoring, cybersecurity logs, and industrial sensors all use ML to flag behaviour that deviates from learned baselines. Isolation forests and autoencoders are workhorses in this space.

Marketplace matching: job platforms, dating platforms, ride-sharing, and marketplaces use ML to predict compatibility between two entities: driver and rider, candidate and role, buyer and listing.

The common thread: ML works best when the rules are too complex to write by hand, the environment shifts over time, and there's feedback data at scale to learn from.

The Three Types of Machine Learning

Not all machine learning works the same way. Understanding the three core categories is fundamental to understanding machine learning properly.

Supervised Learning

Supervised learning is the most common type and the best starting point for beginners.

The model trains on labelled data, every example in the training set comes with the correct answer attached. A spam filter trains on emails labelled "spam" or "not spam." A house price model trains on historical sales records where the price is already known.

What is supervised learning in practice?

The model makes predictions, compares them to the correct labels, measures the error, and adjusts. Repeat this millions of times across thousands of examples and the model gradually gets accurate. At inference time (when it sees new, unlabelled data) it applies everything it learned.

Supervised learning covers two main tasks:

Classification: predicting a category (spam/not spam, disease/no disease, churn/retain)
Regression: predicting a number (house price, temperature, revenue)

Unsupervised Learning

With unsupervised learning, there are no labels. The model receives raw data and must discover structure on its own.

The most common application is clustering. Grouping data points that are similar to each other. Customer segmentation works this way: you feed the model purchase history, browsing behaviour, and demographics, and it discovers natural groupings without anyone telling it what those groups should be.

Other unsupervised applications include anomaly detection (spotting data points that don't fit the pattern) and dimensionality reduction (compressing complex data into simpler representations without losing key information).

Reinforcement Learning

Reinforcement learning is the odd one out. It doesn't learn from a fixed dataset at all.

Instead, an agent takes actions in an environment and receives feedback: rewards for good outcomes, penalties for bad ones. Over time, through trial and error, it learns the strategy that maximises reward.

This is how DeepMind's AlphaGo mastered the game of Go, how robotics systems learn to walk, and how recommendation engines learn to keep users engaged. It's one of the most exciting areas in machine learning today and also the one of the most complex.

What Is the Difference Between Machine Learning and Deep Learning?

This trips up a lot of people new to the field. The short answer: deep learning is a subset of machine learning. Some of my AI engineering collegues would disagree, saying it is another field on its own that comes after machine learning. However, the exact relationship doesnt matter, either is the parent or sibling of machine learning it definitely is a close relative.

Then... what is that relationship? Well, classic machine learning algorithms (linear regression, decision trees, random forests...) work by finding mathematical relationships in structured data. They're transparent, fast, and still dominant in most real-world business applications.

Deep learning uses artificial neural networks with many layers (hence "deep"). Each layer learns increasingly abstract representations: an early layer of an image model might learn to detect edges; a later layer might learn to detect faces. This layered abstraction is what gives deep learning its power on complex, unstructured data.

The bottom line: when someone says "we use machine learning," they may or may not mean deep learning. When someone says "we use deep learning," that's always a subset of machine learning.

So, what you should remember is that both have the same purpose: a system that gets smarter the more data it sees rather than following a fixed set of rules. However, in the case of deep learning, it tends to require more compute, less efficient but can be smarter for harder cases.

Key Terms You'll Keep Seeing

Understanding machine learning means getting comfortable with a core vocabulary. Here are the terms that come up constantly:

Training data: the dataset the model learns from. The quality and size of this data is the single biggest factor in model performance.

Model: the mathematical function that maps inputs to outputs after training. When people say "we trained a model," this is what they mean.

Features: the input variables the model uses to make predictions. In a house price model, features might include square footage, number of bedrooms, and postcode.

Labels: the correct output values in supervised learning. The "answers" in the training data.

Training: the process of exposing a model to data and letting it adjust its internal parameters to minimise error.

Overfitting: when a model learns the training data too well, including its noise and quirks, and fails to generalise to new data. A model that scores 99% on training data and 60% on real data has overfit.

Underfitting: the opposite problem. The model is too simple to capture the real patterns in the data.

Hyperparameters: settings you choose before training begins (number of trees in a forest, learning rate, number of layers). Distinct from parameters, which the model learns during training.

Why Machine Learning Works (The Intuition)

At the heart of basic machine learning is a deceptively simple idea: generalisation.

A model that just memorised its training data would be useless, you'd already have that data. What you want is a model that has learned something general enough to make accurate predictions on data it has never encountered.

The way models achieve this is by minimising a loss function, a mathematical measure of how wrong their predictions are. During training, the algorithm repeatedly adjusts the model's internal parameters in the direction that reduces loss. After enough iterations across enough data, the model has found a set of parameters that capture the underlying structure of the problem.

This is why data quality matters so much. Garbage in, garbage out. if the training data is biased, incomplete, or mislabelled, the patterns the model learns will reflect those flaws, no matter how sophisticated the algorithm.

A Note on Maths

I personally like to bring this one frequently. As we covered in the previous blog, one of the most common questions when people start understanding machine learning: do I need to be good at maths?

The honest answer is: not to start, and not as much as you'd think to go deep.

The mathematical foundations are there (linear algebra, probability, calculus) but they describe what's happening inside the algorithms, not how to use them. Most engineers use libraries like Scikit-learn that handle the implementation entirely. The maths becomes valuable when you want to understand why a model behaves a certain way, not to run it.

Start with intuition. Pick up the maths when a specific question pulls you toward it. That order works far better than studying maths in a vacuum before you've built anything.

What Comes Next

Now that you have the conceptual foundation, the natural next step is getting hands-on. The core skills to tackle in order:

Exploratory Data Analysis (EDA): understand your data before you model it
Data preparation: clean, transform, and structure data for training
Model training: apply the right algorithm for the task
Model evaluation: measure performance properly (accuracy alone isn't enough)
Iteration: improve, tune, and deploy

If you want the full practical path (tools, libraries, timeline, and projects) check out our guide on how to learn machine learning from scratch.

If you want to read our full blog, visit our blog here!

How to Learn Machine Learning from Scratch

Javier Aguirre — Fri, 29 May 2026 03:18:26 +0000

You want to learn machine learning. Great! Now you are staring at a screen full of courses, YouTube videos, Reddit threads, and bootcamp ads, and you have absolutely no idea where to begin.

After being over 10 years in the AI field, I have seen brilliant people give up on machine learning not because it was too hard, but because they started in the wrong place, hit a wall they did not expect, and concluded the whole thing was not for them.

This post is the guide I wish someone had handed me at the beginning. The honest, practical path to go from complete beginner to someone who can actually build and deploy machine learning models. And let me tell you one last thing, it is definitely not as hard as it seems. Trust me :)

Let's get into it.

First, Let's Clear Something Up

When people ask "how do I start machine learning?", they usually follow it up with something like: "do I need a PhD? Do I need to be amazing at math? Do I need to already know how to code?"

The answer to all three is clearly no.

Machine learning for beginners has never been more accessible. The tools are better, the libraries do most of the heavy lifting, and the community has produced genuinely good learning resources. What you need is not genius but a clear path and the willingness to follow it.

Here is that path.

Step 1: Python First, But Not Too Much Python!

Before you touch a single machine learning concept, you need to be able to write basic Python. Not software-engineer-level Python. Just enough to load data, write a function, and run a script.

What you actually need:

Variables and data types
Loops and conditionals
Functions
Lists and dictionaries
Importing libraries

That is it. A few weeks of consistent practice will get you there. Do not disappear into a six-month Python deep dive, that is procrastination dressed up as preparation. Ask your best friends (I mean Claude, Gemini, ChatGPT... They are amazing) for help, they really know how to code and teach coding.

Once you can write simple scripts without completely panicking, you are ready.

Step 2: What Is Machine Learning, Actually?

Here is the simplest way to think about it: normally, when you write a code, you write the rules. You tell the computer exactly what to do in every situation. Machine learning flips that around. Instead of writing the rules, you give the algorithm a pile of examples (historical data where you already know the outcome) and it figures out the rules itself. Sounds pretty cool, doesn't it?

That is genuinely it. A machine learning model is just a pattern-finding machine. You feed it enough examples, it learns what those examples have in common, and then it uses that knowledge to make predictions on data it has never seen before. The more examples, the better the patterns. The better the patterns, the more accurate the predictions.

Step 3: The Core Skills of Machine Learning

Pay attention!! This is where most people get it wrong and go lost. However, it is not hard if you know the path. People think machine learning is about knowing a long list of algorithms. It is not. Others, that they need a math background for it. That is simply not true. It is about mastering a set of core skills that every project requires, in roughly the same order, every single time.

Here is what that actually looks like.

Exploratory Data Analysis (EDA)

Before you build anything you need to understand what you are working with. As an example, you may want to predict who will pay future mortgages based on past data. At that point you should ask yourself: Where does the data come from? What do the columns actually mean? What is missing? What looks suspicious? EDA is the skill that separates people who build models that work from people who build models that silently fail. It is also the step that is least taught and most skipped. Same as with python, do not spend 6 months on learning EDA, spend a few weeks and move next.

Data Preparation

Real-world data is a mess. Missing values, inconsistent formats, outliers that make no sense, categorical variables that need to be converted into numbers. The prior step prepares you to understand all of it, on this step you will focus on how to prepare it for training. Data preparation is where you spend most of your time on any real project. Learn to clean data well, and everything downstream becomes easier.

Model Training

This is the step everyone takes it wrong. People think is the big one. It is not. In practice, once your data is clean, training often takes a few lines of code and you are ready to go. The two previous steps are where the magic of machine learning occurs, on having and preparing good data. However, I have to make a disclaimer, I recommend understanding about the different models so that you know when to use one or another. Hearing a short class on how they work will highly benefit you.

Model Evaluation

This is an important one, it is not a hard one, but can be slightly confusing. Many beginners make mistakes without realizing. Accuracy alone is a terrible metric for most projects. Learn precision, recall, F1-score, ROC-AUC. Understand the difference between overfitting (your model memorised the training data and fails on anything new) and underfitting (your model is too simple to capture the real patterns). Know the difference between your training set, validation set, and test set, and never, ever mix them up.

Model Improvement (optional)

Once you have a baseline model, you can make it better. This means tuning hyperparameters, trying different algorithms, engineering better features from your raw data. This is where craft comes in and where the interesting problem-solving happens.

Deployment (optional)

A model sitting on your laptop is not useful. Learn to put it somewhere that actually does something, an API, a simple web app, a scheduled job. You do not need to become a software engineer to do this, but you need to know the basics.

Step 4: The Algorithms Worth Knowing

Once you understand the core skills, algorithms start to make sense. Now you know what problem they are solving. You do not need to memorise fifty of them. You need to understand a core set deeply.

Linear Regression

Predicting a continuous number (price, temperature, revenue). The simplest model you will build and, by far, the most educational. Understand this one fully before you move on.

Logistic Regression

Despite the name, this is a classification algorithm. Will this customer churn? Is this email spam? Binary decisions. This is your first taste of classification.

K-Nearest Neighbours (KNN)

The most intuitive classifier in ML. To predict something, it looks at the K closest examples in your training data and goes with the majority. No real "training" happens. It just memorises the data and reasons from it at prediction time. A great first algorithm to understand because the logic is completely transparent.

K-Means Clustering

Your entry point into a different kind of ML: unsupervised learning, where you have no labels and let the algorithm find structure on its own. K-Means groups your data points into K clusters based on similarity. Used everywhere from customer segmentation to anomaly detection.

Decision Trees

One of the most intuitive models in all of ML. You can actually visualise how a tree makes decisions, which is invaluable for building intuition.

Random Forests

A collection of decision trees working together. One of the most reliable, robust algorithms in practice. If you are ever in doubt about which model to try first, try Random Forest.

Support Vector Machines (SVM)

Finds the boundary that best separates two classes, with as much margin between them as possible. Works particularly well with smaller datasets and high-dimensional data like text. The intuition behind "maximum margin separation" is one of the most elegant ideas in all of ML.

Gradient Boosting (XGBoost, LightGBM)

The go-to for structured/tabular data in production. These models win Kaggle competitions constantly. Learn them and you will be dangerous on real business problems.

You do not need to implement these from scratch. What you need is to understand: why does this algorithm work? When should I use it? What does it assume about the data?

Step 5: The Libraries You Will Actually Use

Python for machine learning means a short list of libraries that you will use over and over again:

NumPy

Numerical computing. Arrays, matrix operations. Under the hood of almost everything.

Pandas

Your data manipulation workhorse. Load CSVs, clean data, merge tables, explore distributions. You will use this constantly.

Matplotlib / Seaborn

Visualise your data. Plot distributions, spot outliers, understand what you are working with.

Scikit-learn

The gold standard ML library. Every classical algorithm you need, with a consistent API that is genuinely well-designed. This is where you will spend most of your time as a beginner.

XGBoost / LightGBM

Once you are comfortable with Scikit-learn, add these to your toolkit. They are industry workhorses.

Do not chase every new library. Master these first.

Step 6: Build Things That Are Slightly Uncomfortable

Reading is not learning machine learning. Building is learning machine learning.

After each concept, build something. It does not need to be impressive — it needs to be real. Some ideas:

Predict housing prices with linear regression on a public dataset
Build a spam classifier with logistic regression on email data
Predict customer churn with a Random Forest

Go to Kaggle. Find a beginner competition. Download the data. Make a terrible first submission. Then make a slightly less terrible second one. That process teaches you more than any course.

The discomfort of working with messy, real data and not knowing exactly what to do is not a sign that you are doing something wrong. It is the actual learning.

On Math: Stop Worrying About It

Yes, machine learning has mathematical foundations. Linear algebra, probability, calculus, statistics. They are all in there.

Here is the truth: you do not need to master any of that. Most of that is for people who will create the algorithms of the future, but not for building AI. You need enough statistics to understand what a mean and variance are. That is genuinely it.

Now, As you go deeper and start wondering why certain algorithms behave the way they do, you will naturally find yourself reading about the math behind them, but no need to master it. That is when it clicks, because you have context. Learning math in isolation, before you have built anything, is like studying the grammar of a language you have never spoken.

Pick up the math as you need it. Not before.

A Realistic Timeline

For someone starting from scratch and putting in consistent time:

Weeks 1–4

Python basics. Get comfortable with the language.

Months 2–3

Core ML skills and concepts: EDA, data prep, training, evaluation, the main algorithms, Scikit-learn. Build small projects.

Months 4–5

Go deeper. Tackle a real Kaggle dataset. Handle genuinely messy data. Deploy something small.

Months 6+

Expand: gradient boosting, feature engineering, model evaluation at depth.

This assumes a few focused hours per week, not full-time immersion. If you go full-time, compress everything. Consistency matters far more than intensity.

The Summary

Start with Python basics. Learn what machine learning actually is: pattern recognition from data, not magic. Then master the core skills in order: understand your data, prepare it, train a model, evaluate it properly, improve it, and deploy it. Learn the key algorithms well rather than every algorithm superficially. Build real things with real data. Pick up math as you need it, not before. That is how you learn machine learning from scratch.

At Fondra Labs, we are building the step-by-step resources to walk you through exactly this journey. Stay tuned.

Frequently Asked Questions

How to learn machine learning?

Start with Python basics, then work through the core ML skills in order: exploratory data analysis, data preparation, model training, evaluation, and deployment. Use Scikit-learn. Build real projects with real data.

How to get into machine learning?

You do not need a degree or math expertise to start. Pick up Python, learn the fundamentals, and build a portfolio of projects using public datasets. Kaggle is a great place to start.

How to start machine learning?

Write Python first — just the basics. Then pick one beginner dataset and try to build a prediction model with Scikit-learn. That first project, however messy, teaches you more than any course.

How do I start learning machine learning?

Pick one clear resource, follow it from start to finish, and build something at every stage. The most common mistake is jumping between resources constantly instead of going deep on one path.

How to become a machine learning engineer?

Learn the fundamentals, build a public portfolio of projects on GitHub, document what you built and why, and start applying. You do not need a perfect CV, you need demonstrated ability to work with real data and ship real models.

For more content you can visit our blog here!

AI Learning Roadmap: Where to Start if You're a Complete Beginner

Javier Aguirre — Wed, 27 May 2026 04:41:24 +0000

Nowadays, AI is everywhere. More than ever, people want to learn but being the internet flooded with resources makes it incredibly hard to know where to start. It feels like there is too much information, pointing in too many directions. I have been over 10 years in the AI field, and this blog is what you will actually need to understand the dos and don'ts of an effective AI learning roadmap. Keep on reading :)

The problem

Everyone starts in the wrong place

Everyone has heard of ChatGPT. Everyone has heard of LLMs, image generators, voice assistants. And so, naturally, everyone starts there because that's what's visible, exciting and all over the news.

Here's the thing: that is the biggest mistake you can make. What you see in ChatGPT is the latest and most complex technology in the entire AI field. Starting there is like deciding you want to become a chef and showing up to a three-Michelin-star kitchen on day one. It looks great from the outside. Inside, you will be completely lost.

But, don't panic! There is a right way to learn AI. It is not as hard as you think. But it requires starting at the foundation and I am going to show you exactly what that looks like for anyone wondering how to start learning AI.

The golden rule:
"Don't chase the latest. Master the foundations first, and the latest will start making sense on its own."
Javier Aguirre

Before diving in: learn basic Python

Before we talk about AI concepts at all, there is one practical thing to do first: learn the basics of Python. It is the language of AI, and you do not need to become a software engineer — you just need enough to write simple scripts, load data, and run models.

A few weeks of basics is more than enough to get started. Variables, loops, functions, lists. That's it for now. This is why so many people begin with Python for AI beginners courses before moving deeper into machine learning. And here's good news that will surprise most beginners:

On math:
"You do not need math to get into AI. Full stop. You may eventually bump into a concept here and there (a few statistics ideas, basic linear algebra) but those are easy to pick up when the moment comes. Don't let math be the reason you don't start."

The AI Learning Roadmap

How the AI world is actually structured

Before going straight in, you need to understand the landscape. That thing you've heard of — Generative AI — is not the beginning of the story. It is the current top of a much bigger structure. Think of it like a house: Generative AI is the roof. And nobody builds a house starting from the roof.

Let's look at how the AI house actually looks.

This diagram tells you everything you need to know about why most beginners struggle. Generative AI (ChatGPT, Midjourney, and the tools making headlines) sits at the centre of these nested layers. Every concept that powers it comes from the layers around it. Skip those layers and you are building your understanding on nothing.

My personal recommendations

Start with Machine Learning

Machine Learning is the oldest and most foundational part of modern AI. It is also, in many ways, the most powerful. It is driving enormous amounts of revenue across industries right now, from fraud detection to demand forecasting to personalisation engines. Companies are not running it because it's trendy. They're running it because it works.

If you are following a machine learning engineer roadmap, this is where your real understanding begins.

Will it generate text like ChatGPT? No. But here is what it will let you do:

*Predict outcomes: * Will this customer churn? What will sales be next quarter? Which loan applicant is high risk? ML answers these questions with high accuracy, using nothing more than historical data and a well-chosen model.
Classify anything: Is this email spam or not? Is this transaction fraudulent? Does this medical scan show an anomaly? Classification is one of the most commercially valuable things in AI, and ML is the gold standard for it.
Deploy cheap and fast: ML models are lightweight. They run on a basic server, cost little to host, and can be put into production in days. This is the opposite of the expensive GPU-hungry infrastructure that Generative AI requires.
Build real AI intuition: Understanding how a Random Forest learns, why a model overfits, what a training set versus a test set means — these concepts transfer directly to every other area of AI. This is where you grow actual understanding, not just surface-level familiarity.

The core ideas in Machine Learning are: supervised learning (teaching a model with labelled examples), unsupervised learning (finding patterns without labels), and the full process of training, evaluating, and deploying a model. Get comfortable with these, and the rest of AI opens up. If you have ever asked yourself what is machine learning?, this is the practical answer.

Then move to Deep Learning

Have you heard the words neural networks? Backpropagation? Deep learning? If so, this is what those words refer to. Deep learning is the natural evolution of classical machine learning and understanding ML first means you will actually grasp why deep learning exists, not just how to use it.

Instead of traditional algorithms, deep learning uses networks of artificial neurons (layers upon layers of them) that learn extremely complex patterns from data. The results are more powerful for many tasks, and more flexible, but they require significantly more data and computing resources to train.

If you are building your own deep learning roadmap, this is the stage where AI starts becoming truly powerful.

Here is where deep learning truly shines:

Images and computer vision

Deep learning powers every modern image recognition system — from the Face ID on your phone to the quality control cameras in a factory to self-driving car perception. Classical ML simply cannot match its accuracy on visual tasks.

Audio and speech

Voice assistants, real-time transcription, music generation, sound classification — all deep learning. The architecture that understands spoken language is built entirely on neural network layers.

Complex pattern recognition

Anything where the relationship between input and output is extremely non-linear and hard to express as rules, deep learning tends to be the right tool. Drug discovery, genomics, anomaly detection at scale.

Deep learning is also where you start encountering architectures with names such as : CNNs for images, RNNs for sequences, and (most importantly) the Transformer. Remember that name. It is what everything else is built on. This is also the point where people finally understand what is deep learning? in a meaningful way.

Finally, Generative AI (only after ML and DL!!!)

Now (and only now) does Generative AI make sense. Because once you understand ML and deep learning, you understand where GenAI comes from. The Transformer architecture at the heart of every modern large language model is a deep learning architecture. The training techniques are derived from everything you have already learned. The intuition transfers.

Generative AI is extraordinary. It can write, code, reason, create images, generate music, and hold conversations. The commercial excitement around it is real and justified. But here is something most people entering the field do not know:

Important Reality Check:
Most problems companies actually have can be solved with ML or DL — not GenAI. GenAI is incredibly expensive to run, hard to scale reliably, and often complete overkill for the task at hand. Jumping to GenAI head-on, without foundations, is a mistake that costs time, money, and understanding. Do not do it.
Javier Aguirre

GenAI is the mix of everything learned before. Understanding it properly — knowing when to use it, when not to, how to build on top of it rather than just prompting it — requires the foundations you built in steps 1 and 2. That is what separates someone who truly works in AI from someone who just uses it.

If you are wondering how does an AI learn?, the answer starts with these foundations in machine learning and deep learning long before Generative AI enters the picture.

More times than I remember projects fail and get stuck because managers and higher up people ask for genAI when a decision tree would have solved the problem. I am not saying generative AI is not marvelous, but, learning when to use is one of the best favours you can do yourself as a developer.

Summary

The right order, at a glance

1st: Python basics: Just enough to write scripts and work with data. No advanced engineering needed.
2nd Machine Learning: The oldest, most practical, most deployable, and most foundational layer of modern AI.
3rd Deep Learning: Neural networks, images, audio, the Transformer. The powerful evolution that enables everything modern.
4th Generative AI: The exciting frontier, but only makes sense once the foundations are solid.

One last thing

You do not need math. You do not need to be a genius. You do not need expensive bootcamps. You need consistency, the right order, and a willingness to build things even when they break.

At Fondra Labs, we are building the resources to walk you through every step of this journey in depth. Stay tuned, the real learning starts now.