DEV Community: Fahim Uddin

The Shape of Being Wrong

Fahim Uddin — Mon, 06 Jul 2026 14:59:51 +0000

You can build the most elegant neural network architecture in the world, and it will learn absolutely nothing until you answer one question first: wrong compared to what, exactly?

That's what a loss function is. It's not a technical afterthought bolted onto the end of a network — it's the thing that turns "prediction" into "learning." Everything from here on, every gradient that flows backward through every layer, starts as a number produced by this one function. Get it wrong, and the rest of the machinery doesn't matter.

So let's actually derive where these loss functions come from, instead of just memorizing "use MSE for regression, cross-entropy for classification" as a rule of thumb.

Loss functions aren't arbitrary — they're probability in disguise

Here's the thing most intro explanations skip: loss functions aren't hand-picked because they "feel right." They fall directly out of maximum likelihood estimation — you assume a probability distribution over your labels, and the loss function is just the negative log of that distribution's likelihood.

Start with regression. Assume your network's prediction is the mean of a Gaussian, with some noise:

$$p(y \mid x, w, \beta) = \mathcal{N}\big(\hat{y}(x, w), \, 1/\beta\big)$$

Take the negative log-likelihood over your whole dataset, expand the Gaussian, and something clean happens: every term that doesn't depend on your weights — constants, the noise variance — drops away during optimization. What's left is:

$$\frac{\beta}{2} \sum_{m=1}^{M} \big(y_m - \hat{y}(x_m, w)\big)^2$$

That's mean squared error. Not chosen because "distance feels intuitive," but because it's exactly what falls out of assuming Gaussian noise on a continuous target. If you ever get asked "why L2 loss for regression" on an exam, this is the actual answer, not "it penalizes big errors more."

Now do the same thing for classification, except your label distribution isn't Gaussian — it's categorical. For a coin flip, your label model is a Bernoulli distribution. For $K$ classes, it's the multinoulli:

$$\mathfrak{C}(y \mid p) = \prod_{k} p_k^{\,y_k}$$

Take the negative log-likelihood of that, and the product turns into a sum of logs:

$$L(w) = -\sum_{m=1}^{M} \sum_{k} y_{k,m} \, \ln \hat{y}_{k,m}$$

This is cross-entropy. Same derivation pattern, different assumed distribution. Once you see it this way, "which loss function should I use" stops being a lookup-table question and becomes: what's the actual probabilistic story behind my output? Continuous and Gaussian-ish → L2. Discrete and categorical → cross-entropy.

Why one-hot encoding isn't just a formatting choice

If you have 5 classes and the true label is class 3, you write it as:

$$y = \begin{bmatrix}0\0\0\1\0\end{bmatrix}$$

This looks almost too simple to explain, but it's doing real mathematical work. Encode your labels as raw integers (0, 1, 2, 3, 4) instead, and you've accidentally told your model that class 4 is somehow "more" than class 0 — an ordinal relationship that doesn't exist between "cat" and "dog." One-hot encoding removes that false structure entirely, and it maps cleanly onto the multinoulli distribution above, where exactly one $y_k$ is nonzero.

That last part is what makes cross-entropy so cheap to compute. Plug a one-hot vector into $-\sum_k y_k \ln \hat{y}_k$, and every term where $y_k = 0$ vanishes. You're left with:

$$L = -\ln \hat{y}_{k^*}$$

— the negative log of whatever probability your model assigned to the correct class. That's it. Cross-entropy loss, in the one-hot case, is just "how much did the model doubt the right answer." Confident and correct → loss near zero. Confident and wrong → loss explodes toward infinity. That asymmetry is the whole point — it's what makes the model care about fixing its most confident mistakes first.

The elegant accident of softmax and cross-entropy

Softmax converts a network's raw, unbounded output scores (logits) into something that behaves like a probability distribution:

$$\hat{y}_k = \frac{e^{z_k}}{\sum_j e^{z_j}}$$

Every output lands strictly between 0 and 1, and they all sum to exactly 1. Good — but softmax by itself is expensive to differentiate. Because of that normalization term in the denominator, every output depends on every input logit, which means its gradient is a full Jacobian matrix, not a simple per-element derivative.

Here's where it gets genuinely elegant. Pair softmax with cross-entropy — the combination that shows up in essentially every classification network you'll ever train — and take the gradient of the loss with respect to the pre-softmax logits. All the exponentials cancel. All the log terms cancel. The messy Jacobian collapses into:

$$\frac{\partial L}{\partial z_k} = \hat{y}_k - y_k$$

Predicted probability, minus the true one-hot label. That's the entire gradient. No exponentials, no matrix, just a vector subtraction.

Sit with how strange that is for a second. You started with an exponential normalization and a logarithm, stacked two genuinely nonlinear, seemingly-unrelated functions on top of each other, and the derivative simplified into something a first-grader could compute. That's not a coincidence dictated by convenience — it's a direct consequence of softmax being the correct inverse link function for a categorical likelihood, and cross-entropy being the correct loss for that same likelihood. When you pick the mathematically consistent pair, the calculus rewards you.

Practically, this is also why deep learning frameworks fuse softmax and cross-entropy into a single operation internally — softmax_cross_entropy_with_logits-style implementations skip the unstable, wasteful path of computing softmax, then log, then backpropagating through both separately.

Descending the loss surface

Once you have a loss value, you need to actually reduce it. The update rule is deceptively small:

$$W_{\text{new}} = W - \eta \cdot \frac{\partial L}{\partial W}$$

The gradient points toward steepest increase; you subtract it to head downhill. The learning rate $\eta$ controls your stride length. Too large, and you overshoot the valley floor, bouncing between its walls instead of settling into it — in the worst case the loss diverges outright. Too small, and you crawl toward the minimum so slowly that you might never get there in a reasonable number of steps, or get trapped in some shallow dip along the way with no momentum to escape it.

That tension — step too far and oscillate, step too little and crawl — is the entire reason the rest of optimization research exists.

The first fork in the road is how much data you use to compute each gradient step. Use the whole dataset every time (batch gradient descent) and your gradient estimate is smooth and accurate, but each step is painfully slow. Use a single random sample (stochastic gradient descent) and each step is nearly instant, but wildly noisy. Mini-batch gradient descent — a few dozen to a few hundred samples per step — is the practical compromise almost everyone actually uses: fast enough per step, smooth enough to make real progress, and it happens to line up perfectly with how GPUs like to be fed data.

Giving the optimizer memory and self-adjustment

Plain gradient descent has no memory — every step reacts only to the current gradient and forgets everything that came before. Momentum fixes that by keeping a running, decaying average of past gradients:

$$v^{(k)} = \mu \, v^{(k-1)} + \nabla L(W^{(k)})$$

Picture a heavy ball rolling downhill instead of a point mass that stops and restarts at every step. In directions where the gradient keeps pointing the same way, the ball builds up speed. In directions where the gradient keeps flipping sign — the classic symptom of a narrow, steep-walled ravine in the loss surface — the accumulated momentum in one direction cancels the momentum in the other, and the oscillation dampens out. Nesterov's variant refines this further by computing the gradient at the look-ahead position the momentum is already carrying you toward, correcting course slightly earlier and cutting down oscillation even more in badly-conditioned landscapes.

Momentum, though, still uses one global learning rate for every parameter in the network. But not every parameter needs updating at the same rate — some features fire constantly, others rarely. AdaGrad addressed this by tracking each parameter's accumulated squared gradient and scaling its individual learning rate inversely — frequent, large-gradient parameters get gently reined in, rare ones get relatively larger steps. Its flaw: that accumulator only ever grows, so learning rates eventually decay toward zero regardless of whether training is actually done. RMSProp fixes this by decaying the accumulator over time instead of summing forever, so recent gradients matter more than ancient ones.

Adam — by far the most common optimizer you'll encounter in practice — combines both ideas: a momentum term for direction, and an RMSProp-style adaptive scale for step size, with a bias correction added for both since they start at zero. It's not magic, and it's worth knowing it isn't perfect: Adam's original convergence proof turned out to contain an actual bug, later patched by a variant called AMSGrad. And in a lot of published state-of-the-art results, plain SGD with Nesterov momentum and a carefully tuned learning rate schedule still edges out Adam on final performance, even though Adam converges more predictably out of the box. The honest recommendation, and the one worth remembering: start with mini-batch SGD plus momentum, reach for Adam once you have a feel for your data, and keep your eyes open regardless of which one you pick.

Trusting your own backpropagation

There's a quieter, less glamorous idea buried in this chapter that matters just as much as any of the optimizers: how do you know your backprop implementation is actually correct?

You check it against a version that doesn't rely on calculus at all — the definition of a derivative, approximated directly:

$$\frac{\partial f}{\partial x} \approx \frac{f(x+\epsilon) - f(x-\epsilon)}{2\epsilon}$$

This centered version is deliberately more accurate than the simpler one-sided version $\frac{f(x+\epsilon) - f(x)}{\epsilon}$ — the first-order error terms cancel out in the subtraction, leaving an error that shrinks quadratically instead of linearly as $\epsilon$ shrinks. Compute your gradient this way, compare it to what your analytical backprop implementation produces, and if they agree closely, you have real evidence your chain rule is implemented correctly. If they don't, you have a bug, and you have it before wasting hours or days training on top of it.

This isn't a minor footnote. Backprop bugs are famously silent — a broken gradient doesn't crash your program, it just quietly trains a worse network, and you often only notice something's "off" without ever finding the actual cause. Even major, widely-used software frameworks have shipped subtly incorrect gradient computations that went unnoticed for a long time. The lesson generalizes past this one chapter: gradient-based methods are forgiving enough that as long as you're roughly following the right direction, you'll still get some kind of result — which is exactly what makes them so hard to debug when something's subtly wrong.

Where this leaves us

Zoom out, and this chapter is really about one idea wearing several disguises: define what "wrong" means in a way that's grounded in probability, turn that definition into a number you can differentiate, and then descend that number's landscape as efficiently as you can — while occasionally checking, by brute force, that your descent direction is actually correct.

Next up: what happens when the inputs aren't just flat vectors of independent features, but images, where nearby pixels are correlated and position actually matters. That's where convolutions come in.

From a Single Line to a Universal Function Machine: How Feedforward Networks Work

Fahim Uddin — Sun, 05 Jul 2026 19:42:11 +0000

In the last post, we talked about 2012 — the year deep learning stopped being an academic curiosity and started winning. But before we can appreciate why that moment mattered, we need to understand what these networks actually do, mechanically, when you feed them a number. Not the hand-wavy "it's like a brain" version. The actual math, the actual shapes, the actual reason a stack of matrix multiplications can approximate almost any function you throw at it.

So let's go back to the simplest possible neural network — one that isn't even really a network yet — and build up from there.

The perceptron: one line, one decision

Strip away everything, and a perceptron does exactly one thing: it draws a straight line (or a plane, or a hyperplane, depending on dimension) and asks which side are you on?

Given an input vector x and a weight vector w, the perceptron computes:

ŷ = sign(w · x)

That's the whole model. Multiply, sum, check the sign. If w · x is positive, predict one class; if negative, predict the other.

The geometry here is worth sitting with, because it explains both the perceptron's power and its ceiling. The equation w · x = 0 defines a hyperplane, and w is the vector perpendicular to that hyperplane. For any point not sitting exactly on the boundary, w · x is proportional to that point's signed distance from it — positive on one side, negative on the other. The perceptron isn't doing anything conceptually deep; it's just measuring which side of a line you fell on and reporting the sign.

This is also exactly why a single perceptron cannot solve XOR. Plot the four XOR points and you'll see the two classes sitting diagonally opposite each other — there is no single straight line that separates them. The perceptron's decision boundary is always a hyperplane, no matter how you tune the weights, so a problem like XOR is simply out of reach for it. This limitation, formalized in Minsky and Papert's 1969 book Perceptrons, is a big part of why AI research funding dried up for most of the 1970s — the "AI winter" wasn't caused by hype dying naturally, it was caused by a proof that the simplest version of this idea had a hard ceiling.

From one decision to a network of decisions

The fix turns out to be almost embarrassingly simple: stack more of them, and swap the sign function for something smoother.

A multi-layer perceptron (MLP) takes the same building block — weighted sum, then a function applied to it — and arranges many of them into layers. An input layer supplies the raw data, one or more hidden layers transform it, and an output layer produces the final prediction. Each hidden neuron receives a weighted sum of everything in the previous layer, applies a non-linearity, and passes the result forward.

That non-linearity is not optional decoration — it's the entire reason depth matters. Here's the argument: if every layer were purely linear, then a three-layer network would compute

ŷ = W₃(W₂(W₁x)) = (W₃W₂W₁)x

and W₃W₂W₁ is just... another matrix. Multiply as many linear layers together as you like, and you still only get a single linear transformation. Depth would be a complete waste of compute. The non-linear activation function sandwiched between each linear step is what stops this collapse from happening and gives depth an actual reason to exist.

Picking a non-linearity: sigmoid, tanh, ReLU

Not all activation functions are created equal, and the field's history is basically a story of people discovering why their current favorite has a problem, and fixing it.

Sigmoid was the first popular choice — an S-shaped curve squashing everything into (0, 1), with a conveniently simple derivative: f(x)(1 − f(x)). The catch is that this derivative maxes out at just 0.25, and collapses toward zero the moment you move more than a few units away from 0. The function saturates.

Tanh is sigmoid's cousin, squashing into (−1, 1) instead, with the nice property of being zero-centered — which tends to produce better-behaved gradients downstream. But it saturates too, for the same structural reason.

ReLU (rectified linear unit) is almost insultingly simple: max(0, x). No exponentials, dirt cheap to compute, and its derivative is either exactly 1 (for positive inputs) or exactly 0 (for negative ones). No saturation on the positive side, ever. This is the default choice for hidden layers in most modern architectures — not because it's clever, but because it doesn't get in its own way.

Watching data actually move through the network

Let's make this concrete instead of abstract. Take a tiny network: 3 inputs, one hidden layer with 2 ReLU neurons, one sigmoid output neuron.

x  = [1, 2, 3]

W1 = [ 0.1   0.2  -0.1 ]     b1 = [ 0.1 ]
     [ 0.3  -0.2   0.05]          [-0.1 ]

W2 = [ 0.5  -0.3 ]           b2 = [ 0.2 ]

Hidden pre-activation, z1 = W1·x + b1:

z1[0] = 0.1(1) + 0.2(2) - 0.1(3) + 0.1  =  0.3
z1[1] = 0.3(1) - 0.2(2) + 0.05(3) - 0.1 = -0.05

Apply ReLU — the negative value gets clipped to zero, no exceptions:

a1 = [0.3, 0]

Output pre-activation, z2 = W2·a1 + b2:

z2 = 0.5(0.3) + (-0.3)(0) + 0.2 = 0.35

Apply sigmoid:

ŷ = 1 / (1 + e^-0.35) ≈ 0.587

That's it — that's the entire forward pass. Linear combination, non-linearity, linear combination, non-linearity, prediction. Everything a feedforward network does, at any scale, is this pattern repeated more times with bigger matrices.

Which raises the obvious question: why bother writing it as matrix multiplication instead of just... doing this arithmetic neuron by neuron? Two reasons. First, it maps directly onto the kind of vectorized computation GPUs are built for — instead of looping over neurons, you do one matrix multiply. Second, and more importantly for what's coming next, matrix notation gives you clean derivatives. Once you write a layer as ŷ = Wx, matrix calculus hands you ∂ŷ/∂W = xᵀ and ∂ŷ/∂x = Wᵀ — two identities that turn out to be the entire mathematical engine behind training the network.

Dimension bookkeeping, while we're here, is simple: a layer mapping an n-dimensional input to an m-dimensional output has a weight matrix of shape (m × n). The output dimension is always just the number of neurons in that layer — nothing more mysterious than that.

Teaching the network: backpropagation

Forward pass gets you a prediction. It says nothing about how to improve it. That's backpropagation's job, and it answers one specific question: how much did each weight contribute to the final error, and which direction should I nudge it?

The mechanism is two passes. Forward, to compute the prediction and the loss. Backward, to walk from that loss back through the network, layer by layer, figuring out each weight's share of the blame — using the chain rule the entire way.

The chain rule is deceptively simple: if g = f(h(x)), then dg/dx = df/dh · dh/dx. A neural network is nothing but a deeply nested function — layer inside layer inside layer — so the chain rule is the only reason it's even possible to compute how a change to some deeply buried weight ripples all the way forward to affect the final loss.

A concrete example makes this less abstract. Suppose:

a = 2x₁          → a = 2
c = a + b        → c = 11    (with b = 9)
e = c²           → e = 121
g = e + 3        → g = 124

To get ∂g/∂x₁, you don't need to re-derive anything from scratch — you multiply the local derivatives along the path:

∂g/∂x₁ = (∂g/∂e) · (∂e/∂c) · (∂c/∂a) · (∂a/∂x₁)
        =    1    ·   2c   ·    1    ·    2
        =    1    ·   22   ·    1    ·    2   =  44

Each piece is trivial on its own. The chain rule is what lets you chain them into an answer for a variable buried four steps deep — and it's also why backpropagation can reuse intermediate results instead of recomputing everything from scratch for every single weight, which is what makes it efficient enough to actually train networks with millions of parameters.

One more thing worth being precise about: backpropagation is not the training algorithm. It's just the mechanism for computing the gradient. Gradient descent (or a variant of it) is the separate step that actually uses that gradient to update the weights.

The sign function's fatal flaw

This is also the moment where the perceptron's original activation function — sign — permanently disqualifies itself. Its derivative is 0 everywhere except at x = 0, where it's undefined. Plug a derivative of 0 into a chain-rule product, and the entire product becomes 0. No gradient signal reaches any weight upstream. Gradient descent has nothing to work with. This is precisely why the field moved to smooth, differentiable activations — first sigmoid, later ReLU — that have a well-defined, non-zero derivative over a useful range.

The problem hiding inside the chain rule: vanishing and exploding gradients

Here's the uncomfortable part. Backpropagation computes a weight's gradient as a product of many local derivatives — one per layer standing between that weight and the loss. If those local derivatives are consistently less than 1, the product shrinks exponentially with depth. If they're consistently greater than 1, it explodes exponentially. Depth, the thing that makes networks powerful, is also the thing that makes this problem worse the deeper you go.

This is exactly why sigmoid struggles in deep networks. Its derivative tops out at 0.25 and collapses toward zero as inputs move away from the origin. Stack even a modest number of sigmoid layers, and you're multiplying several numbers that are each at most 0.25 — the gradient reaching early layers vanishes almost immediately.

ReLU's derivative, by contrast, is exactly 1 for any active (positive) unit. Multiplying by 1 doesn't shrink anything. This single property — not saturating on the positive side — is a large part of why ReLU became the default and why genuinely deep networks became trainable in the first place.

In practice, vanishing gradients look like a loss curve that drops for a few iterations and then goes nearly flat, even though nothing has technically broken — no NaNs, no divergence, just early layers that have effectively stopped learning. Exploding gradients look like the opposite: wild swings in the loss, sometimes an increase in loss despite gradient descent trying to minimize it, and eventually numerical garbage. Both are tangled up with the learning rate η — too high, and you're closer to exploding; too low, and you get something that looks a lot like vanishing, just from a different cause. Getting η right is the difference between a loss curve that decreases smoothly and one that either stalls or spirals.

Why this matters before anything else

Everything modern — batch normalization, residual connections, better initialization schemes, adaptive optimizers — exists because someone ran into one of these exact problems: a linear collapse, a dead gradient, a vanishing signal, an unstable step size. None of it is solving a new problem. It's all solving this problem, the one sitting quietly inside a network as simple as three inputs and two hidden neurons.

Which is really the whole point of starting here. The perceptron's straight line and the modern deep network's messy, high-dimensional decision surface are running on the same underlying machinery: weighted sums, non-linearities, and a chain rule quietly multiplying its way backward through every layer. Understand that machinery at this scale, and the more sophisticated architectures stop looking like magic — they start looking like reasonable answers to problems you already know exist.

Next up: loss functions and optimization — what actually happens after backpropagation hands you a gradient.

The Big Bang of Deep Learning: How 2012 Changed Everything

Fahim Uddin — Fri, 03 Jul 2026 13:09:51 +0000

Every field has a moment where the story splits into "before" and "after." For deep learning, that moment has a year attached to it: 2012. This is the first post in a series where I'll be working through my Deep Learning course notes and turning them into something more digestible — starting at the very beginning, with the question of why this field exploded when it did.

The problem nobody could crack

Picture the state of computer vision before 2012. Researchers had a benchmark called ImageNet — a database of roughly 14 million images, organized into about 20,000 categories. A subset of this became the ImageNet Large Scale Visual Recognition Challenge (ILSVRC), which asked systems to sort images into one of 1,000 classes, based on nothing but images scraped from the internet, each carrying a single label.

At the time, classifying images into a thousand categories wasn't just hard — it was considered close to impossible. Error rates on the challenge had been stuck around 25% (measured as "Top-5 error," meaning the correct label had to appear among a model's top five guesses) for years. Progress had stalled. Nobody had a clear path forward.

Enter AlexNet

In 2012, a team led by Alex Krizhevsky entered the competition with something different: a convolutional neural network (CNN). Instead of relying on hand-engineered rules for what to look for in an image, the network learned its own representations directly from the pixels.

The result nearly halved the error rate in a single year. This wasn't an incremental improvement — it was the kind of jump that made the rest of the field stop and pay attention. And it kept going: in the years that followed, ILSVRC error rates continued to drop, eventually approaching — and some claimed surpassing — human-level performance.

That claim is worth pausing on, though. "Superhuman performance" sounds impressive, but how many humans had actually gone through the entire test set to establish a real baseline? Barely any. One researcher, Andrej Karpathy, famously did sit down and manually label the whole test set himself — which led to the joke that what these systems achieved wasn't quite "superhuman," but "super-Karpathy-an." It's a good reminder to look closely at benchmark claims rather than taking headline numbers at face value.

It's also worth noting ImageNet wasn't a perfect benchmark. Some images were genuinely ambiguous — a photo labeled "cherry" that also happens to show a dog, for instance. When a dataset only allows one label per image, it inevitably runs into cases where reality doesn't fit neatly into a single box.

Why a GPU company became one of the most valuable companies in the world

Here's a connection that isn't obvious at first: why did NVIDIA's stock price start climbing around the same time deep learning took off?

The answer is compute. Training neural networks means doing enormous numbers of matrix multiplications, and GPUs — originally built to render graphics — turned out to be extremely good at exactly that kind of math. As deep learning adoption grew, so did demand for GPU hardware.

But the story isn't purely a straight line. There's a noticeable dip in NVIDIA's stock around 2018–2019, and deep learning demand alone doesn't explain it. Around the same time, Bitcoin's value dropped sharply, and cryptocurrency mining had also been a major driver of GPU demand. So NVIDIA's rise reflects two overlapping trends — AI compute and crypto mining — not deep learning in isolation. It's a useful reminder that market signals are rarely caused by just one thing, even when the more exciting explanation is tempting.

Deep learning leaves the lab

Once the ILSVRC breakthrough proved CNNs worked, adoption spread fast. A few examples from the era:

Netflix — the Netflix Prize, a $1 million challenge to build a better recommendation engine, was partly solved using deep learning techniques.
Siemens and GE — healthcare imaging and diagnostics.
Daimler and other automakers — the push toward autonomous driving.
Google, Microsoft, IBM, Apple, Samsung — deep learning woven into core products across the board.

This is the shift from "interesting research result" to "technology reshaping industries" — and it happened remarkably quickly after 2012.

A different kind of proof: games

Around the same time, deep learning was also proving itself in a very different arena: games.

Chess had already fallen to computers back in 1997, when Deep Blue beat Garry Kasparov. But chess is, in a sense, a more tractable problem — engines could lean on a database of known opening moves, brute-force search through the middlegame, and another database for endgames.

Go is a different beast entirely. On a 19×19 board, a player can place a stone on almost any open point on any turn. That means the number of possible game states explodes far faster than in chess — so fast that even today's compute power can't brute-force it. Go required something smarter than search.

That "something smarter" arrived in 2016, when AlphaGo beat a professional Go player for the first time. A year later, AlphaGo Zero surpassed every human player — having learned entirely through self-play, without any human game data at all. Then AlphaZero generalized the same approach to other board games, and by 2019, AlphaStar was beating professional players at StarCraft, a real-time strategy game with far messier, less discrete decision-making than Go.

Why this history actually matters

It's tempting to treat this kind of timeline as trivia — dates and milestones to memorize for an exam. But there's a real reason to understand it before diving into the technical machinery of neural networks: it tells you what problem deep learning was actually built to solve.

The throughline across ImageNet, AlexNet, and AlphaGo is the same: traditional approaches relied on humans encoding the rules or features by hand, and that approach hit a ceiling. What changed in 2012 — and what will show up again and again as we get into convolutional layers, architectures, and training techniques — is systems learning their own representations directly from data, at a scale humans never could have hand-engineered.

That's the thread I'll be pulling on for the rest of this series. Next up: what's actually happening inside a neural network when it "learns" a representation.