Every developer working with models eventually hits a bug that has no stack trace. The output is wrong, the code is fine, and the only honest answer is that the weights encode something you did not intend. Understanding what training changes, and what it leaves alone, makes those failures a lot easier to reason about.
The Weights Are The Model
A trained network is a large set of numbers plus the graph that says how to multiply them. Every neuron multiplies its inputs by learned weights, adds a bias, and passes the sum through a nonlinear function. Stack those layers and the whole thing becomes a function from input vector to output vector.
Training touches exactly one thing: the values of those weights and biases. It does not change the architecture, the activation functions, the layer widths, or anything else you configured before the first batch. Every capability a model has, and every failure it exhibits, lives in numbers that were nudged into place by a training loop. This guide to how neural networks work covers the architecture side in more depth if you want the fuller picture.
Why Training Needs Both A Loss And A Gradient
A loss function turns "the output was wrong" into a single number. That is the part people usually remember. The more useful half is the gradient, which answers a much harder question: of the millions of weights involved, how much did each one contribute to that number?
Backpropagation computes that by walking the computation graph backwards and applying the chain rule. Each weight then moves a small step in the direction that reduces the loss. The step size is the learning rate, and it is the single hyperparameter most likely to be the reason a run does not converge.
Nothing in this process writes a rule. There is no moment where the model decides that a certain feature means a certain label. There is only a very long sequence of small corrections that happen to end in weights that produce good outputs on data resembling the training set.
What Overfitting Looks Like From The Inside
Overfitting is usually explained with a chart of two curves diverging. From the inside it is simpler than that: the model has enough capacity to encode specifics of the training examples themselves, so it does, because doing so lowers the loss.
That is why the fixes work the way they do. Holding out validation data gives you a signal that is not part of what the weights were fit to. Dropout stops any single path through the network from becoming load bearing. Weight decay keeps individual weights from growing large enough to encode a single example. Each one limits how precisely the weights can memorize rather than generalize.
Reading Model Failures With This In Mind
Once you accept that a model is fit statistics rather than logic, some common behaviours stop being surprising.
A model confidently produces a wrong answer because confidence is an output of the same fitting process, not an independent check on it. It fails on inputs that look slightly different from the training distribution because nothing in the weights encodes the concept, only the correlations that were present in the data. It cannot explain itself because the reason is spread across millions of weights, none of which is individually meaningful.
The practical version: when a model misbehaves, ask what the training data would have made likely, not what the correct reasoning would have been.
Takeaway
Weights are the entire learned artifact, training only adjusts them, and every strength and weakness of a model traces back to that. It is a small mental model to carry, and it explains far more model behaviour than any amount of prompt tinkering will.
Top comments (0)