This is the second post in my "revisiting my AI/ML notes" series — last time I covered the AI vs ML vs DL vs Data Science taxonomy. This time, going one level deeper into the first deep learning architecture most people learn: the Artificial Neural Network (ANN).
What is an ANN?
An ANN is the baseline deep learning architecture, and it's specifically built for tabular data — the kind of data you'd normally find in a spreadsheet, organized into rows and columns.
Every neural network, no matter how complex it eventually gets, is built out of three types of layers:
- Input layer — where raw features enter the network
- Hidden layer — where the actual processing happens; this is where the network learns patterns
- Output layer — where the final prediction comes out
A concrete example
Say you have tabular information about some object — numeric features like weight, height, ear shape — and your task is to decide whether it's a cat or a dog.
Here's how that flows through the network:
- Each feature (let's call them F1, F2, F3) enters through its own node in the input layer.
- Every input node connects to every node in the hidden layer — this is called a fully connected layer. This is why diagrams of neural networks look like a tangle of crossed lines; every input genuinely does touch every hidden neuron.
- The hidden layer processes this information and works out which combination of features points toward "cat" versus "dog."
- The output layer delivers the final decision.
The number of hidden layer neurons isn't fixed — it depends on the complexity of the problem. More neurons (and more hidden layers) generally mean the network can learn more complex patterns, but also means more computation and a higher risk of overfitting.
Where this all comes from
A couple of names worth knowing if you're going deeper into this field:
- The perceptron was the earliest and simplest version of a neural network — a single neuron making a basic binary decision. Everything since has built on this idea.
- Geoffrey Hinton is one of the researchers most associated with popularizing backpropagation — the algorithm that lets a network adjust its weights based on how wrong its predictions were. This is genuinely the mechanism that makes "learning" possible in a neural network, and it deserves its own deep dive (coming in the next post in this series).
Why this matters
It's tempting to skip past the basic ANN structure to get to the more exciting architectures — CNNs for images, transformers for text. But almost everything in deep learning is a variation on this same input → hidden → output idea, with different constraints on how the layers connect and what kind of data flows through them. Getting comfortable with plain ANN forward flow first makes every architecture after it easier to reason about.
Next up: how a single neuron actually computes its output — weights, bias, and why sigmoid shows up everywhere.
Original notes:
Top comments (0)