They don't. They join many straight lines into shapes, and that's where a network's power comes from.
A single neuron can only draw one straight line. Yet neural networks recognize faces, read handwriting, and trace curved boundaries everywhere. So where do the curves come from?
The surprising answer: from straight lines, lots of them, combined by a hidden layer.
One line can't fence in a cluster
A neuron splits the plane in two with a straight line. Points on one side get a "yes," and points on the other side get a "no."
That works when the classes sit on opposite sides of the page. But look at this case: the blue points form a cluster in the middle, surrounded by the others. Wherever you draw one straight line, it can't fence the cluster in.
The blue points sit in the middle. A single straight line can't separate them from the points around them.
Add a hidden layer
So add a layer of neurons between the inputs and the output. This is a hidden layer. Give it three neurons.
Each hidden neuron draws its own line, and each one asks a single question about any point: "Is this point on my side?" The answer is yes (1) or no (0).
The output neuron asks for agreement
The output neuron adds up the three answers. It only fires when the sum passes 2.5, so it needs all three to say yes.
The only place where all three lines agree is the region between them. Together, the three lines fence off a triangle.
Three hidden neurons draw three lines. The output fires only where all three agree, which is a triangle.
A worked example
Take a point inside the triangle. All three neurons say yes, so the sum is 1 + 1 + 1 = 3. That passes 2.5, so the output fires.
Now take a point just past one of the lines. Only two neurons say yes, so the sum is 0 + 1 + 1 = 2. That's below 2.5, so the output stays off.
That threshold turns the output neuron into an "AND" of all the hidden neurons.
More neurons, rounder shapes
The triangle is a rough fit, and its corners catch a few outsiders. The fix is simple: use more hidden neurons.
- With six neurons, you get six lines and a tighter hexagon.
- With twenty, the edge looks almost like a circle.
The edge is never truly curved. It's made of many short straight pieces, like a polygon with lots of sides. Add enough of them, and the difference stops mattering.
More hidden neurons means more straight edges. With twenty, the boundary looks almost like a circle.
Go deeper
A second hidden layer can combine the shapes from the first one. That lets a network fence off things a single shape can't, like two separate islands, or a ring with a hole in the middle.
This is why depth matters. Each layer builds on the shapes from the layer before it.
A second hidden layer combines shapes: two islands, or a ring with a hole.
Quick check
A hidden layer has five neurons. Does the output fence off a pentagon, or a perfect circle?
Answer: a pentagon. Five neurons draw five straight lines, so the edge has (at most) five sides. A perfect circle would need infinitely many.
Key takeaways
- One neuron draws one straight line.
- A hidden layer draws many lines, and the output neuron fires only where they agree.
- Three lines make a triangle, six a hexagon, and twenty something close to a circle.
- More layers combine shapes into islands, rings, and other complex regions.
- Hidden layers never bend a line. They join many straight ones into a curve.
This article is part of Deep Learning, Visually by Logit Solutions AI: short animated explainers that cover one idea at a time, each with a written version like this one.
Top comments (0)