The perceptron: the 1958 machine that learned from its mistakes

How one artificial neuron makes a decision, and how it fixes itself when it's wrong.

In 1958, psychologist Frank Rosenblatt unveiled the perceptron at the Cornell Aeronautical Laboratory. The press got excited. The New York Times wrote that the Navy expected it to one day walk, talk, see, and write.

It couldn't do any of that. But it introduced an idea that every modern neural network still uses: a machine that improves by fixing its own mistakes.

How a perceptron makes a decision

Under the hood, a perceptron is a single artificial neuron. It does three things:

1. It takes some inputs, like x₁ and x₂.
2. It multiplies each input by a weight (w₁ and w₂) and adds them up.
3. It compares the sum with a threshold, written θ (theta).

If the sum clears the threshold, the perceptron outputs 1. Otherwise, it outputs 0. That's the whole decision: yes or no.

A perceptron: weighted inputs are summed, then a threshold step turns the sum into a 1 or a 0.

The rule draws a straight line

Here's the useful way to picture it. With two inputs, every example is a point on a flat plane, at position (x₁, x₂).

The boundary where the sum exactly equals the threshold, w₁x₁ + w₂x₂ = θ, is a straight line. Points on one side get a 1. Points on the other side get a 0.

So a perceptron is a line-drawer. Its weights decide where the line sits and which way it tilts.

With two inputs, the perceptron's rule splits the plane with a straight line: one side outputs 1, the other outputs 0.

Learning from mistakes

A new perceptron starts with weights that put its line in the wrong place. So how does it get better?

1. It looks at one example at a time and makes a guess.
2. If the guess is right, nothing changes.
3. If the guess is wrong, it nudges each weight by the input times the error.

As a formula: w ← w + x · error, where the error is the target minus the output. In practice, the nudge is often scaled down by a small number called the learning rate, and the threshold is adjusted the same way.

A worked example

Say the perceptron outputs 0 for a point, but the right answer was 1.

• The error is 1 − 0 = +1.
• So each weight moves by its own input. Weights for strong inputs move the most.
• The line swings toward that point, until the point ends up on the correct side.

If it had output 1 when the answer was 0, the error would be −1, and the weights would move the other way.

Target 1, output 0: the error is +1, so the weights grow and the line swings toward the missed point.

When it works, and when it doesn't

Repeat this, example after example, and the line keeps moving until every point lands on the right side.

Rosenblatt proved something remarkable: if one straight line can separate the two classes, this rule always finds such a line, in a finite number of steps. This is known as the perceptron convergence theorem.

The catch is in that "if." Some problems have no single separating line at all. The classic example is XOR, and solving it takes more than one neuron.

Example after example, the line settles where it separates every point. This is guaranteed whenever a straight line can do it.

Quick check

The perceptron gets a point right. Do its weights change, or stay the same?

Answer: they stay the same. When the output is correct, the error is zero, and input times zero changes nothing. A perceptron only learns from its mistakes.

Key takeaways

• A perceptron is one neuron: a weighted sum, then a threshold that outputs 1 or 0.
• With two inputs, its decision boundary is a straight line.
• It learns by nudging each weight by the input times the error, and only when it's wrong.
• If a straight line can separate the classes, it is guaranteed to find one.

---

This article is part of Deep Learning, Visually by Logit Solutions AI: short animated explainers that cover one idea at a time, each with a written version like this one.