I Built a Neural Network's First Neuron From Scratch — the 1958 Perceptron

Before transformers, before backprop, there was one neuron — Frank Rosenblatt's 1958 Perceptron. Build it and you understand the atom that every deep network is made of.

This is Day 1 of DeepLearningFromZero: neural nets built from a single neuron up, no framework magic.

A neuron is shockingly simple

Take inputs, multiply each by a weight, add a bias, then apply an activation. The original used a step: output +1 if the sum is ≥ 0, else −1.

let w = [Math.random(), Math.random()], b = 0;
const sum = x => w[0]*x[0] + w[1]*x[1] + b;
const predict = x => sum(x) >= 0 ? 1 : -1;

The weights ARE a line

w₁·x₁ + w₂·x₂ + b = 0 is the equation of a straight line — the decision boundary. One side is class +1, the other is −1. So a neuron's entire "knowledge" is the tilt and position of one line.

The learning rule: only fix mistakes

Predict each point. If correct, do nothing. If wrong, nudge the weights toward the right answer:

for (const { x, y } of data) {
  if (predict(x) !== y) {        // y is the true label, +1 or -1
    w[0] += lr * y * x[0];
    w[1] += lr * y * x[1];
    b    += lr * y;
  }
}

Geometrically, that rotates and shifts the line so the misclassified point ends up on the correct side. Repeat for a few epochs:

for (let epoch = 0; epoch < 50; epoch++) trainStep(0.1);

The guarantee that launched a field

Rosenblatt proved it: if the two classes can be separated by a straight line, the perceptron will find one in a finite number of steps. Watch the accuracy climb to 100% and training stop.

The catch (and the whole future of deep learning)

A single neuron can only draw one straight line. The famous failure is XOR — no single line separates it. That limitation nearly killed neural nets in the 1970s.

The fix: stack neurons into layers, swap the step for a smooth activation, and train with gradient descent + backprop. That arc — from this one neuron to a transformer — is the rest of the series.

🌐 Train the neuron live (watch the boundary rotate): https://dev48v.infy.uk/dl/day1-perceptron.html

Day 1 of DeepLearningFromZero. From one neuron to transformers, built from scratch.