The year is 1958. Electronic computers are found in some rooms, punch cards clatter through readers, and the idea of a “thinking machine” belongs more to science fiction than to engineering. One day, Frank Rosenblatt publishes a paper that changes the direction of the field of thinking machines : " The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain."
At this moment, most machines are built to follow explicit instructions. If you want a device to behave in a certain way, you must design every rule in advance. There is no standard notion of a system that can adjust its own internal parameters based on " experience ". Rosenblatt’s work proposes something different. It proposes a machine whose response to an input can change as it encounters more examples, without requiring every rule to be manually specified.
He calls this machine the perceptron. To explain its organization, Rosenblatt considers a version that responds to visual patterns. An image first falls on a simple artificial retina, made of sensory points that react to the incoming stimulus. These sensory points send signals to units in the projection area. Some incoming signals push a unit toward becoming active, while others push against it. Rosenblatt calls these excitatory and inhibitory connections. We can think of the unit as adding these positive and negative influences together. If the final amount is large enough to cross a certain threshold, the unit fires. Otherwise, it remains inactive. The active units pass signals to another group of units called the association area, and finally toward one of the several possible response units.
Figure 1 (taken from official paper) shows this complete path clearly.
Looking at the figure from left to right, the retina receives the input.
Its signals travel through the projection area A1 to the association area A2, and from there toward the possible responses R1,R2,…,Rn. Please note that the projection area is not much worth for our further discussion, as it is specific to the example mentioned above.
One thing to note here is that Rosenblatt explicitly labels some connections as random. This does not mean that the machine produces random answers. It means that he does not carefully choose beforehand which unit in one area should connect to which unit in the next. Different inputs will therefore activate different, partly overlapping groups of association units. Learning happens later by changing how effectively the active units can influence the responses.
To understand this, imagine showing the perceptron the same kind of pattern several times. Each time the pattern appears, a particular group of association units becomes active and one of the response units eventually responds. If we now reinforce that response - which essentially means telling the system that this is the response we want, something inside the perceptron changes. Rosenblatt gives each association unit a numerical value , and reinforcement changes the values of the units that were active. So when a similar pattern appears again and activates many of those same units, they no longer influence the response exactly as they did before. The response that was reinforced has now become more likely.
Now notice what has happened. Nobody went inside the perceptron and added a rule saying, “When you see this pattern again, choose this response.” The only thing that changed was the numerical values of some association units. Yet when the pattern appeared again, the machine was more likely to respond differently because of those changed values. This is the key learning of this paper - an experience changes something inside the machine, and that change affects what happens during the next experience.
But something even more interesting can happen. The next input does not have to be exactly the same as one seen before. Suppose a new pattern is similar enough that it activates some of the same association units whose values were changed during learning. Those changed units are still there. They can therefore influence the response even though this exact pattern has never appeared before. So now Rosenblatt can ask a much more important question.
Can the perceptron learn from the examples it has experienced and use those changes to respond correctly to a new example?
Rosenblatt studies exactly this distinction mathematically. He considers two values Pr and Pg.
Pr = probability of correctly responding to something already learned
Pg =probability of correctly responding to a new member of the same class
If the perceptron simply remembers its previous examples, we might expect Pr to improve while Pg remained poor. Rosenblatt finds that, as the system learns from more examples, both probabilities can approach the same limit :
Pr⟶ P∗ and Pg ⟶ P∗
where P∗ represents the limiting probability of a correct response.
In other words, the probability of correctly handling a new member of the class can eventually approach the same level as the probability of correctly handling examples seen during learning. In the paper, Rosenblatt further argues that this limiting probability
P∗ can be brought closer to 1 by increasing the number of association units in the system.
This is called generalization. If the perceptron learns from several similar members of a class, whatever it has learned can help it respond correctly to another member it has never seen before. In the future, the distinction between performing well on examples seen during learning and performing well on new examples from the same class is an idea that will remain central throughout the rest of machine learning.
However, Rosenblatt himself also points to a boundary. The perceptron can learn patterns and associations, but the same becomes difficult in problems involving relationships. He gives examples such as identifying “the object left of the square” or remembering “the pattern that appeared before the circle.” For such problems, Rosenblatt concludes that “some system, more advanced in principle than the perceptron” would be required.
This paper essentially teaches us something that now sounds obvious, but was anything but obvious in 1958.
A machine does not have to be given a separate rule for every situation it may encounter. Its internal numerical state can change through experience, and what changes during learning can help it respond even to inputs it has never seen before.
The perceptron could not learn every kind of relationship, but it had established the idea we needed first i.e a machine could learn.
Stay tuned for our next paper in the sequence that carries the AI journey forward !
Rosenblatt's Paper Link : https://web.engr.oregonstate.edu/~huanlian/teaching/ML/2020spring/extra/rosenblatt-1958.pdf
Top comments (0)