DEV Community

Cover image for The History of AI, Part 6: The First AI Winter
Suraj Nandan
Suraj Nandan

Posted on

The History of AI, Part 6: The First AI Winter

The History of AI, Part 6 of 24. Part 5 toured the programs that made the 1960s feel golden. This chapter is about the hangover: the broken promises, the books and reports that cut funding, and how a field that had been promised the moon found itself out in the cold.

There is an old pattern in technology. First comes a breakthrough. Then comes a wave of excitement, with bold predictions, magazine covers, and generous funding. Then reality arrives, more slowly and more stubbornly than anyone said it would. And finally comes a crash, when the same people who were most excited become the most disillusioned.

Artificial intelligence has gone through that cycle more than once. The first big crash, which began in the late 1960s and bottomed out in the mid-1970s, is the one that gave the phenomenon its name: the AI winter.

Understanding why it happened is more than historical curiosity. Every few years, someone declares that AI has "finally" arrived, and someone else warns that we're in a bubble. Both are echoing arguments first made in this period. If you want a sense of what is hype and what is substance in the current wave, this chapter is the place to start.

The mood in 1966

Let's set the scene. By the mid-1960s, AI was probably the most generously funded corner of computer science. The US Department of Defense's ARPA was handing out millions of dollars to a handful of labs at MIT, Stanford, and Carnegie Mellon, often with very few strings attached. Its attitude, shaped by J.C.R. Licklider's philosophy of backing brilliant people rather than detailed project plans, was to fund the best minds and let them explore.

An IBM 704 at NASA's Langley laboratory, the kind of machine early AI programs ran on
An IBM 704 at NASA's Langley laboratory, the kind of machine early AI programs ran on. Public domain, via Wikimedia Commons.

Researchers, for their part, were making predictions that were hard to resist. A computer would be world chess champion in ten years. Machines would be doing any job a human could within twenty. Translation by machine was a few years from being solved.

Then, over about eight years, four separate blows landed. None of them alone would have been fatal. Together, they changed the field.

Blow 1: Machine translation fails to deliver

Machine translation (MT) was the first great application dream of the computer age, and it was fueled by Cold War anxiety. The US government desperately wanted to read Russian scientific papers faster than human translators could manage.

In January 1954, IBM and Georgetown University staged a public demonstration, the Georgetown-IBM experiment. A computer translated more than sixty Russian sentences into English. The audience was impressed. The press was thrilled. The researchers involved suggested that within three to five years, the problem of machine translation might be solved.

What the audience didn't see was how carefully the demonstration had been prepared. The system's vocabulary consisted of about 250 words, it used only six grammar rules, and the sentences had been selected in advance. It was a proof of concept, not a working translator.

Over the following decade, the US government spent roughly twenty million dollars on machine translation research. The results were disappointing. The core difficulty turned out to be one that the early researchers had underestimated: language is ambiguous, and resolving the ambiguity requires understanding the world. The standard illustration comes from the linguist Yehoshua Bar-Hillel, who pointed out that in the sentence "The box was in the pen," the word "pen" must mean a playpen, not a writing instrument, and that to know this a system needs real-world knowledge about the relative sizes of boxes and pens. No amount of word-for-word substitution could handle that.

In 1966, a US government committee called the Automatic Language Processing Advisory Committee, or ALPAC, chaired by the Bell Labs scientist John R. Pierce, delivered its verdict. The report concluded that machine translation was slower, less accurate, and about twice as expensive as human translation, and that there was no immediate prospect of useful machine translation. It recommended cutting back on MT funding and redirecting money toward basic research in computational linguistics and tools to help human translators.

Funding for machine translation in the United States largely dried up. (A popular, almost certainly apocryphal, story says early MT systems translated "The spirit is willing, but the flesh is weak" into Russian and back as "The vodka is good, but the meat is rotten." It's probably not true, but it captures the public's sense of the failure.) The ALPAC report was among the first major institutional signs that AI's promises were running ahead of its performance.

There is a bitter coda: MT did eventually work, but through a completely different approach. Statistical methods in the 1990s, then neural networks in the 2010s, finally produced useful machine translation, once there was vast amounts of data and computing power. We'll see that in Parts 10 and 19. The 1950s researchers weren't wrong about the goal, only about the timeline and the method.

Blow 2: The book that took aim at neural networks

If you remember the story from Part 5, Frank Rosenblatt's Perceptron was the darling of the connectionist camp. It learned from examples, it attracted media attention, and it attracted Navy funding. But it also attracted a rival.

Seymour Papert, co-author of Perceptrons
Seymour Papert, co-author of Perceptrons. Image: Matematicamente.it, CC BY-SA 3.0, via Wikimedia Commons.

In 1969, Marvin Minsky and Seymour Papert, both at MIT, published a book titled Perceptrons: An Introduction to Computational Geometry. It was short, mathematically rigorous, and devastating in its effect.

What the book actually proved

Minsky and Papert analyzed exactly what single-layer perceptrons can and can't compute. The headline result: a single-layer perceptron can only learn functions that are linearly separable, meaning those where you can draw a straight line (or a flat plane, in higher dimensions) that separates the "yes" cases from the "no" cases.

The most famous example is XOR, the exclusive-or function we've seen in the last two chapters:

Input A Input B Output
0 0 0
0 1 1
1 0 1
1 1 0

Plot those four points on a graph. The two "1" outputs sit on one diagonal; the two "0" outputs sit on the other. There is no way to draw a single straight line that separates them. Here's a quick brute-force demonstration of this fact:

import itertools
import numpy as np

X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])

def separable(y, grid=np.linspace(-2, 2, 41)):
    """Search a grid of weights/biases for a single perceptron that fits y."""
    for w1, w2, b in itertools.product(grid, repeat=3):
        pred = (X @ np.array([w1, w2]) + b > 0).astype(int)
        if (pred == y).all():
            return (w1, w2, b)
    return None

print("AND :", separable(np.array([0, 0, 0, 1])))   # finds a solution
print("OR  :", separable(np.array([0, 1, 1, 1])))   # finds a solution
print("XOR :", separable(np.array([0, 1, 1, 0])))   # None: no solution exists
Enter fullscreen mode Exit fullscreen mode

AND and OR have solutions; XOR doesn't, no matter how fine-grained your search. (The grid search isn't a proof, but the geometry is: no line splits those four points that way.)

The book went further than XOR. It showed that a single-layer perceptron couldn't compute things like parity (whether the number of active inputs is odd or even) or connectedness (whether a drawn shape is a single connected piece), tasks that seem natural for a "pattern recognizer." It was a rigorous demonstration that the perceptron was far more limited than its enthusiasts claimed.

What the book did not prove

This is a subtle point that's often lost. Minsky and Papert's results applied to single-layer perceptrons. It was well known (including to Rosenblatt) that multi-layer networks can compute XOR and far more. The real problem was that nobody knew how to train multi-layer networks reliably. And in the book's discussion, Minsky and Papert expressed a skeptical judgment that the extension to multilayer systems would likely be "sterile," meaning not fruitful.

That sentence did a lot of damage. Readers took away the message that neural networks were a dead end, even though the math didn't say that. The book was brilliant, persuasive, and, in hindsight, overly pessimistic about the broader approach.

The human drama

It's hard to separate the book from the rivalry. Minsky and Rosenblatt had a history: they had attended the same high school, the Bronx High School of Science, one year apart, and they were intellectual competitors for much of their careers. Many commentators have also suggested that the book was partly motivated by a desire to protect the symbolic approach's share of research funding from a flashy competitor. Minsky and Papert have always maintained that their aim was simply to clarify the mathematics.

What's clear is what happened afterward. Funding for neural network research dried up over the following years, and the work essentially went underground. A handful of people kept going, including Stephen Grossberg in the US, Teuvo Kohonen in Finland, Kunihiko Fukushima and Shun'ichi Amari in Japan, and a few others, working in relative obscurity for over a decade. Rosenblatt died in 1971, so he did not live to defend the field or see its return.

In 1988, an expanded edition of Perceptrons appeared, and the authors added an epilogue acknowledging the renewed interest in multilayer networks while defending their original analysis. It's an interesting read because it shows how long the argument lingered.

The deepest irony is this. The very method that would eventually revive neural networks, backpropagation, had been described mathematically by the early 1970s. A Finnish student named Seppo Linnainmaa published the general method of reverse-mode automatic differentiation in 1970, and Paul Werbos described applying it to neural networks in his 1974 Harvard PhD thesis. But almost nobody noticed. The field was too busy turning away. We'll return to this in Part 9.

Blow 3: The combinatorial explosion and the walls of computation

Even in the symbolic camp, researchers were discovering that their favorite techniques ran into mathematical walls.

Remember the idea from Part 4: intelligence as search, with heuristics to tame the combinatorial explosion. For the toy problems of the 1960s, this worked. But as soon as people tried to scale up, the number of possibilities exploded faster than any amount of computing power could handle.

Two developments made the problem precise:

  • In 1971, Stephen Cook proved that a certain logic problem (Boolean satisfiability) was NP-complete, and in 1972, Richard Karp showed that 21 other classic problems were too. These results gave mathematical weight to what practitioners were feeling: a large class of problems, including many that AI researchers cared about, are believed to require time that grows exponentially with their size, and there's no known way around it. (Whether such problems can be solved efficiently is the famous P versus NP question, still unsolved.)
  • Practical limitations bit hard too. The computers of the late 1960s and early 1970s had tiny memories by modern standards, often a few hundred kilobytes at the very most, and slow processors. Hans Moravec, then a young researcher, later estimated that machines of that era were many orders of magnitude too weak to match even the visual processing of an insect or small animal, let alone a human.

Here is a quick script to feel how brutal exponential growth is. Imagine a problem where each step offers just ten options, and a solution needs thirty steps:

branching, depth = 10, 30
total = branching ** depth
print(f"{total:.2e} possible paths")                     # 1.00e+30

# Even at a billion paths examined per second...
seconds = total / 1e9
years = seconds / (60 * 60 * 24 * 365)
print(f"{years:.2e} years to check them all")             # ~3.17e+13 years
print("age of the universe: ~1.38e+10 years")
Enter fullscreen mode Exit fullscreen mode

That's thousands of times longer than the age of the universe, for a problem with only ten options per step. Real-world problems have far larger branching factors. Heuristics could prune the search, but you can't prune what you can't anticipate, and in open-ended settings nobody knew which heuristics to use.

This was the practical meaning of Moravec's paradox from Part 4. The toy problems fit inside the available computing power. The real world did not.

There was also the thorny issue of common sense. John McCarthy and Patrick Hayes identified in 1969 the frame problem: when something changes in the world, how does a reasoning system know what doesn't change? If a robot moves a box, the color of the walls doesn't change, but a logical system has to be told that explicitly, for every action and every property, or it can't draw conclusions. The task of writing down enough common-sense knowledge for a machine to function turned out to be enormous, and it was nowhere near finished.

Blow 4: The outsiders strike

While all this was happening, two critics raised pointed objections from outside the field.

Hubert Dreyfus, the philosopher who criticized early AI
Hubert Dreyfus, the philosopher who criticized early AI. Image: Jörg Noller, CC BY-SA 3.0 de, via Wikimedia Commons.

Hubert Dreyfus

In 1965, a philosopher named Hubert Dreyfus at Berkeley wrote a RAND Corporation paper with the provocative title "Alchemy and Artificial Intelligence." In 1972 he expanded it into a book, What Computers Can't Do. Drawing on the phenomenologists Martin Heidegger and Maurice Merleau-Ponty, he argued that human intelligence depends on embodied, intuitive know-how, the kind of skilled coping you do when you ride a bike, that cannot be captured by explicit rules and symbols. He compared AI researchers to alchemists, claiming that early successes had been mistaken for steps toward a goal that was fundamentally unreachable.

The AI community reacted with contempt. Some of it was personal; Dreyfus was seen as a hostile outsider who didn't understand the technology. The most infamous moment came in 1967, when Dreyfus, who had claimed that no chess program could play even amateur-level chess, played against MIT's Mac Hack VI chess program, written by Richard Greenblatt. He lost. The AI community was delighted. A memo circulated at MIT crowing about it, and the incident became a lasting symbol of the hostility between the two camps.

But time has been kinder to Dreyfus than his contemporaries were. Many of his specific criticisms, particularly about how brittle rule-based systems are and how much human skill depends on tacit, embodied knowledge, were later echoed by insiders. The field's eventual shift toward learning from data is, in a sense, an admission that hand-written rules were the wrong approach for much of intelligence. (Dreyfus was too strong in claiming no machine could ever do certain things; but he was right to identify real problems.)

Weizenbaum

You'll remember Joseph Weizenbaum from Part 5, the creator of ELIZA, who was disturbed by how readily people attributed understanding to his program. His 1976 critique, which we mentioned there, added to the sense that some prominent insiders were questioning the field's claims and ethics.

The Mansfield Amendment and the squeeze on funding

Meanwhile, the money was becoming harder to get. In 1969, the US Congress passed the Mansfield Amendment, named for Senator Mike Mansfield, which required that ARPA fund only research with a direct and apparent relationship to a specific military function. The Vietnam War had made Americans increasingly skeptical of the military funding university research, and Congress wanted accountability.

For AI, this was a big shift. The freewheeling "fund the best people" model gave way to a requirement that projects deliver concrete, military-relevant results on a schedule. DARPA program managers began demanding demonstrable progress. When a major program in speech understanding at Carnegie Mellon and elsewhere failed to meet the more ambitious goals DARPA had hoped for in the early 1970s (even though some of its systems, such as CMU's Harpy, did achieve impressive results on limited vocabularies), patience wore thin, and DARPA cut back its support for some undirected AI research.

The golden years of unconditional funding were over. AI labs in the US weren't wiped out, but they now had to justify themselves.

The Lighthill Report: the British verdict

The most famous and most devastating blow came from the United Kingdom.

By 1971, the British Science Research Council was worried. It was funding a significant amount of AI research, but there was disagreement among scientists about how well it was going. The Council commissioned a review from Sir James Lighthill, the Lucasian Professor of Applied Mathematics at Cambridge (the same chair once held by Isaac Newton and later by Stephen Hawking), and a distinguished fluid dynamicist who was not himself an AI researcher.

Lighthill delivered his report, titled Artificial Intelligence: A General Survey, in 1972, and it was published in 1973. His assessment was withering.

The three categories

Lighthill divided AI work into three categories, labeled A, B, and C:

  • A: Advanced Automation. Practical applications such as industrial robots and machine translation. Lighthill concluded that the field had failed to achieve its "grandiose objectives" here, and that the results had been disappointing.
  • B: Computer-based Central Nervous System research. Using computers to study the brain and psychology. He considered this worthwhile.
  • C: The Bridge between A and B. The attempts to build robots and general-purpose systems that link practical engineering to the study of the mind. Lighthill was skeptical that this bridge category had any real prospects of success.

His central criticism was the combinatorial explosion: AI programs worked on toy problems, but he argued that the methods "would not scale up" to real-world problems, because of the exponential growth of possibilities. He saw little evidence that the work had produced results with real-world impact commensurate with its cost.

The BBC debate

The government arranged a televised debate on the BBC in June 1973, titled "The General Purpose Robot Is a Mirage." Lighthill argued his case against three leading defenders of AI: Donald Michie of Edinburgh, Richard Gregory of Bristol, and, joining by satellite link from Stanford, John McCarthy. The recording survives, and it's fascinating to watch: Lighthill, sharp and combative, repeatedly pressing the defenders to name a concrete real-world success, while the AI researchers pointed to potential and long-term promise. Most observers judged that the defenders failed to land a decisive reply.

The consequences

The effect on British AI was devastating. The Science Research Council sharply cut AI funding. Michie's group in Edinburgh, which had been building a pioneering robot called Freddy, was dismantled. A generation of British researchers either left for the US, switched fields, or saw their careers stall. British AI research would not fully recover until the early 1980s, when a government initiative called the Alvey Programme was launched, largely in response to Japan's Fifth Generation Computer project, which we'll see in Part 8.

The Lighthill Report remains controversial. Some historians argue it was unfairly harsh and misread the field by treating it as a single enterprise. Others say it correctly diagnosed a real overpromising problem. In truth, it was both: it was right about the limits of the then-current methods and about the gap between claims and results, and wrong in its pessimism about long-term prospects.

What was an "AI winter," exactly?

Notice that the first winter was a gradual freeze and a regional one, more than a single dramatic crash. There was no day when AI "died." Instead:

  • Funding agencies became more skeptical and more demanding.
  • Some research areas (machine translation, perceptrons, certain robotics and speech programs) lost support entirely.
  • Researchers rebranded their work, avoiding the phrase "artificial intelligence" in grant applications, and used terms like informatics, machine learning, knowledge-based systems, or cognitive systems instead. (This pattern of avoiding the label would recur in later winters.)
  • Hiring slowed, and students steered toward other fields.

The metaphor of "winter" itself was coined later. At the 1984 annual meeting of the American Association for Artificial Intelligence, Marvin Minsky and Roger Schank, who had both lived through the 1970s downturn, warned the business community that enthusiasm for AI had gotten out of control and that a "nuclear winter"-style collapse was coming. We'll cover that second winter in Part 8.

What survived and what was born in the cold

Here's a point that rarely gets mentioned: the winter was not a period of no progress. A lot of important work happened in the 1970s, partly because funding pressure forced researchers to be more practical.

  • Logic programming. In 1972, Alain Colmerauer and Philippe Roussel in Marseille, with Robert Kowalski's theoretical input in Edinburgh, created Prolog, a language in which you describe facts and rules and let the system deduce answers. It became a staple of European and Japanese AI.
  • Knowledge-based systems. A new idea began to emerge: instead of searching for general-purpose reasoning methods, capture the specific expert knowledge of a narrow domain. Edward Feigenbaum's team at Stanford built DENDRAL (to identify chemical structures from mass spectrometer data) and MYCIN (to diagnose bacterial blood infections and recommend antibiotics), systems that performed at or near the level of human experts in tiny domains. This approach, the "expert system," would drive the next boom. It is the subject of Part 7.
  • Knowledge representation. Researchers developed frames (Minsky, 1974), semantic networks, and scripts (Schank and Abelson, 1977) for organizing real-world knowledge.
  • Underground neural networks. A few dedicated researchers kept the flame alive. Fukushima's Neocognitron (1980), an ancestor of convolutional neural networks, and Kohonen's self-organizing maps came from this quiet period.
  • Foundational theory. The NP-completeness results gave computer science a clearer map of what was easy and what was hard.

So the winter was less a collapse of ideas than a collapse of expectations, and a forced pivot from grand, general ambitions to focused, practical systems. In hindsight, it may have helped the field mature.

Why do AI winters happen?

Historians of AI have noted a recurring pattern, and it's worth stating it explicitly, because it helps interpret today's headlines.

  1. A real breakthrough. Something genuinely new works, often on a narrow problem.
  2. Extrapolation. Excited researchers, journalists, and investors assume the breakthrough will generalize, and make confident predictions about timelines.
  3. Funding surge. Money pours in, expectations rise, and organizations make commitments that depend on the predictions.
  4. Hitting the wall. The methods run into limits (data, compute, brittleness, the messiness of reality) that weren't visible in the early demos.
  5. Disillusionment. Predictions fail. Funders feel misled. Critics gain a hearing. Money retreats, and even good work is tarred by association.
  6. Quiet progress. Researchers who remain keep working, often under different names, and sometimes lay the foundation for the next wave.

The Gartner "hype cycle" is a commercial description of the same thing, with its "peak of inflated expectations" and "trough of disillusionment." What the history of AI adds is the observation that the trough is rarely the end. In the case of neural networks, machine translation, and speech recognition, the early goals were eventually achieved, just not by the methods or on the timeline originally promised.

Lessons worth carrying forward

  • Beware the demo. A system that works in a curated setting says little about how it will do in the wild. (Remember the Mechanical Turk and SHRDLU.)
  • Distinguish the goal from the method. Machine translation was a sound goal; rule-based translation was a limited method.
  • Scale changes everything. Some approaches that fail at small scale succeed when given vastly more data and computing power, and others don't. It's hard to know in advance which is which.
  • Be skeptical of both hype and doom. Lighthill was right about the problems and wrong about the future.
  • Critics are sometimes right. Dreyfus and Weizenbaum were dismissed in their time, but many of their concerns proved prescient.

Timeline of the first winter

  • 1954: Georgetown-IBM translation demonstration.
  • 1965: Dreyfus's "Alchemy and Artificial Intelligence."
  • 1966: ALPAC report cuts machine translation funding.
  • 1969: Perceptrons published; the Mansfield Amendment passes.
  • 1971 to 1972: Cook and Karp establish NP-completeness; Dreyfus's What Computers Can't Do.
  • 1973: The Lighthill Report; the BBC debate in June.
  • 1974: DARPA reduces undirected AI funding; UK research contracts sharply.
  • Late 1970s: Expert systems begin to take shape (Part 7), and the field slowly reorganizes.

Three questions to ask when someone says "AI winter"

The phrase gets thrown around loosely, so it helps to ask:

  1. What exactly froze? Funding, hiring, public interest, or the science itself? In both winters, funding and attention fell while genuine technical progress continued in less visible places.
  2. Who made the promises? Winters follow the gap between what was promised to funders and what was delivered. When researchers and sponsors set modest expectations, nothing much happens, even if the technology is the same.
  3. What was the thing that worked? Each winter left behind durable pieces: search algorithms, planning, logic programming, probabilistic methods. A winter is a pruning, not a death.

Quick glossary

  • Linearly separable: classes that can be divided by a straight line (or flat plane).
  • NP-complete: a class of problems for which no fast exact algorithm is known; solving one efficiently would solve them all.
  • Frame problem: the difficulty of specifying what does not change when something happens.
  • Brittleness: failing badly when conditions move slightly outside what a system was built for.
  • Hype cycle: the typical pattern of inflated expectations followed by disillusionment, then gradual maturity.

Key takeaways

  • The first AI winter (late 1960s to the mid-1970s) was a gradual loss of funding and confidence, not a single event.
  • The ALPAC report (1966) judged machine translation a failure and cut US funding.
  • Minsky and Papert's Perceptrons (1969) showed the limits of single-layer networks (like the XOR problem) and chilled research on neural networks for over a decade, even though multi-layer networks weren't ruled out.
  • Combinatorial explosion and NP-completeness showed that many AI problems couldn't be solved by brute force, and that toy-world successes didn't scale.
  • The Mansfield Amendment (1969) pushed DARPA toward mission-oriented research, and the Lighthill Report (1973) triggered deep cuts in British AI.
  • The era also planted seeds: Prolog, expert systems, knowledge representation, and the underground neural network work that would later revive the field.

Try it yourself

  1. Escape the XOR trap. Using the grid search above as a starting point, build a two-layer network by hand that computes XOR. Verify it with the truth table. Then try parity of three inputs, and notice how the network has to get bigger.
  2. Estimate your own explosion. Choose a real problem (such as scheduling 10 meetings in 5 rooms) and estimate the number of possible arrangements. How long would exhaustive search take? What heuristic would prune it?
  3. Hype audit. Pick an AI claim from today's news and apply the six-stage pattern above. Which stage does it sound like? What would a Lighthill-style skeptic ask? What would a Michie-style defender answer?

Next up

In Part 7, the field comes back from the cold with a new idea: stop trying to build general intelligence, and instead capture the knowledge of human experts. Systems like MYCIN, XCON, and a flood of commercial products would launch the first multi-billion-dollar AI industry, and put AI on the cover of business magazines in the 1980s. It's the story of how a field reinvented itself, and then repeated its old mistakes.

If you've lived through a tech hype cycle yourself, tell me about it in the comments, and follow the series so you don't miss Part 7.

See you in Part 7.


☕ If this series helped you, you can support my writing on Ko-fi.

Support me on Ko-fi

Top comments (0)