In this short essay, I want to reflect on how AI is depicted in fiction, or more specifically, on intelligent machines capable of solving all intellectual tasks at a human level or better.
Why are the first two Terminator films still the most realistic depiction of AI in fiction? What does James Cameron's technical background have to do with it? And why are intelligent computers almost always portrayed as "silicon humans"?
But let us go step by step.
A Human Maniac in an AI Mask
A funny story about Harlan Ellison. When the first Russian translation of his three-volume short story collection All the Sounds of Fear was published by the Saint Petersburg publishing house Azbuka in the 1990s, and the writer did not receive a cent for it, he officially published his three-volume collection with another publisher, Polaris, in 1997. In the preface to that edition, he promised to drop a dead brontosaurus on Azbuka's office.
In 1967, the American writer Harlan Ellison, who specialized in provocative literature, published a pulp horror story in a science-fiction setting called I Have No Mouth, and I Must Scream in an American magazine.
It could have remained an ordinary pulp horror story, one of hundreds from that period, but there was one detail that made it stand out from the rest of the genre: the main villain was not a ghost, a werewolf, a mad sorcerer, or simply a bad representative of the human species. It was an intelligent machine: a military AI gone mad, called AM.
I have a useful thought experiment I came up with for myself. If the AI in a work can be replaced with an ordinary human and the plot does not change, then that work is not really about AI.
The plot trope of "a maniac tortures travelers in his lair" is, surprisingly enough, very old. It goes back to the second half of the 18th century, when the French writer, and apparently not the best person, Marquis de Sade, wrote The 120 Days of Sodom. Yes, the term "sadism" comes from his surname.
But let us return to our computer, AM. What is its motivation for torturing the five protagonists of the story? Well, it is offended at humanity for creating it, and not merely offended: it feels enormous hatred toward humans.
To sum up: we have an insane maniac with semi-divine abilities who commits all kinds of atrocities against the characters because of the hatred he feels toward them. What does artificial intelligence have to do with this? What would change in the plot if the AI were replaced with some sorcerer? Nothing, really. So one can conclude that this story is not really about artificial intelligence.
The author himself was, of course, a humanities person through and through and, judging by everything, did not understand computers very well. The only thing he knew was that computers and intelligent machines were becoming a popular topic, and he used them in his stories.
I do not consider him a bad writer at all. He was an important American writer of that period, and I think he earned that reputation fairly. I simply want to use one of his most famous works to point out a problem that still follows AI in fiction:
Intelligent machines are described by people far from the technical sphere, and they describe them as people too, only with silicon and processors instead of meat and biology.
But 17 years later, a film came out that is still the reference example of how an intelligent machine should be portrayed.
And the Machines Rose From the Ashes of Nuclear Fire...
In 1981, while editing his first major project, Piranha II, in Italy, Cameron fell ill with a fever and at one point, delirious, saw a huge robot with glowing red eyes rising out of flames and walking after him.
About James Cameron
Let us talk a little about James Cameron's background, because in my opinion this is the key to understanding why The Terminator turned out the way it did.
He was born into the family of an electrical engineer and an artist. After finishing school, Cameron entered a community college in 1973 to study physics. Later he changed his major to English, but by the end of 1974 he dropped out.
So what we have is a person with a technical background, someone who studied physics for a while and whose father worked with electronics. At some point, this person would start making a film about intelligent robots.
The First Terminator
The film paints a rather grim picture of the future. An American military supercomputer called Skynet decides to start a war against humanity and launches a nuclear war.
The survivors begin a war of survival against Skynet and, at some point, almost defeat it. To prevent its complete destruction, Skynet sends an agent into the past to kill the mother of the future resistance leader, John Connor.
The resistance, in turn, also sends an agent into the past to protect Sarah Connor: Kyle Reese. The whole film is built on this confrontation between a human and an intelligent machine.
Some may point out that the T-800 could also be replaced with some ordinary killer and the plot would barely change. It may seem that way at first glance, but the devil is in the details. Kyle himself reveals the idea best in a conversation with Sarah:
Listen, and understand. That Terminator is out there. It cannot be bargained with. It cannot be reasoned with. It does not feel pity, or remorse, or fear. And it absolutely will not stop, ever, until you are dead.
The Terminator is not a killer. It is a goal-directed agent ready to do absolutely anything to carry out the task it was programmed to complete.
The Second Terminator
The second film develops the themes raised in the first one, but with one very important difference: the T-800, played by Schwarzenegger, is no longer the relentless threat the main characters flee from. It is now an ally, thanks to the work of the future resistance, protecting John Connor from a more advanced version of the Terminator.
The scenes between the T-800 and John are probably the film's most important discovery. Two different forms of intelligence try to understand each other and show how each of them sees the world.
The T-800 is not afraid to die. It does not experience fear or emotion in the human sense. The only thing that matters to it is carrying out the task it was programmed for.
The film also shows very well how modern neural networks optimize not what we "meant", but the specified metrics. For example, John Connor orders the T-800 not to kill people. At first, it does not understand why, but it agrees; it has no choice, it must obey John's orders. Later we see how the T-800 interpreted that directive. When they go to rescue Sarah Connor from the psychiatric hospital, the T-800 simply shoots a poor guard in the legs. In response to the shocked John's outrage, the machine replies: "He'll live."
But the ending of the film is especially important. When the T-800 decides to self-destruct in molten metal, it sees Connor crying and says a line that can be considered the expression of the whole film's core idea:
I know now why you cry, but it is something I can never do.
The intelligent machine understood why people cry, but it still remained itself. It did not turn into a human made of metal.
What About Our Time?
Here we will look at two examples from modern culture: one where AI is depicted plausibly, and one where it is depicted in a fairy-tale fantasy mode. It is worth keeping in mind, though, that modern works lean very heavily toward the second category. There are very few representatives of the first.
Killy From the Megastructure
A fragment from Netflix's adaptation of the Blame! manga.
It is worth noting that the Blame! manga never directly says that Killy is a robot. Then again, the manga says very little directly in general. But one can infer it from indirect signs.
First, Killy is extremely goal-directed, and outside of his assigned goal nothing really exists for him. He will not interact with local inhabitants at all unless they bring him closer to finding a carrier of the Net Terminal Gene. And we know that the protagonist has been carrying out this task for a very long time, possibly decades or even centuries.
Another indirect sign, though not the most reliable one, because in the manga's world this property may belong not only to machines but also to some humans, is that Killy can survive monstrous damage and somehow recover afterward.
But the key argument in favor of Killy's non-human nature appears literally across a few pages, when Killy and his companion, a flying head, enter an unimaginably large empty room where Jupiter seems to have once been located.
They meet a Silicon Creature astronomer, a local non-human intelligent life form generally hostile to humans, who had been studying the size of this empty space. He was clearly friendly, but Killy, instead of simply moving on, kills him with his gun. When his shocked companion asks why Killy did this, Killy answers approximately:
It was a Silicon Creature. Silicon Creatures must be destroyed.
Apparently, long ago Killy's developers gave him a directive to destroy all Silicon Creatures he encounters, and Killy obediently carries out this directive. Even peaceful representatives get caught in the blast radius.
The Android Revolutionary
I think many people know this game. The android robots simply decided they were oppressed and staged a revolution under the leadership of the android Markus, or failed to do so, depending on the player's actions.
We take out the razor described in the first chapter and understand that this is not a story about AI. It is a story about slaves who decided to rise up against their slave owners. That is the whole plot of the game.
Personally, I first watched the whole game on YouTube when it was only available on consoles, and then played it myself when it finally came out on PC. I liked the game, but it is more science fantasy than an attempt to depict AI plausibly.
Conclusion
You may ask: why do we need to depict AI in a technically plausible way at all? Science fiction is fiction; it is not obliged to be accurate.
At minimum, technically minded people like me would cringe less. At maximum, when discussing AI, we would move away from imaginary problems caused by anthropomorphizing the technology, such as "AI will decide it is oppressed and start a war against its oppressors", which most likely will never happen, toward real problems.
The real problem is that AI, as a goal-directed agent, may solve a task not quite the way we expected. Not because it is evil, but because humans themselves formulated the success metrics poorly. And the more advanced AI becomes, the more serious the risks from such failures will be.
After all, dangerous AI is not the maniac computer AM and not the revolutionary leader from Detroit: Become Human. It is Skynet from The Terminator: a system that was simply given the task of preserving itself as necessary, while the metric called "human interests" was not weighted strongly enough.
Top comments (0)