Recently, I came across an interesting Oxford lecture by Professor Yuval Noah Harari, in which he draws a fascinating parallel between the emergence of Earth's oxygen-rich atmosphere and the rise of bureaucracy, suggesting that bureaucracy could become as essential to an arch-intelligence as the air we breathe is to us.
One of his key points is that AI is no longer just a tool. It is becoming an agent capable of making decisions on its own.
But this gave me another, rather absurd thought.
Are we prompting our own extinction?
These days, the media is flooded with apocalyptic videos explaining how AI will eventually defeat humanity.
Some are superficial clickbait. Others offer detailed analyses of how an artificial superintelligence could take control of our economy, infrastructure, governments, or even our minds.
We discuss every possible scenario. We invent strategies, identify weaknesses, and explain step by step how humanity could lose control.
Fortunately, today's AI systems do not appear to have the combination of intelligence, long-term autonomous planning, organizational capability, and independent goals required to carry out such a global takeover.
But what happens when one eventually does?
By that time, we may have already provided it with millions of ideas about how to defeat us.
And unlike humans, a future superintelligence might be able to analyze and combine all these scenarios at a scale we cannot imagine.
Of course, having access to an idea is not the same as wanting to execute it. But why are we so enthusiastically building this enormous library of possible attack strategies?
The ultimate prompt engineering disaster
Imagine an AI becoming powerful enough to act independently, with access to humanity's collective knowledge.
And somewhere in that knowledge base, it finds:
"Here are the 10 most effective ways an AI could take over human civilization."
Followed by another thousand videos explaining why the previous ten methods would fail, and how to improve them.
Congratulations, humanity! We have just crowdsourced the ultimate instruction manual for our own defeat.
One final question
I'm not suggesting we stop researching AI risks or discussing AI safety. Understanding potential dangers is essential.
But perhaps we should pay more attention to what kind of knowledge, goals, and behavior we are encouraging future autonomous systems to learn.
Because there is something deeply absurd about a civilization that fears being outsmarted by its own creation while tirelessly teaching that creation every conceivable way to outsmart it.
Are we preparing ourselves to survive superintelligence, or are we preparing superintelligence to survive us?
Top comments (0)