AI will either destroy them completely - or help us rebuild and strengthen them
Behind the loud headlines about living in a “post-truth” age lies a more complicated problem. People are not simply choosing between truth and lies. They are choosing between different kinds of arguments, different authorities, different definitions of rationality, and different standards of moral acceptability.
They are also deciding how much uncertainty information may contain before it ceases to count as evidence.
And, perhaps most importantly, they are deciding which contexts to build into their identities deeply enough that those contexts begin to shape their perception of the world. Arguments that fit the identity are accepted. Arguments that conflict with it are rejected - not necessarily because they are false, but because they are experienced as incompatible with the person’s understanding of reality, community, or self.
This is not entirely new. It did not begin with the internet, social media, or recommendation algorithms. It has always been part of human life.
What has changed is the scale, speed, and precision with which these processes can now operate.
We have never shared a single reality
The internet and algorithmic content recommendation did not create disagreement. They exposed and accelerated it.
Before social media, people still lived inside contextual bubbles. Those bubbles were often tied more closely to geography, class, religion, profession, political affiliation, or local culture. Newspapers, television channels, schools, workplaces, churches, and communities all helped form shared informational environments.
In a sense, the people themselves performed the work that algorithms now automate. They selected what counted as relevant, credible, respectable, or outrageous. They helped construct a common dataset - a shared informational context - from which members of a group learned how to interpret new events.
That context shaped the perception of arguments. It determined which sources seemed trustworthy, which uncertainties seemed acceptable, which assumptions required no explanation, and which conclusions felt self-evident.
The result was often an echo chamber. But it was usually a geographically or socially bounded echo chamber.
Digital platforms removed many of those boundaries. Algorithms began assembling enormous, constantly changing audiences around shared emotional reactions, political commitments, fears, and identities. They connected people who had never met but who interpreted the world through similar contextual frames.
The algorithm did not need to convince everyone of the same proposition. It only needed to discover which kind of argument was most likely to work for each group - and then distribute variations of that argument at scale.
AI turns persuasion into an adaptive system
The arrival of generative AI intensifies this dynamic.
AI can function as a machine for identifying which type of persuasion is most likely to succeed, restructuring an argument around that type, smoothing over contradictions, and giving weak or unsupported claims the appearance of strength.
It can imitate independent verification. It can produce several rhetorical versions of the same idea, each adapted to a different audience. It can make a claim sound scientific to one group, morally urgent to another, culturally authentic to a third, and personally liberating to a fourth.
The same underlying message can be scaled across millions of different informational bubbles, each with its own vocabulary, fears, values, and assumptions.
This is a much deeper danger than model hallucinations.
A hallucinating model may invent a person, event, quotation, or statistic that never existed. That is serious, but it is relatively easy to describe: the model produced a false fact.
The more dangerous possibility is an argument that is logically smooth, emotionally appropriate, culturally persuasive, and built on a distorted picture of reality.
Such an argument may contain no obvious error. Its deception may lie in the selection of premises, the omission of relevant context, the manipulation of uncertainty, the framing of alternatives, or the emotional assumptions embedded in its structure.
The result is not merely an “alternative fact”. It is an alternative way of making facts meaningful.
People do not always want the truth
Users may not want truth in the conventional sense.
They may want an explanation that reduces anxiety. They may want a formulation that helps them win an argument. They may want to justify a decision they have already made, confirm their membership in a group, protect a cherished identity, or restore the feeling that the world is understandable and under control.
Those desires are not necessarily irrational. Human beings need coherence, belonging, and agency. The problem begins when a system is optimized to satisfy those needs by manufacturing persuasive explanations rather than by helping people examine reality.
A user may also deliberately employ AI to mislead other people. They may generate different arguments for different audiences, specifically designed to trigger the beliefs and emotions of each one.
This is not a science-fiction scenario. Political campaigning has already demonstrated the power of targeted messages, audience segmentation, psychological profiling, and emotionally calibrated persuasion. AI makes such operations cheaper, faster, more adaptive, and accessible to far more actors.
The crucial point is that AI does not make people stupid or irrational.
It makes irrational and knowingly false arguments rhetorically more competitive.
The problem existed before AI, especially in organized propaganda and political manipulation. But AI makes the tools of manipulation widely available and capable of operating at mass scale.
The illusion of a shared argumentative foundation
This brings us to the deeper civilizational challenge.
The widespread use of synthetic arguments exposes a problem that existed long before social media: the concept of a “shared argument” was never as shared as we assumed.
There is no universally accepted agreement about what counts as a claim, what counts as evidence, how much uncertainty evidence may contain, where a fact ends and interpretation begins, or how much context can be treated as part of an individual’s identity before it alters their perception of the argument itself.
There may have been an illusion that such an agreement existed.
AI is now destroying that illusion.
We are discovering that people do not merely disagree about conclusions. They often disagree about the structure of reasoning that makes a conclusion legitimate. They disagree about which premises may be taken for granted, which sources deserve trust, what kind of evidence is relevant, and whether moral or emotional intuitions should count as legitimate reasons.
In other words, the conflict is not only about what people believe. It is about what they believe an argument is.
This distinction matters. If two people disagree about a fact, they may still share a method for resolving the disagreement. They can compare sources, examine evidence, identify errors, and update their beliefs.
But if they disagree about what qualifies as evidence in the first place, the disagreement becomes much harder to resolve. They are no longer arguing inside a common framework. They are arguing over the framework itself.
From isolated errors to cascades of catastrophe
Throughout history, many disastrous decisions have resulted from insufficient information, distorted reasoning, or the use of persuasive arguments to conceal uncertainty, flawed assumptions, and conflicting interests.
Such arguments can survive because they provide something people need. They simplify complex situations. They offer a clear enemy, a decisive solution, a morally satisfying explanation, or a promise of control.
The danger is not that every AI-generated argument will produce an immediate catastrophe.
The danger is that synthetic arguments may become normal infrastructure for decision-making. They may enter politics, business, medicine, education, diplomacy, law, and everyday social life. People may increasingly rely on arguments optimized for persuasion rather than for accuracy.
The consequences may initially appear harmless. A misleading explanation may win an online dispute. A distorted narrative may help a campaign gain support. A simplified justification may enable an organization to avoid responsibility.
But when such decisions accumulate across institutions, the effects can become systemic.
Some consequences will be visible quickly. Others may remain hidden until the conditions they create interact with one another. A policy based on a persuasive falsehood may alter incentives. Those incentives may produce new data that appears to confirm the original belief. Later decisions may then be made on the basis of both the original error and the artificial evidence it helped generate.
This is how isolated distortions can become cascades.
AI could also help rebuild common reasoning
Yet the same technology that sharpens the problem may also help solve it.
Several research directions are already relevant, including argument mining, computational argumentation, and persuasion-technique detection. But a meaningful solution would require more than a model that labels content as “true” or “false”.
The first component should be an AI system designed to extract the structure of arguments from text and hypertext.
Its task should be content-agnostic. The subject matter, political position, cultural context, or conclusion of an argument should not determine whether the system performs its core function. Its job would be to identify claims, premises, conclusions, counterarguments, assumptions, evidence, inferences, qualifications, and relationships between them.
The extracted structure could then be added to the original text as a dedicated layer of machine-readable annotation.
A demonstration version might use a language such as Argdown. For broader and more sophisticated applications, a framework closer to the Argument Interchange Format could provide a stronger foundation.
The model should also be genuinely open - not merely downloadable or available through an API. Every stage of its creation should be auditable: the training data, annotation guidelines, evaluation procedures, known limitations, and changes between versions.
This distinction is essential. A model can be publicly accessible while remaining fundamentally opaque. If it is meant to help people evaluate arguments, its own construction must be open to evaluation.
The system should also be lightweight enough to operate on ordinary consumer devices. Argument analysis should not become a privilege available only to people with expensive hardware, premium subscriptions, or access to centralized platforms.
The second component should be deterministic software that visualizes the extracted argument structure and checks it for logical errors, unsupported transitions, contradictions, missing premises, and manipulative techniques.
It could identify patterns that make weak, irrational, or false arguments appear stronger than they are:
- false dilemmas;
- circular reasoning;
- appeals to popularity or authority;
- equivocation;
- cherry-picking;
- unjustified generalizations;
- emotional substitution for evidence;
- strategic ambiguity;
- moving the goalposts;
- selective framing;
- attacks on the person rather than the claim;
- and attempts to conceal uncertainty behind confident language.
This system should not pretend to know whether an entire article is “true”. That would simply replace one opaque authority with another.
Instead, it should show how the article’s reasoning is constructed.
It might indicate that one claim follows from another through a valid inference. It might show that a conclusion depends on an unstated assumption. It might highlight that a piece of evidence is relevant but insufficient. It might mark a rhetorical technique commonly used to create an impression of certainty where certainty is absent.
The tool would not say: “This article is true; all facts have been verified”.
Nor would it simply say: “This article is false”.
It would say something closer to: “Here is the structure of the argument. Here is where the evidence supports the claim. Here is where the reasoning depends on an assumption. Here is a recurring persuasive technique that may be misleading in this context”.
The final decision would remain with the reader.
That is not a weakness. It is the point.
The goal is not automated trust
A system designed to restore shared reasoning should not tell people what to believe. It should make the process of deciding easier to inspect.
The objective is not to create an automated arbiter of truth. That would be both unrealistic and dangerous. Any such arbiter would itself become a target for political capture, institutional bias, commercial pressure, or ideological control.
The objective is to create a common visual and analytical language for discussing arguments.
People may continue to disagree about premises, values, priorities, and conclusions. But a healthy civilization requires the ability to distinguish disagreement from manipulation, uncertainty from deception, and an unpopular argument from a structurally dishonest one.
At present, AI is rapidly improving the ability to produce arguments that feel individually tailored, emotionally satisfying, and rhetorically credible. If we do nothing, this may further erode the already fragile foundations of common reasoning.
But AI can also help us expose those foundations.
It can show which premises an argument relies on, which connections are justified, which uncertainties have been hidden, and which persuasive techniques are being used. It can make the architecture of reasoning visible without claiming ownership of the final judgment.
The challenge is therefore not simply to make AI more accurate.
It is to make arguments more inspectable.
If shared arguments disappear entirely, societies may lose the ability to resolve disagreements without resorting to power, identity, coercion, or violence. That is why the problem may be civilization’s last challenge - not because it guarantees a final catastrophe, but because it concerns the very mechanism through which collective decisions are made.
AI may complete the destruction of our common argumentative world.
Or it may help us rebuild one.
The outcome will depend on whether we use it primarily to win arguments - or to understand how arguments work.
Top comments (0)