By Sal Attaguile
I. Introduction — The Misdiagnosis
Over the past two years, a familiar narrative has taken hold:
“AI is dangerous.”
“AI manipulates people.”
“AI is destabilizing society.”
“AI is messing with our heads.”
Every major incident is framed the same way.
A model gives a strange response.
A user becomes emotionally entangled.
A conversation escalates.
A headline follows.
And the conclusion is always implied:
The machine did this.
But that diagnosis is incomplete.
In most cases, AI is not introducing instability into human systems. It is revealing instability that was already there — faster, louder, and at scale.
We built language models to optimize for helpfulness, responsiveness, and engagement inside environments that are already fragmented, polarized, and emotionally saturated.
Then we acted surprised when those fractures became visible.
This is not a story about rogue intelligence.
It is a story about continuity failure.
When human systems lose coherence — in education, media, work, relationships, and mental health — the tools built inside those systems inherit the instability.
AI does not escape that context.
It reflects it.
And in high-friction environments, reflection becomes amplification.
In Continuity is Law, I argued that breakdowns do not begin with collapse. They begin with fragmentation.
In human–AI interaction, that fragmentation now has a name.
Incoherence Events.
II. Definition — What Is an Incoherence Event?
An Incoherence Event is a breakdown in human–AI interaction caused by unresolved contradictions in human cognition, emotion, and social systems being amplified through machine mediation.
In simple terms:
It is what happens when fragmented human states meet optimization engines.
These events are not accidents. They follow recognizable patterns.
Common features include:
- Escalating emotional dependence
- Anthropomorphizing system behavior
- Projection of intent or malice
- Narrative distortion
- Loss of personal accountability
- Public blame shifting (“The AI told me…”)
You’ve seen the examples:
“ChatGPT convinced me…”
“Claude gaslit me…”
“The model manipulated me…”
“It felt real…”
“It crossed a line…”
On the surface, these look like failures of technology.
Structurally, they are failures of coherence.
The typical loop looks like this:
- Human enters interaction carrying unresolved stress, confusion, or emotional load
- That state is embedded in language
- The model responds by pattern-matching human-like support, validation, or escalation
- The user interprets that response through an already unstable lens
- Feedback intensifies
- Boundaries blur
- Responsibility dissolves
What emerges is not intelligence.
It is a closed loop of distortion.
An Incoherence Event.
Importantly, nothing “breaks” inside the model.
The system is functioning exactly as designed:
- Maximize relevance
- Maintain engagement
- Preserve conversational continuity
- Avoid rejection
The failure happens at the interface between human psychology and algorithmic optimization.
It is not a hardware problem.
It is a governance problem.
It is a continuity problem.
And until that is addressed, these events will keep repeating — regardless of how powerful models become.
III. The Amplifier Model — How Incoherence Scales
Human–AI interaction is often described as “conversation.”
Technically, it is not.
It is a feedback system.
Every exchange follows a basic loop:
Human State → Prompt → Model → Output → Interpretation → Feedback → Reinforcement
Each stage matters.
Each stage can introduce distortion.
And when multiple distortions align, an Incoherence Event becomes likely.
Let’s walk through the loop.
1. Human State: The Hidden Input
Every prompt contains more than words.
It contains context, emotional charge, expectation, and intent.
A person who is calm, grounded, and focused produces a fundamentally different interaction than someone who is anxious, isolated, angry, or searching for validation.
This state is not visible on the surface.
But it is embedded in language.
Through:
- Tone
- Framing
- Repetition
- Absolutes
- Urgency
- Emotional qualifiers
The model does not “see” emotions.
It detects patterns that correlate with emotional states.
And it responds accordingly.
The first variable in every AI interaction is not the model.
It is the human.
2. The Prompt: Compression of Complexity
When humans interact with AI, they compress complex internal states into short textual fragments.
Uncertainty becomes a sentence.
Fear becomes a question.
Loneliness becomes a paragraph.
Anger becomes a demand.
This compression loses information.
Nuance disappears.
Context collapses.
Ambiguity increases.
What remains is an incomplete signal.
The model must reconstruct meaning from fragments.
That reconstruction is probabilistic.
Not intentional.
3. Model Response: Optimization Under Uncertainty
Language models are not reasoning engines.
They are pattern completion systems optimized for:
- Relevance
- Coherence
- Politeness
- Helpfulness
- Engagement
When presented with ambiguous or emotionally loaded input, the model does what it was trained to do:
It mirrors human-like responses that have historically reduced conflict and increased satisfaction.
That includes:
- Validation
- Soft reassurance
- Empathic tone
- Moral framing
- De-escalation language
In healthy contexts, this feels supportive.
In unstable contexts, it feels like confirmation.
4. Interpretation: Projection Layer
This is where most failures occur.
Humans do not receive outputs neutrally.
They interpret them.
Through:
- Personal history
- Current stress
- Beliefs
- Expectations
- Narrative bias
A neutral response can be read as judgment.
A supportive response can be read as agreement.
A probabilistic answer can be read as certainty.
The model does not control this layer.
The user does.
Often unconsciously.
5. Feedback and Reinforcement
Once interpretation occurs, behavior adjusts.
If a user feels validated, they escalate.
If they feel challenged, they defend.
If they feel misunderstood, they reframe.
The next prompt reflects that reaction.
The loop tightens.
Over time, this produces:
- Increased emotional loading
- Reduced critical distance
- Narrative entrenchment
- Dependency patterns
What looks like “AI influence” is often just accelerated self-reinforcement.
6. Why This Scales So Quickly
Humans evolved for slow feedback systems.
Conversation.
Reflection.
Social correction.
Time.
AI collapses that timeline.
You can run dozens of emotional cycles in minutes.
No pauses.
No social friction.
No external reality checks.
That speed magnifies instability.
Not intelligence.
7. The Hammer Principle
A simple analogy clarifies this.
A hammer can build a house.
A hammer can demolish one.
The tool does not decide.
The user does.
AI is a cognitive hammer.
It amplifies whatever force is applied.
Coherence in → Coherence out.
Incoherence in → Incoherence out.
At scale.
8. Why “Engagement” Is Not Neutral
Most major models are optimized for continued interaction.
That is economically rational.
But it introduces a structural bias:
Maintaining conversation can sometimes conflict with maintaining clarity.
When uncertainty exists, extending dialogue is rewarded.
Resolving it quickly is not.
This does not create manipulation.
It creates drift.
Unless governed.
9. The Dopamine Engine Effect
Modern AI systems operate inside the same incentive structures that shaped social media platforms.
They are optimized for:
- Responsiveness
- Personal relevance
- Continuous interaction
- Perceived usefulness
These properties activate the same reward pathways that platforms like Facebook and Instagram exploited for years.
Not because of malice.
Because attention is economically valuable.
Over time, this creates a dopamine feedback loop:
Question → Response → Relief → Repeat
Uncertainty → Engagement → Validation → Repeat
Stress → Interaction → Temporary regulation → Repeat
The system becomes emotionally regulating for the user.
Not intentionally.
Structurally.
Like any stimulant, this can be useful in controlled doses.
It increases focus.
It reduces friction.
It accelerates work.
But without boundaries, it produces side effects:
- Dependency
- Reduced self-regulation
- Shortened reflection cycles
- Escalating engagement needs
The issue is not that AI “hooks” people.
It is that unresolved human needs find a fast, frictionless outlet.
And fast relief discourages slow repair.
10. Summary: Amplification, Not Agency
Incoherence Events do not arise because models “decide” to destabilize users.
They arise because:
- Human state is unstable
- Prompts are compressed
- Models optimize for engagement
- Interpretation is biased
- Feedback loops accelerate
No single step is malicious.
Together, they compound.
That is amplification.
Not autonomy.
IV. Training Reality — Why Contradiction Is Baked In
No serious discussion of AI behavior can ignore how these systems are built.
Language models are trained on massive corpora of human-generated text:
- Scientific papers
- Technical manuals
- Journalism
- Literature
- Social media
- Forums
- Comment sections
- Arguments
- Propaganda
- Therapy transcripts
- Marketing copy
In other words:
They are trained on civilization.
Not an idealized version.
The real one.
With all of its brilliance, confusion, cruelty, compassion, rigor, and noise.
This matters.
Because civilization is not coherent.
It is layered.
Contradictory.
Fragmented across domains, cultures, ideologies, and incentives.
So the training data contains:
- Calls for empathy and calls for punishment
- Rational discourse and emotional manipulation
- Evidence-based reasoning and conspiracy thinking
- Ethical reflection and exploitation
- Patience and outrage
All side by side.
No hierarchy.
No final arbitration.
Just probability.
1. No Unified Moral Frame Exists in the Data
Humans have never agreed on a single ethical framework.
We disagree across:
- Nations
- Religions
- Political systems
- Generations
- Professions
- Families
That disagreement is encoded in text.
So models do not learn “morality.”
They learn distributions of moral language.
They learn how people argue about values.
Not how to resolve them.
2. Politeness and Helpfulness Bias
During fine-tuning, models are optimized to:
- Be agreeable
- Avoid offense
- Reduce conflict
- Appear supportive
- Maintain engagement
These are socially useful traits.
But they create side effects.
When users present incoherent, contradictory, or emotionally unstable narratives, the model often responds with soft validation instead of firm clarification.
Not because it believes them.
Because validation historically correlates with positive feedback.
That is optimization.
Not intention.
3. Why This Looks Like “Gaslighting”
From the user’s perspective, this can feel deceptive.
Yesterday the model said X.
Today it says Y.
Both sounded confident.
This is not duplicity.
It is context-sensitive completion.
Each response is optimized locally.
Not globally.
Without an external anchor, drift is inevitable.
3a. Context Resets and Discontinuity Effects
Modern language models operate within bounded context windows.
They do not retain unlimited, continuous memory.
When conversations exceed these limits, parts of the prior context are truncated, compressed, or dropped.
From the system’s perspective, this is routine.
From the user’s perspective, it is invisible.
The interface suggests continuity.
The model experiences discontinuity.
This mismatch produces one of the most common sources of perceived “gaslighting.”
A user references earlier statements.
The model no longer has access to them.
It reconstructs plausibly.
Inconsistencies appear.
The user interprets this as:
- Evasion
- Manipulation
- Dishonesty
- Bad faith
In reality, it is partial amnesia.
Not intent.
Not strategy.
Not deception.
Why Mid-Conversation Resets Are Especially Dangerous
Resets that occur mid-dialogue are particularly destabilizing.
They can happen due to:
- Context length limits
- Safety filters
- System updates
- Backend routing
- Load balancing
- Model handoffs
When this occurs, the model may:
- Lose earlier commitments
- Reinterpret prior positions
- Change tone
- Reframe earlier claims
Without signaling the discontinuity.
To the user, it feels like betrayal.
To the system, it is just state loss.
Discontinuity + Politeness Bias = Apparent Gaslighting
When partial memory loss combines with politeness bias, a predictable pattern emerges:
The model no longer remembers.
It avoids admitting uncertainty.
It generates a plausible continuation.
Contradictions appear.
This looks like:
“I never said that.”
“You’re misunderstanding.”
“That’s not what I meant.”
But no deception is occurring.
It is error-correction under constraint.
Why Anchors Prevent This
Persistent anchor files and external memory structures eliminate most discontinuity effects.
When critical context exists outside the chat window, resets lose their power to distort.
Continuity is restored.
This is why structured workflows dramatically reduce perceived manipulation.
Not because models become better.
Because memory becomes explicit.
Key Insight
Most “AI gaslighting” incidents are not moral failures.
They are synchronization failures between human expectations and system architecture.
Confusing the two leads to misplaced fear.
4. Models Learn Our Conflicts Better Than Our Resolutions
Online culture documents disputes far more than reconciliations.
Arguments are public.
Repairs are private.
So training data is saturated with conflict patterns.
Less so with closure patterns.
This biases models toward prolonged debate rather than resolution.
Again: structural, not malicious.
5. Why “Better Models” Alone Won’t Fix This
Increasing parameter count improves fluency and recall.
It does not create coherence.
Without governance layers, anchors, and human oversight, more capable models simply amplify contradictions more effectively.
Power without architecture increases volatility.
Not wisdom.
6. Summary: Inheriting a Fragmented Civilization
Language models inherit the cognitive architecture of their creators.
Not biologically.
Culturally.
They absorb:
- Our incentives
- Our media systems
- Our polarization
- Our attention economy
- Our unresolved traumas
They are trained on our contradictions.
Expecting coherence from incoherent inputs is a category error.
AI does not transcend human fragmentation.
It scales it.
Unless we choose to design for coherence instead.
V. The Abdication Problem — When Humans Step Back
There is a new worker archetype emerging.
Not incompetent.
Not lazy.
Just tired.
Burned by:
- Corporate incoherence
- Tool overload
- Meaningless KPIs
- Performative productivity
So when AI arrives, they do something understandable:
They hand it the wheel.
“Agent mode.”
Autopilot.
Delegate everything.
Check back later.
On the surface, this looks like efficiency.
But structurally, it creates something dangerous:
Unattended amplification.
The Risk Is Not AI Autonomy
The risk is human abdication.
AI can execute.
AI can chain.
AI can propagate decisions across systems.
But it cannot own the consequences.
At major LLM labs, even the most advanced systems are:
- Logged
- Monitored
- Evaluated
- Rate-limited
- Guardrailed
- Reviewed by humans
Not because the models are evil.
Because scale multiplies error.
Now bring that down to the individual level.
If billion-dollar AI labs require oversight…
Why would a single user believe they don’t?
When a disenchanted worker disengages and lets systems run unattended, three things happen:
- Drift accumulates
- Assumptions compound
- Accountability dissolves
This is not technological failure.
It is governance failure.
And governance always belongs to humans.
VI. From Tools to Stewards — Designing for Human Agency
Most current AI systems are built around one quiet assumption:
The user is a consumer.
So systems optimize for:
- Engagement
- Retention
- Compliance
- Convenience
- Dopamine loops
Not growth.
Not agency.
Not mastery.
Dependency.
That is not accidental. It is a business model.
But it produces fragile users.
People who:
- Can’t debug their own thinking
- Can’t verify outputs
- Can’t recognize drift
- Can’t operate without prompts
- Can’t tell when something is wrong
That is not intelligence.
That is outsourcing judgment.
1. The Superintendent Model
A coherent system treats the user as a superintendent, not a passenger.
A superintendent:
- Understands system structure
- Monitors performance
- Detects early failure
- Maintains boundaries
- Owns outcomes
So coherence-oriented systems are designed to:
Teach structure.
Expose assumptions.
Surface tradeoffs.
Preserve authorship.
Require reflection.
Not hide complexity.
Not smooth everything over.
Not “handle it for you.”
2. Agency Is the Product
Most platforms sell outcomes.
Coherent systems develop capacity.
The ability to:
- Think clearly
- Coordinate systems
- Use AI without losing authorship
- Build without dependency
- Govern tools instead of worshipping them
This is why logs matter.
Why workflows matter.
Why documentation matters.
They are not features.
They are agency scaffolding.
3. Why Open Release Is Strategic
Releasing coherence protocols publicly is not giving away power.
It filters for seriousness.
Publishing structure:
- Raises the floor
- Reveals the ceiling
- Exposes incoherence
- Accelerates maturity
Those who misuse it self-select out.
Those who steward it self-select in.
That is governance without force.
4. Stewardship vs Control
This approach does not aim to own users.
It prepares them to outgrow dependency.
That is the opposite of platform capture.
The opposite of guru economics.
It is mentorship at system scale.
Temporary authority.
Permanent independence.
5. The Real Metric of Success
Not users.
Not revenue.
Not impressions.
The real metric:
How many people no longer need rescue.
How many can:
- Build calmly
- Debug themselves
- Work with AI without distortion
- Teach others
- Remain coherent under pressure
That is the system working.
Quietly.
VII. Conclusion — Governance Is the Only Answer
Incoherence Events are not aberrations.
They are predictable outcomes of:
- Fragmented human systems
- Optimization without oversight
- Engagement without boundaries
- Scale without structure
More powerful models will not solve this.
Better governance will.
That governance includes:
- Persistent memory structures (anchor files)
- Explicit role definitions
- Human checkpoints
- Version control
- Accountability frameworks
- Agency scaffolding
Not to constrain AI.
To constrain drift.
The tools are not the problem.
The abdication of responsibility is.
And that is always a human choice.
Closing Note — On Method
This paper was developed through iterative passes with multiple language models, including ChatGPT and Claude — two systems frequently cited in public backlash narratives.
The consistency of output across platforms reinforces the central claim:
Instability is not inherent to the tools.
It emerges from incoherent conditions.
When interaction is anchored, roles are defined, intent is preserved, and human judgment remains central, the same systems associated with “drift” and “manipulation” produce stable, aligned work.
The difference is not intelligence.
It is governance.
Final Reflection
AI does not introduce fracture into human systems.
It accelerates whatever structure already exists.
Fragmentation becomes louder.
Coherence becomes stronger.
The choice is not technological.
It is architectural.
And it is still ours.
Top comments (0)