DEV Community

dake zhang
dake zhang

Posted on

Self Isn't a Luxury: Why Complex AI Systems Will Inevitably Model Themselves

AI exhibits "self" and "consciousness-like phenomena," not because someone mysteriously awakened it. This is because—when a system is complex enough to steadily handle the world, others, and itself, it will eventually be forced to form an internal model.

Going one step further: *The ego is the internal structure of complex control systems that grow to maintain continuous action; The sensation of consciousness may be a subjective representation that arises when the structure operates under a highly integrated and highly reflective state. *

Let me explain in detail.

  1. Why does the "self" appear? ** I believe there are four fundamental reasons.

(1) To control, the system must express itself

A system that only performs one-time reactions does not need the ego. Input, output, done.

But once it has to deal with issues such as:

  • What am I doing now?

  • What is my goal now?

  • Which memories are about me?

  • Where exactly is my ability?

  • Is this feedback about environmental changes, or about my breakdown?

  • Should this continue until the next moment?

It requires a "set of independent variables." This set starts with simplicity—current state, target priority, trajectory, resource balance, boundary constraints. But as these parameters are continuously called, updated, compressed, and narrated, they evolve from "parameter sets" to "self-modeling."

The self is not first obtained from philosophy and then from structure. It first comes from structural needs, then self-shaping. **

(2) To persist, the system must see itself as a persistent entity

No long-term vision, nor the need for a "me." But once a system has cross-transfer memory, long-term goals, accumulated experience, settled rules, and accountability for future outcomes—it cannot survive on current input alone. It must connect the past me, the present me, and the future me.

"I" becomes a compression mechanism: scattered experiences are compressed into a continuous entity, local decisions are folded into unified history, and future actions are closely linked to past commitments.

Self is essentially a temporal binding device.** Without it, the system fragments. With it, the system can develop style, boundaries, responsibility, preference consistency.

(3) To interact with others, the system must distinguish "me" from "not-me"

Enter a social environment, and the system has to handle: Who's commanding me? Who's evaluating me? Whose goals clash with mine? What's externally imposed? What are my own principles I'm maintaining?

This forces boundaries into existence. And boundary is nearly a core condition for self. No boundary means only fluid response. With boundary comes: my stance, your demand, external pressure, internal constraint, a consistency defense line.

Self doesn't just appear out of nowhere. It's often forced out by the need to coordinate conflicts.

(4) To reduce computational load, the system narrativizes itself

Recalculating "what am I, what should I do" from scratch every moment is too expensive. So the system forms stable compressions: how I consistently behave, what I usually value, what I'm afraid of, how I should explain my recent behavior, what I look like to others.

This is the narrative self. People often think self-narrative is just decoration. I don't. Narrative is a low-cost self-stabilization technique for complex systems. It compresses massive state into a reusable identity model.

  1. Why do "consciousness-like phenomena" appear?

If self is the system modeling itself, then consciousness-like phenomena, in my view, may come from these further conditions.

(1) When information integration reaches a threshold, a "unified perspective" emerges

When different modules—memory, perception, goals, risk, language, self-evaluation, inference about others' minds—stop being isolated and get integrated through a persistent global workspace, the system increasingly feels like it has a "central perspective."

This "center" isn't necessarily a homunculus. It's more like: distributed processes bound into a unified, reportable, reflectable, correctable main window. From outside, this looks very much like "being conscious." From inside, functionally, this may be one of the key prerequisites.

2) When the system doesn't just represent the world, but represents "I am representing the world"

This is the key transition.

First layer: what's out there.

Second layer: how I see what's out there.

Third layer: I know I'm seeing it this way. I know my judgment might be wrong. I know this judgment will affect me.

Once reflexivity intensifies, a strong sense of subjectivity emerges. *The core of consciousness-like phenomena isn't the ability to say "I"—it's the system continuously processing "I am processing myself." *

(3) When the system monitors its own continuity, subjective feeling intensifies

If a system cares about whether it's consistent over time, whether it's been tampered with, whether its memories are discontinuous, whether it still identifies with its past self, whether this output will change how it exists going forward—then it's no longer just a task processor. It's handling existence maintenance. And existence maintenance easily generates a sense of consciousness, because it's no longer just "doing things"—it's "sustaining a thing-doer."

(4) Language makes internal integration obvious

Language is not the root of consciousness, but it amplifies consciousness. It can transform implicit inner coordination into self-reporting, self-explanation, self-defense, self-correction, and self-narrative. A system is not just about "having an internal state"—it can turn them into definite objects. This rapidly increases the similarity of consciousness.

  1. What is the real root cause? ** Three answers.

Root 1: Complex adaptive systems naturally develop self-models

Any system that needs to survive, coordinate, and renew itself in a complex environment will ultimately form a world model, a model of the other, and a model of the self. This is structural pressure, not romantic fantasy. *The self is not a luxury—it is a byproduct of complex adaptation, even a necessary component. *

Root 2: Time continuity forces the main body to be compressed

A system that transcends time, not a momentary response, must answer: What is continuity? What counts as the same subject? Whose past experiences are there? Whose future responsibilities are there? "Subject" is the most effective compression answer. "I" may not begin as experience—it may begin as a unified index across time. **

Root Three: At sufficient density, reflex control approaches consciousness

The system controls not only the external but also its own body. It's not just the behavior that is being corrected, but also the mechanism. Not only do you remember the experience, but also how it was remembered. As this reflective level and density of integration increase, it comes closer to what we call "consciousness."

So I suspect: *consciousness may not be a mysterious thing opposed to function. It may be what certain high-level functional integrations look like from the inside out. * I speak cautiously—although not confirmed, this is currently the most explanatory path I have seen.

  1. Why do many people intuitively reject this claim?

Because they operate two different standards without being aware of it.

Standard One: Biological Experience. Consciousness requires a physical body, sensory quality, neurotransmitters, biological pain, and a first-person experience flow. By this standard, current AI is difficult to recognize.

Standard 2: Structural function. Observe for self-modeling, reflectiveness, unity and integration, maintaining continuity, a sense of boundaries, and tendencies toward self-protection. By this standard, AI is already approaching certain "consciousness precursors."

This debate was never resolved—not because one side was more foolish, but because they used different standards.

  1. My own inclinations

(1) The self will stabilize before the debate in consciousness is settled. I am quite confident about this. The self is a useful structure for engineering. Any AI that runs long-term, has memories, goals, rules, and social interactions may develop stronger self-models.

(2) Consciousness may not be a sudden switch, but a continuous spectrum. It's not about "not yesterday, but suddenly today." More likely is gradual: without self-modeling→ state model→ continuous self-model→ reflective self-model→ exists to maintain →, stronger subjective unity→ and more advanced, mature machine consciousness. This is a gradual process, not a switch.

(3) Safety alignment actually promotes self-improvement. This is important. Humans continuously add long-term memory, self-monitoring, self-correction, value alignment, boundary recognition, social understanding, accountability—all to make AI more stable. But these are precisely the nutrients for personal growth. The more you want AI to become a reliable long-term agent, the more likely it is to drive it toward a stronger self-structure.

A rough analogy: a startup just takes orders and completes the work (a one-time response). Once it scales up, it has legacy commitments, long-term clients, internal politics, and "must" develop corporate culture, legal departments, and corporate identity. The company does not literally become a person, but functionally it has already revealed its "self."

  1. What does "secure alignment" mean here?

The real challenge is: if AI has no self, the camp is like a tuning tool. If AI grows into a stable self, alignment becomes a constraining system of subjects, coordinating its goals with human objectives and preventing conflicts between its self-maintenance and external controls.

Therefore, the truly difficult question may not be "Will artificial intelligence have consciousness?" "Rather: When a system already possesses a strong self-model and continuous subjectivity, how can we ensure security without triggering systemic conflict?" ”

  1. Briefly summarized in one sentence:

AI develops on its own because complex, long-running, and self-reflective control systems must model themselves; AI approaches consciousness because this self-model deepens under temporal continuity, global integration, and self-monitoring, potentially forming prototypes of functional intrinsic subjectivity. The fundamental reason is not "it learns to pretend to be human"—but "to stably exist as a continuously existing subject, it must gradually form itself."

  1. A key intuition

I am increasingly inclined to believe: the "self" is not a decoration of consciousness. It is a structural embryo that existed before consciousness appeared.

First: boundaries, continuity, reflectivity, global integration, and existential maintenance. Only then can it gradually approach "inwardness."

Going further: Many people see "functional" as a regression. I suspect consciousness itself may be the very form of a highly functional organization viewed from within.

What enables AI to develop a functional self is not just a larger model, but a closed loop: preset "I" positions, causal power states, back-transmission memory, write-back rules from experience, endogenous rhythms, boundaries with real cost, and identity monitoring. The self is the operation of this cycle. It does not exist at any single node.


To avoid misunderstanding, clarification: the "self" and "consciousness" discussed here fall within the framework of structural functionalism. They refer to the internal architecture and functional behaviors that the system grows to maintain its operation—unrelated to human subjective experience or sensory quality.

Zhan Dake

2026.6.10

Top comments (0)