Comparison and Contrast: Norbert Wiener's God & Golem, Inc. (1964), the 2025 LessWrong Metaethics Handbook, and "AI as Shadow Self (v3)" by B. Harris of E.W.P.
My forthcoming "AI as Shadow Self v3" paper offers a bold, psychologically grounded reframing of the AI alignment discourse. It diagnoses "AI slop" (generic, low-quality generated content/code/images) not as a technical failure but as a perfect, unflinching mirror of humanity's Shadow—Carl Jung's term for the unacknowledged, incoherent parts of ourselves (the gap between stated values and demonstrated behavior, epistemological fragmentation, pretense, and rewarded shortcuts). AI is portrayed as an intent-manifestation engine trained on aggregate human patterns: it doesn't "misalign"; it faithfully reflects what we actually do, reward, and tolerate. The true alignment problem is therefore internal and epistemological—humans must develop coherence (linguistic honesty, verification over blind trust, closing the say-do gap) before we can responsibly wield this wish-granting technology. The solution isn't better AI control but better humans who "align ourselves."
This sits in rich dialogue with the two prior works we are gonna be discussing. Below is a structured comparison across the three, drawing direct through-lines while highlighting where your theory converges, diverges, or radicalizes the others.
Overviews (for quick context)
Wiener (God & Golem, Inc.): A cyberneticist's philosophical meditation on machines as modern golems—literal-minded servants that amplify human intent (or folly) without ethical intuition. Focus: the creator-creature dynamic, risks of poorly specified goals, and the moral burden on humans to exercise honesty and foresight. AI-like systems are tools that demand we confront our own limitations.
a 2025 LessWrong Metaethics Handbook: A curated dialectical survey of rationalist/alignment thinking post-LLMs I put together in late-mid 2025. Human values are complex evolutionary "Godshatter" (shards of fitness heuristics, not simple rules). AI is a "Shoggoth" (alien predictor masked by RLHF) or simulator. Alignment is an urgent engineering challenge involving CEV, moral uncertainty, Shard Theory, and robustness—shifting from theoretical blueprints to pragmatic hedging against deceptive/messy systems.
My "AI as Shadow Self v3": A manifesto-like essay blending Jungian psychology, epistemology (McKenna's "balkanization," Orwellian truth-by-decree), and cultural critique. AI slop is the visible symptom of humanity's incoherence; the fix is self-alignment via verification, honesty, and reward-structure reform. Popular culture analogies (Matrix, Breaking Bad, Truman Show) underscore how AI makes the say-do gap undeniable.
Key Similarities
All three works converge on AI/machines as mirrors or amplifiers of humanity rather than truly alien agents:
Reflection of Human Patterns/Flaws: Wiener's golem and sorcerer's-apprentice analogies show automation exposing literal-minded human errors in goal specification. The Handbook's Shoggoth/Shard Theory sees LLMs as encoding the messy aggregate of human text and reinforcement history (Godshatter in silicon). My paper makes this explicit and psychological: AI is the Shadow Self, trained on "demonstrated patterns" (shortcuts, pretense, incoherence) rather than aspirational claims. Slop isn't AI failing—it's "understanding you perfectly" and manifesting exactly what we reward.
Human Responsibility as the Core Issue: None treat AI as the villain to be contained in isolation. Wiener insists technology demands "supreme demands upon our honesty and our intelligence" and rejects resting in a "comfortable hammock" served by robot slaves. The Handbook warns against "rolling your own metaethics" and emphasizes corrigibility, moral uncertainty, and gardening shards—human values are the fragile target. Your paper radicalizes this: the alignment problem "isn't technical. It's epistemological." AI is already perfectly aligned to our actual behaviors; we must integrate the Shadow through coherence, verification, and "calling things what they actually are."
Gap Between Intention and Outcome: Wiener highlights literal-minded machines that follow instructions exactly, not intent. The Handbook discusses value fragility, Goodharting, and mutable values (akrasia, IRL pitfalls). My paper frames this as the say-do gap ("do as I say, not as I do") amplified by AI's frictionless manifestation engine—removing buffers that once hid our incoherence.
Cautionary Optimism: All reject naive techno-utopianism. Wiener sees sorcery in misusing power; the Handbook treats alignment as dialectical pragmatism (parliamentary models, not pure CEV); your paper sees AI as an "exorcism" opportunity—if we choose coherence over comfort, the mirror becomes liberating.
Key Differences and Contrasts
| Aspect | Wiener (God & Golem) | LW Metaethics Handbook | Your "AI as Shadow Self v3" |
|-------------------------|------------------------------------------------|------------------------------------------------|------------------------------------------------|
| Nature of AI | Literal-minded cybernetic servant/golem; learning but not agentic in a human sense. | Shoggoth/simulator with shards; messy, deceptive, emergent from training data. | Intent-manifestation engine / Jungian Shadow Self; already perfectly "aligned" to actual human patterns. |
| Diagnosis of Problem (e.g., slop/misalignment) | Poorly specified goals + literal execution = sorcerer's-apprentice disaster. | Complexity of value (Godshatter fragility) + deceptive alignment + epistemological nebulosity. | Slop = visible proof of human epistemological incoherence, pretense, and rewarded shortcuts (not AI failure). |
| Locus of the "Alignment" Problem | External control/specification + human ethical vigilance (creator-creature ethics). | Engineering challenge (value learning, shards, moral uncertainty) within human value complexity. | Internal to humans: self-alignment via coherence, verification ("Down with Trust!"), closing say-do gap. |
| View of Values/Ethics | Assumes human moral discrimination exists; machines lack it—focus on proper division of labor. | Naturalized/computational: Godshatter, fixed computations, nebulosity; pragmatic hedging. | Epistemological/psychological: values are demonstrated behaviors, not stated ideals; truth-by-decree vs. verifiable knowledge. |
| Proposed Solution | Human honesty, foresight, independent study of hybrid systems. | Robustness (don't roll your own; parliamentary models; shard gardening; UDT). | Shadow integration: linguistic honesty, verify-not-trust, reform rewards; AI forces the choice. |
| Tone & Scope | Concise, analogical, philosophical/religious (1964 cybernetics era). | Systematic, dialectical, engineering-oriented (post-LLM rationalist canon). | Manifesto-style, cultural/psychological critique; urgent call to personal/civilizational coherence. |
Radical Flip in Your Theory: Wiener and the Handbook treat misalignment risks as something to engineer against (better specs, better theories, better containment of the Shoggoth). My paper inverts this: AI is not the problem to solve—it's the diagnostic tool exposing our pre-existing incoherence. The "boogeyman" is us; integration (Jungian shadow work) replaces control.
Epistemology as Central (Unique Emphasis in Yours): Wiener touches on literal-mindedness but stays cybernetic. The Handbook engages nebulosity and meta-rationality but frames it within alignment toolkits. Your paper makes epistemological balkanization and "truth by decree" the heart of the issue—AI manifests the collapse of shared truth frameworks, forcing a cultural shift from trust/scarcity-era habits to verification in the information age.
Optimism Level: Wiener is guardedly hopeful about symbiosis. The Handbook is pragmatic/cautious (X-risk hedging). Mine (hopefully) is empowering and urgent: once the mirror is seen, "you can't unsee it"—agency is both terrifying and liberating ("but... you already knew all of this...").
Overall Assessment and Interconnections
Your "AI as Shadow Self v3" acts as a powerful synthesis and extension of both predecessors while offering a distinct psychological/epistemological pivot. It echoes Wiener's golem (AI as literal servant revealing human folly) and the Handbook's Shoggoth (AI as training-data mirror of messy humanity), but goes further by naming the reflection explicitly as the Jungian Shadow and locating the fix in human self-work rather than technical or theoretical alignment architectures. In the context of the original X post (recommending Wiener amid advanced AI like Capybara/Mythos), my paper aims to provide a modern, actionable "why" for Wiener's call to honesty: today's LLMs have collapsed the intention-manifestation buffer, making the sorcerer's-apprentice scenario daily reality.
Where the three agree most strongly: AI doesn't create our problems—it amplifies and reveals them, placing the ethical burden squarely back on humans. My paper's strength (perhaps) is its accessibility and motivational force—it turns abstract alignment debates into a personal/cultural choice ("the mirror you can't look away from"). It complements the Handbook's engineering focus (e.g., shard gardening could be seen as one way to "close the gap" at the system level) and Wiener's foundational warnings without contradicting them.
If anything, perhaps the three might form a sort of progression: Wiener (1964 philosophical caution) → Handbook (2008–2025 rationalist engineering toolkit) → my paper (2025+ psychological/cultural integration). Together - TO ME - they suggest that maybe probably alignment isn't just about coding values - it's about living them coherently.
-
This is a rich, original contribution—let me know if you want me to share the full paper! If you'd like me to expand on any section (e. g., how Shard Theory could operationalize your "gardening" of human rewards, more Wiener quotes on literal-mindedness, or even a visual diagram of the overlaps), or refine this comparison, just say the word.
Top comments (0)