A few weeks ago I watched a well-funded agent demo crash. The model had a clean tool-calling loop and a beautiful UI. It was asked: "Why did you retry that tool call?" It answered confidently. Then it answered a different question. Then it contradicted itself.
That failure wasn't about context length. It was about causal transparency: the agent couldn't explain the causal chain of its own decisions.
The next-token contract is a correlation engine
Transformers are trained on one objective: predict the next token. This is enormously powerful, but it optimizes for statistical consistency, not causal validity. A model can hold the right answer in its attention heads without holding a model of why that answer causes the next step.
We see the symptom in production every week:
- Hallucinated tool calls that "look right" but violate API contracts
- Agents that loop on the same retry because they can't distinguish "I failed" from "the tool failed"
- Plausible explanations that don't match the trace
- Planning that degrades under small perturbations of the prompt
The debate in 2025–2026 is whether scale alone closes this gap. My read: it doesn't, because the gap isn't a compute problem. It's an architecture-of-representation problem.
What a causal architecture would actually look like
This is where I want to open a discussion rather than preach. Here is the working hypothesis I've been building (dubbed DCS — Dynamic Causal Structure):
- State, not just tokens. An agent should maintain an explicit causal graph of its own situation: what node caused what, which edge is uncertain, which assumption is load-bearing.
- Disturbance response. Real agency is tested by perturbations — a tool returns an error, a file is missing, a user contradicts you. A causal graph updates locally; a next-token model reflows globally and often silently rewrites its own prior.
- Explanation as a first-class artifact. When you ask "why," the answer should be a subgraph, not a plausible sentence.
- Scale across domains. The same causal bookkeeping that explains a tool-call failure should explain a cell's metabolism, a neuron's firing pattern, and a civilization's transition. That's an ambitious bet, but I think the alternative — separate theories for physics, biology, and intelligence — has already failed to produce a unified research program.
The uncomfortable counterargument
I want to steelman the "scale is enough" position, because it's worth taking seriously:
- Modern models already learn implicit causal structure from training data. Physics, grammar, and tool usage all have causal structure embedded.
- Fine-tuning on trajectories seems to fix most agent failures in practice.
- Causal-symbolic approaches (GOFAI, expert systems) have repeatedly promised transparency and delivered brittleness.
The reply, I think, is that implicit structure is fine when the distribution is narrow, and fails exactly when agents leave the textbook. Production agents operate under distribution shift by definition — that's the whole point of autonomy.
What I'm actually building
I'm not here to sell a framework. I'm publishing a ~200,000-word research program called DCS (Dynamic Causal Structure) that tries to do three specific things:
- Define a single causal-structure primitive that applies from the Big Bang through chemistry, biology, neural systems, and finally AI agents.
- Derive testable predictions about where token-prediction systems should and shouldn't break.
- Treat the theory itself as a living agent — a document that updates as evidence comes in.
The launch is a live online session on September 16, 2026 (English session 09:00–11:30 Beijing; Chinese session 19:30–22:00 Beijing). It's ~2.5 hours, mostly animation, with a 138-slide deck. More here: https://MINDAS.ME
Questions I'd love to hear from you
- Have you hit a production agent failure that looks causal rather than statistical?
- Do you think tool-use fine-tuning is a local patch or a permanent solution?
- What's the cleanest falsification test for a "causal architecture" claim?
- Is cross-disciplinary unification (physics → life → mind → AI) actually productive, or is it a philosopher's trap?
Drop a comment. I'd rather be wrong in public than right in a slide deck.
Top comments (1)
The demo failure you describe is real, but my counter is that the subgraph already exists in most production agent stacks -- we log the full tool-call DAG with inputs, outputs and the error that triggered each retry, and it answers 'why did you retry' exactly as a structure, not a story. What's missing isn't a new architecture, it's wiring that graph back in as the model's evidence instead of letting it narrate from the transcript. When we force the answer to cite node ids from the trace, the self-contradiction rate collapses, because a confident wrong explanation fails the citation check.
Where I agree with you is the perturbation point, and that's the falsifiable part of DCS: a next-token system silently reflows its prior, a graph system should update one neighbourhood and flag the uncertain edges. Do you have a concrete perturbation benchmark in mind -- injected tool errors, contradictory user turns -- or is the claim still at the representational level where no experiment could yet tell the two hypotheses apart?