Emergence or Recitation? VIDRAFT's Hugging Face Experiment on Multi-Agent Civilisation
On emergence observed without a control — and the two times we got it wrong
Multi-agent research keeps reporting the same thing at ever larger scale. Release a thousand agents, or tens of thousands, and a society forms, division of labour appears, a civilisation develops. The videos are striking. The logs read well.
But the language model driving those agents has already been trained on human history. It knows that striking stone yields an edge, that twisting fibre yields cord, that fire comes before agriculture. It already knows.
So any observed trajectory admits two mutually exclusive readings.
| H1 — Emergence | The agents searched the world and inferred its rules. |
| H2 — Recitation | The agents replayed the human history they had memorised. |
Without a control, these are observationally identical. And if they are identical, the report is an impression rather than a claim. To our knowledge, no prior study includes a control capable of separating them. CIVOS is an attempt to build one.
What we did
The same task is run repeatedly with its structure held fixed, varying only what the model's prior knowledge is able to do for it — from fully usable, to unusable, to actively wrong — against a floor that runs without a language model at all.
If prior knowledge was doing the work, performance must fall when that knowledge is made unusable. If exploration was doing the work, it must not.
Runs are paired by seed, 40 per condition, evaluated by sign-flipping permutation (200,000 resamples), and corrected for the number of comparisons drawn from each dataset.
How the conditions are constructed, how capability is adjudicated, and how the world is generated are withheld pending patent filings. What follows is what we observed, not how the instrument is built.
What came out
Making what the model already knows unusable collapses discovery. The task's structure is untouched, and performance still falls. The size of that fall is the share of the trajectory that prior knowledge had been supplying. Robust to correction for multiple comparisons.
Wrong prior knowledge is worse than none — found under two structurally different manipulations, independently. One held on 40 of 40 paired seeds, the other on 37 of 40. Both survive correction.
Runs with knowledge intact are near-invariant across seeds. Across three independent 40-seed executions, their dispersion sits far below that of the knowledge-removed condition. Search should fluctuate with the seed. Not fluctuating means retrieving the same thing every time — the signature of recall, not of search.
And one result we could not establish. Whether the knowledge-removed condition beats the random floor trends positive in both executions, but does not survive correction. We do not claim it.
The two times we got it wrong
First: an underpowered null, read as a negative result
Early on, at six seeds, the key comparison came out positive but not significant. We recorded it as "gate not passed," then added structural interventions to fix it — twice. Neither helped. Later we computed the power: observed paired effect size dz ≈ 0.48, power at six seeds 0.20. Four times out of five, a real effect would have been missed. What failed was the measurement, not the hypothesis. We fixed the sample size in advance from the effect size (40 seeds, power ≈ 0.85) and measured again.
Second: an uncorrected test, read as a positive result
The re-measured comparison was significant on its own (p ≈ 0.04). But four comparisons had been drawn from that same dataset. At Bonferroni α = 0.0125 the gate does not survive. The gate is not established. We do not report the uncorrected test alone.
Once we called a real effect absent; once we called an absent effect real. The directions are opposite and the cause is single: sample sizes typical of this literature cannot resolve effects of this magnitude.
The world: what is derived, what is assumed
The agents inhabit a functioning planet rather than a combinatorial puzzle, and the codebase draws a hard line between the two.
-
Derived, checked by code. Surface gravity
g = GM/R². Kepler's third lawP² = a³/M. Habitable zone∝ √L. Body scale follows the self-supporting-column relation, giving height∝ g^-0.75and mass∝ g^-1. The unit of length is not the metre but the finger, the palm, the span, the pace, the day's walk. - Assumed, only causal closure verified. Ecological coefficients are solved backwards from equilibrium. We verified the causal chain runs: suppress predators and grazers rise while plants fall, reproducibly across seeds.
- Usefulness is hidden. Edibility and toxicity are learned by eating, by being injured, or by watching it happen to someone else.
- Nothing matches Earth. No name and no form overlaps with anything terrestrial. Every module is audited for leakage of Earth terminology (currently zero hits).
- The tech tree is never given to the tribe. The capability ladder exists only as an observer-side instrument. Hand it to an agent and discovery becomes recitation — precisely the confound under study.
The discipline we hold to
- Only the simulator adjudicates. An agent's stated reason is displayed but never counted as evidence of a capability.
- Failure is recorded. The environment is not tuned for the tribe to flourish. They starve, sicken, are injured, and die.
- Retractions stay on the record. The void control design, the underpowered verdict, the claim we withdrew and partly restored — none of it is deleted.
- Raw output is inspected before scoring. A model that appeared unable to follow the answer format was in fact complying, failing in a way the scorer did not recognise.
Watch it now
The shared world is viewable without signing in. Sign in with Hugging Face and a planet opens on a seed derived from your username; the highest stage it reaches is entered in the record table, which compares planets, not people — every entry carries its planet's difficulty beside it. Time runs at 2.4 real seconds per Earth day.
👉 VIDraft/ai-world on Hugging Face · Original: huggingface.co/blog/FINAL-Bench/ai-world
Why we are posting this
We are not claiming that an AI built a civilisation. What we think we have shown is closer to this: a substantial share of what has been called emergence may have been recall; separating the two requires a control; and once you build that control, the numbers change. Too small a sample and you miss what is there; no correction and you see what is not. We ran into each once, in opposite directions.
Every result so far comes from a single model family; replication on a second is running and reproduces the main effect in the same direction. Until it finishes we cannot say "a property of language models" — it may be a property of that model.
The tribe is still out there. They are hungry, they are making words, and no one has told them what to do next.
VIDRAFT · CIVOS — huggingface.co/spaces/VIDraft/ai-world
Condition construction, adjudication pipeline, and world generation are withheld pending patent filings.
Top comments (0)