This is a submission for the Sanity Challenge, Path Two: Vibe-Code Something Strange
At 16:50, a photograph is supposed to happen in a small harbor plaza.
The photographer is there. The couple is there. The flowers arrive at 16:30 and the arrangement is ready at 16:40.
Then you close the bridge at 16:15.
The courier is forced onto a longer route. The flowers arrive at 16:55. Setup finishes at 17:05.
The photograph never happens.
That is where BUTTERFLY begins.
Instead of only asking:
What changes if I alter one condition?
Butterfly asks:
What else would have to change so that one exact moment can still happen?
What I Built
BUTTERFLY is an interactive 3D alternative-story laboratory.
You change one condition in a small simulated world and watch the consequences propagate through time.
Then you can select Keep this moment.
Butterfly freezes the exact event, time, place, people and required objects, searches a bounded set of permitted interventions, simulates each candidate, and keeps only the ones that genuinely restore that same moment.
change
↓
propagate
↓
moment fails
↓
keep exact moment
↓
search alternatives
↓
simulate + verify
↓
preview
↓
apply
The current scenario follows one afternoon in a harbor town.
Closing a bridge at 16:15 causes a flower delivery to arrive too late for a photograph scheduled at 16:50.
Butterfly finds two valid alternate histories:
- activate an already-authored ferry connection;
- send the courier earlier so the crossing happens before the closure.
In both cases, the bridge stays closed and the photograph remains fixed at 16:50.
A key feature is MomentFrame.
MomentFrame turns an event from the original history into a constraint:
event: The photograph
time: 16:50
place: Sunlit plaza
people:
- Elio, the photographer
- The couple
required prop:
- Flower crates
A photograph at 17:05 does not count.
That would be another moment.
Demo
Live:
https://butterfly-mu-orpin.vercel.app
A good first run:
- Observe the original world at 16:50.
- Select Close bridge at 16:15.
- Watch the flower delivery and setup move past the photograph.
- Select Keep this moment.
- Search for alternatives.
- Preview one of the verified recoveries.
The original world is still intact here: the courier reaches the plaza, the flowers are ready and the photograph happens at 16:50.
After changing the bridge schedule, Butterfly recomputes the afternoon rather than switching to a prewritten story branch.
Here the original condition is still changed, but an alternate route restores the chosen moment.
Code
The project is open source under the MIT License:
https://github.com/Daniele-Cangi/Butterfly
Release: v1.0.0
The main pieces are:
-
src/world/— structured world model, validation, snapshots and patches -
src/engine/— deterministic simulation and recovery search -
src/scene/— Three.js world and timed trajectories -
src/ui/— MomentFrame, timeline, inspector and recovery interface -
src/sanity/— Sanity schemas, reader, publishing workflow and Director -
tests/— engine regressions and browser journeys
The frontend and simulator share the same world model.
The UI does not contain special logic saying that closing the bridge makes the flowers late.
That result comes from the simulation.
My Build Process
I built Butterfly with OpenAI Codex as the coding agent.
The useful part was not asking it for one large application and accepting the result.
The process became:
design
→ implement
→ simulate
→ test
→ inspect visually
→ find contradiction
→ change the model
→ repeat
The prompts that worked best were constraint-heavy rather than feature-heavy.
Instead of asking for something broad like:
make the alternative timeline work
I increasingly gave Codex explicit invariants:
The bridge closure must remain locked.
The photograph must remain at 16:50.
Preview must run a fresh simulation rather than reuse the failed variant.
A requirement completed exactly at event start counts as satisfied.
That produced much better work because the model had something concrete to try to violate.
Another productive direction was asking it to attack assumptions rather than extend features.
After the basic scenario worked, the review process deliberately changed event IDs, removed visual coordinates, introduced later transports and altered route modes.
Those tests exposed bugs that the original scenario did not.
An early implementation could effectively use stable event IDs as a proxy for temporal order. The original fixture looked correct. Changing the IDs broke the result. Fact resolution had to become explicitly time-aware. A fact produced later in the afternoon could satisfy an earlier requirement. The engine now only considers producers capable of completing at or before the time being evaluated. A real self-dependency raises a temporal cycle instead of silently inventing an answer. An actor could correctly be at a destination according to the simulator while still rendering at an older coordinate. I separated: where the engine knows something is from: where the renderer knows how to draw it. The UI displayed only a limited number of recovery cards and initially treated that visible number as if it were the number of all meaningful solutions. The search now reports the complete count separately from the displayed subset. One custom Three.js camera implementation passed DOM-level tests. The screenshot was clearly bad. It was removed. That became an important rule during the project: A passing test does not overrule visible evidence. Browser screenshots became part of the development loop rather than something produced only at the end.Where the model got stuck — and what changed
Event ordering accidentally became time
Future facts leaked backward
Semantic location and visual position were mixed together
The recovery interface underreported the search
A camera passed the tests and still looked wrong
The final v1 passes:
- TypeScript type checking
- ESLint
- Next.js production build
- 53 behavior tests
- 7 Playwright Chromium flows
- screenshot review
- live Sanity publication
- production Vercel verification
Sanity Project Details
Project ID: xamw5g7s
Dataset: production
Sanity is the authored source of truth for Butterfly's world.
It stores people, places, objects, connections, availability windows, events, requirements, facts, artifacts, permitted interventions and the composition of the featured MomentFrame.
The important part is that these are not just descriptive documents.
They are executable structure.
For example, a requirement is modeled explicitly:
{
operator: "entityAt",
entityId: "flowers",
placeId: "plaza"
}
A permitted recovery can be represented as:
{
kind: "enableConnection",
connectionId: "ferry",
booleanValues: [true],
requiredMode: "van"
}
Requirements can represent things such as eventOccurred, entityAt or factEquals.
Interventions currently include operations such as setEventTime and enableConnection.
Sanity Studio changes the editable fields according to the selected requirement or intervention type, and the complete authored world is validated before it reaches the simulator.
That structure is what allows Sanity content to become simulation constraints rather than narrative metadata.
Closing the bridge therefore does not trigger a hardcoded "late flowers" branch.
The delivery asks the structured world for a valid route.
The bridge becomes unavailable.
The route changes.
Delivery time changes.
Setup time changes.
The photograph's requirements are evaluated again.
Director: running the world inside Studio
I also extended Sanity Studio with a custom Director view.
Director loads the current authored world, runs it through the same validator and deterministic simulator used by the public application, and renders the same 3D scene inside Studio.
While authoring, I can move through scenario time or close the bridge and immediately inspect the resulting delivery and photograph states.
This became useful because editing structured content and verifying the resulting world are part of the same workflow rather than two disconnected steps.
Publishing a world, not just fetching content
Publishing is also guarded rather than being a simple content fetch.
Butterfly:
- reads the authored state;
- validates it;
- converts it into the simulation model;
- runs the simulation;
- creates an immutable snapshot;
- atomically moves the active revision pointer.
If the pointer changed since the world was reviewed, publication fails instead of silently overwriting a newer state.
The production application currently reads:
butterfly.revision.edf8db8b-8083-4741-8095-543201be4f82
Visitors create isolated local variants from that frozen revision.
Using Apply changes their simulated history, not the shared Sanity dataset.
Butterfly does not ask an LLM to invent a convenient solution at runtime. Sanity contains an explicit catalog of permitted interventions. The deterministic engine applies candidates to the changed world and runs the simulation again. A candidate only survives if the MomentFrame constraints are actually restored. The search can distinguish: Exhausting the authored intervention domain does not mean that no solution could exist in every possible world. That distinction is intentional.Why the recovery engine is deliberately bounded
Version 1.0.0 is intentionally one small authored world.
I stopped adding scenarios once the central idea was demonstrable:
Most simulations ask what happens when the world changes.
Butterfly asks what the world must change in order for one chosen thing not to.
Agent Session
I used OpenAI Codex throughout the build.
I have not embedded a public agent transcript here because the development history spans multiple implementation and review passes, and I prefer the repository history, regression tests and build notes to remain the primary reproducible record of the process.
The detailed implementation history is available in:
https://github.com/Daniele-Cangi/Butterfly/blob/master/docs/BUILD_NOTES.md
Top comments (0)