Min Choi’s Grok 4.6 thread works because it is a directory of requests that end in visible artifacts, not a page of model parameters. The compilation presents ten examples. Three representative cases in the source material make the range clear: a long-running attempt to build an FPS, a game-trailer workflow that reportedly opened and played the game before recording, editing, and narrating, and a workflow that reportedly looked up dimensions before generating a 3D-printable spacer for a Tesla phone charger.
The thread is a useful discovery surface. It is not a benchmark. The examples come from different creators, use different prompts and tools, expose different amounts of process, and may hide different retry counts or human assistance. The captured first-post evidence also does not contain full details for every one of the ten linked cases, so I will not fill the gaps from the headline.
Developers can still extract more than inspiration from the collection. The right artifact is an evaluation matrix that preserves the differences between demos before choosing which ones deserve replication.
First, classify the task instead of the model
Give each demo a task card with fields that can be checked from the original source:
| Dimension | Questions to record |
|---|---|
| Requested outcome | What did the person ask for, in their own words? |
| Final artifact | Codebase, executable game, video, CAD file, physical part, or something else? |
| Environment | Closed files, browser, desktop UI, game runtime, or physical world? |
| Tool breadth | How many distinct tools or interfaces were required? |
| Time horizon | Seconds, one session, or a multi-hour run? |
| Human role | Prompt only, approvals, corrections, asset preparation, or hidden setup? |
| Evidence available | Screenshot, final clip, raw artifact, trace, repeated runs, independent reproduction? |
| Failure consequence | Cosmetic defect, lost work, false product claim, unsafe physical output? |
This prevents a basic category error. A polished trailer and a printable spacer are both “finished artifacts,” but their acceptance criteria are not interchangeable. The trailer must faithfully represent the game. The spacer must satisfy geometry, tolerances, material assumptions, and physical fit. The FPS case raises a third question: whether a long autonomous run produced a coherent, maintainable project rather than merely many generated files.
Use evidence levels, not viral confidence
Add a proposed evidence level to every task card:
- E0 — claim only: a description without an inspectable result;
- E1 — visible artifact: screenshot, clip, or downloadable output;
- E2 — process evidence: prompt, intermediate artifacts, tool trace, and failure record;
- E3 — repeated runs: the same task succeeds across fresh trials under documented conditions;
- E4 — independent reproduction: another evaluator repeats the task from the published protocol.
These levels do not score quality. They score how much of the claim can be inspected. A beautiful E1 clip can identify a promising capability, while an ugly E3 result may provide stronger evidence about reliability.
This distinction follows NIST’s TEVV guidance, which emphasizes meaningful tasks, testbeds, metrics, and documented evaluation methods. NIST’s AI RMF Measure function also says test sets, metrics, tools, deployment-like conditions, limitations, and results should be documented. A social thread is excellent at proposing tasks; it usually does not supply that evaluation package.
Keep separate score families
Do not average everything into “8.7/10.” Record at least four score families:
1. Completion
- Was the requested artifact produced?
- Did it open, run, render, or fabricate successfully?
- Which required stage failed?
2. Process
- elapsed time;
- retries and full restarts;
- number and duration of human interventions;
- tool errors and recovery path;
- cost, when it can be measured consistently.
3. Artifact quality
This rubric must be task-specific. For the FPS: build validity, playable loop, asset consistency, and maintainability. For the trailer: footage provenance, gameplay coverage, edit continuity, and narration faithfulness. For the spacer: dimensional accuracy, tolerance, printability, fit, and safe use.
4. Reproducibility
- success rate across fresh runs;
- sensitivity to prompt wording;
- sensitivity to environment changes;
- whether another team can repeat the result from the disclosed inputs.
An execution trace can support the process family. OpenTelemetry describes traces as end-to-end journeys composed of spans with timestamps, status, attributes, events, and links. That model is useful for multi-tool agents because a single final artifact otherwise hides where time, retries, and failures accumulated. See the OpenTelemetry observability primer.
Choose replications by information gain and consequence
The matrix should help decide what to test, not crown a winner. A practical replication queue can use two questions:
- What uncertainty would this reproduction remove? A well-instrumented trailer run may reveal whether cross-application orchestration is real. Repeating another visually similar clip may add little.
- What happens if the output is wrong? A game prototype can fail harmlessly in a sandbox. A fabricated part that interacts with hardware deserves measurement and human inspection before use.
That makes physical-world tasks a separate lane, not bonus points for being impressive. A CAD file that looks plausible on screen has not yet passed a fit test. Likewise, “worked for 48 hours” is not automatically a positive metric; duration may reflect persistence, wasted retries, or the legitimate size of the task. The trace must explain it.
Preserve attribution all the way down
Min Choi curated the directory, while the underlying claims and artifacts belong to the linked creators. Keep both relationships in the dataset: discovered_via for the curator and created_by for each original demo. Link to the source instead of downloading and republishing third-party media.
The viral thread’s strongest idea should remain intact: one sentence can now lead to a concrete, inspectable work product. The evaluation matrix adds the question the feed cannot answer: what kind of work product, supported by what evidence, repeatable under which conditions, and risky in what way? That lets ten heterogeneous demos become a useful test backlog without pretending they belong on one leaderboard.
Source list
- Min Choi — original Grok 4.6 compilation thread
- KettlebellDan — original game-trailer example
- NIST — AI test, evaluation, validation and verification (TEVV)
- NIST AI RMF Core — Measure
- OpenTelemetry — Observability primer
AI-assistance disclosure: This article was drafted with AI assistance and manually source-checked and edited. The evaluation schemas are proposals, not implemented or benchmarked results.
Top comments (0)