How I Track AI Video Prompt Experiments in Markdown and JSON

miao cunhui — Sat, 20 Jun 2026 12:42:34 +0000

AI video testing gets messy fast. A creator starts with one product image, tries three prompts, switches models, changes the camera instruction, and then forgets which version produced the best clip.

This is a simple workflow for tracking AI video prompt experiments with Markdown and JSON. It is technical enough for people who like structured files, but the purpose is still creative: make better clips with fewer blind retries.

The Problem

Most AI video failures are hard to learn from because the test notes are incomplete.

Common gaps:

the source image is not recorded
the prompt version changes without a note
the model changes halfway through the test
failures are described as "bad" instead of labeled
the accepted clip is saved without the reason it won

If you want repeatable results, each test needs a small amount of structure.

Folder Structure

ai-video-tests/
  product-bottle/
    source-notes.md
    prompts.md
    runs.json
    review.md
  ugc-hook/
    source-notes.md
    prompts.md
    runs.json
    review.md

This keeps one creative job in one folder. A "job" can be a product ad opener, UGC hook, same-prompt model comparison, explainer background, or product detail close-up.

Source Notes

Use source-notes.md to describe the input before writing prompts.

# Source Notes

Clip job: 5-second vertical product ad
Source image: front-facing product bottle on white background
Target platform: TikTok / Reels
Must preserve: bottle shape, cap, label, color
Caption space: top third
Known risk: small label text may distort

This stops the prompt from carrying all the context. The image and the job are part of the test.

Prompt Versions

Use prompts.md to keep each prompt version intentional.

## v1 Baseline

Make this product image into a clean product video.

## v2 Motion-first

Slow push-in toward the product. Soft light moves across the surface.
Keep the product centered and stable.

## v3 Production prompt

Create a 5-second vertical product ad from the provided image.
Slow push-in toward the product while soft studio light moves across the surface.
Keep the product shape, cap, label, color, and position unchanged.
Leave clean space near the top for a short caption.
No extra objects, no label distortion, no melted edges, no background replacement.

The important part is not that v3 is longer. It is better because it separates the clip job, motion, preservation rules, framing, and negative constraints.

Run Log

Use runs.json when you want sortable records.

[
  {
    "runId": "product-bottle-001",
    "clipJob": "vertical product ad opener",
    "sourceImage": "product-bottle-front.jpg",
    "promptVersion": "v1",
    "model": "Model A",
    "aspectRatio": "9:16",
    "score": 52,
    "failureLabels": ["product-drift", "motion-too-static"],
    "decision": "retry",
    "nextPromptChange": "add preservation constraints and immediate motion"
  },
  {
    "runId": "product-bottle-002",
    "clipJob": "vertical product ad opener",
    "sourceImage": "product-bottle-front.jpg",
    "promptVersion": "v3",
    "model": "Model A",
    "aspectRatio": "9:16",
    "score": 81,
    "failureLabels": ["minor-label-softness"],
    "decision": "keep",
    "nextPromptChange": "add label text in editing instead of regenerating"
  }
]

You can use a spreadsheet instead. JSON is just convenient if you want to reuse the records later.

Scoring Rubric

Criterion	Weight	What to Check
Subject fidelity	30%	Product, character, or object stays recognizable
Motion usefulness	20%	Motion starts early and supports the clip job
Prompt adherence	20%	Camera, framing, and constraints are followed
Editability	20%	Clip can be cropped, captioned, or placed in a sequence
Retry cost	10%	Number of generations needed for a usable result

Do not score style alone. A clip can look impressive and still fail the job if the product label changes or the framing leaves no room for captions.

Failure Labels

Use repeatable labels instead of vague notes.

product-drift
character-drift
background-takeover
text-distortion
motion-too-static
camera-too-fast
prompt-ignored
hard-to-edit
over-stylized

These labels make it easier to see patterns. If product-drift appears in every run, changing models may help more than adding another style phrase. If motion-too-static appears often, the motion instruction may need to start earlier and be more specific.

Review Notes

Use review.md to summarize the decision.

# Review

Best run: product-bottle-002
Why it won: product stayed stable, motion started early, caption space remained usable.
Manual edit needed: add real label-safe text in post-production.
Rejected runs: v1 was too static; v2 improved motion but changed the label.
Next test: compare the same v3 prompt across two other models.

The review is the part your future self will actually read.

Related LumiYing Resource

Image-to-video workflow: https://lumiying.com/tools/image-to-video

Conclusion

A small prompt log is enough to turn AI video generation from guessing into a repeatable creative workflow. Save the source notes, version the prompt, label the failure, score the output, and write down why a clip was kept or rejected.

DEV Community: miao cunhui