DEV Community

miao cunhui
miao cunhui

Posted on

How I Track AI Video Prompt Experiments in Markdown and JSON

#typescript #ai #webdev #tutorial

AI video testing gets messy fast. A creator starts with one product image, tries three prompts, switches models, changes the camera instruction, and then forgets which version produced the best clip.

This is a simple workflow for tracking AI video prompt experiments with Markdown and JSON. It is technical enough for people who like structured files, but the purpose is still creative: make better clips with fewer blind retries.

The Problem

Most AI video failures are hard to learn from because the test notes are incomplete.

Common gaps:

  • the source image is not recorded
  • the prompt version changes without a note
  • the model changes halfway through the test
  • failures are described as "bad" instead of labeled
  • the accepted clip is saved without the reason it won

If you want repeatable results, each test needs a small amount of structure.

Folder Structure 

ai-video-tests/
  product-bottle/
    source-notes.md
    prompts.md
    runs.json
    review.md
  ugc-hook/
    source-notes.md
    prompts.md
    runs.json
    review.md
Enter fullscreen mode Exit fullscreen mode

This keeps one creative job in one folder. A "job" can be a product ad opener, UGC hook, same-prompt model comparison, explainer background, or product detail close-up.

Source Notes

Use source-notes.md to describe the input before writing prompts.

# Source Notes

Clip job: 5-second vertical product ad
Source image: front-facing product bottle on white background
Target platform: TikTok / Reels
Must preserve: bottle shape, cap, label, color
Caption space: top third
Known risk: small label text may distort
Enter fullscreen mode Exit fullscreen mode

This stops the prompt from carrying all the context. The image and the job are part of the test.

Prompt Versions

Use prompts.md to keep each prompt version intentional.

## v1 Baseline

Make this product image into a clean product video.

## v2 Motion-first

Slow push-in toward the product. Soft light moves across the surface.
Keep the product centered and stable.

## v3 Production prompt

Create a 5-second vertical product ad from the provided image.
Slow push-in toward the product while soft studio light moves across the surface.
Keep the product shape, cap, label, color, and position unchanged.
Leave clean space near the top for a short caption.
No extra objects, no label distortion, no melted edges, no background replacement.
Enter fullscreen mode Exit fullscreen mode

The important part is not that v3 is longer. It is better because it separates the clip job, motion, preservation rules, framing, and negative constraints.

Run Log

Use runs.json when you want sortable records.

[
  {
    "runId": "product-bottle-001",
    "clipJob": "vertical product ad opener",
    "sourceImage": "product-bottle-front.jpg",
    "promptVersion": "v1",
    "model": "Model A",
    "aspectRatio": "9:16",
    "score": 52,
    "failureLabels": ["product-drift", "motion-too-static"],
    "decision": "retry",
    "nextPromptChange": "add preservation constraints and immediate motion"
  },
  {
    "runId": "product-bottle-002",
    "clipJob": "vertical product ad opener",
    "sourceImage": "product-bottle-front.jpg",
    "promptVersion": "v3",
    "model": "Model A",
    "aspectRatio": "9:16",
    "score": 81,
    "failureLabels": ["minor-label-softness"],
    "decision": "keep",
    "nextPromptChange": "add label text in editing instead of regenerating"
  }
]
Enter fullscreen mode Exit fullscreen mode

You can use a spreadsheet instead. JSON is just convenient if you want to reuse the records later.

Scoring Rubric

Criterion Weight What to Check
Subject fidelity 30% Product, character, or object stays recognizable
Motion usefulness 20% Motion starts early and supports the clip job
Prompt adherence 20% Camera, framing, and constraints are followed
Editability 20% Clip can be cropped, captioned, or placed in a sequence
Retry cost 10% Number of generations needed for a usable result

Do not score style alone. A clip can look impressive and still fail the job if the product label changes or the framing leaves no room for captions.

Failure Labels

Use repeatable labels instead of vague notes.

product-drift
character-drift
background-takeover
text-distortion
motion-too-static
camera-too-fast
prompt-ignored
hard-to-edit
over-stylized
Enter fullscreen mode Exit fullscreen mode

These labels make it easier to see patterns. If product-drift appears in every run, changing models may help more than adding another style phrase. If motion-too-static appears often, the motion instruction may need to start earlier and be more specific.

Review Notes

Use review.md to summarize the decision.

# Review

Best run: product-bottle-002
Why it won: product stayed stable, motion started early, caption space remained usable.
Manual edit needed: add real label-safe text in post-production.
Rejected runs: v1 was too static; v2 improved motion but changed the label.
Next test: compare the same v3 prompt across two other models.
Enter fullscreen mode Exit fullscreen mode

The review is the part your future self will actually read.

Related LumiYing Resource

Conclusion

A small prompt log is enough to turn AI video generation from guessing into a repeatable creative workflow. Save the source notes, version the prompt, label the failure, score the output, and write down why a clip was kept or rejected.

Top comments (0)