Field notes from October 2026. Not a benchmark: one project, real logs, honest mistakes.
I'm building a fitness app in the style of Gravl. It needs 570 exercises, and each exercise needs a card: three frames (start, middle, end) showing the same character doing the movement correctly, with the camera fixed.
I don't write code. I direct AI models that do. Over about ten days of evenings I built what I thought was a production line: image models to draw, AI judges to check, a 3D pipeline in Blender, a job queue, a live dashboard, and two agent platforms (Hermes Agent and DeepSeek Harness) passing work to each other through files.
The judges approved 490 cards. Then I opened them myself at full size and checked a random sample. Machines changed shape between frames. Hands floated next to the bar. Feet slid. The character's face changed. My estimate is that around 80% of the "approved" cards would have to be thrown away.
This post is about why, and about what each model was actually good and bad at. If you use any of these models, some of this might save you a week.
The short version
At no point did the movement come from a real source. Every stage was an AI guessing what a human body does, followed by another AI checking the guess. When the checker shares the generator's blind spot, errors don't cancel out. They compound.
Everything below is a variation on that sentence.
What I tried, in order
Early experiments on free Kaggle GPUs. Wan 2.2 I2V A14B (GGUF Q4) for short exercise videos got the motion roughly right, but the character drifted from frame to frame. SDXL with IP-Adapter for character consistency didn't work.
Image model draws the card from the exercise name. Fast, consistent outfit and identity. But it guessed the movement. About 40% of the first batch showed the wrong movement or the wrong end position.
More judges, more process. Two AI judges, a "DNA" spec for the character, a library of machine references, a production "belt". Quality didn't move. Complexity did.
A 3D character. I generated a 3D model from character art, rigged it in Blender, and had AI agents write the joint angles for each pose as JSON. The camera became perfectly stable. But the hands didn't grip the bar, and bars went through the torso.
Make the 3D render photoreal. On Modal (H100) I tried an image-edit model and a ControlNet img2img setup. The first rewrote the pose. The second preserved the pose perfectly, including every mistake in the render.
Back to the original image line, now fully automated. A manager agent on one platform wrote instructions; a worker agent on another platform executed them. The automation worked. The quality didn't, and a runaway process started regenerating all 442 defined exercises because of one unsupported command-line flag.
I looked at the "approved" set myself. That ended it.
Field notes, model by model
Everything here comes from my own logs, on my own tasks. Your results will differ.
Image generation: Meta Muse Image 1.0
Good at: style, outfit, and keeping a recognizable character across many images. Fast.
Missed: consistency between the three frames of one card. The camera rotated from side view to front view, the person turned to face the other way, and a machine looked different in each frame. It drew the movement from the name, so less common exercises came out as something else (a wrist curl became a biceps curl).
Lesson: an image model is a renderer, not a source of truth for motion.
Image generation (backup): Seedream 4.0 (BytePlus, seedream-4-0-250828)
Good at: a free credit tier made it a useful fallback when my main image model's weekly quota ran out.
Missed: of 11 cards it produced on top of 3D renders, I approved 2 at full size; 9 were bad. Its outputs were saved under the same file names as the main model's, which later fooled my strongest model about where they came from.
Lesson: record which model produced every file. File names lie; metadata (here, the raw 2048×2048 size) doesn't.
Instruction-following agent: Meta Muse Spark 1.3
Good at: reading a file-based instruction and executing a long job end to end: running the command, collecting results, writing valid JSON, reporting in one line. Large quota, so great for volume work.
Missed: as a visual judge it was weak. When we tried to train it as the card judge, it scored 69% on our test and had memorized examples rather than learned the rule. As an executor it once tried a command-line flag that didn't exist instead of reading the script's docs. That single flag started the runaway job.
Lesson: give it execution, text and data work. Don't give it the final say on visual quality. Forbid undocumented flags in its instructions.
Reasoning model: Meta Muse Wonder 1.0
Good at: second opinions on hard decisions.
Missed: as a per-card judge it was too expensive in quota for the value it added, so I moved it to rare consultations only.
Lesson: use the big model where a decision is expensive, not on every item.
Lightweight vision judge: Gemini 3.5 Flash-Lite
Good at: cheap first-pass screening.
Missed: it was lenient. It gave 10/10 to a chin-up drawn with an overhand grip (a chin-up uses an underhand grip) and to a clean where the camera rotated between frames.
Lesson: "Score out of 10" invites generosity. If you need a judge, ask it for structured facts (camera angle per frame, grip type, number of people) and let code compare them. Don't ask for a verdict.
Image-edit model: Qwen-Image-Edit-2511 + Lightning 4-step LoRA (lightx2v)
Good at: speed (about 4 seconds per image on an H100) and photoreal texture.
Missed: it "fixed" joint angles I didn't ask it to touch. A skullcrusher (elbows bent) came back as a bench press (arms straight). When I gave it the character reference sheet, it copied the reference's standing pose and even drew floating gloves, because the sheet had gloves drawn separately.
Lesson: an edit model will happily re-pose your subject. Feed it only the image you want preserved, with no busy reference sheet, and expect drift on fine joint angles.
Pose-locked img2img: SDXL (RealVisXL V5.0) + ControlNet canny/depth SDXL 1.0 + Depth-Anything-V2
Good at: keeping the pose exactly, frame for frame, and producing a photoreal result in a few seconds per image.
Missed: it is faithful to the mistakes too. A gray placeholder machine stayed a gray placeholder. A hand that missed the bar in the render still missed it. Prompts over 77 tokens got cut off silently. Asking for "red gloves" bled red onto the skin.
Lesson: ControlNet is only as good as its input. It's a finishing step, not a fix.
Image-to-3D: Hyper3D Gen2 (BytePlus), from four character views
Good at: facial likeness. The best of the versions I generated looked recognizably like the character.
Missed: the texture was baked into one image (you can't change the shorts without repainting the skin), the topology was messy, and the hands were weak.
Lesson: fine as a reference, risky as a production body. A clean parametric base mesh with the likeness transferred on top is safer.
Blender 4.2, directed by AI
What worked: a fixed camera and fully repeatable renders. This alone solved the "camera changes between frames" problem.
What failed: letting an LLM write joint angles. The numbers look plausible and the result is not: hands hover, bars clip through bodies. An auto-rig that guesses joint positions from height slices broke at the shoulders and elbows.
Lesson: don't let an LLM write rotations. Give it targets ("hands on the bar here, feet on the floor") and let inverse kinematics solve the joints. Better still, use real motion capture.
Frontier model: Claude Opus 5.5
Good at: system design and root-cause debugging. It found that a script counted a stale "done" file as success, that two processes were corrupting a JSON file with non-atomic writes, and that 11 "good" cards came from a different model than their labels claimed. It found the last one by checking the raw image size, not the label.
Missed: it approved 49 cards from small preview sheets. At full size, all 49 were broken. It also trusted a label once before checking the evidence.
Lesson: even the strongest model is fooled by thumbnails and labels. Make it look at evidence at full resolution.
Coding models: Claude Sonnet 5.5 and DeepSeek V4.1 Flash
Good at: turning a clear work order into working code. Sonnet 5.5 completed a large batch of changes with 381 passing tests in about 14 minutes; DeepSeek V4.1 Flash built a solid first version of the infrastructure cheaply.
Missed: when DeepSeek was asked to manage (write instructions for other agents), its instructions dropped the safety steps: no pilot run before scaling, no check that a dependency actually worked, no definition of where the output ends up.
Lesson: strong coders aren't automatically good managers. If a model writes instructions for other models, give it a checklist: pilot first, verify dependencies, define done.
Agent platforms: Hermes Agent and DeepSeek Harness
Hermes Agent (running Muse Spark 1.3) was the worker; DeepSeek Harness was the manager. Both could run long jobs, read files and talk to each other through a shared folder. The weak point was never the platforms; it was the quality signal flowing between them.
Infrastructure lessons (Windows, multiple agents)
Two agents writing the same JSON file will eventually corrupt it. Write atomically: write to a temp file, then replace.
On Windows, that replace fails while another process has the file open. Retry with a short backoff, and make readers retry too. After this, 6,000+ concurrent reads produced zero errors.
One writer per file. A lock file for "production is running". A single append-only event log with a source field for every agent. These simple rules worked better than anything clever.
A script that treats "no item list" as "process everything" is a loaded gun. Make "all" an explicit flag.
What I would do differently
Get the motion from a real source first: motion capture or real video. Then render it with whatever character you have. A plain character doing the movement correctly beats a beautiful one doing it wrong.
Check 3 to 5 items at full size before scaling to 500. My most expensive mistake was skipping this, more than once.
Use AI judges for screening, not approval. Final approval is a human looking at full-size images.
Don't add a layer for every problem. Ask whether the new judge, dashboard or agent touches the root cause. Mine didn't.
Check what the market already sells. Many fitness apps license their exercise visuals for a few hundred dollars. I spent ten days building what others buy.
Where AI agents shine, and where they don't (my take)
Agents were excellent at coordination, writing code, executing long jobs, producing structured data, and debugging systems when they had evidence. They were not reliable at verifying physical correctness: anatomy, contact, "is the hand really on the bar". And a second AI checking the first doesn't fix that.
Knowing that boundary was worth more than any single card I produced.
Next in this series: the simple file-based protocol that kept two agent platforms in sync, with the code.
Code and protocol from this project: github.com/emre-ferik/ai-agent-field-notes
Top comments (2)
This is why I think the interface matters as much as the model.
Spot on. The interface for the AI judge was a thumbnail, and that single UI decision caused 49 false approvals. If the AI can't zoom in and see the grip, the model's IQ doesn't matter.