How to Automate AI Video Creation with Claude & Higgsfield

#ai #api #python #automation

TL;DR: Claude writes and manages the prompts, the Higgsfield CLI generates the clips, and DaVinci Resolve's Python API assembles the edit. Everything is driven by one CSV shot list. The constraint that shapes the whole design: Higgsfield's "unlimited" applies to the web UI only — the CLI bills credits on every generation.

This is the hub post for a series documenting a real AI-video pipeline: one I built to produce cinematic marketing clips at scale, then ran overnight against a metered API. Below is the whole architecture, the file format that holds it together, and the constraints that dictated most of the design decisions.

What the pipeline does

The goal is narrow: turn a list of scene descriptions into a folder of finished clips with as little clicking as possible. Three tools, three jobs.

Tool	Role	Produces
Claude	Writes and manages cinematic prompts, and the automation code around them	A prompt library and the batch runner
Higgsfield CLI	Generates clips via Seedance 2.0, Kling 3.0, Nano Banana Pro	Raw video and image files
DaVinci Resolve	Assembles clips into a finished edit via its Python API	The rendered video

Claude does not generate video. It orchestrates: prompt authoring at volume, and the glue code that turns a list into a run. The rendering happens in Higgsfield's models.

How the pieces connect

Everything flows through one structured file — a CSV shot list. That single source of truth is what makes the pipeline reproducible rather than a sequence of remembered steps:

shot list (CSV)  ->  Higgsfield CLI (generate)  ->  clips/ + manifest

                                                        |

                                                        v

                                        DaVinci Resolve (Python) -> final.mp4

The generator walks the list. The editor reads the manifest of finished clips and lays them onto a timeline. No stage depends on a human remembering what came before, which is the only way an overnight run is possible.

The shot list schema

This is the actual header the batch runner expects. It is worth getting right early, because every later stage reads from it:

id,product,category,model,prompt,aspect,duration,mode,resolution,priority

Two columns do more work than they look like they do:

category — video, image or audio. This is what lets the runner split the list across separate worker queues rather than processing everything in one line.
priority — because a run gets interrupted. Ordering by priority means the shots that matter are already done when it does.

model being per-row is deliberate too. A hero shot and a background plate do not need the same model, and as the cost table below shows, pretending they do is expensive.

The constraint that shaped everything: credits vs unlimited

Higgsfield advertises unlimited generation. That is true — in the web UI. The CLI and API always bill credits. Their own pricing page states it plainly:

Unlimited models and Free Generations are accessible only via higgsfield.ai and are not accessible on MCP/CLI, Canvas or Supercomputer.

I confirmed it the expensive way: one Nano Banana Pro image through the CLI dropped my balance from 10 credits to 8.

This matters more than it first appears. Automation means the CLI. So the moment you build a pipeline, you are on the metered path — and every design instinct you would carry over from an unlimited tier is wrong. You cannot over-generate and pick the best. You generate one strong take per shot and regenerate only failures.

Generation	Credits
Seedance 2.0, 1080p 5s	45
Seedance 2.0, 720p 5s	22.5
Kling 3.0 pro, 5s video	12.5
Kling 3.0 std, 5s video	10
Kling 3.0 std, 3s video	6
Nano Banana Pro image, 2K	2

Note the top two rows: dropping Seedance from 1080p to 720p halves the cost. On a library of forty-odd shots that single decision is the difference between one plan tier and the next. Always price a shot before batching it:

higgsfield generate cost seedance_2_0 --duration 5 --resolution 1080p --prompt "test"



  

  

  45 credits — drop to 720p and it is 22.5

Two constraints that dictated the architecture

Beyond cost, two properties of the CLI shaped how the batch runner had to be written. Both are the kind of thing you only discover by running it.

1. The relaxed queue allows one video and one image at a time

On a trial, concurrency is not "how many workers can I spawn" — it is two slots, one per media type. That maps to exactly two worker threads, each draining its own queue sequentially. More workers do not go faster; they just queue behind each other. This is why the shot list has a category column: it is the key the runner splits on.

2. The CLI does not reliably save to disk

higgsfield generate create ... --wait --json blocks until the generation finishes and returns JSON containing a result URL. The output is URL-based, so the pipeline downloads the file itself rather than trusting the CLI to write it.

That one detail has consequences everywhere: the runner needs its own download step, its own retry logic around that download, and a disk-space guard — because an overnight run that fills the volume at 3am fails in a much worse way than one that stops cleanly.

Why these three tools

Claude — prompt engineering at volume, and writing the glue. Forty cinematic prompts that share a house style is a language problem, not a video problem.
Higgsfield — a multi-model aggregator, so one CLI reaches Seedance 2.0 and Kling 3.0 for video, Nano Banana Pro for stills, and text-to-speech, each selected per row.
DaVinci Resolve — free, professional, and scriptable through a Python API, so assembly automates too instead of being a manual timeline drag at the end.

The aggregator point is the one worth dwelling on. Integrating three video models directly means three APIs, three auth schemes and three billing models. One CLI with a model column in a CSV is a materially smaller system.

What a run actually looks like

Three phases, and the first two exist because the third costs money:

Dry run. Walk the shot list and print every command without generating anything. Catches malformed rows and bad model names for free.
Measure one. Generate a single clip and check the credit delta against the estimate. This is how you find out that a row is 45 credits rather than 22.5 before multiplying it by forty.
Full run. Both slots saturated, retries with backoff, resume from the manifest if it dies.

Resume-from-manifest is not optional at this scale. A run that cannot resume is a run you cannot afford to interrupt, and something always interrupts it.

Sizing the spend

A roughly 46-shot cinematic library, one take each — Kling pro for hero shots, Seedance 720p for backgrounds, cheap stills where a still will do — comes to about 750–1,000 credits.

Which makes plan choice straightforward: a 1,200-credit tier covers one full library pass plus around 35 regenerations, which is the right shape for a single project. A 3,000-credit tier only makes sense if you genuinely want two or three complete takes of everything. And if budget is zero and time is not, do it by hand in the web UI on a trial — unlimited, but slow and entirely manual.

The rule that falls out of all of this: explore free in the web UI, then pay for credits only once you automate.

What the rest of the series covers

This post — the architecture, the shot list, and the credit model.
Cinematic prompt formula — the prompt anatomy that produces usable Seedance and Kling output instead of dreamlike mush.
Credits vs unlimited — the full cost table and when each path makes sense.
Python batch queue runner — generating a whole library hands-off, with retries, resume and the two-slot worker model described above.