DEV Community

Ramandeep Singh
Ramandeep Singh

Posted on

AI Video creation - Models Research

#ai #llm

Runway ML

Overview

Aspect Details
API Base https://api.dev.runwayml.com/v1
Auth Authorization: Bearer $RUNWAYML_API_SECRET
Version Header X-Runway-Version: 2024-11-06 (required)
Task Model Async: submit → get task ID → poll GET /tasks/{id} until done
Docs https://docs.dev.runwayml.com/
API Key https://docs.dev.runwayml.com/guides/setup

Endpoints

Text-to-Video

  • Endpoint: POST /v1/text_to_video
  • Models: gen4.5, veo3.1, veo3.1_fast, veo3
  • Note: gen4_turbo is not supported for text-to-video

Image-to-Video

  • Endpoint: POST /v1/image_to_video
  • Models: gen4.5, gen4_turbo, veo3.1, gen3a_turbo, veo3.1_fast, veo3
  • Input: promptImage (HTTPS URL, Runway URI, or data URI), promptText, ratio, duration

Models

gen4.5

  • Modes: Text-to-video, Image-to-video
  • Duration: 2–10 seconds
  • Ratios: 1280:720, 720:1280, 1104:832, 960:960, 832:1104, 1584:672
  • Notes: Flagship model; omit promptImage for text-only

gen4_turbo

  • Modes: Image-to-video only
  • Duration: 2–10 seconds
  • Notes: Faster than gen4.5; requires an input image

veo3.1

  • Modes: Text-to-video, Image-to-video
  • Duration: 2–10 seconds
  • Ratios: 1280:720, 720:1280
  • Notes: Google Veo 3.1–based model

veo3.1_fast

  • Modes: Text-to-video, Image-to-video
  • Notes: Faster variant of veo3.1

veo3

  • Modes: Text-to-video, Image-to-video
  • Notes: Base Veo 3 model

gen3a_turbo

  • Modes: Image-to-video only
  • Notes: Earlier-generation turbo model

Task Lifecycle

  1. Submit: POST to /text_to_video or /image_to_video → returns { "id": "task-uuid" }
  2. Poll: GET /v1/tasks/{id} every 5+ seconds
  3. Statuses: submitted, in queue, SUCCEEDED, FAILED
  4. Output: On SUCCEEDED, task.output[0] is the video URL (expires in 24–48 hours)

Input Formats

API Parameters

API Field Type Example
model string gen4.5, gen4_turbo
promptText string "A serene mountain at sunrise"
promptImage string HTTPS URL, Runway URI, or data URI
ratio string 1280:720
duration integer 2–10

Pricing

Credits are the billing unit. 1 credit = $0.01. Credits are purchased in the developer portal; sales tax may apply.

Video Generation (credits per second)

Model Credits/sec $ per 5-sec video $ per 10-sec video
gen4.5 12 $0.60 $1.20
gen4_turbo 5 $0.25 $0.50
gen4_aleph 15 $0.75 $1.50
gen3a_turbo 5 $0.25 $0.50
act_two 5 $0.25 $0.50
veo3 40 $2.00 $4.00
veo3.1 (audio) 40 $2.00 $4.00
veo3.1 (no audio) 20 $1.00 $2.00
veo3.1_fast (audio) 15 $0.75 $1.50
veo3.1_fast (no audio) 10 $0.50 $1.00

Example: A 5-second gen4_turbo video = 25 credits = $0.25.

Sources

fal.ai

Overview

Aspect Details
Queue Base https://queue.fal.run
Auth Authorization: Key $FAL_KEY
Task Model Queue: submit → poll status → GET result
Docs https://docs.fal.ai/
API Key https://fal.ai/dashboard/keys
Pricing Pay-per-use; see https://fal.ai/pricing

Pricing

fal.ai bills by output (per second or per video). Pay only for computing consumed. Check fal.ai/pricing and individual model pages for current rates.

Video Models (representative; verify on fal.ai)

Model Unit Price Output per $1
Wan 2.5 per second $0.05 ~20 seconds
Kling 2.5 Turbo Pro per second $0.07 ~14 seconds
Veo 3 per second $0.40 ~3 seconds
Veo 3 Fast per second $0.10–$0.15 ~7–10 seconds
Ovi per video $0.20 ~5 videos
Pixverse v5.5 (5s) per video $0.15–$0.40 ~2–6 videos

Pixverse v5.5 (by resolution): $0.15 (360p/540p), $0.20 (720p), $0.40 (1080p) for 5-second videos; audio +$0.05.

Pika (on fal.ai): ~$0.20–$0.40 per video depending on endpoint.

Minimax, Kling v1, Luma Dream Machine, Pixverse v4.5: Per-model pricing varies; check the model pages or dashboard.

GPU Compute (custom deployments)

GPU VRAM Price/hour Price/second
H100 80GB $1.89 $0.0005
H200 141GB $2.10 $0.0006
A100 40GB $0.99 $0.0003

Queue API Flow

  1. Submit: POST https://queue.fal.run/{model_id} with { "input": { ... } } → request_id, status_url, response_url
  2. Poll: GET {status_url} every 5+ seconds
  3. Statuses: IN_PROGRESS, COMPLETED, SUCCEEDED, FAILED
  4. Result: GET {response_url} → data.video.url

Models (Image-to-Video)

All models below are image-to-video only. fal.ai hosts multiple providers; each has different capabilities and pricing.

fal-ai/minimax-video/image-to-video

Aspect Details
Provider MiniMax
Input prompt (string), image_url (URL or data URI)
Output video.url
Optional prompt_optimizer (boolean, default true)
Notes Solid default for image animation; good quality/speed balance

Schema:

{
  "input": {
    "prompt": "A woman walks through Tokyo neon streets",
    "image_url": "https://... or data:image/jpeg;base64,...",
    "prompt_optimizer": true
  }
}
Enter fullscreen mode Exit fullscreen mode

fal-ai/kling-video/v1/standard/image-to-video

Aspect Details
Provider Kling AI (Kuaishou)
Input prompt, image_url
Notes Kling 1.0–based; Kling 3.0 (o3) available elsewhere on fal
Alternate fal-ai/kling-video/o3/standard/image-to-video (if listed)

fal-ai/luma-dream-machine/image-to-video

Aspect Details
Provider Luma AI
Input prompt, image_url
Notes Luma Dream Machine v1.5; known for photorealism

fal-ai/pika/v2.2/image-to-video

Aspect Details
Provider Pika Labs
Input prompt, image_url
Resolution Up to 1080p
Notes Pika 2.2; supports camera movement hints (e.g. [Pan left], [Zoom in])
Pikaframes Separate endpoint for multi-keyframe storyboards
Pikascenes fal-ai/pika/v2.2/pikascenes for multi-reference images

fal-ai/pixverse/v4.5/image-to-video

Aspect Details
Provider Pixverse
Input prompt, image_url
Notes High-quality clips from text + image; v4.5 is a recent release

Input: image_url

fal.ai accepts:

  • HTTPS URL: Publicly accessible image URL
  • Data URI: data:image/jpeg;base64,{base64} — works for local files; large payloads may affect latency

Output Structure

Typical response shape:

{
  "data": {
    "video": {
      "url": "https://fal.media/files/.../output.mp4"
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Some models may return video as a string URL directly.

Other fal.ai Video Models

  • Text-to-video: Ovi, Pixverse v5 (text-to-video), AnimateDiff
  • Vidu: Image-to-video
  • Kling 3.0 / O3: Newer Kling models; check fal.ai model list for current IDs

Sources

Top comments (0)