Yes, Sora can generate video from a still image. The useful distinction is that it generates a scene over time: it can infer camera movement, object motion, lighting changes, and content that the original image never showed.
That makes image-to-video useful for short shots, but it also explains the failures. A camera move around an object requires the model to invent its hidden surfaces. A small head turn requires consistent facial geometry across frames.
I approach these renders as constrained shots: one reference image, a specific action, a deliberate camera move, and a short duration. The tighter that brief, the easier the result is to evaluate and refine.
Decide what the reference should preserve
Sora’s image-driven workflow uses an image alongside a text prompt. The image supplies visual cues such as composition, subjects, colors, and lighting; the prompt describes how the scene should develop.
There are two useful ways to approach this:
- Animate the existing composition. Preserve the scene and request limited motion: rising steam, a slow push-in, or subtle background parallax.
- Use the image as a starting point. Allow changes in pose, environment, or action, accepting that the output may depart further from the original.
Remixing operates on an existing generated video. It is useful once a shot is close and the remaining change is narrow. Depending on the product surface, Sora’s creative tools also support extending or stitching clips and reusing consent-controlled characters.
Sora 2 introduced improvements in physical realism, controllability, and synchronized audio. Those capabilities help image-derived shots, but they do not guarantee exact preservation of every detail.
What the model has to infer
A single image does not specify depth, hidden geometry, or how objects behave. Image-to-video systems must infer enough scene structure and motion to synthesize temporally coherent frames.
Depth estimation, learned motion dynamics, and diffusion or transformer-based synthesis are useful concepts for understanding the problem. They should not be treated as a verified description of Sora’s internal implementation.
The practical implication is straightforward: ambiguous depth and complex interactions give the model more opportunities to make incompatible guesses.
Build around an asynchronous video job
The API workflow is a job lifecycle:
- Submit a prompt and image reference.
- Store the returned video ID.
- Poll for completion, or consume a completion/failure webhook.
- Download the generated content.
The core HTTP operations are:
| Operation | Endpoint |
|---|---|
| Create a video | POST /videos |
| Retrieve its status | GET /videos/{id} |
| Download the result | GET /videos/{id}/content |
Webhook event types include video.completed and video.failed.
The models discussed here are sora-2 and sora-2-pro. Documented short-duration options include 4, 8, and 12 seconds; use the values supported by your endpoint. Writing “six seconds” in a prompt does not make seconds=6 a supported API parameter.
If I need several model providers behind one integration, a unified API such as CometAPI can be relevant. Its credentials, base URL, and endpoint compatibility still need to match that provider’s documentation. The example below uses the official OpenAI Python client directly.
Submit an image and download the MP4
Install the client:
pip install openai
Set OPENAI_API_KEY in the environment. This example assumes still_photo.jpg matches the requested output resolution and is eligible for the image-reference workflow.
import time
from pathlib import Path
from openai import OpenAI
client = OpenAI()
image_path = Path("still_photo.jpg")
prompt = (
"Create an 8-second cinematic shot using the reference image. "
"Preserve the subject, composition, colors, and existing props. "
"Hold the camera static for the first 0.5 seconds, then slowly "
"dolly forward with subtle background parallax. "
"Keep warm early-evening lighting consistent. "
"No added characters or objects. Quiet ambient sound, no dialogue."
)
with image_path.open("rb") as image:
job = client.videos.create(
model="sora-2",
prompt=prompt,
input_reference=image,
seconds="8",
size="1280x720",
)
print("Job created:", job.id)
deadline = time.monotonic() + 1800
while job.status in ("queued", "in_progress"):
if time.monotonic() >= deadline:
raise TimeoutError(
f"Stopped polling video {job.id}; retrieve its status later."
)
print(f"{job.status}: {job.progress}%")
time.sleep(3)
job = client.videos.retrieve(job.id)
if job.status != "completed":
raise RuntimeError(f"Video {job.id} ended as {job.status}: {job.error}")
content = client.videos.download_content(job.id)
content.write_to_file("sora_output.mp4")
print("Saved sora_output.mp4")
The 30-minute polling limit is an application choice, not a service completion guarantee. Reaching it stops this client’s polling; it does not cancel the server-side job.
The parameters worth making explicit are:
-
input_reference: the image that anchors generation. -
prompt: action, camera behavior, timing, lighting, and optional audio. -
seconds: a supported duration value. -
size: a supported output resolution.
I would persist the job ID before doing anything else in a service integration. Generation and downloading are separate operations, so a client disconnect should not force another render.
Also, check SDK method names against the installed client. An illustrative files.upload(..., purpose="video.input") call or videos.get() call should not be assumed to exist. The example passes the image directly and uses videos.retrieve().
Write a shot brief with explicit timing
“Make this image move” leaves nearly every meaningful decision to the model. I prefer a prompt that separates framing, action, and timing.
A useful brief covers five things:
| Part | What to specify |
|---|---|
| Framing | Close-up or wide shot, camera height, lens feel, subject placement |
| Action | Which object moves, in which direction, and how far |
| Timing | Initial hold, movement, pauses, and final state |
| Lighting | Existing light to preserve or an intentional change |
| Audio | Ambient sound, effects, or dialogue when appropriate |
For example:
Use the reference image as the starting composition.
Close-up, 50mm lens feel, shallow depth of field.
Preserve the cup, tabletop, background colors, and existing lighting.
Over 8 seconds:
- Hold the camera static for the first 0.5 seconds.
- Slowly dolly forward for the next 2 seconds.
- Keep the camera still for the remainder.
- Let a thin stream of steam rise naturally from the cup.
Warm light, soft shadows, no new objects or people.
Quiet room ambience, no music or dialogue.
The timing communicates intent; it is not a frame-accurate animation timeline.
Camera verbs help distinguish different requests. A pan rotates the view; a dolly moves the camera through space. “Dolly forward with slight parallax” gives clearer direction than “cinematic movement.” Similarly, a push-in is not meaningfully specified in degrees; degrees describe rotation.
I also name what should stay fixed. Existing props, clothing colors, background layout, and light direction are useful anchors. If an element can change, say so explicitly.
Start with one source of motion
For a first pass, I would choose either a camera move or a subject action. Combining a moving camera, a turning subject, new objects, and changing lighting makes diagnosis harder.
Once a restrained render works, add complexity incrementally. Natural movement and stylized stop-motion are different targets, so state which one you want.
Use remix for focused changes
When the composition and movement already work, a narrow remix gives the model a smaller edit to attempt. It can help retain continuity, though I would not assume every remix will be faster or more stable than regeneration.
The official JavaScript client exposes a dedicated remix operation:
npm install openai
Save this as remix.mjs and provide OPENAI_API_KEY plus the ID of a completed video in SOURCE_VIDEO_ID:
import OpenAI from "openai";
const client = new OpenAI();
const videoId = process.env.SOURCE_VIDEO_ID;
if (!videoId) {
throw new Error("Set SOURCE_VIDEO_ID to a completed video ID.");
}
const remix = await client.videos.remix(videoId, {
prompt:
"Keep the scene, camera movement, lighting, and timing unchanged. " +
"Change only the monster's color to bright orange.",
});
console.log("Remix started:", remix.id);
Run it with:
node remix.mjs
The returned job still needs status tracking and downloading. Provider wrappers may expose different remix parameters; do not assume that a remix_video_id field on a create request is interchangeable with the official SDK method.
I would change color and add an extra blink in separate iterations. That makes it easier to identify which edit disturbed the shot.
Diagnose the failure before changing the prompt
Several different problems can look like “the render did not work.”
| Symptom | Likely issue | Next step |
|---|---|---|
| Immediate rejection | Input, policy, or request validation | Inspect the API error before retrying |
| Warped hands or objects | Inferred geometry breaks during motion | Reduce movement and interactions |
| Flickering details | Temporal inconsistency | Simplify the camera move or shorten the clip |
| Unexpected objects or actions | The model extrapolates beyond the brief | Specify preserved elements and smaller action steps |
| A nearly correct shot deteriorates after editing | Too many simultaneous changes | Remix one property at a time |
For early experiments, sora-2 is a reasonable starting point. Testing sora-2-pro can be useful when quality is insufficient, but a model change does not remove the need for a manageable shot.
If an action keeps drifting, split the sequence into smaller jobs and assemble them in an editor. For compositing workflows, clean passes can be easier to control than a single ambitious generation.
Account for likeness restrictions and provenance
Real-person likenesses and copyrighted characters are subject to restrictions. Sora’s character/cameo workflows include consent controls, and input rules can differ between the consumer app and API. A human-face upload that works in one workflow should not be assumed eligible in another.
Policy failures need to be distinguished from runtime failures. Read the returned error instead of repeatedly adjusting unrelated prompt wording.
OpenAI described Sora’s launch outputs as carrying visible watermarks and embedded C2PA provenance metadata. Export behavior can depend on the current product and policy, so check the actual output requirements before planning delivery.
Reported concerns also include stereotyping, biased representation, and convincing false footage. For published work, I would inspect the generated people, context, and implied events as closely as the visual artifacts.
For subtle movement and short visual concepts, an image reference provides a useful starting constraint. For demanding face animation, complicated physical interactions, or VFX delivery, I would budget for editing and compositing alongside generation.
Originally published at cometapi.com
Top comments (0)