Build an Image-to-Video Shot Planner MCP Server with TypeScript

Turning a still image into a convincing short video is usually less about writing a longer prompt and more about separating creative decisions clearly. The subject needs one primary motion, the camera needs a restrained instruction, and the environment should support rather than compete with the main action.

In this tutorial, we will build a small Model Context Protocol (MCP) server that turns a rough creative brief into two useful outputs:

• a structured image-to-video prompt
• a shot plan with continuity and iteration checks

The server runs over stdio, so it can be used by any MCP client that supports local processes.

Project setup

Create a TypeScript project and install the MCP SDK:

mkdir image-to-video-mcp
cd image-to-video-mcp
npm init -y
npm install @modelcontextprotocol/sdk
npm install -D typescript @types/node

Use ESM in package.json and add a build script:

{
  "type": "module",
  "scripts": {
    "build": "tsc"
  }
}

A minimal tsconfig.json can target modern Node.js:

{
  "compilerOptions": {
    "target": "ES2022",
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "outDir": "dist",
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true
  },
  "include": ["src/**/*.ts"]
}

Create the MCP server

Start with the stdio transport and the request schemas required for listing and calling tools:

import {
  CallToolRequestSchema,
  ListToolsRequestSchema,
  Tool,
} from "@modelcontextprotocol/sdk/types.js";
import { Server } from "@modelcontextprotocol/sdk/server/index.js";
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";

const server = new Server(
  { name: "image-to-video-ai", version: "0.1.0" },
  { capabilities: { tools: {} } },
);

The important design choice is to keep the server deterministic. It does not generate media or request private credentials. It organizes the user's intent into a format that a video model can follow more reliably.

Define a prompt-building tool

The first tool accepts a subject and primary motion, then adds optional production controls:

const buildPromptTool: Tool = {
  name: "image_to_video_build_prompt",
  description: "Build a production-ready image-to-video motion prompt.",
  inputSchema: {
    type: "object",
    properties: {
      subject: { type: "string" },
      motion: { type: "string" },
      environment: { type: "string" },
      camera: { type: "string" },
      lighting: { type: "string" },
      timing: { type: "string" },
      style: { type: "string" },
      avoid: { type: "string" },
    },
    required: ["subject", "motion"],
  },
};

A helper keeps optional fields out of the final prompt when they are empty:

function optionalLine(label: string, value: unknown): string {
  return typeof value === "string" && value.trim()
    ? `${label}: ${value.trim()}`
    : "";
}

The request handler can then assemble the prompt in a stable order:

function buildPrompt(args: Record<string, unknown>): string {
  const subject = String(args.subject || "").trim();
  const motion = String(args.motion || "").trim();

  if (!subject || !motion) {
    throw new Error("subject and motion are required");
  }

  return [
    `Subject: ${subject}`,
    `Primary motion: ${motion}`,
    optionalLine("Environment", args.environment),
    optionalLine("Camera", args.camera),
    optionalLine("Lighting", args.lighting),
    optionalLine("Timing", args.timing),
    optionalLine("Style", args.style),
    optionalLine("Avoid", args.avoid),
    "Preserve subject identity, coherent anatomy, stable geometry, and consistent lighting across every frame",
  ].filter(Boolean).join(". ");
}

This ordering matters. Subject and motion come first because they define the visual objective. Camera and environmental motion follow as supporting instructions. Artifact constraints come last.

Add a shot-planning tool

A second tool should help the user reason about the clip before spending generation credits:

const shotPlanTool: Tool = {
  name: "image_to_video_plan",
  description: "Create a concise shot plan with continuity and iteration checks.",
  inputSchema: {
    type: "object",
    properties: {
      goal: { type: "string" },
      image_description: { type: "string" },
      duration_seconds: {
        type: "number",
        minimum: 2,
        maximum: 30,
        default: 5,
      },
      aspect_ratio: {
        type: "string",
        enum: ["16:9", "9:16", "1:1", "4:3", "3:4"],
        default: "16:9",
      },
      end_frame_description: { type: "string" },
    },
    required: ["goal", "image_description"],
  },
};

The plan should be short enough to scan but specific enough to catch conflicting instructions. A useful response includes:

1. The clip goal and duration.
2. The primary subject motion.
3. One camera instruction.
4. Background motion at lower visual priority.
5. Identity, geometry, and lighting continuity checks.
6. A first-pass iteration strategy.
7. Optional end-frame guidance.

The best first test changes only one variable at a time. If the subject motion is wrong, do not simultaneously change the camera, style, timing, and lighting. Controlled iteration makes failures diagnosable.

Register and call the tools

Return both tools from the list handler:

const tools = [buildPromptTool, shotPlanTool];

server.setRequestHandler(ListToolsRequestSchema, async () => ({
  tools,
}));

Then route calls by tool name and wrap text in the MCP content format:

function textResult(text: string) {
  return {
    content: [{ type: "text" as const, text }],
  };
}

server.setRequestHandler(CallToolRequestSchema, async (request) => {
  const { name, arguments: args = {} } = request.params;

  if (name === "image_to_video_build_prompt") {
    return textResult(buildPrompt(args));
  }

  if (name === "image_to_video_plan") {
    return textResult(createShotPlan(args));
  }

  throw new Error(`Unknown tool: ${name}`);
});

Finally, connect the stdio transport:

const transport = new StdioServerTransport();
await server.connect(transport);

Compile the project with npm run build. During development, test the server through an MCP inspector or configure it directly in your client:

{
  "mcpServers": {
    "image-to-video-ai": {
      "command": "npx",
      "args": ["-y", "mcp-imagetovideoai-server"]
    }
  }
}

Practical prompting rules

A tool like this is most useful when it enforces a few constraints consistently:

• Prefer one dominant subject motion.
• Keep camera movement subtle unless it is the point of the shot.
• Describe environmental motion separately.
• Use time-based language for short clips.
• Preserve labels, faces, hands, and product geometry explicitly.
• Add an end frame only when the transition needs a controlled destination.
• Iterate on one failure mode at a time.

For example, a product shot might specify a slow bottle rotation, a gentle dolly-in, stable label typography, and soft moving reflections. That is easier for a model to interpret than a paragraph containing several unrelated cinematic actions.

Once the plan is stable, you can run it through an Image to Video AI generator and compare model outputs without rewriting the creative brief for every attempt.

What to build next

This server can be extended without turning it into a media-processing backend. Useful additions include:

• reusable prompt presets for products, portraits, and landscapes
• validation for contradictory camera instructions
• locale-aware workspace links
• a prompt comparison tool for two motion strategies
• JSON output for production pipelines
• lightweight tests for required inputs and duration limits

The key is to keep the boundary clear: the MCP server structures decisions, while the video platform performs generation. That separation makes the tool easier to audit, safer to run locally, and useful across different image-to-video models.