Kimi K3 reads video natively. Claude still caught the same issues

Moonshot AI provided the Kimi API credits used here ($50) for evaluation. They had no part in designing, running, or concluding the test. Everything K3 couldn't do, and the features that didn't work, are in this post too.

Moonshot AI's Kimi K3 takes video files straight through the API. Claude has no video input, so when I want Claude to look at a video, I cut it into frames first. Codex CLI doesn't have a video channel either, even when you point it at K3 (Kimi API: Use K3 in Codex).

I wanted to know how much that difference matters in real work. My test material was a pair of 40-50 second hands-on videos I made for the SSH series on kenimoto.dev /learn/ (Japanese).

The short version:

• K3 and Claude agreed on what was wrong. Where viewers would drop off, the six fixes, where to insert diagrams, even which video to post first: all the same
• The differences were timestamp precision and effort. K3 pinpointed frozen-screen intervals to fractions of a second, and all I did was hand it the file. Claude estimated from frames sampled every 2 seconds, so its timestamps drift by about ±1 second
• K3 doesn't understand audio. It can't hear the narration or catch audio sync problems
• Claude was faster. The same request took K3 363 seconds and Claude 185

The test material: auto-generated hands-on videos

The videos come from handson-cast, a CLI I'm building. You give it a JSON file of steps. It runs each command for real inside Docker containers while vhs records the terminal, then adds narration from OpenAI's text-to-speech and burns in subtitles. It's Python and ffmpeg underneath. The narration and subtitles are in Japanese.

I made two videos:

• Key authentication (55 s): create a key, install the public key, confirm key-only login works, then disable password auth
• Bastion host (47 s): the same key gets rejected by the bastion with ssh -J, but works through ~/.ssh/config

Every line of terminal output was produced by OpenSSH 10.0p2 running in a container. Nothing is staged. When the rejection message changes from (publickey,password) to (publickey), that really happened on camera.

The first cut (v1) was not watchable, though. The first second shows me typing an internal wrapper script, bash /lab/ssh-config.step-01.sh. You can't tell what the video is about. (I made it, and I couldn't tell either.)

Here's v1 (47 s, with audio):

Click the image to open the mp4 (with Japanese narration).

I showed v1 to K3 and Claude and asked what was wrong with it.

Sending video to K3, and what it costs

The K3 API is OpenAI-compatible. You upload the video as a file first, then reference the returned ID with an ms:// prefix.

from openai import OpenAI
client = OpenAI(api_key=KEY, base_url="https://api.moonshot.ai/v1")

f = client.files.create(file=Path("learn-ssh-key-auth.mp4"), purpose="video")
r = client.chat.completions.create(
    model="kimi-k3",
    max_tokens=32000,
    messages=[{"role": "user", "content": [
        {"type": "video_url", "video_url": {"url": f"ms://{f.id}"}},
        {"type": "text", "text": "Why would viewers drop off in the first 3 seconds?"},
    ]}],
)

Public URLs aren't accepted. The official docs describe video as a set of key frames, and higher resolution means more tokens (Kimi API: Vision).

My first measurement used one 55-second, 1920×1080 video:

Item	Measured
Input tokens	40,659 (almost all video)
Output tokens	2,500 (1,912 of them reasoning)
Cost	about $0.16
Time	83.5 s

Costs use $3 per million input tokens and $15 per million output tokens.

Two practical tips from actually running it:

Assume it can't hear anything

For this run I made a version without burned-in subtitles and asked K3 to transcribe the last sentence of the narration. It answered that it couldn't recognize any audio. Not pretending to hear something it can't is exactly what you want from a reviewer. But it means K3 can't see narration content or subtitle-audio drift. It sees the screen and any subtitles burned into it, nothing else.

It reads the screen well, and accurately. It listed the commands and output for all four steps in order, including the 600 / 644 permissions, the change in the rejection message, and an on-screen comment. The only misread was the wrapper script's filename.

Give it a generous output budget

K3 always reasons before it answers. With max_tokens at 300, the reasoning used everything up and the answer came back empty. It thought hard and then left without saying a word. At 8,000, a full video review got cut off in the middle of the first question. For long answers like reviews, I use 32,000.

The comparison setup

K3 and Claude each reviewed the same two v1 videos, under matched conditions:

	K3	Claude
How the video goes in	The file, once	Frames extracted and read as images
Audio	Not understood	Not used (transcription turned off)
Narration script	Step JSON attached	Same JSON attached
Questions	Same wording (5 questions)	Same wording (5 questions)

The Claude side ran as a subagent that never saw K3's answers. I had already read them, so if I'd done the review myself, I would have been anchored. That's not a comparison.

For frame extraction I used /watch from claude-video, and left the frame count and resolution up to Claude. It ended up re-extracting 4 times. For the opening it pulled frames at 0 / 0.5 / 1 / 2 / 3 seconds, then one every 2 seconds for the rest: 34 frames in total.

The five questions:

1. Where would viewers drop off in the first 3 seconds?
2. Up to six improvements, biggest impact first (what to change / why it works / which pipeline stage implements it)
3. If you combined the article's diagrams with the terminal recording, which diagram goes where, for how long?
4. Which video should go to social media first?
5. What's already good?

Result: nearly identical findings

They mostly matched. Lined up side by side, the answers overlap almost completely:

Finding	K3	Claude
Seconds 0-2 show the wrapper script being typed; the topic is unclear	✓	✓
The first narration line introduces the site, not a reason to watch	✓	✓
Could not resolve hostname in the bastion video is a strong hook, buried	✓	✓
Open with the conclusion or a question (cold open)	✓	✓
Cut the typing and the waits between steps	✓	✓
Bigger font, highlight the key output lines	✓	✓
Make a vertical version	✓	✓
Insert the article's diagrams: permissions at ls -l, the before/after at the climax	✓	✓
End on the article's OG image	✓	✓
Post the bastion video first	✓	✓
Show a progress bar along the top	✓
Rewritten narration lines, as actual sentences		✓
The four ssh-copy-id INFO lines are noise		✓

The fixes themselves are standard short-video advice: open with the conclusion, cut silence, keep subtitles short, put the call to action at the end. Either model will give you that. This comparison didn't surface anything only K3 could find. I expected a model that reads video directly to catch something Claude couldn't see. It didn't.

Where they differed: timestamps

The biggest gap was in how precisely each one placed things in time.

K3 wrote things like "the first meaningful output appears at 3.7 seconds" and "the error at 00:03.2 in the bastion video gets buried," sub-second throughout. When I later had it review an improved version, it came back with intervals: "from 30.5 to 36.1 seconds the screen is frozen after the output appears."

Claude's timestamps were estimates from frames 2 seconds apart, and Claude said so at the end of its review: interval boundaries could be off by about ±1 second. You can sample more densely, but then there are more images to read. It's a trade-off.

"Where exactly is the screen frozen?" is hard to answer from a handful of stills. That's where reading the video directly paid off.

Effort and speed

	K3	Claude
My effort	Hand over the file	Decide how to sample frames (Claude redid it 4 times)
Time	363 s	185 s
Size	84,461 input / 10,863 output tokens, about $0.20 (part of the input billed at the cache rate)	111,313 tokens, 40 tool calls

K3 took less effort; Claude was faster. Each has its edge. Because K3 spends time reasoning, each review took 6-8 minutes. Enough time to make a coffee and come back.

The videos really did get better

I made the fixes. Following the reviews, I added six changes to handson-cast:

• Open with a card: "Same key, but ssh -J gets rejected. Why?"
• Cut the wrapper-script typing and the waits between steps
• Type commands out character by character, with a larger font
• Right after the rejection, show the article's diagram full-screen for 4.5 seconds
• Once the output is complete, slowly zoom into the top-left of the terminal
• End with the article's OG image and URL

Here's the final cut (v4, 41 s, with audio):

Click the image to open the mp4 (with Japanese narration).

Frames from the same timestamps, side by side:

The bastion video went from 46.8 to 41.0 seconds, about 6 seconds shorter. I had K3 grade the improved cut against a 10-item checklist for technical short videos. It covers things like a hook in the first 3 seconds, something changing on screen every 1-3 seconds, and a short call to action only at the end. It scored 7 pass, 2 borderline, 1 pass (estimated). The two borderline items were a frozen stretch after the output appears and an overlong subtitle. Both are fixed now.

I'd meant to have K3 research that checklist too, but that didn't work. The Kimi API has a built-in web search tool ($web_search). With K3, though, the second call, the one that returns the search results, failed every time with 400 tokenization failed. With kimi-k2.6 the first search went through, but after that it stopped searching and wrote text that only looked like search calls into its answer. The docs say this feature is scheduled for retirement on 2026-10-20 (Kimi API: Web Search). I gave up. Claude's web search did the research, and I passed the results to K3.

How I'll split the work

Based on this, here's how I plan to use them:

• K3 when I need to know where in a video something happens. Frozen intervals, the moment the screen gets cut off, how long a caption stays up: questions whose answer is a span of time. I don't have to think about how to sample frames
• Either one for deciding how to fix it. The improvement ideas didn't differ
• Not K3 for anything involving audio. For narration content or sync, I prepare a transcript separately and compare against it
• No confidential screens. The video goes to Moonshot AI's servers. These videos were going public anyway, so that was fine; internal screen recordings need different handling

It's a small comparison: two videos, one set of questions, nothing statistical. Still, it gave me a working answer. Reading video directly didn't change the judgment; it changed timestamp precision and effort. Next I want to plant deliberate defects in a video and count how many each model finds.

References

• MoonshotAI/Kimi-K3 (GitHub): model card. 2.8T total parameters, MoE with 104B active per token, 1M context
• Kimi API: Vision models: how video input works
• Kimi API: Web Search: $web_search and its retirement date
• bradautomates/claude-video: the /watch tool used for Claude's frame extraction
• charmbracelet/vhs: terminal recording
• SSH: five ways to reach a remote machine, compared on seven axes (Japanese): the /learn/ series these videos were made for