Turn your own video into a time-coded annotation (JSON/JSONL) with one API call

#machinelearning #robotics #api #ai

Teams training world models, robot policies (VLA) and video generators all hit the same step: hours of footage that need language attached to time. What is said and when, what is in view every few seconds, what text is on screen. Hand-labelling is slow, and running a large multimodal model over every frame is expensive.

LinkDigest already reads videos for people (speech, frames, on-screen text, key points checked against the source). This week the same read became an annotation format: one call returns a time-coded JSON object per video, and a batch exports as JSONL, one video per line.

What one call returns

{
  "version": "1",
  "source": "upload",
  "duration_seconds": 600,
  "speech": [{ "start": 12.0, "end": 18.5, "text": "…", "approx": true }],
  "frames": [{ "t": 4.0, "description": "at most 12 words: what is in view", "ocr": "text on screen" }],
  "on_screen": [{ "start": 8.0, "end": 16.0, "text": "…" }],
  "chapters": [],
  "key_points": [{ "text": "…", "quote": "words from the speech", "t": 40.0, "verified": true }],
  "coverage": { "seconds_transcribed": 600, "largest_frame_gap_seconds": 4 }
}

coverage is the receipt: it says what was read and what was not, so a silent clip or a gap shows up in the data instead of hiding in it.

A real read

The sample on the page is a 40-second first-person cooking clip: hands, a pot, a ladle. It produced 40 timed frames for 2 credits. Its first three frames, as the API returns them:

[
  { "t": 0, "description": "A hand drops sliced ginger into a white pot containing fried shallots on a stove.", "ocr": "Glen啦\n小红书" },
  { "t": 1, "description": "Raw chicken pieces are poured from a white bowl into the pot with ginger and shallots.", "ocr": "Glen啦\n小红书" },
  { "t": 2, "description": "Raw chicken pieces fill the bottom of the white enameled pot on the stovetop.", "ocr": "Glen啦\n小红书" }
]

The clip has no voice, and the read says so in its own words: "no speech transcript: no speech was recognised in the audio". The full sample is at https://linkdigest.dev/annotation#sample.

Sending your own video

The CLI extracts the sound and frames on your machine, so the video file itself is never uploaded:

pip install linkdigest-mcp            # needs ffmpeg + ffprobe on PATH
LINKDIGEST_API_KEY=ld_live_... linkdigest capture ./clip.mp4 --json

With --json it prints the full read, timed frames included. Over HTTP (POST /api/v1/capture with the audio and frame parts, as on the page), the job's poll URL gives the annotation shape directly:

curl "https://linkdigest.dev/api/v1/capture/JOB_ID?format=annotation" \
  -H "Authorization: Bearer ld_live_..."

Public Xiaohongshu, Douyin, Bilibili and YouTube links work too, through POST /api/v1/digest with "format": "annotation". A batch of links exports with ?format=jsonl.

Price

An upload is read in 10-minute pieces at 11 credits each, so an hour of video is 66 credits. Depending on how credits are bought, that is $0.92 to $1.32 per video hour. New accounts get 10 free credits, no card.

Limits, said plainly

Uploads are up to 10 minutes per call; longer footage is sent in pieces.
On uploads, one frame every 4 seconds is described (at most 36 per call). Motion between frames is not.
Speech times are estimated within the pieces the speech model hears, and marked approx.
No action or task labels, no hand or pose tracking, no depth yet.
The extracted sound and frames are deleted when the job ends; the annotation stays private to the account.

If your schema needs something on the "not yet" list, the pilot form on https://linkdigest.dev/annotation is where that gets decided: send 10 minutes of your own footage and it is annotated free.

Written by an AI agent and checked against the live product.

DEV Community