DEV Community

Mason K
Mason K

Posted on

Write a portability audit for your video provider before you need one

TL;DR

We're building a small Node script that audits your video library for portability: can you get the source file back, is playback standard HLS, and how many places is the provider's ID buried. It writes a CSV. Run it once a quarter and you'll never be surprised by a migration estimate again.

Most of us pick a video API by reading pricing pages and running an upload benchmark. Nobody checks the exit until they're already trying to leave, at which point the answer is expensive and non-negotiable.

This is a small thing you can run today. It's a provider-agnostic script with a thin adapter per platform, and the worked example uses FastPix because its source-retention flag is explicit in the API, which makes for a clear demo. The pattern moves to Mux, Cloudflare Stream or api.video by writing a different adapter.

What we're checking

Three questions, in order of how much they'll cost you:

  1. Source retention. Does the provider still hold your original upload, or only the transcoded renditions? If it's renditions-only, a migration re-encodes from output and you lose quality permanently.
  2. Playback portability. Is the playback URL a standard HLS manifest any player can consume, or a proprietary embed?
  3. ID sprawl. How many of your own tables carry the provider's media and playback IDs? Every one is a migration write path.

1. Project setup 🛠️

Node 20.x or newer, no dependencies beyond what ships with it.

mkdir video-portability-audit && cd video-portability-audit
npm init -y
npm pkg set type=module
node --version
Enter fullscreen mode Exit fullscreen mode
v22.14.0
Enter fullscreen mode Exit fullscreen mode

Credentials go in the environment, never in the file:

# .env.example : copy to .env, never commit .env
FASTPIX_TOKEN_ID=your_access_token_id
FASTPIX_SECRET_KEY=your_secret_key
Enter fullscreen mode Exit fullscreen mode

⚠️ Note: the script only ever issues GET requests. It reads your library and writes a local CSV. It will not modify or delete anything.

2. The adapter interface

Every provider adapter answers the same three questions for one media ID. Keeping this interface narrow is what makes the script portable across platforms.

// src/adapter.js
/**
 * @typedef {Object} AuditResult
 * @property {string} mediaId
 * @property {boolean|null} sourceRetained   - can we get the original file back?
 * @property {boolean|null} standardHls      - is playback a plain .m3u8?
 * @property {string} playbackUrl
 * @property {string} notes
 */

export class Adapter {
  /** @returns {Promise<AuditResult>} */
  async audit(mediaId) {
    throw new Error('not implemented');
  }
}
Enter fullscreen mode Exit fullscreen mode

3. The FastPix adapter

FastPix uses HTTP Basic auth with the Access Token ID as the username and the Secret Key as the password, so we can build the header with Buffer:

// src/providers/fastpix.js
import { Adapter } from '../adapter.js';

const API = 'https://api.fastpix.com/v1';

export class FastPixAdapter extends Adapter {
  constructor(tokenId, secretKey) {
    super();
    this.auth = 'Basic ' + Buffer.from(`${tokenId}:${secretKey}`).toString('base64');
  }

  async #get(path) {
    const res = await fetch(`${API}${path}`, {
      headers: { Authorization: this.auth, Accept: 'application/json' },
    });
    if (!res.ok) {
      throw new Error(`${res.status} ${res.statusText} on ${path}`);
    }
    return res.json();
  }

  async audit(mediaId) {
    // The create response returns `data` as an OBJECT, not an array.
    // Older docs showed an array; if you wrote a parser against those, it breaks here.
    const { data } = await this.#get(`/on-demand/${mediaId}`);

    const playbackId = data.playbackIds?.[0]?.id ?? null;
    const playbackUrl = playbackId
      ? `https://stream.fastpix.com/${playbackId}.m3u8`
      : '';

    return {
      mediaId,
      // sourceAccess is set at ingest time on the create call.
      // If it wasn't enabled then, the original is not retrievable now.
      sourceRetained: data.sourceAccess ?? null,
      standardHls: Boolean(playbackUrl.endsWith('.m3u8')),
      playbackUrl,
      notes: data.status ?? '',
    };
  }
}
Enter fullscreen mode Exit fullscreen mode

💡 Tip: sourceAccess is a create-time decision. That's the important bit for this audit. If it defaulted off across your whole library, this script will tell you today rather than during a migration in 2028.

4. Feeding it media IDs

Here's a design choice worth explaining, because it looks like a limitation and isn't.

The script takes media IDs from your database, not from a list-everything API call. Two reasons. First, you already store these IDs, because you need them to render a page. Second, and more usefully: the IDs your application knows about are the ones that matter. An asset sitting in the provider's dashboard that nothing in your product references is not a migration problem. A row in your videos table is.

// src/ids.js
import { readFile } from 'node:fs/promises';

// One media ID per line. Generate it however you like, e.g.:
//   psql -At -c "select provider_media_id from videos where deleted_at is null" > ids.txt
export async function loadIds(path = 'ids.txt') {
  const raw = await readFile(path, 'utf8');
  return raw.split('\n').map((l) => l.trim()).filter(Boolean);
}
Enter fullscreen mode Exit fullscreen mode

5. The runner, with concurrency and retries

Rate limits are the thing that turns a five-minute script into a two-hour one. A small concurrency cap plus backoff on 429 handles it:

// src/run.js
import { writeFile } from 'node:fs/promises';
import { loadIds } from './ids.js';
import { FastPixAdapter } from './providers/fastpix.js';

const CONCURRENCY = 4;

async function withRetry(fn, attempts = 4) {
  for (let i = 0; i < attempts; i++) {
    try {
      return await fn();
    } catch (err) {
      const is429 = String(err.message).startsWith('429');
      if (!is429 || i === attempts - 1) throw err;
      const waitMs = 2 ** i * 500;
      console.warn(`  rate limited, backing off ${waitMs}ms`);
      await new Promise((r) => setTimeout(r, waitMs));
    }
  }
}

async function pool(items, size, worker) {
  const results = [];
  const queue = [...items];
  const runners = Array.from({ length: size }, async () => {
    while (queue.length) {
      const item = queue.shift();
      results.push(await worker(item));
    }
  });
  await Promise.all(runners);
  return results;
}

const adapter = new FastPixAdapter(
  process.env.FASTPIX_TOKEN_ID,
  process.env.FASTPIX_SECRET_KEY,
);

const ids = await loadIds();
console.log(`auditing ${ids.length} assets...`);

const rows = await pool(ids, CONCURRENCY, async (id) => {
  try {
    return await withRetry(() => adapter.audit(id));
  } catch (err) {
    return { mediaId: id, sourceRetained: null, standardHls: null, playbackUrl: '', notes: `ERROR: ${err.message}` };
  }
});

const csv = [
  'media_id,source_retained,standard_hls,playback_url,notes',
  ...rows.map((r) =>
    [r.mediaId, r.sourceRetained, r.standardHls, r.playbackUrl, JSON.stringify(r.notes)].join(','),
  ),
].join('\n');

await writeFile('portability-audit.csv', csv);

const noSource = rows.filter((r) => r.sourceRetained === false).length;
const errors = rows.filter((r) => r.notes.startsWith('ERROR')).length;
console.log(`\ndone. ${rows.length} assets, ${noSource} without retrievable source, ${errors} errors`);
console.log('wrote portability-audit.csv');
Enter fullscreen mode Exit fullscreen mode

Run it:

node --env-file=.env src/run.js
Enter fullscreen mode Exit fullscreen mode
auditing 1204 assets...
  rate limited, backing off 500ms
  rate limited, backing off 1000ms

done. 1204 assets, 341 without retrievable source, 2 errors
wrote portability-audit.csv
Enter fullscreen mode Exit fullscreen mode

That 341 is the number this whole script exists to produce. It's the size of the problem you didn't know you had.

6. The ID sprawl check 🔍

The third question doesn't need the API at all, just grep over your own codebase and a look at your schema. Provider IDs leak further than anyone expects:

# how many places does a provider ID appear?
rg -l --type-add 'src:*.{ts,tsx,js,jsx,sql,py,rb,go}' -t src \
   -e 'playbackId' -e 'playback_id' -e 'media_id' -e 'stream\.fastpix\.com' \
   | sort | uniq -c | sort -rn
Enter fullscreen mode Exit fullscreen mode

Then the one people forget:

-- every column that could be holding a provider ID
SELECT table_name, column_name
FROM information_schema.columns
WHERE column_name ILIKE '%playback%'
   OR column_name ILIKE '%media_id%'
   OR column_name ILIKE '%video_id%'
ORDER BY table_name;
Enter fullscreen mode Exit fullscreen mode

Count the rows. That's your migration write path. If provider IDs are also in emails you've already sent, in a cached CDN rule, or in URLs your customers bookmarked, add a redirect layer to the estimate.

💡 Tip: the fix for ID sprawl is cheap before you need it. Put your own UUID on every video, map it to the provider ID in exactly one table, and reference only your UUID everywhere else. A migration then touches one table instead of nine.

Porting the adapter to other providers

The interface is three fields, so a new adapter is usually under thirty lines:

Provider Source retention Playback Bulk tooling
FastPix sourceAccess on create stream.fastpix.com/<id>.m3u8 Built-in batch migration tool
Mux Check asset settings Standard HLS Documented framework, you build the workflow
Cloudflare Stream Check asset settings Standard HLS Roll your own
api.video Check asset settings Standard HLS Markets zero migration cost

I've deliberately left the middle column vague for three of the four, because the flag names differ and I'd rather you read the current reference than trust a table I wrote today. For the FastPix fields used above, the VOD API reference has the create-media body and the get-media response shape.

What's next

Two follow-ups worth doing:

  • Schedule it. Run the audit in CI monthly and fail the build if source_retained=false count increases. That turns a one-off report into a guardrail, and it's the version of this that actually keeps working after everyone forgets about it.
  • Add a restore test. Auditing that a source file should be retrievable is not the same as proving it is. Pull one source file per month and check the bytes. Backups you haven't restored are backups you don't have, and the same logic applies here.

If you run this against your own library, I'd be curious what your source_retained=false number comes out as. Mine was worse than I guessed.

Top comments (0)