Handle HTML error pages before parsing a video metadata response

A download interface can show Unexpected token '<' even when the user's post link is valid. The metadata endpoint may have returned an HTML login page, an upstream error page, or a proxy response instead of JSON. Validate the HTTP response contract before parsing its body so users get an actionable failure and developers keep the actual cause.

I maintain Twitter Viewer and its twitter video downloader. This article uses a fictional metadata endpoint and a small executable response parser. It describes an interface boundary, not Twitter Viewer's private implementation.

Check status before interpreting a successful result

fetch resolves for HTTP errors such as 404 and 500. A resolved promise only means a response arrived. Code that immediately calls response.json() can hide an upstream 502 behind a JSON syntax error if the gateway generated an HTML page.

Give each kind of failure a stable code. A non-success status is different from a successful response with an unexpected media type, and both differ from a JSON object with missing fields.

export async function readMetadata(response) {
  const fail = (code, message) => {
    throw Object.assign(new Error(message), { code });
  };
  if (!response.ok) fail('HTTP', `Metadata request returned ${response.status}`);
  const type = (response.headers.get('content-type') || '')
    .split(';', 1)[0].trim().toLowerCase();
  if (type !== 'application/json') {
    fail('TYPE', 'Metadata endpoint did not return application/json');
  }
  let value;
  try {
    value = await response.json();
  } catch {
    fail('JSON', 'Metadata endpoint returned invalid JSON');
  }
  if (!value || typeof value !== 'object' || Array.isArray(value) ||
      typeof value.postId !== 'string' || !Array.isArray(value.variants)) {
    fail('SCHEMA', 'Metadata response is missing postId or variants');
  }
  return value;
}

Save this as metadata.mjs. The schema check is intentionally small: it establishes that the outer object can be consumed by a caller. A real application should validate each variant's properties before rendering a link or initiating a transfer. A string with the right property name is not automatically a trusted fetch destination.

This contract accepts application/json with optional parameters, including application/json; charset=utf-8. It intentionally rejects vendor media types. If your API uses a vendor JSON type, add it explicitly to the contract and its tests instead of accepting any header that happens to contain the substring json.

Exercise failures without making a network request

Modern Node.js includes the Response class. A test can construct the exact bad response a proxy might return:

import test from 'node:test';
import assert from 'node:assert/strict';
import { readMetadata } from './metadata.mjs';

const json = text => new Response(text, {
  headers: { 'content-type': 'application/json; charset=utf-8' }
});

test('accepts the expected envelope', async () => {
  assert.deepEqual(await readMetadata(json('{"postId":"123","variants":[]}')),
    { postId: '123', variants: [] });
});

test('classifies status, content, and shape separately', async () => {
  await assert.rejects(readMetadata(new Response('<h1>Bad gateway</h1>',
    { status: 502 })), { code: 'HTTP' });
  await assert.rejects(readMetadata(new Response('<html>Sign in</html>',
    { headers: { 'content-type': 'text/html' } })), { code: 'TYPE' });
  await assert.rejects(readMetadata(json('{')), { code: 'JSON' });
  await assert.rejects(readMetadata(json('{"variants":[]}')), { code: 'SCHEMA' });
});

Save that as metadata.test.mjs and run node --test metadata.test.mjs. No API credentials, social account, or live video are needed. These are response-contract tests, so a platform outage cannot make the test results ambiguous.

Keep diagnostics out of the visible error

For the user, an HTTP failure might warrant “Video lookup is temporarily unavailable; retry shortly.” An HTML response might warrant the same recovery advice while retaining a different internal code. Log the status, response type, request ID, and endpoint name where appropriate. Avoid pasting an entire unexpected HTML body into an error toast: it may contain account details, tokens, or internal infrastructure text.

Once metadata is valid, downloading remains a separate operation with its own response contract. Receiving a list of variants does not prove a file reached the user's device. Native public videos, externally embedded players, and inaccessible posts also require distinct source handling; a generic JSON parsing message should never be the explanation for all of them.