DEV Community

Jason Patrick Shedden
Jason Patrick Shedden

Posted on

whip-whep-android-chrome-findings

#android #cloud #networking #nextjs

In-Browser Live Streaming with WHIP + WHEP on Cloudflare Stream: Everything That Can Go Wrong (and How to Fix It)

A complete technical post-mortem of getting Android Chrome → Cloudflare Stream → desktop viewer working via WebRTC.

The goal

Build an owner-only /broadcast page that streams live video from a phone's camera directly to paying members, with no third-party app, no stream keys in the user's hands, and sub-second latency. Stack: Next.js + Cloudflare Stream using WHIP (WebRTC HTTP Ingestion Protocol) to publish and WHEP (WebRTC HTTP Egress Protocol) to play back.

The approach looked simple on paper:

Phone browser → WebRTC/WHIP → Cloudflare → WebRTC/WHEP → Desktop viewer
Enter fullscreen mode Exit fullscreen mode

In practice there were four distinct, non-obvious failures. Each one looked like a Cloudflare problem. None of them were.

Failure 1: Black video / no video despite successful WHIP handshake (codec mismatch)

Symptom

The WHIP SDP exchange completed cleanly. ICE connected. Cloudflare's dashboard showed the stream as "connected". But viewers got black video or no video at all.

Root cause: RFC 3264 + Android hardware H.264 encoder

Cloudflare's SDP answer is allowed by RFC 3264 to reorder codec entries. It puts H.264 first, before VP8, regardless of the offer order. Chrome obeys this and switches its video encoder to H.264 — which is fine on desktop (software encoder produces Constrained Baseline). On Android, however, Chrome's H.264 encoder is a hardware MediaCodec. Android hardware encoders produce Main profile or High profile, not Constrained Baseline.

Cloudflare Stream only accepts Constrained Baseline H.264 on its WHIP endpoint. It silently drops the non-CB H.264 stream. No error. Just black.

This means: even if you write VP8 first in your SDP offer, Cloudflare can legally swap it to H.264 in the answer, Chrome switches encoder, and Android hardware produces unusable frames.

Fix (two-pronged, belt and suspenders)

Client side — lock to VP8 before createOffer():

pc.addTrack(videoTrack, stream);
pc.addTrack(audioTrack, stream);

// setCodecPreferences MUST be called BEFORE createOffer()
const videoTransceiver = pc.getTransceivers().find(
  t => t.sender.track?.kind === "video"
);
if (videoTransceiver) {
  const caps = RTCRtpSender.getCapabilities("video");
  if (caps) {
    const vp8 = caps.codecs.filter(c => c.mimeType === "video/VP8");
    if (vp8.length > 0) {
      videoTransceiver.setCodecPreferences(vp8);
    }
  }
}

const offer = await pc.createOffer();
Enter fullscreen mode Exit fullscreen mode

Server side — strip all H.264 profiles from the offer SDP before forwarding to Cloudflare:

// In your WHIP proxy route
function mungeSdpOffer(sdp: string): string {
  const lines = sdp.split("\r\n");
  const stripPts = new Set<number>();

  // Pass 1: find H264 payload types (all profiles: 42=CB, 4d=Main, 64=High)
  for (const line of lines) {
    const m = line.match(/^a=rtpmap:(\d+) H264\/90000/i);
    if (!m) continue;
    const pt = parseInt(m[1]);
    // Also match via fmtp profile-level-id
    const fmtpLine = lines.find(l => l.startsWith(`a=fmtp:${pt} `));
    const idc = fmtpLine?.match(/profile-level-id=([0-9a-f]{2})/i)?.[1]?.toLowerCase();
    if (idc === "42" || idc === "4d" || idc === "64") {
      stripPts.add(pt);
    } else {
      stripPts.add(pt); // strip all H264 regardless
    }
  }

  // Pass 2: rebuild m=video line without H264 pts
  // Pass 3: strip a=rtpmap, a=fmtp, a=rtcp-fb lines for stripped pts
  // ... (see full implementation below)
}
Enter fullscreen mode Exit fullscreen mode

If setCodecPreferences succeeds, H.264 never appears in the offer SDP at all. The server-side strip is a safety net for browsers where setCodecPreferences is unavailable or partial.

Why this matters: This affects all Android Chrome users (any version) and any other browser that uses hardware H.264 encoding with Main/High profile. It is not a Cloudflare bug — it is correct RFC 3264 behaviour combined with a platform limitation.

Failure 2: ICE disconnects after ~30 seconds

Symptom

Stream connects. Video flows. After roughly 30 seconds, ICE state changes to disconnected, then failed. Reconnect attempts repeat the cycle.

Root cause: NAT binding expiry without TURN

WebRTC ICE uses STUN to discover the public IP:port, then sends media directly (peer-to-peer style) to Cloudflare's servers. The direct UDP path requires the NAT binding on the mobile carrier's NAT to stay alive. Mobile carrier NATs are aggressive — many expire UDP bindings after 20–30 seconds if there is no traffic in the reverse direction.

RFC 7675 "ICE Consent Freshening" sends keep-alives, but if the NAT has already discarded the binding, those keep-alives go nowhere. The connection dies.

Fix: Cloudflare Realtime TURN

Cloudflare offers a separate "Realtime" product with TURN servers. Using TURN, media flows phone → TURN relay → Cloudflare, so there is always traffic on the path and the relay keeps the NAT binding alive.

Critical pricing note: Traffic between Cloudflare TURN and Cloudflare Stream is free. You only pay for TURN egress to non-Cloudflare destinations. For WHIP ingestion this means your TURN cost is zero.

TURN key creation: Keys can only be created via the Cloudflare dashboard (Realtime → TURN → Create key). There is no API to create new keys. Once a key exists, you generate short-lived credentials via API.

Server endpoint to generate credentials:

// GET /api/live/turn-credentials (SUPER_ADMIN only — keep server-side!)
export async function GET() {
  const session = await auth();
  if (session?.user?.role !== "SUPER_ADMIN") {
    return NextResponse.json({ error: "Forbidden" }, { status: 403 });
  }

  const keyId = process.env.CLOUDFLARE_TURN_KEY_ID;
  const token = process.env.CLOUDFLARE_TURN_KEY_API_TOKEN;

  if (!keyId || !token) {
    // Fallback: STUN only (stream may disconnect after 30s on bad NAT)
    return NextResponse.json({
      iceServers: [{ urls: "stun:stun.cloudflare.com:3478" }]
    });
  }

  const resp = await fetch(
    `https://rtc.live.cloudflare.com/v1/turn/keys/${keyId}/credentials/generate-ice-servers`,
    {
      method: "POST",
      headers: {
        Authorization: `Bearer ${token}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({ ttl: 86400 }),
    }
  );

  const data = await resp.json();
  return NextResponse.json({ iceServers: data.iceServers });
}
Enter fullscreen mode Exit fullscreen mode

Client — use TURN credentials when creating RTCPeerConnection:

let iceServers: RTCIceServer[] = [{ urls: "stun:stun.cloudflare.com:3478" }];
try {
  const resp = await fetch("/api/live/turn-credentials");
  if (resp.ok) {
    const data = await resp.json();
    if (data.iceServers) iceServers = data.iceServers;
  }
} catch {
  console.warn("Could not fetch TURN credentials, using STUN only");
}

const pc = new RTCPeerConnection({ iceServers, bundlePolicy: "max-bundle" });
Enter fullscreen mode Exit fullscreen mode

Never expose your TURN key ID or API token to the browser. Always generate credentials server-side and return just the iceServers array. Short TTL (86400s = 1 day) is fine for broadcaster flows.

Failure 3: Canvas-based encoding produces malformed frames

What we tried first

To avoid the H.264 codec mismatch, we tried routing the camera through a canvas element and calling canvas.captureStream(), reasoning that Canvas VP8 encoding would bypass the hardware encoder:

// DON'T DO THIS — it doesn't work reliably
const canvas = document.createElement("canvas");
const ctx = canvas.getContext("2d");
const videoEl = document.createElement("video");
videoEl.srcObject = stream;
videoEl.play();

function draw() {
  ctx.drawImage(videoEl, 0, 0);
  requestAnimationFrame(draw);
}
draw();

const canvasStream = canvas.captureStream(30);
pc.addTrack(canvasStream.getVideoTracks()[0], canvasStream);
Enter fullscreen mode Exit fullscreen mode

Why it failed

Canvas captureStream() works, but the frame timestamps produced by requestAnimationFrame are based on the display clock, not the camera's capture clock. When those frames reach Cloudflare's WebRTC ingest, the RTP timestamps are irregular. Cloudflare's relay dropped the session after 1–2 minutes. Additionally, the canvas path added visible frame latency.

The correct fix

Once setCodecPreferences([VP8 only]) is in place (see Failure 1), you don't need the canvas workaround at all. Use the raw camera track directly:

pc.addTrack(stream.getVideoTracks()[0], stream);
pc.addTrack(stream.getAudioTracks()[0], stream);
// set VP8 codec preference on transceiver
// createOffer() — VP8 is now locked in
Enter fullscreen mode Exit fullscreen mode

The native camera track has correct RTP timestamps. Cloudflare accepts it. Canvas adds complexity and bugs without solving anything that setCodecPreferences doesn't already solve.

Failure 4: Viewer player spins forever (iframe vs WHEP)

Symptom

Publisher streams successfully. ICE connected, video flowing. Viewer opens the page, sees the LIVE badge, but the player just spins. No video ever appears.

Root cause: Cloudflare's /iframe URL is an HLS player

Cloudflare Stream's /iframe embed URL serves an HLS player (based on hls.js or a similar library). It waits for an HLS manifest. WHIP-ingested streams produce no HLS output — WebRTC input goes through a passthrough relay and comes out as WebRTC only.

The HLS player never gets an HLS manifest. It just hangs.

Key Cloudflare architecture facts:

  • WHIP → WHEP is a passthrough relay, not a transcode
  • Whatever codec the WHIP publisher sends, WHEP viewers receive — same codec, same stream
  • preferLowLatency: true on live inputs applies to HLS, not WebRTC
  • The iframe player does not auto-switch to WHEP — it is always HLS

Fix: native WHEP RTCPeerConnection client

Replace the Cloudflare iframe with a native WebRTC viewer:

async function startWhep(
  videoEl: HTMLVideoElement,
  signal: AbortSignal
): Promise<RTCPeerConnection> {
  const pc = new RTCPeerConnection({
    iceServers: [
      { urls: "stun:stun.cloudflare.com:3478" },
      { urls: "stun:stun.l.google.com:19302" },
    ],
    bundlePolicy: "max-bundle",
  });

  // Attach incoming tracks to the video element
  pc.ontrack = (e) => {
    const stream = e.streams[0] ?? new MediaStream([e.track]);
    if (videoEl && !videoEl.srcObject) {
      videoEl.srcObject = stream;
      videoEl.play().catch(() => null);
    }
  };

  // Viewer is receive-only
  pc.addTransceiver("video", { direction: "recvonly" });
  pc.addTransceiver("audio", { direction: "recvonly" });

  const offer = await pc.createOffer();
  await pc.setLocalDescription(offer);

  // Wait for ICE gathering to complete (or 3s timeout)
  await new Promise<void>((resolve) => {
    if (pc.iceGatheringState === "complete") { resolve(); return; }
    const t = setTimeout(resolve, 3000);
    pc.onicegatheringstatechange = () => {
      if (pc.iceGatheringState === "complete") { clearTimeout(t); resolve(); }
    };
  });

  if (signal.aborted) { pc.close(); throw new DOMException("aborted", "AbortError"); }

  // POST SDP offer to your server-side WHEP proxy
  const resp = await fetch("/api/live/whep", {
    method: "POST",
    headers: { "Content-Type": "application/sdp" },
    body: pc.localDescription!.sdp,
    signal,
  });

  if (!resp.ok) {
    pc.close();
    throw new Error(`WHEP ${resp.status}`);
  }

  const sdpAnswer = await resp.text();
  await pc.setRemoteDescription({ type: "answer", sdp: sdpAnswer });
  return pc;
}
Enter fullscreen mode Exit fullscreen mode

Your WHEP proxy (server-side) forwards the SDP offer to:

POST https://<customer-subdomain>.cloudflarestream.com/<uid>/webRTC/play
Content-Type: application/sdp
Authorization: Bearer <your-api-token>
Enter fullscreen mode Exit fullscreen mode

And returns the SDP answer body verbatim.

Bonus: IPv6 ICE candidates cause silent failure on Cloudflare

Symptom

Occasionally the stream connects but immediately disconnects, or connection quality is poor.

Root cause

Cloudflare's WHIP endpoint has a known issue with IPv6 ICE candidate path selection. If IPv6 candidates are included, Cloudflare may select the IPv6 path and then fail silently.

Fix: strip IPv6 candidates from the offer before forwarding 

// In your WHIP proxy, when processing the SDP offer:
for (const line of lines) {
  if (line.startsWith("a=candidate:") && line.includes(" IP6 ")) {
    continue; // drop IPv6 candidates
  }
  out.push(line);
}
Enter fullscreen mode Exit fullscreen mode

This is a server-side-only change — the client is unaffected.

Complete server-side WHIP proxy (/api/live/whip-publish/route.ts) 

import { NextRequest, NextResponse } from "next/server";
import { auth } from "@/lib/auth";

const WHIP_URL = process.env.CLOUDFLARE_WHIP_URL!; // never expose to browser

function mungeSdpOffer(sdp: string): string {
  const lines = sdp.split("\r\n");
  const stripPts = new Set<number>();

  // Pass 1: collect H264 payload types
  for (const line of lines) {
    const m = line.match(/^a=rtpmap:(\d+) H264\/90000/i);
    if (m) stripPts.add(parseInt(m[1]));
  }

  // Pass 2: rewrite m=video line
  const out: string[] = [];
  for (const line of lines) {
    if (line.startsWith("m=video ") && stripPts.size > 0) {
      const parts = line.split(" ");
      const filtered = parts.filter((p, i) => i < 3 || !stripPts.has(parseInt(p)));
      out.push(filtered.join(" "));
    } else {
      out.push(line);
    }
  }

  // Pass 3: strip H264-related attribute lines AND IPv6 candidates
  const out2: string[] = [];
  for (const line of out) {
    const rtpMatch = line.match(/^a=(?:rtpmap|fmtp|rtcp-fb):(\d+) /);
    if (rtpMatch && stripPts.has(parseInt(rtpMatch[1]))) continue;
    if (line.startsWith("a=candidate:") && line.includes(" IP6 ")) continue;
    out2.push(line);
  }

  return out2.join("\r\n");
}

export async function POST(req: NextRequest) {
  const session = await auth();
  if (!session?.user) return NextResponse.json({ error: "Unauthorized" }, { status: 401 });
  if (session.user.role !== "SUPER_ADMIN") return NextResponse.json({ error: "Forbidden" }, { status: 403 });

  const offerSdp = await req.text();
  const mungedSdp = mungeSdpOffer(offerSdp);

  const cfResp = await fetch(WHIP_URL, {
    method: "POST",
    headers: { "Content-Type": "application/sdp" },
    body: mungedSdp,
  });

  if (!cfResp.ok) {
    const body = await cfResp.text();
    console.error(`[WHIP proxy] Cloudflare error ${cfResp.status}:`, body);
    return NextResponse.json({ error: "Cloudflare WHIP failed" }, { status: 502 });
  }

  const answerSdp = await cfResp.text();
  return new NextResponse(answerSdp, {
    status: 200,
    headers: { "Content-Type": "application/sdp" },
  });
}
Enter fullscreen mode Exit fullscreen mode

Summary: the four changes required for Android Chrome → Cloudflare Stream → browser viewer

# Problem Fix
1 Android H.264 hardware encoder produces Main/High profile (not CB) setCodecPreferences([VP8 only]) on video transceiver before createOffer(). Strip all H.264 from offer SDP server-side as belt-and-suspenders.
2 ICE disconnects after ~30s on mobile carrier NAT Cloudflare Realtime TURN. Generate credentials server-side, never expose key in browser.
3 Canvas captureStream() produces malformed frame timestamps Remove canvas entirely. Use native camera track with VP8 codec preference — correct timestamps, no intermediate encoding step.
4 Cloudflare iframe is an HLS player, not WHEP Replace iframe with native WHEP RTCPeerConnection using addTransceiver("video", { direction: "recvonly" }).

None of these are Cloudflare bugs. They are:

  • RFC 3264 codec reordering behaviour (intended)
  • Android platform limitation (no SW H.264 encoder in Chrome)
  • NAT binding expiry (network infrastructure behaviour)
  • Cloudflare documentation gap (iframe → HLS player is not prominently documented)

Environment variables required 

# Server-side only — never expose to browser
CLOUDFLARE_WHIP_URL=https://customer-xxx.cloudflarestream.com/<uid>/webRTC/publish
CLOUDFLARE_TURN_KEY_ID=<from dashboard.cloudflare.com → Realtime → TURN>
CLOUDFLARE_TURN_KEY_API_TOKEN=<from same page>

# These are fine to have server-side (used by your WHEP proxy and status API)
CLOUDFLARE_ACCOUNT_ID=...
CLOUDFLARE_API_TOKEN=...
CLOUDFLARE_LIVE_INPUT_UID=...
Enter fullscreen mode Exit fullscreen mode

Platform notes

  • Tested on: Android Chrome (Samsung Galaxy, Android 14), iOS Safari 17+, Desktop Chrome 124+, Desktop Firefox 125+
  • WHIP and WHEP are IETF standards (RFC 9725 and RFC 9726 respectively) — not Cloudflare-specific
  • Cloudflare Stream WHIP/WHEP is in GA as of early 2025
  • setCodecPreferences() is Baseline 2024 — available in all modern browsers

Written from real debugging experience building a live fitness streaming platform. Every failure listed here was hit in production testing, not theory.

Top comments (0)