Join the Call Even When Beauty AR Fails: Build a Browser Preflight Gate

A Beauty AR demo usually proves one thing: an effect can run on the developer’s device.

A production join flow has a harder contract. The user still needs to enter the call if effect initialization times out, the GPU context is lost, a low-end device cannot sustain processing, or an asset fails to load.

That distinction helps clarify the role of “AI in the browser.” Real-time computer vision can produce useful beauty filters, makeup, segmentation, avatars, and gesture-driven effects. It cannot decide how much delay is acceptable, whether an altered image may be published without confirmation, or what the application should do when processing fails. Those remain product and engineering decisions.

In this tutorial, we will build a preflight gate with one invariant:

Beauty AR is a candidate output until it has produced frames and the user explicitly chooses to publish it.

If the candidate fails, the clean camera remains available. The call does not become dependent on the effect.

The workflow we want

The browser join sequence will be:

1. Acquire the camera.
2. Show the clean preview.
3. Prepare a Beauty AR candidate off-air.
4. Verify that the candidate is producing frames before a deadline.
5. Let the user choose Join with effect or Join without effect.
6. Publish only the selected stream.
7. If the live effect later becomes unhealthy, attempt to replace it with the clean stream.

This is deliberately different from testing whether an API call resolved. A resolved initialization promise does not prove that useful video is reaching the preview.

Tencent RTC’s Beauty AR overview describes real-time capabilities including beauty filters, makeup, stickers, virtual backgrounds, avatars, gesture recognition, and image or video enhancement. Its low-end device optimization guide recommends adapting effects to device tiers, using appropriate performance modes, avoiding expensive segmentation or 3D/GAN effects where necessary, and controlling resolution and frame rate.

Our controller turns those recommendations into an application-level join policy without inventing a universal definition of a “fast device.”

Model the states users can actually encounter

A boolean such as beautyEnabled cannot distinguish loading from failure, qualification, publication, or recovery.

Use explicit phases instead:

type Phase =
  | "idle"
  | "acquiring-camera"
  | "camera-ready"
  | "preparing-effect"
  | "qualifying-effect"
  | "ready-clean"
  | "ready-effect"
  | "publishing"
  | "live-clean"
  | "live-effect"
  | "blocked";

type ViewState = {
  phase: Phase;
  notice?: string;
};

These states produce a straightforward UI contract:

Phase	User-visible behavior
acquiring-camera	Disable join and show camera setup progress
camera-ready	Show the clean preview
preparing-effect	Keep the clean preview visible; allow cancellation
qualifying-effect	Indicate that the effect is being checked
ready-clean	Enable clean join and explain why the effect is unavailable, if relevant
ready-effect	Show both join choices; do not select the effect silently
publishing	Prevent duplicate join actions
live-clean / live-effect	Display which output is currently being sent
blocked	Explain the camera or publication failure and offer a relevant retry

The important emotional detail is that “your device could not run this effect” should not be presented as “your device cannot join.” Those are separate capabilities.

Create the reproducible project

Start with a framework-independent TypeScript core. It can later sit behind React, Vue, or another UI.

mkdir beauty-preflight
cd beauty-preflight
npm init -y
npm install -D typescript vitest @types/node
npx tsc --init

Add these scripts to package.json:

{
  "scripts": {
    "test": "vitest run",
    "test:watch": "vitest"
  }
}

For deterministic tests, the core uses a small Stream value rather than requiring real browser media objects:

// src/preflight.ts
export type Stream = { id: string };

export type EffectProfile = {
  id: string;
  allowSegmentation: boolean;
  allow3D: boolean;
  allowGAN: boolean;
};

export interface CameraPort {
  open(): Promise<Stream>;
  close(stream: Stream): Promise<void>;
}

export interface BeautyPort {
  create(input: Stream, profile: EffectProfile): Promise<Stream>;
  dispose(stream: Stream): Promise<void>;
}

export interface FrameProbe {
  hasProgress(stream: Stream, windowMs: number): Promise<boolean>;
}

export interface PublisherPort {
  publish(stream: Stream): Promise<void>;
  replace(stream: Stream): Promise<void>;
  unpublish(): Promise<void>;
}

export type Phase =
  | "idle"
  | "acquiring-camera"
  | "camera-ready"
  | "preparing-effect"
  | "qualifying-effect"
  | "ready-clean"
  | "ready-effect"
  | "publishing"
  | "live-clean"
  | "live-effect"
  | "blocked";

export type ViewState = {
  phase: Phase;
  notice?: string;
};

export type Policy = {
  prepareDeadlineMs: number;
  qualificationWindowMs: number;
};

CameraPort, BeautyPort, and PublisherPort are application-owned boundaries. They are not names of Tencent RTC APIs. The production adapters should call the documented browser, Beauty AR, and RTC integration surfaces available to your application.

Race the preparation, but clean up the loser

A timeout does not cancel the operation underneath it. If initialization finishes after the application has fallen back, that late result must be disposed rather than attached to the preview.

// src/preflight.ts

type DeadlineResult<T> =
  | { kind: "ok"; value: T }
  | { kind: "error"; error: unknown }
  | { kind: "timeout" };

async function beforeDeadline<T>(
  operation: Promise<T>,
  deadlineMs: number
): Promise<DeadlineResult<T>> {
  let timer: ReturnType<typeof setTimeout> | undefined;

  const timeout = new Promise<DeadlineResult<T>>((resolve) => {
    timer = setTimeout(() => resolve({ kind: "timeout" }), deadlineMs);
  });

  const settled: Promise<DeadlineResult<T>> = operation.then(
    (value) => ({ kind: "ok", value }),
    (error) => ({ kind: "error", error })
  );

  const result = await Promise.race([settled, timeout]);
  if (timer) clearTimeout(timer);
  return result;
}

The deadline is an application policy, not a Tencent RTC performance guarantee. Choose it from your join experience requirements and verify it on representative devices.

Implement the preflight coordinator

The coordinator retains the clean camera while the effect is prepared. An incrementing epoch invalidates callbacks from cancelled or superseded attempts.

// src/preflight.ts

export class BeautyPreflight {
  private state: ViewState = { phase: "idle" };
  private epoch = 0;
  private raw?: Stream;
  private candidate?: Stream;

  constructor(
    private readonly camera: CameraPort,
    private readonly beauty: BeautyPort,
    private readonly probe: FrameProbe,
    private readonly publisher: PublisherPort,
    private readonly policy: Policy,
    private readonly onState: (state: ViewState) => void
  ) {}

  get snapshot(): ViewState {
    return this.state;
  }

  private transition(state: ViewState): void {
    this.state = state;
    this.onState(state);
  }

  async start(profile: EffectProfile): Promise<void> {
    const attempt = ++this.epoch;
    this.transition({ phase: "acquiring-camera" });

    let raw: Stream;
    try {
      raw = await this.camera.open();
    } catch {
      if (attempt !== this.epoch) return;
      this.transition({
        phase: "blocked",
        notice: "Camera access failed. Check permission or device availability."
      });
      return;
    }

    if (attempt !== this.epoch) {
      await this.camera.close(raw);
      return;
    }

    this.raw = raw;
    this.transition({ phase: "camera-ready" });
    this.transition({ phase: "preparing-effect" });

    const pending = this.beauty.create(raw, profile);
    const prepared = await beforeDeadline(
      pending,
      this.policy.prepareDeadlineMs
    );

    if (prepared.kind === "timeout") {
      // The operation may still resolve. Dispose that late candidate.
      void pending
        .then((lateStream) => this.beauty.dispose(lateStream))
        .catch(() => undefined);

      if (attempt === this.epoch) {
        this.transition({
          phase: "ready-clean",
          notice: "The effect took too long to prepare. You can still join normally."
        });
      }
      return;
    }

    if (prepared.kind === "error") {
      if (attempt === this.epoch) {
        this.transition({
          phase: "ready-clean",
          notice: "The effect could not be loaded. You can still join normally."
        });
      }
      return;
    }

    const candidate = prepared.value;

    if (attempt !== this.epoch) {
      await this.beauty.dispose(candidate);
      return;
    }

    this.candidate = candidate;
    this.transition({ phase: "qualifying-effect" });

    let healthy = false;
    try {
      healthy = await this.probe.hasProgress(
        candidate,
        this.policy.qualificationWindowMs
      );
    } catch {
      healthy = false;
    }

    if (attempt !== this.epoch) {
      await this.beauty.dispose(candidate);
      if (this.candidate === candidate) this.candidate = undefined;
      return;
    }

    if (!healthy) {
      await this.beauty.dispose(candidate);
      this.candidate = undefined;
      this.transition({
        phase: "ready-clean",
        notice: "The effect preview stopped producing frames. Clean video is available."
      });
      return;
    }

    this.transition({ phase: "ready-effect" });
  }

  async chooseClean(): Promise<void> {
    ++this.epoch;

    const oldCandidate = this.candidate;
    this.candidate = undefined;

    if (oldCandidate) await this.beauty.dispose(oldCandidate);

    if (this.raw) {
      this.transition({ phase: "ready-clean" });
    }
  }

  async join(useEffect: boolean): Promise<void> {
    if (!this.raw) {
      this.transition({
        phase: "blocked",
        notice: "The camera is not ready."
      });
      return;
    }

    if (useEffect && this.state.phase !== "ready-effect") {
      this.transition({
        phase: "ready-clean",
        notice: "The requested effect is not ready. Choose clean video or retry it."
      });
      return;
    }

    const previous = this.state;
    const selected = useEffect ? this.candidate : this.raw;

    if (!selected) {
      this.transition({
        phase: "ready-clean",
        notice: "No publishable video is available."
      });
      return;
    }

    this.transition({ phase: "publishing" });

    try {
      await this.publisher.publish(selected);
      this.transition({ phase: useEffect ? "live-effect" : "live-clean" });
    } catch {
      this.transition({
        ...previous,
        notice: "Joining failed. Your preview is still available; retry when ready."
      });
    }
  }

  async recoverToClean(reason: string): Promise<void> {
    if (!this.raw) return;

    try {
      await this.publisher.replace(this.raw);

      if (this.candidate) {
        await this.beauty.dispose(this.candidate);
        this.candidate = undefined;
      }

      this.transition({
        phase: "live-clean",
        notice: `The effect was removed: ${reason}`
      });
    } catch {
      await this.publisher.unpublish().catch(() => undefined);
      this.transition({
        phase: "blocked",
        notice: "Video recovery failed. Video was stopped instead of leaving a frozen effect on-air."
      });
    }
  }
}

recoverToClean() encodes an important trade-off: if replacing an unhealthy effect fails, this implementation stops video rather than intentionally leaving a frozen or corrupted effect output published. Your application can choose a different policy, but it should be explicit and tested.

Test the failures before connecting a camera

Create src/preflight.test.ts:

import { describe, expect, it, vi } from "vitest";
import {
  BeautyPreflight,
  type EffectProfile,
  type Stream
} from "./preflight";

const raw: Stream = { id: "raw" };
const effected: Stream = { id: "effected" };

const profile: EffectProfile = {
  id: "conservative",
  allowSegmentation: false,
  allow3D: false,
  allowGAN: false
};

function deferred<T>() {
  let resolve!: (value: T) => void;
  let reject!: (error: unknown) => void;
  const promise = new Promise<T>((res, rej) => {
    resolve = res;
    reject = rej;
  });
  return { promise, resolve, reject };
}

function fixture(overrides: Record<string, unknown> = {}) {
  const states: string[] = [];

  const camera = {
    open: vi.fn().mockResolvedValue(raw),
    close: vi.fn().mockResolvedValue(undefined)
  };

  const beauty = {
    create: vi.fn().mockResolvedValue(effected),
    dispose: vi.fn().mockResolvedValue(undefined)
  };

  const probe = {
    hasProgress: vi.fn().mockResolvedValue(true)
  };

  const publisher = {
    publish: vi.fn().mockResolvedValue(undefined),
    replace: vi.fn().mockResolvedValue(undefined),
    unpublish: vi.fn().mockResolvedValue(undefined)
  };

  Object.assign({ camera, beauty, probe, publisher }, overrides);

  const controller = new BeautyPreflight(
    camera,
    beauty,
    probe,
    publisher,
    { prepareDeadlineMs: 20, qualificationWindowMs: 10 },
    (state) => states.push(state.phase)
  );

  return { controller, camera, beauty, probe, publisher, states };
}

describe("BeautyPreflight", () => {
  it("keeps clean join available when effect creation fails", async () => {
    const f = fixture();
    f.beauty.create.mockRejectedValueOnce(new Error("asset failed"));

    await f.controller.start(profile);

    expect(f.controller.snapshot.phase).toBe("ready-clean");
    await f.controller.join(false);
    expect(f.publisher.publish).toHaveBeenCalledWith(raw);
  });

  it("rejects an effect that initializes but produces no frames", async () => {
    const f = fixture();
    f.probe.hasProgress.mockResolvedValueOnce(false);

    await f.controller.start(profile);

    expect(f.beauty.dispose).toHaveBeenCalledWith(effected);
    expect(f.controller.snapshot.phase).toBe("ready-clean");
  });

  it("disposes an effect that resolves after the deadline", async () => {
    const late = deferred<Stream>();
    const f = fixture();
    f.beauty.create.mockReturnValueOnce(late.promise);

    await f.controller.start(profile);
    expect(f.controller.snapshot.phase).toBe("ready-clean");

    late.resolve(effected);
    await vi.waitFor(() => {
      expect(f.beauty.dispose).toHaveBeenCalledWith(effected);
    });
  });

  it("does not publish an effect merely because it is ready", async () => {
    const f = fixture();

    await f.controller.start(profile);

    expect(f.controller.snapshot.phase).toBe("ready-effect");
    expect(f.publisher.publish).not.toHaveBeenCalled();

    await f.controller.join(true);
    expect(f.publisher.publish).toHaveBeenCalledWith(effected);
  });

  it("falls back to clean video when a live effect becomes unhealthy", async () => {
    const f = fixture();

    await f.controller.start(profile);
    await f.controller.join(true);
    await f.controller.recoverToClean("frame processing stalled");

    expect(f.publisher.replace).toHaveBeenCalledWith(raw);
    expect(f.controller.snapshot.phase).toBe("live-clean");
  });
});

Run the suite:

npm test

These tests verify authority as well as mechanics. The effect can become eligible for publication, but only join(true) can publish it.

Connect frame qualification to the browser

A production FrameProbe should observe the output users will see—not only the input camera and not only a successful initialization callback.

Where supported by your browser targets, a video-element adapter can observe presented frames with requestVideoFrameCallback. Keep that browser-specific code outside the state machine:

export class VideoElementProbe {
  constructor(
    private readonly resolveVideo: (stream: Stream) => HTMLVideoElement
  ) {}

  async hasProgress(stream: Stream, windowMs: number): Promise<boolean> {
    const video = this.resolveVideo(stream);

    if (!("requestVideoFrameCallback" in video)) {
      // Do not pretend unsupported observation is a successful check.
      return false;
    }

    return new Promise<boolean>((resolve) => {
      let frames = 0;
      let finished = false;

      const finish = (result: boolean) => {
        if (finished) return;
        finished = true;
        resolve(result);
      };

      const onFrame = () => {
        frames += 1;
        if (frames >= 2) return finish(true);
        video.requestVideoFrameCallback(onFrame);
      };

      video.requestVideoFrameCallback(onFrame);
      setTimeout(() => finish(false), windowMs);
    });
  }
}

The count above is an example liveness rule for this tutorial, not a universal quality threshold. A production policy may also consider frame spacing, sustained output, application responsiveness, and visibility state.

Do not qualify an effect while the tab is hidden and then treat that result as representative. Pause the attempt or restart qualification when the document becomes visible again.

CPU/GPU policy: use capability hints, then verify output

Browser capability flags are useful inputs, but poor final authorities.

For example:

• A GPU interface being available does not prove enough headroom for a particular effect.
• CPU core count does not describe thermal throttling or competing workloads.
• A device model or user-agent string does not reveal the current session’s resource pressure.
• Successful initialization does not prove sustained frame delivery.

Use a two-stage decision:

Stage 1: Select a conservative candidate profile

An application-owned profile can restrict expensive categories:

const conservativeProfile: EffectProfile = {
  id: "conservative",
  allowSegmentation: false,
  allow3D: false,
  allowGAN: false
};

For constrained device tiers, follow the official optimization guidance: choose an appropriate performance mode, avoid expensive segmentation or 3D/GAN effects, and control capture resolution and frame rate. Do not copy arbitrary thresholds from another product and call them device tiers.

Stage 2: Verify the candidate in the current session

Regardless of the initial profile, require actual output progress. Continue monitoring after publication so thermal pressure, GPU context loss, or backgrounding can trigger recoverToClean().

This creates graceful degradation rather than a binary “AI-capable” label:

1. Requested effect profile.
2. Simpler effect profile, if the user agrees.
3. Clean video.
4. Video stopped if clean recovery itself cannot be published safely.

Automatic fallback between visibly different effects should also be disclosed. “We silently changed your appearance” is not an acceptable interpretation of graceful degradation.

React should render the state, not reconstruct it

A React component can subscribe to ViewState and map it to controls:

function JoinControls({
  state,
  join,
  chooseClean
}: {
  state: ViewState;
  join: (withEffect: boolean) => void;
  chooseClean: () => void;
}) {
  const canJoinClean =
    state.phase === "ready-clean" || state.phase === "ready-effect";

  return (
    <section aria-live="polite">
      {state.notice && <p>{state.notice}</p>}

      <button disabled={!canJoinClean} onClick={() => join(false)}>
        Join without effect
      </button>

      <button
        disabled={state.phase !== "ready-effect"}
        onClick={() => join(true)}
      >
        Join with effect
      </button>

      {(state.phase === "preparing-effect" ||
        state.phase === "qualifying-effect") && (
        <button onClick={chooseClean}>Skip effect</button>
      )}
    </section>
  );
}

Avoid deriving readiness from a collection of React booleans such as loading, enabled, and failed. Let the coordinator own legal transitions; let React render them.

Failure drills to run on real devices

Unit tests cannot reproduce every browser and hardware behavior. Before release, perform these drills with representative devices and workloads.

Camera permission is denied

Expected result: no effect initialization starts, no join button implies video is ready, and the UI explains how to retry.

Effect assets fail to load

Expected result: the clean preview remains visible and clean join is enabled. Repeated automatic downloads should not trap the user in setup.

The preparation deadline expires

Expected result: the UI moves to ready-clean. If initialization later completes, its output is disposed and never replaces the preview.

The user skips while initialization is pending

Expected result: the epoch invalidates the pending result. A late callback cannot re-enable or publish the effect.

The candidate initializes but frames do not advance

Expected result: qualification fails, resources are disposed, and the clean stream remains available.

The tab is backgrounded during qualification

Expected result: do not certify that run. Pause or restart qualification after visibility returns.

The live effect stalls

Expected result: replace it with the clean stream and show a neutral explanation. If replacement fails, follow the declared stop-video or reconnect policy.

Publishing fails after a successful preview

Expected result: retain the local preview and offer a retry. Preview readiness and RTC publication are separate states.

Release verification checklist

Before shipping, verify that:

• [ ] The clean camera is acquired independently of Beauty AR initialization.
• [ ] No effect output is published before explicit user confirmation.
• [ ] A deadline leads to clean join rather than an endless spinner.
• [ ] Late initialization results are disposed.
• [ ] Qualification observes processed output frames, not merely input frames.
• [ ] Hidden-tab qualification is paused or invalidated.
• [ ] Low-end profiles avoid effects your device policy classifies as too expensive.
• [ ] Resolution and frame-rate choices are tested on representative hardware.
• [ ] Runtime stalls trigger a clean-output recovery path.
• [ ] Failed stream replacement has a defined, user-visible outcome.
• [ ] Logs distinguish camera failure, effect failure, qualification failure, and publication failure.
• [ ] The user can always tell whether clean or altered video is being sent.

Browser-based computer vision is a demonstrated capability. Reliable participation is the product contract. Keeping those ideas separate lets developers use Beauty AR without making a user’s ability to join depend on an effect—or making hardware limitations feel like personal failure.

Implementation references and disclosure

Implementation guidance was checked against the official Tencent RTC Beauty AR overview and Low-End Device Performance Optimization Practice Guide.

Disclosure: I’m writing in collaboration with Tencent RTC, and I used official Tencent RTC documentation as the implementation reference for this tutorial.