An Avatar Failure Must Not Reveal the Camera: Build a Consent-Safe Beauty AR Fallback Governor

An avatar can make someone comfortable joining a live session. That creates a product promise: if the avatar stops working, the application must not quietly replace it with the person’s camera feed.

This is where a convincing avatar demo often becomes an unreliable product. Asset loading can fail. A device can heat up. Background segmentation or optional 3D effects can consume too much of the frame budget. Camera permission can disappear after an operating-system interruption.

The difficult part is not selecting another effect. It is deciding which fallback is both technically sustainable and acceptable to the user.

In this tutorial, we will build an application-side governor for a Tencent RTC Beauty AR experience. It will:

• start with a profile appropriate for the device tier;
• degrade only after repeated performance pressure;
• recover conservatively instead of oscillating between profiles;
• distinguish avatar, sticker, camera, and paused presentation modes;
• require separate consent before revealing the camera;
• reject stale asynchronous completion events;
• fail closed when no approved visual fallback remains.

Tencent RTC's Beauty AR overview documents capabilities including avatars, beauty effects, stickers, virtual backgrounds, gesture recognition, and image or video enhancement. Its Low-End Device Performance Optimization Practice Guide recommends adapting effects to device capability, choosing suitable performance modes, controlling resolution and frame rate, and disabling expensive segmentation or optional 3D/GAN effects when necessary.

The policy governing those capabilities still belongs in the application.

The fallback ladder is a consent decision

A common fallback sequence looks technically reasonable:

quality avatar → performance avatar → sticker → camera

But the final step changes what the participant reveals. It is not equivalent to reducing resolution.

Use separate consent for separate presentation modes:

Presentation	What the participant expects	Consent required
Avatar	Their chosen avatar represents them	Avatar tracking
Sticker	A simpler visual representation replaces the avatar	Sticker fallback
Camera	Their actual camera image may be shown	Camera fallback
Paused	No visual representation is published	No additional disclosure

If camera fallback was not approved, the safe ladder ends at paused.

This is also the useful way to frame the “AI avatar” question. The demonstrated capability is real-time visual processing and avatar presentation. The human decision underneath it is whether the software may change how a participant is represented when that processing fails. Calling the feature AI does not answer that question.

Create the project

The governor is deliberately independent of any UI framework or platform-specific SDK method names. That lets us reproduce policy behavior without requiring a camera.

mkdir avatar-fallback-governor
cd avatar-fallback-governor
npm init -y
npm install --save-dev typescript tsx @types/node
mkdir src test

Add the following scripts to package.json:

{
  "scripts": {
    "demo": "tsx src/demo.ts",
    "test": "tsx --test test/**/*.test.ts",
    "typecheck": "tsc --noEmit"
  }
}

Create tsconfig.json:

{
  "compilerOptions": {
    "target": "ES2022",
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "strict": true,
    "noUncheckedIndexedAccess": true,
    "outDir": "dist"
  },
  "include": ["src", "test"]
}

Model approved presentation separately from performance

Create src/governor.ts:

export type DeviceTier = "high" | "medium" | "low" | "unknown";

export type Profile =
  | "avatarQuality"
  | "avatarPerformance"
  | "sticker"
  | "camera"
  | "paused";

export interface Consent {
  avatarTracking: boolean;
  allowStickerFallback: boolean;
  allowCameraFallback: boolean;
}

export interface PolicyConfig {
  badWindowsBeforeDegrade: number;
  goodWindowsBeforeUpgrade: number;
}

export interface State {
  phase: "idle" | "applying" | "live" | "paused" | "error" | "stopped";
  ladder: readonly Profile[];
  active: Profile | null;
  target: Profile | null;
  badWindows: number;
  goodWindows: number;
  operationToken: number;
  lastError?: string;
}

export type Event =
  | { type: "START" }
  | { type: "SAMPLE"; overBudget: boolean }
  | { type: "APPLIED"; profile: Profile; token: number }
  | { type: "APPLY_FAILED"; profile: Profile; token: number; message: string }
  | { type: "CAMERA_INTERRUPTED"; message: string }
  | { type: "RETRY" }
  | { type: "STOP" };

export type Action = {
  type: "APPLY_PROFILE";
  profile: Profile;
  token: number;
};

export interface Transition {
  state: State;
  action?: Action;
}

export function buildLadder(
  tier: DeviceTier,
  consent: Consent,
): readonly Profile[] {
  if (!consent.avatarTracking) return ["paused"];

  const ladder: Profile[] = [];

  // Product-defined profiles, not Tencent RTC API or edition names.
  if (tier === "high") ladder.push("avatarQuality");
  ladder.push("avatarPerformance");

  if (consent.allowStickerFallback) ladder.push("sticker");
  if (consent.allowCameraFallback) ladder.push("camera");

  // There is always a fail-closed destination.
  ladder.push("paused");
  return ladder;
}

export function initialState(ladder: readonly Profile[]): State {
  return {
    phase: "idle",
    ladder,
    active: null,
    target: null,
    badWindows: 0,
    goodWindows: 0,
    operationToken: 0,
  };
}

function schedule(state: State, profile: Profile): Transition {
  const token = state.operationToken + 1;

  return {
    state: {
      ...state,
      phase: "applying",
      target: profile,
      operationToken: token,
      badWindows: 0,
      goodWindows: 0,
    },
    action: { type: "APPLY_PROFILE", profile, token },
  };
}

function indexOfActive(state: State): number {
  if (state.active === null) return -1;
  return state.ladder.indexOf(state.active);
}

export function reduce(
  state: State,
  event: Event,
  config: PolicyConfig,
): Transition {
  switch (event.type) {
    case "START":
    case "RETRY": {
      const first = state.ladder[0];
      if (!first) {
        return {
          state: { ...state, phase: "error", lastError: "Empty profile ladder" },
        };
      }
      return schedule(state, first);
    }

    case "APPLIED": {
      if (
        event.token !== state.operationToken ||
        event.profile !== state.target ||
        state.phase !== "applying"
      ) {
        return { state }; // stale completion
      }

      return {
        state: {
          ...state,
          phase: event.profile === "paused" ? "paused" : "live",
          active: event.profile,
          target: null,
          badWindows: 0,
          goodWindows: 0,
          lastError: undefined,
        },
      };
    }

    case "APPLY_FAILED": {
      if (
        event.token !== state.operationToken ||
        event.profile !== state.target
      ) {
        return { state };
      }

      const failedIndex = state.ladder.indexOf(event.profile);
      const fallback = state.ladder[failedIndex + 1];

      if (!fallback) {
        return {
          state: {
            ...state,
            phase: "error",
            active: null,
            target: null,
            lastError: event.message,
          },
        };
      }

      const next = schedule(
        { ...state, active: null, lastError: event.message },
        fallback,
      );
      return next;
    }

    case "SAMPLE": {
      if (state.phase !== "live" || state.active === null) {
        return { state };
      }

      const activeIndex = indexOfActive(state);

      if (event.overBudget) {
        const badWindows = state.badWindows + 1;

        if (badWindows < config.badWindowsBeforeDegrade) {
          return {
            state: { ...state, badWindows, goodWindows: 0 },
          };
        }

        const fallback = state.ladder[activeIndex + 1];
        return fallback
          ? schedule(state, fallback)
          : { state: { ...state, badWindows, goodWindows: 0 } };
      }

      const goodWindows = state.goodWindows + 1;

      // Recovery requires a longer stable period than degradation.
      if (
        activeIndex > 0 &&
        goodWindows >= config.goodWindowsBeforeUpgrade
      ) {
        const upgrade = state.ladder[activeIndex - 1];
        return upgrade ? schedule(state, upgrade) : { state };
      }

      return {
        state: { ...state, goodWindows, badWindows: 0 },
      };
    }

    case "CAMERA_INTERRUPTED":
      return {
        state: {
          ...state,
          phase: "error",
          active: null,
          target: null,
          operationToken: state.operationToken + 1,
          lastError: event.message,
        },
      };

    case "STOP":
      return {
        state: {
          ...state,
          phase: "stopped",
          active: null,
          target: null,
          operationToken: state.operationToken + 1,
          badWindows: 0,
          goodWindows: 0,
        },
      };
  }
}

There are two important design details here.

First, the ladder is created from consent. Runtime performance logic cannot insert the camera profile later if camera fallback was not originally allowed.

Second, degradation and recovery use different windows. This hysteresis prevents a device near its limit from switching profiles every few seconds.

The values for those windows are application policy, not universal Tencent RTC recommendations. Tune them using measurements from your supported devices.

Drive asynchronous profile changes

Create src/demo.ts:

import {
  buildLadder,
  initialState,
  reduce,
  type Action,
  type PolicyConfig,
  type Profile,
  type State,
} from "./governor.js";

interface BeautyAdapter {
  apply(profile: Profile): Promise<void>;
  hardStop(): Promise<void>;
}

const adapter: BeautyAdapter = {
  async apply(profile) {
    // Replace this switch with documented SDK calls for your target platform.
    console.log("Applying:", profile);

    switch (profile) {
      case "avatarQuality":
        // Configure the selected avatar and your quality-oriented settings.
        break;
      case "avatarPerformance":
        // Select performance-oriented settings. Reduce optional workload,
        // resolution, or frame rate according to measured device behavior.
        break;
      case "sticker":
        // Disable the avatar path before enabling an approved sticker.
        break;
      case "camera":
        // Publish an unmodified/approved camera presentation only if consented.
        break;
      case "paused":
        // Disable visual effects and stop visual publication/capture as required
        // by the product's privacy contract.
        break;
    }
  },

  async hardStop() {
    // Map this to your platform's local capture/publication shutdown path.
    console.log("Hard stop visual pipeline");
  },
};

const config: PolicyConfig = {
  badWindowsBeforeDegrade: 3,
  goodWindowsBeforeUpgrade: 8,
};

let state: State = initialState(
  buildLadder("low", {
    avatarTracking: true,
    allowStickerFallback: true,
    allowCameraFallback: false,
  }),
);

async function execute(action?: Action): Promise<void> {
  if (!action) return;

  try {
    await adapter.apply(action.profile);
    dispatch({
      type: "APPLIED",
      profile: action.profile,
      token: action.token,
    });
  } catch (error) {
    if (action.profile === "paused") {
      await adapter.hardStop();
    }

    dispatch({
      type: "APPLY_FAILED",
      profile: action.profile,
      token: action.token,
      message: error instanceof Error ? error.message : "Unknown apply failure",
    });
  }
}

function dispatch(event: Parameters<typeof reduce>[1]): void {
  const result = reduce(state, event, config);
  state = result.state;
  console.log(event.type, "=>", state.phase, state.active ?? state.target);
  void execute(result.action);
}

dispatch({ type: "START" });

Run it:

npm run demo

The adapter names above are application interfaces, not invented Tencent RTC APIs. Map each branch to the documented methods and settings for the Beauty AR SDK and target platform you use.

For the performance profile, make optional workload explicit. For example, if the experience does not require background segmentation or additional 3D/GAN effects, do not keep those features active merely because the quality profile used them. Resolution and frame rate should also be controlled as part of the profile rather than left as accidental global state.

Turn measurements into one policy signal

The reducer accepts overBudget rather than pretending one metric works on every device.

Your platform layer can derive that signal from a rolling observation window containing measurements such as:

• missed rendering deadlines;
• sustained frame-processing time;
• dropped or late frames;
• platform thermal or resource-pressure notifications;
• camera or effect-processing errors.

Do not degrade after one slow frame. Aggregate a window, record why it exceeded budget, and feed one policy result into the reducer.

A useful event record is:

interface PerformanceWindow {
  startedAt: number;
  endedAt: number;
  processedFrames: number;
  lateFrames: number;
  effectErrors: number;
  thermalPressure?: "normal" | "elevated" | "critical";
  overBudget: boolean;
  reason: string;
}

Keep the raw measurements in telemetry even though the reducer consumes only the boolean. Otherwise, you will know that degradation happened but not whether resolution, segmentation, avatar rendering, or another workload caused it.

Prove that privacy survives degradation

Create test/governor.test.ts:

import test from "node:test";
import assert from "node:assert/strict";
import {
  buildLadder,
  initialState,
  reduce,
  type PolicyConfig,
  type State,
} from "../src/governor.js";

const config: PolicyConfig = {
  badWindowsBeforeDegrade: 3,
  goodWindowsBeforeUpgrade: 8,
};

function step(state: State, event: Parameters<typeof reduce>[1]) {
  return reduce(state, event, config);
}

test("camera is absent when camera fallback was not approved", () => {
  const ladder = buildLadder("low", {
    avatarTracking: true,
    allowStickerFallback: false,
    allowCameraFallback: false,
  });

  assert.deepEqual(ladder, ["avatarPerformance", "paused"]);
});

test("three bad windows degrade to paused, never camera", () => {
  let state = initialState(
    buildLadder("low", {
      avatarTracking: true,
      allowStickerFallback: false,
      allowCameraFallback: false,
    }),
  );

  let result = step(state, { type: "START" });
  state = result.state;
  const start = result.action!;

  result = step(state, {
    type: "APPLIED",
    profile: start.profile,
    token: start.token,
  });
  state = result.state;

  state = step(state, { type: "SAMPLE", overBudget: true }).state;
  state = step(state, { type: "SAMPLE", overBudget: true }).state;
  result = step(state, { type: "SAMPLE", overBudget: true });

  assert.equal(result.action?.profile, "paused");
  assert.notEqual(result.action?.profile, "camera");
});

test("a stale profile completion cannot revive a stopped session", () => {
  let state = initialState(
    buildLadder("high", {
      avatarTracking: true,
      allowStickerFallback: true,
      allowCameraFallback: false,
    }),
  );

  const starting = step(state, { type: "START" });
  state = starting.state;
  const oldAction = starting.action!;

  state = step(state, { type: "STOP" }).state;
  state = step(state, {
    type: "APPLIED",
    profile: oldAction.profile,
    token: oldAction.token,
  }).state;

  assert.equal(state.phase, "stopped");
  assert.equal(state.active, null);
});

test("recovery requires a sustained stable period", () => {
  let state = initialState(["avatarQuality", "avatarPerformance", "paused"]);

  let result = step(state, { type: "START" });
  state = result.state;
  state = step(state, {
    type: "APPLIED",
    profile: "avatarQuality",
    token: result.action!.token,
  }).state;

  for (let i = 0; i < 3; i++) {
    result = step(state, { type: "SAMPLE", overBudget: true });
    state = result.state;
  }

  state = step(state, {
    type: "APPLIED",
    profile: "avatarPerformance",
    token: result.action!.token,
  }).state;

  for (let i = 0; i < 7; i++) {
    result = step(state, { type: "SAMPLE", overBudget: false });
    state = result.state;
    assert.equal(result.action, undefined);
  }

  result = step(state, { type: "SAMPLE", overBudget: false });
  assert.equal(result.action?.profile, "avatarQuality");
});

Run the checks:

npm test
npm run typecheck

These tests verify product invariants, not visual quality. You still need device testing for visual artifacts, heat, frame pacing, and SDK lifecycle behavior.

Failure drills that change the design

Avatar assets fail while the camera is already available

Camera availability is not camera consent. Descend through the precomputed ladder. If it ends at paused, show a clear message such as “Avatar unavailable; video remains off.”

Do not offer a camera fallback as if it were a retry. Make it a separate user action with an accurate preview.

An old profile finishes applying after the session stops

The operation token prevents the stale callback from changing application state. However, a token alone cannot undo an SDK side effect that already occurred.

Serialize adapter operations and reconcile the actual media state after backgrounding, stopping, or replacing a profile. The desired state and the observed SDK state should both be visible in diagnostics.

The paused profile itself fails

Treat stopping visual publication as a safety operation, not just another cosmetic profile. Invoke a lower-level capture/publication shutdown path and report the failure. Do not continue publishing the previous profile while displaying a paused badge.

The device repeatedly crosses the budget boundary

Use hysteresis: degrade relatively quickly, but require a longer healthy interval before upgrading. Also add a minimum dwell time if profile changes themselves are expensive.

If the session keeps moving down the ladder, preserve the lower profile instead of repeatedly chasing quality.

Tracking is lost but rendering still appears healthy

Performance telemetry does not cover tracking correctness. Add a separate tracking-health signal. Depending on the representation contract, the application can freeze the avatar briefly, show a reconnecting state, or pause visuals. It should not reveal the camera unless that mode was approved.

A decision framework for each fallback

Before adding a profile to the ladder, ask four questions:

1. Representation: Does this change what other participants believe they are seeing?
2. Consent: Did the participant approve this presentation mode, not merely camera access?
3. Budget: Which measured workload does the fallback remove?
4. Reversibility: Can the application restore the previous profile without flicker, stale assets, or an accidental camera frame?

A fallback that saves computation but violates representation consent is not graceful degradation. A fallback that preserves consent but removes no meaningful workload is not performance engineering. You need both.

Device verification checklist

Run these checks on every supported device tier:

• [ ] A device with an unknown tier starts conservatively.
• [ ] Camera fallback never appears without separate approval.
• [ ] Denying avatar tracking results in a paused visual state.
• [ ] Repeated over-budget windows move only downward through the approved ladder.
• [ ] One transient slow window does not trigger a switch.
• [ ] Recovery requires sustained healthy measurements.
• [ ] An avatar asset failure selects the next approved profile.
• [ ] Backgrounding during profile application cannot revive an old session.
• [ ] Camera interruption produces a visible recoverable state.
• [ ] Pausing actually stops the intended capture or publication path.
• [ ] Logs contain device tier, desired profile, applied profile, operation token, and degradation reason.
• [ ] Resolution, frame rate, segmentation, and optional effects match the selected profile.

The strongest avatar implementation is not the one that preserves every effect on every device. It is the one whose behavior remains understandable when the visual pipeline cannot preserve the ideal experience.

What should happen when a participant approves an avatar but rejects every visual fallback: a frozen last frame, a neutral placeholder, or no visual publication at all? That is a product and consent decision worth making explicitly before the SDK callback forces it.

---

Disclosure: I have a content relationship with Tencent RTC. Official Tencent RTC documentation was used as the implementation reference for this article.