Don’t Stack Every Beauty AR Effect: Build a Runtime Budget Arbiter for GAN, Segmentation, and Avatars

A shared demo has a predictable failure mode: every contributor adds something visually impressive, and nobody wants to remove anything.

A beauty filter is joined by background segmentation. Then someone adds a GAN effect, a 3D avatar, and a sticker layer. The pull requests all work independently. The combined application does not.

The tension is not that the team lacks AI features. It is that saying “no” to an effect can feel less ambitious than stacking everything—especially during a hackathon or community build. But a reliable Beauty AR experience needs an explicit rendering budget, not optimism.

In this tutorial, we will build an application-level runtime budget arbiter that:

• allows only one intensive presentation mode at a time;
• distinguishes the user’s requested mode from the mode currently visible;
• blocks unsupported GAN, segmentation, or 3D modes on constrained devices;
• applies a basic fallback only when the user has approved it;
• removes an intensive mode if the device is reclassified during the session;
• restores the last working mode after an application failure; and
• exposes every decision as testable state.

Tencent RTC’s Beauty AR overview describes capabilities including real-time beauty filters, makeup, stickers, virtual backgrounds, avatars, gesture recognition, and image or video enhancement. Its Low-End Device Performance Optimization Practice Guide recommends adapting effects to device tiers, including disabling expensive segmentation or 3D/GAN effects and controlling resolution and frame rate.

The controller below turns that guidance into deterministic application behavior without inventing a universal device benchmark.

Start with the collision, not the SDK call

Consider these user actions:

1. User selects a segmented background.
2. The background becomes visible.
3. User selects a GAN portrait mode.
4. The application removes segmentation before enabling GAN.
5. Device pressure becomes sustained.
6. The device is reclassified as constrained.
7. GAN is removed.
8. Basic enhancement is enabled only if the user approved that fallback.

There are three different facts in this sequence:

• Requested mode: what the user most recently chose.
• Desired mode: what policy currently permits.
• Visible mode: what the renderer last applied successfully.

Collapsing those into one currentEffect variable creates misleading UI. For example, the application may say “GAN selected” while the asset is still loading or while policy has already rejected it.

We will keep those facts separate.

The admission policy

This tutorial uses five application-owned modes:

Mode	Classification	Constrained device	Standard device
off	No Beauty AR presentation	Allow	Allow
basic	Basic enhancement profile	Allow	Allow
ganPortrait	Intensive	Block or consented fallback	Allow
segmentedBackground	Intensive	Block or consented fallback	Allow
avatar3d	Intensive	Block or consented fallback	Allow

These are product-policy names, not Tencent RTC API names. Your adapter will map them to the supported SDK operations for your target platform.

The important rule is that a mode represents an exact presentation, rather than another layer to append. Selecting ganPortrait replaces segmentedBackground; it does not silently stack with it.

This policy deliberately leaves stickers and makeup combinations out of the first version. Add combinations only after measuring them on your supported devices. “The SDK can express it” and “our session can afford it” are different statements.

Create the TypeScript project

mkdir beauty-budget-arbiter
cd beauty-budget-arbiter
npm init -y
npm install -D typescript vitest @types/node
mkdir src

Replace package.json with:

{
  "name": "beauty-budget-arbiter",
  "private": true,
  "type": "module",
  "scripts": {
    "test": "vitest run",
    "test:watch": "vitest"
  },
  "devDependencies": {
    "@types/node": "latest",
    "typescript": "latest",
    "vitest": "latest"
  }
}

Add tsconfig.json:

{
  "compilerOptions": {
    "target": "ES2022",
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "strict": true,
    "noUncheckedIndexedAccess": true,
    "outDir": "dist"
  },
  "include": ["src"]
}

Implement the arbiter

Create src/arbiter.ts:

export type BeautyMode =
  | "off"
  | "basic"
  | "ganPortrait"
  | "segmentedBackground"
  | "avatar3d";

export type DeviceCapability = "constrained" | "standard";
export type Phase = "active" | "applying" | "blocked" | "failed";

export interface Selection {
  mode: BeautyMode;

  // This is separate consent. Selecting GAN does not automatically mean
  // that the user accepts a different appearance when GAN is unavailable.
  allowBasicFallback: boolean;
}

export interface BeautyPort {
  /**
   * Apply exactly this mode, removing any mutually exclusive previous mode.
   * Reject if the operation cannot be completed.
   */
  applyMode(mode: BeautyMode): Promise<void>;
}

export interface ArbiterSnapshot {
  capability: DeviceCapability;
  requested: BeautyMode;
  desired: BeautyMode;
  visible: BeautyMode;
  phase: Phase;
  notice: string | null;
}

type Admission =
  | { kind: "admitted"; target: BeautyMode; notice: string | null }
  | { kind: "blocked"; notice: string };

export function isIntensive(mode: BeautyMode): boolean {
  return (
    mode === "ganPortrait" ||
    mode === "segmentedBackground" ||
    mode === "avatar3d"
  );
}

export function admit(
  selection: Selection,
  capability: DeviceCapability,
): Admission {
  if (capability === "standard" || !isIntensive(selection.mode)) {
    return { kind: "admitted", target: selection.mode, notice: null };
  }

  if (selection.allowBasicFallback) {
    return {
      kind: "admitted",
      target: "basic",
      notice: `${selection.mode} is unavailable in the current device profile; basic mode was selected.`,
    };
  }

  return {
    kind: "blocked",
    notice: `${selection.mode} is unavailable in the current device profile and no fallback was approved.`,
  };
}

export class BeautyBudgetArbiter {
  private requested: BeautyMode = "off";
  private desired: BeautyMode = "off";
  private visible: BeautyMode = "off";
  private phase: Phase = "active";
  private notice: string | null = null;
  private lastSelection: Selection = {
    mode: "off",
    allowBasicFallback: false,
  };

  // All renderer mutations are serialized through this queue.
  private tail: Promise<void> = Promise.resolve();

  constructor(
    private capability: DeviceCapability,
    private readonly port: BeautyPort,
  ) {}

  select(selection: Selection): void {
    this.lastSelection = selection;
    this.requested = selection.mode;

    const decision = admit(selection, this.capability);

    if (decision.kind === "blocked") {
      // If an intensive mode is already visible and has become invalid,
      // remove it. Otherwise preserve the last permitted presentation.
      this.desired = isIntensive(this.visible) ? "off" : this.visible;
      this.phase = "blocked";
      this.notice = decision.notice;
    } else {
      this.desired = decision.target;
      this.phase = this.desired === this.visible ? "active" : "applying";
      this.notice = decision.notice;
    }

    this.scheduleReconcile();
  }

  updateCapability(capability: DeviceCapability): void {
    if (capability === this.capability) return;

    this.capability = capability;

    // Re-evaluate the user's last explicit selection. Do not invent a new
    // preference merely because the device classification changed.
    this.select(this.lastSelection);
  }

  snapshot(): ArbiterSnapshot {
    return {
      capability: this.capability,
      requested: this.requested,
      desired: this.desired,
      visible: this.visible,
      phase: this.phase,
      notice: this.notice,
    };
  }

  async whenSettled(): Promise<void> {
    await this.tail;
  }

  private scheduleReconcile(): void {
    this.tail = this.tail.then(() => this.reconcile());
  }

  private async reconcile(): Promise<void> {
    const target = this.desired;

    if (target === this.visible) return;

    const previous = this.visible;

    try {
      await this.port.applyMode(target);
      this.visible = target;

      // A newer selection may have arrived while the port was awaiting.
      this.phase = this.desired === target ? "active" : "applying";
    } catch (error) {
      try {
        // applyMode is an exact-mode contract, so reapplying the previous
        // mode also compensates for a partially completed mutation.
        await this.port.applyMode(previous);
        this.visible = previous;
      } catch {
        // If restoration also fails, prefer an explicit neutral state.
        try {
          await this.port.applyMode("off");
        } finally {
          this.visible = "off";
        }
      }

      this.phase = "failed";
      this.notice = `Could not apply ${target}; restored ${this.visible}.`;
    }
  }
}

There are two intentional constraints here.

First, renderer mutations are serialized. Rapid clicks cannot launch three competing effect changes and let the slowest callback win.

Second, applyMode() means “make this the exact active mode.” The application does not ask the SDK adapter to “add GAN” without accounting for segmentation already being active.

Reproduce the unsafe paths in tests

Create src/arbiter.test.ts:

import { describe, expect, it } from "vitest";
import {
  BeautyBudgetArbiter,
  type BeautyMode,
  type BeautyPort,
} from "./arbiter.js";

class FakeBeautyPort implements BeautyPort {
  history: BeautyMode[] = [];
  failures = new Set<BeautyMode>();

  async applyMode(mode: BeautyMode): Promise<void> {
    this.history.push(mode);

    if (this.failures.has(mode)) {
      throw new Error(`Failed to apply ${mode}`);
    }
  }
}

describe("BeautyBudgetArbiter", () => {
  it("blocks GAN on a constrained device without inventing consent", async () => {
    const port = new FakeBeautyPort();
    const arbiter = new BeautyBudgetArbiter("constrained", port);

    arbiter.select({ mode: "ganPortrait", allowBasicFallback: false });
    await arbiter.whenSettled();

    expect(arbiter.snapshot()).toMatchObject({
      requested: "ganPortrait",
      desired: "off",
      visible: "off",
      phase: "blocked",
    });
    expect(port.history).toEqual([]);
  });

  it("uses basic mode only when fallback consent exists", async () => {
    const port = new FakeBeautyPort();
    const arbiter = new BeautyBudgetArbiter("constrained", port);

    arbiter.select({ mode: "ganPortrait", allowBasicFallback: true });
    await arbiter.whenSettled();

    expect(arbiter.snapshot()).toMatchObject({
      requested: "ganPortrait",
      desired: "basic",
      visible: "basic",
      phase: "active",
    });
    expect(port.history).toEqual(["basic"]);
  });

  it("ends in the latest mode after rapid selections", async () => {
    const port = new FakeBeautyPort();
    const arbiter = new BeautyBudgetArbiter("standard", port);

    arbiter.select({ mode: "segmentedBackground", allowBasicFallback: false });
    arbiter.select({ mode: "ganPortrait", allowBasicFallback: false });
    arbiter.select({ mode: "off", allowBasicFallback: false });

    await arbiter.whenSettled();

    expect(arbiter.snapshot().visible).toBe("off");
    expect(port.history).not.toContain("ganPortrait");
  });

  it("removes an intensive mode after a capability downgrade", async () => {
    const port = new FakeBeautyPort();
    const arbiter = new BeautyBudgetArbiter("standard", port);

    arbiter.select({ mode: "avatar3d", allowBasicFallback: false });
    await arbiter.whenSettled();
    expect(arbiter.snapshot().visible).toBe("avatar3d");

    arbiter.updateCapability("constrained");
    await arbiter.whenSettled();

    expect(arbiter.snapshot()).toMatchObject({
      requested: "avatar3d",
      visible: "off",
      phase: "blocked",
    });
  });

  it("restores the last working mode when the new mode fails", async () => {
    const port = new FakeBeautyPort();
    const arbiter = new BeautyBudgetArbiter("standard", port);

    arbiter.select({ mode: "basic", allowBasicFallback: false });
    await arbiter.whenSettled();

    port.failures.add("ganPortrait");
    arbiter.select({ mode: "ganPortrait", allowBasicFallback: false });
    await arbiter.whenSettled();

    expect(arbiter.snapshot()).toMatchObject({
      requested: "ganPortrait",
      visible: "basic",
      phase: "failed",
    });
    expect(port.history).toEqual(["basic", "ganPortrait", "basic"]);
  });
});

Run the suite:

npm test

These tests verify policy and ordering without requiring a camera. They do not prove that a real device can sustain a selected mode. That remains an on-device verification task.

Connect the port to Tencent RTC Beauty AR

Keep SDK-specific operations outside the arbiter. One useful adapter shape is:

import type { BeautyMode, BeautyPort } from "./arbiter.js";

type BeautyBindings = {
  disableCurrentPresentation(): Promise<void>;
  enableBasic(): Promise<void>;
  enableGanPortrait(): Promise<void>;
  enableSegmentedBackground(): Promise<void>;
  enableAvatar3d(): Promise<void>;
};

export class TencentBeautyAdapter implements BeautyPort {
  constructor(private readonly bindings: BeautyBindings) {}

  async applyMode(mode: BeautyMode): Promise<void> {
    await this.bindings.disableCurrentPresentation();

    switch (mode) {
      case "off":
        return;
      case "basic":
        return this.bindings.enableBasic();
      case "ganPortrait":
        return this.bindings.enableGanPortrait();
      case "segmentedBackground":
        return this.bindings.enableSegmentedBackground();
      case "avatar3d":
        return this.bindings.enableAvatar3d();
    }
  }
}

The binding names above are your own interface, not claims about Tencent RTC API names. Implement them using the official Beauty AR integration documentation for the platform and SDK version you selected.

Disabling the previous presentation before enabling the next one has a trade-off: users may briefly see a neutral transition. The alternative—enabling the candidate first—may temporarily overlap two intensive workloads. Choose deliberately, then verify the transition on real devices.

If your selected SDK integration offers a safe staging mechanism, the adapter can use it without changing the policy controller.

Where the device classification comes from

Do not turn a phone model list into permanent truth. Firmware, thermal conditions, camera resolution, concurrent applications, and the rest of the live session can all affect available budget.

Use two inputs:

1. A conservative initial classification based on the devices and configurations your team actually supports.
2. Session measurements from the complete camera, Beauty AR, and RTC workload.

Your capability estimator can call:

arbiter.updateCapability("constrained");

when degradation is sustained, and later return to standard after a separate recovery condition.

Use hysteresis: entering and leaving constrained mode should not depend on the same single threshold or one bad frame. Otherwise, the application can oscillate between GAN and basic modes. This tutorial intentionally does not prescribe universal frame-rate, temperature, or timing thresholds; establish them from repeatable tests on your workload.

Resolution and frame rate are another part of the budget. The official low-end optimization guide recommends controlling them, but reducing them should be represented as an explicit media policy—not hidden inside an effect callback.

Failure modes to drill on hardware

The GAN asset cannot load

Expected behavior:

• phase becomes failed;
• the previous working mode is restored;
• the UI does not claim GAN is visible;
• retry is a user-visible action rather than an infinite background loop.

Test with unavailable or invalid test assets in a non-production environment where your integration permits it.

Disabling segmentation succeeds, but enabling the next mode fails

This is why the adapter receives an exact-mode command and the controller reapplies the previous mode. Verify that restoration really reconstructs the previous presentation; do not assume an SDK rejection is atomic.

The user revokes fallback consent

Update the stored selection and run it through the arbiter again. Do not leave basic active merely because it was once approved.

Consent belongs in application state that can be changed and audited—not only in the label of a button shown at first launch.

The application enters the background during an effect change

Decide whether your platform integration pauses, clears, or reconstructs Beauty AR state. On return, compare renderer reality with desired; do not assume the pre-background callback still represents the current session.

Remote configuration changes the policy

Treat remote configuration as an input to admission, not as permission to alter someone’s appearance silently. A remotely disabled GAN mode can become blocked, but switching to basic enhancement still requires the fallback choice already modeled here.

Capability classification flaps

If the application alternates between standard and constrained, fix the estimator. Adding debounce logic to the renderer adapter merely hides an unstable policy signal.

What AI contributes—and what remains a human decision

GAN-based effects and real-time computer vision can produce visual transformations. That is a demonstrated capability, not evidence that an “AI agent” should decide how a person appears.

The human decisions remain:

• whether any appearance-changing mode is enabled;
• whether a different fallback appearance is acceptable;
• whether reduced resolution is an acceptable trade-off;
• whether an effect should be removed when the device is under pressure; and
• which devices and session conditions the team is willing to support.

The durable engineering skill here is not finding the most impressive model. It is turning visual capability into a bounded system with visible state, consent, measurements, and recovery.

Release verification checklist

Before shipping, verify the complete experience rather than only the TypeScript tests:

• [ ] Selecting each mode updates requested immediately.
• [ ] The UI shows an applying state until the renderer confirms success.
• [ ] Only one intensive mode is visible at a time.
• [ ] A constrained classification blocks GAN, segmentation, and 3D modes according to policy.
• [ ] Basic fallback is never applied without separate consent.
• [ ] A mid-session capability downgrade removes an intensive mode.
• [ ] A failed activation restores the previous working presentation.
• [ ] A failed restoration reaches an explicit neutral state.
• [ ] Rapid selections end in the latest requested and admitted mode.
• [ ] Background and foreground transitions reconcile renderer state.
• [ ] Tests use the full RTC, camera, and Beauty AR workload at the chosen resolution and frame rate.
• [ ] UI language distinguishes requested, unavailable, applying, active, and failed states.

Discussion: should the policy allow combinations?

Eventually, perhaps—but “allow combinations” should mean a measured compatibility table, not an unbounded array of effects.

A sensible next iteration is to identify profiles such as basic + sticker or makeup + segmented background, qualify each complete profile, and continue applying one exact profile at a time. This preserves deterministic rollback and prevents independently contributed features from accidentally creating an unsupported workload.

The arbiter is therefore not a ceiling on creativity. It is the place where the team records which combinations it can responsibly operate.

Disclosure: I have a relationship with Tencent RTC, and I used the official Tencent RTC documentation as the implementation reference for this article.