AR makeup try-on rarely fails in the demo. It fails on the third page load, on a mid-range Android, in the Instagram in-app browser, or on a skin tone nobody on the team has. Six failure modes account for most of it: camera permission handling, double-mounted effects, shade fidelity, frame rate collapse, SKU mapping drift, and mobile browser policy.
Each one below has the same shape: what actually breaks, why React or the mobile browser makes it worse, and the test you can run before a shopper finds it.
The code examples use the Banuba virtual try-on widget because that is what we build, but every test method here is vendor-neutral. If you are integrating a different engine, the checks still apply.
Why does a try-on demo pass and the storefront fail?
A vendor demo runs one product, one browser, one lighting setup, and one face. A storefront runs your whole catalog across every device your analytics has ever seen. The gap between those two is where the six failures live.
It is worth being precise about what a web try-on actually is before testing it. It is a camera stream, a face tracking model, a rendering layer, and a mapping from your SKUs to render parameters. Three of the six failures below are in code you wrote, not in the engine.
1. What happens when the shopper denies camera access?
This is the most common one and the cheapest to fix. getUserMedia can be denied, dismissed, blocked by enterprise policy, or unavailable because the page is not on a secure origin. Per MDN, getUserMedia is only available in a secure context, meaning HTTPS or localhost, so a staging box on plain HTTP will fail every time and look like a bug in the widget.
A try-on with no fallback shows a black rectangle and the shopper leaves the page. A try-on with a fallback shows the product on a model photo and keeps the session alive.
async function openTryOn(widget) {
await widget.open();
const stream = await widget.useWebcam();
if (!stream) {
// Camera denied, dismissed, or unavailable. Do not leave the modal empty.
const model = await fetch("/model.webp").then((r) => r.blob());
await widget.useImage(model);
}
}
How to test it
In Chrome DevTools, open the three-dot menu, go to More tools, then Sensors, and block camera permission for the origin. Then repeat on plain HTTP. Both paths must render something. Track the denial rate as an event, because if a third of your sessions never get a camera, the fallback image is your real try-on experience and deserves the same design attention.
2. Why does the camera light stay on after the modal closes?
This one is specific to React and it is the failure most teams ship without noticing. React 18 in development deliberately mounts, unmounts, and remounts every component. The React documentation is blunt about the reason: Strict Mode calls effects twice so that "bugs caused by missing cleanup logic" surface in development instead of production.
If your effect opens the widget and never releases it, you get two widget instances and two camera streams. In development the symptom is a doubled modal.
In production the symptom is worse: the camera indicator stays lit after the shopper closes the try-on, which reads as spyware and generates support tickets.
import { useEffect, useRef } from "react";
export function TryOn({ merchantId, sku }) {
const ref = useRef(null);
useEffect(() => {
const widget = ref.current;
let cancelled = false;
(async () => {
await widget.open();
if (cancelled) await widget.close();
})();
return () => {
cancelled = true;
widget.close(); // release the stream
};
}, [sku]);
return <tint-vto ref={ref} merchant-id={merchantId} isolated-sku sku={sku} />;
}
How to test it
Open the try-on, close it, and watch the browser tab's camera indicator. It must go dark within a second. Then navigate between two product pages five times and confirm you still have exactly one widget instance in the DOM. Run this with Strict Mode on, not off. Turning Strict Mode off to make the double mount go away hides the bug rather than fixing it.
3. Does the shade look right on every skin tone, or only on yours?
This is the failure that costs money rather than support tickets, and it is the hardest to catch because the team testing it is usually not diverse enough to catch it.
The single most common complaint reported by virtual try-on buyers is filed, almost word for word, as "product color not displayed correctly": the shade on screen does not match the shade in the box. A naive color overlay washes out on deep skin and looks artificial on light skin, so the same lipstick reads as two different products depending on who is looking at it.
The fix is not something you test by eye once. Build a fixed test matrix and run it every release. The dermatology standard for this is the Fitzpatrick scale, which defines six skin phototypes, and it gives you a defensible set of rows rather than an arbitrary one.
Written out rather than only in the table: cover all six Fitzpatrick phototypes, four lighting conditions including backlit, at least four finishes including one high-coverage foundation, and both the live webcam path and the uploaded photo path. Twenty-four to forty-eight screenshots per release is a small price against a shade that only works on half your customers.
On the engine side, skin-tone-aware rendering is the specific capability that separates a try-on from a filter. The Banuba virtual try-on widget applies 16 makeup product types with skin-tone-aware application for exactly this reason, and supports 16 or more product categories including eyewear, hair color, contacts, jewelry, and accessories.
Whichever engine you pick, ask for its output across your matrix before you sign, not after.
4. What is your frame rate on the cheapest phone in your analytics?
Real-time face tracking plus rendering is a per-frame GPU and CPU budget. On a flagship phone you will never see the ceiling. On a three-year-old midrange Android the frame rate drops, the render lags behind the face by a few frames, and the try-on stops feeling like a mirror.
Pick a floor and enforce it. A practical one is 24 frames per second sustained over sixty seconds, since that is where motion stops reading as stuttering to most people.
Measure sustained rather than peak, because thermal throttling is what actually gets you: the first ten seconds look fine and second thirty do not.
let frames = 0;
const start = performance.now();
function tick() {
frames += 1;
if (performance.now() - start < 60000) requestAnimationFrame(tick);
else console.log("sustained fps:", frames / 60);
}
requestAnimationFrame(tick);
How to test it
run it on a real device, not the emulator, and not immediately after a charge. Pull the three most common device models out of your analytics and test the slowest one. If you only have flagships in the office, a cheap prepaid Android is the highest-value QA hardware you can buy.
5. Why does one SKU render as a generic tint?
Try-on engines do not render your product. They render the parameters someone mapped to your product: a region, a color, a finish, a coverage level.
When merchandising adds a shade and nobody digitizes it, the try-on either shows nothing or falls back to something approximate, and the shopper sees a color that is not the one on the box.
This is a data integrity problem, so treat it as one. Add a contract test in CI that walks your live catalog and asserts every purchasable SKU exists on the try-on side.
// Fails the build when merchandising adds a SKU that nobody digitized.
const skus = await getPurchasableSkus(); // your catalog
const ok = await widget.isCustomerSkuExist(skus); // try-on catalog
if (!ok) throw new Error("SKU present in catalog but missing from try-on");
That isCustomerSkuExist call is from the Banuba widget API, and most engines expose an equivalent lookup; the open web sample has a minimal page you can diff your own wiring against.
How to test it
Run that check nightly against production data, not just on the fixture set in your test suite. Fixtures never drift. Catalogs always do. Then decide the product behavior for a missing SKU: hiding the try-on button is almost always better than opening a try-on that shows the wrong color.
6. Does it work inside the Instagram browser?
A large share of beauty traffic arrives from social, which means the page opens in an in-app webview rather than Safari or Chrome.
Those webviews have their own camera permission behavior, and iOS has a long-standing quirk that catches teams out: WebKit requires the playsinline attribute on a video element, otherwise the stream takes over the whole screen in fullscreen playback instead of rendering in place.
If you embed the try-on in your own native app rather than the mobile web, the same constraint applies from the other direction. Camera access inside a webview needs a real HTTPS origin, so loading inline HTML without setting a proper base URL will fail even though the identical markup works in a browser tab:
// Android WebView: JS on, and a real HTTPS base URL or the camera never opens.
webView.settings.javaScriptEnabled = true
webView.settings.domStorageEnabled = true
webView.loadDataWithBaseURL("https://shop.example.com/", html, "text/html", "UTF-8", null)
How to test it
Send yourself the product URL through Instagram, Facebook, and TikTok on both iOS and Android, and open it from inside each app. Do not test by pasting the link into Safari. That is a different browser with different rules, and it is not where your traffic is.
What does a release checklist look like?
Pulling the six together, the pre-release pass is short:
- Camera denied and plain-HTTP origin both render a fallback, not a black box.
- Camera indicator goes dark within a second of closing, with Strict Mode on.
- Shade matrix screenshots taken across six phototypes and four lighting setups.
- Sustained 24 fps over sixty seconds on the slowest device in your top three.
- Nightly SKU contract test green against production catalog data.
- Product URL opened from inside Instagram on iOS and Android.
For the integration itself, the reference worth having open alongside the checklist is the widget integration guide, which documents the element, its attributes, and the lifecycle methods used above.
Bottom line
The engineering work in AR makeup try-on has moved. Face tracking and rendering are the vendor's problem now, and a first web integration is realistically a two-week front-end task rather than a computer vision project.
What has not moved is everything around it: permissions, component lifecycle, device spread, catalog integrity, and the browsers your customers actually use.
Those six are yours regardless of which engine you integrate, and they are the difference between a try-on that converts and one that quietly loses sessions. Build the test matrix before the launch, not after the first support ticket.
FAQ
Do I need a native app to run AR makeup try-on?
No. A web-based try-on loads as a script bundle and a custom HTML element on your existing product page, so it runs in the mobile browser without a separate app. A native app can embed the same widget through a webview when you want it inside an existing app experience.
Why does the try-on work on desktop but not on my staging server?
Almost always the secure context rule. Browsers only expose getUserMedia over HTTPS or localhost, so a staging box served over plain HTTP will never get a camera stream no matter how correct the integration is.
How do I test shade accuracy without a diverse test team?
Use a fixed matrix instead of ad hoc checks. Six Fitzpatrick phototypes against four lighting conditions gives you 24 screenshots per finish, which you can collect from a stock photo set for the upload path and from contractors or colleagues for the live path. The point is that it is repeatable between releases.
What frame rate is good enough?
Set a floor rather than chasing a number. Sustained 24 fps over a full minute on your slowest common device is a reasonable bar, because motion below that starts reading as stutter and thermal throttling shows up after the first few seconds.
Can one integration cover more than makeup?
Yes, if the engine supports it. The Banuba widget covers 16 or more product categories including eyewear, hair color, contacts, jewelry, and accessories, so a storefront that integrates for lipstick can enable additional categories later through configuration rather than a second integration.
What breaks most often after launch?
Catalog drift. The integration is static once it ships, but merchandising keeps adding SKUs, and any SKU that is not digitized on the try-on side either disappears or renders approximately. A nightly contract test catches it before a customer does.
Top comments (0)