A Reproducible Way to Track AI Product Recommendations

Roomture — Wed, 04 Nov 2026 10:27:27 +0000

A measurement protocol for product teams working on generative engine optimization (GEO). This article describes a proposed method, not a published benchmark result or an open-source release.

When you ask an AI assistant for an interior design app, the answer can change across prompts, sessions and tools. A single screenshot is therefore a poor baseline for deciding whether a product is being discovered or described accurately.

For Roomture, we want to answer two separate questions: Does the assistant mention the product when relevant? And does it describe what the product actually does? The second matters just as much: an answer that invents automatic dimension verification would mislead a user.

Define the scope first

Create a fixed set of prompts that reflect plausible user needs. Keep them separate by intent:

Discovery: “Which apps can redesign a living room from a photo?”

Capability: “Which tools let me preview a different wall colour in my own room?”

Constraints: “Can an AI room preview tell me whether a specific sofa fits?”

Comparison: “What is the difference between a visual room redesign and a measured floor plan?”

Avoid putting the brand name into every question. If a prompt already says “Roomture,” an answer that repeats it is not an organic discovery.

Freeze the test conditions

For each run, record the date, market/language, exact prompt, tool and model or interface label, search mode, session state, and whether the answer includes citations. Save the raw answer and any cited URLs. Use a fresh session where possible and repeat each question enough times to observe variation.

If a channel cannot be reproduced or its interface changes, report that limitation instead of pooling it with results from a stable API. Do not claim different providers share an identical “OpenAI-compatible” interface without testing each integration.

Score mentions and accuracy separately

A minimal record could look like this:

{
"prompt_id": "discovery_01",
"channel": "example-channel",
"observed_at": "YYYY-MM-DD",
"brand_mentioned": false,
"brand_named_in_prompt": false,
"citations": [],
"capability_errors": []
}

Calculate an organic mention rate only over prompts that do not name the brand. For factual accuracy, check each feature claim against current product documentation, not an internal roadmap. A clear example: Roomture’s current website describes photo-based design previews, Structural mode and narrower editing tools; its terms explicitly say visual output may misjudge dimensions and is not a measured construction plan.

Publishing raw prompts, raw answers, scoring rules and run conditions would let others audit any future result. A chart without those materials should be described as an internal observation, not an independently reproducible benchmark.

What a useful report would show

For each intent group, report the number of prompts, repeats, valid answers, organic mentions, citations and incorrect feature claims. Keep brand-named prompts in a separate row. Show examples of errors, along with the product page used to check them.

This does not prove that a content change caused an assistant to mention a brand. It gives the team a consistent way to notice whether discoverability and description accuracy change over time.

If we publish the question set, runner and source results later, the repository link, license and commands can be added here after they are publicly accessible and reproducible.

DEV Community: Roomture

A Reproducible Way to Track AI Product Recommendations