We Ran the Same Four Questions Past Nine AI Engines. The Split Was Clean.

We are Proofmend, a search and answer-engine services shop. This month we ran a controlled sampling pass across nine AI engines to see how brand recall and recommendation differ inside these systems. The numbers were cleaner than we expected, and they change what "getting cited by AI" should mean in a scope document.

How the pass was run

Nine engines: ChatGPT, Claude, Gemini, Perplexity, Grok, Doubao, DeepSeek, Kimi, Qwen. Four questions each. Every question in its own fresh session, default model tier, no deep-research modes. Two question types kept separate, branded probes ("what is X and is it legitimate") and unbranded recommendation queries ("what is the best tool for this job").

One engine's account carried persistent memory that recognized the operator, so all four of its answers were discarded rather than cleaned. That is the kind of thing that quietly ruins a sampling pass if you do not check for it.

The result

On branded probes, eight of nine engines described the tested products accurately, and most cited the official site directly.

On unbranded recommendation queries, the tested brands were mentioned zero times.

Same engines. Same minute. Opposite outcomes.

Why the two question types diverge

They travel different retrieval paths.

A branded query sends the engine looking for pages about that name. Official site, product pages, third-party descriptions. Any of those can satisfy it, which is why a well-structured site wins this half.

An unbranded query sends the engine looking for pages that already rank things. Roundups, comparisons, "best X tools in 2026" listicles. The engine does not invent a ranking; it assembles one from sources that already contain rankings.

The sharpest evidence we got: on one engine, a tested brand's own domain appeared in the citation pool for a recommendation query, while the brand itself was absent from the answer's shortlist. The page was retrieved and then not used. A product page tells the model "this is a tool." A roundup hands the model a ready-made ordering. Only the second one is usable when assembling a recommendation.

What this means for scoping the work

	Classic search work	Answer-engine work
Target	Ranking on a results page	Presence in an assembled answer
Content that wins	Keyword coverage, page authority	Extractable blocks: comparisons, numeric facts, definitions, steps
Measurement	Position and traffic	Mention rate, citation source mix, ordering

Two more findings worth carrying into any scope document.

Rankings inside AI answers are unstable. We asked one engine the same question twice a few hours apart and got substantially different shortlists. Changing the reasoning tier on the same engine produced two shortlists with almost no overlap, and ordinary users are on the default tier.

That instability has a hard limit though. Volatility reshuffles names already in the candidate pool. It never pulls in a name that is absent from the sources. Zero mentions is not bad luck, it is structural.

The honest boundary

Because of all this, we do not promise rankings, AI mentions, citations, traffic, or revenue. Three numbers from this pass sit behind that sentence: nine engines tested, four questions each, zero mentions on unbranded queries. What we commit to is recording the conditions, questions, pages, platforms, modes and timestamps, labeling anything unmeasured as unknown rather than estimating it, and separating written deliverables from platform-side outcomes.

Sampling reflects August 2026 and a limited sample. Run the same question twice yourself and you will see the same variance we did.

Written by the Proofmend team. Service scope and stated limits at https://proofmend.com, checked August 22, 2026.