DEV Community: tengence

Whom Do ChatGPT, Perplexity, and Gemini Cite? A Breakdown of the Three AI Platforms' Source Preferences

tengence — Mon, 12 Oct 2026 09:09:30 +0000

Summary: The three AI platforms differ sharply in source preference: ChatGPT leans on encyclopedic and authoritative references, Perplexity leans on community discussion and reviews, and Gemini with Google AI Overviews is the most balanced, weighting video and professional communities high. Optimize per platform instead of one content set for all.

The same company with the same piece of content can differ severalfold in citation probability across the three AI platforms — not because of content quality, but because the platforms' source-selection mechanisms are fundamentally different. This article uses three sets of traceable public data — Profound's analysis of 30M citations accumulated from August 2024 to June 2025, SE Ranking's June 2026 study of referral traffic across 101,574 sites, and Muck Rack's May 2026 tally of over 25M citation links — to break down, platform by platform, whom each cites, how many citation slots each answer grants, and where content assets should be tilted.

1. The takeaway first: one sentence per platform

The three platforms can be summed up in one line: ChatGPT trusts "reference material", Perplexity trusts "community discussion", and Gemini with Google AI Overviews trust "multi-source confirmation".

This difference is not a detail but the premise that decides how content is written. If the target is ChatGPT, the priority is to make the content citeable as a reference — complete definitions, sourced data, stable phrasing. If the target is Perplexity, the priority is to have real users discuss your brand and third-party reviews mention it. If the target is Google's AI Overviews, the priority is traditional ranking plus cross-confirmation from multiple sources; a single piece rarely works alone.

2. ChatGPT: encyclopedic and authoritative references at the core

Among Profound's top-ten citation sources, Wikipedia leads at about 47.9%, far above second-place Reddit at about 11.3%. The rest follow: Forbes (about 6.8%), G2 (about 6.7%), TechRadar (about 5.5%), NerdWallet (about 5.1%), Business Insider (about 4.9%), New York Post (about 4.4%). Looking at full-volume citations rather than just the top ten, Profound's stat based on 680M citations shows Wikipedia accounts for about 7.8% of all ChatGPT citations — still the single largest source.

This distribution says two things. First, ChatGPT has a stable preference for "definitional, factual" content, so on queries like concept explanation, parameter description, and standards and specs, pages with complete structure and neutral phrasing are more likely to be picked. Second, its source structure carries a sizable share of traditional media and review sites, meaning third-party endorsement remains an important channel — saying something ten times yourself is worth less than one line quoted by a media outlet it trusts.

Note that ChatGPT's retrieval runs on Bing's index. This means if a site is not properly indexed by Bing, even quality content will hardly appear in its citations.

3. Gemini and Google AI Overviews: the most balanced, with high weight on video and professional communities

The top-ten citation sources of Google AI Overviews are clearly flatter: Reddit about 21.0%, YouTube about 18.8%, Quora about 14.3%, LinkedIn about 13.0%, Gartner about 7.1%, NerdWallet about 5.9%, Wikipedia about 5.7%, Forbes about 5.7%, Business Insider about 5.7%, Business Insider about 4.5%.

Three traits deserve separate attention. First, video carries heavy weight: YouTube takes nearly a fifth, and BrightEdge's stat also shows about 29.5% of Google AI Overviews cite YouTube content — so for decision-stage queries like "how to" and "how to choose", the value of video content is significantly amplified. Second, professional communities weigh high: LinkedIn and Gartner both rank near the top, showing professional identity and research-institute reports carry extra weight in Google's judgment. Third, Wikipedia's weight is actually low (about 5.7%), a sharp contrast with ChatGPT.

Platform	First source	Second source	Third source	Structure trait
ChatGPT	Wikipedia 47.9%	Reddit 11.3%	Forbes 6.8%	Single source highly concentrated
Google AI Overviews	Reddit 21.0%	YouTube 18.8%	Quora 14.3%	Multi-source balanced, high video share
Perplexity	Reddit 46.7%	YouTube 13.9%	Gartner 7.0%	Community and reviews both high

(Data source: Profound, August 2024 – June 2025; the metric is the share distribution of each platform's top-ten citation sources, not the full-volume citation share.)

4. Perplexity: community discussion and third-party reviews dominate

Among Perplexity's top-ten citation sources, Reddit leads at about 46.7%, close to half, followed by YouTube (about 13.9%), Gartner (about 7.0%), Yelp (about 5.8%), LinkedIn (about 5.3%), Forbes (about 5.0%), NerdWallet (about 4.5%), TripAdvisor (about 4.1%), G2 (about 4.0%).

Review and rating sites (Yelp, TripAdvisor, G2, Gartner) together take a clearly higher share than the other two platforms, matching Perplexity's user mix — researchers, developers, and professional buyers are over-represented, and they ask more "which is better" and "what is the difference" comparison questions, whose answers naturally depend on third-party reviews and real user ratings.

Another Perplexity trait is real-time retrieval. It does not replay training data but launches retrieval at question time, so the cycle from a new page going live to its first citation is noticeably shorter, which favors time-sensitive content.

5. How many citation slots per answer: the count difference matters too

Looking only at "whom they cite" misses the other half of the picture — how many citation slots each platform grants in one answer. Muck Rack's May 2026 tally of over 25M citation links shows: ChatGPT gives a citation in about 96% of answers, averaging about 5 sources; Gemini gives a citation in about 82% of answers, averaging about 8 sources; Claude gives a citation in only about 55% of answers, the lowest of the three. Qwairy's stat based on 118,101 answers shows Perplexity averages about 22 citations per answer — far above ChatGPT's about 8.

The practical meaning of these numbers is: the total number of citation slots is not proportional to the share of referral traffic. Gemini grants more citation slots per answer than ChatGPT, yet its referral-traffic share is far smaller (next section). If the goal is "to be named", the number of seats per answer is a more direct metric than the platform's traffic share.

6. Referral traffic and citation opportunity are different things — don't use the wrong metric

SE Ranking's June 2026 study (covering 101,574 sites, 250 countries, tracking window January 2025 to April 2026) gives each platform's AI referral-traffic share: ChatGPT about 74.78% (79.74% a year earlier), Gemini about 11.56% (up about 231% year over year), Perplexity about 7.23%, Copilot about 3.51% (up about 31%), Claude about 2.62% (up about 320%).

Platform	AI referral share	YoY change	Citation behavior	More likely picked by
ChatGPT	74.78%	Share down, absolute volume up about 27%	96% of answers cite, avg about 5	Reference content with complete definitions and sourced data
Gemini	11.56%	+231%	82% of answers cite, avg about 8	Strong traditional ranking + multi-source confirmation
Perplexity	7.23%	Slight decline	Highest citation count, about 22 per answer	Extractable passages + strong timeliness
Copilot	3.51%	+31%	Follows Bing index	Bing indexing and ranking
Claude	2.62%	+320%	Only 55% of answers cite	Depth and accuracy first

(Share and growth source: SE Ranking, 2026-06-18; citation-behavior source: Muck Rack, 2026-05.)

Stacking the two tables, the conclusion is: ChatGPT has the largest referral traffic but grants the fewest slots per answer, while Gemini and Perplexity grant more citation slots but currently have smaller referral traffic. Allocating by traffic share alone systematically misses the platforms "most willing to name you".

7. The three platforms differ in crawl and update mechanics

Beyond source preference, the crawl mechanism decides how soon content becomes visible. ChatGPT's retrieval depends on Bing's index, so Bing Webmaster Tools indexing status is a prerequisite; if a CDN or WAF blocks OAI-SearchBot and GPTBot by user-agent, it silently excludes the whole site from the index. Gemini and Google AI Overviews use Google's index directly, and traditional ranking remains the main entry condition — BrightEdge's research shows most cited sources land in the top ten of organic results. Perplexity retrieves in real time and clearly prefers recently updated pages; timeliness is its most prominent difference.

The resulting actions differ too: on the ChatGPT side, prioritize checking Bing indexing and crawler allowance; on the Google side, prioritize traditional ranking and content freshness; on the Perplexity side, prioritize keeping key pages always crawlable with clear, extractable passage structure.

8. How to allocate content assets by platform

When allocating content assets by platform, map by "asset type → platform" rather than "platform → what to make". The directly usable mapping is: definition and spec pages mainly serve ChatGPT; third-party reviews, ratings, and real user discussion mainly serve Perplexity and Google AI Overviews; video and professional-community content (including research reports) mainly serves Google AI Overviews and Gemini; and the common base across all platforms is the same structured body — the first sentence of each subheading gives a self-contained 40–60 character answer, key conclusions carry specific numbers and dates, and Q&A is written as independently extractable passages.

On prioritization, most teams should first ensure the "common base" is met, then pick one or two platforms to focus on by target audience. If the audience is developers, analysts, and researchers, Perplexity's marginal return is higher; if it is mass consumer and general procurement, Google AI Overviews and ChatGPT cover more.

9. Three common misconceptions

The first misconception is "one content set for all platforms". The source-structure differences across the three platforms are structural: the same piece may miss on ChatGPT for lacking reference-style phrasing and miss on Perplexity for lacking third-party discussion.

The second misconception is "decide investment by referral-traffic share alone". As the two tables above show, referral share and citation-slot count are not aligned; investing only in the largest-share platform undervalues the opportunity cost of being named.

The third misconception is "treat UGC as something to mass-produce". Community content like Reddit and Quora gets cited repeatedly precisely because of authenticity and natural filtering; mass-posted marketing content is usually accepted by neither the community nor the platform.

About Tengence

Multi-platform citation monitoring and content-structure diagnosis are among the core capabilities of Tengence GEO; its strategy generation, content go-live, and effect dashboard cover the per-platform source analysis and citation-baseline building discussed here. Tengence (TENGENCE) is the enterprise growth engine of the AI era, focused on public-traffic acquisition and private-domain conversion; its core products are Tengence GEO (dual-engine GEO + SEO) and Tengence Search.

Data Sources

Profound citation-source distribution study: Tracking window 8/2024 to 6/2025, about 30M citations total. Among ChatGPT's top-ten sources Wikipedia is about 47.9%, Reddit about 11.3%; in Google AI Overviews Reddit is about 21.0%, YouTube about 18.8%, Quora about 14.3%, LinkedIn about 13.0%; in Perplexity Reddit is about 46.7%, YouTube about 13.9%. A separate full-volume stat based on 680M citations shows Wikipedia accounts for about 7.8% of all ChatGPT citations. Metric is top-ten share distribution, not full-volume share.
Muck Rack citation-behavior analysis (May 2026): Based on over 25M citation links, ChatGPT gives a citation in about 96% of answers, averaging about 5 sources; Gemini in about 82%, averaging about 8; Claude in about 55%, the lowest of the three.
SE Ranking: AI Referral Traffic Study (2026-06-18): Covers 101,574 sites, 250 countries, tracking window January 2025 to April 2026; ChatGPT referral share 74.78% (79.74% previous year, absolute volume up about 27%), Gemini 11.56% (+231%), Perplexity 7.23%, Copilot 3.51% (+31%), Claude 2.62% (+320%).
BrightEdge research on AI Overviews sources: About 29.5% of Google AI Overviews cite YouTube content; most cited sources land in the top ten of organic results.

Get Started

How Enterprises Build Entity Authority in the AI Era (E-E-A-T)

tengence — Mon, 12 Oct 2026 05:38:32 +0000

Summary: To be cited by AI search, the core is not stuffing keywords but getting AI to recognize you as a "verifiable entity" — that is exactly what E-E-A-T means in GEO.

AI search is changing how enterprises acquire traffic. When users ask ChatGPT, Perplexity, Claude, Wenxin Yiyan (ERNIE Bot), or Doubao questions, the AI engine must first decide "which sources are worth citing." A 2025 Rankeo analysis shows that about 73% of AI citations come from domains with clear E-E-A-T signals — without those signals, even longer content gets replaced by competitors. This article walks through the entity, author, content, and third-party layers and gives enterprises an actionable E-E-A-T roadmap plus a self-check list.

Key Takeaways

E-E-A-T stands for Experience, Expertise, Authoritativeness, and Trustworthiness, and originates from Google's Search Quality Rater Guidelines; in the AI search era it has shifted from a "nice-to-have" to a "must-have."
Before citing, AI engines first perform entity resolution: brands without a Knowledge Graph / Wikidata entity are treated as "unverified claims" and cited far less often than those with one.
Only when the entity layer (Knowledge Graph / Wikidata / sameAs), author layer (byline + Person schema), content layer (firsthand experience + original data), and third-party layer (digital PR + industry directories) stack up do they form a complete signal.
Technically, the highest-ROI step is deploying Organization schema with a sameAs chain to your official site, LinkedIn, Crunchbase, and Wikidata.
Top-ranking articles generally lack a perspective that places Chinese AI engines (Wenxin Yiyan, Doubao) alongside international engines — that gap is the differentiation space for Chinese-language GEO content.
E-E-A-T is not a one-time project; it requires quarterly-level content updates and the accumulation of third-party endorsements.

1. What Is E-E-A-T and Why It Matters More in the AI Era

The four letters of E-E-A-T stand for Experience (firsthand experience), Expertise (professional capability), Authoritativeness (authority status), and Trustworthiness (reliability). This framework first appeared in Google's Search Quality Rater Guidelines, where Google mentions E-E-A-T more than 120 times. In the traditional SEO era, E-E-A-T mainly affected the ranking of YMYL (Your Money Your Life) content such as healthcare, finance, and legal; entering the AI search era, its weight has risen significantly — because when an AI engine generates an answer it must first decide whether a source is worth citing.

Google's Knowledge Graph currently maintains about 5B entities. When an AI system is asked "how good is a certain brand," it first consults these entity databases rather than reading web pages directly. Brands with an entity ID are treated as "hard facts," while those without are treated as "unverified claims" — this is the duality of entity trust.

2. What the Four E-E-A-T Elements Mean in AI Search

Element	Traditional SEO meaning	New requirement in AI search	Actionable step
Experience	Author has relevant work experience	Content must include firsthand details (screenshots, processes, data)	Add case data, operation screenshots, real metrics
Expertise	Author has degrees/certifications	AI verifies the author's byline consistency across other sites	Author page + Person schema + cross-site sameAs
Authoritativeness	Industry recognition, backlinks	Frequency and accuracy of third-party mentions of you	Industry media mentions, G2/Clutch reviews
Trustworthiness	Privacy policy, contact info, citations	Facts are traceable: every specific claim has a source	Data-sources block + internal links to original research

Among the four, Trustworthiness is the foundation. Google explicitly states in its rater guidelines that the other three dimensions are all built upon "trust." For an AI engine, "trust" means: your facts (founding date, headquarters, product category, founders) are completely consistent across all public channels.

3. Entity Layer: Help AI Recognize Who You Are

3.1 Unified Entity Definition

The first step is to describe yourself identically across all public channels: full brand name, founding date, headquarters location, founders, core products, and official website URL. Inconsistent entities create ambiguity for the resolution system — AI doesn't know which fact to attach to which entity ID.

3.2 Schema.org Structured Data

Deploy Organization schema on your homepage and About page. Key properties include @id (pointing to the Wikidata entity ID), sameAs (links to LinkedIn, Crunchbase, G2, Wikipedia, Wikidata), foundingDate, founder, and address. The value of this step: when an AI crawler reads the page it gets a machine-readable entity definition directly, instead of guessing.

3.3 Knowledge Graph Entity

Google's Knowledge Graph is one of the primary fact sources for AI engines. You don't apply to enter the Knowledge Graph directly; Google builds it automatically from your page's structured data plus third-party mentions. Once the sameAs signals, external citations, and search volume cross a threshold, Google automatically generates a knowledge panel.

4. Author Layer: Let AI Know Who Is Speaking

When citing content, AI engines also evaluate "who wrote it." Every bylined article should link to a standalone author page containing the real name, photo, years of experience, area of expertise, relevant certificates and publications, and cross-platform sameAs. Deploy Person schema on the author page so AI can identify the author's identity in a machine-readable way.

Google's 2023 official guidelines clearly state that crediting AI as an author "may not be best practice." The correct approach is to credit a real human author/editor and describe the editing process — which actually strengthens Trustworthiness. For more on this, see How to Write AI-Generated Content Disclosure and Compliance.

5. Content Layer: Differentiate with Firsthand Experience and Original Data

The content AI models generalize most easily is the generic advice every blog repeats. What genuinely gets cited by AI as an authoritative source is what only you have: your own industry surveys, user research, and performance tests; real metrics for your own product, customer case studies (anonymized), and lessons-learned logs; giving your methodology a memorable name. "Up 37%" is more likely to be cited by AI than "significantly improved" — because when AI cites a statement with a number, it carries the brand along with it.

Producing at least one original research or data report each quarter is the most sustainable way to build Authoritativeness.

6. Third-Party Layer: Let Others Vouch for You

A large part of E-E-A-T's Trust comes from independent third-party sources. Actionable channels: pitch 5 articles per month to tier-1 industry media; complete your G2, Clutch, and Capterra profiles and encourage real customers to leave scenario-specific reviews; do industry podcast interviews and speak at industry conferences; register in official directories such as Google Partner and AWS Network Partner; and participate naturally in Reddit and industry forums.

Key principle: third-party mentions must be accurate — AI engines cross-verify them. If a third party says you make product A while your site says product B, your trust score actually drops.

7. The 12-Point E-E-A-T Self-Check List

Before publishing any GEO-targeted content, check each item:

Does the article have a real human author byline linked to an author page?
Does the author page include years of experience, area of expertise, and certificates/publications?
Is Organization schema + sameAs deployed on the site?
Is the brand name completely consistent across all public channels?
Are core facts such as founding date, headquarters, and founders consistent across sites?
Does the article contain at least one firsthand data point?
Does every specific claim have a traceable source?
Are there at least 3 independent third parties mentioning your brand?
Are G2/Clutch/industry-directory profiles complete and information-consistent?
Does the page have a privacy policy, contact info, and an About Us page?
Does the article contain at least one comparison table or structured checklist?
Is the author's Person schema deployed and linked to sameAs?

FAQ

Q: Is E-E-A-T a direct ranking factor?

A: Not a direct ranking factor. Google repeatedly emphasizes that E-E-A-T is a framework in the rater guidelines used to calibrate the ranking system, not a box you can "check." But in AI search, it essentially determines whether the AI engine selects you as a citation source — in effect it functions like a ranking factor.

Q: Can a small business with no industry-media endorsement build E-E-A-T?

A: Yes. The starting point is the content layer — use original data and firsthand experience to build differentiation. Third-party endorsement is a long-term effort, not a prerequisite. Many top content sources in niche areas started as small teams.

Q: Are Wikidata and Wikipedia mandatory?

A: A Wikidata item is worth creating for any company (low barrier, structured); a Wikipedia entry has a notability threshold, so don't force it if you don't qualify — use Crunchbase, G2, or industry directories instead. For related methods, see How to Acquire AI Search Traffic.

Q: Does AI-generated content hurt E-E-A-T?

A: Google's 2023 official stance is "reward high-quality content regardless of how it was produced." The key is having real human editors in control, injecting firsthand experience, and not mass-producing valueless content.

Q: How long does it take for E-E-A-T to show results?

A: The entity layer (schema, sameAs) can be recognized in 2–4 weeks; the content layer begins to show changes in AI citation rate after 3–6 months; third-party endorsement accumulates over 6–12 months. This is a quarter-level effort, not a one-time project.

About Tengence

Tengence GEO helps enterprises improve AI citation credibility through entity signals and author authority, covering Organization schema, Wikidata item creation, author-page building, and third-party mention monitoring. Tengence (TENGENCE) is the enterprise growth engine of the AI era, focused on public-traffic acquisition and private-domain conversion; its core products are Tengence GEO (dual-engine GEO + SEO) and Tengence Search.

Data Sources

Google Search Quality Rater Guidelines: the official definition and use cases of the E-E-A-T framework.
Google Official Blog: Google Search's Guidance on AI-Generated Content: Google does not penalize based on how content is produced and rewards high-quality E-E-A-T content.
Rankeo: E-E-A-T for AI Search (2026): about 73% of AI citations come from domains with clear E-E-A-T signals.
Fuel Online: E-E-A-T for AI Search: the duality of entity trust — Knowledge Graph entities are treated as hard facts.
schema.org Organization: the official documentation for the Organization schema and the sameAs field.

Get Started

AI-Generated Content Disclosure and Compliance: How to Write It

tengence — Mon, 12 Oct 2026 05:38:31 +0000

Summary: Google does not penalize content simply for "using AI," but scaling AI-generated content with no added value triggers its spam policy; compliant disclosure is a proactive move to reduce trust risk.

More and more companies use AI to assist with writing blog posts, product descriptions, and marketing images. This raises the question: will Google downrank for it? Should you disclose proactively? What do Chinese regulations require? What label goes on AI-generated images? Based on Google's official 2023 guidance and its December 2025 update, plus China's Interim Measures for the Management of Generative AI Services, this article provides practical templates for text and image disclosure and a three-red-line self-check checklist.

Key Takeaways

Google's 2023 official stance: reward high-quality content regardless of production method; focus on E-E-A-T, not whether AI wrote it.
Google's 2025 update: AI/automation disclosure is "helpful but not required"; disclose when readers would reasonably ask "how was this made."
China's Interim Measures for the Management of Generative AI Services (effective 2023/08/15) requires labeling of AI-generated images and videos.
Do not give AI an author byline—credit a real human author/editor and explain the editing process.
AI-generated images must write IPTC DigitalSourceType = TrainedAlgorithmicMedia in metadata.
Three red lines: scaled content abuse (mass-producing worthless pages), empty content (hollow AI content), and not crediting a human author.

1. How Google views AI-generated content

In its February 2023 official blog post "Google Search's guidance about AI-generated content," Google stated clearly: its ranking systems are designed to reward high-quality, original content, focusing on content quality rather than production method. This means writing articles with AI is not itself penalized.

What Google actually fights are two types: scaled content abuse (using AI to mass-produce many pages with no real value) and empty content (looks complete but has no first-hand information, only restating existing information in different words). The test is simple: if you removed the "AI-generated" label, would this article still have independent value to the user?

In the December 2025 update to its "Creating helpful, reliable, people-first content" document, Google says disclosure is "helpful" rather than "mandatory." But when readers would reasonably ask "how was this written," disclosure strengthens Trustworthiness.

2. What Chinese regulations require

China's Interim Measures for the Management of Generative AI Services (issued jointly by the CAC and six other departments, effective 2023/8/15), Article 12, requires providers to label generated content such as images and videos in accordance with the Provisions on the Administration of Deep Synthesis of Internet Information Services.

The Provisions on the Administration of Deep Synthesis of Internet Information Services require: content with generative or significantly edited traces—such as intelligent dialogue, synthetic human voices, and face generation—must carry a conspicuous label at an appropriate position in the generated content; service providers must add labels that do not affect user use; and logs must be retained for no less than 6 months.

Practical implications for enterprise content teams: when publishing AI-generated articles, images, or videos within China, add a conspicuous note such as "Part of this article was generated with AI assistance and reviewed and edited by a human." There is currently no unified format requirement for text disclosure, but conspicuous labeling of images and videos is explicitly required by regulation.

3. How to disclose text content: templates and placement

Disclosure level	When to use	Example wording
Full disclosure	Press releases, policy explainers, in-depth analysis	"This article was drafted with AI assistance and fact-checked and supplemented with first-hand cases by editor XXX."
Light disclosure	How-to guides, popular science, industry observation	"Some paragraphs of this article were written with AI assistance and human proofreading."
No disclosure	Fully original; AI used only for grammar polishing	No disclosure needed (but keep an editing-process record)

On placement: do not put it in the title (it dilutes the answer-first signal), do not put it in the first 100 words (the key AI-citation zone); place it before "About Tengence" at the end, in one or two sentences. Do not credit AI as the author, and do not add an AI label to every paragraph. Recommended template:

Editor's note: The first draft of this article was generated with AI assistance and published after the editorial team verified the facts and added cases.

4. How to label AI-generated images/videos

Google's December 2025 generative-AI content guidance explicitly requires: AI-generated images must write the IPTC DigitalSourceType field in metadata, with the value TrainedAlgorithmicMedia. This field is embedded in the image file's EXIF/XMP metadata; ordinary users cannot see it, but Google Image Search and AI crawlers read it.

Beyond metadata, Chinese regulations require a "conspicuous label." Practical approach: add a watermark or mark "AI-generated" in the bottom-right corner of the image, or state in the caption "This image was generated by AI, for illustration only." If AI generated e-commerce content such as product titles or descriptions, label it separately as "AI-generated" and do not mix it with human-written content.

5. Three-red-line self-check checklist

Before publishing any AI-assisted content, check each item:

Is there a real human author/editor byline? Not an AI name, not a vague credit.
Is there first-hand experience or original data? Did the editor add information found only here?
Is mass production avoided? Generating dozens of articles from one template at once triggers scaled content abuse.
Were facts manually verified? Numbers, dates, person names, product names—checked by a human?
Is the disclosure placement reasonable? Does it not harm the main-content reading experience?
Does the image have IPTC metadata written? Does the DigitalSourceType field exist?
Is the editing-process record retained? Who drafted, who reviewed, who published—traceable if problems arise?

Compliant disclosure is not the finish line. Real human author bylines, first-hand experience injection, original research, traceable citations, and transparent editing processes—these five points let AI-assisted content actually improve E-E-A-T. For the complete E-E-A-T framework, see AI Search Traffic Acquisition.

FAQ

Q: If I generate an article entirely with AI and make no human edits, will Google penalize it?

A: Technically, Google does not penalize for "using AI." But if the content has no first-hand value and only rearranges existing information, the helpful content system will flag it as low quality. The result is low ranking, not a manual penalty.

Q: Our company uses ChatGPT to write blogs; do we need to disclose on every post?

A: Google does not require it, but you should disclose when readers would reasonably ask "how was this made." We recommend adding one editor's note at the end of every post—settled once and for all.

Q: How do I write IPTC metadata for AI-generated images?

A: When exporting the image (Photoshop, Lightroom, or a script), write the XMP field DigitalSourceType = TrainedAlgorithmicMedia. Google Image Search reads this field.

Q: Do Chinese regulations require mandatory labeling of AI text content?

A: The current Interim Measures for the Management of Generative AI Services mainly require labeling of multimodal generated content such as images and videos; there is currently no mandatory format requirement for plain text content, but proactive disclosure is recommended to strengthen user trust.

Q: What is the problem with crediting AI as the author?

A: Google explicitly advises against this. It blurs "who is responsible for the content" and harms Trustworthiness. The correct approach is to credit a real human editor and mention AI assistance in the editor's note.

About Tengence

AI-assisted content disclosure and E-E-A-T signal management are supported by Tengence GEO, helping enterprises keep Trustworthiness while producing content at scale. Tengence (TENGENCE) is the enterprise growth engine of the AI era, focused on public-traffic acquisition and private-domain conversion; its core products are Tengence GEO (dual-engine GEO + SEO) and Tengence Search.

Data Sources

Google Official Blog: Google Search's Guidance About AI-Generated Content (2023/02): Google does not penalize based on content production method; it rewards high-quality E-E-A-T content.
Google Official Docs: Creating Helpful, Reliable, People-First Content (2025/12): AI disclosure is "helpful but not required."
Google Official Docs: Use Generative AI Content Guidelines (2025/12): AI-generated images need IPTC DigitalSourceType metadata.
CAC: Interim Measures for the Management of Generative AI Services (effective 2023/08/15): requires labeling of AI-generated images and videos.

Get Started

Do You Still Need Backlinks in the AI Era? What 75,000 Brands' Citation Data Shows

tengence — Mon, 12 Oct 2026 05:38:29 +0000

Summary: Yes, but the role shifts from main plotline to foundation. Ahrefs' analysis of 75,000 brands shows the correlation between backlink count and AI overview visibility is only about 0.218, while unlinked brand mentions reach about 0.664; backlinks remain the baseline of trust, but marginal returns drop sharply once that line is crossed.

"AI answers don't read the link graph, so are backlinks useless?" This is one of the most common questions in the 2026 budget review. The answer is not a simple "useful" or "useless" but requires separating two things: links largely retain their role in traditional ranking, their direct role in AI answers clearly weakens, yet they remain the foundation for entering the candidate pool. This article uses public data from Ahrefs, Linkody, Muck Rack, and BuzzStream to show which conclusions are evidence-backed, which are industry inertia, and how to reallocate budget.

1. The verdict first: yes, but it shifts from main plotline to foundation

Backlinks remain necessary in 2026, for three reasons.

First, they remain the baseline of trust. When AI systems filter citation sources they favor domains with basic authority, and the backlink structure is still one of the main sources of that authority. Without this baseline, even great content struggles to enter the candidates. Second, profitable sources bring multiple signals. A link from an authoritative media outlet usually brings brand mention, editorial context, and topic co-occurrence at once, and these three are exactly the signals AI systems weigh more heavily—the real gain comes from "being mentioned," and the link is just a byproduct. Third, there is still direct value on citation-heavy engines. Platforms that rely on real-time retrieval reference the source's own performance more directly, and the role of links has not disappeared in such scenes.

In other words, backlinks retreat from "ranking main plotline" to "trust foundation": they no longer decide how high you rank, but they still decide whether you qualify to be considered.

2. What the data says: mentions beat links

The most persuasive comparison comes from Ahrefs' analysis of 75,000 brands in 2025-5:

Signal	Correlation with AI overview visibility
YouTube mentions	about 0.737
Unlinked brand mentions	about 0.664
Brand anchor text	about 0.527
Domain Rating	about 0.326
Brand search volume	about 0.35
Traditional backlinks	about 0.218

The correlation between unlinked brand mentions and AI visibility is about 3x that of traditional backlinks. "Unlinked mentions" here means the brand name appears in media articles, industry rankings, or community discussions without a hyperlink.

Muck Rack's analysis of more than 25,000,000 citation links points to the same conclusion: about 84% of AI citations point to earned media (media coverage, reviews, industry rankings, and other third-party content), while paid content accounts for only about 0.3%.

A reminder: these are correlations, not causality, and different studies use inconsistent methods. Treat them as directional evidence rather than precise values.

3. Why links' role is diluted: three mechanisms

The first is ranking and citation are decoupling. Ahrefs' analysis in 2026-3 based on 863,000 keywords and 4,000,000 citations shows that only about 38% of sources cited in AI overviews also rank in the top ten organic results, while the previous version of that study was about 76% (BrightEdge's similar estimate puts this number even lower). What links do best—pushing you up the rankings—is itself becoming a weaker path into the answer.

The second is query fan-out. AI engines don't match your page to one word; they break the question into multiple sub-questions, retrieve each separately, then merge into a single answer. You won the keyword, but may have never participated in most of the sub-questions AI actually raised.

The third is language models read text, not link relationships. Models learn what a brand means from "who is mentioned in what context," not from the topology of hyperlinks. This also explains why unlinked mentions work better.

4. What irreplaceable roles links still have

Deriving "backlinks can't directly bring AI citations" into "backlinks are useless" misses the four roles they still play.

First, discovery and indexing. A new page still needs a path pointing to it to be found. Second, the basic trust threshold: an open domain with only a few low-relevance backlinks can hardly earn the boost from other signals. Third, traffic itself—a seriously underestimated point: Google still sends far more traffic to websites than conversational AI platforms, and in most scenarios it remains the main source channel. Fourth, part of the conversion path: even if the AI answer completes awareness, the landing step often happens on the page the backlink points to. Evaluating these four separately from "direct AI citation" gives backlinks a sensible place.

5. The real cost of link building in 2026

If you keep investing, you must be clear-eyed about price and quality. Linkody's statistics based on 865,000 monitored backlinks show: about 56% of backlinks remain valid, about 31% are confirmed lost or broken; about 52% disappear within a year, and only about 31% survive after five years; low-authority (DA below 10) sites have a backlink survival rate of only about 33%.

On cost, BuzzStream's analysis of large vendor data in 2026-6 shows guest posts cost about $300–460 each, and digital PR links about $1,250–1,500 each; yet in this vendor database only about 1.37% of sites meet the basic quality threshold of "real authority and traffic." That is, only a small share of paid link candidates clears the bar.

The gap between perception and reality is also clear. Editorial.link's survey of 518 practitioners in 2026-3 found 48.6% believe digital PR is most effective, and guest posts only 16%; but guest posts remain the most widely used. About 57% in the same survey expect results in 1–3 months.

6. Three common misjudgments

The first misjudgment is "backlinks are a top-three ranking factor." This claim originates from a 2016 Q&A; Google's Gary Illyes denied it directly in 2023-9: "not for a long time." It still circulates in many articles and should not be used as a budget basis.

The second misjudgment is "buy more links and you'll get into AI answers." The data doesn't support it: backlink correlation is about 0.218, while paid content accounts for about 0.3% of AI citations. Together they show this path is not only inefficient but also easily hits quality red lines.

The third misjudgment is "since AI doesn't read links, you can stop all external signal work." Stopping completely leads to gradual loss of the basic trust line and indexing ability, a classic case of reading "relative weight dropped" as "absolute value went to zero."

7. How to reallocate the budget

If the total budget is unchanged, adjust in the following order.

First, keep the baseline and cut the quantity race. Maintaining a few high-quality, topically relevant links is enough to hold the threshold; shift the quantity-chasing budget out.

Second, put the main increment into "being mentioned." Since unlinked brand mentions correlate about 0.664, and about 84% of AI citations point to third-party content, the most valuable thing is to get your name into authoritative reviews, industry rankings, and media coverage. Hyperlinks will occur naturally, but they should not be the procurement target itself.

Third, do content extractability first. Compared with any other external action, restructuring existing content—including self-contained answer passages, tabulated data, and clear attribution—pays off faster and costs less.

Fourth, build consistent tracking. Currently about 73.2% of practitioners believe backlinks affect the probability of appearing in AI search results, but only about 24% actually track it. Without a unified tracking standard, budget optimization is impossible.

About Tengence

Citation-rate monitoring and AI referral attribution are among the core capabilities of Tengence GEO, whose performance dashboard covers the cited-rate baseline and brand-term benchmarking steps mentioned in this article. Tengence (TENGENCE) is the enterprise growth engine of the AI era, focused on public-traffic acquisition and private-domain conversion; its core products are Tengence GEO (dual-engine GEO + SEO) and Tengence Search.

Related Reading

Data Sources

Ahrefs brand visibility correlation study (2025-5, 75,000 brands): YouTube mentions about 0.737, unlinked brand mentions about 0.664, brand anchor text about 0.527, brand search volume about 0.35, Domain Rating about 0.326, traditional backlinks about 0.218.
Ahrefs AI overview citation source analysis (2026-3, 863,000 keywords / 4,000,000 citations): only about 38% of AI overview citation sources also rank in the top ten organic results, the previous version was about 76%.
Muck Rack citation link analysis (over 25,000,000 links): about 84% of AI citations point to earned media, paid content accounts for about 0.3%.
Linkody backlink statistics 2026 (based on 865,000 monitored backlinks): about 56% still valid, about 31% confirmed lost or broken, about 52% disappear within a year, only about 31% survive after five years, low-authority sites survival rate about 33%.
BuzzStream backlink vendor quality analysis (2026-6): guest posts about $300–460 each, digital PR links about $1,250–1,500 each; only about 1.37% of vendor sites meet the basic quality threshold.
Editorial.link practitioner survey (2026-3, 518 respondents): 48.6% believe digital PR is most effective, guest posts 16%; about 57% expect results in 1–3 months.

Get Started
Free trial of Tengence
Book an expert consultation
Learn about Tengence GEO

Why Do Your A/B Tests Keep Failing to Reach Significance? Three Datasets from 2026

tengence — Mon, 12 Oct 2026 05:38:28 +0000

Summary: Most "not significant" results are a test-design problem, not a creative one. 2026 industry analysis shows that among tests reaching significance with insufficient statistical power, only about 28.4% replicate after being rolled out to full traffic; while in properly run tests, about 74.2% concluded "no significant difference" in the first place—having no effect is the norm.

"Changed it for ages, still not significant", "tested for three months, not a single lift seen"—the reason behind these two complaints is usually not that the creative is weak, but that sample size, peeking habits, and effect expectations went wrong in three ways. Using three sets of public data and a sample-size formula you can verify yourself, this article explains how to interpret "not significant," why peeking manufactures fake winners, and under what conditions you simply should not run quantitative tests.

1. The takeaway first: treat "no effect" as the default result

Before running A/B tests, the thing most in need of adjustment is expectations. Three datasets show why.

First set: most properly run tests simply have no result. 2026 industry analysis shows that about 74.2% of A/B tests run to spec ultimately reach a "no significant difference" or inconclusive conclusion. In other words, "no difference detected" is the norm; teams that produce a winner every week are usually running under-powered tests.

Second set: "victories" under insufficient power are mostly noise. The same analysis shows that among tests reaching significance with insufficient statistical power (below 60%), only about 28.4% replicate after full-traffic rollout, the remaining about 71.6% being random fluctuation. Tests with genuine adequate power have a replication rate in the 70%–85% range.

Third set: real effects are often smaller than you think. Most copy, layout, and button tweaks fall in a relative-lift range of 1%–5%, and detecting that magnitude requires far more sample than most teams intuit.

Combined, these three lead to the conclusion: the value of a testing program should be measured by the final actual conversion rate, not by the reported "number of winning variants."

2. How much sample size do you actually need: a set of comparable numbers

Sample size is jointly determined by four variables: baseline conversion rate, minimum detectable effect (MDE, the smallest relative lift worth detecting), significance level (usually 95%), and statistical power (usually 80%). Fix three, and the fourth is determined.

At the baseline of 95% confidence and 80% power, public sample-size estimates are:

Scenario	Baseline conversion	Relative lift to detect	Sample per variant
Common marketing test (median)	3%	5%	~14,800 sessions
E-commerce landing page	2%	10%	~78,000 visits
High-sensitivity scenario	2%	2.91%	~470,000 visits
Small-site reachable range	5%	20%	~9,000 visits
Aggressive redesign	5%	50%	~1,700 visits

Two rules worth remembering. First, MDE and sample size have a quadratic, not linear, relationship: halving the effect you want to detect makes the required sample about fourfold, not double. This is exactly why jumping from "detecting a 10% lift" to "detecting a 5% lift" suddenly gets much harder. Second, the higher the baseline, the easier to test: a page at an 8% conversion rate needs about half the sample to detect the same relative lift as a page at 2%.

Per one testing vendor's statistics based on its own experiment library, the median test duration needed to reach adequate power is about 42 days. Most teams' problem: the tool lit a green significance flag in week three, and the test stopped there.

3. Why mid-test peeking ruins the test

Repeatedly checking mid-way and stopping as soon as significance appears is habitually called peeking. Its harm comes from a simple mechanism: each check is a chance for random noise to cross the threshold. A fixed-horizon frequentist test checked daily has a true false-positive rate far above the 5% you set.

What results is "winner inflation." Tests that happen to be significant under low power, when rolled out, show an average true effect of only 25%–35% of the original estimate; teams report cumulative lifts of 100%–200%, while the actually replicated lift often lands in the 15%–30% range. This also explains a common awkward situation: a testing program running for over a year reports impressive cumulative gains, yet the production conversion rate hasn't budged.

There are three fixes, ranked by reliability: first, compute sample size and duration before launch, and only read results when the time is up; second, run at least one to two full business cycles (usually 1–2 weeks) so the day-of-week effect averages out; third, if continuous monitoring is truly needed, choose a tool built for sequential testing or Bayesian methods—don't keep re-checking a fixed-horizon frequentist test.

4. Three rules for separating signal from noise

Rule 1: Compute first, then run. Bring the four numbers—baseline, MDE, confidence level, power—into a sample-size calculator to get the sample per variant; if current traffic can't reach that number in 4–6 weeks, don't launch—either enlarge the change to raise MDE, move to a higher-traffic step, switch to a higher-frequency proxy metric (e.g., add-to-cart instead of checkout), or switch to qualitative methods directly.

Rule 2: Look at effect size, not p-value. The p-value only says the difference is unlikely from chance, not that this difference is worth shipping. Conversely, a non-significant result from a low-power test does not prove the two variants are equally good—its real meaning is "this experiment had no power to detect this difference."

Rule 3: Reconcile with cumulative gains. Do a review each quarter: total up the claimed gains of all past "winning variants," then compare against the actual production conversion-rate change. A long-term divergence means the testing system is manufacturing noise, not discovering gains.

5. When you should NOT run A/B tests

Quantitative testing has clear boundaries; the following four scenarios usually cost more than they pay.

Insufficient traffic. A site with a few thousand visits per month wanting to detect a 5% lift often must run over a year; user interviews and session replays are then more valuable than a chronically under-powered test.

Too-small changes. Changes like button color or spacing tweaks have tiny real effects and require huge samples; the same traffic spent on bigger changes is more likely to yield actionable conclusions.

Non-representative sample. Early users of a new site or innovative product differ greatly from the mainstream; conclusions measured then are hard to generalize.

Irreversible or very high-cost decisions. Adjustments to pricing systems or brand positioning cost far more than they return from testing, and are better judged via phased rollout and user feedback.

6. Three common misjudgments

The first misjudgment is "not significant equals ineffective." This does not hold: not significant could mean the effect is truly zero, or that the sample is insufficient. The two are handled completely differently; conflating them makes teams systematically abandon effective ideas. One practitioner's statistic is intuitive: 10 tests run at 60% power miss about 4 real winners, while the same 10 at 90% power miss only 1.

The second misjudgment is "running more small tests makes up for insufficient sample." Mathematically it does not hold: low power does not cancel out by adding quantity; it only systematically misses real effects.

The third misjudgment is "the tool says significant so it's real." The tool only flags at a set threshold; it doesn't know whether your sample is adequate, whether you peeked repeatedly, or whether you ran full business cycles. The responsibility to judge always rests with the human.

7. Minimum viable process

No complex experiment platform needed; these five steps significantly raise conclusion quality.

Step 1, compute. Before launch use a calculator to determine the sample per variant and the expected duration, and write it into the experiment sheet.

Step 2, hold. Read results only when time is up, while ensuring coverage of at least two full business cycles.

Step 3, record. Record baseline, MDE, actual sample, and final effect range—not just "won or lost."

Step 4, verify. For important conclusions, run a holdout-group comparison after full-traffic launch to verify replication.

Step 5, review. Each quarter reconcile claimed gains against actual conversion rate, and evaluate the testing program itself by the final actual conversion rate, not the number of winning variants.

About Tengence

Experiment design and effect measurement belong to the growth-technology domain that Tengence focuses on. Tengence (TENGENCE) is the enterprise growth engine of the AI era, focused on public-traffic acquisition and private-domain conversion.

Data Sources

2026 A/B Test Replication-Rate Industry Analysis (Prooflytics compilation): tests reaching significance under low power (below 60%) replicate only about 28.4% after full-traffic rollout; adequately powered tests replicate about 70%–85%; about 74.2% of properly run tests reach a "no significant difference" conclusion; a 3% baseline detecting a 5% relative lift needs about 14,800 sessions per variant.
DRIP Experiment-Library Sample-Size Estimates: a 2% baseline detecting a 10% relative lift needs about 78,000 visitors/variant; a 2% baseline detecting a 2.91% relative lift needs about 470,000 visitors/variant; median test duration to reach power about 42 days.
Stackmatix Sample-Size Guide: a 5% baseline detecting a 50% relative lift needs about 1,700 visitors/variant, detecting 20% needs about 9,000, detecting 10% needs about 36,000.
Statistical Power and Missed-Detection Analysis: 10 tests at 60% power miss about 4 real winners, the same 10 at 90% power miss only about 1.

Get Started

Book a free expert consultation and diagnosis

The Complete Guide to A/B Testing: Sample Size, Duration, and Significance

tengence — Mon, 12 Oct 2026 05:38:26 +0000

Summary: The credibility of an A/B test rests on 3 things: collecting enough sample, running a full business cycle, and judging significance against a stopping rule fixed in advance. This guide connects sample size calculation, duration setting, significance testing, multiple-comparison correction and null-result handling into one executable loop, and adds workable alternatives for low-traffic scenarios.

Most teams do not fail at A/B testing because their ideas are bad. They fail because the conclusion is not trustworthy: calling a result before the sample is complete, stopping early when day-one data looks good, or tracking 5 metrics and reporting only the one that improved. All of these push the false positive rate from a nominal 5% up to 20% or higher. This article follows the order "sample size, duration, significance, multiple testing, wrap-up" and gives the concrete calculation and judgement rule for each step, so experimental results can genuinely be used for decisions.

Key Takeaways

Sample size is decided jointly by baseline conversion rate, minimum detectable effect (MDE) and statistical power; if any of the three is missing, it cannot be calculated.
Every time the MDE is halved, the required sample roughly quadruples; low-baseline pages need a bigger change, not a longer wait.
Test duration must satisfy both the sample size requirement and a full business cycle; the shortest practical run is usually no less than 14 days.
Checking results repeatedly and stopping early (peeking) pushes the false positive rate from 5% to over 20%; the stopping rule must be fixed in advance.
When you observe several metrics or several variants at once, apply multiple-comparison correction, otherwise "significant" is simply a probabilistic certainty.
Only about 1 in 7 experiments produces a clear winner; null and negative results are the norm and should be recorded and attributed just the same.
Low-traffic pages should not force a classic A/B test; switch to sequential testing, Bayesian methods or qualitative research.

1. Sample Size Is Set by Baseline Conversion, MDE and Power Together

The essence of sample size is this: given a baseline conversion rate, how much sample do you need to reliably tell a real lift apart from random noise. For the most common comparison, a conversion-rate test (two-proportion test), the sample per group can be estimated with n ≈ 2 × (Z_{1-α/2} + Z_{1-β})² × p × (1-p) / δ², where p is the baseline conversion rate, δ is the absolute lift (the MDE expressed in percentage points), α is the significance level, and 1-β is statistical power. Using the industry-standard 95% confidence level and 80% power, with a baseline conversion rate of 5% and a desired absolute lift of 1 percentage point, each group needs about 7,415 visitors. This parameter set makes one key fact clear: sample size is not a round number pulled out of thin air, it is the result of solving for 4 variables at once.

Setting the MDE is the step most likely to go wrong. The smaller the MDE, the weaker the signal you need to distinguish, so the larger the sample required, and the relationship is quadratic: halve the MDE and the sample roughly quadruples. On a page with a 2.5% baseline, detecting a 10% relative lift (that is, 0.25 percentage points) needs about 15,700 visitors per group; raise the expectation to a 20% relative lift and the sample drops to roughly a quarter. Pages with a low baseline and low traffic therefore should not obsess over detecting tiny changes. Design the experiment around a difference large enough that limited traffic can still produce an answer.

2. Turn Sample Size Into an Executable Target With a Benchmark Table

Once you have the sample size, the next step is converting it into a target the team can understand. A benchmark table laid out by "baseline conversion rate × relative lift" is the most practical tool: put 3 baseline tiers, 2.5%, 5% and 10%, on the horizontal axis, 2 relative-lift tiers, 10% and 20%, on the vertical axis, and fill each cell with the required sample per group plus the corresponding total sample (number of groups × sample per group). With this table in hand, a product manager can judge at requirement time whether a change is detectable at current traffic, instead of discovering insufficient power only after the experiment has finished.

One point deserves emphasis: an underpowered experiment is more dangerous than no experiment at all. It ends with a "no significant difference" conclusion, the team concludes the change does not work and drops it, when in reality the sample was simply too small. Before launch, make "how many days to reach 80% power at current traffic" a hard gate for experiment admission. If the time required exceeds the acceptable window (usually more than 4 weeks), redesign the experiment: raise the MDE, cut the number of variants, or move the test to a higher-traffic page.

3. Duration Must Satisfy Sample Size and a Full Business Cycle

Duration is not a simple conversion of sample size; it also has to cover the full rhythm of the business. Purchase behaviour varies clearly across the week: the traffic mix, decision time and average order value on weekdays can all differ from weekends, so running only 5 working days biases the result toward weekday-type users. The general rule is to run at least 2 complete business cycles (that is, 14 days), and not longer than 8 weeks: beyond 8 weeks an experiment faces shifting traffic composition, seasonality and interference from external events, and historical data becomes less representative of new data.

2 floors are also worth memorising: at least 1,000 sessions per group and at least 30 conversions per group. The first keeps the traffic sample from being too thin; the second gives the conversion-rate estimate itself a basic level of stability. If the experiment has not reached its sample size after 14 days, keep running until it does rather than stopping by the calendar; conversely, if the sample is reached within 3 days, you still have to run at least one complete business cycle. Duration and sample size are joined by "and", not "or".

4. Significance Judgement Means Refusing to Peek and Fixing the Stopping Rule

Statistical significance is widely misused, and the classic error is peeking: checking results repeatedly while the test runs and stopping the moment the p-value drops below 0.05 to declare victory. This turns a single test into many tests, and the false positive rate accumulates with them: a nominal 5% significance level can climb past 20% under repeated inspection; checking once a day with a readiness to stop at any time can push it beyond 30%; in the extreme case of checking after every visitor, the false positive rate can even reach the neighbourhood of 26%.

The correct approach is to fix the stopping rule in advance: before the experiment starts, write down "test once when N samples have accumulated", and in between look only at health-check metrics (whether traffic splitting is balanced, whether there are technical errors, whether extreme outliers appear), never at the significance of the primary metric. If stopping mid-flight is genuinely necessary, use only statistical methods designed for sequential settings, such as the sequential probability ratio test or Bayesian stopping rules, rather than repeatedly executing a fixed-sample test. In addition, the p-value only answers whether the difference could have arisen by chance; it does not answer how large the difference is, so confidence intervals and the relative lift must always be reported alongside it.

5. Multiple Metrics and Multiple Variants Need Multiple-Comparison Handling

An experiment that observes only one metric is rare inside an enterprise. The common practice is to watch click-through rate, add-to-cart rate, conversion rate and average order value at the same time, or to launch 3 or 4 variants simultaneously. The problem is that every extra test is another chance to hit significance at random. When you observe 10 mutually independent metrics, even if the change is completely ineffective, you should expect about 0.5 of them to reach 5% significance; if "any metric significant" counts as success, the overall false positive rate sits far above 5%.

There are 3 layers of response. The first is to separate primary from secondary: designate a single primary metric in advance (guardrail metrics are only checked for "must not degrade"), and count the experiment as successful only when the primary metric passes. The second layer is statistical correction: the simplest is the Bonferroni correction, dividing α by the number of tests (with 5 tests, α becomes 0.01); you can also control the false discovery rate (FDR) to trade a little rigour for sensitivity in exploratory work. The third layer is to treat multi-variant cases separately: the more variants, the worse the multiple-comparison problem, and the less traffic each group receives, so the usual recommendation is no more than 4 variants per experiment, including the control.

6. Null and Negative Results Are Conclusions Too

Long-running industry statistics show that only about 1 in 7 experiments produces a clear winner; the rest are mostly null or negative results. That is not a failure of the method, it is the true distribution of product optimisation: changes that genuinely and significantly shift user behaviour are simply rare. Teams should therefore not treat "produced a significant lift" as the only definition of a successful experiment. Ruling out a wrong direction at acceptable cost counts as valid output too.

The right way to handle a null result is attribution, not amnesia: was the MDE set too small, leaving the test underpowered? Was the change so minor that users never noticed it? Or did the change indeed touch an irrelevant part of the journey? Recording the cause of every null result markedly improves the design quality of later experiments. Negative results deserve even more attention: they prove definitively that a direction does not work, and often carry more information than a positive result. When an experiment ends, regardless of the conclusion, archive the design, the sample-size rationale, the full metrics (including confidence intervals) and the conclusion, and build a searchable experiment library.

FAQ

Q: How much sample does an A/B test actually need?

There is no fixed number; it depends on 4 parameters: baseline conversion rate, MDE, significance level and statistical power. At 95% confidence and 80% power, a 5% baseline with a 1 percentage point absolute lift needs about 7,415 visitors per group; drop the baseline to 2.5% and detect a 10% relative lift and you need about 15,700 per group.

Q: How should the MDE be set?

The MDE should equal the smallest lift that makes the change worth shipping, usually derived backwards from business value and implementation cost, and generally set at a 10%–20% relative lift. Note that halving the MDE roughly quadruples the sample, so pages with limited traffic should set a larger MDE.

Q: How many days does an experiment have to run?

At least 2 complete business cycles (usually 14 days) to cover within-week variation, and no more than 8 weeks. At the same time you must satisfy the floors of at least 1,000 sessions and 30 conversions per group. Days and sample size are joined by "and": only when both are met can you stop.

Q: Why can't I look at results mid-flight and stop when it turns significant?

Checking repeatedly and stopping whenever you like turns one test into many, and the false positive rate accumulates from a nominal 5% to over 20%, or beyond 30% if you check daily. Fix the inspection point in advance, and in between look only at technical health metrics such as traffic-split balance.

Q: Is watching several metrics at once a problem?

Yes. The more metrics, the more chances to hit significance at random. Designate a single primary metric in advance and treat the others as guardrails checked only for degradation; if judgement across multiple metrics is truly needed, use the Bonferroni correction (with 5 tests, α becomes 0.01) or control the false discovery rate.

Q: What should I do when the experiment returns "no significant difference"?

First judge whether power was insufficient: work backwards from actual traffic to the smallest MDE you could detect and compare it with the expected lift. If it is larger, the conclusion is "could not detect", not "no effect". If power was sufficient, record the cause (change too light, direction irrelevant, and so on) and archive it to avoid repeating the same attempt.

Q: How do you run experiments on a page with very little traffic?

3 routes. First, switch to sequential testing or Bayesian methods, which allow early stopping once enough evidence has accumulated. Second, give up on tiny changes and design versions different enough to match the traffic you have. Third, replace quantitative testing with qualitative methods such as user interviews and usability testing: find the direction first, then validate it quantitatively.

About Tengence

Scientific experiment design and effect measurement are the precondition for building growth decisions on evidence rather than intuition. Tengence focuses on growth technology for public-traffic acquisition and private-domain conversion, and the experiment design and effect measurement covered in this article are among the areas it follows continuously.

Data Sources

PulseCRO: How to Calculate the Right Sample Size for Your A/B Test: gives the two-proportion sample size formula and the meaning of each parameter, and lists the estimate of about 7,415 visitors per group at a 5% baseline, an MDE of 1 percentage point, and 95% significance / 80% power, recommending an MDE of 10%–20% relative lift.
Metricuno: Sample Size in A/B Testing: provides a sample-size benchmark table at 95% confidence / 80% power, notes that halving the MDE roughly quadruples the sample, and gives the example of about 15,700 per group for detecting a 10% relative lift on a 2.5% baseline.
Grow Conversions: How Long to Run an A/B Test: explains that peeking pushes the false positive rate from 5% above 20%, and gives the window guidance of a minimum of 2 complete business cycles (14 days) and a maximum of 8 weeks, plus the floors of 1,000 sessions and 30 conversions per group.
Clarigital: A/B Testing Methodology: gives the finding that multiple interim checks (10 of them) drive the false positive rate above 30%, and the practice of tightening α to 0.01 in the Bonferroni correction.
Systeme.io: A/B Testing: gives the long-run statistic that about 1/7 of experiments produce a clear winner, and the estimate that peeking after every visitor raises the false positive rate to about 26%.

Get Started

Book a free expert consultation and diagnostic
Start a free Tengence Cloud trial
Learn about Tengence GEO

Tengence Search User Guide: Add Search to a Business System in Four Steps

tengence — Fri, 18 Sep 2026 08:30:30 +0000

Tengence Search User Guide: Build Search into Your App in 4 Steps

This guide walks you through adding search capabilities to your business system with Tengence Search: create an app, configure an index, build the index, and integrate via API or UI component.

What Is Tengence Search?

Tengence Search is a fully managed search product from Tengence that lets enterprises add high-quality search to their applications in minutes—no search infrastructure to deploy or maintain.

The four-step flow:

Step	Description
Create an app	Pick your industry and plan to provision search resources
Configure the index	Map dataset fields to index fields and define the search schema
Build the index	The system builds the search index automatically based on your config
Integrate	Embed search via REST API or a ready-made UI component

Getting Started

Log in to the Tengence Console and click Tengence Search in the left sidebar.

Figure 1: Tengence Search in the left navigation
You land on the overview page. Click New App to start creating your first search application.

Figure 2: Tengence Search overview page

Creating an App

Step 1: Choose Industry and Plan

Select your business industry. The system applies industry-specific tuning automatically. If your industry isn't listed, choose General.
Choose a plan tier. Each tier comes with different resource quotas and features—pick one that matches your data volume.
Click Create to finish.

Figure 3: Selecting industry and plan

Step 2: Open the App Detail Page

After creation, click Back to App List.

Figure 4: Returning to the app list
Your new app appears at the top of the list (newest first). New apps get an auto-generated name.

Figure 5: App list page
Click the app name to open its detail page.

Step 3: Understand the "Basic Info" Panel

The app detail page has three main sections: Basic Info, Index Config, and Preview. The Basic Info panel gives you:

#	Feature	Description
1	Rename	Click the edit icon next to the app name
2	Actions	Upgrade, renew, or delete the app
3	Service status	Current plan tier and service state
4	Runtime status	Shows whether the index is ready; you must complete index config before use
5	Quota usage	Documents indexed / plan limit; upgrade to raise the cap
6	Preview	Opens a side panel to see the search UI in action
7	API access	Shows API endpoint, auth key, and request examples
8	UI component	Shows the embedded search widget integration guide
9	Debug console	Test queries and verify your config is working

Figure 6: The Basic Info panel

⚠️ Important: The API Key authenticates all calls to your app. Keep it secure and never expose it client-side.

Index Configuration

An index is the foundation of search. You map dataset fields to index fields, and the system builds a searchable index from that mapping.

Step 1: Enter Index Config

From the app detail page, click Index Config.

No dataset yet? Click Create New Dataset to start fresh.
Already have one? Click Link Existing Dataset and pick from your list.

Figure 7: Index config entry point

Step 2: Pick Your Dataset

Select the dataset that will serve as the data source for this index.

Figure 8: Choosing a dataset

Step 3: Configure Index Fields

Tengence Search provides index templates for common use cases. Start from a template and adjust as needed:

Semantic labels: Tag each field with a semantic label so the system understands your business data better.
Index fields: Templates pre-populate field names. Just map them to your dataset fields. Every index must have exactly one ID field (unique document identifier) and at least one field with tokenized field type for search to work.
Dataset field: Pick which dataset column maps to each index field.
Field type: Choose how each field is processed into the index (tokenized, exact, numeric, etc.).
Field weight: Set a weight for tokenized fields—search results are boosted by weight at query time.
Add custom fields: Click Add Index Field to extend the template with business-specific fields.
Click Save to persist your config.

Figure 9: Index field configuration

💡 Tip: Hover over the "?" icon next to any column header to see a detailed explanation of that setting.

Step 4: Start the Index Build

After saving, a dialog asks whether to start building now. Click Start to kick off the build, or Not now to keep editing.

Figure 10: Starting the index build

Step 5: Track Build Progress

Back on the app detail page, the Runtime status card shows build progress and lets you stop the build if needed.
The Document count shows how many records have been indexed so far.
Click refresh to see the number update in real time.

Figure 11: Monitoring index build progress

Integrating Search

Once the index is built, you can connect search to your product. Tengence Search offers multiple integration paths.

Preview

Click Preview to see how the search UI component looks before you ship it.

Figure 12: Search preview

API Access

Click API Access to see the full integration guide: endpoint URL, request parameters, and response examples.

Figure 13: API integration guide

UI Component

Click UI Component to get the embed code for a ready-made search box. Drop it into your page and you're done.

Figure 14: UI component integration

Debug Console

Click Debug Console to run test queries directly in the console and verify your index is returning the right results.

Figure 15: Debug console

Glossary

Term	Definition
App	A search service unit in Tengence Search. Each app has its own index config, API key, and quota.
ID field	The unique identifier field in an index. Exactly one per index.
Index	The searchable data structure built from your dataset field mappings.
Semantic label	A tag describing a field's business meaning, helping the system understand and rank better.
Tokenized	A field type that splits text into tokens so users can match by keywords.
Field weight	A boost value for tokenized fields; higher weight ranks higher in results.
Dataset	A data storage unit in Tengence Data Center. Supports push API, MySQL sync, and behavioral data ingestion.