When a public page changes according to request headers, test which HTML your cache reuses before treating a search mismatch as a content problem. Compare the origin response with repeated edge requests, then reverse the order in which the variants arrive. An order-dependent result is a useful clue that the cache is mixing responses.
The Vary header describes request fields that influenced a response. It is not an instruction to create a special page for a crawler, and adding it without checking the delivery configuration is not a complete fix. The important question is whether each request receives the intended public content through the whole delivery path.
The method below uses a fictional documentation site. It is a diagnostic design, not a report of a test performed on a real service.
Define the content difference before testing the header
Choose one URL with a concrete discrepancy. For example, a product's desktop help page contains a compatibility warning while the mobile response does not. Write down the exact sentence, link, or directive that differs. “The pages look different” is too broad to diagnose.
Separate layout differences from missing information. Responsive styling can rearrange the same HTML without creating a separate server response. Dynamic serving can return different HTML at the same URL. Google's mobile-first indexing guide distinguishes those configurations and describes the use of Vary: User-Agent for dynamic serving.
Record which request property is meant to select the response. If nobody can name it, investigate the application before editing cache rules. A cookie, language header, device classification, or deployment version can each point toward a different cause.
Understand what Vary actually tells a cache
The protocol rule is about selecting stored responses. Under RFC 9111, section 4.1, a cache must account for the request fields named by Vary before reusing a response. Matching the URL alone is insufficient when the stored response declares additional selection fields.
MDN's Vary reference explains the header's role in separating cached responses and calls for consistent values across responses for the same URL, including validation responses. Read the actual header, rather than inferring it from a framework setting.
That still leaves implementation work. A CDN may have its own cache eligibility rules, custom cache keys, or configuration requirements. Verify the current documentation for the service in your delivery path. Do not assume that a header copied from an example changes every cache layer in the way you intend.
Create a small request matrix
For the fictional site, define request A as an anonymous desktop request and request B as an anonymous mobile request. Keep the URL, language preference, authentication state, and other relevant fields unchanged. Change only the property under investigation.
For each response, retain the status, relevant request headers, response headers, and the body section containing the compatibility warning. Record time and the observed cache-status information when the service exposes it. A body digest can help compare responses, but keep the meaningful text difference alongside it.
Do not use a personalized session as your public baseline. A signed-in page may legitimately contain different material. Keep secrets, cookies, and private content out of shared test notes. The purpose is to compare public representations under controlled conditions.
This small matrix is easier to interpret than a large crawl that changes geography, language, cookies, and device type at once.
Reverse which request warms the cache
Assume the intended origin behavior is that both variants contain the warning, despite different layout markup. First check that assumption through an authorized staging or origin diagnostic route. Do not expose a protected origin to make testing easier.
Using the project's established cache-testing controls, begin with a known empty or isolated test cache. Send A, repeat A, then send B. Record the warning and the available cache signal each time. Repeat the experiment from a fresh test state, this time sending B before A.
Suppose the first sequence gives both requests the desktop body, while the reversed sequence gives both the mobile body. That fictional pattern suggests the first stored response is being reused across requests that should remain distinguishable. It does not by itself identify the exact layer or configuration error.
If the response changes even without a shared cache, the origin's selection logic deserves attention first. The order reversal helps distinguish these possibilities instead of assuming every mismatch is a Vary problem.
Trace the first layer where the body becomes wrong
Compare the same meaningful body section at the application, the edge, and the final browser view where those observations are available. Retain the request conditions at every point. A correct origin response and incorrect edge response narrow the investigation; an incorrect origin response calls for a different repair.
Check whether the cache key includes the intended selection property or a documented normalized version of it. Inspect rules that remove headers, rewrite the URL, bypass storage, or serve an older object. Make one change at a time so the next result remains interpretable.
Avoid broad fixes such as varying every page by every cookie. Those changes can create many stored combinations and make the system harder to reason about. The right selection rule follows the actual public content difference and the service's supported configuration.
When the content does not need to vary, serving equivalent primary information is often a simpler design than maintaining two independent content versions.
Check the SEO consequence separately
A cache defect becomes a discovery problem when the wrong response omits or changes information a search system needs. Inspect the main answer, important links, title, canonical, and indexing directives. A successful status alone does not establish that the intended page arrived.
Google states that it uses the mobile version of content for indexing and ranking. Its mobile-first guidance also calls for equivalent primary content across desktop and mobile versions. That makes a missing mobile warning or important answer worth fixing even when a desktop screenshot looks correct.
Keep the claim engine-specific. Google's documented rendering and indexing behavior does not establish how every AI search crawler handles the same site. For another engine, consult its current documentation and examine actual delivery evidence.
Changing a browser's user-agent string is a diagnostic input, not proof that the request came from a real crawler. Use verified request records when identifying actual crawler traffic.
Verify the repair with the same sequences
After the approved change, repeat both warm-up orders using the same controlled inputs. Both A-then-B and B-then-A should now produce the intended body for each request. Record the configuration version and any narrowly scoped purge performed through the project's normal process.
Recheck the response after a cache hit, not only after a fresh origin fetch. A fix that works only on the first request can leave the original problem unresolved. Also check an ordinary anonymous visit to ensure the repair did not create a new user-facing discrepancy.
Retain a compact evidence bundle: request definition, response headers, meaningful body comparison, and test order. That is enough for another developer to assess the conclusion without publishing sensitive request logs.
If the result remains inconsistent, mark it unresolved. Do not convert a few correct responses into a claim that indexing or AI visibility has recovered.
Keep delivery tests tied to useful public answers
For a SaaS team, prioritize templates whose public answers affect evaluation: supported platforms, API capability, product limits, and implementation requirements. A delivery test is most useful when it protects information buyers actually rely on.
After the delivery behavior is stable, monitor the affected URL through the relevant search tools. Separate successful fetching from indexing, search appearance, linked AI citations, and recommendations. Those are later observations, each with its own limits.
The durable improvement is a repeatable check that proves the intended public answer survives a warm cache. Begin with one URL, one meaningful content difference, and two request orders. Expand the test only when the first result gives you a reason to do so.
Hey I'm Uriel Bitton. I write about SEO, AEO, GEO, and helping brands get found in AI answers.
Subscribe for practical ways to grow your brand's visibility in search and AI answers.
Explore HoneyWeRank for SEO and AI visibility.
Top comments (0)