DEV Community

Jangwook Kim
Jangwook Kim

Posted on • Originally published at jangwook.net

The meta Line That Decides If AI Overviews Can Quote You

I used to think a robots meta line only turned off the little summary under a search result. Now that same line decides whether Google's AI Overview is allowed to quote your page at all. Google Search Central rewrote its docs to say so explicitly. And yet plenty of sites still carry a nosnippet someone pasted into a layout template years ago — quietly locking AI search out of their entire content, without anyone noticing.

Today I didn't just read the directives. I built one page that's deliberately broken and one that's fixed, then wrote a small audit script that parses the HTML and decides what an AI system could actually quote from each. Every log below is real output from that sandbox.

Snippets, AI Overviews, and what robots meta actually governs

Terms first. A snippet is the summary line under a title in search results. It used to be nothing more than a click-bait preview. AI Overviews and AI Mode are Google's generative answers pinned to the top of search (or in a conversational view). They summarize several pages into prose and cite the source pages as evidence. Here's the pivotal shift: Google decided that whether a page is used as input to those generative answers is governed by the same switches as the old snippet directives.

That switch is a snippet directive inside <meta name="robots">. This is where a common confusion starts. robots.txt and the robots meta tag are entirely different levers. Controlling AI crawler access itself with robots.txt and llms.txt decides whether they get in; the robots meta tag decides what may be indexed and displayed once they're in. Order matters because of this. If you block crawling in robots.txt, Google never reads the page's meta tags in the first place. For a snippet directive to take effect, the page has to be crawlable and indexable. Confuse "block access" with "tune display" and you get the exact opposite of what you intended.

Why now? Because a meaningful chunk of search traffic is drifting from "a list of links" toward "a summarized answer." Being cited as evidence in an AI answer has itself become a distribution channel. And the doorway to that channel hangs on a very old line of markup.

The official rules — four directives, and the exact values

Straight from the Google Search Central robots meta docs. Not a guess — what the page actually says.

nosnippet. The definition: "Do not show a text snippet or video preview in the search results for this page." Then the decisive sentence follows. This applies to "all forms of search results (web search, Google Images, Discover, AI Overviews, AI Mode) and will also prevent the content from being used as a direct input for AI Overviews and AI Mode." So nosnippet no longer means "hide the summary." It means "drop me from the citation pool."

max-snippet:[number]. Sets the maximum character count for a text snippet. 0 means no snippet (effectively the same as nosnippet), -1 lets Google choose the length, a positive number caps it. The docs say the same thing about AI: it "will also limit how much of the content may be used as a direct input for AI Overviews and AI Mode." In other words, max-snippet:0 blocks citation, max-snippet:-1 allows the whole thing.

max-image-preview:[none|standard|large]. The maximum size of an image preview in results. You have to open it to large for a large preview to be eligible. The default is usually standard, so if you want your hero image shown big and never touch this, you're stuck with a thumbnail.

data-nosnippet. This one is an HTML attribute on an element, not a meta tag. Use it when you want to exclude just one block from the snippet rather than the whole page. Two traps here. First, the docs are explicit that this attribute only works on span, div, and section. A <p data-nosnippet> is simply ignored. Second, there's a warning not to "add or remove the data-nosnippet attribute of existing nodes through JavaScript." The reason is simple: many AI crawlers don't render JavaScript, so an attribute you attach at runtime doesn't exist as far as the crawler is concerned. It has to be baked into the initial HTML your server returns.

When they conflict, the most restrictive one wins

This is where real sites break. A single page can carry several directives: a generic robots tag, a googlebot-specific tag, plus whatever a CMS plugin injected. Which wins? The docs are blunt: "In the case of conflicting robots rules, the more restrictive rule applies. For example, if a page has both max-snippet:50 and nosnippet rules, the nosnippet rule will apply."

The scary part is the direction. Rules never merge toward looser; they only merge toward tighter. You can put max-snippet:160 in the googlebot tag to "give Google a generous snippet," but if a nosnippet still sits in the generic robots tag, the result is a snippet of zero. You thought the door was open; it was locked. That's exactly why eyeballing two tags and concluding "looks fine" is dangerous.

So I decided to audit it with a parser.

I built two pages and audited them with a parser

In a throwaway sandbox outside the repo, I created two static HTML files. One, broken.html, packs in the mistakes you actually see in the wild. The other, fixed.html, does what was intended.

The <head> and body of broken.html contain these:

<!-- mistake 1: nosnippet on the generic robots tag — the classic template-wide paste -->
<meta name="robots" content="index,follow,nosnippet">
<!-- mistake 2: googlebot tries to open a snippet, but the nosnippet above is "most restrictive" and wins -->
<meta name="googlebot" content="max-snippet:160">
...
<!-- mistake 3: data-nosnippet on a p → unsupported element, ignored -->
<p data-nosnippet>Internal note: I want this out of the snippet, but p doesn't work.</p>
Enter fullscreen mode Exit fullscreen mode

fixed.html opens the page up and isolates the single block it wants excluded onto a supported element:

<!-- page level: allow full snippet + large image preview -->
<meta name="robots" content="index,follow,max-snippet:-1,max-image-preview:large">
...
<!-- element level: internal note only, on a supported span -->
<span data-nosnippet>Internal note: excluded from the snippet.</span>
Enter fullscreen mode Exit fullscreen mode

Then I wrote audit.mjs: it parses the HTML with node-html-parser, reads both the generic robots and the googlebot directives, merges them under "most restrictive wins," and checks the tag name of every data-nosnippet element. The merge core looks like this:

// if any nosnippet or max-snippet:0 is present, everything is blocked
function effectiveSnippetPolicy(dirsList) {
  let hardZero = false, cap = -1; // -1 = Google chooses the length
  for (const d of dirsList) {
    if (d.nosnippet) hardZero = true;
    if (d.maxSnippet === 0) hardZero = true;
    else if (d.maxSnippet > 0) cap = cap === -1 ? d.maxSnippet : Math.min(cap, d.maxSnippet);
  }
  if (hardZero) return { chars: 0, aiInput: 'blocked' };
  return { chars: cap, aiInput: cap === -1 ? 'full' : `capped@${cap}` };
}
Enter fullscreen mode Exit fullscreen mode

Here's the actual output for both files:

========================================================
FILE: broken.html
  effective text snippet : 0 chars
  AI Overview text input : blocked
  image preview          : standard(default)
  data-nosnippet elements: 1
  [ERROR] PAGE_SNIPPET_BLOCKED
  [WARN]  CONFLICT_MOST_RESTRICTIVE: robots=nosnippet vs googlebot=max-snippet:160 → the more restrictive nosnippet wins (official).
  [INFO]  IMAGE_PREVIEW_LIMITED
  [ERROR] DATA_NOSNIPPET_BAD_ELEMENT: <p data-nosnippet> ignored. Only span/div/section (official).
========================================================
FILE: fixed.html
  effective text snippet : full (Google chooses)
  AI Overview text input : full
  image preview          : large
  data-nosnippet elements: 1
  findings               : none — clean
========================================================
Enter fullscreen mode Exit fullscreen mode

Audit of robots snippet directives — broken.html blocks AI input, fixed.html allows full input

The numbers make it plain. broken.html, no matter how good its content, is dropped wholesale from AI Overview input. The developer who put max-snippet:160 in the googlebot tag and believed "the snippet is open" was standing at a locked door. fixed.html is fully quotable, has a large image preview open, and excludes exactly one line — the internal note. The four problems the audit flagged (page-wide block, wrong element, conflict, image limit) are all patterns that recur on live sites.

One rule for the audit: run it against the HTML your server actually returns. The Elements panel in DevTools shows the DOM after JavaScript runs, so any meta tag manipulated at runtime will differ from what the crawler sees. Pull the raw response instead — curl -s <URL> | grep -i 'name="robots"'. That's the trap I once hit: DevTools showed a clean max-snippet:-1, while the raw server response still carried a nosnippet the CMS had injected. The truth is in the first bytes, not the rendered screen.

Don't judge this by eye. Just as I validated JSON-LD structured data in CI, snippet directives deserve an automated parser check in the build pipeline. A human misses the moment one tag lands wrong; a parser doesn't.

An honest limit — eligibility, not a guarantee

Time to lower expectations. Opening max-snippet:-1 and max-image-preview:large does not make AI Overviews quote your page. These directives only open the eligibility to be quoted; whether you actually are is Google's call. They don't raise your ranking either. Google has never said snippet directives are a ranking signal. There's no promise that removing nosnippet brings more visitors.

Look at the trade-off in the other direction honestly, too. nosnippet isn't always a mistake. For the body of paid content, information that should only appear behind a login, or a page whose click incentive evaporates if it's shown whole in results, tightening the snippet is reasonable. What's new is that this choice now carries a cost: you're giving up the chance to be cited in an AI answer. You used to hide only the snippet; now you're also hiding your presence in generative search.

My position: for public content — especially the docs, guides, and product explanations people arrive at looking for an answer — there's rarely a reason to tighten the snippet. If you must, exclude the offending block with data-nosnippet rather than the whole page. A page-wide nosnippet usually lingers in a state where nobody remembers why it was added, quietly eroding your citation chances.

What to do today

The checklist for today:

  • Open your layout template's global robots meta first. If a shared header carries nosnippet or max-snippet:0, your entire site is already out of the AI citation pool.
  • Default for content you want quoted: max-snippet:-1, max-image-preview:large. That's the explicit signal that says "fine to use me as evidence in an AI answer."
  • Exclude blocks, not pages. Isolate internal notes, boilerplate, and paid-content teasers with data-nosnippet on a span, div, or section. It won't work on a p or anything else.
  • Don't toggle data-nosnippet with JavaScript. Bake it into the initial server HTML. In front of a crawler that doesn't render, a runtime attribute doesn't exist.
  • Catch conflicts with a parser. Read both the generic robots and googlebot tags, merge under "most restrictive wins," and verify the effective policy with a CI script, not your eyes.

If you need structured data emitted reliably server-side, or want an existing site audited from the snippet-and-crawler angle to see how it's exposed to AI search, I take on consulting and implementation work personally — reach me through the contact link on my profile. An old line of meta blocking an entire traffic path is more common than you'd think.

Top comments (0)