Google, OpenAI and Microsoft are moving toward a common way for website owners to express whether their content may be used for AI training. The effort centers on a machine-readable preference in robots.txt, designed to reduce the need for publishers to manage separate training controls for every AI provider.
The alignment is significant, but it is not a universal on-off switch for AI visibility. Training preferences, grounding in AI answers, and ordinary search crawling remain related but distinct decisions. Google and Microsoft have also set provider-specific rules that publishers should understand before changing existing directives.
What the shared AI training preference is
On October 30, 2026, Google announced plans to support a shared AI training control for websites. The proposed approach uses the __usage_ai-training token in robots.txt, allowing a site to communicate an AI-training preference to participating providers through one policy signal.
Google's announcement on the shared AI training control describes the initiative as a way to give website owners a more consistent choice across services. OpenAI made a parallel announcement the same day, saying it will honor the common robots.txt signal in the coming months while retaining its existing crawler controls.
The underlying specification is an Internet-Draft from the Internet Engineering Task Force. The machine-readable training preference signal draft defines how the token is intended to work among participating providers. An Internet-Draft is a proposed technical specification, not a guarantee that every AI company or crawler will implement identical behavior.
Anthropic has also been identified as supporting the IETF-driven approach. Its practical value will depend on adoption and on each provider's interpretation and rollout of the preference.
Training, grounding and search are not the same thing
A central issue for publishers is that a training choice does not necessarily answer whether a page can be retrieved as a cited source in an AI response.
AI training concerns whether content may be used to train or improve a model. Grounding concerns whether a system can retrieve or use current web content to support an answer, often with a citation or link. Search crawling and indexing concern discovery and ranking in conventional search results.
These functions can involve different systems, crawlers and controls. A publisher that wants to restrict model training may still want to be eligible for search referrals or grounded answers. Conversely, a site that permits training may decide it does not want its content used for a particular form of AI retrieval.
Google's implementation makes this distinction especially important. Google says that when a site uses the shared opt-out but does not separately specify Google-Extended, Google will treat that as a disallow for Google-Extended. That affects Gemini grounding in Gemini Apps.
Google also says that Google-Extended remains a separate control. Sites that specifically opt in to Google-Extended can be eligible to appear in Gemini Apps groundings. In other words, adopting the shared training preference without reviewing Google-Extended could have implications beyond a publisher's intended training decision.
Microsoft's separate grounding transition
Microsoft's Bing documentation adds another layer. Alongside the shared training preference, Microsoft is introducing an AI Grounding preference at the page level, with options to disallow or allow grounding.
Microsoft says it will begin honoring the new preferences on February 1, 2027. Its guidance also states that NoArchive and NoCache will be superseded, with a March 1, 2027 cutoff for the older signals. Publishers relying on those directives for Bing-related AI behavior should review the transition rather than assuming historic controls will continue to carry the same meaning.
| Date or period | Development | Publisher implication |
|---|---|---|
| October 30, 2026 | Google and OpenAI announced support for a common AI-training preference; the IETF draft was published. | Review the shared signal and existing provider-specific controls before making policy changes. |
| Coming months | OpenAI said it will implement support for the common signal. | Keep existing OpenAI crawler controls in place until implementation details and operational status are clear. |
| February 1, 2027 | Microsoft plans to begin honoring its new AI preferences. | Prepare any Bing training and page-level grounding decisions before the stated start date. |
| March 1, 2027 | Microsoft's stated cutoff for old NoArchive and NoCache signals. | Replace reliance on superseded signals with the applicable new preferences. |
What publishers should do now
The new shared signal can simplify policy management, but it does not eliminate the need for a platform-by-platform review. Start by documenting the organization's actual policy rather than treating an opt-out as a routine technical change.
Consider these practical steps:
- Separate the decisions. Define positions on model training, grounded AI answers, ordinary search indexing and content licensing. They may not all be the same.
-
Audit current directives. Identify existing
robots.txtentries, crawler-specific rules and MicrosoftNoArchiveorNoCacheusage. Record why each was added and which system it was intended to affect. - Review Google-Extended deliberately. A shared opt-out without a Google-Extended specification is treated by Google as a disallow for Google-Extended. Assess whether that outcome matches the site's Gemini Apps grounding strategy.
- Plan for Microsoft's dates. If Bing AI grounding matters to the business, prepare for the February 1, 2027 start and the March 1, 2027 transition from older signals.
- Apply controls consistently. Large sites, subdomains, multilingual properties and third-party publishing arrangements can make policy implementation more complicated. Ensure technical directives reflect the content owner's decision.
- Monitor provider documentation. The common signal is an emerging standard, and providers retain their own implementation details. A policy should be revisited as support becomes operational.
For SEO teams, this is also a governance issue. A technical directive may be deployed by web operations, while its effects can be felt in content distribution, brand visibility, partnerships and search performance. Stakeholders should agree on trade-offs before rules are changed in production.
The shared signal should not be interpreted as a measurement tool for AI visibility. It communicates a preference. It does not establish whether a page will rank, be crawled, be cited by an AI system or receive visits.
If you need a structured way to assess how technical controls and content accessibility may affect AI discovery, create a GEO Search Leads account. It can help your team evaluate website GEO readiness and identify optimization opportunities while you develop a deliberate policy for AI training and grounding preferences.
Frequently Asked Questions
What is the shared AI training opt-out signal?
It is a proposed common preference for robots.txt, using the __usage_ai-training token, that lets websites express an AI-training preference to participating providers.
Does opting out of AI training also block Google Search?
Not necessarily. AI training, grounding and traditional search crawling or indexing are separate functions. Publishers should review the controls that apply to each function rather than assuming one preference governs all of them.
How does the shared signal affect Google-Extended?
Google says that if a website uses the shared opt-out and does not specify Google-Extended, Google will treat it as a disallow for Google-Extended. Google-Extended is a separate control, and opting into it can allow eligibility for Gemini Apps groundings.
When will Microsoft honor its new AI preferences?
Microsoft says it plans to begin honoring the new preferences on February 1, 2027. It also identifies March 1, 2027 as the cutoff for the older NoArchive and NoCache signals.
Conclusion
A shared AI training preference is a meaningful step toward clearer publisher controls, but it does not create a single policy for every kind of AI and search visibility. The immediate priority is to distinguish training from grounding, preserve the controls that match business goals, and prepare for the provider-specific changes scheduled through 2027.
Top comments (0)