Google has said it intends to back a shared AI training control in robots.txt, according to a Google Search Central post by John Mueller dated October 30, 2026. The control is built on work from the AIPREF working group at the IETF.
What is planned
At the moment, a website that wants to state its AI training preferences has to address each AI developer separately. The new approach adds a shared policy token to robots.txt. A site can use it to set a broad preference that participating AI providers read.
Google describes the mechanics for its own case. If a site uses the token to opt out of training and has not explicitly specified Google-Extended, Google plans to read that as a Google-Extended disallow. Google-Extended is the training control it has offered since 2023. Other providers can handle the token in a similar way, so a site could opt out across all of them or choose provider by provider.
Existing controls stay
Company-specific controls will keep working next to the shared signal. A site can set a general preference with the token and still give Google its own instruction through Google-Extended.
Google further states that websites which explicitly permit Google-Extended can be featured in answers and gain visits through grounding within Gemini Apps.
Status and timing
This is a draft proposal, not a finished standard. Google says it reflects feedback from publishers, civil society groups, regulators and other stakeholders, and that broad backing across major AI providers would enable a future cross-industry standard. The token carries the AI training definition from the IETF, while discussions on the rest of the AIPREF vocabulary continue.
Google says the proposal fits the EU Code of Practice for General Purpose AI model providers under the EU AI Act, which recognises robots.txt as a main way for publishers to state machine-readable preferences.
Google aims to support the control in 2027, together with other AI providers, and says it will post updates. The post does not name the other providers, and it does not give the token's exact syntax.
What to check
The practical step is an audit. Read your current robots.txt, note any Google-Extended rule, and settle your position on AI training for your public content before the shared token arrives.
Top comments (0)