OpenAI and Google Back One 'No AI Training' Switch for Websites

OpenAI and Google both said on October 30 that they will honor a single, shared signal in robots.txt that lets a website say it doesn’t want its content used for AI training, across participating companies at once. Microsoft made the same commitment the same day.

One rule instead of many

Today, a site that wants to opt out has to find each AI company’s crawler or control and write a separate rule for it. As OpenAI put it, operators had to “identify individual crawlers and maintain separate rules for each and every provider.”

The new approach uses a shared policy token in an ordinary robots.txt group:

User-Agent: __usage_ai-training
Disallow: /

That expresses a preference against AI training for the whole site. Paths can be allowed or disallowed individually, the same way robots.txt already works.

Who supports it, and when

None of the companies reads the signal yet:

• Microsoft says it will honor the new preferences as it crawls starting February 1, 2027.
• Google says it aims to support the control “along with other AI providers in 2027.”
• OpenAI says it aims to implement it “in the coming months” and will share details when it does.

The specification itself, an IETF Internet-Draft dated October 30, was written by Kevin Kelley of Anthropic, the company behind Claude.

The fine print

This is a statement of preference about how content is used, not a block on crawling. The draft says it “is not a directive for crawling behavior,” and a blanket User-Agent: * rule doesn’t count as an opt-out. It’s also still an individual draft: the IETF notes it is “not endorsed by the IETF and has no formal standing” in its standards process.

Existing company-specific controls stay in place. At Google, an opt-out through the shared token, when a site doesn’t mention Google-Extended, will be treated as if the site had disallowed Google-Extended. Google says sites that specifically allow Google-Extended are eligible to appear in Gemini Apps answers through grounding, so a blanket opt-out could also cost a site those citations.

Microsoft is pairing the training signal with a separate page-level setting for grounding, and says sites relying on its older NoArchive and NoCache tags should switch before March 1, 2027.