Cloudflare announced a Disallow AI Training setting on September 15 for sites that want search visibility without permitting training use of the same crawled content. The control addresses mixed-use crawlers operated for both purposes.
Previously, refusing that crawler could also mean losing its search indexing. Cloudflare lists Apple, Google and Microsoft as participating operators, with different controls already available. At announcement, the new setting did not automatically convey a no-training preference to Bing through robots.txt. Microsoft targeted that support for early 2027; site owners still needed Bing’s existing NOARCHIVE control to express the preference.
The company combines the published preference with crawler identification and network controls. A declaration in robots.txt alone cannot establish that a visitor follows it; Cloudflare describes checking operator behavior and restricting crawlers that ignore applicable rules.
Search, model training and AI-generated summaries remain separate uses. The announcement also discusses future controls over how much content appears in summaries, but those planned controls should not be confused with the training preference being introduced.