OpenAI will honor a shared robots.txt AI-training preference signal
OpenAI will honor a common robots.txt signal so site operators can express one generative-AI training preference to multiple participating providers; existing crawler controls remain, with implementation in coming months.
On 30 October 2026, OpenAI announced it will honor a common signal in robots.txt for preferences about using site content for foundational generative AI training. Operators previously maintained separate rules per crawler; a shared signal reduces that work.
OpenAI aims to implement the signal in the coming months and will share implementation details later. It does not replace existing crawler controls such as GPTBot rules. The move builds on IETF AI Preferences working-group work; OpenAI encourages other providers to honor the signal.