Making it easier for websites to express AI training preferences
OpenAI will honor robots.txt signals for AI training data preferences from websites
OpenAI announced it will respect a shared robots.txt standard allowing website operators to signal their AI training data preferences to participating providers. This is a notable governance move that could reshape how AI companies handle web data consent at scale. It also signals industry momentum toward standardized opt-out mechanisms, potentially influencing how other labs approach training data sourcing.