Definition
AI Preferences, usually shortened to aipref, is an IETF standards effort that gives publishers one shared vocabulary for saying how their content may be used by AI systems, such as whether it may train a generative model, and defines how to attach those preferences to content through robots.txt or an HTTP header. As of November 2026 it is still a set of Internet-Drafts, not a finished standard.
The IETF chartered the AI Preferences Working Group in January 2025, chaired by Mark Nottingham and Suresh Krishnan. In its announcement, the IETF said AI vendors 'use a confusing array of non-standard signals in the robots.txt file' and that publishers were losing confidence their preferences would be followed. Today each AI company publishes its own tokens, such as Google-Extended or ClaudeBot, and a site has to name every one. aipref aims to replace that with statements about uses rather than about bots.
The work has two documents. The vocabulary draft (draft-ietf-aipref-vocab-08, September 13, 2026) defines three categories of use, each with a short label:
train-ai: using an asset to modify the learned parameters of a generative AI model.ai-use: using an asset as input to a generative AI model when the user did not supply it directly, which covers grounding and retrieval for AI answers.search: using an asset in an application whose main purpose is to point users to its location, with links and short excerpts. The draft says this category does not include generating summaries, and that it overrides the other two.
Each category can be set to y (allow) or n (disallow). A category left out is 'unknown', and the draft takes no position on what default a crawler should assume. When several statements conflict, the most restrictive one wins.
The attachment draft (draft-ietf-aipref-attach-05, August 18, 2026, edited by Gary Illyes of Google and Martin Thomson of Mozilla) adds a Content-Usage rule to robots.txt and a Content-Usage HTTP response header. A robots.txt group can say Allow: / and Content-Usage: train-ai=n, and a rule can be scoped to a path, for example Content-Usage: /ai-ok/ train-ai=y. This separates acquisition (Allow and Disallow decide what may be fetched) from usage (Content-Usage says what may be done with it).
The drafts are not done. The working group's charter set August 2026 as the date to send both documents to the IESG, and by mid-November 2026 neither had been sent. The group's agenda for IETF 127, with sessions on November 16 and 19, 2026, still lists open issues on both, including how to distinguish access from use and how preferences apply to caches, embeddings and other derived forms. The vocabulary draft itself warns that its core sections 'do not yet have consensus', so labels and definitions may still change. Adoption is minimal: a ProxyPicker scan on October 8, 2026 of 6,735 robots.txt files from Chrome's 10,000 most visited origins found 13 using Content-Usage, 12 of them Vinted country sites. The same scan found 122 files with Cloudflare's Content-Signal line and 950 blocking GPTBot by name.
The biggest change in this period came from outside the working group documents. On October 30, 2026 Kevin Kelley of Anthropic published an individual Internet-Draft, draft-kelley-aipref-training-preference-signal-00, that expresses one preference, AI training, through an ordinary robots.txt group keyed to the policy token __usage_ai-training. It adds no new directives: Disallow means a preference against training for that path and Allow a preference for it. A User-agent: * group never matches the token, a file without the group expresses no preference, and the draft says the token is not a crawling directive. It reuses the vocabulary draft's definition of AI training. The same day OpenAI, Google, Microsoft and Anthropic said they will honor it: Anthropic has begun rolling it out, OpenAI aims to 'in the coming months', Microsoft starts on February 1, 2027 and Google aims for 2027 (OpenAI, Google, Bing). Microsoft also extends the training preference to individual pages through a meta tag or HTTP header and adds a separate page-level AI Grounding preference. As an individual draft it has no IETF standing yet, and it expires on May 3, 2027 unless revised.
aipref sits next to several related signals. Cloudflare's content signals, launched in September 2025, express a similar idea with different labels (search, ai-input, ai-train). The EU's TDM rights reservation gives opt-outs legal weight in Europe, and the vocabulary draft is written to work both where preferences carry legal obligations and where they do not. Licensing schemes such as Really Simple Licensing go further and attach payment terms. None of these are technical blocks: the draft states plainly that a recipient decides whether to follow a preference.
For AI visibility, the category that matters most is ai-use. Under the draft's definitions, AI-generated search answers fall under AI use rather than search, so a site that sets ai-use=n is asking AI search engines not to ground answers on its pages, which would mean no citations from systems that honor it. A brand that only objects to AI training data collection should set train-ai=n and leave ai-use and search alone. Adding a Content-Usage line costs little, but per-crawler robots.txt rules remain the mechanism AI companies document and follow today, with the __usage_ai-training group as the first shared signal the largest providers have committed to.
Examples of AI Preferences (IETF aipref)
- A regional newspaper adds 'Content-Usage: train-ai=n' under its 'User-Agent: *' group while keeping 'Allow: /', so crawlers may still fetch and index its articles but are asked not to use them for model training.
- A stock photo library cannot put robots meta tags inside JPEG files, so its CDN sends a 'Content-Usage: train-ai=n' HTTP header with every image response.
- A SaaS company wants its docs cited in AI answers but not used for training. Its SEO lead sets train-ai=n and deliberately leaves ai-use unset, because ai-use=n would ask AI search engines not to ground answers on the docs.
- A publisher's legal team maps its existing EU text and data mining opt-out to train-ai=n in robots.txt, alongside its named blocks for GPTBot and ClaudeBot, so the same preference is stated in both the new and the old format.
