Emerging standards for AI-usage preferences: aipref, Content-Usage, and RSL

A robots.txt file addresses named crawlers, and the companion note on robots.txt and AI crawlers sets out what that can and cannot do. A separate line of work asks a different question: not which crawler may fetch a page, but for which uses the content may be processed once fetched — training a model, feeding a model at answer time, building a search index — and defines new places to state that preference. Three formal texts now specify parts of this. Two are Internet-Drafts of the IETF’s AI Preferences (aipref) working group; one is a version-1.0 specification from an industry group. This note reads those texts as published and states, precisely, what each defines and how mature each is. It is distinct from the note on Content Signals in robots.txt, which reads one vendor’s robots.txt policy; the vocabularies overlap, and the connection is drawn here only where the standards’ own text draws it.

The IETF vocabulary: two usage categories, three values

The working group’s vocabulary document, draft-ietf-aipref-vocab-06, “A Vocabulary For Expressing AI Usage Preferences,” is an Internet-Draft dated 28 April 2026 [1]. Its abstract states that it “defines a vocabulary for expressing preferences regarding how digital assets are used by automated processing systems” [1]. The model is three-valued: after processing a statement of preference, a recipient assigns each category of use one of “allowed”, “disallowed”, or “unknown”, and “in the absence of a statement of preference, all usage categories are assigned a preference value of ‘unknown’” [1]. The draft defines two categories. AI Model Training is “the act of using an asset in the production or refinement of an AI model that can generate content in one or more modalities” [1]. Search is “use of an asset in an application where the primary purpose of the application is to select assets and direct users to the location of those assets,” conditioned on the search output including “a direct reference or link to the original location,” and it “does not include the use of assets to generate summaries” [1]. The draft’s exemplary serialization is a Structured-Fields dictionary whose keys are the category labels train-ai and search and whose values are y (allow) or n (disallow); its own example reads train-ai=y, search=n [1]. One caveat is stamped on the vocabulary itself: the section that defines the categories carries the note “This section does not yet have consensus” [1].

Attaching a preference to content: a header and a robots.txt rule

A companion draft, draft-ietf-aipref-attach, “Associating AI Usage Preferences with Content in HTTP,” defines where such a statement is placed [2]. Its abstract says it “defines how preferences can be signaled as part of the acquisition of content in HTTP” and “updates RFC 9309 to allow for the inclusion of usage preferences” [2]. It specifies two mechanisms, each carrying a statement built from the vocabulary draft: “a Content-Usage header field for HTTP” and “a Content-Usage directive for the Robots Exclusion Protocol” [2]. In an HTTP response the header appears as Content-Usage: train-ai=n; in robots.txt the same statement appears as a Content-Usage: line inside a group [2]. The robots.txt case introduces a distinction the draft states explicitly: it creates “a two-stage arrangement that distinguishes acquisition and usage,” where “acquisition relies on Allow/Disallow rules” and “usage preference relies on Content-Usage rules,” and where “usage preferences apply only to those resources that can be crawled according to Allow/Disallow rules” [2]. Access — whether a crawler may fetch the page — and usage — what may be done with the page once fetched — are thereby made separate statements in the same file.

RSL 1.0: a usage vocabulary carried inside a license

Really Simple Licensing (RSL) 1.0, document RSL-SPEC-1.0, is an XML-based specification published on 10 December 2025 [3]. It “builds on the ideas of the RSS syndication format and the Robots Exclusion Protocol [RFC 9309],” and for robots.txt it “extends the Robots Exclusion Protocol (REP) [RFC 9309] with a new directive, License, that associates a website or section of a website with one or more governing RSL license documents” [3]. Here the robots.txt line is a pointer rather than the preference itself: License: https://example.com/rsl-license.xml names an RSL XML document, and the usage preference lives inside that document’s <permits> and <prohibits> elements [3]. RSL’s usage vocabulary is wider than the IETF draft’s two categories: its normative usage tokens are all, ai-all, ai-train, ai-input, ai-index, and search [3]. ai-train is “training or fine-tuning AI models”; ai-input is “input into AI models, including retrieval-augmented generation, grounding, or other use of the content to produce generative AI responses or search summaries”; ai-index is “inclusion in an AI system’s internal index or retrieval database”; and ai-all “explicitly includes the ai-train, ai-input, and ai-index usage types” [3]. RSL states that this vocabulary “includes the Cloudflare Content Signals vocabulary and MAY include terms from additional standardized vocabularies as they become available (e.g., IETF AI Preferences)” [3] — the Content Signals policy that names those tokens is read in the companion note. RSL also does something the IETF drafts do not: alongside the usage vocabulary it defines <payment>, <legal>, and licensing terms, plus OPTIONAL network protocols for license acquisition and enforcement. Its own guidance frames the division of labor — the IETF work “defines a vocabulary of allowed and prohibited usage cases … but does not define a mechanism for obtaining permission or compensating publishers,” and RSL “complements” it by adding “machine-readable licensing and compensation rules” [4].

Reading the maturity labels precisely

The three texts sit at three different points on the standards path, and the word “standard,” in the sense of a published RFC, applies to none of them. The two IETF documents are Internet-Drafts: working documents that, in the boilerplate each carries, are “valid for a maximum of six months” and which “it is inappropriate to use … as reference material or to cite … other than as ‘work in progress’” [1]. draft-ietf-aipref-vocab-06 lists an intended status of Standards Track, but that is a target for a future RFC, not a current standing, and — as above — the section defining its categories does not yet have working-group consensus [1]. RSL 1.0 is not an IETF or W3C document at all: it is published by the RSL Technical Steering Committee under its own status label, “Recommendation,” which the specification defines as “considered stable, has received broad review, and is suitable for implementation and deployment,” while adding that “future revisions may supersede or update this specification” [3]. That is an industry-group status term, distinct from a W3C Recommendation and from an IETF RFC. The one fully published standard in the picture is the base that both robots.txt mechanisms extend, RFC 9309, whose text is also the reason none of these preferences is self-enforcing: it states that robots.txt “rules are not a form of access authorization” [5].

A usage axis, not a crawler axis

What these texts share is that they legislate over uses, not over crawler identities. That is the distinction the taxonomy of AI-related crawlers (BA-C-6) draws for robots.txt tokens, seen from the other side: BA-C-6 sorts crawlers into five functional classes — training, retrieval, user-fetch, agent, and search — each reached by naming a token, whereas the vocabularies here name usages — training, search, AI input, AI indexing — reached without naming any crawler. The attach draft’s explicit separation of acquisition (Allow/Disallow, which name crawlers) from usage (Content-Usage, which names uses) is a formal statement of that same two-axis structure [2]. The axes map onto each other only loosely: a usage such as ai-input can be served by more than one crawler class, and a single crawler can perform more than one use. These usage vocabularies also sit upstream of what this publication measures — a preference governs whether content may be used for a purpose, whereas a mention or a citation is an observation of whether content later appears in an answer, a separate axis this note does not conflate.

Limitations

This note reads three documents as published and archived on 2026-07-10, alongside RFC 9309. Two of them are Internet-Drafts and will change: revision numbers advance, categories may be added or removed, and the vocabulary section is explicitly marked as lacking consensus, so every token and value stated here is current-as-read rather than settled [1][2]. The attach mechanism is cited from the working group’s editor’s copy, which tracks ahead of the numbered revision on the datatracker; both are the same work in progress [2]. RSL 1.0 is stable in its own terms but may be superseded by later revisions under a new namespace [3]. This note does not report how widely any of these mechanisms is deployed; the presence or absence of these fields in the wild is a measurement question for a later signal census, not a claim to be made from the standards text. Nor does it assert that any crawler reads or honors any of these signals — as with robots.txt, compliance is a decision made by whoever operates a crawler, not a property of the file [5].

@techreport{ai-usage-preference-standards,
  author       = {{Barkhausen AI}},
  title        = {Emerging standards for AI-usage preferences: aipref, Content-Usage, and RSL},
  institution  = {Barkhausen AI},
  year         = {2026},
  url          = {https://barkhausen.ai/notes/ai-usage-preference-standards/}
}