Skip to content
Barkhausen AI

Topic

Conventions & policy

NoteBA-DI-32026

Two ways to count an llms.txt file, two adoption rates

A site is said to have adopted llms.txt when it publishes a /llms.txt file. But 'has one' can be operationalized two ways, and the 2026 crawler-access census measured both: present, an HTTP 200 at /llms.txt, and valid, a content check that the 200 is markdown-like rather than an HTML page returned for any path. The two disagree widely. Counting presence, e-commerce adoption is 18.2% and government 15.2%; counting validity, the same sectors read 8.8% and 1.8% — a gap that reaches better than eight-to-one for government, where 76 domains answer /llms.txt with a 200 but only 9 return something that resembles the file. The census already reported these counts; this note isolates what the gap is about. Neither number is wrong, and the choice between them is a measurement decision, not a fact about the world. An 'adoption rate' is interpretable only when it says which rule it counted — the same disclosure discipline the minimum-disclosure convention requires of any visibility claim.

Method noteBA-MN-52026

How to read an 'accessibility improves rankings' claim

A claim that improving a website's accessibility improves its visibility in search results or AI answers arrives in three forms, each answered by a different question. A direct-factor claim holds that an accessibility property is itself a ranking input; the test is whether the engine's own documentation says so, property by property and surface by surface — and in the clearest page-ranking cases it says the opposite. A correlational claim reports that accessible sites rank or are cited more often; the test is n, window, controls, and causal direction, against a web where detectable accessibility defects are near-universal. A mechanism claim proposes a pathway from page structure to answer inclusion; the test is whether each link is documented, and two load-bearing links are verified absences in the sampled record. None of the three, as usually stated, licenses a quantified expectation. A closing checklist collects the disclosures a reader should require.

Note2026

Content Signals in robots.txt: what the proposal says and what it can enforce

The Content Signals Policy adds a line to robots.txt that expresses preferences by purpose rather than by crawler: three named signals (search, ai-input, ai-train) each carry a yes or no, stating whether content may be used for search, model input, or training. This is a different axis from the per-token crawler groups the taxonomy (BA-C-6) classifies: one names purposes and reaches every crawler, the other names crawlers and is silent on purpose. This note reads the published policy, states who authored it and its relation to the IETF effort it points to, and places it on robots.txt's enforcement boundary: a request and, in the policy's framing, a reservation of rights, not an access control. A re-analysis of this publication's census corpus adds a first field measurement: a Content-Signal line on 3.0% of parsed domains, most carrying a CDN's managed default that also deploys a fourth key beyond the announced three.

Note2026

Protocols for agent access: the landscape at concept level

The top of the Barkhausen Ladder (BL-8) describes an entity that exposes a machine-actionable interface an agent can invoke directly, rather than one an agent reaches by parsing a rendered page. This note maps, at concept level, the emerging protocols by which a site can expose such an interface — the Model Context Protocol, Microsoft's NLWeb, and OpenAPI-described endpoints of the kind OpenAI's GPT Actions consume — and marks the boundary that separates them from the page-parsing agents described elsewhere in this publication and from robots.txt, which is a crawler-access mechanism rather than an agent protocol. Every characterization is drawn from the projects' own documentation. The note takes no position on which protocol will prevail and reports that no public adoption statistics exist to quantify any of them.

WhitepaperBA-W-2026-042026

The measurement gap in AI-visibility tooling

Commercial AI-visibility tools have converged on a recognizable form: a daily score per tracked query, a trend line of those scores, a cited-or-not verdict per topic, and per-engine coverage badges. This paper argues the genre, as a class, cannot meet the disclosure floor a measurement requires. Walked against the ten minimum-disclosure items of BA-C-4, the common form omits the load-bearing ones: a daily single run is one Bernoulli draw per query, so the score carries no usable sample size; a line of bare points hides the day-to-day drift that is its own variance; one frozen query per topic measures the wording, not the need; API collection cannot support a claim about what users see; refusals and engine-wide updates go unmarked. The paper states what a conforming tool would show instead, and closes with five questions a buyer can ask. No tool is named or assessed.

ConventionBA-C-72026

Terminology for AI-visibility measurement

Measurements of AI visibility are comparable only when the words that name them are stable: two studies that both report "visibility" compare nothing if one counts entity mentions and the other counts source links. This convention fixes the field-level vocabulary the rest of the series relies on and that no other convention owns — AI visibility, the phenomenon; answer engine and AI assistant, the systems; generative engine optimization, the practice; retrieval visibility versus parametric, closed-book visibility, two causal paths; mention and citation, two distinct observables; information need, query, and phrasing, three levels of a question; and recommendation set. It states why one canonical term per concept is a precondition for comparability, gives usage rules including first-use linking and a deprecated-synonym table, and defines conformance as using the canonical terms or explicitly mapping one's own. It claims authority over usage within this series only, not over the field's language.

ConventionBA-C-82026

A query-intent taxonomy for visibility measurement

Answer engines respond differently to different kinds of question, and an entity's visibility on one kind does not carry to another. This convention classifies the query space a credible visibility study must cover into five intent classes — navigational, informational, comparative, constraint-based, and recommendation-seeking — defined by the goal a user's information need expresses, and states what a study must disclose about its coverage. Each class behaves differently under measurement: navigational visibility is near-degenerate, informational mentions are incidental, comparative needs are Share of Voice's natural home, constraint-based needs are where Discovery Depth applies, and recommendation-seeking needs yield the least stable and most commercially consequential recommendation sets. A study must disclose which classes it covers and in what proportion, must not generalize a class-specific result across classes, and should report per-class estimates where classes are mixed — all at the level of classes, never the concrete queries.

Note2026

Reading the accessibility tree: what platform documentation says software sees

The accessibility tree is the structured representation a browser builds from a web page's markup. As defined by the WAI-ARIA 1.2 specification, it is a tree of accessible objects, each node exposing an element's role, states, and properties through the platform accessibility API; Chromium's engineering documentation describes its shape as derived from the Document Object Model (DOM) and modifiable through ARIA attributes. This note reads what platform and vendor documentation says consumes that representation: assistive technology, the Playwright browser-testing framework, and browser-automation and agent systems. The documentation shows a split — some named agent systems are documented as operating on the accessibility tree or ARIA semantics, others as working from screenshots. What the tree carries is structural and semantic, not the pixel-level appearance a screenshot captures, and that distinction maps onto how differently these systems are documented to perceive a page.

WhitepaperBA-W-2026-032026

Machine readers of the web: how search optimization, generative engine optimization, and accessibility relate

A web page is read by four kinds of machines: search crawlers, the retrieval pipelines behind AI answer engines, assistive technology, and autonomous browser agents. Three practices — search engine optimization, generative engine optimization (GEO), and web accessibility — each optimize for one or more of these readers. Working from primary documentation, specifications, regulations, peer-reviewed studies, and, for one 2022 statement lacking an official transcript, a flagged trade-press transcription (sampled 2026-07-09), this paper states each practice as a reader–objective–evaluator tuple, maps twelve page-level signals against the four readers, and audits the circulating claim that accessibility improves AI visibility. Five signals have documented consumers in multiple reader categories. But both load-bearing links of the proposed accessibility-to-GEO mechanism are verified documentary absences, Google states accessibility is not a direct ranking factor, and agent systems split between accessibility-tree and screenshot perception. It closes with five falsifiable propositions that no public study yet measures.

ConventionBA-C-52026

Reporting checklist for AI-visibility studies

AI-visibility findings are published as full study reports — a research paper, a public-data census, a vendor white paper — and a reader must be able to assess the whole study, not only its individual figures. This convention is a reporting checklist, in the tradition of the CONSORT and PRISMA guidelines, that an AI-visibility study report satisfies to claim methodological completeness. It organizes twenty-six items by report section: title and abstract, introduction, methods (queries, engines and channels, sampling, detection, analysis), results, limitations, and availability. Each item is a MUST or SHOULD requirement traced to the convention it derives from. The checklist composes rather than replaces the series — figure-level metric reporting (BA-C-2), sampling-design disclosure (BA-C-3), and single-claim disclosure (BA-C-4) — and requires no raw data, query lists, or tooling. Conformance is claimed per report, with partial conformance stated as named exceptions.

ConventionBA-C-42026

Minimum disclosure requirements for AI-visibility claims

Claims about visibility in AI assistants circulate widely — in dashboards, case studies, vendor material, and research summaries — and most cannot be evaluated, because they omit the facts a reader needs to judge them. This convention states the minimum any published AI-visibility claim must disclose: the entity and information need; the point estimate with a confidence interval and interval method; the sample size; the engine with interface and version; the measurement channel; the time window; the region; the phrasing representation; refusal handling; and the mention-detection method. A claim that omits an applicable item is not a measurement under this convention. Claims that visibility changed must additionally satisfy the Barkhausen Criterion defined in BA-C-3. The requirements are deliberately minimal: they demand no raw data, no query lists, and no tooling disclosure, so that any measurer — commercial or academic — can comply.

ConventionBA-C-62026

A taxonomy of AI-related crawlers

"AI crawler" names a function, not a piece of software, and it conflates several. A single site is visited by crawlers that collect content for model training, that build retrieval indexes AI answers draw from, that fetch one page in real time to answer a specific prompt, that act autonomously on a user's behalf, and that maintain traditional search indexes AI features also consume. Each function has different consequences for a site, and a robots.txt rule aimed at one silently binds or misses the others. This convention defines five functional classes — training, retrieval, user-fetch, agent, and search — from vendor documentation and public observation, states how a published crawler registry or census must classify by function rather than operator, and sets out how robots.txt semantics interact with each class. It records where the scheme's boundaries are unstable, including the agent class, which user-agent strings systematically undercount.

Note2026

The GEO experiment: what a controlled study showed about content and AI answers

In 2024, a controlled experiment introduced the term Generative Engine Optimization and tested, rather than assumed, which content changes shift how often a source is cited in an AI-generated answer. Working from a 10,000-query benchmark and a simulated generative engine, the study found that adding quotations, statistics, or cited sources measurably increased a source's contribution to the answer, that rewriting text into a more authoritative tone did not, and that traditional keyword stuffing performed worse than making no change at all. A further test found that when all competing sources apply the same techniques, lower-ranked sources gain the most. This note reports what the study found and what its design does and does not support.

WhitepaperBA-W-2026-012026

Measuring AI visibility: statistical requirements and common failures

A brand's visibility in AI assistants is routinely 'verified' with a single screenshot or one daily query. This paper argues such verification is not measurement. Answer engines are stochastic and their retrieval changes continuously: lightly rewording a query while holding its intent fixed cut the overlap of the brands an assistant recommended to a Jaccard similarity near 0.3 — far below the 0.50–0.61 overlap of a plain re-run — and an identical prompt re-issued a day later overlapped only 34–42% in cited sources and 45–59% in mentioned brands. A single observation of a moving distribution estimates nothing. The paper enumerates what voids a visibility claim — no sample size, no interval, no window, no engine version, one fixed phrasing, uncontrolled personalization, discarded refusals — and shows a three-sigma jump can be pure drift. It then states what valid measurement requires — repeated sampling to a declared precision, bounded intervals near the extremes, a phrasing distribution, partial pooling, explicit windows and versions, change-point monitoring, recorded refusals — specified in BA-C-2 and BA-C-3.

ConventionBA-C-22026

Visibility metrics

This convention defines the metrics a credible measurement of entity visibility in AI assistants must report, and the form each report must take. It specifies three primary metrics: Visibility Probability (VP), the probability that an entity is mentioned in an answer to a query drawn from a distribution of real user phrasings, on a given engine, time window, and region; Share of Voice (SoV), the entity's share of all brand mentions in the same answers; and Discovery Depth (DD), the degree of query constraint at which an entity first enters the recommendation set. It adds supporting measures for prominence, citation support, rank-aware list overlap, and sentiment; requires every estimate to carry a confidence interval, sample size, engine version, window, region, and a stated detection method; mandates boundary-valid intervals near zero and one and composition-aware intervals for SoV; and states what claiming conformance requires. Sampling procedure is deferred to BA-C-3.

ConventionBA-C-32026

Sampling-protocol requirements

AI assistants return different answers to the same question from one request to the next, so any measurement of how visible a brand is inside those answers is a statistical estimate, not a reading. This convention states the requirements a sampling design must meet for its estimates to be credible. It summarizes the public evidence that a single observation is uninformative and derives the sample size a stated confidence interval requires. It requires interval methods that stay valid near zero and one, representation of each query as a distribution over real phrasings, partial pooling, observations spread across the stated window, engine-version and measurement-channel disclosure, controlled region and personalization, change-point monitoring for silent engine updates, multiplicity disclosure, and refusals recorded as availability observations. It formalizes the Barkhausen Criterion — the conditions under which a claimed visibility change counts as real — and closes with a reporting checklist.

ConventionBA-C-12026

The Barkhausen Ladder

The Barkhausen Ladder is a nine-level maturity model (BL-0 through BL-8) describing an entity's readiness to be found, cited, and acted upon by AI assistants and answer engines. Each level is defined by three things: what the level means, the observable criteria that place an entity at it, and why it matters. The levels run from basic search hygiene (BL-0) through rendering and crawler access, structured data, content form, distribution, measurement, algorithmic optimization, and corpus presence to agent-readiness (BL-8). The Ladder is diagnostic, not prescriptive: it states conditions an assessment can verify, not techniques to apply. Levels are cumulative, and an entity is placed at the highest level whose criteria, and all lower levels', it satisfies. This convention uses normative keywords (MUST, SHOULD, MAY), cross-references the metric and sampling conventions (BA-C-2, BA-C-3) that define how visibility itself is measured, and closes with an assessment checklist for each level.