Note2026
A 2026 study measured how much an AI recommendation set changes when the same underlying request is reworded, rather than assuming it stays fixed. Across roughly 12,000 runs on two production models — one from OpenAI, one from Anthropic — it compared a same-prompt rerun baseline against two kinds of paraphrase. Rerunning an identical prompt reproduced the recommendation set with a Jaccard similarity of 0.50–0.61 within an engine. A cosmetic reword, preserving the need, cut the corpus-mean overlap to 0.288, 21–32 percentage points below that baseline; a reword that added a constraint cut it to 0.135, 37–48 points below. Increasing reasoning effort did not remove the gap. The authors argue that progress requires different measurement units, not merely more prompt sampling. This note reports what the study measured and what its single-day design does and does not support.
Data insightBA-DI-32026
A site is said to have adopted llms.txt when it publishes a /llms.txt file. But 'has one' can be operationalized two ways, and the 2026 crawler-access census measured both: present, an HTTP 200 at /llms.txt, and valid, a content check that the 200 is markdown-like rather than an HTML page returned for any path. The two disagree widely. Counting presence, e-commerce adoption is 18.2% and government 15.2%; counting validity, the same sectors read 8.8% and 1.8% — a gap that reaches better than eight-to-one for government, where 76 domains answer /llms.txt with a 200 but only 9 return something that resembles the file. The census already reported these counts; this note isolates what the gap is about. Neither number is wrong, and the choice between them is a measurement decision, not a fact about the world. An 'adoption rate' is interpretable only when it says which rule it counted — the same disclosure discipline the minimum-disclosure convention requires of any visibility claim.
Method noteBA-MN-52026
A claim that improving a website's accessibility improves its visibility in search results or AI answers arrives in three forms, each answered by a different question. A direct-factor claim holds that an accessibility property is itself a ranking input; the test is whether the engine's own documentation says so, property by property and surface by surface — and in the clearest page-ranking cases it says the opposite. A correlational claim reports that accessible sites rank or are cited more often; the test is n, window, controls, and causal direction, against a web where detectable accessibility defects are near-universal. A mechanism claim proposes a pathway from page structure to answer inclusion; the test is whether each link is documented, and two load-bearing links are verified absences in the sampled record. None of the three, as usually stated, licenses a quantified expectation. A closing checklist collects the disclosures a reader should require.
Note2026
A 2026 study sampled four AI answer engines every day for about six weeks to measure how much their answers move on their own. Running identical prompts across ChatGPT, Gemini, Google AI Mode, and Perplexity for 45–46 collection days between 2026-01-24 and 2026-03-20, over four Swiss-German commercial verticals from a Swiss vantage, it found that answers on consecutive days shared only 34–42% of their cited sources and — across the three verticals that cleared a brand-detection threshold — 45–59% of their mentioned brands. Simultaneous same-day reruns overlapped by 32–43% in sources, so much of the turnover is request-to-request stochasticity rather than genuine day-over-day movement. Within-24-hour source stability differed sharply by engine, from 0.233 to 0.505. This note reports what the study measured — instability magnitudes per engine and vertical for that window — and what its single-window, single-region design does not support.
Data insightBA-DI-22026
Crawl-delay is a non-standard robots.txt field that asks a crawler to wait between requests. It is not part of RFC 9309, and the operators of several major crawlers treat it differently: Google's documentation lists the fields it supports and states that crawl-delay is not among them; Bing documents that its crawler honors it; and Yandex documents that it stopped taking the directive into account in 2018, directing operators to a crawl-rate control instead. Against that mixed and partly negative support, the 2026 crawler-access census finds crawl-delay written into 228 of 1,381 parsed robots files — 16.5%, spread across all four sectors measured. This note pairs the support documentation with the census count to make one observation: a directive's presence in robots.txt is a separate fact from whether any crawler acts on it, and the two need not track each other. That gap is a base-rate reminder for the newer fields now appearing in robots.txt, whose eventual honoring their presence today does not establish.
Note2026
The Content Signals Policy adds a line to robots.txt that expresses preferences by purpose rather than by crawler: three named signals (search, ai-input, ai-train) each carry a yes or no, stating whether content may be used for search, model input, or training. This is a different axis from the per-token crawler groups the taxonomy (BA-C-6) classifies: one names purposes and reaches every crawler, the other names crawlers and is silent on purpose. This note reads the published policy, states who authored it and its relation to the IETF effort it points to, and places it on robots.txt's enforcement boundary: a request and, in the policy's framing, a reservation of rights, not an access control. A re-analysis of this publication's census corpus adds a first field measurement: a Content-Signal line on 3.0% of parsed domains, most carrying a CDN's managed default that also deploys a fourth key beyond the announced three.
Data insightBA-DI-12026
A robots.txt group begins with a User-agent line, and the value on that line is a product token — a name matched, case-insensitively, against a crawler's own token. RFC 9309 fixes the characters a product token may contain: letters, hyphens, and underscores, and nothing else. A value carrying a comma is therefore not a product token, and a conformant parser matches no crawler to the group, so whatever Disallow rules follow never take effect. This note demonstrates the mechanism with the census's pinned parser on a constructed two-token line and a single-token control, then reports what the census corpus actually holds: ten news-sector robots files carry a comma-bearing User-agent value, none of them a crawler token — they are whole browser user-agent strings, a quoted bot description, and offline-downloader names with bracket flags — and not one file in the corpus joins two recognized crawler tokens with a comma. The finding is small and mechanical, but it separates a rule that is written from a rule that runs.
Method noteBA-MN-42026
Two statistical phenomena manufacture visibility success stories that no intervention produced. The first is base rates: for most entity–need combinations the true probability of appearing in an AI answer sits near zero, so a portfolio-wide scan across many monitored combinations will surface some appearances by chance alone — it samples the lucky tail rather than measuring an effect. The second is regression to the mean: engagements tend to begin just after an unusually poor measurement, and because repeated AI-search measurements are noisy, the next measurement improves on its own, with no intervention. This note works both mechanisms with explicit arithmetic — including a hypothetical portfolio in which chance alone yields roughly four spurious appearances — and states what defeats them: pre-registered cells, interval estimates rather than point comparisons, and the sustainment condition of the Barkhausen Criterion.
Note2026
The top of the Barkhausen Ladder (BL-8) describes an entity that exposes a machine-actionable interface an agent can invoke directly, rather than one an agent reaches by parsing a rendered page. This note maps, at concept level, the emerging protocols by which a site can expose such an interface — the Model Context Protocol, Microsoft's NLWeb, and OpenAPI-described endpoints of the kind OpenAI's GPT Actions consume — and marks the boundary that separates them from the page-parsing agents described elsewhere in this publication and from robots.txt, which is a crawler-access mechanism rather than an agent protocol. Every characterization is drawn from the projects' own documentation. The note takes no position on which protocol will prevail and reports that no public adoption statistics exist to quantify any of them.
Method noteBA-MN-22026
A visibility percentage is often reported as though it were a fixed property of an AI engine, but the engines are non-stationary: identical prompts submitted on consecutive days return substantially different answers, overlapping by only 34–42% in the sources they cite and, on the study's brand-qualifying verticals, 45–59% in the brands they mention. A percentage measured over one span of dates therefore estimates a quantity that exists only for that span. This note argues that the time window is part of a visibility figure's identity, not metadata attached to it: a percentage reported without its window names no fixed quantity, and two figures drawn from undisclosed windows cannot be compared — the comparison is undefined, not merely imprecise. It sets out how to choose a window (short enough for approximate stationarity, long enough for the repetition a precision target requires), a house notation for stating one, and the perishability disclosure every figure should carry.
Method noteBA-MN-32026
Published case studies of visibility gains — an entity's mentions in AI answers rising over some period after an intervention — are drawn from a survivor sample. The cases that regressed or went nowhere are not written up, so a file of success stories carries almost no information about whether the intervention works. Two mechanisms produce winners regardless of any real effect: a portfolio of noisy visibility series will always contain some that rose by chance, and any entity engaged just after an unusually poor measurement will tend to improve on the next one by regression to the mean alone. This note frames the case study as a selection problem, separates it from evidence, and states what would count as evidence instead: pre-registered cohorts, all-entity aggregates, or at minimum a disclosed denominator — of how many tracked entities is this the best case?
Note2026
The accessibility tree is the structured representation a browser builds from a web page's markup. As defined by the WAI-ARIA 1.2 specification, it is a tree of accessible objects, each node exposing an element's role, states, and properties through the platform accessibility API; Chromium's engineering documentation describes its shape as derived from the Document Object Model (DOM) and modifiable through ARIA attributes. This note reads what platform and vendor documentation says consumes that representation: assistive technology, the Playwright browser-testing framework, and browser-automation and agent systems. The documentation shows a split — some named agent systems are documented as operating on the accessibility tree or ARIA semantics, others as working from screenshots. What the tree carries is structural and semantic, not the pixel-level appearance a screenshot captures, and that distinction maps onto how differently these systems are documented to perceive a page.
Note2026
In 2024, a controlled experiment introduced the term Generative Engine Optimization and tested, rather than assumed, which content changes shift how often a source is cited in an AI-generated answer. Working from a 10,000-query benchmark and a simulated generative engine, the study found that adding quotations, statistics, or cited sources measurably increased a source's contribution to the answer, that rewriting text into a more authoritative tone did not, and that traditional keyword stuffing performed worse than making no change at all. A further test found that when all competing sources apply the same techniques, lower-ranked sources gain the most. This note reports what the study found and what its design does and does not support.
Note2026
Operators publishing a robots.txt file often intend to make one decision — keep this site out of AI training data — and instead make several, because AI-related crawlers are not one thing. The same operator, and often the same AI company, runs separate crawlers for training-data collection, for building a retrieval index, and for fetching a page in real time when a user asks about it, and each is controlled by a distinct token. Disallowing the training crawler does not disallow the others, and a rule written broadly enough to catch more than one class can remove a site from AI-generated answers as an unintended side effect. This note reads the public documentation for named crawler classes and states plainly what robots.txt is: a request, not an enforcement mechanism.
Method noteBA-MN-12026
A claim such as "62% of AI answers mention this brand" is only as trustworthy as the sampling behind it, and most published versions of this claim omit the information needed to judge it. This note sets out the minimum arithmetic a reader needs: what a sample size (n) contributes to a proportion's precision, how the standard error of a proportion behaves as n grows, why proportions near 0% or 100% require a different interval than the standard formula, and which disclosures — engine, version, time window, and query construction — a claim must carry before it is evaluable at all. It closes with a five-item checklist for interrogating any AI-visibility percentage encountered outside a peer-reviewed setting.
