Reporting checklist for AI-visibility studies

A single visibility figure can be reported flawlessly and still sit inside a study a reader cannot evaluate. A study report makes many figures, compares them, and draws conclusions across them; assessing whether those conclusions follow requires more than assessing each figure in isolation. This convention is a reporting checklist for a complete AI-visibility study report — the account a reader uses to judge the study end-to-end. Its form follows the reporting-guideline tradition of the health sciences, where an itemized, section-by-section checklist became the mechanism that made study reports comparable and complete; the CONSORT statement for randomized trials is the canonical example [1], and PRISMA plays the same role for systematic reviews. Checklists of this kind are adopted because they are useful to authors and reviewers alike.

This convention is the report-level member of a three-part system, and the three members govern different units. BA-C-4 governs a single published claim — its ten disclosure items are what any one figure, quoted anywhere, must state to be evaluable. BA-C-3 governs the sampling design behind each figure — its own fourteen-item checklist is what a figure’s design must disclose to be credible. This convention governs the study report as a whole — everything a reader needs to assess the study from title to data availability, organized by the section of the report in which each requirement belongs. The three do not overlap so much as nest: a conforming report satisfies BA-C-2 and BA-C-3 for its figures and BA-C-4 for any claim excerpted from it, and this checklist adds the report-level structure that binds those figure-level and claim-level requirements into a whole a reader can follow.

The checklist therefore composes the series rather than restating it. Most of its items trace directly to a section of the metric convention (BA-C-2), the sampling convention (BA-C-3), or the disclosure convention (BA-C-4); the traceability column names that section for every item. Adopting this checklist is a way to adopt the whole series at once, at the granularity of a report.

Normative note. The key words MUST, MUST NOT, SHOULD, SHOULD NOT, and MAY carry their established normative meanings. MUST is an absolute requirement: a report that violates one is not complete under this convention. SHOULD is a strong recommendation whose departure requires a stated, defensible reason recorded in the report. MAY marks a genuinely optional element. Every item is written so that conformance can be checked from the published report alone, without access to its raw data.

1. Scope

This convention applies to any published report of a study that estimates entity visibility in AI assistants or answer engines — a research paper, a public-data census, a commissioned or vendor study, or a recurring measurement edition. It governs the report as a whole: the document a reader consults to judge, end to end, whether the study’s conclusions follow from its design and its data. It does not govern a single figure read in isolation, which is the province of BA-C-2 and BA-C-3, nor a single claim excerpted from a report, which is the province of BA-C-4. A report may be complete under this convention and still be wrong, in the way any well-documented study can be wrong; completeness of reporting is what this convention certifies, and validity is a separate judgment the reader makes from the report the checklist requires (see §4).

2. The reporting checklist

The checklist has twenty-six items grouped by the section of a report in which each belongs. The Requirement column states each as a single MUST or SHOULD; the Traces to column names the convention section the requirement derives from, so that a report satisfying the checklist is, section by section, satisfying the rest of the series. One item — the disclosure of a commercial interest, item 25 — is introduced by this convention rather than inherited from an earlier one, because no figure-level or claim-level convention has a natural home for it. A report claims conformance by confirming each applicable item; items that do not apply to a given study (for example, the change-claim item for a purely cross-sectional report) are marked not applicable rather than omitted silently.

The availability item settles a recurring misunderstanding. A complete report is not a report that publishes everything; it is a report that publishes the aggregate estimates and the definitions needed to interpret them. Publishing raw answer transcripts, the exact query phrasings, or collection tooling is neither required for completeness nor a substitute for it, mirroring the floor drawn in BA-C-4 §5: the disclosures the checklist demands are counts, definitions, and conditions, not the operational layer beneath them. A study that cannot state its counts and definitions has a measurement problem, not a disclosure problem.

3. How to claim conformance

A report claims conformance with this convention when every applicable MUST is satisfied and every applicable SHOULD is either met or accompanied by a stated reason. The citation form is: reported in conformance with BA-C-5 v1.0. Because the checklist composes the series, a conforming report will ordinarily also state conformance with BA-C-2 and BA-C-3 for its figures and note that any excerpted claim meets BA-C-4; a report that claims BA-C-5 conformance while its figures do not meet BA-C-2 and BA-C-3 is mislabeled, since the figure-level requirements are reached through this checklist’s own items.

Partial conformance MUST be stated as named exceptions, never implied by silence — for example, conformant with BA-C-5 v1.0 except items 12 and 18 (clustering handling and multiplicity control not applied). This mirrors the exception form of BA-C-2 §7 and keeps a partial claim honest: a reader learns exactly which items were not met rather than inferring completeness from an unqualified badge. Reviewers, editors, and procurement evaluators MAY cite non-conformance neutrally — the report does not meet the reporting checklist of BA-C-5 — which records that the report, as published, cannot be fully assessed, and makes no claim about whether its findings are correct.

4. Limitations

A checklist certifies completeness of reporting, not validity of findings. Every item in §2 can be satisfied by a report whose underlying study is nonetheless flawed — too few phrasings, an unrepresentative query set, an engine window chosen after the fact. Completeness is what makes those flaws visible and arguable; it does not remove them. The value of the checklist is that it moves the argument from “the study did not say” to “the study said, and here is why the design is or is not sound,” which is the argument worth having. A conforming report is an assessable report, not a validated one.

The checklist also cannot prevent false disclosure. A stated sample size, window, or engine version is a concrete claim a challenger can attempt to reproduce, so the checklist makes fabrication checkable in principle; it cannot make it impossible, and no reporting convention can. Conformance is a claim by the author, subject to the same scrutiny as any other claim in the report.

Finally, the checklist reflects the answer engines as they are currently built — retrieval and generation conditioned on phrasing, region, personalization, channel, and a perishable engine version. Its items are load-bearing because those features are load-bearing now. If engine architecture changes such that new features become decisive, or existing ones cease to matter, the set of items will change with a new version of this convention rather than through silent reinterpretation. The report that a complete study must produce is defined against the systems of its time, and this convention is versioned for that reason.

@techreport{BA-C-5,
  author       = {{Barkhausen AI}},
  title        = {Reporting checklist for AI-visibility studies},
  institution  = {Barkhausen AI},
  year         = {2026},
  url          = {https://barkhausen.ai/conventions/reporting-checklist/}
}