CensusBA-D-2026-042026
On 2026-07-10 the Common Crawl index for CC-MAIN-2026-25 — the June 2026 monthly crawl — was queried for 2,000 domains: the four documented frames of 500 universities, news outlets, e-commerce sites, and U.S. federal government domains from crawler-access census BA-D-2026-01. Coverage is reported two ways, because 'is a domain in Common Crawl' has two operationalizations that disagree. Captured — at least one archived record with HTTP 200 or a revisit — holds for 89.0% (1,781 of 2,000); presence — at least one index record of any status — for 95.3% (1,907). Universities are wall-to-wall (100%); news is lowest (82.8% captured, 69 fully absent). Joined to the same domains' robots.txt policy, 354 that root-block CCBot still appear in the June crawl — a timing relationship, not a compliance finding. Presence at this pipeline's front door implies nothing about a model retaining or reproducing the content.
CensusBA-D-2026-012026
On 2026-07-09 the robots.txt policy of 2,000 domains — four documented frames of 500 universities, news outlets, e-commerce sites, and U.S. federal government domains, each built from a cited public source and ordered by a public ranking (Tranco traffic rank, or reported enrollment for the U.S. universities) — was fetched and evaluated against sixteen crawler tokens. Among domains with a determinable policy, news sites root-blocked AI crawlers far more than any other sector: 66.7% (288/432) blocked at least one AI-specific token, against 7.8% (33/423) of universities. A distinct pattern recurs in news: 45.8% (198/432) root-blocked a retrieval-class crawler that supplies AI answer engines while staying open to general search. Non-response was itself a finding — the policy was undeterminable for 40.6% (203/500) of government and 37.8% (189/500) of e-commerce domains. This census enumerates the four frames completely; it makes no sampling inference to all universities, news sites, stores, or agencies.
WhitepaperBA-W-2026-022026
When a person asks an AI assistant a question — which schools to consider, which vendor to trust, which clinic to visit — the answer names some organizations and omits others. Being named is AI visibility, and it is not the same as ranking on a search results page. This introduction defines the phenomenon and explains why it must be measured as a probability rather than checked once. It distinguishes the two ways an entity becomes known to an assistant — live retrieval of sources at answer time, and parametric memory formed during training — and summarizes the public evidence that answers are unstable: rewording a question slightly, or asking it again tomorrow, changes the sources and the names returned. It introduces three metrics — Visibility Probability, Share of Voice, and Discovery Depth — and the Barkhausen Ladder, the field's maturity map, pointing to the conventions for formal definitions.