Skip to content
Barkhausen AI

DataBA-D-2026-02

University international-readiness census 2026 — raw-HTML homepage signals

500 universities — the 300 largest U.S. institutions by enrollment (College Scorecard) plus 200 international universities by Tranco traffic rank (Hipolabs) — one homepage each. Of the 500, 429 returned an analyzable homepage (HTTP 200 with a non-empty body) and carry the readiness signals; 71 did not (33 non-200 responses, 38 transport-level failures) and carry identity fields only.

Collection window: 2026-07-09 · License: CC-BY-4.0 · Published with BA-D-2026-02: International-readiness of university homepages: a raw-HTML census of hreflang, canonical, and language signals

Download

CC-BY-4.0 · cite as a dataset with the parent report.

Schema

ColumnTypeDescription
domainstringThe registrable domain from the sample frame (input to the fetch), lowercased. One row per domain.
countrystringCountry code assigned by the sample frame: US for the enrollment-ordered U.S. portion; the frame-assigned code for the international portion. May be empty where the frame source supplied none.
analyzedbooleantrue when the homepage returned HTTP 200 with a non-empty body and was analyzed for readiness signals; false when a non-200 status was returned; empty when the fetch failed at the transport layer (DNS/TLS/timeout). The 429 analyzed=true rows are the denominator for every figure in BA-D-2026-02.
http_statusintegerHTTP status of the final response after redirects. Empty for transport-level failures where no response was received.
hreflang_countintegerNumber of <link rel="alternate" hreflang> elements found in the raw HTML. 0 on analyzed rows with no hreflang; empty on non-analyzed rows.
hreflang_valid_countintegerNumber of the counted hreflang alternates whose language code passes the BCP-47-lite check (see limitations for the check's known false negatives on script subtags).
hreflang_invalid_codesstringList of hreflang codes that failed the BCP-47-lite check. Empty when none — which is the case for every row in this release (0 invalid codes among the 68 adopters).
hreflang_has_x_defaultbooleantrue if any alternate declares hreflang="x-default" (the fallback for unmatched locales).
hreflang_has_self_referencebooleantrue if any alternate href, resolved against the final fetched URL and normalized, equals that final URL (the page listing itself among its alternates).
canonical_presentbooleantrue if a <link rel="canonical"> element is present in the raw HTML.
canonical_self_pointingbooleantrue if the first canonical's href, resolved and normalized, equals the final fetched URL. false with canonical_present=true means the canonical points elsewhere (a different path on the same host, or cross-domain).
canonical_multiplebooleantrue if more than one canonical link is present — an ambiguity defect a crawler must resolve on its own.
html_langstringThe value of the <html lang> attribute. Empty if the attribute is absent (html-lang presence is therefore derivable as html_lang non-empty).
html_lang_validbooleantrue if html_lang passes the BCP-47-lite check. Read as "valid under the lite check", not "valid under full BCP 47".
detected_langstringTop language code returned by langdetect on the page's visible text (fixed seed for determinism). Empty if detection was skipped (see insufficient_text).
detected_lang_probnumberlangdetect's probability for detected_lang, from 0 to 1. Empty if detection was skipped.
lang_mismatch_flagbooleantrue only when the text is sufficient, an html lang is present, detection probability is at least 0.9, and the html-lang primary subtag differs from the detected primary subtag. Deliberately conservative — high-confidence candidates rather than a full count; close-language pairs can produce false flags.
noindexbooleantrue if a <meta name="robots"> content value or an X-Robots-Tag response header contains "noindex" (case-insensitive) — instructing a compliant crawler not to index the page. Captured even on some non-200 responses via the header.
site_movedbooleantrue if the final served host differs from the input domain. Treats an apex-to-www change or any subdomain change as "moved" — an identity note about which host answered, not a defect.
insufficient_textbooleantrue if fewer than 200 characters of visible text were extracted, in which case language detection was skipped.
redirect_chain_lengthintegerNumber of redirect hops followed (0 = the first request returned the final response; maximum 5 hops).
final_hoststringThe host that served the final response after redirects. Empty for transport-level failures.
fetched_atstringISO 8601 UTC timestamp of the fetch.

Known limitations

FAQ

Does no hreflang mean the university has no international pages?
No. This dataset records only what each homepage declared in its raw, server-delivered HTML. Many institutions run internationalized inner pages, language microsites, or country landing pages that the homepage does not reference with hreflang, and hreflang injected by client-side JavaScript is not captured. A blank means "no hreflang alternate in the homepage's raw HTML", not "no international content anywhere".
Why does a university show a canonical mismatch (canonical_present true, canonical_self_pointing false)?
Because the homepage's canonical link resolves to a different URL than the homepage itself. This is a single homepage snapshot: a canonical can legitimately point to a different path (for example /home) or, less desirably, to another domain entirely. Both same-host-different-path and cross-domain canonicals occur in this release. The dataset flags the condition and lets you inspect the row; it does not infer intent.
What do analyzed = false and an empty analyzed field mean?
analyzed=false means the domain returned a non-200 status (often a 403 bot-challenge) so no page-body signals were computed. An empty analyzed field means the fetch failed at the transport layer (DNS, TLS, timeout, or other connection failure) and no response was received. Either way the row carries no hreflang, canonical, or language signals and is excluded from the readiness denominators.
Why is a code marked invalid (or html_lang_valid false) when it looks correct, e.g. zh-Hant?
The validity screen is a conservative BCP-47-lite check that rejects script subtags (zh-Hant, zh-Hans, sr-Latn) and extended or variant tags by design, so it over-reports rather than under-reports structural problems. Treat "valid" as "valid under the lite check", not "valid under full BCP 47". In this release no hreflang code was marked invalid.

Changelog

How to cite

Barkhausen AI (2026). University international-readiness census 2026 — raw-HTML homepage signals (dataset). https://barkhausen.ai/data/university-intl-readiness-2026/

BibTeX
@techreport{BA-D-2026-02,
  author       = {{Barkhausen AI}},
  title        = {University international-readiness census 2026 — raw-HTML homepage signals (dataset)},
  institution  = {Barkhausen AI},
  year         = {2026},
  url          = {https://barkhausen.ai/data/university-intl-readiness-2026/}
}

Published under the Creative Commons Attribution 4.0 International (CC-BY-4.0).