What does AI actually read? [Interactive Dashboard]

What does AI actually read? Mostly the open web. And Common Crawl, the page-by-page archive that sits at the foundation of GPT, LLaMA, Gemini and other major language models, looks nothing like the world that speaks.

Hindi: 610 million speakers, 0.22% of pages; three percent of the rate it is spoken. Bengali sits at the same ratio. English is 41% of pages, falling slowly. The training data that shapes how machines understand language is wildly uneven.

Eight years of snapshots, 29 languages, read page by page, and a new AEO index showing exactly where AI under-reads the world.