A note on method
Most reports on "languages of the internet" come from W3Techs, which detects the language of the homepage of each of the top 10 million websites. That's a useful view but it counts a multilingual site like apple.com or microsoft.com as a single English site, even when much of their content is multilingual.
This dashboard uses Common Crawl instead: the language of every single page across roughly three billion pages per crawl. It's the same data the major large language models (GPT, Claude, Gemini) are pre-trained on. For localization strategy and AI search optimization, pages are what matter. That's what AI reads, indexes and answers from.
Side note: The two methodologies don't fully agree. W3Techs reports English at ~49 % and a sharp decline since 2020; Common Crawl puts English at ~43 % and roughly stable. The difference is what gets counted as "the web." I've chosen the page-level view because it's the one your audience actually reads and your AI tools actually train on.
Chapter I · Time series
Annual share of pages crawled by Common Crawl, by content language, 2019 to 2026. Toggle a language to add or remove it from the chart.
Y-axis on a logarithmic scale so the 0.06–46 % range reads on one chart. Source: Common Crawl monthly archive statistics (CC-MAIN-YYYY-WW). Earliest crawl of each year. Each crawl covers ~3 billion pages; language is detected per page via Compact Language Detector 2 (CLD2).
Chapter II · The Top 29
The 29 most-published content languages on the open web, ranked by their 2026 share. The original 25 plus four large-speaker languages — Hindi, Bengali, Thai and Malay — added in June 2026 to surface the most under-read languages in AI training data. Click a column to sort; type to filter.
| # | Language | 2019 | 2020 | 2021 | 2022 | 2023 | 2024 | 2025 | 2026 | Δ 2019→26 | Trend |
|---|
Sparklines show each language's 2019–2026 trajectory at a common vertical scale (0%–50%). Source: Common Crawl.
Chapter III · The mismatch
For every language, I compare its share of web pages (Common Crawl, 2026) with its share of the world's speakers according to Ethnologue, L1 (first language) + L2 (second language). Bars to the right of zero indicate languages overrepresented online; bars to the left, underrepresented.
Note: If we consider only speakers for whom English is a first language, the figure drops from 1,493M to 372M (Ethnologue, 2026). In that case, English’s overrepresentation would be even more pronounced.
Speaker share = (L1 + L2 speakers, Ethnologue 29th edition) ÷ 8.2 billion world population. Chinese, Hindi, Arabic and Bengali are highlighted; all four are vastly underrepresented online relative to the number of people who speak them.
Chapter IV · Side by side
A direct comparison. The gap column is web % minus speaker %. Negative values mark languages whose web presence trails their speaker base.
| Language | Speakers (M, L1+L2) | Speaker share of world | Web share, 2026 | Gap (web − speaker) | Status |
|---|
Chapter V · The AEO index
This is a single ratio: a language's share of web pages divided by its share of the world's speakers. Below 1.0, AI under-reads the language relative to its speaker base; the localization and AEO opportunity is largest. Above 1.0, the language is over-represented in the training data; quality is high and the marginal payoff to adding more content is lower.
Read the bars on a log scale. Hindi and Bengali sit at ≈0.03 (AI sees them at 3 % of their speaker share). Arabic at 0.16 is severely under-read; Czech at 8.9 means AI sees Czech at almost nine times its speaker share. The dashed line at 1.0 marks parity. Languages with an index of 4 or higher reflect very prolific publishing cultures relative to their population: the European bloc, plus Japanese.
In summary