We Added 324,523 X Accounts Found on the Open Web. The Blue-Check Rate Fell From 19% to 5%.

Four studies ago, we described our X users index as 325,553 public profiles seeded from Wikidata, GitHub, startup founders and journalists. That number was already out of date by the time the third study published — organic crawling had carried the curated tiers to 471,437. Then, over four days in late July, the index jumped again: to 796,611. Not because we added a fifth curated seed list. Because we turned on Common Crawl as a discovery channel — finding X accounts the same way a search engine finds web pages, by reading what the open web already links to and mentions, rather than starting from a notability list.

That one channel added 324,523 accounts almost overnight. It's now the second-largest tier in the index, just behind Wikidata and just ahead of every other source combined. Which raises the obvious question: does an X account discovered by reading the open web look like an X account discovered by checking who has a Wikidata entry? It does not.

The new half looks nothing like the old half

Split the index in two — the 324,523 Common Crawl profiles against the 472,088 profiles from every other tier (Wikidata, GitHub, X search, the following graph, founders, journalists, and the rest) — and compare the same signals the rest of this dataset is built around:

Blue check — Common Crawl4.9%

Blue check — other tiers18.9%

Has bio — Common Crawl81.4%

Has bio — other tiers91.8%

Every one of those gaps points the same direction: Common Crawl found a more ordinary slice of X. Fewer of its accounts pay for verification, fewer bothered to fill in a bio, and nearly half follow more people than follow them back — the accounts-follow-strangers shape of an everyday social account, not the accounts-get-followed shape of a notable one. Only 1.6% clear 100,000 followers, a sixth of the rate in the curated tiers.

One number breaks the pattern, and it's worth sitting with: 63.6% of Common Crawl accounts still list an external link, only 13 points behind the curated tiers. That's not a coincidence of the discovery method — it's the mechanism. Common Crawl finds an X account because some archived web page links to it, and a disproportionate share of those pages are the account's own website, crediting itself. The channel is structurally biased toward accounts with a link, even though it isn't biased toward accounts with a big following.

The single largest Common Crawl account by followers is @NetflixBrasil, Netflix's Brazilian brand account, at 37.5 million followers — proof the tier isn't only small fry, just that small fry dominates it numerically.

Same shape, sharper at the bottom

The rest of this dataset shows verification climbing steadily with follower count — a paid-subscription signal, not an identity marker, that scales with reach. That gradient survives inside Common Crawl too, but it starts from a much lower floor:

At the top, the two populations converge — over half of both groups' 100K+ accounts are verified, because X's 2023 policy of granting free checks to large accounts doesn't care which seed tier found the profile. At the bottom, they diverge sharply: a sub-100-follower account is more than fifteen times as likely to be verified if it came from a curated tier (where "curated" already implies some baseline of notability or professional presence) than if it surfaced from a random web mention. The gradient is the same shape; Common Crawl just starts from further down the hill.

They're not new accounts — the open web just hadn't linked to them yet

The obvious hypothesis for a channel that finds accounts through web mentions is that it's mostly picking up recent activity — new sites, new links, new accounts. It isn't:

Created before 201578.0%

Created 2015–201916.3%

Created 2020 or later5.8%

Nearly four in five Common Crawl accounts predate 2015. These are old accounts on old web pages — a decade-old community blog, a company's "follow us" footer, a forum signature — that a search-engine-style crawl of the web's archive eventually surfaces, and that a notability-first seed list (Wikidata, GitHub, a founder database) had no reason to ever pick up. The tier isn't finding new users. It's finding old, ordinary ones the rest of the index was never built to catch.

What this shows, and what it doesn't

What we think holds up: adding a discovery channel that finds accounts through open-web mentions rather than notability seeding materially changes the composition of an X profile index — lower verification, thinner bios, fewer mega-accounts, more everyday follow behavior — without changing how old those accounts are. The reach-verification gradient this dataset already documented replicates inside the new population, just from a lower floor.

What limits it:

• Common Crawl has its own selection bias, not none. It surfaces accounts that some archived web page happened to link to or mention — which skews toward accounts with a website, a press mention, or a community presence online, and away from accounts that only ever existed inside X itself. It is a different sample of "established elsewhere on the web," not a random draw from X's user base.
• This is two snapshots four days apart, not a controlled experiment — some of the "every other tier" growth in the same window came from ordinary re-crawling of existing seeds, not just the Common Crawl addition, though it accounts for less than 0.2% of the delta.
• The 325,553 figure in our four earlier X studies was already a stale subset before this, and now describes 41% of the current index. Their specific percentages (blue-check rates, follow-limit walls, bio patterns) were measured correctly against the population they drew from; they are not automatically the population-wide truth of the index as it stands today.
• We publish aggregates only — cohort counts and percentages, not individual profile records beyond the one public account named above.

None of this makes the earlier studies wrong — it makes them dated in a specific, checkable way: they describe the curated slice, and the curated slice hasn't moved much (its own internal ratios reconcile against the pre-Common-Crawl snapshot almost to the record). What changed is that the index now also includes the accounts nobody had thought to seed for, and on every signal that matters, they answer differently.

Sources

Methodology

We queried the X users dataset live, scoping every metric to source_tier=common-crawl and comparing it against the same metric across the full index (blue-check, bio and external-link facets; follower bands; the follow/follower ratio; creation-year cumulative totals). The "every other tier" figures in this piece are the full-index total minus the Common Crawl-scoped total for each metric, not a separate query — a subtraction that reconciles exactly, since every boolean facet and follower band sums to the full index total. As a consistency check, the derived "every other tier" numbers for zero-follower accounts, sub-100-follower verification and 100K+ verification match our pre-Common-Crawl snapshot (recorded 2026-07-21) almost to the record, confirming the curated population itself barely moved in the four-day window Common Crawl was added.

Creation-year cohorts use the same cumulative created_before=YYYY-01-01 diffing as our other X studies, since a direct combined-range query is inconsistent on this live index. All figures are live as of the 2026-07-25 crawl; the full index now stands at 796,611 profiles, reflected on the X users dataset page.

Want to run your own cut? The X users dataset is queryable by source_tier directly, the how to scrape Twitter/X guide covers the calls, and pricing has the free tier. See also the four studies this one updates: the most prolific poster on X is a machine, X's real follow limit is 7,500, a random developer is 5x more likely than a notable person to have a blue check, and what X bios reveal. For what this composition shift did to X's follower inequality — it made it worse, not better — see X's attention economy is more unequal than any country's wealth. And for the one completeness signal where Common Crawl accounts actually beat the curated ones, see where X accounts say they live.