Tony Wang15 Min. LesezeitWe Fingerprinted the Top 1,000,000 Websites. The Top 1,000 Run a Completely Different Stack.
Split our 1,000,000-site tech-stack census by Tranco rank: bot management and React frameworks concentrate at the top, WordPress dominates the tail.
A million-domain census answers "what does the internet run on" the same way a national census answers "what does the average citizen earn" — technically true, and hiding almost everything interesting. Split the same 1,000,000-site fingerprint by how much traffic each domain actually gets, and two pairs of numbers move in opposite directions at once: the technologies you'd associate with well-funded, professionally engineered sites concentrate hard at the top, while the technologies you'd associate with the self-serve, managed-platform web climb the further down the list you go.
The stack flips as you go down the list
Top 1,000
% of reachable sites in the band
Top 1,000
At the very top of the web, bot management (26.3%) and React meta-frameworks (16.4%) are both far more common than anywhere else on the list — and Cloudflare's free-tier CDN (20.8%) is the least common of the four technologies in this group.
1K–10K
One order of magnitude down the list, bot management (21.0%) and React (14.1%) have both already started to thin out, while WordPress (12.4%) and Cloudflare's free CDN (32.8%) have both climbed past their top-1,000 levels.
10K–100K
By 10,000–100,000, WordPress (17.0%) has overtaken React (11.1%) as the more common of the two, and Cloudflare's free CDN now fronts more than a third of the whole band (38.0%).
100K–1M
In the long tail, bot management has fallen to 9.8% — a third of its top-1,000 level — while WordPress (19.8%) and Cloudflare's free CDN (42.2%) are both at or near their peak across the whole list.
| Tranco rank band | Bot management | React frameworks | WordPress | Cloudflare CDN |
|---|---|---|---|---|
| Top 1,000 | 26.3% | 16.4% | 10.6% | 20.8% |
| 1K–10K | 21.0% | 14.1% | 12.4% | 32.8% |
| 10K–100K | 15.6% | 11.1% | 17.0% | 38.0% |
| 100K–1M | 9.8% | 6.0% | 19.8% | 42.2% |
Two things fall together and two things rise together, and it isn't a coincidence. Enterprise bot management (DataDome, PerimeterX, Akamai Bot Manager, Kasada, or Cloudflare's own paid tier) and modern React meta-frameworks (Next.js and its siblings) are both, in practice, decisions that take engineering budget and traffic that justifies the spend — a site fighting off scrapers at scale is usually also a site that can afford a dedicated frontend team. WordPress and Cloudflare's free-tier CDN are the opposite: the default choice when a site owner wants something that works without a platform team behind it, which describes the overwhelming majority of the million-domain tail far more than it describes the top 1,000.
Ten more categories, and the sharpest gradient isn't bot management
The four technologies above aren't the only ones sensitive to rank — they're just the most dramatic example most people would guess. Run the same rank-band cut across ten more technology categories and one gradient dwarfs every number already in this piece: ecommerce software runs on just 1.0% of the top 1,000 sites, climbing to 9.7% of the 100K–1M tail — a 9.7× swing, more than three times sharper than bot management's own fall (2.7×) or WordPress's own climb (1.9×). A detectable programming-language signature (PHP, ASP.NET, and similar server-language fingerprints) follows the same shape: 2.8% → 15.9%, a 5.7× climb.
| Category | Top 1,000 | 1K–10K | 10K–100K | 100K–1M |
|---|---|---|---|---|
| JavaScript framework2.3× top-heavy | 23.7% | 20.6% | 16.6% | 10.4% |
| Web framework1.6× top-heavy | 22.4% | 20.6% | 19.8% | 13.9% |
| Security1.5× top-heavy | 36.2% | 32.7% | 29.7% | 24.4% |
| CDNflat | 63.3% | 65% | 65.2% | 61.4% |
| Tag managerpeaks mid-list | 32.8% | 42.5% | 44.3% | 37.9% |
| Advertisingpeaks mid-list | 14.8% | 20.3% | 20.5% | 18.1% |
| SEO1.6× tail-heavy | 6% | 6% | 8.4% | 9.5% |
| Analytics1.4× tail-heavy | 30.8% | 40.6% | 45.5% | 43.5% |
| Programming language5.7× tail-heavy | 2.8% | 6.6% | 11.4% | 15.9% |
| Ecommerce9.7× tail-heavy | 1% | 1.3% | 4.1% | 9.7% |
Read the shape, not just the headline number. CDN is nearly flat (63.3% → 61.4%) — a reminder that not every technology tracks rank; fronting a site through some CDN at all is close to universal regardless of size. JavaScript framework (a broader category than the React-specific meta-frameworks above) falls even more sharply — 23.7% → 10.4%, a 2.3× drop — sharpening rather than contradicting the earlier finding. And the two steepest category-level climbs in the census, ecommerce (9.7×) and programming-language exposure (5.7×), tell a coherent story together: independent online stores and simpler, less-abstracted backends are both disproportionately long-tail phenomena, while the top of the list skews toward organizations large enough to have moved past both.
The sharpest gradient in the whole piece is one specific vendor, not a category
That category-level cut isn't the final word. Narrow "web server" down to one specific vendor — LiteSpeed — and the climb gets steeper than anything else in this piece, category-level cuts included: 0.5% of the top 1,000 run LiteSpeed, climbing to 5.5% of the 100K–1M tail — a 10.5× swing, edging out even ecommerce's 9.7×.
| Tranco rank band | LiteSpeed |
|---|---|
| Top 1,000 | 0.5% |
| 1K–10K | 1.2% |
| 10K–100K | 2.9% |
| 100K–1M | 5.5% |
LiteSpeed isn't a general-purpose web server choice the way Nginx or Apache are — it's the default bundled into budget cPanel shared-hosting plans, and its own caching plugin is one of the most widely installed WordPress performance add-ons. Its rank-band curve reads less like a technology preference and more like a hosting-tier signal: the same budget-hosting economics that put WordPress ahead in the long tail put its most common companion web server ahead even further, and sharper. For contrast, Nginx — the general-purpose default nearly every kind of site reaches for — stays close to flat across the same four bands (15.2% → 18.5%, a 1.2× spread), confirming LiteSpeed's climb is a real vendor-specific signal, not an artifact of web-server detection generally getting easier further down the list.
An independent confirmation, measured a completely different way
Strip Cloudflare's own bot-management product out of the picture — leaving only the specialist enterprise vendors (DataDome, Akamai Bot Manager, Kasada, PerimeterX) — and this static-HTML fingerprint pass lines up with a dataset built for an entirely different purpose: Crawlora's own Anti-Bot Adoption Index, which measures active challenge behavior (does a probe actually get blocked or challenged) rather than reading a script tag off the homepage.
| Tranco rank band | Tech Stack dataset | Anti-Bot Index |
|---|---|---|
| Top 1,000 | 9.7% | 7.6% |
| 1K–10K | 5.7% | 5.3% |
| 10K–100K | 2.9% | 2.6% |
| 100K–1M | 0.7% | 0.6% |
Both datasets fall roughly 13–14× from the top of the list to the tail, and every band lands within 1–2 points of the other despite being scanned at different times, with different detector code, by different methods (a passive script/header read here, an active probe there). That's the same cross-validation trick as our CAPTCHA cross-check in the parent post — two independent measurements agreeing is much stronger evidence than either one alone that "enterprise bot management is a head phenomenon" is real, not a detector artifact.
The biggest sites are actually the leanest
If you assumed the biggest sites on the internet run the most third-party tooling — the most trackers, the most widgets, the most vendor sprawl — the data says the opposite. The top 1,000 average 6.91 detected technologies per homepage; that climbs to a peak of 7.95 in the 10K–100K band before easing back slightly to 7.80 in the 100K–1M tail — still well above the top of the list either way. A plausible read: the very top of the web includes a disproportionate share of infrastructure itself (search engines, CDNs, cloud consoles, other companies' own APIs) that deliberately runs minimal third-party code, while a mid-tier commercial site loaded with every marketing tool its team has ever purchased is a far more typical shape for "large audience, unremarkable engineering discipline."
A measurable blind spot: the detector sees less of .cn
Before the number: this section measures what our detector can see on a .cn domain, not how sophisticated or modern the underlying sites are. Our signature database was built primarily against Western CDN, analytics, and CMS vendors — a .cn site returning zero detected technology very likely still runs a real, working stack, just one built on Baidu/Alibaba/Tencent-ecosystem infrastructure this detector doesn't yet recognize. Read what follows as a coverage gap in our tooling, not a claim about the Chinese web.
With that said, the gap is real and worth stating plainly. Reachable .cn domains average 3.56 detected technologies per homepage — less than half the 7.84 average everywhere else in the census, a 2.2× gap.
Push to the extreme end and the gap sharpens further: 5.09% of reachable .cn sites return literally zero detected technology, vs. just 0.54% everywhere else — a 9.4× gap.
.cn domains are also somewhat less likely to resolve a working homepage at all — 70.0% reachable (6,914 of 9,873) vs. 84.9% for the rest of the census — though that gap is far smaller than the detection-coverage gap once a site does load. This exact refresh added 10 new Chinese-market technology signatures specifically to narrow that gap, and it's narrowing — treat the current numbers as a coverage floor, not a fixed property of the Chinese web. Full caveats and what this finding does and doesn't support: METHODOLOGY.md in the open dataset.
What BuiltWith charges $295 a month for
BuiltWith's "Market Analysis" product segments technology adoption by exactly this cut — top 10K, top 100K, top 1M — and gates it behind a paid plan starting at $295/month (Pro: $495, Team: $995). The segmentation above is the same idea, computed from Crawlora's own open Tranco census instead of BuiltWith's proprietary crawl, and published here for free. If you need the underlying per-domain records rather than the aggregate curve, the full dataset is one REST API call away.
Limits, stated plainly
The top-1,000 band has the smallest sample (n=763 reachable), so read its exact percentages as directional, not decimal-precise — a handful of domains moving between "has a detector match" and "doesn't" swings a percentage point at that scale far more than it would in the 764,146-domain tail. This is also correlation, not causation: using Next.js doesn't cause a site to get more traffic, and running WordPress doesn't cause it to get less. The far more likely direction of causality runs the other way — well-funded organizations independently choose both a modern engineering stack and already operate at a scale that draws automated traffic worth defending, while a small or resource-constrained site reaches for whichever platform bundles hosting, security, and content management into one product it doesn't have to assemble itself. And as with the parent census, this is a homepage-only, HTTP-tier fingerprint — it sees what a plain request to / returns, not what loads behind a login or on a checkout page.
Query the full Tech Stack dataset
1,000,000 fingerprinted domains — CDN, CMS, ecommerce, analytics, web server and more — over one REST API, ranked by Tranco traffic rank.
Häufig gestellte Fragen
Do the biggest websites run more third-party technology than smaller ones?
No - the opposite. In Crawlora's 1,000,000-site tech-stack census, the top 1,000 sites by Tranco rank average 6.91 detected technologies per homepage, while the 100K-1M tail averages 7.80 (peaking at 7.95 in the 10K-100K band). The largest sites tend to be leaner, not more instrumented; a lot of the very top of the web is infrastructure (search engines, clouds, other companies' APIs) that runs minimal third-party code by design.
Does enterprise bot management concentrate at high-traffic sites?
Yes, sharply. Enterprise bot-management vendors (Cloudflare Bot Management, PerimeterX, DataDome, Akamai Bot Manager, Kasada) protect 26.3% of the top 1,000 sites by Tranco rank, falling to 9.8% of the 100K-1M tail - a 2.7x gradient. Excluding Cloudflare specifically, this independently matches Crawlora's separately-run Anti-Bot Adoption Index (a different, challenge-response-based detection method): 9.7% to 0.7% here vs. 7.6% to 0.6% there.
Does WordPress usage vary by site traffic rank?
Yes, in the opposite direction from bot management. WordPress runs 10.6% of the top 1,000 Tranco-ranked sites, climbing to 19.8% of the 100K-1M tail - a 1.9x increase. Cloudflare's own (free-tier) CDN follows the same rising pattern, 20.8% to 42.2%, even as Cloudflare's own paid bot-management product falls with rank.
Do React frameworks like Next.js concentrate at high-traffic sites?
Yes. React meta-frameworks (Next.js, Nuxt.js, Astro, Gatsby, SvelteKit, Remix) run 16.4% of the top 1,000 Tranco-ranked sites, falling to 6.0% of the 100K-1M tail - a 2.7x gradient, the same direction as enterprise bot management and the opposite direction from WordPress.
How is technology adoption segmented by website traffic rank normally measured?
BuiltWith's paid "Market Analysis" product segments technology adoption by top-10K/100K/1M Tranco-style rank bands, starting at $295/month. Crawlora computes the same kind of segmentation from its own open Tranco top-1M tech-stack census and publishes it for free.
Which technology category shows the sharpest gradient by site rank?
Not bot management or React frameworks - ecommerce. Ecommerce software runs on just 1.0% of the top 1,000 Tranco-ranked sites, climbing to 9.7% of the 100K-1M tail, a 9.7x swing - more than three times sharper than bot management's own 2.7x fall. A detectable programming-language signature (PHP, ASP.NET, and similar server-language fingerprints) shows a similar 5.7x climb, 2.8% to 15.9%. Independent online stores and simpler, less-abstracted backends are both disproportionately long-tail phenomena, while CDN usage (63.3% to 61.4%) stays nearly flat across every rank band.
Does this census miss technology on Chinese (.cn) websites?
Yes, measurably - and it's a detector-coverage limitation, not a claim about the sites themselves. Reachable .cn domains average 3.56 detected technologies vs. 7.84 for the rest of the census (a 2.2x gap), and return zero detected technology at 5.09% vs. 0.54% elsewhere (9.4x). Our signature database was built primarily against Western CDN/analytics/CMS vendors, so a .cn site showing no detected technology very likely still runs a real stack - just one built on Baidu/Alibaba/Tencent-ecosystem infrastructure this detector doesn't yet recognize. This exact refresh added 10 new Chinese-market signatures specifically to narrow that gap.