Data catalog
Crawlora maintains four web-data datasets you can query over one REST API: a 4.1M-app App Store + Google Play catalog, a Google Maps local-business dataset, and two live web studies — the Dead-Web Index and the Anti-Bot Index. Search them, pull clean JSON, pay on success — no crawl to run.
4.1M
apps indexed, plus 3 more datasets — all one REST API away.
1.2M
iOS apps
2.9M
Android apps
998K
sites scanned
Normalized JSON · pay on success · 2,000 free credits / month.
| Dataset | What it covers | Records | Updated | REST endpoint |
|---|---|---|---|---|
| App Store + Google Play apps | Every app's category, rating, ratings count, price, install scale and popularity | 4.1M apps | Continuously enriched | datasets/apps/search |
| Google Maps businesses | Scraped Google Maps business records held in a search index | Local business listings | Continuously expanded | datasets/google-map-businesses/search |
| Anti-Bot Index | Which of the world's top sites run anti-bot / WAF protection, and which vendor | 998K sites scanned | Snapshot June 14, 2026 | — |
| Dead-Web Index | How much of the most popular web is actually dead, blocked, or parked | Top global domains, probed | Probed live | — |
The app and Google Maps datasets return JSON from the REST endpoints above; the Anti-Bot and Dead-Web indices are interactive studies. Counts are the latest indexed totals and grow as the crawl runs.
A few cuts from the two most data-rich datasets — hover any chart to isolate a series. The full breakdowns live on each dataset's page.
The app catalog skews Android
Google Play is ~2.5× the App Store — 2.9M Android apps to 1.2M iOS.
2,938,575 apps
1,170,541 apps
Most apps are invisible
61% of Android apps have under 1,000 installs; only 171 have crossed a billion.
Most of the web doesn’t fight back
Of 998,497 top sites scanned for the Anti-Bot Index, 437,857 run a managed wall — but only 79,835 (8%) actively challenged a bot.
| Scanned → Reachable | 818,614 (36.3%) |
| Reachable → Has a wall | 437,857 (19.4%) |
| Reachable → No wall | 380,757 (16.9%) |
| Has a wall → Passively present | 358,022 (15.9%) |
| Scanned → Unreachable | 179,883 (8%) |
| Has a wall → Actively challenged | 79,835 (3.5%) |
Stored, structured, and deduplicated — query the index directly without a live crawl. Filter, facet, sort, and page through normalized records over the REST API.
App intelligence
4.1M apps
1.2M iOS · 2.9M Android
Every app's category, rating, ratings count, price, install scale and popularity — deduplicated to one record per app per store, across the iOS App Store and Google Play.
Key fields
titledevelopercategoryscoreratings_countprice / freeinstalls (Android)popularitycountries_available (iOS)Local business
Local business listings
search · facets · nearby · detail
Scraped Google Maps business records held in a search index — query by keyword and location, facet by category, find places near a point, or fetch one business by place_id. Reads the stored index, so there's no live scraping or proxy routing.
Key fields
namecategoryaddressratingreview_countcoordinatesplace_idTwo living indices that turn large-scale crawls into browsable, citable studies — with per-entity pages, charts, and machine-readable tables.
Web data study
998K sites scanned
Which of the world's top sites run anti-bot / WAF protection, and which vendor — from a full top-1,000,000 scan, broken down by site category and Tranco rank band.
Explore the Anti-Bot IndexWeb data study
Top global domains, probed
How much of the most popular web is actually dead, blocked, or parked — live reachability and HTTP status across the world's top domains. Search any domain for its verdict.
Explore the Dead-Web IndexEvery dataset is a documented REST endpoint. Authenticate with an x-api-keyheader, filter and sort with query params, and get normalized JSON — no HTML parsing, no proxies to manage. Queries read the stored index, and you’re billed pay on success: charged for results, not failed requests.
Search the app catalog
curl "https://api.crawlora.net/api/v1/datasets/apps/search?store=android&sort=popularity_desc&size=20" \
-H "x-api-key: $CRAWLORA_API_KEY"Swap in datasets/google-map-businesses/search for local businesses.
Crawlora runs 300+ live endpoints across search, maps, marketplaces, social, finance and more — most public web data can become a structured dataset. Browse what’s live, or tell us what you need.
Crawlora maintains four web-data datasets: an App Store + Google Play app catalog (4,109,116 apps — 1,170,541 iOS and 2,938,575 Android), a Google Maps local-business dataset, and two live web studies — the Anti-Bot Index (998,497 sites scanned) and the Dead-Web Index. The app and Maps datasets are queryable as JSON over the REST API; the two indices are interactive studies you can browse and cite.
Query them through Crawlora's REST API with an x-api-key header. `datasets/apps/search` searches the app catalog (filter by store, category, rating, price, sort by popularity) and `datasets/google-map-businesses/search` searches stored Google Maps businesses with keyword, location, facets and nearby queries. Both return normalized JSON and bill pay-on-success. See the Datasets API docs to start.
The app catalog is continuously enriched (latest snapshot June 19, 2026) and the Google Maps dataset is continuously expanded. The Anti-Bot Index is a periodic full top-1M scan (latest June 14, 2026); the Dead-Web Index probes domains live on request.
Crawlora's catalog has indexed 4,109,116 apps across both stores — 1,170,541 on the iOS App Store and 2,938,575 on Google Play — each deduplicated to one record per store with category, rating, price, install and popularity fields.
Dataset queries are billed pay-on-success on credit-based plans (you're charged for results, not failed requests), with 2,000 free credits a month and no card to start. See pricing for plan limits.
Query 4.1M apps, Google Maps businesses and the live web indices through one REST API — clean JSON, no parsing, pay on success.