Datasets API endpoint
Lists the read-only scraped datasets available, with no parameters. Each item has its `id`, name, description and the capabilities it supports: `search`, `detail`, `facets`, `nearby`, `history`, `trending` or `changes`. It reads stored data and does not start a live scrape.
/datasetsDataset id enum:
google-map-businessesgithub-usersappschrome-extensionsapps-chartsapps-reviewscreatorsairbnb-marketsCapability enum (per dataset):
searchdetailfacetsnearbyhistorytrendingchangesResponse behavior:
| Field | Filled in, of 48 checked | Values in the sample |
|---|---|---|
| id | 48 | |
| name | 48 | |
| description | 48 | median 211 characters |
| capabilities | 48 | median 3 entries |
| Parameter | Type | Required | Default | Description | Example |
|---|---|---|---|---|---|
| x-api-key (header) | string | Yes | API key required |
curl -X GET "https://api.crawlora.net/api/v1/datasets" \ -H "x-api-key: $CRAWLORA_API_KEY"
{
"code": 200,
"msg": "OK",
"data": {
"items": [
{
"id": "google-map-businesses",
"name": "Google Maps Businesses",
"capabilities": [
"search",
"detail",
"facets",
"nearby"
]
},
{
"id": "github-users",
"name": "GitHub Users",
"capabilities": [
"search",
"detail",
"facets",
"nearby"
]
},
{
"id": "apps",
"name": "Mobile Apps",
"capabilities": [
"search"
]
},
{
"id": "chrome-extensions",
"name": "Chrome Extensions",
"capabilities": [
"search",
"detail",
"facets",
"history",
"trending",
"changes"
]
},
{
"id": "apps-charts",
"name": "App Store Charts",
"capabilities": [
"search"
]
},
{
"id": "apps-reviews",
"name": "App Reviews",
"capabilities": [
"search"
]
},
{
"id": "creators",
"name": "TikTok Creators",
"capabilities": [
"search"
]
},
{
"id": "airbnb-markets",
"name": "Airbnb Markets",
"capabilities": [
"search",
"detail",
"facets",
"nearby"
]
}
]
}
}Request schema
No body schema
Response schema
#/definitions/datasets.listResponseDoc
| Field | Type | Required | Enum | Bounds | Example | Description |
|---|---|---|---|---|---|---|
| code | integer | No | 200 | |||
| data | datasets.DatasetListResponse | No | ||||
| data.items | array | No | Datasets available through the datasets endpoints. | |||
| data.items[].capabilities | array | No | Operations the dataset supports, such as `search`, `detail`, `facets` or `nearby`. | |||
| data.items[].description | string | No | Short description of what the dataset contains. | |||
| data.items[].id | string | No | Dataset identifier, for example `google-map-businesses`. | |||
| data.items[].name | string | No | Display name of the dataset. | |||
| msg | string | No | OK |
| Status | Description | Schema |
|---|---|---|
| 429 | Too Many Requests | #/definitions/app.Response |
| 500 | Internal Server Error | #/definitions/app.Response |
Use environment variables for secrets and keep Crawlora API keys server-side.
curl -X GET "https://api.crawlora.net/api/v1/datasets" \
-H "x-api-key: $CRAWLORA_API_KEY"