Substack API endpoint
Use Crawlora's Substack Explore Feed API to extract supported public Substack data as structured JSON. This page includes request parameters, cURL examples, response schema, error behavior, credit cost, and a Playground link for testing before integration.
/substack/exploreReturns one page of Substack's Explore surface. Explore is a feed of heterogeneous modules rather than a flat list, so each page returns the modules it contains and every module names its own type. Known module types are `trendingTopicsExplore` (trending topics and suggested searches), `trendingPostsExplore`, `newBestsellersExplore`, `videoSuggestionsExplore`, `categoryLeaderboard`, `comment` (a single Note) and `post` (a single post). Only the collections a module actually carries are populated. Pagination is cursor-based; early pages are curated shelves and later pages are individual Notes and posts. Developers commonly use this endpoint for data enrichment, monitoring, research dashboards, internal automation, and agent-native workflows that need repeatable structured public web data. Authentication uses the documented Crawlora headers, and usage is metered with the credit cost shown on this page.
Request parameters are generated from the active endpoint catalog. Required values must be sent before Crawlora can call the upstream public web data source.
| Parameter | Type | Required | Default | Description | Example |
|---|---|---|---|---|---|
| tab | string | No | Explore tab id from the tabs endpoint with surface=explore | ||
| type | string | No | Tab type Allowed values: base, secondary, category | ||
| cursor | string | No | Opaque cursor from a previous response's next_cursor | ||
| x-api-key (header) | string | Yes | API key required |
curl -X GET "https://api.crawlora.net/api/v1/substack/explore?type=base" \ -H "x-api-key: $CRAWLORA_API_KEY"
Send your scraping API key in the x-api-key header. Use the console API Keys page to rotate or select the active key.
Endpoint usage is metered in credits. The plan prices, included credits, limits, and overage rates below match the active backend billing configuration.
| Plan | Price | Included credits | Daily cap | Rate limit | Overage |
|---|---|---|---|---|---|
| Free | $0/mo | 2,000 | 500 daily credits | 5/min | No overage |
| Starter | $9/mo | 20,000 | 5,000 daily credits | 15/min | $0.75/1,000 overage credits when enabled |
| Growth | $29/mo | 100,000 | 25,000 daily credits | 45/min | $0.45/1,000 overage credits when enabled |
| Pro | $79/mo | 400,000 | No daily cap | 120/min | $0.30/1,000 overage credits |
| Business | $199/mo | 1,200,000 | No daily cap | 300/min | $0.20/1,000 overage credits |
| Enterprise | $499/mo | 5,000,000 | No daily cap | 1,000/min | $0.12/1,000 overage credits |
This endpoint is executed through Crawlora's managed scraping infrastructure.
Crawlora does not silently return bad data when the upstream page cannot be used.
| Status | Common failure case |
|---|---|
| 400 | Invalid input or missing required parameter |
| 429 | Plan or endpoint rate limit exceeded |
| 500 | Internal execution error |
| 502 | Upstream platform failed, returned unusable HTML, or served a challenge page that could not be resolved |
When possible, Crawlora returns structured error context so your integration can retry, back off, or inspect the request.
| Status | Description | Schema |
|---|---|---|
| 400 | Bad Request | #/definitions/app.Response |
| 429 | Too Many Requests | #/definitions/app.Response |
| 503 | Service Unavailable | #/definitions/app.Response |
{
"code": 200,
"data": {
"tab": "explore",
"type": "base",
"next_cursor": "1",
"has_more": true,
"modules": [
{
"type": "trendingTopicsExplore",
"topics": [
{
"query": "Trump campaign rhetoric 2026",
"description": "Coverage of campaign messaging",
"source": "search_history",
"alternate_queries": [
"david"
],
"featured": true
}
]
}
]
},
"msg": "success"
}Request schema
No body schema
Response schema
#/definitions/substack.exploreResponseDoc
| Field | Type | Required | Enum | Bounds | Example | Description |
|---|---|---|---|---|---|---|
| code | integer | No | 200 | |||
| data | substack.ExploreResult | No | ||||
| data.has_more | boolean | No | ||||
| data.modules | array | No | ||||
| data.modules[].category_id | string | No | ||||
| data.modules[].category_name | string | No | ||||
| data.modules[].notes | array | No | ||||
| data.modules[].notes[].attached_post | substack.NotePost | No | ||||
| data.modules[].notes[].attached_post.id | integer | No | ||||
| data.modules[].notes[].attached_post.title | string | No | ||||
| data.modules[].notes[].attached_post.url | string | No | ||||
| data.modules[].notes[].author_bestseller_tier | integer | No | ||||
| data.modules[].notes[].author_handle | string | No | jameelajamil | |||
| data.modules[].notes[].author_id | integer | No | 138271663 | |||
| data.modules[].notes[].author_name | string | No | Jameela Jamil | |||
| data.modules[].notes[].author_photo_url | string | No | ||||
| data.modules[].notes[].author_url | string | No | ||||
| data.modules[].notes[].body | string | No | ||||
| data.modules[].notes[].edited_at | string | No | ||||
| data.modules[].notes[].id | integer | No | 316240438 | |||
| data.modules[].notes[].is_ai_generated | boolean | No | ||||
| data.modules[].notes[].language | string | No | ||||
| data.modules[].notes[].published_at | string | No | ||||
| data.modules[].notes[].reaction_count | integer | No | 12846 | |||
| data.modules[].notes[].reactions | object | No | ||||
| data.modules[].notes[].reply_count | integer | No | 129 | |||
| data.modules[].notes[].restack_count | integer | No | 1139 | |||
| data.modules[].notes[].url | string | No | ||||
| data.modules[].posts | array | No | ||||
| data.modules[].posts[].audience | string | No | everyone | |||
| data.modules[].posts[].comment_count | integer | No | ||||
| data.modules[].posts[].cover_image_url | string | No | ||||
| data.modules[].posts[].description | string | No | ||||
| data.modules[].posts[].id | integer | No | 213613469 | |||
| data.modules[].posts[].paywalled | boolean | No | ||||
| data.modules[].posts[].published_at | string | No | ||||
| data.modules[].posts[].reaction_count | integer | No | ||||
| data.modules[].posts[].slug | string | No | ||||
| data.modules[].posts[].subtitle | string | No | ||||
| data.modules[].posts[].title | string | No | ||||
| data.modules[].posts[].type | string | No | newsletter | |||
| data.modules[].posts[].url | string | No | ||||
| data.modules[].profiles | array | No | ||||
| data.modules[].profiles[].bestseller_tier | integer | No | 10000 | |||
| data.modules[].profiles[].bio | string | No | ||||
| data.modules[].profiles[].handle | string | No | pragmaticengineer | |||
| data.modules[].profiles[].id | integer | No | 2067309 | |||
| data.modules[].profiles[].name | string | No | Gergely Orosz | |||
| data.modules[].profiles[].photo_url | string | No | ||||
| data.modules[].profiles[].profile_set_up_at | string | No | ||||
| data.modules[].profiles[].publications | array | No | ||||
| data.modules[].profiles[].publications[].custom_domain | string | No | ||||
| data.modules[].profiles[].publications[].id | integer | No | 458709 | |||
| data.modules[].profiles[].publications[].is_primary | boolean | No | ||||
| data.modules[].profiles[].publications[].name | string | No | The Pragmatic Engineer | |||
| data.modules[].profiles[].publications[].role | string | No | admin | |||
| data.modules[].profiles[].publications[].subdomain | string | No | ||||
| data.modules[].profiles[].publications[].url | string | No | ||||
| data.modules[].profiles[].slug | string | No | ||||
| data.modules[].profiles[].subscribers | substack.SubscriberCounts | No | ||||
| data.modules[].profiles[].subscribers.follower_count | integer | No | 365727 | |||
| data.modules[].profiles[].subscribers.paid_rough | string | No | Thousands of paid subscribers | PaidRough and PaidRoughInt are the only paid-subscriber figures Substack publishes, and both are buckets rather than counts. PaidRoughInt is the finer of the two and is always present; PaidTier is the coarser threshold behind the public bestseller badge. | ||
| data.modules[].profiles[].subscribers.paid_rough_int | integer | No | 1000 | |||
| data.modules[].profiles[].subscribers.paid_tier | integer | No | 1000 | |||
| data.modules[].profiles[].subscribers.total | integer | No | 315000 | |||
| data.modules[].profiles[].subscribers.total_display | string | No | 315K+ | |||
| data.modules[].profiles[].subscribers.total_hidden | boolean | No | ||||
| data.modules[].profiles[].subscribers.total_rough | string | No | Hundreds of thousands of subscribers | |||
| data.modules[].profiles[].subscriptions | array | No | ||||
| data.modules[].profiles[].subscriptions[].is_founding | boolean | No | ||||
| data.modules[].profiles[].subscriptions[].membership_state | string | No | free_signup | |||
| data.modules[].profiles[].subscriptions[].publication_id | integer | No | ||||
| data.modules[].profiles[].subscriptions[].publication_name | string | No | ||||
| data.modules[].profiles[].subscriptions[].subdomain | string | No | ||||
| data.modules[].profiles[].url | string | No | https://substack.com/@pragmaticengineer | |||
| data.modules[].publications | array | No | ||||
| data.modules[].publications[].author_handle | string | No | ||||
| data.modules[].publications[].author_id | integer | No | ||||
| data.modules[].publications[].author_name | string | No | Dylan Patel | |||
| data.modules[].publications[].custom_domain | string | No | ||||
| data.modules[].publications[].explicit | boolean | No | ||||
| data.modules[].publications[].hero_text | string | No | ||||
| data.modules[].publications[].id | integer | No | 6349492 | |||
| data.modules[].publications[].language | string | No | ||||
| data.modules[].publications[].logo_url | string | No | ||||
| data.modules[].publications[].name | string | No | SemiAnalysis | |||
| data.modules[].publications[].payments_enabled | boolean | No | ||||
| data.modules[].publications[].subdomain | string | No | semianalysis | |||
| data.modules[].publications[].subscribers | substack.SubscriberCounts | No | ||||
| data.modules[].publications[].subscribers.follower_count | integer | No | 365727 | |||
| data.modules[].publications[].subscribers.paid_rough | string | No | Thousands of paid subscribers | PaidRough and PaidRoughInt are the only paid-subscriber figures Substack publishes, and both are buckets rather than counts. PaidRoughInt is the finer of the two and is always present; PaidTier is the coarser threshold behind the public bestseller badge. | ||
| data.modules[].publications[].subscribers.paid_rough_int | integer | No | 1000 | |||
| data.modules[].publications[].subscribers.paid_tier | integer | No | 1000 | |||
| data.modules[].publications[].subscribers.total | integer | No | 315000 | |||
| data.modules[].publications[].subscribers.total_display | string | No | 315K+ | |||
| data.modules[].publications[].subscribers.total_hidden | boolean | No | ||||
| data.modules[].publications[].subscribers.total_rough | string | No | Hundreds of thousands of subscribers | |||
| data.modules[].publications[].subscription | substack.Subscription | No | ||||
| data.modules[].publications[].subscription.annual_disabled | boolean | No | ||||
| data.modules[].publications[].subscription.app_store_plans | array | No | ||||
| data.modules[].publications[].subscription.app_store_plans[].active | boolean | No | ||||
| data.modules[].publications[].subscription.app_store_plans[].amount_cents | integer | No | 1400 | |||
| data.modules[].publications[].subscription.app_store_plans[].currency | string | No | usd | |||
| data.modules[].publications[].subscription.app_store_plans[].display_name | string | No | ||||
| data.modules[].publications[].subscription.app_store_plans[].display_price | string | No | $14 | |||
| data.modules[].publications[].subscription.app_store_plans[].is_founding | boolean | No | ||||
| data.modules[].publications[].subscription.app_store_plans[].period | string | No | month | |||
| data.modules[].publications[].subscription.app_store_plans[].sku | string | No | ||||
| data.modules[].publications[].subscription.founding_benefits | array | No | ||||
| data.modules[].publications[].subscription.founding_plan_name | string | No | Founding Member | |||
| data.modules[].publications[].subscription.free_benefits | array | No | ||||
| data.modules[].publications[].subscription.free_trial_enabled | boolean | No | ||||
| data.modules[].publications[].subscription.group_discount_percent | number | No | 20 | |||
| data.modules[].publications[].subscription.monthly_disabled | boolean | No | ||||
| data.modules[].publications[].subscription.paid_benefits | array | No | ||||
| data.modules[].publications[].subscription.payments_state | string | No | enabled | |||
| data.modules[].publications[].subscription.plans | array | No | ||||
| data.modules[].publications[].subscription.plans[].active | boolean | No | ||||
| data.modules[].publications[].subscription.plans[].amount_cents | integer | No | 5000 | |||
| data.modules[].publications[].subscription.plans[].currencies | array | No | ||||
| data.modules[].publications[].subscription.plans[].currencies[].amount_cents | integer | No | 900 | |||
| data.modules[].publications[].subscription.plans[].currencies[].currency | string | No | eur | |||
| data.modules[].publications[].subscription.plans[].currency | string | No | usd | |||
| data.modules[].publications[].subscription.plans[].id | string | No | monthly50usd_982ddb86 | |||
| data.modules[].publications[].subscription.plans[].interval | string | No | month | |||
| data.modules[].publications[].subscription.plans[].interval_count | integer | No | 1 | |||
| data.modules[].publications[].subscription.plans[].kind | string | No | monthly | |||
| data.modules[].publications[].subscription.plans[].nickname | string | No | $50 a month | |||
| data.modules[].publications[].subscription.plans[].trial_period_days | integer | No | ||||
| data.modules[].publications[].subscription.pledges_enabled | boolean | No | ||||
| data.modules[].publications[].url | string | No | https://semianalysis.substack.com | |||
| data.modules[].title | string | No | Trending posts | |||
| data.modules[].topics | array | No | ||||
| data.modules[].topics[].alternate_queries | array | No | ||||
| data.modules[].topics[].category | string | No | ||||
| data.modules[].topics[].description | string | No | ||||
| data.modules[].topics[].featured | boolean | No | ||||
| data.modules[].topics[].first_trending_at | string | No | ||||
| data.modules[].topics[].id | string | No | ||||
| data.modules[].topics[].query | string | No | Trump campaign rhetoric 2026 | |||
| data.modules[].topics[].source | string | No | search_history | |||
| data.modules[].topics[].url | string | No | ||||
| data.modules[].type | string | No | trendingPostsExplore | Type is the upstream module kind. Known values: trendingTopicsExplore, trendingPostsExplore, newBestsellersExplore, videoSuggestionsExplore, categoryLeaderboard, comment, post. | ||
| data.next_cursor | string | No | ||||
| data.tab | string | No | explore | |||
| data.type | string | No | base | |||
| msg | string | No | success |
Use environment variables for secrets and keep Crawlora API keys server-side.
curl -X GET "https://api.crawlora.net/api/v1/substack/explore?type=base" \
-H "x-api-key: $CRAWLORA_API_KEY"Crawlora is designed for responsible structured public web data workflows. Customers are responsible for using Crawlora in compliance with applicable laws, third-party rights, target-platform rules, and Crawlora terms.
Read Crawlora terms