Datasets API endpoint
Use Crawlora's Search the SEC companies dataset API to search or inspect stored structured datasets as JSON. This page includes request parameters, cURL examples, response schema, validation behavior, credit cost, and a Playground link for testing before integration. Dataset endpoints read indexed records and do not apply proxy routing.
/datasets/sec-companies/searchSearches SEC-reporting companies stored in a search index — normalized filing history, financial-statement rollups (latest annual/quarterly revenue, net income, total assets) and trailing-90-day insider (Form 3/4/5) activity. Sort enum: `relevance`, `name_asc`, `revenue_desc`, `net_income_desc`, `filing_recent_desc`, `insider_activity_desc`. `entity_type`, `sic`, `sic_description`, `exchange`, and `state_of_incorporation` are open filters over the exact values EDGAR reports for each filer (not a fixed enum) — discover real values via the matching facet. Developers commonly use this endpoint for repeatable dataset search, filtering, facets, local business enrichment, analytics, exports, and internal tools that need structured records beyond the limited manual refinement available in the Google Maps app. Authentication uses the x-api-key header, usage is metered with the credit cost shown on this page, and the request does not trigger live scraping or proxy routing.
Request parameters are generated from the active endpoint catalog. Dataset parameters filter, page, facet, or locate stored structured records; they do not configure a live scraper or proxy path.
| Parameter | Type | Required | Default | Description | Example |
|---|---|---|---|---|---|
| q | string | No | Full-text query over the company name, or an exact ticker match, max 256 characters | ||
| ticker | string | No | Exact ticker filter (case-insensitive), e.g. AAPL, max 32 characters | ||
| cik | string | No | Exact CIK filter, numeric or zero-padded, e.g. 320193 or 0000320193 | ||
| sic | string | No | Exact SIC industry-code filter, e.g. 3571, max 32 characters | ||
| sic_description | string | No | Exact SIC description filter, e.g. Electronic Computers, max 128 characters | ||
| exchange | string | No | Exact exchange filter as reported by EDGAR, e.g. Nasdaq, NYSE, max 64 characters | ||
| state_of_incorporation | string | No | Exact state/country-of-incorporation filter as reported by EDGAR, e.g. DE, CA, max 32 characters | ||
| entity_type | string | No | Exact entity-type filter as reported by EDGAR (e.g. operating), max 64 characters | ||
| reporting_currency | string | No | Exact reporting-currency filter, ISO-4217 code, e.g. USD, JPY, EUR | ||
| has_financials | boolean | No | When true, keep only companies that have XBRL financial statements | ||
| min_revenue | number | No | Minimum latest-annual revenue in USD (normalized from the filer's reporting currency at reference rates), 0 or greater | ||
| max_revenue | number | No | Maximum latest-annual revenue in USD (normalized), 0 or greater | ||
| min_net_income | number | No | Minimum latest-annual net income in USD (normalized; negative allowed) | ||
| min_total_assets | number | No | Minimum latest-annual total assets in USD (normalized), 0 or greater | ||
| form_filed | string | No | Exact form-type filter; keeps only companies that have ever filed this form, e.g. 10-K, 8-K | ||
| min_insider_txn_count_90d | integer | No | Minimum insider (Form 3/4/5) transaction count in the trailing 90 days, 0 or greater | ||
| sort | string | No | Sort enum: relevance, name_asc, revenue_desc, net_income_desc, filing_recent_desc, insider_activity_desc Allowed values: relevance, name_asc, revenue_desc, net_income_desc, filing_recent_desc, insider_activity_desc | ||
| page | integer | No | 1 | Page number, defaults to 1 | |
| page_size | integer | No | 20 and maxes at 100 | Page size, defaults to 20 and maxes at 100; page * page_size must be <= 10000 | |
| x-api-key (header) | string | Yes | API key required |
curl -X GET "https://api.crawlora.net/api/v1/datasets/sec-companies/search?q=coffee&has_financials=true&sort=relevance&page=1" \ -H "x-api-key: $CRAWLORA_API_KEY"
Send your scraping API key in the x-api-key header. Use the console API Keys page to rotate or select the active key.
Endpoint usage is metered in credits. The plan prices, included credits, limits, and overage rates below match the active backend billing configuration.
| Plan | Price | Included credits | Daily cap | Rate limit | Overage |
|---|---|---|---|---|---|
| Free | $0/mo | 2,000 | 500 daily credits | 5/min | No overage |
| Starter | $9/mo | 20,000 | 5,000 daily credits | 15/min | $0.75/1,000 overage credits when enabled |
| Growth | $29/mo | 100,000 | 25,000 daily credits | 45/min | $0.45/1,000 overage credits when enabled |
| Pro | $79/mo | 400,000 | No daily cap | 120/min | $0.30/1,000 overage credits |
| Business | $199/mo | 1,200,000 | No daily cap | 300/min | $0.20/1,000 overage credits |
| Enterprise | $499/mo | 5,000,000 | No daily cap | 1,000/min | $0.12/1,000 overage credits |
This endpoint reads stored indexed dataset records. It does not execute a live upstream Google Maps request, browser session, or proxy-routed scraping job.
- Each item carries `cik`, `cik_padded`, `name`, `former_names`, `tickers`, `primary_ticker`, `exchange`, `sic`, `sic_description`, `state_of_incorporation`, `fiscal_year_end`, `entity_type`, `filing_count`, `latest_filing_date`, `forms_filed`, `recent_filings` (capped detail list), `reporting_currency`, the `latest_annual_*`/`latest_quarterly_*` financial rollups (each with a native `_revenue`/`_net_income`/`_total_assets` and a `_usd` normalized counterpart), `revenue_band`, and the `insider_txn_count_90d`/`insider_net_shares_90d`/`insider_last_transaction_at` trailing-90-day rollup. - Currency: financial figures are stored in each filer's native `reporting_currency` exactly as filed; the `*_usd` fields (and `revenue_band`, and the `revenue_desc`/`net_income_desc` sorts and `min_*`/`max_revenue` filters) are that value converted at fixed reference rates so they compare across currencies. Read the native figure for exactness; screen and sort on the USD fields. - Reference rates: the `*_usd` conversion uses a single fixed rate per currency, **as of 2026-07-17**, refreshed about annually — not the exchange rate on each filing's own reporting date. So a `*_usd` figure is accurate enough to sort, band and range-filter across currencies, but it is not a restatement: for a non-USD filer's exact dollar value, take the native figure and apply your own as-of rate. Conversion is applied only to currencies we hold a rate for; a filer reporting in any other currency keeps its native figures and simply has no `*_usd` value (so it sorts last rather than being given a guessed one). - The maximum result window is `10000`; `page * page_size` must not exceed `10000`. - Invalid enum values return the standard invalid params envelope. - Returns an empty `items` array (not an error) when nothing matches or the dataset has no data yet. - Does not trigger live scraping — for a single company's current data right now, use `GET /sec/company/submissions` instead. Example response: ```json { "code": 200, "msg": "OK", "data": { "dataset": "sec-companies", "items": [ { "cik": 320193, "cik_padded": "0000320193", "name": "Apple Inc.", "tickers": ["AAPL"], "primary_ticker": "AAPL", "exchange": ["Nasdaq"], "sic": "3571", "sic_description": "Electronic Computers", "entity_type": "operating", "filing_count": 342, "latest_filing_date": "2025-10-31", "forms_filed": ["10-K", "10-Q", "8-K", "4"], "latest_annual_fiscal_year": 2024, "latest_annual_revenue": 391035000000, "latest_annual_net_income": 93736000000, "revenue_band": "over_10b", "insider_txn_count_90d": 6 } ], "page": 1, "page_size": 20, "total": 1, "sort": "relevance" } } ```
Crawlora does not silently return invalid dataset search results when filters, pagination, coordinates, or stored record lookups cannot be satisfied.
| Status | Common failure case |
|---|---|
| 400 | Invalid input, missing required parameter, invalid enum, bad coordinate pair, or result window beyond the dataset limit |
| 404 | Requested stored dataset item is not present |
| 429 | Plan or endpoint rate limit exceeded |
| 500 | Internal dataset query or storage error |
When possible, Crawlora returns structured error context so your integration can adjust filters, page size, location inputs, or lookup identifiers.
| Status | Description | Schema |
|---|---|---|
| 400 | Bad Request | #/definitions/app.Response |
| 429 | Too Many Requests | #/definitions/app.Response |
| 500 | Internal Server Error | #/definitions/app.Response |
{
"code": 200,
"msg": "OK",
"data": {
"dataset": "sec-companies",
"items": [
{
"cik": 320193,
"cik_padded": "0000320193",
"name": "Apple Inc.",
"tickers": [
"AAPL"
],
"primary_ticker": "AAPL",
"exchange": [
"Nasdaq"
],
"sic": "3571",
"sic_description": "Electronic Computers",
"entity_type": "operating",
"filing_count": 342,
"latest_filing_date": "2025-10-31",
"forms_filed": [
"10-K",
"10-Q",
"8-K",
"4"
],
"latest_annual_fiscal_year": 2024,
"latest_annual_revenue": 391035000000,
"latest_annual_net_income": 93736000000,
"revenue_band": "over_10b",
"insider_txn_count_90d": 6
}
],
"page": 1,
"page_size": 20,
"total": 1,
"sort": "relevance"
}
}Request schema
No body schema
Response schema
#/definitions/datasets.secCompaniesSearchResponseDoc
| Field | Type | Required | Enum | Bounds | Example | Description |
|---|---|---|---|---|---|---|
| code | integer | No | 200 | |||
| data | datasets.SecCompanySearchResponse | No | ||||
| data.dataset | string | No | ||||
| data.items | array | No | ||||
| data.items[].cik | integer | No | ||||
| data.items[].cik_padded | string | No | ||||
| data.items[].crawled_at | string | No | ||||
| data.items[].entity_type | string | No | ||||
| data.items[].exchange | array | No | ||||
| data.items[].filing_count | integer | No | ||||
| data.items[].financials_synced_at | string | No | ||||
| data.items[].fiscal_year_end | string | No | ||||
| data.items[].former_names | array | No | ||||
| data.items[].forms_filed | array | No | ||||
| data.items[].insider_last_transaction_at | string | No | ||||
| data.items[].insider_net_shares_90d | number | No | ||||
| data.items[].insider_txn_count_90d | integer | No | ||||
| data.items[].latest_annual_fiscal_year | integer | No | ||||
| data.items[].latest_annual_net_income | number | No | ||||
| data.items[].latest_annual_net_income_usd | number | No | ||||
| data.items[].latest_annual_revenue | number | No | ||||
| data.items[].latest_annual_revenue_usd | number | No | ||||
| data.items[].latest_annual_total_assets | number | No | ||||
| data.items[].latest_annual_total_assets_usd | number | No | ||||
| data.items[].latest_filing_date | string | No | ||||
| data.items[].latest_quarterly_fiscal_year | integer | No | ||||
| data.items[].latest_quarterly_revenue | number | No | ||||
| data.items[].name | string | No | ||||
| data.items[].primary_ticker | string | No | ||||
| data.items[].recent_filings | array | No | ||||
| data.items[].recent_filings[].accession_number | string | No | ||||
| data.items[].recent_filings[].filing_date | string | No | ||||
| data.items[].recent_filings[].form | string | No | ||||
| data.items[].recent_filings[].primary_doc_url | string | No | ||||
| data.items[].reporting_currency | string | No | Native-currency figures are exactly as the filer reported them (see ReportingCurrency); the *USD fields are the same values converted at fixed reference rates (services/sec.CurrencyToUSD) so they can be sorted, banded and range-filtered ACROSS currencies. Screen on the USD fields; display the native value + currency. | |||
| data.items[].revenue_band | string | No | ||||
| data.items[].schema_version | integer | No | ||||
| data.items[].seed_source | string | No | ||||
| data.items[].sic | string | No | ||||
| data.items[].sic_description | string | No | ||||
| data.items[].state_of_incorporation | string | No | ||||
| data.items[].submissions_synced_at | string | No | ||||
| data.items[].tickers | array | No | ||||
| data.page | integer | No | ||||
| data.page_size | integer | No | ||||
| data.sort | string | No | ||||
| data.total | integer | No | ||||
| msg | string | No | OK |
Use environment variables for secrets and keep Crawlora API keys server-side.
curl -X GET "https://api.crawlora.net/api/v1/datasets/sec-companies/search?q=coffee&has_financials=true&sort=relevance&page=1" \
-H "x-api-key: $CRAWLORA_API_KEY"Crawlora is designed for responsible structured public web data workflows. Customers are responsible for using Crawlora in compliance with applicable laws, third-party rights, target-platform rules, and Crawlora terms.
Read Crawlora terms