How to Scrape SEC EDGAR in 2026 (API & Python)

SEC EDGAR is one of the few platforms in this series where the honest answer to "do I need to scrape this" starts with "EDGAR already gives it to you for free." Its official APIs at data.sec.gov and efts.sec.gov are public, unauthenticated, and cover company filings, submissions history, and XBRL financial facts as JSON — no key, no signup. What EDGAR's own API doesn't do is parse a 10-K into the risk-factors section you actually want, or turn XBRL tags into a normalized income statement with ratios. This guide covers EDGAR's own free API and its real limits, no-code options, and a structured API for the parsing work, with the legal reality of each laid out up front.

Why scrape SEC EDGAR?

SEC EDGAR holds the filing history of every U.S. public company, which powers:

• Company financials research — pull normalized income statement, balance sheet, and cash-flow data with ratios for a ticker or CIK.
• Filing and disclosure monitoring — track new 10-K, 10-Q, and 8-K filings as they land, or re-run full-text search for a topic or phrase across all filers.
• Due diligence and risk research — extract specific filing sections (Risk Factors, MD&A) instead of reading a 100-page document top to bottom.
• Insider and institutional ownership analysis — follow Form 4 insider transactions and 13F institutional holdings for a company or manager.
• Cross-referencing market data — pair SEC fundamentals with a live quote or market cap from Yahoo Finance or Google Finance.

Is it legal to scrape SEC EDGAR?

Option 1: EDGAR's own free API (and its real limits)

The SEC's developer resources and EDGAR APIs overview point to data.sec.gov for company submissions and XBRL data, and efts.sec.gov for full-text search — both public JSON, no key required, as long as you send a compliant User-Agent:

curl "https://data.sec.gov/submissions/CIK0000320193.json" \
  -H "User-Agent: YourCompany research@yourcompany.com"

That returns Apple's full filing history — every 10-K, 10-Q, 8-K, and Form 4 with its accession number and document URL — for free, directly from the SEC. It's a legitimate, no-signup way to get raw filing metadata and XBRL company facts.

Where it starts to take real engineering time:

• A 10-K is a document, not a record. primary_doc_url points to the full filing HTML — often 100+ pages. Getting just the Risk Factors or MD&A section means fetching the whole document and parsing HTML yourself; EDGAR doesn't segment it for you.
• XBRL "frames" need taxonomy knowledge. Pulling a financial concept across companies (https://data.sec.gov/api/xbrl/frames/us-gaap/Assets/USD/CY2024Q4I.json) requires knowing the exact US-GAAP tag (Assets, Revenues, NetIncomeLoss, …) and period format (CY2024Q4I) — there's no fuzzy lookup, and tags vary in how consistently companies use them.
• Fair-access limits are real, not decorative. 10 requests/second per IP with a required User-Agent is generous for interactive lookups but throttles fast for a bulk backfill across thousands of CIKs — the SEC's own guidance for that case is the nightly bulk ZIP archives (companyfacts.zip, submissions.zip), not per-company API calls.

Option 2: No-code tools

Marketplace scraper actors and BI connectors exist for SEC data, mostly wrapping the same data.sec.gov endpoints rather than adding new access. They save setup time for a one-off pull into a spreadsheet, but they don't solve the parsing problem — you still get raw filing HTML or XBRL tags out the other end, and they add a layer of indirection over an API the SEC already publishes directly.

Option 3: A structured SEC EDGAR API (via Crawlora)

If what you actually want is a parsed filing section or a normalized financial statement — not raw HTML or XBRL tags — a SEC EDGAR API does that on top of EDGAR's own data. Search a company to get its CIK:

curl "https://api.crawlora.net/api/v1/sec/company/search?q=apple" \
  -H "x-api-key: $CRAWLORA_API_KEY"

{
  "code": 200,
  "msg": "OK",
  "data": {
    "query": "apple",
    "count": 1,
    "matches": [
      { "cik": 320193, "cik_padded": "CIK0000320193", "ticker": "AAPL", "name": "Apple Inc." }
    ],
    "source_url": "https://www.sec.gov/files/company_tickers.json"
  }
}

Then pull a specific filing's sections and normalized financials in Python:

import requests
h = {"x-api-key": "YOUR_API_KEY"}
base = "https://api.crawlora.net/api/v1/sec"

company = requests.get(f"{base}/company/search", headers=h, params={"q": "apple"}).json()["data"]["matches"][0]
cik = company["cik"]

sections = requests.get(f"{base}/filing/sections", headers=h, params={
    "ticker": "AAPL", "accession": "0000320193-25-000079", "items": "1A"
}).json()["data"]

financials = requests.get(f"{base}/financials", headers=h, params={
    "ticker": "AAPL", "statement": "income", "period": "annual"
}).json()["data"]

filing/sections returns extracted item text instead of a full HTML document (real fields — check the docs):

{
  "data": {
    "cik": 320193,
    "accession_number": "0000320193-25-000079",
    "form": "10-K",
    "sections": [
      { "item": "1A", "title": "Risk Factors", "text": "The Company's business...", "char_count": 68562, "truncated": true }
    ],
    "source_url": "https://www.sec.gov/Archives/edgar/data/320193/000032019325000079/aapl-20250927.htm"
  }
}

financials returns normalized income-statement (or balance-sheet, cash-flow) lines with computed ratios, instead of raw XBRL tags:

{
  "data": {
    "cik": 320193, "ticker": "AAPL", "statement": "income", "period": "annual",
    "line_order": ["revenue", "net_income"],
    "periods": [{
      "fiscal_year": 2025, "end_date": "2025-09-27", "form": "10-K", "currency": "USD",
      "lines": { "revenue": 383285000000, "net_income": 96995000000 },
      "ratios": { "gross_margin": 0.4413, "operating_margin": 0.2982, "net_margin": 0.2531, "revenue_growth_yoy": -0.028 }
    }]
  }
}

/sec/company/submissions (filing history filtered by form, from/to), /sec/company/intelligence (a merged profile, financial snapshot, and market data), /sec/full-text-search (q, forms, from/to across all filers), /sec/frames (a US-GAAP concept across companies for one period, without you tracking the taxonomy tag list yourself), /sec/insider (Form 3/4/5 transactions), and /sec/institutional-holdings (13F holdings by manager CIK) round out the group. Store one row per filing, period, or transaction and re-run on a schedule.

What you can collect

Public EDGAR filing and financial data: company search (CIK, ticker, name); filing history and metadata (accession number, form type, filing/report dates, document URLs, XBRL flag); parsed filing sections by item number (Risk Factors, MD&A, and others, with character counts); normalized income-statement, balance-sheet, and cash-flow data with computed ratios and year-over-year growth; a merged company profile with financial snapshot, market data, and recent filings; full-text search across all EDGAR filings; XBRL concept frames across companies for one period; Form 3/4/5 insider transactions; and 13F institutional holdings by manager.

Limitations and common challenges

• EDGAR itself is the free, official source — know when a structured API adds value. Raw submissions, filings, and XBRL facts are already free from data.sec.gov; a structured API earns its place when you need parsed sections or normalized statements, not "access" you don't already have.
• Fair-access limits apply either way. 10 requests/second per IP and a required User-Agent govern the underlying EDGAR calls regardless of which layer you call them through.
• XBRL tagging varies by company. Not every filer tags every concept the same way; some smaller filers have sparser or inconsistent XBRL coverage than large accelerated filers like Apple.
• Filing text is long and item-numbered, not free-form. Section extraction depends on the filing following EDGAR's standard item numbering (1A, 7, 7A, …), which most — but not all — filings do consistently.
• Public data only. This collects what companies already disclosed to the SEC and what the SEC already publishes — never a way to infer non-public information or bypass a company's actual disclosure timing.

Where this gets used

• Financial research and screening — pull normalized statements and ratios across a watchlist of tickers.
• Due-diligence pipelines — extract Risk Factors and MD&A sections for AI-assisted review instead of manual reading.
• Compliance and disclosure monitoring — track new filings, insider trades, and institutional-holdings changes as they're reported.
• Cross-source financial research — pair SEC fundamentals with live pricing from Yahoo Finance or Google Finance.

Sources

Start collecting

Try it first, free: run any public URL through the Free Web Scraper, or check whether a site blocks bots with the Anti-Bot Checker — no signup.

Test the company search, filing sections, and financials endpoints in the Playground, check the schema in the API docs, and review pricing. SEC EDGAR tells you what a company officially disclosed and when; Yahoo Finance and Google Finance add the live price and market context around it — pair fundamentals with market data instead of trusting one source alone. EDGAR only covers public companies, though; for the private-company side of the same funding story, how to scrape PitchBook covers deals, rounds, and investors before a company ever files. See the broader alternative data use case for this kind of disclosure-driven signal. See also how to choose a web scraping API and is web scraping legal.

Part of our how-to-scrape guide series — every platform we cover, in one index.