How to Scrape Indeed in 2026 (API & Python)

Indeed doesn't offer an open developer API in 2026 — the old Publisher API and XML feed are deprecated, and what remains is a partner-gated program for employers and ATS vendors, not a public signup. This guide covers what's realistic to collect from Indeed's public job search, the legal and technical reality of doing it yourself, and a structured API that returns job search, job detail, and location data as JSON.

Why scrape Indeed job data?

• Hiring-demand research — track which roles, companies, and locations are actively posting on the largest general job board.
• Job-board aggregation — pull listings into a meta-search or niche job board without maintaining Indeed's own frontend.
• Salary benchmarking — aggregate posted salary ranges by title, location, and seniority.
• AI/LLM pipelines — feed structured postings into a resume-matching or career-advice tool instead of scraping raw HTML.
• Recruiting tools — surface open roles across companies in one searchable feed alongside other sourcing signals.

Is it legal to scrape Indeed?

Option 1: DIY in Python (and why it breaks)

A minimal scrape looks like a normal search-and-parse job:

import requests
from bs4 import BeautifulSoup

headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64)"}
params = {"q": "software engineer", "l": "Austin, TX", "start": 0}

resp = requests.get("https://www.indeed.com/jobs", headers=headers, params=params)
soup = BeautifulSoup(resp.text, "html.parser")

for card in soup.select("div.job_seen_beacon"):
    title = card.select_one("h2.jobTitle span")
    company = card.select_one("span.companyName")
    print(title.get_text(strip=True) if title else None, company.get_text(strip=True) if company else None)

It runs, once — then the usual list shows up:

• Bot detection. Indeed serves interstitials and blocks to requests that don't look like a real browser session; a bare requests call with no cookies, no JS execution, and no residential IP gets flagged fast, and you're back to a headless-browser-plus-proxy stack for a "simple" list page.
• The individual job pages are the part robots.txt disallows. The search results themselves are indexable, but /viewjob — where the full description, salary, and apply link live — is specifically excluded for general bots.
• Pagination is capped. Indeed's start parameter only walks a limited window of results per query; broad market coverage means slicing the same search many ways by location and keyword, not paging to the end.
• Markup drift. Class names like job_seen_beacon and jobTitle change without notice, and a selector that worked last quarter silently returns nothing this quarter.

Option 2: No-code / ready-made tools

Visual scraping tools can point-and-click a selector onto Indeed's search results the same way the Python script does, but they inherit the identical problems — bot detection on the request layer, a disallowed job-detail page, and brittle selectors — because the underlying access method hasn't changed. They save you writing the parser, not the anti-bot or ToS problem underneath it.

Option 3: A structured Indeed API

Crawlora's Indeed API returns job search, job detail, and location autocomplete as normalized JSON with one key — no HTML parsing, no session to maintain:

curl "https://api.crawlora.net/api/v1/indeed/search?q=software+engineer&l=Austin,+TX" \
  -H "x-api-key: $CRAWLORA_API_KEY"

The same flow in Python — search, then pull one job's full detail by its job_key:

import requests

h = {"x-api-key": "YOUR_API_KEY"}
base = "https://api.crawlora.net/api/v1/indeed"

search = requests.get(f"{base}/search", headers=h, params={
    "q": "software engineer", "l": "Austin, TX", "page": 1, "fromage": 7,
}).json()["data"]

job_key = search["jobs"][0]["job_key"]
job = requests.get(f"{base}/job", headers=h, params={"jk": job_key}).json()["data"]

/indeed/search takes q (required), l for location, page (1-based), sort (relevance or date), radius in miles, and fromage to cap results to postings within N days. Response shape (trimmed):

{
  "code": 200,
  "msg": "OK",
  "data": {
    "query": "software engineer",
    "location": "Austin, TX",
    "page": 1,
    "total": 3040,
    "jobs": [
      {
        "job_key": "5e52165ce276aa5a",
        "title": "Staff HPC Applications Engineer",
        "company": "NextSilicon",
        "location": "Austin, TX",
        "city": "Austin",
        "state": "TX",
        "posted_at": "2026-08-03T05:00:00Z",
        "url": "https://www.indeed.com/viewjob?jk=5e52165ce276aa5a"
      }
    ]
  }
}

/indeed/job takes the jk (job key) from a search result and returns that posting's full detail — description, salary, and benefits included where Indeed publishes them. And since l on /indeed/search needs a valid Indeed location string, /indeed/locations/suggest turns a partial query into one:

locations = requests.get(f"{base}/locations/suggest", headers=h, params={
    "q": "Austin", "limit": 5,
}).json()["data"]
# {"query": "Austin", "suggestions": [{"name": "Austin, TX (Travis County)", "source": "PRECISELY"}, ...]}

What you can collect

• From search — job_key, title, company, location (city/state/postal code), salary range and currency where posted, job type(s), remote flag, sponsored/urgently-hiring flags, a result snippet, and posted_at.
• From job detail — full description_text/description_html, salary min/max/period, benefits list, employment type, remote flag, direct_apply, company rating and review count, and valid_through.
• From location suggest — normalized location strings (with source, e.g. PRECISELY) to feed back into l on the search endpoint.

Limitations

• No full-text or arbitrary filter beyond q/l/radius/fromage/sort — build coverage by running multiple targeted searches (keyword × location) rather than one broad crawl.
• Job-detail fields depend on what the employer posted. Salary, benefits, and remote flags are frequently blank when Indeed itself doesn't have that data.
• Closed postings aren't flagged in real time — a job_key that returns a 404 or stale data on /indeed/job usually means the listing closed; re-check periodically rather than assuming it's still open.
• This covers public search and detail data only — no login-gated employer dashboards, applicant data, or Indeed Apply flows.

Where this gets used

• Hiring-demand and market research — track posting volume and salary trends by role, company, and metro.
• Meta job-search and niche job-board products — aggregate Indeed listings alongside other sources in one feed.
• Compensation benchmarking tools — roll up posted salary ranges across comparable titles and locations.
• Pairs with the Job Postings dataset and the job postings guide for ATS-sourced roles Indeed doesn't carry.

Sources

Start collecting

Try it first, free: run any public URL through the Free Web Scraper, or check whether a site blocks bots with the Anti-Bot Checker — no signup.

Test the search, job, and location-suggest endpoints in the Playground, check the schema in the API docs, and review pricing. Indeed is one source in a fragmented hiring market — how to scrape job postings covers the 14 ATS platforms companies post through directly, and how to scrape LinkedIn covers the public company pages behind the employers doing the hiring. For engineering-team signal specifically, how to scrape GitHub is a useful companion, and how to scrape Upwork covers the freelance side of the same hiring demand. For the legal lines on scraping generally, see is web scraping legal.

Part of our how-to-scrape guide series — every platform we cover, in one index.