Tony Wang8 min readThe Career-Page Graveyard: More Than a Third of the Job Boards We've Found Are Gone or Blocked
We track 17,064 company career-page boards across 14 ATS platforms. 3,510 now block our crawler, 2,698 have vanished — 36.4% of everything we've ever found.
A "list of company career pages" sounds like the kind of thing you'd build once and reuse forever — a static index, like a phone book. It isn't. We maintain one of these lists ourselves, as part of Crawlora's Jobs dataset, and the honest answer to "how many of the boards you've ever found are still there" is: more than a third of them aren't.
The dataset currently tracks 17,064 company career-page boards across 14 hosted applicant-tracking platforms — Greenhouse, Lever, Ashby, Workday, SmartRecruiters, Workable, Recruitee, Rippling, Personio, Teamtailor, Oracle, UKG, iCIMS, and Eightfold — and indexes 900,907 live open postings today. Every board in that registry carries a status, and that status is the real story here.
More than a third of what we've found is gone or locked
Sort all 17,064 discovered boards into their current status, and the picture is immediate:
9,678 boards (56.7%) are active. That's a majority, and it's the number behind the 900,907 postings we index today. But add the two failure states — blocked (3,510, 20.6%) and gone (2,698, 15.8%) — and you get 6,208 boards, 36.4% of the entire registry. Of every three career-page boards this crawl has ever found, more than one is now either locked out or simply not there anymore.
Two different ways a board dies
"Blocked" and "gone" sound similar but describe opposite failure modes, and they're worth separating because they say different things about the company on the other end.
A blocked board still exists. The page is there, the company is presumably still hiring through it, but something on their side — a new bot-detection vendor, a stricter WAF rule, a CDN configuration change — now actively rejects the crawler. This is usually the board getting more defended, not less used. 3,510 boards, 20.6% of the registry, are in this state.
A gone board is a different kind of failure: the URL no longer resolves, or resolves to something that isn't a careers page anymore. The company migrated to a different ATS vendor and the old board URL 404s. A subsidiary got folded into its parent and its standalone board disappeared. The domain changed. Whatever the specific cause, we can no longer find that board, full stop. 2,698 boards, 15.8% of the registry, are in this state.
Neither number is a claim that the underlying company stopped hiring — plenty of gone boards are just a company that moved from, say, Lever to Workday and left the old board dangling. But it does mean that any static export of "company career pages" taken today will have measurably decayed by the time you use it next quarter.
The other 63%: still there, most of it still useful
It's worth being fair to the rest of the registry. Add active (9,678) to empty (1,157) — reachable boards, whether or not they currently have open roles — and 10,835 boards, 63.5% of everything discovered, are still there and functioning normally. Empty isn't decay; it's a company between hiring cycles, still on the same ATS, still reachable the next time we check. And despite the churn on the edges, the dataset still indexes 900,907 open postings today, which is the number that actually matters if you're searching for a job or tracking a competitor's headcount.
Where the boards live
Board count isn't evenly split across the 14 providers. Two platforms — Workday and Greenhouse — together account for 40.3% of every board we've discovered:
| Provider | Boards | Share |
|---|---|---|
| Workday | 3,571 | 20.9% |
| Greenhouse | 3,314 | 19.4% |
| Workable | 2,950 | 17.3% |
| Personio | 1,325 | 7.8% |
| Ashby | 1,196 | 7.0% |
| Teamtailor | 1,080 | 6.3% |
| Rippling | 919 | 5.4% |
| SmartRecruiters | 891 | 5.2% |
| UKG | 760 | 4.5% |
| Recruitee | 710 | 4.2% |
| Oracle | 272 | 1.6% |
| Lever | 71 | 0.4% |
| iCIMS | 4 | <0.1% |
| Eightfold | 1 | <0.1% |
This board-count ranking is a different question from posting volume by provider — Workday's boards run enormous (a single enterprise board can carry thousands of open reqs), so it dominates postings even more than it dominates board count. We wrote about that skew separately in The Job Market, by Volume, Is a Workday Market; read the two pieces together and you get both halves of the picture — how many boards a platform has, and how much hiring actually moves through them.
Query the boards yourself
Every status in this piece is a live filter, not a one-time export. The company-registry endpoint takes a status parameter with exactly the five values above:
curl --get "https://api.crawlora.net/api/v1/datasets/jobs/companies" \
-H "x-api-key: $CRAWLORA_API_KEY" \
--data-urlencode "status=blocked" \
--data-urlencode "sort=crawled_desc" \
--data-urlencode "page_size=20"
Swap status=blocked for status=gone, status=active, or status=empty to pull any slice of the registry. Pair it with /datasets/jobs/search for the live postings themselves, or /datasets/jobs/facets for a provider-level rollup of what's currently open. Full parameter lists live in the Jobs API docs.
What it means
For anyone building on top of a scraped "list of companies" — career pages, store listings, storefronts, any registry sourced from crawling the open web — the lesson isn't specific to jobs. It's that a discovery-based registry is a living thing, not a fixed asset: it needs to be re-crawled and re-classified, not exported once and trusted. We've seen the same shape in the Chrome Web Store, where the population that stays reachable and useful is a fraction of everything ever indexed. Career pages just decay for a more specific reason — the ATS a company runs, and even whether it keeps a public board at all, is a decision every company can and does change.
Query 17,064 company career boards directly
Search live postings, filter the company registry by ATS provider or board status, and pull provider-level aggregates over one REST API. 2,000 free credits a month, no card.