Tony Wang10 min readWe Searched 1.9 Million Job Postings for a "Web Scraper." We Found Two.
We checked 1,935,047 open job postings for genuine web-scraping duties. Found 280 (0.0145%) — only 2 are titled 'Web Scraper.' Here's what exists instead.
Search "web scraping jobs" and you'll find salary aggregator pages — Glassdoor, ZipRecruiter, Salary.com — all reporting a healthy-looking $120-160K average. What none of them show is how many of those jobs actually exist. So we counted them directly, in Crawlora's own Jobs dataset: 1,935,047 open postings, scraped from every major company applicant-tracking system. The honest number is startling — 280 postings genuinely describe scraping as a job duty. Two are literally titled "Web Scraper."
The naive search fails immediately
The obvious first move is to search job postings for "web scraping" and count the results. We tried it. The top matches by frequency: "Multi-Media Account Executive" (54 postings), "Merchandising Service Associate" (25), "Senior Software Engineer" (22), and "Janitorial Cleaner" (6).
None of these are scraping jobs. The likely explanation: many ATS templates carry a standard privacy or data-handling notice — something like "we may use web-scraping tools to verify applicant information" — that gets reused verbatim across every posting an employer publishes, regardless of role. A full-text search has no way to tell "the company mentions scraping in a boilerplate paragraph" from "this role does scraping." Of 2,000 postings matched this way, only 3 had the word "scrap" anywhere in the job title.
Building an honest count instead
We threw out the keyword search and built a proper classifier instead: require the exact phrase (not a loose relevance match), require real technical context around it, and explicitly exclude the privacy-boilerplate pattern. Run against a scoped subset first — the "Information and communications technology professionals" occupational family, about 20,197 open postings — it found 33 genuine matches. That's a real number, but it turned out to badly undercount: restricting to tech-titled postings misses scraping-duty roles classified under other job families (a Business Intelligence Analyst, a Data Operations Associate) entirely.
So we scanned the full population instead — all 1,935,047 open postings, no job-family restriction — using the same phrase-plus-context classifier. That surfaced 314 candidates; a manual pass removed 37 more (mostly Oxylabs, Apify, and Infatica postings — Product Manager, Marketing, SRE, Legal roles at companies whose own product is web scraping, where the mention describes the business, not the job) and added back a handful found via a second check on "reverse engineering" as a related term (see below).
Final count: 280 genuine scraping-duty postings out of 1,935,047 — 0.0145%, or 1 in every 6,911 open jobs.
What the 280 actually are
Show the full table (companies with 2+ matching postings)
| Company | Postings | What they are |
|---|---|---|
| Toloka AI | 59 | 2 titles, both explicitly "Freelance" |
| LSEG | 10 | Data Scientist (all levels), 7 countries |
| WIN Home Inspection | 7 | Senior/Data Engineer |
| PWC | 5 | Data Scientist, consulting/internship track |
| Getwingapp | 5 | GTM Automation Engineer |
| NielsenIQ | 4 | Web Scraper (2), Software Engineer (2) |
| Infatica | 4 | Senior Scraping Engineer — full-time, no freelance qualifier |
| UNUM | 3 | Data Scientist, CX/People Analytics |
| Microsoft | 3 | Software Engineer, various |
| Reflectionai | 3 | Data Ingestion / Web Crawl Engineer |
| Oxylabs | 3 | Web Scraping Engineer, Data Analyst, PM (Web Scraper APIs) |
| OpenAI | 3 | Software Engineer, Data Acquisition |
| Contentful | 3 | Senior Product Architect |
| Comtech LLC | 3 | Data Engineer (Java/Scala/Spark) |
Two patterns dominate. First: a handful of companies systematically mention scraping in every posting from one team — LSEG's Data Scientist req template runs identically from Bangalore to Beijing to Gdynia, which is why 10 of the 280 are a single company's hiring pipeline, not 10 independent market signals. Second, and more telling: only 2 of the 280 — both at NielsenIQ — are literally titled "Web Scraper." Everything else is a Data Scientist, Data Engineer, Data Analyst, or Software Engineer role where scraping is one requirement among several, not the job title.
When scraping is its own job, it's freelance
Toloka AI accounts for 59 of the 280 matches — 21% of everything we found — and every single one is one of exactly two titles: "Senior Python Data Scraping Engineer (Freelance)" and "Freelance Data Scraping Engineer (Python)." Both say "Freelance" in the title itself, posted repeatedly across locations.
This is the clearest evidence in the dataset for a simple explanation: when a company does carve out "scraping" as its own distinct job, it's overwhelmingly structured as contract or freelance work, not permanent headcount. That matches what the open freelance market shows independently — ZipRecruiter lists live "$51-$113/hr Remote Web Scraping Jobs" and "$52-$90/hr Python Web Scraping Jobs" as of this year, a gig-rate market that exists in parallel to, and largely instead of, full-time hiring.
The one clean counterexample: Infatica, a proxy/scraping-infrastructure company, has 4 postings for "Senior Scraping Engineer (Web scraping & Anti-bot)" — no freelance qualifier, a real permanent title. Oxylabs has one too ("Web Scraping Engineer - Python"). Both companies sell scraping infrastructure for a living. Outside that narrow set — companies whose product is scraping — a standing "Scraping Engineer" headcount line is close to nonexistent.
Compared to a real job title
To know whether 280-out-of-1.9M is unusually rare or just how narrow job titles work in general, we ran the same population through a much simpler filter: does the title literally contain "software engineer"? Within the same tech-job population (the ICT professional occupational family, ~20,138 postings scanned), that returned 11,823 matches — 58.7% of the entire family — with 3,279 disclosing a USD salary, median $185,000 (p25 $150,262, p75 $215,000), 20.6% remote.
| Group | Postings | USD-disclosed | Median (USD) | Remote % |
|---|---|---|---|---|
| Scraping-duty (classified) | 280 | 112 (40%) | $93,600 | 47.1% |
| "Software Engineer" (title match) | 11,823 | 3,279 (28%) | $185,000 | 20.6% |
Read this carefully — it's not an apples-to-apples salary comparison (the scraping-duty group skews toward smaller, non-US employers with a much lower disclosure rate, and Toloka's freelance postings alone are more than a fifth of the sample), and neither is the median an evaluation of what scraping work is "worth." The number that actually matters is the count: 11,823 vs. 280 is a 42x gap in the same population, using the same methodology. "Software Engineer" is a job title companies routinely hire for. "Scraping Engineer" essentially isn't — except at the handful of companies that sell scraping for a living.
We checked "reverse engineering" too — it doesn't change the picture
Scraping and API reverse-engineering are closely related skills, so we also searched the full corpus for "reverse engineering," "reverse engineer," and "reverse-engineer" — 1,485 candidates, nearly 5x more raw volume than the scraping-phrase search. Almost none of it was relevant: 901 were clearly about firmware, malware analysis, or hardware (chips, PCBs, embedded systems) — a completely different discipline that happens to share the same two words — and a further 256 failed a stricter context check requiring co-occurrence with web/API/app-specific terms. "Endpoint Engineer," for instance, matched the search but means IT device management (laptops, not API endpoints).
After filtering, exactly 3 postings turned out to be genuinely about reverse-engineering an API or website for data access, at two companies (Belvo, an open-banking data platform; Adthena, a competitive-intelligence company) — both already the kind of company you'd expect to need this work internally. We added all 3 to the final count above. The takeaway: "web scraping" and its direct synonyms already capture essentially all of the real signal. Reverse-engineering doesn't unlock some larger hidden population of scraping jobs sitting under a different name.
The bottom line
Scraping is a real, widely-used technical skill — Crawlora's own business depends on it, and companies clearly need the output badly enough that entire vendors exist to sell it. What the labor market shows is that almost nobody hires a person to do it as their job. Out of 1.9 million open postings, 280 (1 in 6,911) genuinely mention it as a duty, 2 use it as the actual job title, and the single largest source of "scraping" as its own listed role — Toloka's 59 postings — is explicitly freelance work, not a career. When companies do need it, the labor market's answer looks less like "hire a scraper" and more like "hire a data scientist who can also do this" or "pay a contractor by the project" — or, the option this whole exercise makes an unusually strong case for, buy the data instead of building the pipeline.
How we did this (and the caveats)
Data source: Crawlora's Jobs dataset, which crawls open postings from every discovered company board across Workday, Greenhouse, Lever, Ashby, SmartRecruiters, Workable, Recruitee, Rippling, Personio, Teamtailor, Oracle, UKG, iCIMS, Eightfold, Gem, and Pinpoint, plus 5 single-company big-tech boards (Amazon, Apple, Google, Meta, Tesla). Snapshot pulled 2026-08-04; 1,935,047 open postings at that moment.
The classifier is a heuristic, not ground truth. Requiring an exact phrase plus 2+ technical terms plus no privacy-boilerplate context is precision-oriented — it will still miss real scraping-duty postings that describe the work without ever using one of our six trigger phrases (a posting that just says "collect competitor pricing daily" without naming the technique, for instance), and the manual refinement pass (removing Oxylabs/Apify/Infatica's unrelated company-wide postings) is a judgment call, not an algorithm. We're confident in the order of magnitude — a few hundred out of nearly two million, not a few thousand or a few tens of thousands — more than we are in the exact figure of 280.
Salary figures are illustrative, not statistical, at n=111-112 disclosed values out of 280-11,823 postings. Salary-disclosing postings skew toward US jurisdictions with pay-transparency laws, which is a real skew in who discloses, not a flaw in the sample.
This measures full-time and contract postings on company ATS boards — it does not capture freelance-marketplace listings (Upwork, Fiverr), which is exactly where the qualitative evidence above suggests a meaningful share of this work actually lives.
Skip hiring for it — pull scraped data as an API instead
Crawlora turns hundreds of sites and platforms into normalized JSON — no scraping team, proxy rotation, or anti-bot maintenance required. 2,000 free credits a month, no card.
Frequently asked questions
Is "web scraper" a real job title?
Almost never. Out of 1,935,047 open postings in Crawlora's Jobs dataset (2026-08-04 snapshot), only 2 are literally titled "Web Scraper" — both at NielsenIQ. A genuine, permanent "Scraping Engineer" title exists at a handful of companies, but almost exclusively at ones that sell scraping or proxy infrastructure themselves (Infatica, Oxylabs).
How many jobs actually involve web scraping?
280 out of 1,935,047 open postings (0.0145%, or 1 in 6,911) genuinely describe scraping as a job duty, after filtering a naive keyword search down with a classifier requiring the exact phrase, real technical context, and no privacy-boilerplate false positives. Most of those 280 are Data Scientist, Data Engineer, or Software Engineer roles that list scraping as one requirement among several — not standalone scraping jobs.
Why does searching job postings for "web scraping" return unrelated results like retail or sales jobs?
Many applicant-tracking-system templates carry a boilerplate privacy notice — something like "we may use web-scraping tools to verify applicant information" — that gets reused across every posting an employer publishes, unrelated to the role. A naive full-text search can't distinguish that from an actual job duty; of 2,000 postings matched by a plain keyword search, only 3 had "scrap" anywhere in the job title.
Is scraping work mostly freelance instead of full-time?
The data points that way. Toloka AI alone accounts for 59 of the 280 genuine scraping-duty postings we found (21%) — and both of its titles say "Freelance" explicitly. Live freelance-market listings (ZipRecruiter: $51-$113/hr remote web scraping work) show a real gig-rate market running in parallel to, and largely instead of, full-time hiring for the same skill.
How does this compare to a real job title like Software Engineer?
In the same tech-job population, postings titled "Software Engineer" numbered 11,823 — a 42x gap versus the 280 scraping-duty postings found with the same methodology. Software Engineer roles also disclosed salary more often (28% vs 40%, though on a much larger base) with a higher median ($185,000 vs an illustrative $93,600 among scraping-duty postings, at a much smaller n).
Does "reverse engineering" describe more of these jobs under a different name?
No, not meaningfully. We checked 1,485 postings mentioning reverse engineering and found the term overwhelmingly describes firmware, malware analysis, or hardware work — a different discipline that happens to share the same words. After filtering for web/API/app-specific context, only 3 postings turned out to be genuinely about reverse-engineering an API or website for data access, confirming "web scraping" and its direct synonyms already capture the real signal.