Selenium is an open-source browser-automation framework, originally built for testing in 2004, that drives real browsers through the W3C WebDriver protocol and a separate driver binary per browser — the oldest and broadest-supported tool in its category, though scraping projects mostly favor Playwright or Puppeteer today.
Every Selenium command goes over HTTP to a driver process specific to the browser — ChromeDriver for Chrome, GeckoDriver for Firefox, msedgedriver for Edge — which then translates that command into actual browser actions. That extra hop, versus Puppeteer and Playwright's direct WebSocket connection over the Chrome DevTools Protocol, adds real per-command latency and a version-compatibility surface: browser and driver versions have to stay in sync or commands start failing in ways that don't reproduce cleanly.
The upside of that architecture is breadth: because WebDriver is a W3C standard rather than a single vendor's protocol, Selenium supports the widest range of browsers and has official client libraries in Java, Python, C#, Ruby, and JavaScript — more languages than Playwright or Puppeteer officially cover, which is a real reason it persists in large, multi-language test suites.
Selenium has no built-in auto-waiting — scripts need explicit or implicit wait calls, and getting that wrong produces the classic flaky-test failure where an element "wasn't there yet." Playwright's actionability checks and Puppeteer's event-driven model both handle this automatically, which matters more for scraping than testing since a scraper runs unattended at scale, without a human noticing and re-running a failed step.
Selenium is also more fingerprintable out of the box in its default configuration — navigator.webdriver and related automation tells are well documented — and its slower per-command round trips compound at scraping volume in a way a single test suite running occasionally never surfaces. Selenium Grid still earns its keep for genuinely cross-browser regression testing at scale; for new scraping infrastructure specifically, Playwright or Puppeteer are the more common default now.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome()
driver.get("https://example.com/products")
prices = WebDriverWait(driver, 10).until(
EC.presence_of_all_elements_located((By.CSS_SELECTOR, "span.price"))
)
values = [p.text for p in prices]
driver.quit()How Crawlora handles this
Crawlora's browser-rendering tier is built on the same direct-protocol approach that made Playwright and Puppeteer faster than Selenium — real Chromium over a fast internal connection, with stealth patches and matched residential IPs applied server-side — so migrating scraping infrastructure off Selenium doesn't mean building that rendering layer yourself.
Related reading
Glossary
FAQ
Mostly for legacy codebases or genuine multi-language, cross-browser needs — new scraping projects usually default to Playwright or Puppeteer for the auto-waiting and lower per-command latency, both of which matter more at scraping scale than in an occasional test run.
Every command travels over HTTP to a separate driver process (ChromeDriver, GeckoDriver) that then relays it to the browser, versus Playwright and Puppeteer's direct WebSocket connection over the Chrome DevTools Protocol. That extra hop adds latency per command, which compounds at scraping volume.
In principle yes — WebDriver is a vendor-neutral W3C standard with drivers for more browsers and official client libraries in more languages. In practice, Playwright's three engines (Chromium, Firefox, WebKit) cover what most scraping and testing work actually needs.
Beyond Selenium, Crawlora's own docs cover the rest of the stack — browse the APIs, test a request in Playground, and move from scraping infrastructure work to production data workflows.