XPath (XML Path Language) is a query language for selecting nodes in an HTML or XML document tree by describing a path to them — by tag, attribute, position, or text content — giving scrapers a way to reach elements that CSS selectors alone can't target.
A path like //div[@class='price']/span selects any span that's a direct child of a div with class "price", anywhere in the document — the leading // means "search the whole tree", not just from the root. Predicates in square brackets filter by attribute (@class='price'), position ([1] for the first match), or a function call.
XPath's functions are where it earns its keep for scraping: contains(@class, 'price') matches a class among several, text() reads an element's own text content, and parent::/following-sibling:: walk the tree in directions CSS selectors have no syntax for at all.
CSS selectors (div.price > span) are shorter, more familiar to anyone who's written a stylesheet, and cover the majority of scraping cases — most libraries, including BeautifulSoup, support them directly. XPath adds capability CSS structurally lacks: selecting by text content, walking to a parent or a preceding sibling, and combining multiple conditions in one expression.
The common pattern is CSS-first: use it for anything CSS can express cleanly, and drop into XPath specifically for the awkward cases — "find the div that contains the text 'In Stock'" or "select the label two elements before this price" — where CSS has no equivalent syntax at all.
import requests
from lxml import html
page = requests.get("https://example.com/products", timeout=15).text
tree = html.fromstring(page)
prices = tree.xpath("//div[@class='product-card']//span[contains(@class, 'price')]/text()")How Crawlora handles this
Crawlora's structured endpoints remove the selector layer entirely — no CSS or XPath expressions to write or maintain against a target's markup — since normalized JSON fields come back directly and the extraction logic lives on Crawlora's side, not in a selectors file in your repo.
Related reading
Glossary
FAQ
Learn CSS selectors first — they cover most extraction needs and are more widely understood. Add XPath once you hit a case CSS can't express: matching by text content, or navigating to a parent or sibling element instead of only descendants.
Yes — most browsers support document.evaluate() for running XPath expressions directly in DevTools, which makes it easy to test a selector against the live page before writing it into a scraper.
Like CSS selectors, XPath expressions are coupled to the page's structure — a changed class name, added wrapper div, or reordered elements can break a path that depended on the old layout. Selectors of any kind need maintenance when a target's markup changes.
Beyond XPath, Crawlora's own docs cover the rest of the stack — browse the APIs, test a request in Playground, and move from scraping infrastructure work to production data workflows.